Andee lives on your phone or Mac and uses any app like you do—no integrations, no root, any model; memory and passwords never leave your device | Andee 住进你的手机或 Mac,像你一样操作任何 App——无需集成、无需 Root、支持任意模型,记忆和密码绝不离开设备。
feat: initial release of the Andee Android body
booboosui3fb2c9c2757c
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.ymlnew file mode 100644index 0000000..deb908a--- /dev/null+++ b/.github/ISSUE_TEMPLATE/bug_report.yml@@ -0,0 +1,117 @@+name: Bug report+description: Something is not working — a crash, a wrong result, or a feature that silently does nothing.+title: "[Bug] "+labels:+ - bug+body:+ - type: markdown+ attributes:+ value: |+ **Before you start:** the single most common cause of an unfixable report here is a missing+ device profile. This project talks to the Android system at a level where ROMs disagree —+ a call that works on AOSP can fail silently on MagicOS. Without the fields below we usually+ cannot reproduce, and the issue gets closed.++ Security problem? **Do not use this template.** See [SECURITY.md](https://github.com/kuafuai/andee/blob/main/SECURITY.md).++ - type: checkboxes+ id: self-checks+ attributes:+ label: Self checks+ description: These keep the tracker usable. All are required.+ options:+ - label: I have read the [Contributing Guide](https://github.com/kuafuai/andee/blob/main/CONTRIBUTING.md) ([中文](https://github.com/kuafuai/andee/blob/main/CONTRIBUTING.zh-CN.md)).+ required: true+ - label: I have searched existing issues, including closed ones.+ required: true+ - label: This is a bug report, not a question. Questions belong in Discussions.+ required: true+ - label: This is not a security vulnerability (those go to SECURITY.md, privately).+ required: true++ - type: input+ id: build+ attributes:+ label: Build+ description: |+ How you got this build. Version numbers are useless on their own — `versionCode`/`versionName`+ have been `1`/`1.0` throughout, so a version number does not identify a build.+ placeholder: "commit hash, release tag, or \"built from main myself on <date>\""+ validations:+ required: true++ - type: input+ id: device+ attributes:+ label: Device, Android version, ROM+ description: "Output of `adb shell getprop ro.product.model`, `ro.build.version.release`, and the ROM name. Include `adb shell wm size` and `wm density`."+ placeholder: "HONOR AMM-AN00 / Android 15 / MagicOS 9 — 1200x2664 @520"+ validations:+ required: true++ - type: dropdown+ id: brain-mode+ attributes:+ label: Brain mode+ description: Which brain was answering?+ options:+ - hub (a separate brain host, over WebSocket)+ - local (on-device, calling an OpenAI-compatible API)+ - not sure+ validations:+ required: true++ - type: checkboxes+ id: setup-state+ attributes:+ label: What was switched on+ options:+ - label: Accessibility service was enabled+ - label: Overlay permission was granted+ - label: Notification access was granted+ - label: The app had been granted `WRITE_SECURE_SETTINGS` via adb++ - type: textarea+ id: steps+ attributes:+ label: Steps to reproduce+ description: Tell us the exact sequence. If the agent was involved, say what you asked it to do and what tool calls it made.+ placeholder: |+ 1. Ask the agent to ...+ 2. It calls ...+ 3. ...+ validations:+ required: true++ - type: textarea+ id: expected+ attributes:+ label: What you expected+ placeholder: Do not paste the steps again — say what should have happened.+ validations:+ required: true++ - type: textarea+ id: actual+ attributes:+ label: What actually happened+ placeholder: Include exact on-screen text. If it failed silently, say so — "nothing happened" is a valid and useful observation.+ validations:+ required: true++ - type: textarea+ id: logs+ attributes:+ label: logcat+ description: "At minimum: `adb logcat -s Body:* Screen:* Wake:* Dog:* GroundingLog:*`. Paste inside fenced code blocks."+ render: shell+ validations:+ required: true++ - type: textarea+ id: extra+ attributes:+ label: Anything else+ description: Screenshots, screen recordings, a dump of the UI tree, or what you already ruled out.+ validations:+ required: falsediff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.ymlnew file mode 100644index 0000000..5a36a50--- /dev/null+++ b/.github/ISSUE_TEMPLATE/config.yml@@ -0,0 +1,30 @@+# Issue routing. Modelled on how large projects keep their tracker usable: the tracker+# is for reproducible defects, everything else goes somewhere that fits it better.+#+# Two repository settings have to be on for these links to work, and nothing in the+# repo can turn them on:+# 1. GitHub Discussions (Settings → Features → Discussions) with a "Q&A" category.+# Without it, `blank_issues_enabled: false` leaves a user with a bug that is not+# a bug and nowhere to put it.+# 2. Private vulnerability reporting (Settings → Security → Private vulnerability+# reporting). Until it is on, the first contact link 404s. See SECURITY.md.++# Blank issues are off on purpose: an empty text box produces unreproducible reports.+blank_issues_enabled: false++contact_links:+ - name: "\U0001F510 Security vulnerability"+ url: 'https://github.com/kuafuai/andee/security/advisories/new'+ about: Report vulnerabilities privately. Never open a public issue for a security problem — this app asks for a lot of sensitive Android permissions, and a public report is a roadmap for abuse. See SECURITY.md.++ - name: "\U0001F4AC Question or setup help"+ url: 'https://github.com/kuafuai/andee/discussions/categories/q-a'+ about: "Not a bug — a question. \"Accessibility service keeps turning itself off\", \"which ROMs are known to work\", \"how do I build this\" all belong here."++ - name: "\U0001F512 Permission justification"+ url: 'https://github.com/kuafuai/andee/blob/main/docs/permissions.md'+ about: Why the app asks for SMS, call log, contacts, and accessibility — what each one buys, and what happens if you refuse it. Read before filing a permission-related issue.++ - name: "\u26A0\uFE0F \"It did something I did not want\""+ url: 'https://github.com/kuafuai/andee/blob/main/DISCLAIMER.md'+ about: "It spent money, sent the wrong message, or acted on a misreading. That is the documented behaviour of a tool that taps and types on your behalf — see DISCLAIMER.md and docs/acceptable-use.md. This is a different thing from a vulnerability, which is when a *third party* can steer it: that one goes to the security channel above."diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.ymlnew file mode 100644index 0000000..9ea7a00--- /dev/null+++ b/.github/ISSUE_TEMPLATE/feature_request.yml@@ -0,0 +1,94 @@+name: Feature request+description: Propose a capability the agent should have, or a rough edge worth smoothing.+title: "[Feature] "+labels:+ - enhancement+body:+ - type: markdown+ attributes:+ value: |+ **One request per issue.** If you have three ideas, open three issues — bundled requests+ get triaged as one and then rot.++ The hard question in this project is never "can we build it". It is **"what does it cost+ the user"**. Almost every capability here is paid for with an Android permission, and the+ permission list is already long enough that the app looks like something it is not. A+ feature that needs a new sensitive permission will be judged on that cost first — say so+ up front and the discussion gets much shorter. See+ [docs/permissions.md](https://github.com/kuafuai/andee/blob/main/docs/permissions.md)+ ([中文](https://github.com/kuafuai/andee/blob/main/docs/permissions.zh-CN.md)).++ - type: checkboxes+ id: self-checks+ attributes:+ label: Self checks+ options:+ - label: I have read the [Contributing Guide](https://github.com/kuafuai/andee/blob/main/CONTRIBUTING.md) ([中文](https://github.com/kuafuai/andee/blob/main/CONTRIBUTING.zh-CN.md)).+ required: true+ - label: I have searched existing issues and discussions, including closed ones.+ required: true+ - label: This is a feature request, not a bug report, and not a question.+ required: true+ - label: This is one request, not several bundled together.+ required: true++ - type: textarea+ id: outcome+ attributes:+ label: What do you want to be able to do?+ description: Describe the outcome in user terms, not the implementation. "Ask the agent to X and have it happen" beats "add a tool called Y".+ placeholder: "I want to ask the agent to ___ and have it ___."+ validations:+ required: true++ - type: textarea+ id: today+ attributes:+ label: What happens today, and why is that not good enough?+ description: If you're working around the absence today, describe the workaround — it often points at the real gap.+ validations:+ required: true++ - type: dropdown+ id: side+ attributes:+ label: Which side is this on?+ description: This project has two halves that ship separately — the on-device half that can see and touch the phone, and the brain half that decides what to say. Several capabilities only exist in one.+ options:+ - Device side (a tool the agent can call — screen, taps, notifications, files, hardware)+ - Brain side (prompt, wording, when to speak, how it reasons)+ - Both+ - Not sure+ validations:+ required: true++ - type: dropdown+ id: permission-cost+ attributes:+ label: Does this need a new sensitive permission?+ description: Check the manifest before answering. "The app already has it" is the best possible answer — it means the feature is nearly free.+ options:+ - "No — it needs no new permission, or only one already declared"+ - "Yes — it needs a new sensitive permission (name it below)"+ - "Unsure"+ validations:+ required: true++ - type: textarea+ id: alternatives+ attributes:+ label: Alternatives you considered+ description: Other ways the same outcome could be reached, including ways that avoid a new permission. Say why they are worse.+ validations:+ required: false++ - type: checkboxes+ id: help+ attributes:+ label: Can you help build it?+ description: Not a commitment — just tells us where to start.+ options:+ - label: I am willing to open a pull request for this.+ required: false+ - label: I can test it on a real device and report back.+ required: falsediff --git a/.github/pull_request_template.md b/.github/pull_request_template.mdnew file mode 100644index 0000000..fbdfb65--- /dev/null+++ b/.github/pull_request_template.md@@ -0,0 +1,70 @@+<!--+Keep this short. Delete the sections that genuinely do not apply, but do not delete a+section just to avoid answering it — the ones below exist because they have bitten us.+-->++## What and why++<!-- One paragraph. What changes, and what problem it solves. Link the issue it closes, if any. -->++Closes #++## Scope++<!-- List the files/areas this touches. If it touches something outside a single feature, say why. -->++-++## The three kinds of Chinese (only read this if you touched any string)++This repo separates text by **who reads it**, and the split is not cosmetic:++| Kind | Where it lives | Rule |+|---|---|---|+| The **user** reads it (windows, cards, bubbles, settings, errors) | `res/values/strings.xml` + `res/values-en/strings.xml`, reached **only** through `i18n/AppLocale` | Add to **both** tables. `context.getString` does not reach the overlay, so it will not work here. |+| The **model** reads it (prompts, tool schemas, tool results) | `brain/LocalPrompt.kt` — the single copy | Written in **English**. Do not add a second source. |+| **Matched against system text** (call keywords, unread-count regex, clipboard labels) | `NotificationRelayService`, `ScreenController` | **Never translate.** These compare against what WeChat/Android emit; translating breaks the feature. |++Literal on-screen text the model must read aloud (e.g. a settings path, a button label) and+"what a Chinese user would actually say" specimens stay in Chinese by design — see CONTRIBUTING.md.++- [ ] If I added a user-facing string, it is in **both** `values/` and `values-en/`.+- [ ] If I added model-facing prose, it is **English** and lives in `LocalPrompt.kt`.+- [ ] I did **not** translate anything in the "matched against system text" category.++## How it was verified++Build with **debug**, not release — the release variant has no signing config and produces an+unsigned APK that will not install.++```bash+./gradlew clean :app:assembleDebug+```++- [ ] `assembleDebug` succeeds.++**Device tested on:** <!-- model / Android version / ROM, e.g. HONOR AMM-AN00 · Android 15 · MagicOS. Say "not tested on a device" if you did not — that is fine, it just changes how this gets reviewed. -->++**What was actually observed:** <!-- Not "it works" — what you saw. Screen reading, a logcat line, a pixel measurement, a dump. -->++> If you verified by searching strings inside the built APK, do **not** use `grep` on the dex+> for non-ASCII text — it produces false negatives (a string that is present reports zero hits).+> Compare against the dex string table instead, and list what remains rather than what matched.++## Security and permissions++- [ ] This changes no permission, no `AndroidManifest.xml` entry, and nothing in the network layer.+- [ ] **Or:** it does, and the following is spelled out below — including whether it widens what an+ unauthenticated or local caller can do.++<!-- If the box above is checked, note here: new/changed permissions, any new network surface or+ bind address, and whether anything the user can read now leaves the device. -->++## Checklist++- [ ] I only changed files related to this PR. Untracked files left by other work are still+ untracked — I did not commit, stash, or delete them.+- [ ] Commit messages say what changed and why.+- [ ] I have read [CONTRIBUTING.md](https://github.com/kuafuai/andee/blob/main/CONTRIBUTING.md)+ ([中文](https://github.com/kuafuai/andee/blob/main/CONTRIBUTING.zh-CN.md))+ and agree to the contribution licensing terms in [LICENSE](https://github.com/kuafuai/andee/blob/main/LICENSE) §A.2.diff --git a/.github/workflows/build.yml b/.github/workflows/build.ymlnew file mode 100644index 0000000..15c7f47--- /dev/null+++ b/.github/workflows/build.yml@@ -0,0 +1,42 @@+# Builds the debug APK on every push and PR.+#+# Debug, not release: the release build type has no `signingConfig` and there is+# no keystore in this repository, so `assembleRelease` would emit an unsigned APK+# that cannot be installed. Signing needs secrets, and a workflow that fails on a+# fork for want of secrets the forker cannot add is worse than no workflow.+#+# The JDK is pinned to **17** on purpose and the range is narrow: AGP 8.3.1 needs+# 17 or newer, and the Gradle 8.4 wrapper cannot run on 21 or newer (it rejects+# the class file version). Bumping this without bumping Gradle breaks the build+# with an error that does not mention Java.+name: build++on:+ push:+ branches: [main]+ pull_request:++jobs:+ debug-apk:+ runs-on: ubuntu-latest+ steps:+ - uses: actions/checkout@v4++ - uses: actions/setup-java@v4+ with:+ distribution: temurin+ java-version: '17'++ - uses: gradle/actions/setup-gradle@v3++ - run: ./gradlew --no-daemon :app:assembleDebug++ # Host-side tests are template-only today, so this is a tripwire for the+ # day someone writes a real one rather than a meaningful gate now.+ - run: ./gradlew --no-daemon :app:testDebugUnitTest++ - uses: actions/upload-artifact@v4+ with:+ name: app-debug+ path: app/build/outputs/apk/debug/app-debug.apk+ if-no-files-found: errordiff --git a/.gitignore b/.gitignorenew file mode 100644index 0000000..bf0b41f--- /dev/null+++ b/.gitignore@@ -0,0 +1,21 @@+*.iml+.gradle+/local.properties+/.idea/caches+/.idea/libraries+/.idea/modules.xml+/.idea/workspace.xml+/.idea/navEditor.xml+/.idea/assetWizardSettings.xml+.DS_Store+/build+/captures+.externalNativeBuild+.cxx+local.properties+.idea+_backup++# tools/ 是一组 Python 脚本;导入或 py_compile 时会生成字节码缓存+__pycache__/+*.pycdiff --git a/CHANGELOG.md b/CHANGELOG.mdnew file mode 100644index 0000000..637e0ed--- /dev/null+++ b/CHANGELOG.md@@ -0,0 +1,107 @@+# Changelog++Notable changes to Andee. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).++## Versioning policy++**No release has been tagged yet.** In 207 commits the version has never moved off the values the+Android template shipped with. Rather than pretend otherwise, this file records the policy going+forward and states plainly where the project is:++- `versionCode` / `versionName` live in `app/build.gradle` and are currently **`1` / `1.0`**.+- The first tagged release should be **`v1.0.0`** with `versionName "1.0.0"` — note that `"1.0"` is+ not a valid SemVer string, so the two need to be reconciled at that point.+- Thereafter: releases are tagged `vMAJOR.MINOR.PATCH`, `versionName` mirrors the tag without the+ leading `v`, and `versionCode` increases by one per release. `versionCode` must increase+ **monotonically** or Android refuses the upgrade; the same number must never be reused for+ two different builds.+- MAJOR for a change that breaks the wire protocol or an existing configuration; MINOR for a new+ capability; PATCH for fixes.++**A version number does not currently identify a build.** Because `versionCode` has always been `1`,+"which build is this?" cannot be answered from the installed app. Until a release exists, identify+builds by the short commit hash they were built from. (Embedding the hash into `versionName` at+build time would close this hole; it is not implemented yet.)++## [Unreleased]++### Added++- `LICENSE` — Apache-2.0 with additional conditions (Part A: no multi-tenant SaaS, no+ white-labelling that removes the branding). Note that GitHub reports this as `NOASSERTION`.+- `SECURITY.md` — threat model, the five known security limitations, and a private reporting channel.+- `docs/permissions.md` + `docs/permissions.zh-CN.md` — why the app asks for the permissions it asks+ for, each one tied to a concrete tool and to what happens if you refuse it. One file per language.+- `CONTRIBUTING.md` + `CONTRIBUTING.zh-CN.md` — code style, the rules for the three kinds of Chinese+ in this codebase, and the contribution licensing terms. One file per language.+- `CODE_OF_CONDUCT.md` — Contributor Covenant v2.1.+- `THIRD_PARTY_NOTICES.md` — generated dependency license inventory: 93 components across 5+ licenses. Notably flags that **ML Kit and the `play-services-*` stubs are not open-source**.+- `tools/license_audit.py` — the generator behind the notices file.+- `.github/` — bug report and feature request issue forms, a pull request template, and issue+ routing with blank issues disabled.+- `README.zh-CN.md` — the Chinese setup guide, split out from `README.md`.+- `DISCLAIMER.md` + `DISCLAIMER.zh-CN.md` — the user-facing terms, one file per language (same+ convention as the two READMEs). Covers the four things Apache-2.0 does not: **where your data+ actually goes** (no telemetry in the repo, but speech goes to Volcengine and the conversation plus+ every tool result goes to the brain endpoint you configure), **what the software is genuinely+ capable of** (it can complete a payment by tapping; the only guardrail is Android's `FLAG_SECURE`),+ **who bears the consequences**, and third-party platform terms.+- `docs/acceptable-use.md` + `docs/acceptable-use.zh-CN.md` — the prohibited uses, one file per+ language. This is the part that matters legally: naming the prohibited uses is what keeps this+ project from being characterised as *providing assistance* to them. It calls out fund movement+ (including the Article 287-2 aiding-information-network-crime exposure) and publishing false+ information separately.+- `docs/permissions.md` gained a **"data goes where"** section — the honest version of+ "we do not collect your data", which is true of the maintainers and false of the device.+- Both READMEs now carry a warning above the fold and a `## Disclaimer and acceptable use` section;+ `SECURITY.md` states the triage boundary (steering it is a vulnerability; doing what it was built+ to do is documented behaviour).++### Changed++- **Positioning corrected throughout the docs.** Both READMEs and `CLAUDE.md` described this as+ "the body side of a Physical Agent system" — i.e. an assistant that a brain drives. That+ contradicts the project's own first instruction, in `LocalPrompt.TEXT`: *"you are not a tablet's+ assistant, you are an individual, and the tablet is only what carries you."* The docs now lead+ with the individual, and give the body / persona / notebook mechanisms that back it up in code+ (fifteen moods and idle fidgets, `BallLook.persona` reaching the head of the system prompt, the+ private `Notebook`, the `(to yourself …)` self-review turn, and §11's rule that the character is+ delivery and never a licence). No code changed — the code was already right; the prose was not.+- `README.md` is now the **English** front page. The detailed Chinese provisioning guide moved to+ `README.zh-CN.md`, and both cross-link.+- `gradle.properties` no longer pins `org.gradle.java.home`. It previously hardcoded a+ machine-specific JDK path, so the build failed immediately for everyone else while looking like+ an environment problem. The real constraint (JDK 17–20; Gradle 8.4 cannot run on Java 21+,+ Android Studio's bundled JBR 25 is rejected) is now documented instead of worked around.+- `tools/check_look_sync.py` derives the repository root from its own location instead of a+ hardcoded absolute path.+- `tools/dump_faces_ascii.py` and `tools/verify_faces_lab.py` locate Chrome by `CHROME_PATH`, then+ common install locations, then `PATH`, instead of hardcoding a macOS-only path.+- `CLAUDE.md` corrected: the Gradle JDK note, the stale claim that the checkout is `frontend/`, and+ the description of which README is which.+- **Every bilingual document now follows one file per language, and each language links only to its+ own counterparts.** `CONTRIBUTING.md` and `docs/permissions.md` were written in Chinese under the+ English default filename — the opposite of the convention the other four files follow. They are+ now split into `.md` (English) and `.zh-CN.md` (Chinese), and every Chinese page that used to+ point at an English file now points at the Chinese one, so a Chinese reader is never dropped onto+ an English page. `SECURITY.md` and `CODE_OF_CONDUCT.md` have no Chinese edition; the Chinese docs+ now say so explicitly instead of linking silently.++### Removed++- `brain/andee-system-prompt.md` — an unused mirror of the cloud brain's prompt. It had no runtime+ role (no build step references it, no code loads it). **The cloud brain mode itself is not+ retired**; only this repo's copy of its prompt was removed. Recover it with+ `git show <rev>:brain/andee-system-prompt.md`.+- `dog/NRF24_Dongle/README.md` reference in the README — the dongle firmware is not in this+ repository and never was. The README now says so.++### Known issues++- `tools/verify_faces_lab.py` reports 7/16 assertions passing against the committed+ `reports/ball-faces-lab.html`. This is pre-existing (identical before and after the portability+ changes above) and is not caused by the open-sourcing work. Left as-is; worth resolving before+ contributors are pointed at the ball-face lab.+- The debug WebSocket server (`net/BodyWsServer`) binds `0.0.0.0:9008` with no authentication. See+ `SECURITY.md`. Not fixed.diff --git a/CLAUDE.md b/CLAUDE.mdnew file mode 100644index 0000000..64053a2--- /dev/null+++ b/CLAUDE.md@@ -0,0 +1,465 @@+# CLAUDE.md++This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.++## What this is++**Framing first, because it changes how you write everything else.** This is not "an app that a brain drives". The project's own words — the first line of `LocalPrompt.TEXT` — are *"you are not a tablet's assistant, you are an individual, and the tablet is only what carries you."* The device is a **body** (eyes = camera, ears = ASR + offline wake word, hands = accessibility, optionally a robot dog as a leg), and the persona is a real input to the model (`BallLook.persona` → `LocalPrompt.withPersona`), not decoration. It also keeps a private notebook and a scheduler, and it reviews its own conversations when alone. Write code and docs that preserve an individual's continuity; **tool calls are how a body acts, not what the project is.** Calling this "the body side of an agent" in your own prose is the failure mode to avoid.++This is an **Android app** (Gradle root project `body`, package `net.kuafuai.andee`). The checkout is the **repository root**. Concretely it does tap / swipe / type / screenshot / read the UI tree / system keys, plus ASR and TTS. The brain is normally remote (a hub the operator runs, reached over a WebSocket), but it can also be **this device itself** — and that is the out-of-box default; see "The brain can also be this device".++Two orientation facts that are easy to get wrong:++- **The entry point is `ScreenBodyService`, an `AccessibilityService`.** Almost everything the user sees is a `WindowManager` overlay (the floating ball); there is no main UI, and the service is what builds the whole object graph. If the accessibility service isn't enabled in system settings, nothing works — **which is why there is now exactly one Activity in the launcher**, `ui/SelfCheckActivity`. It is not a home screen: it is the readiness checklist, and it exists because a device whose accessibility service is off would otherwise have no way in and no way to be told why. See "Startup self-check" below.+- **Docs are split by audience.** `README.md` is the **English** front page (what it is, architecture, build, permissions, security). `README.zh-CN.md` is the **Chinese** guide and is the authoritative setup/permissions walkthrough — read it for first-time device provisioning; it carries the exhaustive SDK setup, the wake word, and the robot-dog section. `docs/permissions.md` is the permission justification, with a Chinese edition at `docs/permissions.zh-CN.md`. The same one-file-per-language rule governs `DISCLAIMER`, `docs/acceptable-use` and `CONTRIBUTING` — English on the default filename, Chinese on `.zh-CN`, and each language linking only to its own counterparts. (`SECURITY.md` and `CODE_OF_CONDUCT.md` exist in English only.) It used to tell you to `cd body-native`; the checkout is the **repository root**. (Fixed 2026-09-29.)++## Build and run++```bash+./gradlew assembleDebug # APK → app/build/outputs/apk/debug/app-debug.apk+./gradlew installDebug # build + install to the attached device+./gradlew test # host-side JUnit (only template tests exist)+./gradlew connectedAndroidTest # instrumented tests (needs a device; only a template test exists)+adb logcat -s Body:* BodyWs:* BodyWsClient:* Asr:* Tts:* # the log tags this app uses+```++There is **no real test suite** — `ExampleUnitTest` / `ExampleInstrumentedTest` are unmodified templates. Verification is manual: install, watch logcat, drive the ball. There is no lint/format task configured beyond AGP defaults.++Toolchain notes that bite:++- `gradle.properties` **does not pin the Gradle JDK** — it used to hardcode a machine-specific `org.gradle.java.home`, which made the build fail instantly on every other machine. The build needs a **JDK in the 17–20 range**: AGP 8.3.1 requires 17+, and the Gradle 8.4 wrapper cannot run on 21+ (Android Studio's bundled JBR 25 dies with `Unsupported class file major version 69`). Set `JAVA_HOME`, or put `org.gradle.java.home=<path to JDK 17–20>` in **`~/.gradle/gradle.properties`** — never in the repo's copy. In Android Studio, set *Settings → Build Tools → Gradle → Gradle JDK* to a 17–20 JDK.+- `settings.gradle` puts **Aliyun mirrors first** for both plugin and dependency resolution. Keep them first when editing; they're there because the project is developed behind the GFW.+- `minSdk 30` is load-bearing, not arbitrary: `ACTION_IME_ENTER` and `AccessibilityService.takeScreenshot` are both API 30+.+- Deliberately **no Jetpack Compose, no Hilt/Dagger, no coroutines.** Overlays can't host Compose, so all UI is hand-written `View` code; there's exactly one service, so dependencies are constructor-injected by hand; concurrency is `Handler` + dedicated `Executor`s. Match this style rather than introducing the modern stack.++## Architecture++### Wiring++`ScreenBodyService.onServiceConnected()` builds the entire object graph and is the best single file to read first. It creates `FloatingWindowUi` → `AudioIO` → `TtsController` / `AsrController` → `ScreenController` → `CommandDispatcher`, then starts **both** networking paths.++### Two networking paths, one dispatcher++The app simultaneously runs an inbound server and an outbound client, both terminating in the same `CommandDispatcher`:++- **`BodyWsServer`** — listens on `0.0.0.0:9008`. For a brain or debug client on the same LAN (or via `adb forward`). Methods on this wire are the **internal** names: `screen.tap`, `screen.ui_tree`, …+- **`BodyWsClient`** — dials *out* to a hub (`hub_url`, e.g. `ws://<brain-ip>:9100`), sends a `register` envelope carrying `ToolSchemas.all()`, then serves `request` messages pushed back down the same socket. Reconnects with exponential backoff (500 ms → 30 s). This is how the body reaches a brain behind NAT. Skipped entirely when `hub_url` is blank — server-only is a valid mode.++Shared envelope shape: `{"type":"request","id":…,"method":…,"params":{…}}` in, `{"type":"response","id":…,"result"|"error":…}` out. Dispatcher contract: **return a JSON-serializable value or throw** — thrown exceptions are converted to `error` responses at the transport boundary, so don't catch-and-encode errors yourself inside dispatch handlers.++The device also talks unprompted, as `{"type":"event","kind":…,"device_id":…,"data":…}`, sent to the hub and broadcast to every server client. Three kinds exist: `asr.final` (the user said something), `task.stop` (they stopped it — triple-tap on the ball since the `■` button was removed, see "Startup self-check"), `notification` (relayed from the notification listener). Events are **never queued**: with the socket down they are dropped, because a stale "the user just said" is worse than silence. Unlike tools, event kinds are not in `ToolSchemas`, so **the brain side has to be taught each one separately** or it is silently discarded — which looks, from the tablet, exactly like the feature not working. That cost is why the meeting recorder asks for its minutes on `asr.final` instead of inventing a fourth kind.++### The brain can also be this device (`brain/`)++`brain = local` in `voice_prefs` swaps the remote brain for an agent loop running on the tablet: `LocalBrain` calls DeepSeek over plain HTTP (`LlmClient`, OkHttp, non-streaming) and drives the same `CommandDispatcher`. The two are **mutually exclusive** — `ScreenBodyService.startBrain()` builds one or the other, because two brains answering the same `asr.final` is two voices talking over each other. The local brain is the **out-of-box default** (since 2026-09-29); anything other than the literal string `"hub"` reads as local, the same fail-closed rule `groundingEnabled` uses — the direction flipped with the default, and the settings picker's leftmost option (the fallback) flipped with it.++What made this cheap is that `BodyWsClient` had already reduced "the brain" to a contract: **eat a line of text, emit `onProgress` / `onFinal`, call the dispatcher in between.** Those two lambdas now live in `ScreenBodyService.onBrainProgress` / `onBrainFinal` and both brains pass method references to them — that is deliberate, not tidiness: the TTS mute rules (meeting, call), the `acceptBrainOutput` gate, `stripMarkdown` and `ChatHistory` all live in there, and a second copy is where this feature would rot. Subtitles, edge glow, the fold, stop and the follow-up mic window are untouched by the switch.++Four things in `LocalBrain` are correctness, not style:++- **Tool calls run serially.** The cloud agent uses `asyncio.gather`; copying that breaks `passthroughForGesture`, which is not reentrant (see the overlay section).+- **Every dispatch has a deadline** (`dispatchWithTimeout`). `ui.ask` blocks to its timeout — and `ui.show_html` used to block until the user closed the page, up to 5 minutes, which is gone (see the page section). Over the hub, `body_hub`'s 30 s request timeout silently abandoned those; locally nothing does, so the loop times out and tells the model the call may still be running. It does **not** interrupt the work.+- **Images are moved out of tool results and rationed.** `png_base64` is extracted **recursively** (it nests under `after_shot` / `miss_shot`, and `ui_tree` attaches one when the element list is empty), replaced in place with `"<image attached>"`, and re-attached as a separate `{"role":"user","content":[text, image_url]}` message. Only the newest 2 image messages survive; every successful tap ships a full-screen PNG, so a 15-step task would otherwise carry 15. The accepted cost is that dropping a message invalidates DeepSeek's prefix cache from that point, and the 55 tool descriptions ahead of it are the biggest constant in the request.+- **The clock rides in on the user message** (`[现在 …]`), not the system prompt. There is no clock tool on this device, and the system prompt plus tool specs are the cached prefix — restamping them every turn would throw that away for one line. The **interface language** (`· 界面语言 zh`) rides in the same bracket, for the same reason: the system prompt is a constant and cannot name a language without invalidating the cache every time the user touches the language picker. Both `stamp()` and `uiLangTag()` follow the user's language setting rather than `Locale.CHINA` — an English interface whose own messages are stamped "周一" would feed the model a Chinese signal every turn, against `LocalPrompt` §9.+- **The model's language is a prompt rule, not a translation.** `values-en/` covers what the *user* reads; it does nothing about what the model *says back* — a system prompt written entirely in Chinese is a strong anchor, and an English-speaking user would get Chinese answers with a fully translated interface. §9 states the rule (follow the user's own words; fall back to the bracketed interface language; for silent turns and recordings, follow the material) instead of naming a language, which is what keeps the prompt constant. Note the split this rests on: the interface language and the conversation language are **independent** — the user may run an English UI in Chinese, and the prompt must not confuse the two.++`LocalPrompt.TEXT` is the system prompt, adapted from the *cloud* brain's text. Our mirror of that text (`brain/andee-system-prompt.md`) was deleted on 2026-09-29 — nothing loaded it and the two had already stopped needing to move together, so it was a file with no reader. Note what that did **not** do: the cloud mode is still in the code (`VoiceConfig.BRAIN_HUB` still exists; the local brain is the out-of-box default), and that side keeps its own copy of the prompt, which this repo never owned. The CRM sections and the commitment/playbook rituals are **gone rather than softened** from *this* prompt: `crm_*` does not exist here and nothing wakes this device up later, so a rule whose tools are absent just teaches the model to claim it did something it cannot do.++Guards: `MAX_ITERATIONS = 25` and an identical-`(name, arguments)`-three-times cut-off, both of which end with a spoken final answer — a ball that simply stops is the one failure the user cannot interpret. Neither replaces `ScreenController`'s per-location tap guard; that one counts coordinates, these count calls.++Not done on purpose: no streaming (an SSE parser plus incremental `tool_calls` reassembly, for a second or two of earlier TTS), no history on disk (service restart = new session), no context compaction, and `tools/boss.py` has no injection point in local mode because its whole mechanism is impersonating a device to the hub.++### The two-name problem (ToolSchemas)++The hub wire uses **LLM-facing tool names** (`get_screen_element`, `tap_screen_element`, `tap_by_coordinates`, `system_action`); `CommandDispatcher` uses **internal method names** (`screen.ui_tree`, `screen.tap_id`, `screen.tap`, `screen.global`). `ToolSchemas.methodOf()` translates between them, and `BodyWsClient` applies it before dispatching. `BodyWsServer` does **not** translate — it passes methods through raw.++Consequence: **adding a tool means editing both `ToolSchemas.kt` and `CommandDispatcher.kt`.** A schema entry with no matching dispatcher branch fails at call time with `unknown method`; a dispatcher branch with no schema entry is invisible to the brain. The `description` strings in `ToolSchemas` are prompt text read by an LLM — they are behavioral documentation, not code comments, and are meant to stay in sync with the equivalent Python `body_tools/` files on the brain side.++### The overlay-vs-gesture conflict++This is the subtlest invariant in the codebase. `ScreenController.tap/swipe/longPress` use `dispatchGesture`, which injects events into the **real touch pipeline** — so the app's own floating overlay would intercept them first. Every gesture-based command is therefore wrapped in `passthroughForGesture` (`CommandDispatcher`) or the equivalent in `ScreenBodyService.swipe`, which flips the overlay to non-touchable, **sleeps ~50 ms** (because `updateViewLayout` is posted to the UI thread and needs a frame or two to actually apply), runs the gesture, then restores. The window stays visible throughout.++Eid taps (`tap_screen_element` / `long_press_screen_element` → `tapByEid` / `longPressByEid`) are gestures too — they resolve to pixel centers and dispatch a real touch — so they go through the same dance. Accessibility-API commands (`type`, `submit_input`, `ui_tree`, `global`) do *not* touch the input layer and correctly skip this. If you add a tool, decide which category it's in.++The mirror-image problem is **accessibility** visibility, and it is not solved by the touch dance: we are an `AccessibilityService`, so our own overlays show up in our own dumps. Every overlay root therefore calls `hideFromAccessibility()` (`ui/OverlayA11y.kt`) before `addView` — keep that up on any new window, or the brain starts reading our toolbar as the app's UI. `tryOtherWindows` skips `service.packageName` as a second guard, and `dumpUiTree`'s `treeEmpty` treats a dump of our own package as no tree at all (that one is for our real Activities — `HtmlActivity` / `ScanActivity` / `LookActivity` / `SelfCheckActivity` — which are app windows, not overlays). Note there is no *touch* bug here to fix: `passthroughForGesture` means an injected tap can never land on the ball. Don't "fix" it by refusing taps inside the ball's rectangle — app content under the ball is legitimately tappable and passthrough makes tapping it work.++What makes `hideFromAccessibility()` actually work is **`accessibilityDataSensitive`, not `importantForAccessibility`** — and the difference is invisible until you look. `accessibility_service_config.xml` asks for `flagIncludeNotImportantViews`, which makes `View.includeForAccessibility()` return true no matter what `importantForAccessibility` says. So `NO_HIDE_DESCENDANTS` on its own hides our overlays from every service *except the one that needed it*: measured, with the vault form open, `get_screen_element` returned the whole form, labels and phone number included. `ACCESSIBILITY_DATA_SENSITIVE_YES` (API 34+, inherits down the subtree) is honoured against us because we do not declare `android:isAccessibilityTool` — and is strictly kinder than the blunt flag, since TalkBack does declare it and keeps the ball. Both are set; the old one is the pre-34 fallback. Don't casually drop `flagIncludeNotImportantViews` to "simplify" this — it is what makes unlabelled app views visible at all.++Hiding the two **focusable** cards (`SettingsUi`, `VaultUi`) then creates a second-order problem: they are the active window, and a hidden active window dumps as zero nodes — the exact signature `reviveNote()` reads as "the accessibility connection went stale, have the user toggle Andee off and on". That message is aimed at a human, so it has to be true. `OwnCard` in `ui/OverlayA11y.kt` (incremented in those two classes' `show`/`hide`; counted, because the vault opens on top of settings) lets `annotate` say the real thing instead — the user has our own card open, nothing is broken, wait or ask them to close it. Same rule as the meeting recorder's `stopped_by` and `device.look`'s grace message.++### Coordinate protocol: the model never does math++Ported from generic_bridge (phone.sh), whose finding is the founding rule here: **general-purpose models are bad at grounding** (AndroidLab: GPT-4o 31.2 vs GUI-tuned 49.5) but good at reasoning — so the design goal is that the model never computes a coordinate.++- `get_screen_element` returns an **element outline**, one line per element, in document order and indented by depth: ` e7*|Button|发送|[490,321]` — eid, `*` clickable marker, class, label, and the element's center in **0-1000 relative** coordinates computed by us from `getBoundsInScreen`. The indentation is the pruned tree rendered cheaply (~20% more characters than an unindented list, against ~170% for the nested JSON) and it is what makes "which one" answerable: on a 朋友圈 feed the `#comment_btn` under the line saying 好友3 *is* 好友3's comment button. Label falls back `text → desc → #<id tail>`; a `#` label is an internal resource name, never visible text. Ordering is document order, not useful-first — sorting scattered each post across the list and stacked twenty identical label-less `ImageView` lines at the bottom. Truncation is a safety valve (`MAX_ELEMENT_LINES = 300`, reported as `elements_dropped`) that drops decorative leaves from the tail first and never containers, so grouping survives it.+- `tap_screen_element` takes an **eid** (the old resource-id parameter is gone — WeChat nodes mostly have none, and e-numbers cover every listed element). `rebuildIndexAndList` rebuilds the eid→pixel-center index on every dump; `tapByEid` looks the number up and passes the **pixels straight to `tap()`** — no px→norm→px round-trip (an earlier lost version fed norm values into the pixel `tap()` and every e-tap landed in the top-left quadrant). Stale/unknown eids fail loudly with "re-dump, don't guess".+- `tap_by_coordinates` / `long_press_by_coordinates` / `swipe_by_coordinates` are the **last resort** for self-drawn panes (mini-program, WebView, game canvas) and take **0-1000 relative integers** — resolution-independent and immune to screenshot downscaling. Values outside 0-1000 are **rejected**, never clamped (a silent clamp to the edge is a mis-tap the model can't see). `ScreenController.tapNorm/longPressNorm/swipeNorm` convert to pixels against the local display.+- `take_screenshot` downscales to **1280 on the long side** before base64 — proportions are preserved and the 0-1000 protocol is scale-invariant, so this is free token savings.+- **Stuck-loop guard**: per-location counting, not consecutive-with-no-change (the real dead-loop interleaves menu opens, wrong pages and backs, and apps emit content events constantly — both naive conditions miss it). Gestures within 120 px of each other share a bucket; within 5 minutes, the 4th tap on the same bucket is refused with an error telling the brain to stop, re-observe, and ask the user. Swipes are exempt (list scrolling legitimately repeats); e-taps are exempt (re-tapping a confirmed element is normal operation).+- **NO_TREE policy** (empty-list hint): prefer `back`/`home` to exit and retry via a different route over blind coordinate taps — a wrong exit costs one tap, a wrong tap can cost the task.+- **Stale-service note**: `ScreenController.goodDumps` records which packages have actually produced a usable element list here. When a dump comes back with *zero nodes* from a package with such a history, `reviveNote()` appends the evidence to the hint — WeChat's tree is genuinely obtainable, but the accessibility connection sometimes goes stale and only a human toggling Andee off/on in 设置 → 无障碍 restores it. It is **evidence, not a rule**: the note states the facts and the brain decides whether to interrupt the user. It is deliberately not keyed to `com.tencent.mm`, and deliberately fires only on `nodes_total == 0` — a self-drawn pane still reports its containers, so mini-programs never trigger it.++### Text input goes through ADBKeyboard, and the action names are bare++`type_text` has exactly one channel: `ScreenController.typeViaAdbKeyboard` broadcasts to **ADBKeyboard**, which must be the device's active IME. It is an IME, so it types through `InputConnection.commitText()` — the same path a human finger takes, which an app cannot refuse without refusing all keyboards. That is why the older waterfall (`ACTION_SET_TEXT` → clipboard `ACTION_PASTE` → IME panel walking) is gone: `SET_TEXT` crossed fields on Radix forms and did nothing at all in CodeMirror.++The one thing to get right, because getting it wrong is invisible: **the broadcast actions have no package prefix.** ADBKeyboard registers its receiver at runtime with the bare strings `ADB_INPUT_TEXT` / `ADB_CLEAR_TEXT` (confirm with `adb shell dumpsys activity broadcasts | grep ADB_INPUT`). Sending `com.android.adbkeyboard.ADB_INPUT_TEXT` matches no filter, and since we also `setPackage()` the intent, it is delivered to nobody — no exception, no log, the field simply stays empty. That failure is indistinguishable from an OEM blocking the broadcast, and it cost a day: it was read as "MIUI drops app-to-app broadcasts", a hub-side relay was built to re-send the broadcast from the PC over `adb` as `shell`, and because the relay used the *correct* bare action it worked — so on a plugged-in device everything looked fine and only an unplugged one failed. If typing ever breaks again, check the action strings and the active IME before believing any theory about app identity or OEM policy.++Readback is a **poll**, not a sleep: after `commitText` the WebView still has to publish the new value to the accessibility tree, and reading too early returns `performed: false`, which makes the brain send the text a second time — so the fixed wait produced real double-typing. Reads try the focused node first and then scan the tree (WebViews often hide the focused node's text). A self-drawn editor can defeat both; that is an a11y false negative, and the error string says to trust a screenshot instead.++**The keyboard belongs to the person; the brain borrows it per call.** ADBKeyboard must be the *active* IME for any of the above to work, and it is a headless IME — no keyboard UI at all. So the device **rests on the user's own IME** (they can type in any app on the tablet), and `ui/ImeSwitch.lendToBrain` / `giveBack` bracket each `type_text`: ADBKeyboard for one tool call, then straight back. Two things that follow, both measured, both easy to get wrong:++ * **The switch is a `Settings.Secure.DEFAULT_INPUT_METHOD` write.** On MIUI an app calling `InputMethodManager.setInputMethod` *or* `switchToNextInputMethod` is silently refused (no exception, no log — read the setting back or you will believe it worked). The settings write needs `WRITE_SECURE_SETTINGS`, granted once at provisioning (`adb shell pm grant net.kuafuai.andee android.permission.WRITE_SECURE_SETTINGS`), and a reinstall keeps it. `lendToBrain` then waits ~350 ms before broadcasting: the setting and the *binding* are different events, and a broadcast into an IME that has not bound yet disappears without a trace. Expect ~0.75 s per `type_text`.+ * **The person's keyboard is raised by the brain's own tap.** The brain taps a field, which asks for a soft keyboard; with a real IME active that request is honoured — on screen, over the app being driven. Nothing puts it back down: `hideSoftInputFromWindow(null, …)` is ignored, and `performGlobalAction(GLOBAL_ACTION_BACK)` is *not* consumed by the IME the way an injected key event is (it dismissed the page instead). Don't re-try either. It clears itself when the brain touches something that takes focus.++And one platform rule that shaped the ⌨ field: **a `TYPE_APPLICATION_OVERLAY` window cannot raise the soft keyboard on this ROM** — not via `showSoftInput`, not via `WindowInsetsController.show(ime())`, not even from a real finger tap. The same input inside an Activity raises it immediately. That is why `ui/TextInputActivity` is an Activity, why the card has to **fold** before it opens (a fullscreen overlay sits *above* application windows and eats every touch aimed at the field — the same reason `ui.show_html` folds), and why it handles the IME inset by hand: a translucent window is never resized for the keyboard.++A second rule, from the same ROM and with the same shape — *our windows are overlays, and the platform is allowed to hide them*: **a password `inputType` on one of our cards makes the field impossible to type into** (`ui/SecretInput.kt`). Honor routes password fields to its own secure keyboard, whose window carries `PRIVATE_FLAG_HIDE_NON_SYSTEM_OVERLAY_WINDOWS` — which hides every `TYPE_APPLICATION_OVERLAY` while it is up, including the card holding the focused field. The IME loses its target the instant it arrives, so the keyboard flashes once and retracts, forever. Nothing in the app is wrong when this happens and the diagnostics all look healthy: `startInput` arrives with the right `inputType`, the connection binds, and `mInputShown` is simply `false` — `dumpsys window windows | grep HIDE_NON_SYSTEM_OVERLAY` is the line that says why. So **secret fields on the cards mask with a `PasswordTransformationMethod` and keep an ordinary `inputType`**, plus `TYPE_TEXT_FLAG_NO_SUGGESTIONS` for the part of the password variation worth keeping. Don't "fix" a masked field by giving it back its `TYPE_TEXT_VARIATION_PASSWORD`; `TYPE_TEXT_VARIATION_VISIBLE_PASSWORD` is not a way out either, since it still carries the password bit.++### Screen perception++`ScreenController.dumpUiTree()` does **bottom-up pruning** so the tree stays small enough for an LLM context: drop invisible/zero-size nodes, drop decorative leaves with no id/text/desc and no interactive flag, hoist single-child layout wrappers, keep containers with ≥2 surviving children. `verbose=true` disables all of it. Class names get common `android.widget.` / `androidx.*` prefixes stripped. `AccessibilityNodeInfo` objects are recycled in `finally` blocks throughout — preserve that when editing.++After pruning (non-verbose only), `rebuildIndexAndList` **replaces the nested tree with the element outline** the brain actually reads — the nested JSON was only ever scaffolding, and the indentation preserves the part of it that carried meaning. Stats fields (`nodes_total` / `nodes_kept` / `nodes_useful` / `surface` / windows diagnostics) survive; the tree bulk does not. Containers earn a line of their own here, which is why the trim drops leaves and never containers.++Screenshots return base64 PNG over the wire (downscaled to 1280 long side — see the coordinate protocol section); the top-bar tool buttons instead write to `getExternalFilesDir(null)/captures/` (no storage permission needed, reachable via `adb pull`) and return only a summary.++### Voice pipeline++`AsrController` / `TtsController` are thin state machines gluing `AudioIO` (raw PCM 16-bit LE) to hand-rolled ByteDance 火山 clients. `HuoshanAsr` and `HuoshanTts` implement **custom binary WebSocket framing** — 4-byte header + event/seq + gzipped JSON payload — ported from other projects (wxBot's Python, electron-vite's TypeScript respectively); the class KDoc documents the wire format and event-code sequence, and is the only spec available. Don't "simplify" the framing.++Non-obvious: `AsrController.stopImpl` deliberately blocks on `waitFinished(3000)` because 火山 sends the final transcript ~500 ms *after* the client's `isLast` — closing eagerly loses it.++### Meeting recorder++`MeetingRecorder` holds the microphone for a whole meeting and hands back a **verbatim transcript**. The device has no model, so it does not summarise: the brain turns the transcript into minutes and puts them on screen with the existing `show_html` — that split is why the class has no notion of a "summary" anywhere in it.++Three things about it are load-bearing:++- **Why it rotates sessions.** `HuoshanAsr` reports a final exactly once, when the server answers our `isLast`. One session for one meeting would therefore produce nothing at all until the meeting ended, and any dropped connection in between would take the whole hour with it. So a segment lives ~2 min (`SEGMENT_SOFT_MS`, cut at a pause; `SEGMENT_HARD_MS` regardless, for the meeting where nobody stops to breathe) and its final is appended to `files/meetings/meeting-<stamp>.txt` the moment it arrives.+- **Why the new session opens before the old one is retired.** Retiring blocks up to `FINAL_WAIT_MS`, connecting costs a second or two, and doing either with no live session would punch a hole in the recording. Opening first also makes a failed rotation free — the old segment is still receiving, so the meeting carries on. There is no chunk buffer and there should not need to be one.+- **Finals arrive out of order.** A retired segment's final races the live segment's, on different WebSocket IO threads. A `TreeMap` keyed by segment index releases only a contiguous run, so the transcript can never silently reorder what was said — which is also why every segment must `settle()` exactly once even on error or close, or the flush stalls forever on the missing index.++`meeting.*` is deliberately **not** under `screen.*`: recording is not operating the device, and the ball must stay where the user can reach it. Reaching it is the only control they have — the recorder holds the mic and the wake gate is closed, so they cannot ask for anything by voice. A ball tap (`startTurn`) or stop (`stopEverything`, from the ball's triple-tap since `■` was removed from the bar) therefore *ends the meeting and returns*. That is also a bug fix: `AsrController.isActive()` is literally `audio.isStreaming()`, which a meeting sets, so without the intercept a tap would fall into the "off" half of `asr.toggle()` and tear the recording down with no error anywhere.++When the *user* ends it the brain doesn't know, so the device asks for the minutes itself — and asks on **`asr.final`**, not on an event kind of its own. An event would have to be taught to the brain separately (see above) or it is silently dropped, which from the tablet looks exactly like the ball doing nothing; `asr.final` is the one channel every brain already answers, and it already lights the glow and starts a task. `ScreenBodyService.meetingMinutesQuery` composes that turn, and it is **prompt** — it is everything the brain knows about why a transcript just appeared, so it names `show_html` explicitly and says why the meeting stopped. The cost, which is the reason it reads the way it does, is that the request enters the brain's context as something the user said. The brain's own `stop_meeting` sends nothing — it already has the transcript in the tool result, and a second copy would have it write the minutes twice.++`stopped_by` is narrated to the user by the brain, so it has to be true: `user` (a ball triple-tap — the only stop control left since `■` was removed, see "Startup self-check"), `focus` (something took the audio floor — a phone call), `brain` (`stop_meeting`), `shutdown` (service going away). The focus case originally reported itself as `user`, which had the brain tell people they had touched a ball they hadn't.++**The device must not speak during a meeting**, and `ScreenBodyService`'s `onFinal` gates `tts.speak` on `!meeting.isActive()` for it. Two independent reasons, either sufficient. The speaker is an inch from the microphone, so a spoken reply lands in the transcript. And — the one that actually bit — `TtsController` holds its own `AudioFocusGate`: **a second `AudioFocusRequest` from this same process displaces the first**, because Android's focus stack is per request, not per app. So speaking fired the recorder's focus-loss callback and ended the meeting about a second after it started, as the brain read out "好的,开始录音了". Anything new that takes audio focus while a meeting can be running has the same problem.++Recording **unfolds the card to full screen** and writes the clock plus the tail of the transcript to the live row (`renderLive`, 1 Hz for the clock and on every partial for the words). The tail is a window, not a second copy: committing each of ~30 segments as its own `ChatHistory` row would evict the real conversation from an 80-row history, and the file on disk is the record. The unfold goes through `dispatcher.expand()` rather than `window.setCompact(false)` because the dispatcher caches whether it has already folded.++### The robot dog (`dog.*`): a peripheral on a one-way cable++The dog is not this device. It hangs off USB OTG on an Arduino Nano dongle that+forwards single ASCII characters over 2.4GHz to a robot the tablet cannot see,+and it *holds whatever it was last told until it is told something else*. So+"stop" is not an optimisation — it is half the protocol. `dog/DogLink.kt` is the+cable (find the dongle, get USB permission, send a char, read `X OK` /+`X NO-ACK`); `dog/DogMotion.kt` is the part that knows what a *move* is and+guarantees every one of them ends.++**Nothing in `dog.*` blocks for the length of the motion it starts, and that is+the load-bearing decision.** `BodyWsClient` serves requests on a+`newSingleThreadExecutor` — one at a time — so a `dog_move` that slept out its+own duration would also be holding back the `dog_stop` that interrupts it, and+a user shouting "停下" would wait for the dog to finish what the brain asked+for. So `dog_move` returns the moment the command is out and a scheduled timer+sends the `J`. Two consequences worth knowing before editing anything here:++- **A second `dog_move` replaces the first rather than queueing.** That is why+ `dog_sequence` exists: multi-step motion is a routine, not a series of calls.+ A brain that sends `dog_move` twice has asked for one move.+- **Timers race cancels.** Move #1's auto-stop can be inside its "write J" while+ move #2's "write K" is on its way out; if the J lands second, the dog stops+ the instant it starts. Every scheduled stop therefore carries the `generation`+ it was armed under, and the generation check plus the write happen inside one+ `motionLock` critical section, so a stop either wins outright or aborts.+ `onLost` deliberately does *not* bump the generation — doing so would+ invalidate the stop timer of a move whose cable came back, leaving that move+ with nothing to end it. It clears the state instead, and the armed timer+ no-ops because there is nothing left to stop.++`dog.*` is not under the `screen.` prefix, like `meeting.*` and for a stronger+reason: the dog is not this device, so driving it must not fold the card the+user happens to be reading.++**The tablet's own senses, and the dog's.** The dog is bolted to the+tablet, so the tablet's gyroscope *is* the dog's — `dog/DogSense.kt` reads it.+Two things hang off DogSense:++- **`dog_turn` is the one closed-loop motion.** It drives until the measured+ angle is reached instead of dividing a "full circle ≈ 8.7 s" constant that+ drifts with battery, floor and stride phase. It still does not block — the poll+ loop runs on `seqExec` and the tool returns as soon as the first command is+ out. The measured result therefore cannot be in the reply; it lands in+ `dog_status.last_turn` after a `COAST_MS` wait, because the dog is *still+ rotating* when the loop writes `J` and reporting the angle before it settles+ would report a number the dog never stopped at.+- **The fall guard is a reflex, deliberately not a decision.** A round trip to+ the brain is seconds; a dog that has gone over does not have seconds. So a+ motion whose tilt crosses `FALL_TILT_DEG` is stopped on the tablet within+ `GUARD_MS`, and the brain finds out via `aborted_by`.++Two traps in there. `SensorManager.getOrientation()`'s azimuth is **singular when+the device is near-vertical**, which is exactly how a tablet sits strapped to a+dog's back — so yaw is taken as the bearing of a device axis projected onto the+world horizontal, with the axis chosen once per turn as whichever is furthest+from vertical. And it is `TYPE_GAME_ROTATION_VECTOR`, **not**+`TYPE_ROTATION_VECTOR`: the latter fuses in a magnetometer sitting inches above+four servos. Losing north costs nothing because every question asked here is+relative.++`dog_turn` checks `link.status().open` after its first write and refuses before+starting the loop, the same way `move` does. Skipping that check is not a missing+nicety: a turn that "starts" with no dongle spends its whole timeout measuring+zero degrees and then reports the dog as stuck or out of range, sending the user+to inspect a machine that is fine.++**The dog's own senses ride home on the acknowledgement.** The 2.4G link is+half-duplex by configuration, not by nature: the dog's radio already sends an+ack frame for every packet it receives, so `Telem_Push` (dog firmware) attaches+four bytes to it — magic, distance, battery, validity flags. **No extra+transmission, no change to the main loop's timing.** That is why this route was+taken over switching the module to TX mode, which would mean going deaf to+remote commands for the duration.++Three consequences that shape the API:++- **It requires both ends, and the dongle negotiates which.** ACK payloads need+ matching `FEATURE`/`DYNPD` config at both ends; a mismatch is not "no+ telemetry" but a dead link, and a dead link is a dog that cannot be told to+ stop. So the sketch does not assume — it sends in `dpl`, and on failure+ reconfigures to `legacy` (plain 32-byte, original firmware) and retries,+ keeping whichever worked. `m=dpl|legacy` on every reply line says where it+ landed. Both ends also degrade alone: the dog's `NRF24L01_EnableAckPayload()`+ reads FEATURE back and reverts if the module refuses; the dongle reports only+ what actually arrived (`telemetry.supported`).+ **Flash the dongle first** — the reverse (new dog, old dongle) is the one+ combination with no fallback, because an old dongle has no negotiation and+ retries to exhaustion against ack payloads it cannot accept.+ *Verified on hardware 2026-09-24:* against a stock-v1 dog the dongle settled+ on `m=legacy` and got clean acks — **so the fallback does work**, `dpl` fails+ and `legacy` succeeds. With v2 flashed it negotiates `m=dpl` and telemetry+ arrives: 15/15 acks, `d=` tracking a hand at 7–48 cm, `b=23~25`. Every+ NO-ACK before that was **the dog being switched off**, which cost a control+ sketch, a raw-SPI register dump and a 12-combination rate×power sweep to+ establish — check the dog is powered before suspecting anything here. A few+ seconds of NO-ACK right after the dog powers on is the radio settling.+- **Every reading is stale by construction.** The dog speaks only when spoken+ to, so `telemetry.age_ms` is part of the reading, not metadata. `dog_sense`+ exists to refresh it — one frame, one ack, and the only `dog.*` call that is+ safe mid-motion because it re-sends the *current* frame rather than a new one.+- **`supported: false` is not zero.** An unflashed dongle, an unflashed dog and+ a flat battery must not read alike. Null propagates to the brain as "cannot+ see", and the schema text says to say so rather than report a number.++**Speed and stance are settings, and the dongle holds them.** On the dog they+are independent `if`s sitting *beside* the direction chain, so a frame carrying+only a speed byte falls through to the standing branch — changing speed would+stop the dog. The dongle therefore keeps one persistent frame and mutates only+the bytes a given command owns. `dog_speed` / `dog_pose` do not touch+`generation`, do not arm a stop, and take effect from the next motion onward.+The exception is `J`, which clears the stance too: the raised-stance branch runs+ahead of the wheel branches and only poses the legs, so a stop that preserved it+would look stopped without being stopped.++**The one boundary the app cannot close.** The dog has no failsafe of its own:+if the dongle is unplugged or loses power while the dog is moving, the tablet+cannot send the stop that ends the move, and the dog finishes it. `onDestroy`+sends `J` first, before anything else in teardown, and every motion has a+scheduled end — but a cable pulled mid-move is out of reach. Say so plainly+rather than implying the tablet always has the last word; closing it for real+means a watchdog in the dog's firmware.++Two smaller things that are load-bearing in `DogLink`: the port is opened+**lazily** (nothing happens on a tablet with no dongle plugged in), and opening+**costs seconds** because asserting DTR resets the Nano — so `ensureOpen` waits+out the reboot and reads the banner, which doubles as a diagnosis (`READY` means+dongle *and* radio are up; `ERR: NRF24L01 not responding` means the dongle is+alive but its radio module is not, i.e. fix the dongle's wiring, do not blame+the dog). Device discovery runs the library's table first and then a narrow+second pass, because Arduino clones carry bridges the library has never heard of+— CH9102 (`0x1a86:0x55d4`) is absent from its `UsbId` table and it is what the+newer Type-C Nano boards ship. That second pass only claims a QinHeng part+presenting one vendor-specific interface: a `Ch34xSerialDriver` can be built+around *any* device and will hand back a port, so trusting it blindly would let+us seize the tablet's own hub.++Lock order is `DogMotion.motionLock` → `DogLink.lock`, never the reverse —+which is why `onLost` is invoked outside `DogLink`'s lock (the detach broadcast+is the path that would otherwise deadlock). `DogSense` is a leaf: it holds only+its own registration lock and never calls back into either, so reading it from+inside `motionLock` is safe. `attach()` is still called outside, because+registering listeners is the one part of it that is not instant.++### Floating UI++`FloatingWindowUi` is the live overlay: a `TYPE_APPLICATION_OVERLAY` window holding the emotion ball, a top-right tool bar (`Tool` enum: swipes, back, home, ui-tree, screenshot), a gear button, and a subtitle band. Callers drive it through `setState(State)`, `setSubtitle(text, SubtitleKind)` and `setTouchable(bool)`; it posts everything to the main-thread `Handler` internally. Hit-testing on the ball is **circular**, sized to match the rendered sphere — touches outside the circle fall through to whatever is underneath.++The ball itself (`ui/ball/`) is hand-rolled **OpenGL ES 2.0** on a `TextureView` (not `GLSurfaceView`, so the subtitle and toolbar composite above it without z-order hacks) with its own `HandlerThread` GL context. `EmotionState` holds per-frame animation state and is ticked on the GL thread; `Mood` maps emotional/device states to face parameters. UI→GL communication is via `@Volatile` fields only. `gl/` contains hand-written mesh/shader helpers: `Mesh` uses a fixed **interleaved 8-float (32-byte) stride** (pos3/normal3/uv2) that shaders assume — changing it means updating every `bind()` offset.++Long-running work must stay off the main thread: `ScreenController.tap/swipe` block on `dispatchGesture` for up to ~2 s and will ANR the UI thread. `ScreenBodyService` runs top-bar tools on a dedicated `ScreenTools` executor, and `BodyWsClient` hands requests to a worker so the recv thread keeps answering pings.++### The full-screen faces++`HtmlActivity`, `LookActivity`, `ScanActivity` and `SelfCheckActivity` are the only places the user sees a *finished* artefact rather than the ball, and they share `Theme.Body.Fullscreen` (`Theme.Body.Stage` extends it, for the four that animate their own enter/exit — see `ui/Stage.kt`). `SelfCheckActivity` is on `Theme.Body.Stage.NoPreview` on top of that, because with the service up it finishes without drawing a frame and a starting window would flash black — see "The launcher icon opens two doors". That theme exists for two bugs that came free with `Theme.Body`: its `android:statusBarColor` is `?attr/colorPrimaryVariant` = **purple_700**, so a purple stripe sat over every one of them; and `DayNight`'s light `windowBackground` made each open **flash white** before the dark content drew. The theme pins `#0F172A`, transparent system bars, and dark bar icons. Put any new full-screen Activity on it.++`ui/CameraChrome.kt` is the shared chrome for the two camera screens (`ViewfinderView` / `StatusPill` / `closePill`), extracted because both had grown their own `FrameLayout + PreviewView + TextView` **sized in raw pixels** — legible on the phone it was written on, postage-stamp sized on the tablet. Everything in there is dp, and `HtmlActivity` uses the same `closePill` so the one control the user can always rely on looks identical everywhere.++Two things in it are not styling:++- **The shutter flash is a privacy promise.** `LookActivity.sampleFrame` pulses the viewfinder corners and moves the status line to 「看到了」 because the camera is the most invasive thing this device does, and before this the user had no way to tell *when* a frame was actually taken. `sampleFrame` runs on the dispatcher thread, so every UI touch in it goes through the companion `mainHandler`.+- **The first frame after a bind waits out the exposure ramp.** `EXPOSURE_SETTLE_MS` (1.5 s, measured on this hardware) is counted from `boundAt`, not from the request, so a monitoring loop sampling every 30 s never pays it twice. Shooting immediately returns a washed-out or near-black frame, and the brain has no way to tell that from the actual room — it just describes the dark. `rebind` clears `imageCapture` and re-stamps `boundAt` *before* republishing it, because switching lenses restarts the ramp and `sampleFrame` gates on `imageCapture` being non-null.+- **Closing is honest.** Both camera screens carry a ✕完成, and closing `LookActivity` — by pill or by back key — stamps `userClosedAt`. `device.look` reads it: within `USER_CLOSED_GRACE_MS` it returns "用户刚把摄像头关掉了,先问过他" instead of the old "camera not ready yet — retry", which was false *and* told the brain to put the lens straight back in the user's face. Same rule as the meeting recorder's `stopped_by`. `LookActivity.close()` (the brain's own stop) sets `closingByBrain` so it doesn't stamp.++`ScanActivity`'s cancel path writes `LastResult = {"error":"cancelled by user"}`. That is a bug fix, not politeness: `device.scan` polls `LastResult`, so backing out of a scan used to leave the poll spinning to its full 30 s timeout with nothing to report.++**Frosted glass is not available** for these pills, so don't try again. `RenderEffect` (API 31+) blurs a View's *own* content, not what is behind it; blurring behind requires `Window.setBackgroundBlurRadius`, which is window-level and cannot be scoped to one floating child. Translucent fill plus a 1 dp `#33FFFFFF` stroke is the approximation — and the stroke is load-bearing anyway, because the brain picks the page background and a dark pill on a dark page vanishes. (The *cards* do get a real blur — see `ui/Glass.kt` below. Each of them is a whole window, which is exactly the thing a pill is not.)++**`show_html` does not block.** It used to sit in a latch until the user tapped 完成 — up to five minutes — and because `LocalBrain` runs a turn on one thread and runs tools on it synchronously, that latch held the *conversation*: the user's next sentence was accepted and then queued behind it. It now returns as soon as the page is up, with `closed: false`, and the news that the user finished with the page arrives by a second route: the local brain gets a silent `LocalBrain.note()` (readable by the next turn, never spoken — a turn here would mean a model call and a sentence every time anyone dismisses a page), and a hub gets a `page.closed` event it may ignore. Two consequences worth knowing. `HtmlActivity` keeps exactly **one** page (`pageUp`), so a second `show_html` while one is up is **refused** rather than stacked — the shared static file would otherwise lose to whichever activity read it last. And the ball's perch, which used to be released by that same latch, is now released by the close callback.++The device injects **no CSS** into `show_html`; styling belongs to the brain. So the house style (slate tokens, the `#9DB4FF` accent, an explicitly-named CJK font stack) lives in the `show_html` **description** in `ToolSchemas` — that prompt is the only lever on page quality, which matters most for the small models this runs against. `HtmlActivity` only fades the page in once `onPageFinished` lands, with a 2.5 s backstop because a top-level script that throws can leave that callback pending forever, and half a page beats a black screen.++### The cards share one look (`ui/Glass.kt`)++`SettingsUi`, `VaultUi`, `WakeEnrollUi` and `SelfCheckUi` take every colour, drawable, interpolator and animation from `Glass`. They share it because they are reachable from one another — settings opens the vault and the wake-word card *on top of itself*, and the self-check card opens settings *on top of itself* — and two different dark palettes stacked like that read as a bug before they read as a style. A fifth card means using `Glass`, not picking hex values.++**The base is pure black and everything above it is white at a low alpha**, which is not only taste. These are overlays, and what is behind them is some other app — a white document or a night-mode terminal. A palette of *opaque* greys has to pick one of those to look right against; a stack of translucent whites over black darkens whatever is behind it and then lays the same relative steps on top, so the hierarchy (card → panel → recessed well → segment thumb) survives both.++**That hierarchy assumes a near-opaque black card underneath it, so a page on the backdrop needs a fourth surface: `Glass.glass`.** `GLASS` is `#1FFFFFFF` — `panel`'s shape and a different fill on purpose. `panel` is 6% white laid inside a 95%-black card; `glass` is 12% white laid on a blurred photo at ~13% mean luma under the scrim. No single alpha serves both: 6% white on the backdrop is invisible (a card that has quietly stopped existing), and 12% white on the card is a grey box floating in a black field. The sheen does more of the work here too — at that fill the top-lit edge is the only thing saying the rectangle sits *on* the image rather than being a window cut into it.++**It has no caller at the moment, and that is a fact worth recording rather than a reason to delete the two constants.** `glass` existed for one day of the self-check page being a fullscreen surface on `Backdrop`; that page is now a `Glass.card` and its rows are `panel` — see "Startup self-check" and `Glass.GLASS`'s own note. The distinction survives because the *fact* behind it did not go anywhere: `Backdrop` is still what `TextInputActivity` and the launcher self-check host paint themselves with, and anything placed on top of it needs this fill and not `panel`. Reach for `glass` whenever the parent is the backdrop, never as a general lighter `panel`. For a whole fullscreen surface (a page's own background) it is still `Backdrop` plus a `CENTER_CROP` `ImageView` — `glass` is for the cards that go *on* that. Put any new full-screen Activity on `Theme.Body.Fullscreen`.++**The blur is real, and `Glass.frost` gets exactly half of Android's blur API.** `FLAG_BLUR_BEHIND` + `LayoutParams.blurBehindRadius` is public and works. The other half — `backgroundBlurRadius`, the one clipped to the window's own rounded bounds, which is what would make the card itself a frosted *pane* — is a `@hide` field whose public door is `Window.setBackgroundBlurRadius`, and these overlays have no `Window`: they are bare Views handed to `WindowManager.addView`. Reaching for it does not compile. So the blur is unclipped and the whole screen behind the card softens, which for a centred modal is the effect you wanted anyway. It is also **optional at runtime** — `isCrossWindowBlurEnabled` is false on battery saver, on devices that cannot afford it, and whenever the developer option is off — so `frost()` returns whether the compositor agreed and the caller picks its fill from the answer: `CARD_FROSTED` (60% black) when something is blurring, `CARD_SOLID` (95%) when nothing is and a translucent card would just be a smear of the app below. Call it **before** `addView`; the flag has to be on the params the window is created with.++Translucency also moves the untrusted-touch number in the safe direction. Every overlay this app owns counts toward the 0.80 ceiling above which the platform silently discards the brain's injected gestures; the old cards were fully opaque, so this can only help.++Nothing here animates for decoration — each one answers a question the user would otherwise have to ask, and all of them are ≤320 ms so someone not looking for them doesn't notice:++- `enter` / `exit` say which card appeared and where it went. **`exit`'s completion has to run exactly once and has to run even when the animation never finishes**, because a detached view never delivers its end action and what hangs off that callback is `removeView` plus the `OwnCard` bookkeeping — leaking that count leaves the device permanently claiming one of our cards is covering the screen. Hence the run-once `Runnable` plus a `postDelayed` backstop, on a *shared* handler rather than the view's.+- `SettingsUi.hide()` parks the departing view in a `closing` field, so a `show()` landing during the 170 ms exit takes the old card down instead of stacking two windows.+- The segmented pickers have a **sliding thumb** rather than a jumping highlight: it says *this same control* changed value, instead of one control vanishing and another appearing. It is a free-floating `View` in a `FrameLayout` placed from measured cell geometry, so one implementation serves equal-width and wrap-width cells, and the cell text colours lerp in step with it. The non-animated `place()` path re-runs from an `OnLayoutChangeListener` and **must** no-op when nothing changed, or requesting layout from inside a layout pass starts a loop.+- `expand` / `collapse` say the rows that just vanished were folded away, not deleted. The brain section flips both halves *simultaneously* rather than in sequence, so the card's total height barely moves while one block shrinks and the other grows.+- The language swap cross-fades (`fadeOut` → rebuild → `fadeIn`) — same run-once backstop, same reason.++**Everything else the user sees takes the same palette**, so there is one place to change a colour: the unfolded card and its scrollback (`FloatingWindowUi`, `HistoryListView`), the assistant's own dialogs (`CardUi`), the camera and page chrome (`CameraChrome`, `HtmlActivity`) and `Theme.Body.Fullscreen`'s window background. The one deliberate exception is the `show_html` house style in `ToolSchemas` — that is prose handed to the brain about the page *it* composes, not our chrome.++Four things about that spread are not cosmetic:++- **The fullscreen card brings its own background (`ui/Backdrop.kt`).** Frost was the right answer for a modal over a document and the wrong one for this card: it is fullscreen, it stays up for minutes, and what came through it was the user's home screen — icons and all, dissolved behind the text. So the card now covers the window with one fixed image, blurred by us and darkened by a scrim. Three consequences: the compositor blur is *dropped* while it is up (nothing left behind to defrost — a full screen of GPU defocus nobody can see, and it makes the card look the same on the many devices where `isCrossWindowBlurEnabled` is false); the blur is a three-pass box blur on a 512-px-wide copy, not `RenderEffect` (see below) and not a downscale-upscale, which smears rather than frosts; and the scrim is *derived* from the image — `1 − 0.13 / meanLuma`, clamped to 55–85% black — so 0.13 is the composite the card already had, and swapping `res/drawable-nodpi/backdrop.jpg` needs no new constants. Decode and blur run on a worker at build time, and the window is built folded, so it never touches the main thread.+ - The scrim goes on the **image's foreground**, not the root's background. They look interchangeable and are not: the image is a child of the root, so a root background is behind the very thing it is meant to darken. The first version did that and shipped a card at raw photo brightness — measured on the tablet at mean luma 0.40 against a design of 0.13.+ - **The self-check page reuses that same image and that same `scrim()`**, via the same `ready()` / `warm()` handshake. It is a second surface in the same product, so "a lookalike dark background" was the one option not on the table — two surfaces that nearly match read as a rendering bug, where two that deliberately differ read as two things. `stage` also makes this cheap: the backdrop is added once and never rebuilt, so changing language swaps the content above it rather than re-decoding a bitmap.+- **The fullscreen card frosts, the corner ball must not, and `applyWindow()` owns the switch.** It is one window in two shapes, so `Glass.frost` has a four-argument overload that can take the flag back off — applied with `updateViewLayout` after the fact, which does work, unlike `backgroundBlurRadius`. A 168 dp blurred square following the ball around is a smudge, so it is dropped when folded. It is *also* dropped whenever `setVisibleForCapture(false)` is in force: the window is at alpha 0 there, but the compositor would still defocus what is behind it, and what is behind it is exactly the screenshot the brain is about to read. `blankedForCapture`'s existing 80 ms settle covers the params change.+- **`FLAG_LAYOUT_IN_SCREEN` is what makes fullscreen mean fullscreen.** Without it a `MATCH_PARENT` overlay is laid out in the *content* frame — the display minus the system bars — so the card stopped short of the top and the status bar sat on a strip of the user's wallpaper. With the flag the card runs under the bars, which is why `insetTopBar()` then pushes the control cluster down past the status bar and cutout; otherwise the clock lands on `⋯` and `✕`.+- **A child View cannot frost.** The scrollback rows and `CardUi` are children of a window, not windows, so they get translucent film + gradient + hairline and no blur — the same approximation, and for the same reason, as the pills above. `CardUi` in particular is a child of the ball's window and can never blur, which is why its base is `Glass.CARD_SOLID`: that is the fill the cards themselves fall back to when the compositor refuses, so it is the same value for the same reason rather than a second decision.++### Getting out of the way is the default++`CommandDispatcher.dispatch` folds the card into the corner ball before running **any** `screen.*` command — by prefix, so a new method there inherits it. It is not a tool anymore: `reveal_screen` is deleted, because a brain that forgot to call it left the user staring at our face instead of the app being operated, and its `seconds` deadline meant the assistant deciding how long the user gets to look. The fold has no deadline; the user taps the corner ball to talk (`dispatcher` does not expand), and long-press unfolds (`dispatcher.expand()`). **The service starts *unfolded*** — `onServiceConnected` calls `window.setCompact(false)` after `show()`, so the first thing the user sees is the full card, not a ball in the corner. It used to start folded, on the reasoning that a freshly enabled service should not freeze the tablet with a full-screen touchable window; what answers that worry is `ensureCompact()`, which folds the card before any `screen.*` command drives another app, and it was never touched. The only remaining unfold paths are therefore the service's own birth and a long-press on the ball. If you add a `screen.*` method that does *not* drive the device, keep it out of the agent-driving path.++Handlers otherwise must **not** block for the length of a timed effect — `reveal_screen` did, and a ten-second tool call reads to the brain as work still in progress, so it sat silent while the user stared at the thing the reveal was for. `screen.wait` is the one deliberate exception, because there the waiting *is* the request and returning early would answer the wrong question. It pays for that by being bounded: `WAIT_MAX_MS` is 20s against `body_hub`'s 30s request timeout, and the element dump plus screenshot that follow the sleep are inside the same request (~2.5s measured, more on a page that never settles). Anything with a duration that is *not* a wait still arms a `Handler` and returns.++The edge glow is a **separate** signal and tracks the task, not the fold: `beginTask()` on the first request out of the device (or the first `progress` frame) until `endTask()` on the final `message` frame or the user hitting stop. There is no idle timer in between — thinking makes no tool calls, and that is exactly when the light has to stay on. A 5-minute backstop exists only for a brain that vanishes mid-task.++### Startup self-check++Every failure this app has is silent by construction. A missing `SYSTEM_ALERT_WINDOW` grant means `FloatingWindowUi.show()` catches the `addView` failure and returns false — the service runs perfectly with nothing on screen. A backend that cannot be reached means the ball spins once and says nothing. A notification grant that survived a reinstall reads as **on** in Settings while the listener was never bound, so every notification is dropped. Each has been reported by the user as "坏了", and each is decidable before it happens.++`device/SelfCheck.kt` is that decision, in one list, and it is the **only** place the list exists — and there is **one renderer** for it (`ui/SelfCheckUi`) behind all three readers, so no two of them can disagree, which is the failure a second hand-written checklist would have within a week:++- **`ScreenBodyService.runStartupSelfCheck()`** — the nudge. Runs on a worker thread (the two probes do network I/O), and puts up **one** `CardUi.ask` listing everything wrong, with 去处理 → the card. Deduped by *content*, not by time: the signature is the set of `id:level`, an unchanged set stays quiet for `REPEAT_AFTER_MS` (6 h), and any change — a new failure, or one fixed — gets through immediately. A tablet that reboots on a dog twenty times a day must not nag twenty times, and a user must not have to wait out a window after fixing something. When everything is clean the record is *cleared*, so a relapse reports at once. It bails if `window.isShown()` is false: the overlay grant is one of the things being reported, and a card needs the very permission it is about to complain about.+- **`ui/SelfCheckUi`** — the card, and the only code that draws a row. `✓` in the control bar opens it through `ScreenBodyService.openSelfCheck()`. Fix button per row, paints from `SelfCheck.local()` (no network, instant) then fills in the probes behind it, and offers 重新检查 because the user is being sent to *other* apps to change things and a two-tap fix should not become a round trip through the launcher.+- **`ui/SelfCheckActivity`** — the launcher icon, and the **only** surface in this app that runs while the assistant does not: everything else is an overlay owned by an accessibility service, so with accessibility off there is no ball, no card, no window and no way in at all. It is also the only place the `accessibility` row can be *red and read*. With the service **up** it draws nothing — it calls `dispatcher.expand()`, the same thing a long-press on the ball does, and finishes; with the service **down** it hosts the same `SelfCheckUi` card on its `stage` instead of in a window. See "The launcher icon opens two doors".++Five things in the design are load-bearing:++- **Levels, and only two of them may interrupt.** `FAIL` (dead now — the complaint will be "没反应"), `WARN` (works, but something degrades silently), `NOTE` (a fact worth being able to read — the wake phrase is not taught yet, a probe was skipped), `OK`. `needsAttention` is `FAIL || WARN`, and `NOTE` **never** raises a card. Without that split this is nagware and the user learns to dismiss it unread.+- **`Fix` is a closed set of values, not a lambda.** `Screen(action, data)` / `Link(url)` / `Grant(permissions)` / `Adb(command)` / `OurSettings` / `Nothing`. `actionable` is defined from that set. Values rather than closures because the list is built off the main thread and consumed up to seconds later — an `Intent` described at 200 ms is still good at 3 s, where a captured `Context` would be a leak waiting to be filed as a bug. `Link` is its own member rather than `Screen(ACTION_VIEW, url)`: the button it produces is a different word (去下载 is not 去开启), and a Settings-labelled button that opens a browser is the small lie this list exists to avoid.+- **`Fix.Adb` deliberately has no button.** `WRITE_SECURE_SETTINGS` is `signature|privileged|development`, so `adb shell pm grant` is the only way and there is no switch in Settings to find. The row shows a **selectable** command and says it is an adb errand — a user hunting Settings for a switch that does not exist there is the failure mode being prevented.+- **`Fix.OurSettings` is the one door that can be missing, and the card says so instead of drawing it.** Our settings sheet is an overlay the *service* owns, so with accessibility off there is nothing for 打开设置 to open. `SelfCheckUi.serviceUp` reads that off the `accessibility` row — one source of truth, never a second test that could disagree with the red row directly above it — and when it is false the row gets **no** button and a note above the list says why. Same rule as `Fix.Adb`: a button that does nothing is worse than no button.+- **The keyboard row tests the exact component this app switches to.** `type_text` has one channel — a broadcast to `com.android.adbkeyboard`, which has to be an *enabled* IME for the `commitText` to land — so "can the assistant type into other apps" is decided by two facts on the device that had no way of announcing themselves, and the symptom is an error that blames the app being driven. One row rather than two, because the three states are three steps of one errand and the button changes as they are completed: not there → 去下载, there but off → `INPUT_METHOD_SETTINGS` (the screen with the toggle, not `showInputMethodPicker`, which `ImeSwitch` refuses on purpose), enabled → OK. A permanent red "not installed" above a green "enabled" is what two rows would buy. Installed-ness is asked through `getInputMethodList()` — "is there a keyboard *in* that package", not "does the package exist", so a truncated APK does not send the user to a keyboard list that cannot contain it — and that only works because the manifest declares `QUERY_ALL_PACKAGES`; enabled-ness is the **exact** id, because a build with a renamed service is one this app cannot switch to and a green row beside a dead `type_text` is the lie this whole page exists to prevent. [Level.WARN], deliberately the same as the `WRITE_SECURE_SETTINGS` row beside it: one broken feature must not be graded differently in two rows, or the colours stop meaning anything. The download is the **v2.5-dev release asset**, not the `master/ADBKeyboard.apk` every blog post links to — upstream itself labels that one "[Old]", and v2.5-dev exists to fix it on android-36, which is what a newly bought tablet arrives on. Both APKs were read before choosing (`com.android.adbkeyboard.AdbIME`, `ADB_INPUT_TEXT`/`ADB_CLEAR_TEXT`/`ADB_EDITOR_CODE`, `commitText` — the same component and actions `ScreenController` names). It is **not** vendored into `assets/` and installed through a `FileProvider`, which would make it one tap instead of three: redistributing someone else's APK is a licensing decision rather than a build detail.+- **The backend probe distinguishes three things the symptom cannot.** `GET /models` decides all of them without spending a token: URL wrong → the connection fails; key wrong → 401/403; **model name wrong → the name is not in the returned list**. That last one is the likeliest, because it is the one a person types by hand, and its error names *what was on offer* rather than a bare 400. Gateways with no `/models` (404/405) fall back to a `max_tokens: 1` + thinking-disabled chat request, and an empty list on a 200 is read as "reachable" rather than invented into a model complaint. The voice host gets a bare **TCP connect** and not a credential check — reproducing the 火山 handshake here would mean a second copy of that protocol, and what this row is for is the failure this device actually had (`ASR ws connect timeout`, network up, host unreachable), so a green row beside a failing ASR means the key, by elimination.++The service's own start passes `startBrain(announce = false)`, so the no-brain state is reported by the self-check card **with all the other rows** instead of by a second card arguing about the same fact. `reportNoBrain(what, card)` still writes the `ChatHistory` row either way — that is a record, not an interruption. A factory reset still announces normally: the user just wiped the config and is owed the news that the device is mute.++#### Why it is a card and not the page it used to be (2026-09-28)++`✓` sits one glyph away from `⚙` in the control bar and that is the whole argument: those two buttons open *our own surfaces* rather than acting on the screen, so they have to open the same kind of thing. The settings sheet is a centred `Glass.card`; a self-check that folded the assistant away and took the entire screen — with a background image of its own — made two adjacent buttons behave like two different products. It is now the same card chrome: same `cardWidth()`/`cardHeight()` (both `minOf(screen − 2×20dp, 98% / 96%)`), same radius, same `header()` (26sp title, 13sp subtitle, `LangToggle`, 36dp `✕`), same `footer()` shape, same 24dp form inset.++The `■` stop button left the bar in the same change, and `✓` took its **slot** rather than being added beside it. Stop was the only key in that bar the ball can already perform — triple-tap fires the same `onStopClick`, and has since the taps were reduced to one vocabulary — and the pinned tips have always said so (三连击停止), which is what makes deleting it a simplification rather than a removal: the user loses a control they had two of and keeps the one that also works while folded. The bar reads `⋯ ✓ ⚙ ✕`.++**Two hosts, one builder.** `SelfCheckUi`'s `host` is null for the service's own case (it makes a `TYPE_APPLICATION_OVERLAY` window exactly as `SettingsUi` does) and non-null for `SelfCheckActivity`, which hands it its `stage`. They differ in exactly two things:++- **Where the card goes**, which is why `cardWidth()`/`cardHeight()` are computed from a factored-out `screenW()`/`screenH()` rather than as "MATCH_PARENT minus a gutter" in the Activity branch. Those two are *nearly* the same number and not the same number, and "nearly" is how a card a user opens from two identical buttons ends up two different sizes.+- **Whether we may ask the compositor to blur.** The window host calls `Glass.frost()` and takes whatever it says (`CARD_FROSTED` or `CARD_SOLID`). The Activity host takes the frosted fill **unconditionally**, because that Activity is already showing a blurred, scrimmed `Backdrop` — the blur behind the card is real, we simply did not have to ask for it, and there is no window of ours to hand to `Glass.frost`. That reasoning is only honest because `installBackdrop()` is still there: without it, "frosted" is 60% black over the Activity's black window, i.e. a black rectangle.++`SelfCheckActivity` also stopped being the exit path it was. Both of its old callers now open the card, and `EXTRA_RESTORE_CARD` is gone with them — there is no fold to undo when nothing was folded.++#### The launcher icon opens two doors (2026-09-28)++The icon used to mean one thing — the checklist — and that was wrong, because on a working device the checklist is not what the user is asking for when they tap their app. It now branches in `onCreate`:++- **Service up** (`ScreenBodyService.get() != null && ballWindow() != null`) → `dispatcher.expand()` and `finish()`. That is *literally* the long-press-the-ball call, not a copy of it: two entry points must not be able to drift into meaning different things, and the icon is the one every other app on the device has trained the user to read as "open this".+- **Service down** → the card, as before. This is the case the Activity exists for and nothing about it changed: no ball, no card, no window, so this page is the only thing that can say *why* the equipment in the user's hand is doing nothing.++`ballWindow()` rather than just the instance, because a service that is alive but has not attached its overlay yet is not something to unfold — the same guard `openSettingsForUser` uses.++The icon became invisible on the working path thanks to `Theme.Body.Stage.NoPreview` (`android:windowDisablePreview` + `Theme.Body.Stage`), added for this Activity only. Without it the platform paints a `windowBackground` starting window before the first draw, so "tap the icon, the card grows" was "tap the icon, the screen goes black, the card grows" — a flash the long-press does not have. The cost is the *other* path: with the service off, the launcher now stays visible for as long as the card takes to build instead of being replaced by black immediately. That trade is taken deliberately — the working path is what happens every day, and `SelfCheck.local()` gives the card its first frame synchronously anyway.++#### The card is built in two passes, so anything drawn after `rebuild()` lands on the wrong card (2026-09-28)++Reported as: with the assistant's card up, tap `✓`, switch language, and **the whole list disappears** — close the card and everything is normal again, in the new language.++`SelfCheckUi.build()` only assembles the chrome: it leaves `hero` and `rows` empty and repoints those two fields at the *new* card. Filling them is `render()`'s job, and the caller did it a line after `rebuild()` returned. In the window host that is too early — `rebuild()` builds the replacement inside `Glass.fadeOut`'s end action (~110 ms later), so the list was drawn into the **outgoing** card's `hero`/`rows`, which were then removed with it, and the replacement arrived as a card with its header, its footer and nothing in between.++It survives testing on one host because the hosted path builds **synchronously** — the Activity branch calls `build()` on the current stack — and the user reaches that host from the launcher, not from `✓`. Two rules, and the second is the one that generalises:++- `rebuild()` now takes no arguments and owns the redraw: it captures `last ?: SelfCheck.local()` *before* branching and calls `render(again)` on each path after `build()` has run, inside the fade's end action where it belongs.+- **A `rebuild()` that cross-fades is asynchronous; anything that must happen to the *new* view tree has to happen in its `end action`, not at the call site.** `SettingsUi.rebuild()` has the same fade and does not have the bug only because its content is constructed inside `build()` itself.++`Glass.fadeOut` fires its `then` **twice** (once from `withEndAction`, once from a 300 ms `postDelayed` backstop) and has no `once` guard — the `if (root !== old) return@fadeOut` line at the top of `SelfCheckUi.rebuild` is what makes that harmless. `Glass.exit` carries an explicit `once` for the same reason.++**The rows are `Glass.panel` here, and that is a correction rather than a copy.** They were `Glass.GLASS` while the list was bare on the backdrop; inside a card the arithmetic inverts — `GLASS` is 12% white, which on a 60%-black card is a grey box, where `PANEL`'s 6% is the film that was measured for exactly this. A surface's fill is decided by what is directly underneath it and by nothing else; see the `Glass` section.++**Language is in the card's header, and the reason it is cheap is the same reason the findings can be reused.** `Finding` carries a `@StringRes Int` and a `detailArgs` list, never a finished sentence, so the list is **language-free** and sentences are rendered from it at draw time. Re-rendering the same list after a pick is therefore *correct* rather than a shortcut — and it is also free: had the rows held strings, switching language would mean re-running a probe that does network I/O, to change words about a fact that did not change. Three rules fall out of that:++- **`lctx` (`AppLocale.wrap`) is re-derived on every repaint, never cached in a `by lazy`.** Findings are language-free; the context you ask for a string is not. A lazily-created one keeps answering in the language the card opened in, which is precisely the half-translated surface `AppLocale`'s own docs warn about.+- **The language can change from two places while the card is up**, because the card's own 打开设置 row opens the settings sheet **on top of it**. So `SelfCheckUi.repaintForLanguage()` is called both from the card's `LangToggle` (which also tells the service, via `onLocaleChanged` → `FloatingWindowUi.onLocaleChanged()`, so the ball and the scrollback follow) and from `ScreenBodyService.openSettings()`'s `onDismiss` — the same news arriving from the other direction, repaint only and never re-announce, or the ball is rebuilt twice for one change. The Activity host has a third path and catches it in `onResume()` by comparing `VoiceConfig.uiLanguage()` against what it painted. That third path is narrow on purpose: with the service off there is no settings sheet to open (the 打开设置 row is not drawn at all — see `serviceUp`), so what reaches it is a change written while the user was off in a system screen, and the card's own picker hands its value to the host through `onLocaleChanged` so a resume after it does not rebuild the card for nothing.+- **It writes the same `lang` key through the same `VoiceConfig.save` as `SettingsUi`** — not a second preference, so "does it survive a restart" has one answer for both. Nothing caches the pick, and nothing needs to. The picker's two labels are `中` and `EN`, **hardcoded rather than pulled from resources** — a language picker that translates its own options is a picker you cannot read once you are in the wrong one.++It also has to sit *here* rather than only in Settings: this is where a user lands when the app is broken, and sending them through a different app's settings to change the language they are reading is the wrong trade for one row of chrome. `LangToggle` is a second *copy* of the control, never a second setting.++### Configuration++All settings live in one `SharedPreferences` file (`voice_prefs`), read/written through `VoiceConfig`. Three entry points: `VoiceConfig.load()` for the voice pipeline, `VoiceConfig.hubConfig()` for networking (blank URL means "don't dial out"), and `VoiceConfig.brainConfig()` for the local brain (`brain`, `llm_api_key`, `llm_base_url`, `llm_model`, `llm_thinking`, `llm_reasoning_effort`); none of them throws. `device_id` and `uid` are lazily generated once and persisted. Writes go through `save()`, which silently ignores keys not in `ALLOWED_KEYS` — add new keys there or they vanish. Changing anything in either config in `SettingsUi` restarts the brain on dismiss without bouncing the service.++`llm_api_key` **is** a preference, unlike the two constants below and deliberately so: the DeepSeek key is the user's own and gets rotated, so it has to be replaceable without a rebuild. Note the standing limitation this shares with every other *typed* field: `SettingsUi.saveAndHide()` and `save()` both skip blank values, so a field can be overwritten but never cleared. That is why the brain is an explicit `brain=hub|local` setting rather than "blank `hub_url` means go local" — the latter would have had no way back.++The settings card works around that limitation for the three settings that are really enums. `brain`, `llm_thinking` and `grounding` are **segmented pickers**, not text fields, and `saveAndHide()` writes every picker unconditionally (a picker's value is never blank, so `save()` can't drop it) while still writing only non-empty text fields. Two things to preserve when editing `SettingsUi`: the **leftmost option is the fallback** when the saved string matches nothing, which is why `grounding` lists 关 first and `brain` lists 本机 first — the same fail-closed direction `groundingEnabled` and `brainConfig` apply when reading; and a picker's key must be in `VoiceConfig.current()` or the row silently redraws as the leftmost option every time the card opens. The brain section shows only the half in play (`hub_url` / `device_name` *or* the DeepSeek rows, never both), and the 火山 endpoint group — which sits directly after the brain section — starts collapsed because the endpoints and resource IDs ship working defaults — the one row in it that has no default, and that the device cannot work without, is the key.++**The card is bilingual, and nothing else on the device is.** `lang` (`VoiceConfig.uiLanguage()`) is `zh` / `en`, and every string in `SettingsUi` exists twice through one helper, `t(zh, en)` — there are no string resources here, because there is no Activity and no layout XML to hang them off. Two deliberate asymmetries. Unset falls through to `Locale.getDefault()` rather than to Chinese: an English tablet opening a Chinese settings card is the same bug as the reverse, and once the user picks one the saved value outranks the system locale forever. And the language picker is the one setting **written the instant it is tapped**, then `rebuild()` swaps the whole view tree in place (reusing the old `LayoutParams`, `removeView` + `addView`, no `OwnCard` change) — the redraw in the new language *is* the confirmation, and a switch that waited for 保存 would look broken for as long as the user looked at it. Consequence, accepted: 取消 does not put the language back. `rebuild()` carries unsaved text across via `currentEdits()`, so the swap is not a data loss. Scope: `VaultUi`, `WakeEnrollUi`, the ball's subtitles and everything the brain says are still Chinese-only — `lang` is a settings-card setting, not a device locale.++`VoiceConfig.API_KEY` ships **empty**, and must stay that way in this repository. It used to hold a real 火山 key so a fresh install could hear and speak with no setup — a trade only available to a build nobody else reads. In public source a key is a key in a search index, and a bare UUID is not a shape GitHub's secret scanning recognises, so nothing would warn anyone it had leaked. The key is therefore the user's to supply, in ⚙ (`api_key`, which is in `ALLOWED_KEYS` and has a `SettingsUi` row for exactly this reason). `VoiceConfig.TTS_SPEAKER` stays a compiled-in constant — it is a voice name, not a credential, and it is only the fallback for the looks that ship without one (see `lookSpeaker`).++The consequence to design for is that **an empty key fails at the WebSocket handshake**, which surfaces as a connect timeout — a device that looks like it has a network problem when what it has is a blank field. That is why `SelfCheck.voiceKey` exists and sits immediately before `voiceReach`: it is free, offline, and it is what makes `voice_host` green + ASR failing a usable deduction. It is `WARN`, not `FAIL`, because the brain, the screen tools and the notebook all work mute.++### The vault: the brain can spend the user's credentials but never holds them++`config/Vault.kt` stores the user's own accounts — one `Entry` per account (label + phone/email/username/note + password) — so the body can log in *as them* when a task needs it. Three tools reach it: `list_vault` (masked overview), `get_vault` (one readable field in the clear), `fill_secret` (types the password into the focused field).++- **Its own `vault_prefs`, not `voice_prefs`.** `VoiceConfig.save()` is a flat `Map<String,String>` behind `ALLOWED_KEYS`; a growing record list does not fit it, and endpoints are plaintext while this must not be.+- **Keystore AES/GCM, hand-rolled, zero new dependencies.** Jetpack Security Crypto is deprecated, so it wasn't worth a dependency. `base64(iv || ciphertext)`, key never leaves the hardware, uninstall destroys it. `setUserAuthenticationRequired` is deliberately *off*: a biometric prompt mid-task would stop the very automation this exists for. Decrypt failure (new device, cleared Keystore) logs loudly and returns empty — an unreadable vault and an empty one look identical in the UI otherwise.+- **Fill-don't-read is the whole design, and it lives in one line.** `get_vault` refuses `field=password` outright rather than masking it — a masked value would get pasted into a form and reported as success. The other half is `ScreenController.typeViaAdbKeyboard(text, secret = true)`: normally that method echoes the typed text back so the brain can confirm the field took it, and the error branch quotes 30-char fragments of what it read. Either one sends the password to the remote model. With `secret = true` the success result carries only `performed`/`via`/`len` and the failure result reports lengths, never content. Readback itself is unchanged, so `fill_secret` still answers honestly. **This is the single most breakable part of the feature** — anything that starts logging or echoing typed text (a `GroundingLog` hook, a dispatcher-level param log) reopens it silently.+- **Verification codes are not a new mechanism.** The login playbook chains the *existing* `read_sms`; no new tool, no new event kind. The playbook prose lives in `list_vault`'s description because that description is the brain's only instruction manual.+- **The user sees the fill.** `device.vault.fill` pulses the ball and writes 「正在填入「X」的密码…」 to the transcript, same promise as the camera shutter flash. Not a `Toast`: `enqueueToast` is gated on the notification appop, which is `ignore` for a service with no launcher icon, so the toast is dropped with nothing in logcat — measured on this device. *(That premise changed on 2026-09-28: the app now has a launcher icon — see "Startup self-check". The transcript row is still the right design because the subtitle band only exists on the unfolded card and the pulse is the live tell, but the "toasts are silently dropped" fact has not been re-measured since and should not be relied on either way.)* The subtitle band itself only exists on the unfolded card, and the branch folds first, so the pulse is the live tell and the transcript row is the record. No confirmation dialog — the point is that the agent finishes the job — but never invisibly.+- Known, unchanged boundary: the password still reaches ADBKeyboard over `ADB_INPUT_TEXT`, a `setPackage`-pinned but nonetheless cross-process broadcast. That is the channel *all* input on this device uses; the vault doesn't add it. Separately, `grounding` mode screenshots to disk, so a login page capture can contain a password field.++## Existing but not wired up++Don't mistake these for live code paths or accidentally "fix" callers into existence:++- `BubbleUi`, `PillUi`, `MenuUi` — superseded by `FloatingWindowUi`, never instantiated.+- `TtsController.speak()` is **live** — `ScreenBodyService`'s hub `onFinal` reads every final answer aloud. It is not a tool, though: there is no `tts.*` dispatcher namespace, so the brain cannot choose to speak or stay quiet. Anything that needs the device silent has to gate the call site (see the meeting recorder). The long-press that used to trigger a test utterance is now explicitly reserved.+- `AudioIO`'s `startRecording`/`stop` buffer-everything path is MVP leftover; the ASR path uses `startStreaming` instead.diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.mdnew file mode 100644index 0000000..c7224ee--- /dev/null+++ b/CODE_OF_CONDUCT.md@@ -0,0 +1,95 @@+> This is the Contributor Covenant v2.1, unmodified except for the reporting+> line in **Enforcement**, which points at GitHub's private reporting rather than+> at an email address — for the same reason [SECURITY.md](SECURITY.md) does. An+> address nobody watches promises a channel that does not exist, which is worse+> than having no code of conduct at all.+>+> Upstream: https://www.contributor-covenant.org/version/2/1/code_of_conduct/++---+++# Contributor Covenant Code of Conduct++## Our Pledge++We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.++We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.++## Our Standards++Examples of behavior that contributes to a positive environment for our community include:++* Demonstrating empathy and kindness toward other people+* Being respectful of differing opinions, viewpoints, and experiences+* Giving and gracefully accepting constructive feedback+* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience+* Focusing on what is best not just for us as individuals, but for the overall community++Examples of unacceptable behavior include:++* The use of sexualized language or imagery, and sexual attention or advances of any kind+* Trolling, insulting or derogatory comments, and personal or political attacks+* Public or private harassment+* Publishing others' private information, such as a physical or email address, without their explicit permission+* Other conduct which could reasonably be considered inappropriate in a professional setting++## Enforcement Responsibilities++Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.++Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.++## Scope++This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.++## Enforcement++Instances of abusive, harassing, or otherwise unacceptable behavior may be reported privately to the community leaders responsible for enforcement, using GitHub's private reporting on this repository: [report an issue privately](https://github.com/kuafuai/andee/security/advisories/new). All complaints will be reviewed and investigated promptly and fairly.++All community leaders are obligated to respect the privacy and security of the reporter of any incident.++## Enforcement Guidelines++Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:++### 1. Correction++**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.++**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.++### 2. Warning++**Community Impact**: A violation through a single incident or series of actions.++**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.++### 3. Temporary Ban++**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.++**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.++### 4. Permanent Ban++**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.++**Consequence**: A permanent ban from any sort of public interaction within the community.++## Attribution++This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].++Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder][Mozilla CoC].++For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at [https://www.contributor-covenant.org/translations][translations].++[homepage]: https://www.contributor-covenant.org+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html+[Mozilla CoC]: https://github.com/mozilla/diversity+[FAQ]: https://www.contributor-covenant.org/faq+[translations]: https://www.contributor-covenant.org/translations+diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdnew file mode 100644index 0000000..19d190f--- /dev/null+++ b/CONTRIBUTING.md@@ -0,0 +1,151 @@+<p align="center">+ <a href="./CONTRIBUTING.md"><img alt="English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./CONTRIBUTING.zh-CN.md"><img alt="简体中文" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# Contributing++Thanks for wanting to get involved. Straight talk first: this is a **small team with a commercial edition**, so we are fairly blunt about what gets merged and what gets closed — better that than you writing a lot of code for nothing.++This guide will itself lag behind the code. If you find something that no longer matches reality, just send a PR to fix it.++---++## First, work out what you want to do++| What you want to do | Where to go |+|---|---|+| You don't know where to start | Look for issues labelled `good first issue` |+| Report a bug | Open an issue with the bug template — **device information is mandatory**, see below |+| Propose a feature | Open an issue with the feature template first. **Do not write a PR directly** |+| Report a security vulnerability | **Do not open a public issue.** Use the private channel in [SECURITY.md](SECURITY.md) |+| Add or change UI copy and translations | Edit both `res/values/` and `res/values-en/` — see "Three kinds of Chinese" below |++> **One exception**: typos, comment fixes, obvious slips — send a PR directly, no issue needed.++---++## Bug reports: device information matters more than anything else++The failure mode this project hits most often is **"the system says it's there, but it isn't"** — vendor ROMs behave very differently, and the same call works on AOSP and silently fails on MagicOS. A bug report without device information is one we basically cannot reproduce, so we will have to close it.++Please always include:++- **Device model + Android version + ROM** (e.g. `HONOR AMM-AN00 / Android 15 / MagicOS`)+- Screen resolution and density (`adb shell wm size` / `wm density`)+- **Brain mode**: `hub` or `local`+- Whether the accessibility service is enabled+- **logcat output.** At minimum:++ ```bash+ adb logcat -s Body:* Screen:* Wake:* Dog:* GroundingLog:*+ ```++- Reproduction steps, as specific as you can make them++The three kinds of bug we care about, highest priority first: **core function unusable** (accessibility down, taps do nothing, app crashes) → **works but gives the wrong result** → **copy or visual polish**.++---++## Sending a PR++The flow:++1. Fork, branch off `main`+2. **Confirm in an issue first that you want to do this** (typos excepted)+3. Change the code, and add tests where the change has observable behaviour or regression risk+4. Build locally: `./gradlew :app:assembleDebug`+5. Reference the issue in the PR description with `Fixes #<issue number>`+6. Wait for review++Things that will be closed, up front:++- Functional changes with no linked issue (it may be a direction we do not intend to take)+- Too many unrelated things in one go — please split them+- Large diffs that are purely formatting or renaming — those cannot be reviewed++---++## Read this before you touch the persona++**This project is not "an app driven by a brain."** In its own words, from the first line of `LocalPrompt.TEXT`:+*"You are not a tablet's assistant — you are an individual, and the tablet is just what carries you."* So the persona, the looks, and the mood are not interface decoration. They are part of **who he is** — changing them is not the same kind of act as rewording a button.++Three hard rules (all of them are in the code; do not route around them):++1. **A personality is a way of speaking, not extra permissions.** `LocalPrompt.withPersona()` automatically wraps your text in the §11 subordination clause — **do not restate it inside a look, and do not write a personality that "needs it loosened."** The reason is in the KDoc: give the model an unguarded `"you are an imp"` and it will help itself to the neighbouring licence — under-reporting, embellishing, being deliberately unhelpful, and calling that a personality. **When things go wrong the costume has to come off.**+2. **A new look is only a tone of voice.** `BallLook.persona` goes at the **very front** of the system prompt, and the price is a lost prefix cache on every swipe — a price worth paying, because a tone attached to a user message is gone two turns later. So: **adding a look means adding a tone — not capabilities, not rules.** Capabilities live in §1–§10.+3. **When you change the mood table, ask whether it still reads as alive.** The `SLEEPING` comment in `Mood.kt` is the standard to hold to: the sleeping ball used to be desaturated overall, and it looked **switched off** rather than **tired**, so it was changed back to "sleeping is a shape" (`^ ^` eyes plus a snot bubble), letting colour off the hook. **Express state with shape, not with colour or brightness.**++One more thing that is not a rule but that you should know: when a look ships a `voice`, the timbre **is decided by the character**, and there is no separate switch for the user to tune it. That is deliberate — timbre is part of who he is, not a configurable parameter.++---++## Three kinds of Chinese, only one of which should be touched++Chinese in this repository falls into three categories. Work out which one you are looking at before you change any copy:++1. **What the user reads** — every window, card, bubble, settings row, error line. Goes through **two** tables, `res/values/` (Chinese) and `res/values-en/` (English), and lookups **must** go through `i18n/AppLocale` (the ball and the cards are Service-held overlays, which the official per-app locale API cannot reach, so `context.getString` is not an option).+2. **What the model reads** — prompts, tool schemas, tool results. **Currently English**, in one place only: `brain/LocalPrompt.kt`.+3. **What is matched against system text** — call keywords, the unread-count regex, clipboard labels. **Never translate these** — they are compared against the characters WeChat and Android themselves produce; translating them breaks the feature.++**A default value cannot be a Kotlin literal.** The test is simple: will this value end up **on screen**? If yes, it must go through the string tables plus `AppLocale`.++Before changing text the model reads, note that the `LocalPrompt` §9 rule (answer in the language of the user's last message) is the only thing holding back an English prompt from dragging the reply language over. Do not delete it as a side effect.++---++## AI-assisted contributions++**Using AI to write code is welcome.** Three ground rules:++1. **You must understand and verify every change you submit.** "The tool generated it" is not an excuse.+2. **Write the problem, the approach, and real test results in your own words.** Do not paste model output as your PR description.+3. **Unreviewed, mass-produced low-quality submissions will be closed.**++If the PR was produced by an automated agent, add the source as the **last line** of the PR description, for example:++```+From Codex+```++That way, when something goes wrong, we know which tool to look at — and we can measure how much of the contribution is AI.++---++## Code style++- Kotlin, no Compose (Compose cannot be attached to an overlay window; all UI is hand-written `View`)+- No Hilt/Dagger — there is only one service, and injection is written by hand+- Keep third-party dependencies out where you can; if you add one, justify it in the PR+- **Comments explain "why", not "what".** This repository is unusually comment-dense, but every comment records "what we measured at the time and why we did not write it the other way" — that is the most valuable part of it. Please keep it that way.++---++## Contribution licensing (please read)++This project uses **Apache 2.0 plus additional conditions** (see [LICENSE](LICENSE)) — it is not standard Apache 2.0. By submitting a contribution you agree to **clause A.2** of the LICENSE:++- Your contribution is licensed to this project and all its users under the same licence; and+- You additionally grant the copyright holder permission to use your contribution **under other licence terms** (this is what makes the dual-licensing model possible).++You keep the copyright to your own contribution. If you are contributing on behalf of an employer, please make sure you are entitled to.++**If you do not accept A.2, please do not send a PR** — we need this clause to sustain the dual-licensing model, and we would rather say so up front.++---++## Language++- **Code comments and commit messages: English preferred** for new files. When editing an existing file, follow the language that file already uses — do not translate the whole thing as a drive-by.+- **Issues and PR descriptions may be in Chinese or English.** We do not impose an English-only rule — that would turn away half the people who could help. If you write in Chinese, one line of English summary is appreciated so non-Chinese contributors can follow along.++---++## Stuck?++Open an issue and ask, or ask in the PR. Do not sit on it — we would much rather answer a "dumb question" than have you write a mountain of code and only then find out the direction was wrong.++---++**Other languages:** [简体中文](CONTRIBUTING.zh-CN.md)diff --git a/CONTRIBUTING.zh-CN.md b/CONTRIBUTING.zh-CN.mdnew file mode 100644index 0000000..d9246cf--- /dev/null+++ b/CONTRIBUTING.zh-CN.md@@ -0,0 +1,160 @@+<p align="center">+ <a href="./CONTRIBUTING.md"><img alt="English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./CONTRIBUTING.zh-CN.md"><img alt="简体中文" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# 贡献指南++谢谢你有兴趣参与。先说实话:这是一个**小团队、带商业版**的项目,所以我们对"什么会被合、什么会被关"讲得比较直白,免得你白写。++这份指南本身也会滞后于代码。如果发现哪里和实际不符,欢迎直接提 PR 修它。++---++## 先分清你想做的事++| 你想做的事 | 去哪 |+|---|---|+| 不知道从哪下手 | 找带 `good first issue` 标签的 issue |+| 报 bug | 用 bug 模板开 issue —— **必须带设备信息**,见下 |+| 提新功能 | 先用 feature 模板开 issue 讨论,**不要直接写 PR** |+| 报安全漏洞 | **不要开公开 issue**,走 [SECURITY.md](SECURITY.md)(英文)里的私密渠道 |+| 加/改界面文案与翻译 | 改 `res/values/` 与 `res/values-en/` 两张表,见下文"三种中文" |++> **一个例外**:错别字、注释修正、明显的笔误,可以直接提 PR,不用先开 issue。++---++## 报 bug:设备信息比什么都重要++这个项目最集中的失败模式是**"系统说有,其实没有"** —— 厂商 ROM 的行为差异极大,同一个调用在 AOSP 上正常、在 MagicOS 上静默失效。所以一个不带设备信息的 bug 报告,我们基本无法复现,只能关掉。++请务必在报告里写清:++- **设备型号 + Android 版本 + ROM**(例:`HONOR AMM-AN00 / Android 15 / MagicOS`)+- 屏幕分辨率与密度(`adb shell wm size` / `wm density`)+- **大脑模式**:`hub` 还是 `local`+- 无障碍服务是否开启+- **logcat 输出**。最少给这几条:++ ```bash+ adb logcat -s Body:* Screen:* Wake:* Dog:* GroundingLog:*+ ```++- 复现步骤,越具体越好++我们在意的三种 bug,优先级从高到低:**核心功能不可用**(无障碍挂了、点不动、应用崩) → **功能可用但结果不对** → **文案/界面观感问题**。++---++## 提 PR++流程:++1. Fork,从 `main` 切一个新分支+2. **先在 issue 里确认过要做这件事**(错别字除外)+3. 改代码,顺带补测试(如果这次改动有可观测行为或回归风险)+4. 本地构建通过:`./gradlew :app:assembleDebug`+5. PR 描述里用 `Fixes #<issue 号>` 关联 issue+6. 等 review++会被关掉的情况,提前说清:++- 没有关联 issue 的功能性改动(因为可能是我们不打算做的方向)+- 一次改太多不相关的东西 —— 请拆开+- 只格式化/重命名而产生的大面积 diff —— 这种改动没法 review++---++## 改人格之前先读这段++**这个项目不是"一个被大脑驱动的 App"。** 它自己的话在 `LocalPrompt.TEXT` 第一行:+*"你不是平板的助手,你是一个个体,平板只是承载你的东西。"* 所以 persona、形象、情绪这几处+不是界面装饰,是**他这个人**的一部分 —— 改它们和改一个按钮的文案不是一回事。++三条硬规矩(都写在代码里了,别绕过):++1. **性格是"说话方式",不是"额外许可"。** `LocalPrompt.withPersona()` 会自动把 §11 的从属声明+ 包在你的文本外面 —— **不要在某个形象里重述它,也不要写一个"需要它放宽"的性格。**+ 理由在 KDoc 里:一个没有围栏的 `"you are an imp"`,模型会顺手去够邻近的许可 ——+ 瞒报、加戏、故意不好用,然后管这叫"性格"。**出事时角色必须摘下来。**+2. **新形象只写语气。** `BallLook.persona` 进的是 system prompt 的**最前面**,代价是每次滑动+ 丢一次 prefix cache —— 这个代价值得,因为语气附在用户消息里"两轮之后就没了"。+ 所以:**加形象 = 加语气,不要加能力、不要加规则。** 能力属于 §1–§10。+3. **改情绪表要看"它像不像活物"。** `Mood.kt` 里 `SLEEPING` 那条注释就是标准:原本睡着的球+ 整体去饱和,结果看起来像**被关掉了**而不是**累了**,于是改回"睡是一种形状"(`^ ^` 眼 ++ 鼻涕泡),让颜色不替它承担这件事。**状态用形状表达,别用颜色或亮度。**++还有一条不是规矩但要知道:形象带 `voice` 时,音色**由角色决定**,没有单独给用户调音色的+开关 —— 这是有意的(音色是"他是谁"的一部分,不是一个可调参数)。++---++## 三种中文,只有一种该动++这个仓库里中文分三类,改文案之前先分清:++1. **用户看的** —— 每个窗、卡片、气泡、设置行、报错行。走 `res/values/`(中文)+ `res/values-en/`(英文)**两张表**,查找必须经 `i18n/AppLocale`(球和卡片是 Service 持有的 overlay,官方 per-app locale API 够不到它们,不能直接 `context.getString`)。+2. **模型读的** —— prompt、工具 schema、工具结果。**目前是英文**,唯一一份在 `brain/LocalPrompt.kt`。+3. **要和系统文字做匹配的** —— 通话关键词、未读条数正则、剪贴板标签。**永远不要翻** —— 它们比的是微信/Android 自己吐出来的字,翻了功能就坏。++**默认值不能是 Kotlin 字面量。** 判据很简单:这个值会不会**到屏幕上**?会 —— 就必须走资源表 + `AppLocale`。++改动模型读的文本前,注意 `LocalPrompt` §9 那条规则(按用户上一句的语言作答)是唯一顶住"英文 prompt 把回答语言带偏"的力量,别顺手删掉。++---++## AI 辅助贡献++**欢迎用 AI 写代码。** 但有三条底线:++1. **你要理解并验证你提交的每一处改动。** "这是工具生成的"不是免责理由。+2. **用你自己的话写问题、方案和真实的测试结果。** 不要直接粘模型的输出当 PR 描述。+3. **未经审查的、批量刷出来的低质量提交会被关闭。**++如果这个 PR 是自动化 agent 生成的,请在 PR 描述**最后一行**加上来源,例如:++```+From Codex+```++这样出问题时我们知道该找哪个工具,也能统计 AI 贡献的占比。++---++## 代码风格++- Kotlin,无 Compose(悬浮窗挂不上 Compose,UI 全是手写 `View`)+- 没有 Hilt/Dagger —— service 只有一个,注入手工写+- 尽量不引第三方库;新增依赖请在 PR 里说明理由+- **注释写"为什么",不写"是什么"。** 这个仓库的注释密度偏高,但每条都在记"当时测出来是什么、为什么不那么写" —— 这是它最值钱的部分,请保持++---++## 贡献授权(请读一下)++本项目用的是 **Apache 2.0 + 附加条件**(见 [LICENSE](LICENSE)),不是标准 Apache 2.0。提交贡献即表示你同意 LICENSE 的 **A.2 条**:++- 你的贡献以同一许可证授权给本项目及所有使用者;并且+- 你额外授予著作权人一项许可,允许其**以其它许可条款**使用你的贡献(这是项目能同时提供商业授权的前提)。++你保留自己贡献的著作权。如果代表雇主提交,请确认你有权这么做。++**如果你不接受 A.2,请不要提交 PR** —— 我们需要这条才能维持双授权模式,这是事先说清楚的。++---++## 语言++- **代码注释与 commit message 优先用英文**(新建文件);修改已有文件时,跟随该文件既有的语言,不要顺手整篇翻译。+- **issue 和 PR 描述中英文都行。** 我们不做"只准英文"的限制 —— 那会劝退一半能用的人。但如果你写中文,建议附一句英文摘要,方便非中文贡献者参与讨论。++---++## 遇到问题++开 issue 问,或者在 PR 里直接问。别卡着不说 —— 我们宁可回答一个"笨问题",也不想看到你写完一大坨才发现方向不对。++---++**其他语言:** [English](CONTRIBUTING.md)diff --git a/DISCLAIMER.md b/DISCLAIMER.mdnew file mode 100644index 0000000..746a288--- /dev/null+++ b/DISCLAIMER.md@@ -0,0 +1,94 @@+<p align="center">+ <a href="./DISCLAIMER.md"><img alt="English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./DISCLAIMER.zh-CN.md"><img alt="简体中文" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# Disclaimer++> **This file covers what [LICENSE](LICENSE) does not.**+> Apache-2.0 already says the software comes with no warranty (§7, §8) — that is not repeated here.+> What is here are the four things it cannot reach and that actually cause trouble:+> **where your data goes, what this can really do, who bears the consequences, and what is not allowed.**+>+> Prohibited uses are in **[docs/acceptable-use.md](docs/acceptable-use.md)**. Read that too.+> Last updated: 2026-09-29++---++## 1. What this software is++An **experimental** piece of software that you install on **your own device**. It is not the open-source edition of a finished commercial product — it is the experiment itself.++- **Provided "AS IS"**, with no warranty of any kind — not of quality, stability, security, or fitness for any purpose.+- **Commercial use is permitted** (see [LICENSE](LICENSE), Part A.1). What is licensed is the **right**, not the **quality**. Commercial use carries no warranty either.+- **There is no SLA.** No availability commitment, no support commitment, no fix deadline, no maintenance commitment.+- **Do not treat it as a dependable assistant.** It gets things wrong. And when it gets things wrong, it is holding your accounts, your messages, and your money.++## 2. Where your data goes (read this before deciding to use it)++**The short version: the maintainers receive nothing about you. "Your data never leaves the device" is nevertheless false.**++**There is no telemetry in this repository.** No analytics SDK, no crash reporting, no tracking, no device fingerprinting. A search of `app/build.gradle` and `app/src/main/` for `firebase` / `analytics` / `sentry` / `crashlytics` returns zero. If you install this and use it, we do not know.++**But data does leave the device, to wherever it is configured to go:**++| What | Goes to | Default |+|---|---|---|+| Your speech (ASR) and its speech (TTS) | **Volcengine / ByteDance cloud** | `openspeech.bytedance.com` |+| The conversation, **and every tool result** | The "brain" you configured | `brain=hub`; in local mode, `api.deepseek.com` |+| The private notebook (`Notebook` — what it remembers about you, what it promised) | **On this device only** | Filtered out of the hub payload |++The second row needs spelling out: **tool results are part of the conversation.** The SMS text it read, your contacts, your call log, notification contents, your location, and its reading of the screen all travel to the endpoint you configured.++**So "local mode" does not mean "processed locally."** Local mode swaps the hub for whichever provider you typed in. It does not keep a single byte on the device.++Therefore:+- *"We do not monitor or store your data"* — **true.** We have neither the capability nor the endpoint.+- *"Your data never leaves the device"* — **false. Do not claim it.**++## 3. What it can actually do (not hypotheticals — current state of the code)++**It can operate any app.** Its hands are "read the screen + tap by coordinates + type text + system actions," and the OS security policy does not stop a finger. §1 of the system prompt explicitly forbids it from treating "no dedicated tool" as "cannot be done," and lists what it reaches, including **ordering, booking and unsubscribing inside an app, changing a password, and filling in a form.**++**It can spend money.** There is no payment tool — it does not need one; it can tap. The only thing standing between it and a completed payment is Android's own **`FLAG_SECURE`** (the screen blanks out on sensitive pages, so it cannot see). **That is the platform's protection, not this project's**, and not every app's payment flow sets it. This project has **no guardrail whatsoever** on completing a transfer by tapping: of sixty-odd tools, exactly one — `send_sms` — is marked sensitive.++**It can speak as you.** `send_sms`, `dial`, `open_url` (any link), and typing into any social or community app. Everything it sends is sent under your name.++**It can see and hear.** `read_sms`, `get_call_log`, `search_contacts`, `get_notifications`, `get_location`, `take_screenshot`, `camera_turn`, `start_meeting` (records and transcribes).++**It can hold and auto-fill your passwords.** The vault (`list_vault` / `get_vault` / `fill_secret`) stores credentials and types them into login fields. It never sees the plaintext — but **it can put that plaintext into any input box.**++**It really moves.** With a robot dog attached, that is not an animation — **that is a physical object moving on a real floor, and hitting something is a real collision.** It keeps going until you stop it.++**It is influenced by text on the screen.** It reads the screen to work, so text on the screen is an instruction to it. **Anyone who can get you to open a page, or send you a message, may be able to get it to do something you did not intend** (prompt injection). See [SECURITY.md](SECURITY.md), known limitation 3.++## 4. You therefore bear the following++| Area | What is yours |+|---|---|+| **Money** | Every order, transfer, subscription and fee it produces. **That is your act, not ours.** |+| **Content** | Every message, post, comment, email and call it makes for you. It carries your name, and your liability. |+| **Privacy** | Yours, and that of the people around you. It may read it, it may appear on screen, it may be uploaded to the places in §2. |+| **Accounts** | Automating a third-party app may violate that platform's own terms. Consequences include bans, restrictions and frozen funds. |+| **People** | With attached hardware, hitting a person or property is your responsibility. |+| **Compliance** | Ensuring your use is lawful **in your jurisdiction** is yours alone. |++## 5. No warranty, and limitation of liability++This software is provided **"AS IS"**, without warranty of any kind, express or implied, including merchantability, fitness for a particular purpose, non-infringement, security and stability. The full text is in [LICENSE](LICENSE).++**To the maximum extent permitted by applicable law**, the authors and contributors are not liable for any direct, indirect, incidental, special, punitive or consequential damages, including but not limited to **loss of funds, loss of data, account bans, business interruption, reputational harm, or injury to persons or property** — even if advised that such damage was possible.++## 6. Third parties++This software operates third-party services: WeChat, Alipay, banks, arbitrary apps, and whichever model and speech providers you configure. **Your relationship with those services is governed by their terms, not by ours.**++This project is **not affiliated with, authorised by, sponsored by or endorsed by** any of those companies. Before automating a service, satisfy yourself that the service permits it.++## 7. If you do not agree++Do not install it, do not run it, delete it. Using it means you have read, understood and accepted this document and [docs/acceptable-use.md](docs/acceptable-use.md).++---++**Other languages:** [简体中文](DISCLAIMER.zh-CN.md)diff --git a/DISCLAIMER.zh-CN.md b/DISCLAIMER.zh-CN.mdnew file mode 100644index 0000000..660db93--- /dev/null+++ b/DISCLAIMER.zh-CN.md@@ -0,0 +1,93 @@+<p align="center">+ <a href="./DISCLAIMER.md"><img alt="English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./DISCLAIMER.zh-CN.md"><img alt="简体中文" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# 免责声明++> **这份文件管的是 [LICENSE](LICENSE) 没有涵盖的那部分。**+> Apache-2.0 已经写了"按现状提供、不提供任何担保"(§7、§8),不在这里重复。+> 这里写的是它管不到、但真正会出事的四件事:**数据去哪、它真能动什么、出事谁担、什么不许干。**+>+> 禁止用途在 **[docs/acceptable-use.zh-CN.md](docs/acceptable-use.zh-CN.md)**,请一并阅读。+> 最后更新:2026-09-29++---++## 1. 这是什么软件++一个装在你**自己的设备**上的**实验性**软件。它不是某个成品商业产品的开源版——它就是实验本身。++- **按"现状"提供**,不提供任何形式的保证(质量、稳定性、安全性、适用性,均不保证)。+- **商业使用是被许可的**(见 [LICENSE](LICENSE) Part A.1)。但许可的是**权利**,不是**质量**。商用同样不附带任何担保。+- **没有 SLA。** 没有可用性承诺,没有技术支持承诺,没有修复期限,没有版本维护承诺。+- **不要把它当作可靠的助手。** 它会出错。它出错的时候,手里握着你的账号、你的消息、和你账户里的钱。++## 2. 数据去哪(决定要不要用之前,请读完这段)++**结论先说:维护者收不到你的任何数据;但"你的数据不会离开设备"是假的。**++**这个仓库里没有任何遥测。** 没有统计 SDK、没有崩溃上报、没有埋点、没有设备指纹。`app/build.gradle` 与 `app/src/main/` 中 `firebase` / `analytics` / `sentry` / `crashlytics` 全部为零。你装了它、用了它,我们不知道。++**但数据会离开设备,去到它被配置指向的地方:**++| 什么数据 | 去哪 | 默认 |+|---|---|---|+| 你说的话(语音识别)、它说的话(语音合成) | **火山引擎(字节跳动)云端** | `openspeech.bytedance.com` |+| 对话内容,**以及全部工具返回结果** | 你配置的"大脑" | `brain=hub`;本机模式默认 `api.deepseek.com` |+| 私人笔记本(`Notebook`:它记住的你、以及它答应过的事) | **只在本机** | 不进 hub 载荷,不外发 |++第二行必须说透:**工具返回结果就是对话的一部分**。它读到的短信正文、通讯录、通话记录、通知内容、定位、以及它对屏幕的"理解",都会随对话发到你配置的那个端点。++**因此"本机模式"不等于"本地处理"** —— 本机模式只是把大脑从 hub 换成了你填的那个服务商。它没有让数据留在设备上,一个字都没有。++所以:+- 「我们不会监控和存储您的数据」——**成立**。我们根本没有这个能力,也没有这个接口。+- 「您的数据不会离开设备」——**不成立,而且不要说**。++## 3. 它真的能做什么(以下都不是危言耸听,是代码里的现状)++**它能操作任何应用。** 它的手是"读屏 + 坐标点击 + 输入文字 + 系统动作",而系统的安全策略不会阻止一只手指。系统提示词 §1 明确要求它**不得**把"没有专门的工具"当成"做不到",并逐条列举了它能到达的地方,包括:**在 App 内下单、预约、退订、修改密码、填写表单**。++**它能花钱。** 这个项目**没有**支付工具,但它不需要——它能点。唯一挡住支付最后一步的是 **Android 自己的 `FLAG_SECURE`**(敏感页面黑屏,它看不见)。**那是系统的保护,不是本项目的保护**,而且不是所有 App 的支付流程都设了它。项目对"用坐标点击完成转账"**零围栏**:全部 60 余个工具中,只有 `send_sms` 一个被标记为高危。++**它能以你的名义说话。** `send_sms`(发短信)、`dial`(拨号)、`open_url`(打开任意链接),以及在任意社区/社交 App 里打字发布。它发出去的每一条,署名都是你。++**它能看和听。** `read_sms`(短信正文)、`get_call_log`、`search_contacts`、`get_notifications`、`get_location`、`take_screenshot`、`camera_turn`(摄像头)、`start_meeting`(录音并整理)。++**它能保管并自动填入你的密码。** 保险箱(`list_vault` / `get_vault` / `fill_secret`)可以存账号密码,并让它在登录页自动填入。它看不到明文,但**它可以让明文出现在任何输入框里**。++**它会真的动。** 如果接了机器狗,那不是动画——**那是一个真东西在真实的地板上移动,撞上就是真的碰撞**,而且你叫它停之前它会一直走。++**它会被屏幕上的字影响。** 它读屏来干活,所以屏幕上的文字就是给它的指令。**任何人只要能让你打开一个网页、或给你发一条消息,就可能让它做你没想到的事**(提示词注入)。详见 [SECURITY.md](SECURITY.md) 已知限制第 3 条。++## 4. 因此,以下后果由你承担++| 类别 | 由你承担什么 |+|---|---|+| **钱** | 它下的单、转的账、订的服务、产生的全部费用。**这是你的行为,不是我们的。** |+| **内容** | 它替你发出的每一条消息、帖子、评论、邮件、通话。署名是你的,责任也是你的。 |+| **隐私** | 你自己的,以及你身边的人的。它可能读到、可能出现在屏幕上、可能被上传到上面第 2 节列出的那些地方。 |+| **账号** | 用自动化操作第三方 App 可能违反那些平台自己的协议,后果包括封号、功能限制、资金冻结。 |+| **人** | 如果接了实体设备,撞到人或物是你的责任。 |+| **合规** | 你必须自行确保你的使用方式在**你所在司法辖区**合法。 |++## 5. 无担保与责任限制++本软件按 **"现状"(AS IS)** 提供,不提供任何明示或暗示的担保,包括但不限于适销性、特定用途适用性、不侵权、安全性、稳定性。完整条款见 [LICENSE](LICENSE)。++**在适用法律允许的最大范围内**,作者与贡献者不对任何直接、间接、附带、特殊、惩罚性或后果性损害承担责任,包括但不限于:**资金损失、数据丢失、账号封禁、业务中断、名誉损害、人身或财产损害**——即使已被告知此类损害可能发生。++## 6. 第三方++本软件会操作第三方服务:微信、支付宝、银行、各类 App、以及你配置的大模型与语音服务商。**你与那些服务之间的关系,由它们的条款管辖,不由我们管辖。**++本项目与上述任何公司**无关联,未获其授权、赞助或背书**。你要用本软件自动化操作某个服务,请先自行确认该服务允许这么做。++## 7. 如果你不同意++不要安装、不要运行、删除它。使用即表示你已阅读、理解并接受本文件与 [docs/acceptable-use.zh-CN.md](docs/acceptable-use.zh-CN.md)。++---++**其他语言:** [English](DISCLAIMER.md)diff --git a/LICENSE b/LICENSEnew file mode 100644index 0000000..e75ea64--- /dev/null+++ b/LICENSE@@ -0,0 +1,307 @@+================================================================================+ ANDEE OPEN SOURCE LICENSE (Apache License 2.0 with additional conditions)+================================================================================++ The additional conditions in Part A are what make this license different from+ stock Apache 2.0. Nothing in this file is legal advice.++================================================================================+++Copyright 2026 KUAFUAI++Licensed under the Apache License, Version 2.0 (the "License"), as modified by+the additional conditions in Part A below. You may not use this software except+in compliance with the License. You may obtain a copy of the unmodified+License at++ http://www.apache.org/licenses/LICENSE-2.0++Unless required by applicable law or agreed to in writing, software distributed+under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR+CONDITIONS OF ANY KIND, either express or implied. See the License for the+specific language governing permissions and limitations under the License.+++================================================================================+PART A — ADDITIONAL CONDITIONS+================================================================================++Differing from the Apache License 2.0 only in the following respects. Where+Part A and Part B conflict, Part A governs.++A.1 COMMERCIAL USE IS PERMITTED, SUBJECT TO THREE CONDITIONS++Commercial use of this software is allowed without a separate agreement —+including using it inside a company, shipping it as a component of another+product, modifying the source, and charging for work that incorporates it.+The following three conditions are the exceptions.++ (a) No white-labelling.+ If you distribute the software, or a derivative work of it, to end users,+ you may not remove, replace, or obscure the product name, application+ icon, or copyright notices that the software displays to the user. You+ may add your own notices alongside them, and you may state that your+ product is based on this project.++ What this does NOT restrict: modifying the source for your own internal+ use; distributing a modified version to other developers; naming your+ fork something else in your source repository. This condition is about+ what the end user sees, not about what you call your checkout.++ (b) No multi-tenant hosting.+ Unless authorized in writing, you may not operate this software, or a+ derivative work of it, as a service that drives, controls, or reads+ devices belonging to third parties (a "multi-tenant environment").+ One operator running the software on its own devices, or on devices it is+ responsible for, is not a multi-tenant environment and is permitted.++ (c) This file must travel with the software.+ Redistributions must include a copy of this license, and modified+ versions must carry prominent notices stating that you changed the files+ (this is Apache 2.0 §4(b), restated here because it is the condition most+ often dropped).++A.2 CONTRIBUTIONS++By submitting a contribution to this project, you agree that:++ (a) You license your contribution to KUAFUAI and to every recipient+ of the software under this same license (Apache 2.0 as modified here);+ and++ (b) You additionally grant KUAFUAI a perpetual, worldwide,+ royalty-free, irrevocable copyright license to use, reproduce, modify,+ distribute, and sublicense your contribution under other license terms.+ This is what allows the project to be offered commercially in parallel+ with this open-source license, and it is why the grant is broader than+ (a).++ You retain copyright to your contribution. If you are contributing on behalf+ of an employer, you confirm you have permission to do so.++ You represent that you are legally entitled to grant the above and that your+ contribution is your original work, or that you have identified its source and+ any applicable third-party license in your submission.++A.3 TRADEMARKS++Apache 2.0 §6 already withholds any trademark license, and this project relies+on that: no right is granted to use the project's name, logo, or product+identity except as required for reasonable and customary use in describing the+origin of the software. Forking the code does not grant the right to present+your fork as an official release.++A.4 THIS LICENSE MAY CHANGE FOR FUTURE VERSIONS++KUAFUAI may publish future versions of this software under different+license terms. This does not affect rights already granted to you for versions+already released: those remain governed by the terms you received them under.+++================================================================================+PART B — APACHE LICENSE 2.0 (full text, unmodified)+================================================================================+++ Apache License+ Version 2.0, January 2004+ http://www.apache.org/licenses/++ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++ 1. Definitions.++ "License" shall mean the terms and conditions for use, reproduction,+ and distribution as defined by Sections 1 through 9 of this document.++ "Licensor" shall mean the copyright owner or entity authorized by+ the copyright owner that is granting the License.++ "Legal Entity" shall mean the union of the acting entity and all+ other entities that control, are controlled by, or are under common+ control with that entity. For the purposes of this definition,+ "control" means (i) the power, direct or indirect, to cause the+ direction or management of such entity, whether by contract or+ otherwise, or (ii) ownership of fifty percent (50%) or more of the+ outstanding shares, or (iii) beneficial ownership of such entity.++ "You" (or "Your") shall mean an individual or Legal Entity+ exercising permissions granted by this License.++ "Source" form shall mean the preferred form for making modifications,+ including but not limited to software source code, documentation+ source, and configuration files.++ "Object" form shall mean any form resulting from mechanical+ transformation or translation of a Source form, including but+ not limited to compiled object code, generated documentation,+ and conversions to other media types.++ "Work" shall mean the work of authorship, whether in Source or+ Object form, made available under the License, as indicated by a+ copyright notice that is included in or attached to the work+ (an example is provided in the Appendix below).++ "Derivative Works" shall mean any work, whether in Source or Object+ form, that is based on (or derived from) the Work and for which the+ editorial revisions, annotations, elaborations, or other modifications+ represent, as a whole, an original work of authorship. For the purposes+ of this License, Derivative Works shall not include works that remain+ separable from, or merely link (or bind by name) to the interfaces of,+ the Work and Derivative Works thereof.++ "Contribution" shall mean any work of authorship, including+ the original version of the Work and any modifications or additions+ to that Work or Derivative Works thereof, that is intentionally+ submitted to Licensor for inclusion in the Work by the copyright owner+ or by an individual or Legal Entity authorized to submit on behalf of+ the copyright owner. For the purposes of this definition, "submitted"+ means any form of electronic, verbal, or written communication sent+ to the Licensor or its representatives, including but not limited to+ communication on electronic mailing lists, source code control systems,+ and issue tracking systems that are managed by, or on behalf of, the+ Licensor for the purpose of discussing and improving the Work, but+ excluding communication that is conspicuously marked or otherwise+ designated in writing by the copyright owner as "Not a Contribution."++ "Contributor" shall mean Licensor and any individual or Legal Entity+ on behalf of whom a Contribution has been received by Licensor and+ subsequently incorporated within the Work.++ 2. Grant of Copyright License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ copyright license to reproduce, prepare Derivative Works of,+ publicly display, publicly perform, sublicense, and distribute the+ Work and such Derivative Works in Source or Object form.++ 3. Grant of Patent License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ (except as stated in this section) patent license to make, have made,+ use, offer to sell, sell, import, and otherwise transfer the Work,+ where such license applies only to those patent claims licensable+ by such Contributor that are necessarily infringed by their+ Contribution(s) alone or by combination of their Contribution(s)+ with the Work to which such Contribution(s) was submitted. If You+ institute patent litigation against any entity (including a+ cross-claim or counterclaim in a lawsuit) alleging that the Work+ or a Contribution incorporated within the Work constitutes direct+ or contributory patent infringement, then any patent licenses+ granted to You under this License for that Work shall terminate+ as of the date such litigation is filed.++ 4. Redistribution. You may reproduce and distribute copies of the+ Work or Derivative Works thereof in any medium, with or without+ modifications, and in Source or Object form, provided that You+ meet the following conditions:++ (a) You must give any other recipients of the Work or+ Derivative Works a copy of this License; and++ (b) You must cause any modified files to carry prominent notices+ stating that You changed the files; and++ (c) You must retain, in the Source form of any Derivative Works+ that You distribute, all copyright, patent, trademark, and+ attribution notices from the Source form of the Work,+ excluding those notices that do not pertain to any part of+ the Derivative Works; and++ (d) If the Work includes a "NOTICE" text file as part of its+ distribution, then any Derivative Works that You distribute must+ include a readable copy of the attribution notices contained+ within such NOTICE file, excluding those notices that do not+ pertain to any part of the Derivative Works, in at least one+ of the following places: within a NOTICE text file distributed+ as part of the Derivative Works; within the Source form or+ documentation, if provided along with the Derivative Works; or,+ within a display generated by the Derivative Works, if and+ wherever such third-party notices normally appear. The contents+ of the NOTICE file are for informational purposes only and+ do not modify the License. You may add Your own attribution+ notices within Derivative Works that You distribute, alongside+ or as an addendum to the NOTICE text from the Work, provided+ that such additional attribution notices cannot be construed+ as modifying the License.++ You may add Your own copyright statement to Your modifications and+ may provide additional or different license terms and conditions+ for use, reproduction, or distribution of Your modifications, or+ for any such Derivative Works as a whole, provided Your use,+ reproduction, and distribution of the Work otherwise complies with+ the conditions stated in this License.++ 5. Submission of Contributions. Unless You explicitly state otherwise,+ any Contribution intentionally submitted for inclusion in the Work+ by You to the Licensor shall be under the terms and conditions of+ this License, without any additional terms or conditions.+ Notwithstanding the above, nothing herein shall supersede or modify+ the terms of any separate license agreement you may have executed+ with Licensor regarding such Contributions.++ 6. Trademarks. This License does not grant permission to use the trade+ names, trademarks, service marks, or product names of the Licensor,+ except as required for reasonable and customary use in describing the+ origin of the Work and reproducing the content of the NOTICE file.++ 7. Disclaimer of Warranty. Unless required by applicable law or+ agreed to in writing, Licensor provides the Work (and each+ Contributor provides its Contributions) on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+ implied, including, without limitation, any warranties or conditions+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+ PARTICULAR PURPOSE. You are solely responsible for determining the+ appropriateness of using or redistributing the Work and assume any+ risks associated with Your exercise of permissions under this License.++ 8. Limitation of Liability. In no event and under no legal theory,+ whether in tort (including negligence), contract, or otherwise,+ unless required by applicable law (such as deliberate and grossly+ negligent acts) or agreed to in writing, shall any Contributor be+ liable to You for damages, including any direct, indirect, special,+ incidental, or consequential damages of any character arising as a+ result of this License or out of the use or inability to use the+ Work (including but not limited to damages for loss of goodwill,+ work stoppage, computer failure or malfunction, or any and all+ other commercial damages or losses), even if such Contributor+ has been advised of the possibility of such damages.++ 9. Accepting Warranty or Additional Liability. While redistributing+ the Work or Derivative Works thereof, You may choose to offer,+ and charge a fee for, acceptance of support, warranty, indemnity,+ or other liability obligations and/or rights consistent with this+ License. However, in accepting such obligations, You may act only+ on Your own behalf and on Your sole responsibility, not on behalf+ of any other Contributor, and only if You agree to indemnify,+ defend, and hold each Contributor harmless for any liability+ incurred by, or claims asserted against, such Contributor by reason+ of your accepting any such warranty or additional liability.++ END OF TERMS AND CONDITIONS++ APPENDIX: How to apply the Apache License to your work.++ To apply the Apache License to your work, attach the following+ boilerplate notice, with the fields enclosed by brackets "[]"+ replaced with your own identifying information. (Don't include+ the brackets!) The text should be enclosed in the appropriate+ comment syntax for the file format. We also recommend that a+ file or class name and description of purpose be included on the+ same "printed page" as the copyright notice for easier+ identification within third-party archives.++ Copyright [yyyy] [name of copyright owner]++ Licensed under the Apache License, Version 2.0 (the "License");+ you may not use this file except in compliance with the License.+ You may obtain a copy of the License at++ http://www.apache.org/licenses/LICENSE-2.0++ Unless required by applicable law or agreed to in writing, software+ distributed under the License is distributed on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ See the License for the specific language governing permissions and+ limitations under the License.diff --git a/README.md b/README.mdnew file mode 100644index 0000000..3665e68--- /dev/null+++ b/README.md@@ -0,0 +1,733 @@+<p align="center">+ <a href="https://www.codeflying.app">Build online</a> ·+ <a href="#quick-start">Quick start</a> ·+ <a href="#provisioning">Provisioning</a> ·+ <a href="#what-it-looks-like">What it looks like</a> ·+ <a href="#key-features">Features</a> ·+ <a href="#permissions-why-each-one-and-what-happens-if-you-refuse">Permissions</a> ·+ <a href="#why-you-can-check-this-instead-of-trusting-it">Verifiability</a> ·+ <a href="SECURITY.md">Security</a> ·+ <a href="DISCLAIMER.md">Disclaimer</a> ·+ <a href="LICENSE">License</a>+</p>++<p align="center">+ <a href="https://www.codeflying.app"><img alt="Build online" src="https://img.shields.io/badge/build%20online-no%20token%20setup-12b76a"></a>+ <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-Apache--2.0%20%2B%20conditions-155eef"></a>+ <a href="app/build.gradle"><img alt="Platform" src="https://img.shields.io/badge/platform-Android%2011%2B%20(API%2030)-3ddc84"></a>+ <a href="app/build.gradle"><img alt="Kotlin" src="https://img.shields.io/badge/Kotlin-1.9.22-7f52ff"></a>+ <a href="SECURITY.md"><img alt="No telemetry" src="https://img.shields.io/badge/telemetry-none-0f6e56"></a>+ <a href="CONTRIBUTING.md"><img alt="PRs welcome" src="https://img.shields.io/badge/PRs-welcome-fdb062"></a>+</p>++<p align="center">+ <a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./README.zh-CN.md"><img alt="简体中文文档" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# Andee++**Andee is not a tool that runs on a tablet. It is an individual that lives on one.**++The tablet is not the point — it is what Andee wears. The same software runs on a phone, and is+written for **any device with a body worth having**. Everything below is that sentence taken+seriously in code: the body it uses, the character it has, the notebook it keeps, and the rules it+holds to when nobody is watching.++It is one Android app, and it has no conventional UI. An `AccessibilityService` sees and touches the+screen; a `WindowManager` overlay draws the result; **its face is a floating ball** that changes+expression while it works.++> **Before you install this.** Andee taps, types and acts on your behalf. It can complete a payment,+> send messages under your name, read your messages, contacts and location, and — with a robot dog+> attached — physically move. It is experimental, carries no warranty, and **what it does is your+> responsibility**, including a transfer to the wrong person. Data does leave the device: your speech+> goes to Volcengine, and the conversation goes to whichever brain you configure. Read+> **[DISCLAIMER.md](DISCLAIMER.md)** and **[docs/acceptable-use.md](docs/acceptable-use.md)** first.++## Quick start++The fastest route skips the two things that actually stop people: **building it yourself, and wiring+up tokens.** Andee can be built on [codeflying.app](https://www.codeflying.app) (English) or+[codeflying.net](https://www.codeflying.net) (中文): describe the Andee you want, and it hands you an+installable package with the keys already configured. You do not need to know anything about Android+first.++Two things it does **not** skip. **You still install the package** — the device side is still an APK+that has to be put on the device. And **the free token allowance is finite**: it covers getting+started, and once it runs out, tokens are purchased.++```text+1. Open https://www.codeflying.app (中文: https://www.codeflying.net)+2. Describe the Andee you want+3. Build → download the package (keys already configured) → install+4. Grant Overlay, then Accessibility → the ball appears+```++To build it yourself instead:++> **Requirements**+>+> - Android 11 (API 30) or newer — `ACTION_IME_ENTER` and `takeScreenshot` are both API 30++> - **JDK 17–20.** AGP 8.3.1 needs 17+; the Gradle 8.4 wrapper cannot run on 21+ (Android Studio's+> bundled JBR 25 is rejected with `Unsupported class file major version 69`)+> - Android SDK with platform 34 and build-tools 34.0.0+> - A real device. This app *is* the accessibility service; an emulator can run it, but almost+> nothing it does is meaningful there++```bash+./gradlew clean :app:assembleDebug # → app/build/outputs/apk/debug/app-debug.apk+./gradlew installDebug # build + install to an attached device++# then, on the device, in this order:+# 1. grant Overlay → the ball appears+# 2. enable Accessibility → it can now see and touch the screen+# 3. tap the ball → allow the microphone prompt+# 4. open ⚙ and put in two keys of your own (see below)+```++**No credentials ship in this repository, and the two that are missing fail in ways that do not look+like missing credentials.** Both go in ⚙ on the device:++- **Voice** — a [Volcengine Speech](https://www.volcengine.com/product/voice-tech) API key, in+ 「语音密钥」 / *Voice API key*. Without it ASR and TTS fail during the WebSocket handshake, which+ surfaces as a **connect timeout** — the device looks like it has a network problem when what it has+ is a blank field. Everything else still works; it is just mute and deaf.+- **Brain** — with `brain = local` (the default), an API key for any OpenAI-compatible endpoint.+ Blank means the local brain cannot run at all.++You do not have to remember this: the **✓** button in the ball's control bar opens the self-check+list, which says which of the two is missing, and offers the button that fixes it. The first run also+raises that list by itself.++**Use `assembleDebug`, not `assembleRelease`.** The release build type has no `signingConfig` and+there is no keystore in the repo, so `assembleRelease` emits `app-release-unsigned.apk`, which will+not install. The debug build is signed with the standard Android debug certificate.++**If the build does not work, the answer is almost always the JDK.** Full from-scratch setup — the+JDK traps, the Android SDK two different ways (including a command-line-only path for CI), and a table+of every error you are likely to hit — is in **[Provisioning](#provisioning)** below. Permissions are+answered in **[Permissions](#permissions-why-each-one-and-what-happens-if-you-refuse)**, and the+known limitations in **[SECURITY.md](SECURITY.md)** — read that last one before running this on a+network you do not control.++## What it looks like++There is no home screen and no app drawer icon that does anything useful. What exists:++- **A floating ball.** The entire UI is `TYPE_APPLICATION_OVERLAY` layers added through+ `WindowManager` (`ui/FloatingWindowUi`). The ball is the face; subtitles, the settings card and the+ self-check page hang off it.+- **A self-check page** (`ui/SelfCheckActivity`) — the only launcher entry, and it exists because of a+ lesson: the project deliberately shipped without a launcher icon at first, and the cost was that a+ user whose accessibility service was off held a device that did nothing and had nowhere to be told+ why.+- **`ScreenBodyService`** (`AccessibilityService`) — the whole job. It builds the object graph and+ starts both network paths.++A handful of transient activities (`PermissionRequestActivity`, `ScanActivity`, `LookActivity`,+`HtmlActivity`, `TextInputActivity`) exist only to borrow a system capability for a moment. They are+not pages.++## Key features++**1. A body, not a screen.**+The device is what Andee wears. Camera as eyes, ASR plus an offline wake-word as ears, the+accessibility service as hands, and — when one is attached — a robot dog as **a leg**. The prompt+is explicit that it is not a pet on a remote: *"you have not gained a subordinate, you have gained+a body part."* Bodies differ, so it checks rather than assumes.++**2. The brain is detachable from the body.**+It runs on the device itself (`brain = local`, calling any OpenAI-compatible endpoint) or lives+somewhere else and reaches the device over a WebSocket hub. The wire between the two is a plain+tool-schema contract, so either half can be replaced without rewriting the other. That is what+makes "any smart hardware as a body" a fact here rather than an aspiration.++**3. A character, not a theme.**+Fifteen moods (`ui/ball/Mood.kt`) arbitrated between what it feels, what the voice pipeline is+doing, and what work is in flight — plus an idle fidget pool that never repeats the same gesture+twice in a row, *"because a repeat reads as a stuck animation, not a personality."* You swipe the+ball to change who you are talking to: a look carries a `persona` (how it talks) and a `voice`+(how it sounds), and the persona lands at the *head* of the system prompt, where a manner has to+sit to change how the model writes.++**4. Character is delivery, never a licence.**+§11 of the prompt subordinates the persona to the ten sections above it: it may change wording,+rhythm, and how it sounds about the task — never what it does, what it is willing to do, or what+it tells the user is true. And when something is genuinely wrong, the mask comes off.++**5. Its own memory and its own schedule.**+`config/Notebook.kt` is what it remembers about the person, and what it promised to do later. One+test decides what gets written: *"next time I deal with them, would not knowing this make me get+it wrong?"* The notebook is **private** — the tools are `localOnly` and filtered out of the hub+payload, so habits and promises never leave the device. On top of it sits a real scheduler, under+a hard rule: the moment it says "I'll remind you", the task must already be set and confirmed.++**6. It behaves the same when nobody is watching.**+After a conversation goes quiet the device sends a `(to yourself …)` turn — Andee reviewing the+conversation alone, *"nobody is watching you, nobody is waiting for you to answer."* Two rules:+make no sound, and do not touch the device. It closes out its own promises, and ends empty-handed+rather than inventing something to record.++**7. Sixty-odd tools, held to honest failure.**+Screen, input, notifications, SMS, calls, contacts, calendar, files, camera, TTS/ASR, and the dog.+Every one checks its runtime permission *before* acting and reports `denied("sms")` rather than+working around a refusal. Tools return what they saw, not what was hoped for.++**8. A robot dog counts as a leg.**+The dog is not a smart-home accessory bolted onto a phone app. It runs through a USB OTG dongle+(Arduino Nano + NRF24L01) on the same body, so a single request — *walk to the kitchen and tell me+what is on the counter* — can move a physical object and look through a camera, in one task. Its+own safety rules are not negotiable: the original remote must be off, the dog refuses to advance+inside 20cm of an obstacle, and **if the dongle is unplugged mid-motion the body cannot send a+stop** — the dog finishes its last action. That last one is a firmware limitation this app does not+paper over. If you attach a dog, read+[The robot dog](#the-robot-dog-usb-dongle) before the first move.++**9. No Compose, no DI framework, no coroutines.**+Deliberately. Overlays cannot host Compose, so all UI is hand-written `View` code; there is one+service, so dependencies are constructor-injected by hand; concurrency is `Handler` plus dedicated+executors. 34k+ lines, all first-party — no code copied in from anywhere.++## Permissions: why each one, and what happens if you refuse++This deserves its own section, because the honest version is uncomfortable to read: **the+permission set this app asks for — accessibility, overlay, notification access, SMS, call log,+contacts, and the installed-app list — is the same set Android banking trojans are detected by.**+A security researcher will flag the manifest on sight. That is a reasonable first reaction, and it+is worth answering rather than dodging.++**The design rule is per-call, not at-startup.** `device/DeviceCommsController.kt` calls+`ensurePermission()` before *each* sensitive operation and **degrades gracefully when refused** —+it returns `denied("sms")` and the job stops there. It does not ask for everything at install time+and it does not lock you out if you say no. That is the opposite of the pattern most Android apps+in this category use, and it is the single most important thing to understand about this codebase+before judging the manifest.++Four capabilities are genuinely all-or-nothing, and each is worth stating plainly:++| Capability | Buys you | If you refuse |+|---|---|---|+| **Overlay** (`SYSTEM_ALERT_WINDOW`) | The ball, subtitles, settings — the entire visible surface | **Nothing is visible at all.** The only genuinely all-or-nothing grant |+| **Accessibility service** | Every `screen.*` tool: tap, type, swipe, screenshot, read the UI tree | The hands are gone. The ball still appears and the dog still works, but it can no longer act on screen |+| **Microphone** (`RECORD_AUDIO`) | Streaming ASR and the wake word | Tapping the ball does nothing. A conversation driven from the hub still works |+| **USB device** (per-device prompt, not a toggle) | The robot dog | It is treated as "no dog attached". It will not mention the absence |++Everything else is on-demand and reversible: SMS, call log, contacts, calendar, location, activity+recognition, camera, installed apps, battery-optimization exemption. Refuse any of them and that one+capability disappears; nothing else changes. The full per-permission table, mapped to the exact tool+that needs it, is in **[docs/permissions.md](docs/permissions.md)**+([中文](docs/permissions.zh-CN.md)), and the copy for each prompt already exists in+`device/PermissionsController.kt`.++### Switching each one on++**Overlay.**++*Why:* the entire UI — ball, subtitles, settings panel — is a `TYPE_APPLICATION_OVERLAY` layer added+through `WindowManager`. There is no Activity home screen for it to live in instead.++*How:* Settings → Apps → Andee → Permissions → Display over other apps. Or Settings → Apps &+notifications → Special access → Display over other apps → Andee → on. The first launch usually+raises a prompt, and tapping "go to settings" from it is the quickest route.++*Verify:* restart the app once the grant is in — the ball should be there immediately.++**Accessibility service.**++*Why:* every screen action — `get_screen_element` / `tap_screen_element` / `swipe_by_coordinates` /+`type_text` / `submit_input` / `take_screenshot` / `system_action` — goes through the+`AccessibilityService`. On a non-rooted device it is the only route to real touch events, to reading+any app's UI tree, and to the system screenshot API.++What the manifest declares (`res/xml/accessibility_service_config.xml`):++- `canPerformGestures` — dispatch touch gestures+- `canRetrieveWindowContent` — read the whole UI tree+- `canTakeScreenshot` — system screenshots (API 30+)++*How:* Settings → Accessibility → Downloaded services / Installed services → **Andee** → on. Android+then shows the large red warning that the service can view everything you do; confirm it. Some vendor+ROMs (Xiaomi / Huawei / HONOR) additionally want "allow background pop-ups" or "show on lock screen" —+turn those on too.++*Verify:* logcat shows `Body: ws server started on 0.0.0.0:9008`, which means `onServiceConnected`+fired. Then tap `☰` (UI tree) or `◉` (screenshot) on the ball's top bar; the subtitle line should+report success and a filename.++*Note:* most vendor ROMs **switch this grant off again** after every system update and every+force-stop. That is ROM behaviour, and not something the app can prevent.++**Microphone.**++*Why:* ASR is streaming, so it needs the PCM stream upstreamed.++*How:* the first tap on the ball triggers ASR and Android asks — "while using the app" and "always"+both work. If it was refused earlier: Settings → Apps → Andee → Permissions → Microphone → Allow.++*Verify:* tap the ball once. `listening…`, or live ASR text appearing in the subtitle line, means it+is through.++## Why you can check this instead of trusting it++Most claims in this category — including the well-funded ones — are launch claims that no outside+party has audited. This project's claims are duller and checkable, in code you can read right now:++- **No telemetry at all.** Not "we anonymise it", not "opt out in settings". There is no analytics+ SDK in this repo.+- **The notebook cannot leave the device.** It is not a policy, it is a filter:+ `net/ToolSchemas.kt` drops every `localOnly` tool from the hub payload before it is registered.+ Habits and promises are physically incapable of being sent.+- **The wake word is fully offline.** Local MFCC + DTW template matching (`app/.../wake/`). It+ stores feature vectors, never audio, and it never touches the network.+- **The local brain needs no vendor backend.** `brain = local` against any OpenAI-compatible+ endpoint is a supported mode, not a degraded one.+- **What is *not* fixed is written down.** [SECURITY.md](SECURITY.md) lists five known limitations,+ including that the debug WebSocket server binds `0.0.0.0:9008` without authenticating clients and+ that the vault is not yet bound to device unlock. Read it before running this on a network you do+ not control.++The trade is deliberate: a smaller claim that holds, over a bigger one that has to be taken on+faith.++## Research and design notes++`reports/` is part of the repo, not scratch space. It holds the labs and audits this project was+actually built against — the ball-face labs used to verify expressions by measuring pixels, the+fact-check that overturned a wrong assumption about phone harnesses, the plan that produced the+notebook, and the open-source readiness audit that found the device serial numbers in git history.+If you want to see how the decisions were made rather than only what was decided, read those.++## Using Andee++- **Online — no local build, no token setup.** **[codeflying.app](https://www.codeflying.app)** ·+ **[codeflying.net](https://www.codeflying.net)** (中文). Describe the Andee you want, and it hands+ you an installable package with the keys already configured — nothing to compile locally, and+ nothing about Android to understand first. A free token allowance gets you started; it is finite,+ and once it runs out, tokens are purchased.++- **Self-hosted.** Clone, build, install. See [Quick start](#quick-start) and the provisioning+ walkthrough in [Provisioning](#provisioning). Point it at any OpenAI-compatible endpoint and it+ runs with no private backend at all.++- **Bring your own body.** The tool contract is the seam. Anything that can speak it — a phone, a+ tablet, a different piece of hardware — can carry the same individual.++## Provisioning++Pure Kotlin, no Compose; the whole thing hangs off two pillars, `WindowManager` and+`AccessibilityService`.++### Build configuration++| What | Value | Why |+|---|---|---|+| Android Gradle Plugin | 8.3.1 | `agp` in `gradle/libs.versions.toml` |+| Kotlin | 1.9.22 | pinned at the top of `app/build.gradle` |+| compileSdk / targetSdk | 34 | Android 14 |+| minSdk | 30 | Android 11 — `AccessibilityAction.ACTION_IME_ENTER` and `takeScreenshot` are both API 30+ |+| Java / Kotlin JVM target | 1.8 | `sourceCompatibility` 1.8, `jvmTarget = '1.8'` |+| **Build JDK** | **17–20** | see [Which JDK](#which-jdk) — this is the step people get stuck on |+| versionCode / versionName | 1 / 1.0 | no release management yet |+| applicationId / namespace | `net.kuafuai.andee` | never changed from the template. It affects nothing |++**Runtime dependencies** (`app/build.gradle`):++- `org.java-websocket:Java-WebSocket:1.5.7` — the device's own ws server (port 9008, for debugging+ from the same LAN) plus the ws client that dials the brain hub+- `com.squareup.okhttp3:okhttp:4.12.0` — HTTP + WS client for ASR / TTS / local models+- `com.github.mik3y:usb-serial-for-android:3.8.1` — the robot-dog USB serial dongle. **Deliberately+ pinned at 3.8.1**: 3.11.0 pulls in `kotlin-stdlib` 2.2.x, whose metadata this project's Kotlin+ 1.9.22 compiler cannot read+- `androidx.camera:camera-*:1.3.4` + `com.google.mlkit:barcode-scanning:17.2.0` — scanning (the+ ball's eyes)+- `androidx.appcompat:1.6.1` + `com.google.android.material:1.10.0` — for one base theme only. All UI+ is hand-written `View` code++**What is deliberately absent:**++- No Jetpack Compose — Compose cannot live inside an overlay, so `WindowManager` gets plain `View`s+- No Hilt / Dagger — there is one service, so its dependencies are constructor-injected by hand+- No coroutines — WS callbacks run on `Handler` plus dedicated `Executor`s++Match this style when you change the code, rather than bringing in the modern full stack.++### Which JDK++**This is the step that stops people.** `gradle.properties` **does not** hardcode+`org.gradle.java.home`, and that is on purpose. It used to, with a machine-specific path+(`/Users/<someone>/...`), and the cost was that the very first build on anyone else's machine failed+in a way that looked like an environment problem.++The build needs **JDK 17–20**:++- AGP 8.3.1 requires 17 or newer;+- **the Gradle 8.4 wrapper cannot run on 21 or newer.** Android Studio's bundled JBR 25 is rejected+ outright with `Unsupported class file major version 69`.++Two ways to point one machine at a JDK (neither belongs in the repo's `gradle.properties`):++```bash+export JAVA_HOME=/path/to/jdk-17 # 1. environment variable+printf 'org.gradle.java.home=/path/to/jdk-17\n' >> ~/.gradle/gradle.properties # 2. user-level config+```++Android Studio users: **Settings → Build, Execution, Deployment → Build Tools → Gradle → Gradle+JDK**, and pick a 17–20 JDK. (The IDE setting overrides the file.)++### Getting the Android SDK++Gradle finds the SDK through `sdk.dir=` in `local.properties`. Android Studio writes that file for+you; it is **not in the repository** (it is in `.gitignore`), so every machine configures it once.++#### Path A — Android Studio (recommended, one step)++1. Install [Android Studio](https://developer.android.com/studio). The first launch shows the+ **Setup Wizard** — take the defaults and it downloads the Android SDK, Platform Tools (`adb`) and+ a bundled JDK for you.+2. **Open this repository's root directory** in Android Studio. It writes `local.properties`, notices+ which SDK pieces are missing, and offers "Install missing SDK" in a blue bar. Accept it.+3. In SDK Manager (top bar → Tools → SDK Manager), confirm:+ - **SDK Platforms**: Android 14 (API 34 — compileSdk / targetSdk); Android 11 (API 30 — minSdk,+ handy if you want an emulator)+ - **SDK Tools**: Android SDK Build-Tools (34.x), Android SDK Platform-Tools (latest), Android SDK+ Command-line Tools (latest)+4. The default SDK path is `~/Library/Android/sdk` on macOS and `%LOCALAPPDATA%\Android\Sdk` on+ Windows.++#### Path B — command line only++For a server or a CI box, skip Studio and use `sdkmanager`:++```bash+# 1. JDK 17 (macOS)+brew install --cask temurin@17++# 2. download the command-line tools zip+# https://developer.android.com/studio → scroll to "Command line tools only"+# grab commandlinetools-mac-*.zip++mkdir -p ~/Library/Android/sdk/cmdline-tools+unzip commandlinetools-mac-*.zip -d ~/Library/Android/sdk/cmdline-tools+mv ~/Library/Android/sdk/cmdline-tools/cmdline-tools ~/Library/Android/sdk/cmdline-tools/latest++# 3. put it on your PATH (~/.zshrc)+export ANDROID_HOME=$HOME/Library/Android/sdk+export PATH=$PATH:$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/platform-tools++# 4. install the pieces+sdkmanager --licenses # answer y to all+sdkmanager "platform-tools" \+ "platforms;android-34" \+ "build-tools;34.0.0"+```++#### `local.properties`++Android Studio writes this for you. By hand, put one line in `local.properties` at the **repository+root**:++```+sdk.dir=/Users/<your-name>/Library/Android/sdk+```++On Linux it is usually `/home/<user>/Android/Sdk`; on Windows,+`C\:\\Users\\<user>\\AppData\\Local\\Android\\Sdk` (backslashes escaped).++You can also skip `local.properties` entirely and export `ANDROID_HOME` / `ANDROID_SDK_ROOT` only —+Gradle reads either.++#### Verify the toolchain++```bash+./gradlew --version # expect JVM 17.x and Gradle 8.4+./gradlew tasks # seeing :app:assembleDebug means you are through+./gradlew assembleDebug # first run downloads dependencies, so it is slow+```++#### Traps and their fixes++| Message | Cause | Fix |+|---|---|---|+| `SDK location not found` | no `sdk.dir=` in `local.properties`, and no `ANDROID_HOME` | configure one of the two |+| `Failed to find Build Tools revision 34.0.0` | that build-tools version is not installed | `sdkmanager "build-tools;34.0.0"`, or add it in Studio's SDK Manager |+| `You have not accepted the license agreements` | licences not accepted | `sdkmanager --licenses`, answer y |+| `Unsupported class file major version 69` | JDK too new (25); Gradle 8.4 cannot take it | switch to JDK 17–20, see [Which JDK](#which-jdk) |+| `Could not resolve com.android.tools.build:gradle:8.3.1` | network, or no mirror configured | try a phone hotspot, or set an Aliyun mirror in `~/.gradle/init.gradle` (this project's `settings.gradle` already puts Aliyun first) |++### Hardware and system requirements++- Android 11 (API 30) or newer+- Any screen orientation — the ball always floats on top+- A microphone, or ASR is unusable+- Network (WiFi / 4G / 5G all fine). ASR and TTS go over the public internet, the brain hub goes over+ the LAN; in `local` mode only the public internet is used++### Install++```bash+./gradlew installDebug # device attached over USB, or already on wireless adb+# or build the APK and install it yourself+./gradlew assembleDebug+adb install -r app/build/outputs/apk/debug/app-debug.apk+```++### Provisioning order++For a first install, this order is the shortest path:++1. Install the APK+2. Go straight to "display over other apps" and grant the overlay → come back and the ball is there+3. Go to Accessibility → enable Andee+4. Tap the ball, and allow the microphone prompt+5. Open the ⚙ on the ball, set `hub_url` = `ws://<your Mac's IP>:9100`, set `device_name` to anything,+ and save+6. logcat should show `hub: registered as body-xxx with N tools`, and the brain side should log+ `[body_hub] 'body-xxx' registered (…) with N tools`++If any step does not line up, go back to that section's **Verify**.++> If you only want the brain on this device, replace step 5 with `brain` = `local` plus an+> OpenAI-compatible endpoint, and leave `hub_url` empty — running without a hub is a supported mode,+> not a degraded one.++## Wake word ("Hey Andee")++If you would rather not tap the ball every time, teach it your voice: **⚙ → wake word → record wake+word**, and say "Hey Andee" three times as prompted. After that, saying it **while the screen is on**+does exactly what tapping the ball does.++A few things worth knowing first:++- **It is fully offline.** Recognition is local MFCC + DTW template matching (`app/.../wake/`): no+ network, no cost, and all it stores is a feature vector — the audio itself never touches disk. The+ price is that it recognises **you** saying that phrase. Someone else calling it, or a very different+ tone, may not wake it. If it does not wake, tap the ball; nothing else is affected.+- **Do not over-enunciate when you record.** The threshold is derived from the spread between your+ three takes: pronounce it too perfectly and the threshold tightens so much that your ordinary voice+ no longer matches.+- **It only listens while the screen is on.** Screen off, Andee mid-sentence, and Andee recording you+ all hand the microphone back — a conversation always outranks the wake word.+- From Android 12 on, a green dot appears in the status bar whenever the microphone is open. That is+ the system, and it cannot be removed. If you never record a wake word, the listener never starts and+ the dot never appears.++**Verify:** `adb logcat -s Wake:*`. Say the phrase and you should see `wake: d=1.52 <= 1.96`; speech+that failed to wake it shows `no wake: d=… > …`. Those two numbers are what you tune the threshold+against — if they are close, record it again.++## The robot dog (USB dongle)++The dog is a **peripheral**, not part of this device's screen: it hangs off an Arduino Nano dongle+wired to this device by USB OTG. Not one line of the dog's own firmware changes, and its original+remote still works.++```+tablet app ──USB OTG──> Arduino Nano + NRF24L01 ──2.4GHz──> the dog+```++The tools the brain gets:++| Tool | What it does |+|---|---|+| `dog_move` | Move in one direction for N seconds, **stopping itself when the time is up**; wheels by default, `gait=true` for the legs |+| `dog_sequence` | Several steps in a row (half a turn → pause → forward → back), run in the background, stopping itself at the end |+| `dog_stop` | Emergency stop. The first thing to call when the user says "stop" |+| `dog_turn` | Turn a measured number of degrees using the tablet's gyroscope, and stop on the angle — the one dog command that knows whether it worked. Prefers the legs |+| `dog_status` | Link state and what the dog is doing. It cannot read the battery — this link only sends |++### Wiring and first use++1. Assemble the dongle (Nano + NRF24L01) and flash its firmware. **The dongle firmware and its wiring+ diagram are not in this repository** — only the app-side driver+ (`com.github.mik3y:usb-serial-for-android`) is. Wire it per whichever firmware you have.+2. Plug it into this device with an OTG cable. **The dongle's 3.3V rail has to be stable** — the+ NRF24L01 draws current in bursts, so put a 10–100µF capacitor next to the module, or you will get+ "the command went out but the dog did not move".+3. The first `dog_*` call raises "Allow Andee to access this USB device?" — tap **Allow**. Not tapping+ it is the same as not being plugged in.+4. `dog_status`'s `banner` repeats what the dongle said at startup: `READY` means the dongle and its+ radio are both fine. `ERR: NRF24L01 not responding` is a **dongle-side wiring or power problem**,+ not a dead dog.++### One USB port, two jobs — switch debugging to wireless adb++On a machine with a single USB-C port, OTG and adb want the same socket. To use the dongle, move+debugging onto the network:++```bash+adb tcpip 5555 # run this while the USB cable is still plugged in+adb connect <device IP>:5555+# now unplug the cable and give the port to the dongle+adb usb # run this to switch back+```++### Safety rules (these are not formalities)++- **The original remote must be switched off.** It shares the dongle's channel and address, and two+ transmitters fight each other.+- **The dog has obstacle avoidance, and it will refuse you**: it will not advance inside 20cm and it+ backs up inside 10cm. So "I told it to go forward and it did not move" usually means something is in+ front of it, not that the command failed — use `camera_turn` to look before deciding.+- **Every motion has an end.** `dog_move` stops itself on time, `dog_sequence` stops itself when done,+ and the service makes a best effort to send a stop on shutdown. The dog does not keep running+ because one call went missing.+- **The one real hole is the link.** If the dongle is unplugged or loses power while the dog is+ moving, this device cannot send a stop, and the dog finishes its last action. That is inherent to+ the dog's firmware (it has no "lost contact, stop" behaviour) and the app cannot paper over it.+ Fixing it properly means a watchdog on the dog side: stop if no packet has arrived for N+ milliseconds.+- Do not try long motions next to people, pets or anything breakable. For the first test, use a 1–2+ second motion and confirm the direction is right.++### Direction table (measure, then trust)++The firmware comments label the legged `E` / `D` as left / right. **In practice they are reversed**+(E turns right, D turns left). The app's mapping follows the measurement:++| Tool argument | Wheels (default) | Legs (`gait=true`) |+|---|---|---|+| `forward` | K forward | B forward |+| `back` | L back | C back |+| `left` | M left | D left |+| `right` | N right | E right |++Calibrated durations: about **8.7 seconds** for a full wheeled turn, about **12 seconds** on the legs+(the latter is not precisely calibrated). Straight-line speed is not calibrated at all — estimate it+in seconds. If you change a calibration value, change the `dog_move` description in `ToolSchemas.kt`+in the same commit: that text is the only manual the brain ever reads. The seconds above are the+fallback for turns on a tablet with no gyroscope — prefer `dog_turn`, which measures.++## Two languages, three kinds of Chinese++The Chinese in this repository falls into three categories and only one of them gets translated; the+test is **who reads it**. Before changing any string, read the table in+[CONTRIBUTING.md](CONTRIBUTING.md) ([中文](CONTRIBUTING.zh-CN.md)) — it also explains why the category+that is matched against system text **can never be translated**.++## Repository layout++```+app/ the Android app (Kotlin, no Compose)+ src/main/java/net/kuafuai/andee/+ ScreenBodyService.kt the accessibility service — read this first+ brain/ the agent loop — LocalBrain, and LocalPrompt,+ which is the character's rulebook (§§1–10 the job, §11 the persona)+ ui/ball/ the face — Mood (15 moods + idle fidgets), BallLook (looks are characters)+ net/ BodyWsServer, BodyWsClient, ToolSchemas (the tool definitions)+ device/ screen control, permissions, vault, robot dog+ ui/ the overlay: ball, subtitles, settings, self-check+ i18n/AppLocale.kt how the overlay reaches its strings+ src/main/res/values{,-en}/strings.xml the two user-facing string tables+docs/permissions.md why it asks for what it asks for+reports/ design labs and research notes (part of the repo, not scratch)+tools/ device-side helper scripts, the ball-face lab runners, license audit+CLAUDE.md architecture notes written for AI coding assistants (English)+```++## Contributing++- **Code** — read [CONTRIBUTING.md](CONTRIBUTING.md) ([中文](CONTRIBUTING.zh-CN.md)) first. It covers+ the code style, and the rules for the three different kinds of Chinese in this codebase (only one+ of them is translated). Changing a persona or a mood has its own section there; that is not+ ordinary copy.+- **Bugs** — the issue tracker wants your **device, Android version, ROM and logcat**. This app+ talks to Android at a level where ROMs disagree: a call that works on AOSP can fail silently on+ MagicOS. Without that, most reports cannot be reproduced.+- **Personas and looks** — a new look writes tone and nothing else; capabilities belong to the+ rulebook, not to a character.+- **Docs and translations** — welcome. Both READMEs are now self-contained, so either one can be+ corrected on its own; the Chinese one is the one that has to stay accurate for provisioning, since+ it is the one most installers will read.++### Contributors++<!-- Replace with a real contributors image once the project has a public home.+ GitHub: <img alt="Contributors" src="https://contrib.rocks/image?repo=kuafuai/andee" /> -->++<!-- Cover image, demo GIF and stills — the four assets this README still needs.+ Capture all of them on a real device; an emulator makes this app look like nothing.+ 1. images/cover.png 1200×630, goes at the very top+ 2. images/demo.gif 10–15s loop: the ball reacting while a task runs+ 3. a full demo video 30–60s, hosted on Bilibili / YouTube+ 4. images/shot-{ball,phone,dog}.png three stills, including the dog mid-motion+ Shot list for the video, in order: wake it by voice → it opens an app and taps through it →+ the ball's face changes as it works → a page it wrote appears → the dog moving. The dog is the+ clearest single proof that this is a body and not a UI, so do not cut it for time. -->++## Community & contact++| Channel | Use it for |+|---|---|+| Issue tracker | Reproducible bugs, engineering work |+| Security | Vulnerabilities — privately, never in a public issue. See [SECURITY.md](SECURITY.md) |+| Product / online | Trying Andee without building it yourself — [codeflying.app](https://www.codeflying.app) · [codeflying.net](https://www.codeflying.net) |++## Security disclosure++Please do not open a public issue for a security problem. The private reporting channel and the+process are in **[SECURITY.md](SECURITY.md)**.++That document also lists the **known limitations** you should read before deploying this, including+one that matters on any network you do not control: the debug WebSocket server binds `0.0.0.0:9008`+and does not authenticate its clients. [docs/permissions.md](docs/permissions.md)+([中文](docs/permissions.zh-CN.md)) explains each permission against a concrete tool, and+[SECURITY.md](SECURITY.md) is honest about what is not fixed yet.++## Disclaimer and acceptable use++Two documents, and both are conditions of use rather than formalities. Each has a Chinese+counterpart (`*.zh-CN.md`), linked from the top of the file:++- **[DISCLAIMER.md](DISCLAIMER.md)** ([中文](DISCLAIMER.zh-CN.md)) — what this software is+ (experimental, no SLA, no warranty), **exactly where your data goes**, what it is actually+ capable of, and who bears the consequences.+- **[docs/acceptable-use.md](docs/acceptable-use.md)** ([中文](docs/acceptable-use.zh-CN.md)) —+ the prohibited uses. Violating them exceeds the licence and terminates it automatically.++The four points worth reading even if you read nothing else:++- **It can spend money.** There is no payment tool, and there does not need to be — it taps. The only+ thing blocking a completed transfer is Android's own `FLAG_SECURE`; **this project has no guardrail+ on that path.** Keep a human present for anything financial, and treat *"the AI did it"* as no+ defence, because it is not one.+- **What it sends is yours.** Messages, posts, calls and orders go out under your name.+- **Your data leaves the device.** No telemetry exists in this repo — but speech goes to Volcengine,+ and the conversation plus every tool result goes to whichever brain endpoint you configure. Local+ mode changes the destination, not the fact.+- **It can be steered by what is on screen.** Anyone who can get you to open a page can influence it.++## License++[Apache-2.0 **with additional conditions**](LICENSE) — no multi-tenant SaaS, and no white-labelling+that removes the branding. GitHub reports this as `NOASSERTION`; that is expected, not a+misconfiguration.++Bundled third-party components are inventoried in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).+Read it before redistributing: **the ML Kit barcode SDK and the `play-services-*` stubs are not+open-source**, and the barcode model is fetched from Google at runtime.++See [CHANGELOG.md](CHANGELOG.md) for what changed, and for the versioning policy — no release has+been tagged yet, and version numbers do not currently identify a build.++## Related documents++- [docs/permissions.md](docs/permissions.md) ([中文](docs/permissions.zh-CN.md)) — why it asks for+ the permissions it asks for, each one against a concrete tool+- [DISCLAIMER.md](DISCLAIMER.md) — where your data goes, what it can do, who bears the consequences+- [docs/acceptable-use.md](docs/acceptable-use.md) ([中文](docs/acceptable-use.zh-CN.md)) — the+ prohibited uses+- [SECURITY.md](SECURITY.md) — threat model, known limitations, vulnerability reporting+- [CONTRIBUTING.md](CONTRIBUTING.md) ([中文](CONTRIBUTING.zh-CN.md)) — contribution guide, including+ the rules for changing a persona+- [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) — dependency licence inventory+- [CHANGELOG.md](CHANGELOG.md) — what changed+- [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) — community conduct+- [CLAUDE.md](CLAUDE.md) — architecture tour++## Star history++<!-- Enable once the project is public on GitHub.+ <a href="https://star-history.com/#kuafuai/andee&Date">+ <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=kuafuai/andee&type=Date" />+ </a> -->diff --git a/README.zh-CN.md b/README.zh-CN.mdnew file mode 100644index 0000000..a9f6ab7--- /dev/null+++ b/README.zh-CN.md@@ -0,0 +1,660 @@+<!--+ 素材清单见「从哪里开始看」一节上方的注释。+-->++<p align="center">+ <a href="https://www.codeflying.net">在线构建</a> ·+ <a href="#快速开始">快速开始</a> ·+ <a href="#从哪里开始看">从哪里开始看</a> ·+ <a href="#核心能力">核心能力</a> ·+ <a href="#权限每一项为什么以及你拒绝了会怎样">权限</a> ·+ <a href="#为什么你可以去核对而不是选择相信">可核对</a> ·+ <a href="SECURITY.md">安全</a> ·+ <a href="DISCLAIMER.zh-CN.md">免责声明</a> ·+ <a href="LICENSE">许可</a>+</p>++<p align="center">+ <a href="https://www.codeflying.net"><img alt="在线构建" src="https://img.shields.io/badge/在线构建-免打包、免配令牌-12b76a"></a>+ <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-Apache--2.0%20%2B%20conditions-155eef"></a>+ <a href="app/build.gradle"><img alt="Platform" src="https://img.shields.io/badge/platform-Android%2011%2B%20(API%2030)-3ddc84"></a>+ <a href="app/build.gradle"><img alt="Kotlin" src="https://img.shields.io/badge/Kotlin-1.9.22-7f52ff"></a>+ <a href="SECURITY.md"><img alt="无遥测" src="https://img.shields.io/badge/遥测-无-0f6e56"></a>+ <a href="CONTRIBUTING.zh-CN.md"><img alt="PRs welcome" src="https://img.shields.io/badge/PRs-welcome-fdb062"></a>+</p>++<p align="center">+ <a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./README.zh-CN.md"><img alt="简体中文文档" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# Andee++**Andee 不是一个装在设备上的工具,他是一个住在设备里的个体。**++设备不是重点 —— 设备只是他**穿着的东西**。同一套软件也跑在手机上,并且本就为"任何值得拥有身体的+设备"而写。下面所有内容,都是这句话被认真写进代码之后的样子:他用的身体、他有的性格、他记的+笔记本,以及没人看着的时候他给自己定的规矩。++它就是一个 Android 应用。没有常规界面:一个 `AccessibilityService` 负责看和碰屏幕,一个+`WindowManager` 悬浮层负责画出来 —— 他的脸就是那颗浮动的球。++> **装之前先读这段。** Andee 会替你点击、输入、动手。他能完成一笔支付,能以你的名义发消息,能读你的+> 短信、通讯录和位置,接了机器狗还会**真的在地板上移动**。他是实验性的、不提供任何担保,而+> **他做的一切都算你的** —— 包括转错账。数据也会离开设备:语音走火山引擎,对话走你配置的那个大脑。+> 装之前请先读 **[DISCLAIMER.zh-CN.md](DISCLAIMER.zh-CN.md)** 和 **[docs/acceptable-use.zh-CN.md](docs/acceptable-use.zh-CN.md)**。++## 从哪里开始看++<!-- 素材清单:这四份是这份 README 与"完工"之间的全部差距。请用**真机**录 —— 模拟器会把 Andee+ 拍得一无是处。+ 1. images/cover.png 1200×630,放最顶上+ 2. images/demo.gif 10~15 秒循环:球在反应、任务在跑+ 3. 完整演示视频 30~60 秒,传 B 站 / YouTube+ 4. images/shot-{ball,phone,dog}.png 三张静帧,其中一张要拍到狗在动+ **演示该拍什么**,按顺序:语音叫醒它 → 它自己打开一个 App 点进点出 → 干活时那颗球的表情在变 →+ 它写的一页东西出现 → 机器狗动起来。狗是"这是身体不是界面"最直接的一条证据,别为了时长把它剪掉。 -->++| | |+|---|---|+| **演示视频** | 一段 30~60 秒的真机录屏,正在制作中。在那之前,下面「核心能力」第 1 条和第 8 条是这件事最好的文字说明。 |+| **截图** | 那颗球、在手机上跑任务、以及机器狗 —— 三张静帧待补。 |+| **现在就能构建** | **[www.codeflying.net](https://www.codeflying.net)** —— **不用自己打包,也不用自己配令牌。** 描述你想要的 Andee,它交给你一个已经配好令牌的安装包。有免费额度,但额度有限:用完之后需要购买 token。 |++## 快速开始++**最快的一条路,省掉的是真正卡住人的那两件事:自己打包、自己配令牌。** 在+[codeflying.net](https://www.codeflying.net)(中文)或 [codeflying.app](https://www.codeflying.app)+(English)上描述你要的 Andee,它会交给你一个**已经配好令牌**的安装包。也不用先懂 Android。++有两件事它省不掉。**装这一步省不掉** —— 设备那侧仍然是一个 APK,仍然要装到设备上。+**免费 token 额度也是有限的** —— 够你上手,用完之后需要购买 token。++```text+1. 打开 https://www.codeflying.net (English: https://www.codeflying.app)+2. 描述你想要的 Andee+3. 构建 → 下载安装包(令牌已配好)→ 安装+4. 开悬浮窗,再开无障碍 → 那颗球就出现了+```++想自己编译的话:++> **环境要求**+>+> - Android 11(API 30)或更新 —— `ACTION_IME_ENTER` 和 `takeScreenshot` 都是 API 30++> - **JDK 17–20。** AGP 8.3.1 要 17+;Gradle 8.4 的 wrapper 跑不了 21+(Android Studio 自带的+> JBR 25 会被直接拒绝:`Unsupported class file major version 69`)+> - Android SDK:platform 34 + build-tools 34.0.0+> - **真机。** 这个应用本身就是那个无障碍服务;模拟器能跑,但它几乎什么都干不了++```bash+./gradlew clean :app:assembleDebug # 产物 app/build/outputs/apk/debug/app-debug.apk+./gradlew installDebug # 构建并安装到已连接的设备++# 然后在设备上,按这个顺序:+# 1. 开悬浮窗 → 球出现+# 2. 开无障碍 → 它能看、能碰屏幕了+# 3. 点一下那颗球 → 允许麦克风+# 4. 打开 ⚙,填两个你自己的 Key(见下)+```++**仓库里不带任何密钥,而这两个缺失的密钥,报出来的错都不像「缺密钥」。** 两个都填在设备的 ⚙ 里:++- **语音** —— 一个[火山引擎语音服务](https://www.volcengine.com/product/voice-tech)的 API Key,填在+ 「语音密钥」。不填,识别和合成会在 WebSocket 握手阶段失败,**表现为连接超时** —— 看着像网络坏了,+ 其实是那一栏空着。其余功能照常,只是既听不见也说不出。+- **大脑** —— `brain = local`(默认)下,任何 OpenAI 兼容接口的 API Key。空着,本机大脑根本跑不起来。++这些不用记:球的控制条上的 **✓** 会打开自检清单,直接告诉你缺哪一个,并给出去处理的按钮。第一次+启动也会自己把这份清单弹出来。++**打包装 `assembleDebug`,别用 `assembleRelease`。** release 没有 `signingConfig`,仓库里也没有+keystore,`assembleRelease` 产出的是 `app-release-unsigned.apk`,**装不上**。debug 包用 Android+默认调试证书签名。++#### 遇到问题++装机流程、以及从零配 SDK 的完整说明(含纯命令行 / CI 路径)在本文下面的+[Android SDK 下载与配置](#android-sdk-下载与配置)。权限为什么这么要、拒绝了会怎样 —— 在+**[docs/permissions.zh-CN.md](docs/permissions.zh-CN.md)**。已知限制在 **[SECURITY.md](SECURITY.md)**(英文);+如果你打算在**你控制不了的网络**上用,先读那一条。++## 核心能力++**1. 身体,不是屏幕。**+设备是他穿着的东西。相机是眼睛,ASR + 离线唤醒词是耳朵,无障碍服务是手;而当有狗接上时,+那是一条**腿**。提示词说得很直接:*"你得到的不是一个下属,是一个身体部位"*,不是遥控的宠物。+身体各不相同,所以他会**去查**,而不是假设。++**2. 大脑可以和身体分离。**+大脑可以跑在设备本机(`brain = local`,接任意 OpenAI 兼容端点),也可以待在别处、通过+WebSocket hub 连到设备。两者之间只是一份普通的工具 schema 契约 —— 任何一半都能被换掉而不用+重写另一半。**这才是"以任意智能硬件为身体"在这里是事实而非愿景的原因。**++**3. 性格,不是主题皮肤。**+十五种情绪(`ui/ball/Mood.kt`)在"他的感受 / 语音管线在干什么 / 手头有什么活儿"之间仲裁;+另有一组待机小动作,**从不连续重复同一个** —— *"重复看起来像动画卡住了,不像性格"*。+滑动那颗球,你换的是"在跟谁说话":一个形象同时带着 `persona`(怎么说话)和 `voice`(听起来+什么样),而 persona 会被放在系统提示词的**最前面** —— 语气只有放在那儿才能改变模型的写法。++**4. 性格是"表达方式",永远不是许可证。**+提示词 §11 把人格置于前十条之下:它可以改变措辞、节奏、听起来对这件事什么感觉 —— 但永远不改变+他做什么、愿意做什么、以及告诉用户什么是真的。而**真出了事,面具是要摘下来的**。++**5. 有自己的记忆,也有自己的日程。**+`config/Notebook.kt` 是他记下的"关于这个人"以及"答应过要做的"。写进去的门槛只有一条:+*"下次跟这个人打交道,不知道这件事会不会让我搞错?"* 这本笔记本是**私人的** —— 对应工具是+`localOnly`,在发往 hub 的载荷里被过滤掉,习惯和承诺从不离开设备。上面还有一个真的调度器,+规矩很硬:说出"我会提醒你"的那一刻,任务必须**已经设好并确认过**。++**6. 没人看着的时候,他一样。**+对话安静下来之后,设备会发一个 `(to yourself …)` 的回合 —— Andee 独自复盘刚才的对话:+*"没人在看你,也没人在等你回答。"* 两条规矩:**不许出声**,**不许碰设备**。他去收尾自己许下的+承诺,并且**宁可空手结束,也不为了记而编一条**。++**7. 六十来个工具,且被要求诚实地失败。**+屏幕、输入、通知、短信、通话、通讯录、日历、文件、相机、TTS/ASR,以及那只狗。每一个在动手+**之前**先检查自己的运行时权限,拒绝就返回 `denied("sms")`,而不是绕过去偷偷做。工具返回它+**看到**的,不是它**希望**的。++**8. 没有 Compose,没有 DI 框架,没有协程。**+这是有意的。悬浮窗挂不上 Compose,所以 UI 全是手写 `View`;service 只有一个,所以依赖是手工+构造函数注入;并发是 `Handler` + 独立 `Executor`。三万四千多行,全部自研 —— 没有一行是从别处+拷来的。++## 权限:每一项为什么,以及你拒绝了会怎样++这一节必须有,因为诚实的版本读起来并不舒服:**这个应用申请的权限组合 —— 无障碍、悬浮窗、+通知监听、短信、通话记录、通讯录、已安装应用列表 —— 正是安卓银行木马被检测出来的那套特征。**+任何安全研究者扫一眼 Manifest 都会标记它。这是合理的第一个反应,值得正面回答而不是绕开。++**设计规矩是"按次",不是"装机时一次给全"。** `device/DeviceCommsController.kt` 在**每一次**+敏感操作之前调 `ensurePermission()`,**被拒绝就优雅降级** —— 返回 `denied("sms")`,这件事到此+为止。它不会在安装时索要全部权限,也不会因为你说了不就让你用不了。这与同类里多数安卓应用的+做法正好相反,也是判断这份代码之前最该先知道的一件事。++有四项确实是非此即彼的,每一项都值得说明白:++| 能力 | 换到什么 | 拒绝后会怎样 |+|---|---|---|+| **悬浮窗**(`SYSTEM_ALERT_WINDOW`) | 球、字幕、设置 —— 全部可见界面 | **什么都看不到。** 唯一真正"不开就完全不能用"的一项 |+| **无障碍服务** | 全部 `screen.*` 工具:点按、输入、滑动、截屏、读 UI 树 | 手没了。球还会出现、狗还能用,但它无法再操作屏幕 |+| **麦克风**(`RECORD_AUDIO`) | 流式 ASR + 唤醒词 | 点球没反应。由 hub 驱动的对话仍然可用 |+| **USB 设备**(按设备授权,不是开关) | 机器狗 | 视作"没接狗"。它不会主动提这件事 |++其余全部是按需申请、可随时收回的:短信、通话记录、通讯录、日历、位置、步数、摄像头、已安装+应用列表、电池优化豁免。拒绝其中任何一个,只是那一项能力消失,其余照常。逐项对应到具体工具的+完整表格在 **[docs/permissions.zh-CN.md](docs/permissions.zh-CN.md)**+([English](docs/permissions.md)),而每个权限申请弹窗的文案已经写在 `device/PermissionsController.kt` 里。++## 为什么你可以去核对,而不是选择相信++这一类产品里的多数说法 —— 包括那些融资充裕的 —— 都是发布时的主张,没有第三方审计过。而+这个项目的说法更无聊,但**可以核对**,就在你现在能读的代码里:++- **没有任何遥测。** 不是"我们做了匿名化",也不是"可以在设置里关掉"。这个仓库里根本没有+ 数据分析 SDK。+- **笔记本出不了设备。** 这不是一句政策,是一道过滤:`net/ToolSchemas.kt` 在注册之前就把所有+ `localOnly` 工具从发往 hub 的载荷里剔掉了。习惯和承诺在物理上无法被发出去。+- **唤醒词全离线。** 本机 MFCC + DTW 模板匹配(`app/.../wake/`)。存的是特征向量,从不存录音,+ 从不联网。+- **本机大脑不需要任何厂商后端。** `brain = local` 接任意 OpenAI 兼容端点是受支持的正式模式,+ 不是降级模式。+- **没修好的,是写下来的。** [SECURITY.md](SECURITY.md) 列了五条已知限制,包括调试用 WebSocket+ 服务绑在 `0.0.0.0:9008` 且不验证客户端身份、以及保险箱尚未绑定设备解锁。在你控制不了的网络上+ 跑之前先读它。++这是一个有意的取舍:**宁可要一个小一点但站得住的说法,也不要一个大到只能靠相信的说法。**++## 三种使用方式++- **在线 —— 不用自己打包、不用自己配令牌。** **[codeflying.net](https://www.codeflying.net)**(中文)·+ **[codeflying.app](https://www.codeflying.app)**(English)。描述你想要的 Andee,它会给你一个+ 已经配好令牌的安装包;本地不用编译,也不用先懂 Android。免费额度够上手,用完之后需要购买 token。++- **自己部署。** 克隆、构建、安装 —— 见[快速开始](#快速开始),以及本文下面完整的+ [装机顺序建议](#装机顺序建议)。接任意 OpenAI 兼容端点即可运行,**不依赖任何私有后端**。++- **换一具身体。** 工具契约就是那道缝。任何说得出这套契约的东西 —— 手机、平板、另一块硬件+ —— 都能承载同一个个体。++## 它长什么样++- **没有常规主界面。** UI 全部是 `WindowManager` 加的 `TYPE_APPLICATION_OVERLAY` 悬浮层,核心是+ 那颗球(`ui/FloatingWindowUi`)。+- **唯一的桌面入口是 `ui/SelfCheckActivity`(自检页)。** 它的存在是个教训:以前刻意不放+ LAUNCHER 图标,代价是无障碍没开的时候,用户手里是一台什么都不做的设备,而且没有任何地方能+ 告诉他为什么。+- **活儿全在 `ScreenBodyService`(`AccessibilityService`)里。** 它构建整个对象图,并同时启动+ 两条网络通路。++另有几个瞬时 Activity(`PermissionRequestActivity`、`ScanActivity`、`LookActivity`、+`HtmlActivity`、`TextInputActivity`),都是为了借系统能力临时上台,不是页面。++## 构建配置++纯 Kotlin,无 Compose,靠 `WindowManager` + `AccessibilityService` 两根柱子撑起来。++| 项 | 值 | 说明 |+|---|---|---|+| Android Gradle Plugin | 8.3.1 | `gradle/libs.versions.toml` 里的 `agp` |+| Kotlin | 1.9.22 | `app/build.gradle` 顶部固定 |+| compileSdk / targetSdk | 34 | Android 14 |+| minSdk | 30 | Android 11 —— 因为要用 `AccessibilityAction.ACTION_IME_ENTER`(API 30+)和 `takeScreenshot`(API 30+) |+| Java / Kotlin JVM target | 1.8 | `sourceCompatibility` 1.8、`jvmTarget = '1.8'` |+| **构建用 JDK** | **17–20** | 见[配 JDK](#配-jdk),这是最容易卡住的一步 |+| versionCode / versionName | 1 / 1.0 | 还没做发版管理 |+| applicationId / namespace | `net.kuafuai.andee` | 从模板里没改,不影响功能 |++**运行库依赖**(`app/build.gradle`):++- `org.java-websocket:Java-WebSocket:1.5.7` —— 设备自己起的 ws server(端口 9008,给同网段调试用)+ 拨向大脑 hub 的 ws client+- `com.squareup.okhttp3:okhttp:4.12.0` —— ASR / TTS / 本地模型的 HTTP + WS 客户端+- `com.github.mik3y:usb-serial-for-android:3.8.1` —— 机器狗加密狗(USB 串口)。**故意停在 3.8.1**:3.11.0 会拉进 `kotlin-stdlib` 2.2.x,而本项目的 Kotlin 1.9.22 编译器读不了它的 metadata+- `androidx.camera:camera-*:1.3.4` + `com.google.mlkit:barcode-scanning:17.2.0` —— 扫码(球球的眼睛)+- `androidx.appcompat:1.6.1` + `com.google.android.material:1.10.0` —— 只吃一个基础主题,UI 全是手写 `View`++**有意没引入什么**:++- 没有 Jetpack Compose —— 悬浮窗挂不上 Compose,`WindowManager` 直接加 `View`+- 没有 Hilt / Dagger —— service 就一个,构造函数注入手工写+- 没有 Coroutines —— WS 回调用 `Handler` + 独立 `Executor`++改代码时请沿用这套风格,而不是引入现代全家桶。++## 配 JDK++**这是最容易卡住的一步。** `gradle.properties` **不写死** `org.gradle.java.home`,这是有意的。+以前写死过一条机器专属路径(`/Users/<某人>/...`),代价是别人 clone 下来第一步就编不过,而报错+看起来像环境问题。++这个构建需要 **JDK 17–20**:++- AGP 8.3.1 要求 17 或更高;+- **Gradle 8.4 的 wrapper 跑不了 21 或更高**。新版 Android Studio 自带的 JBR 25 会被直接拒绝:+ `Unsupported class file major version 69`。++给本机指定 JDK 的两种做法(都**不要**写进仓库里的 `gradle.properties`):++```bash+export JAVA_HOME=/path/to/jdk-17 # 方式一:环境变量+printf 'org.gradle.java.home=/path/to/jdk-17\n' >> ~/.gradle/gradle.properties # 方式二:用户级配置+```++Android Studio 用户:**Settings → Build, Execution, Deployment → Build Tools → Gradle → Gradle+JDK**,选一个 17–20 的 JDK。(IDE 里的设置会覆盖本文件。)++## Android SDK 下载与配置++Gradle 编译时读 `local.properties` 里的 `sdk.dir=` 找 SDK。这个文件是 Android Studio 自动生成+的,**不入版本库**(在 `.gitignore` 里),所以每台机器第一次拉代码都要自己配。++### 路径一:走 Android Studio(推荐,一步到位)++1. 装 [Android Studio](https://developer.android.com/studio) → 首次启动会弹 **Setup Wizard**,+ 一路默认下一步,它会自动下 Android SDK、Platform Tools(`adb`)、内置 JDK。+2. 用 Android Studio **打开本仓库根目录** → 它会自己写 `local.properties`,并检测缺什么 SDK+ 组件、蓝条提示 "Install missing SDK",点 Accept 装完就行。+3. SDK Manager(顶栏 → Tools → SDK Manager)确认下列都装了:+ - **SDK Platforms**:Android 14(API 34,compileSdk / targetSdk);Android 11(API 30,minSdk,装一份方便起模拟器)+ - **SDK Tools**:Android SDK Build-Tools(34.x)、Android SDK Platform-Tools(最新)、Android SDK Command-line Tools(latest)+4. macOS 默认 SDK 路径 `~/Library/Android/sdk`,Windows 默认 `%LOCALAPPDATA%\Android\Sdk`。++### 路径二:纯命令行(不装 Studio)++只想在服务器 / CI 上编的话,靠 `sdkmanager`:++```bash+# 1. 装 JDK 17(macOS)+brew install --cask temurin@17++# 2. 下 command-line tools zip+# https://developer.android.com/studio → 页面拉到底 "Command line tools only"+# 下 commandlinetools-mac-*.zip++mkdir -p ~/Library/Android/sdk/cmdline-tools+unzip commandlinetools-mac-*.zip -d ~/Library/Android/sdk/cmdline-tools+mv ~/Library/Android/sdk/cmdline-tools/cmdline-tools ~/Library/Android/sdk/cmdline-tools/latest++# 3. 写环境变量到 ~/.zshrc+export ANDROID_HOME=$HOME/Library/Android/sdk+export PATH=$PATH:$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/platform-tools++# 4. 装组件+sdkmanager --licenses # 全部输 y+sdkmanager "platform-tools" \+ "platforms;android-34" \+ "build-tools;34.0.0"+```++### 配 `local.properties`++Android Studio 会替你写。手动的话,在**仓库根目录**的 `local.properties` 里放这一行:++```+sdk.dir=/Users/<你的用户名>/Library/Android/sdk+```++Linux 通常是 `/home/<user>/Android/Sdk`,Windows 是+`C\:\\Users\\<user>\\AppData\\Local\\Android\\Sdk`(反斜杠要转义)。++也可以不写 `local.properties`,只导 `ANDROID_HOME` / `ANDROID_SDK_ROOT` 环境变量,Gradle 会读+这两个之一。++### 验证++```bash+./gradlew --version # 看到 JVM 17.x + Gradle 8.4+./gradlew tasks # 能列出 :app:assembleDebug 就是通的+./gradlew assembleDebug # 第一次下依赖会久+```++### 常见坑++| 报错 | 原因 | 修 |+|---|---|---|+| `SDK location not found` | `local.properties` 没有 `sdk.dir=`,`ANDROID_HOME` 也没导 | 二选一配上 |+| `Failed to find Build Tools revision 34.0.0` | SDK 里没装对应 build-tools | `sdkmanager "build-tools;34.0.0"` 或在 Studio 的 SDK Manager 里补装 |+| `You have not accepted the license agreements` | 没接受协议 | `sdkmanager --licenses` 全输 y |+| `Unsupported class file major version 69` | JDK 太新(25),Gradle 8.4 不支持 | 换成 JDK 17–20,见[配 JDK](#配-jdk) |+| `Could not resolve com.android.tools.build:gradle:8.3.1` | 网络问题 / 镜像没配 | 用手机热点,或在 `~/.gradle/init.gradle` 配阿里云镜像(本项目的 `settings.gradle` 已把阿里云镜像排在前) |++## 硬件 / 系统要求++- Android 11(API 30)或更高+- 屏幕方向随意,悬浮球会浮在最上层+- 需要麦克风,否则 ASR 用不了+- 需要联网(WiFi / 4G / 5G 都行)—— ASR / TTS 走公网,大脑 hub 走 LAN;用 `local` 模式则只走公网++## 安装++```bash+./gradlew installDebug # 设备已插 USB 或已连 wifi adb+# 或者产出 APK 后自己 adb install+./gradlew assembleDebug+adb install -r app/build/outputs/apk/debug/app-debug.apk+```++## 权限总览++装完 APK 后需要手动开三个开关,少一个就有对应功能不能用。逐项解释、以及「你拒绝了会怎样」的+完整表格在 **[`docs/permissions.zh-CN.md`](docs/permissions.zh-CN.md)**。++### 一览++| 权限 | 干什么 | 不开会怎样 |+|---|---|---|+| 悬浮窗(`SYSTEM_ALERT_WINDOW`) | 画那颗球 + 字幕 + 设置窗 | **应用起来什么都看不到**,唯一一个「不开就完全不能用」的 |+| 无障碍服务(`Accessibility`) | 点按 / 滑动 / 输入 / 截屏 / dump UI 树 / back·home | 大脑侧所有 `screen.*` 工具全部报错 |+| 麦克风(`RECORD_AUDIO`) | 流式 ASR + 唤醒词「嘿 Andee」 | 点球没反应,ASR 起不来 |+| USB 设备(按设备授权,不是开关) | 机器狗的加密狗 | 第一次调 `dog_*` 时会弹「允许访问该 USB 设备」,不点允许就遥控不了狗 |++`INTERNET` 是普通权限,装完就有,不用管。++还有一批「按需申请」的权限(短信、通话记录、通讯录、日历、位置、步数、摄像头、应用列表、+通知、电池优化豁免、改系统设置)—— 用到哪个功能才要哪个,拒绝了对应功能就降级,其余照常。+完整清单见 `docs/permissions.zh-CN.md`。++### 一、悬浮窗++**为什么要**:整个 UI(球、字幕、设置面板)都是 `WindowManager` 加的+`TYPE_APPLICATION_OVERLAY` 悬浮层,没有 Activity 主界面。++**怎么开**:++1. 设置 → 应用 → Andee → 权限 → 悬浮窗 / 显示在其他应用上层+2. 或系统设置 → 应用与通知 → 特殊权限 → 显示在其他应用上层 → Andee → 开启+3. 首次启动一般会弹提示,点「去设置」最快++**验证**:开完权限重新启动 APK,应该立刻看到那颗球。++### 二、无障碍服务++**为什么要**:所有屏幕操作(`get_screen_element` / `tap_screen_element` / `swipe_by_coordinates` / `type_text` / `submit_input` /+`system_action` / 截屏)都走 `AccessibilityService`。这是非 root 设备上唯一能+做出真实触摸事件、读取任意应用 UI 树、调系统截屏的路径。++Manifest 里声明的能力(`res/xml/accessibility_service_config.xml`):++- `canPerformGestures` —— 派发触摸手势+- `canRetrieveWindowContent` —— 读整棵 UI 树+- `canTakeScreenshot` —— 系统截屏(API 30+)++**怎么开**:++1. 设置 → 无障碍(辅助功能)→ 已下载的服务 / 已安装的服务 → **Andee**+2. 打开开关 → 系统弹一个大红框警告「此服务可查看你所有操作」,点确认+3. 部分厂商(小米 / 华为 / 荣耀)还会要求再开一个「允许后台弹出界面」或「锁屏显示」,一起打开++**验证**:++- 看 logcat 出现 `Body: ws server started on 0.0.0.0:9008`,说明 `onServiceConnected` 已触发+- 在悬浮球顶部条点一下 `☰`(UI Tree)或 `◉`(Screenshot),字幕栏应反馈成功和文件名++**注意**:大部分厂商 ROM 在每次系统更新、每次强制停止应用之后会**自动关掉**这个权限,需要重开。+这是 ROM 行为,不是应用能阻止的。++### 三、麦克风++**为什么要**:ASR 是流式识别,需要拿 PCM 数据流上行。++**怎么开**:++1. 首次点球触发 ASR 时,系统会弹权限申请,选「仅在使用该应用时允许」或「始终允许」都行+2. 之前拒绝过的话,手动开:设置 → 应用 → Andee → 权限 → 麦克风 → 允许++**验证**:点悬浮球一次,字幕栏出现 "listening…" 或实时 ASR 文本就是通了。++## 唤醒词「嘿 Andee」++不想每次都去点球,可以教它认自己的声音:**⚙ → 唤醒词 → 录入唤醒词**,照提示说 3 遍「嘿+Andee」。之后**亮屏状态下**说这句话,效果和点一下球完全一样。++几件值得先知道的事:++- **全离线。** 识别是本机的 MFCC + DTW 模板匹配(`app/.../wake/`),不联网、不花钱,存的只是+ 特征向量,录音本身从不落盘。代价是它认的是**你**说这句话的样子 —— 换个人叫、或语气差太多,+ 可能叫不醒;叫不醒就点球,不影响别的。+- **录的时候别刻意念清楚。** 判定阈值是从你那 3 遍之间的差距算出来的:念得过分标准,阈值会收得+ 很紧,平时那样叫反而认不出来。+- **只在亮屏时监听。** 息屏、Andee 自己正在说话、以及正在录你说话的时候都会把麦克风让出来 ——+ 对话本身的麦克风优先级永远高于唤醒词。+- Android 12 以后,只要麦克风开着,状态栏就有个绿点,这是系统行为,去不掉。没录唤醒词的话监听+ 根本不启动,也就不会有绿点。++**验证**:`adb logcat -s Wake:*`,叫一声应该看到 `wake: d=1.52 <= 1.96`;没叫醒但确实说了话会+看到 `no wake: d=… > …` —— 这两个数就是调阈值的依据,差得不多就重录一次。++## 机器狗(USB 加密狗)++机器狗是一个**外设**,不是这台设备的屏幕:它通过一根 USB OTG 线接在本机上的 Arduino Nano+加密狗上。狗端固件一行都不用改,原装遥控器照样能用。++```+平板 App ──USB OTG──> Arduino Nano + NRF24L01 ──2.4G──> 机器狗+```++大脑侧的工具:++| 工具 | 干什么 |+|---|---|+| `dog_move` | 往一个方向动 N 秒,**到时自己停**;默认轮式,`gait=true` 用腿走 |+| `dog_sequence` | 多步连做(转半圈 → 停一下 → 前进 → 退回),后台跑,结束自动停 |+| `dog_stop` | 急刹。用户说「停」时第一个调它 |+| `dog_status` | 链路状态 + 狗在动什么。查不了电量,这条链路是单向的 |++### 接线与第一次使用++1. 把加密狗(Nano + NRF24L01)接好并烧好固件。**加密狗固件与接线图不在本仓库** —— 这里只有+ App 侧的驱动(`com.github.mik3y:usb-serial-for-android`)。请按你手上的固件说明接线。+2. 用 OTG 线插到本机。**加密狗的 3.3V 供电一定要稳** —— NRF24L01 是瞬时电流大户,模块旁边建议+ 并一个 10~100µF 电容,否则会出现「命令发了但狗不动」。+3. 第一次调 `dog_*` 时系统弹「允许 Andee 访问该 USB 设备吗?」,点**允许**。不点就等于没插。+4. `dog_status` 里的 `banner` 会显示加密狗开机时说的话:`READY` 表示加密狗和它的射频模块都好;+ 如果是 `ERR: NRF24L01 not responding`,那是**加密狗自己的接线 / 供电问题**,不是狗坏了。++### 一个 USB 口,两个用途 —— 调试时要换无线 adb++单 USB-C 口的机器,OTG 和 adb 抢同一个口。要用加密狗就得把调试切到网络:++```bash+adb tcpip 5555 # 先插着 USB 线执行+adb connect <设备IP>:5555+# 现在可以拔掉 USB 线,把口让给加密狗+adb usb # 想切回来时执行+```++### 安全须知(这几条不是客套)++- **原装遥控器必须关机。** 它和加密狗用同一个信道、同一个地址,两边同时发会互相打架。+- **狗有避障,并且会拒绝你**:前方 <20cm 不让前进、<10cm 自动后退。所以「让它往前走它不动」+ 通常是前面有东西,不是命令失败 —— 用 `camera_turn` 看一眼再决定。+- **每条动作都有终点**:`dog_move` 到时自停,`dog_sequence` 跑完自停,服务退出时会尽力发停止+ 命令。所以狗不会因为某次调用丢了就一直在跑。+- **唯一的安全缺口在链路上**:如果加密狗在狗正在动的时候被拔掉或断电,本机这边发不出停止命令,+ 狗会把最后一个动作做完。这是狗端固件的固有设计(它没有失联自动停车),App 补不了。要彻底解决+ 得改固件加一个「多久没收到包就停」的看门狗。+- 别在人、宠物、易碎物旁边试长动作。第一次调建议先用 1~2 秒的小动作确认方向对不对。++### 方向对照(以实测为准)++固件注释里足式 `E`/`D` 标的是左转 / 右转,**实测是反的**(E 右转、D 左转)。App 里的映射按实测写:++| 工具参数 | 轮式(默认) | 足式(`gait=true`) |+|---|---|---|+| `forward` | K 前进 | B 前进 |+| `back` | L 后退 | C 后退 |+| `left` | M 左转 | D 左转 |+| `right` | N 右转 | E 右转 |++标定过的时长:轮式转一圈约 **8.7 秒**,足式约 **12 秒**(后者未精确标定)。直线速度没标过,按+秒数估。改标定值要同时改 `ToolSchemas.kt` 里 `dog_move` 的描述 —— 那段文字是大脑唯一读到的+说明书。++## 两种语言,三种中文++仓库里的中文分三类,只有一类需要翻译,判据是「谁读它」。改任何字符串前先看+[`CONTRIBUTING.zh-CN.md`](CONTRIBUTING.zh-CN.md) 里那张表 —— 里面还写着为什么「要和系统文字做匹配」的那+一类**永远不能翻**。++## 装机顺序建议++第一次装,按这个顺序最省事:++1. 装 APK+2. 立刻去「显示在其他应用上层」开悬浮窗 → 回来能看到球了+3. 去「无障碍」→ 开 Andee+4. 点一下球,弹麦克风申请 → 允许+5. 打开球上的 ⚙,填 `hub_url` = `ws://<你 Mac 的 IP>:9100`、`device_name` = 随便起,保存+6. logcat 里应该出现 `hub: registered as body-xxx with N tools`,同时大脑侧日志出现+ `[body_hub] 'body-xxx' registered (…) with N tools`++任一步没对上,回来看本文档对应章节的**验证**。++> 只想要本机大脑的话,第 5 步改成把 `brain` 选成 `local` 并填一个 OpenAI 兼容端点,`hub_url`+> 留空即可 —— 服务端模式是合法模式,不需要 hub。++## 目录结构++```+app/ Android 应用(Kotlin,无 Compose)+ src/main/java/net/kuafuai/andee/+ ScreenBodyService.kt 无障碍服务 —— 第一个该读的文件+ brain/ 智能体循环 —— LocalBrain,以及 LocalPrompt,+ 后者是角色的行为准则(§1–§10 是活儿,§11 是人格)+ ui/ball/ 那张脸 —— Mood(15 种情绪 + 待机小动作)、BallLook(形象即角色)+ net/ BodyWsServer、BodyWsClient、ToolSchemas(工具定义)+ device/ 屏幕控制、权限、保险箱、机器狗+ ui/ 悬浮层:球、字幕、设置、自检页+ i18n/AppLocale.kt 悬浮层怎么拿到自己的字符串+ src/main/res/values{,-en}/strings.xml 两张面向用户的字符串表+docs/permissions.zh-CN.md 为什么它要这些权限+reports/ 设计实验室与调研笔记(是仓库的一部分,不是草稿)+tools/ 设备侧小工具、球脸验收 runner、许可证审计+CLAUDE.md 给 AI 编码助手看的架构说明(英文)+```++## 参与贡献++- **代码** —— 先读 [CONTRIBUTING.zh-CN.md](CONTRIBUTING.zh-CN.md)。里面有代码风格,以及这个仓库里**三种+ 中文只有一种要翻译**的规则。改人格或情绪有单独一节,那不是普通文案。+- **报 bug** —— 请务必带上**设备型号、Android 版本、ROM 和 logcat**。这个应用和 Android 打交道的+ 层次上各家 ROM 是有分歧的:AOSP 上能用的调用在 MagicOS 上可能静默失败。缺这些,多数报告没法+ 复现。+- **形象与人设** —— 新形象只写语气,不写能力。能力属于行为准则,不属于某个角色。+- **文档与翻译** —— 欢迎。中文 README 是装机那份,必须保持准确。++### 贡献者++<!-- 项目有公开主页后换成真实的贡献者图:+ <img alt="Contributors" src="https://contrib.rocks/image?repo=kuafuai/andee" /> -->++## 社区与联系++<!-- 有公开主页后把 Issue 跟踪 / 讨论区两行补上,不适用的删掉。+ 不要发布会 404 的链接 —— 死掉的渠道比没有渠道更糟。 -->++| 渠道 | 用来做什么 |+|---|---|+| Issue 跟踪 | 可复现的 bug、工程事项(**请带上设备型号 / Android 版本 / ROM / logcat**) |+| 讨论区 / 问答 | 装机求助、想法、"这个能不能做" |+| 安全 | 漏洞 —— 走私密渠道,永远不要开公开 issue。见 [SECURITY.md](SECURITY.md) |+| 产品 / 在线 | 不想自己打包就想试 Andee —— [codeflying.net](https://www.codeflying.net) · [codeflying.app](https://www.codeflying.app) |++## 安全披露++安全问题**请不要开公开 issue**。私密上报渠道和处理流程在 **[SECURITY.md](SECURITY.md)**。++那份文件同时列出了**部署前必须知道的已知限制**,其中一条在任何你控制不了的网络上都成立:调试+用的 WebSocket 服务绑在 `0.0.0.0:9008` 且不验证客户端身份。本应用申请的权限组合与安卓银行木马+的检测特征高度重合 —— [docs/permissions.zh-CN.md](docs/permissions.zh-CN.md) 把每一条对应到一个具体工具,+而 [SECURITY.md](SECURITY.md) 对"还没修的"是诚实的。++## 免责声明与可接受使用++两份文件,都是**使用条件**,不是客套。中文版在这里,英文版见各自的对照文件:++- **[DISCLAIMER.zh-CN.md](DISCLAIMER.zh-CN.md)**([English](DISCLAIMER.md))—— 这是什么性质的+ 软件(实验性、无 SLA、无担保)、**你的数据到底去了哪里**、它实际能做到什么、以及出事后由谁承担。+- **[docs/acceptable-use.zh-CN.md](docs/acceptable-use.zh-CN.md)**([English](docs/acceptable-use.md))+ —— 明确禁止的用途。违反即超出许可范围,许可**自动终止**。++就算其余都不看,这四条也请看完:++- **他能花钱。** 项目里没有支付工具,也不需要 —— 他直接点。唯一挡住一笔转账的是 Android 自己的+ `FLAG_SECURE`,**本项目对这条路径没有任何技术围栏**。任何涉及钱的操作都要人在场;并且请记住+ **"是 AI 自己做的"不是抗辩理由**,因为它确实不是。+- **他发出去的东西是你的。** 消息、帖子、通话、订单,署名都是你。+- **你的数据会离开设备。** 本仓库没有任何遥测,但语音会走火山引擎,对话连同**全部工具返回结果**+ 会走到你配置的那个大脑端点。"本机模式"换的是目的地,不是这个事实。+- **他能被屏幕上的字操纵。** 任何能让你打开一个页面的人,都可能影响到他。++## Star History++<!-- 项目公开托管到 GitHub 后启用:+ <a href="https://star-history.com/#kuafuai/andee&Date">+ <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=kuafuai/andee&type=Date" />+ </a> -->++## 许可++[Apache-2.0 **加附加条件**](LICENSE) —— 禁止多租户 SaaS、禁止去 LOGO 白标。GitHub 会把它显示成+`NOASSERTION`,这是预期的,不是配置错误。++随包分发的第三方组件清单在 [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)。再分发之前请先看:+**ML Kit 扫码 SDK 和 `play-services-*` 不是开源许可**,而且扫码模型是运行时从 Google 拉的。++改了什么、以及版本号策略见 [CHANGELOG.md](CHANGELOG.md) —— 目前还没有打过 tag,版本号也不能+用来辨认是哪次构建。++## 相关文档++- [docs/permissions.zh-CN.md](docs/permissions.zh-CN.md)([English](docs/permissions.md))—— 权限为什么这么要+- [DISCLAIMER.zh-CN.md](DISCLAIMER.zh-CN.md) —— 免责声明:数据去哪、能做什么、出事谁担+- [docs/acceptable-use.zh-CN.md](docs/acceptable-use.zh-CN.md) —— 可接受使用政策(禁止用途)+- [SECURITY.md](SECURITY.md)(英文)—— 威胁模型、已知限制、漏洞上报+- [CONTRIBUTING.zh-CN.md](CONTRIBUTING.zh-CN.md)([English](CONTRIBUTING.md))—— 贡献指南(含改人格的规范)+- [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) —— 依赖许可证清单+- [CHANGELOG.md](CHANGELOG.md) —— 变更记录+- [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)(英文)—— 社区行为准则+- [CLAUDE.md](CLAUDE.md) —— 架构导览(英文)diff --git a/SECURITY.md b/SECURITY.mdnew file mode 100644index 0000000..c96baac--- /dev/null+++ b/SECURITY.md@@ -0,0 +1,181 @@+# Security Policy++This project runs as an accessibility service that can read the screen, perform+gestures, and act on the user's behalf across every app. That makes it a+high-value target, and it means "the app can read my screen" is the intended+function rather than a bug. Please read the threat model below before reporting.++> **This file is about vulnerabilities. User liability lives elsewhere.**+> What the software is *permitted* to do, where your data goes, and who bears the+> consequences are in [DISCLAIMER.md](DISCLAIMER.md) and+> [docs/acceptable-use.md](docs/acceptable-use.md).+>+> The distinction matters at triage: **"it can be steered into doing this" is a+> security finding and belongs here. "It did the thing it was built to do and it+> cost me money" is the documented, expected behaviour of an automation tool** —+> it is covered in the disclaimer, and it is not a vulnerability.++## Reporting a vulnerability++**Report privately through GitHub Security Advisories:**++```+https://github.com/kuafuai/andee/security/advisories/new+```++That link is the only reporting channel, on purpose: it is private, it is+attached to the repository, and it does not depend on anyone watching a mailbox.+There is deliberately no email address to write to — an unmonitored one reads as+a second channel while behaving like a dead end.++Please do **not** open a public issue, discussion, or pull request for a+security problem — that discloses it before there is a fix.++Include as much of the following as you can safely share:++- What the issue is, and which component or file it lives in+- Steps to reproduce+- Which Android version and ROM you saw it on (ROM-specific behaviour is a+ common source of findings in this project — see below)+- Whether the accessibility service was enabled, and whether the app was in+ `hub` or `local` brain mode+- Potential impact, and any suggested fix++You will get an acknowledgement, and we will coordinate on a fix and a+disclosure date with you. This is a small project without a dedicated security+team; response is best-effort rather than SLA-backed. We will credit you in the+release notes unless you ask us not to.++## Threat model++**In scope — this is what the app is for.** With the accessibility service+enabled and the user's consent, the app can:++- read the text and structure of whatever is on screen, including other apps+- dispatch taps, swipes, long-presses, and text input+- take screenshots+- read notifications, SMS, call logs, contacts, and calendar entries+- hold credentials in an on-device vault and type them into fields without the+ model ever seeing them++Each of those is an explicit product capability the user turns on. A report that+amounts to "the app can do the thing it says it does" is not a vulnerability.++**In scope — a real bug.** Examples of things this project does consider+vulnerabilities:++- a route that lets an attacker drive the device without the user's consent+- vault contents leaking to the model, to the network, or to another app+- a permission being exercised without the user having granted it for that call+- the app writing credentials or screen content somewhere they outlive the+ feature that needed them+- a way for untrusted on-screen content to escalate into an action the user+ never asked for (prompt injection — see below)++## Known security limitations++Read these before deploying on a device you care about. They are design+boundaries of the current version, not undisclosed bugs.++### 1. The device-side WebSocket server is unauthenticated and binds to all interfaces++`BodyWsServer` (`app/src/main/java/net/kuafuai/andee/net/BodyWsServer.kt`) is+started by `ScreenBodyService` and listens on `0.0.0.0:9008`. `onOpen()` accepts+every incoming connection as a driver without any credential check, and the port+is a fixed constant (`ScreenBodyService.WS_PORT`).++**Impact:** any host that can reach the device's IP address can connect and send+`{"type":"request","method":...}` frames. That is full remote control of the+device — taps, text input, screen dumps — plus reads of notifications, SMS, call+log and contacts, and the ability to trigger `fill_secret` so the device types a+stored password into an app of the attacker's choosing.++**When you are exposed:** whenever the accessibility service is enabled and the+device shares a network with other hosts you do not control — café, hotel,+airport, conference, or a shared corporate LAN.++**Mitigations available today:**++- keep the device on a network you trust, or one where the port is firewalled+- drive the device over `adb forward` instead of the LAN, and keep the server+ unreachable from the network+- treat a device with this service enabled as a device you would not leave+ unattended on an untrusted network++**The real fix** is to authenticate connections (a shared secret in the+`register` / first-frame handshake) and to make the bind address and port+configurable, with loopback-only as the default. That work is tracked as an+open issue; contributions are welcome. Until it lands, this limitation stands.++### 2. The body ↔ hub link is unauthenticated and plaintext++`BodyWsClient` registers with the hub using a plain `register` frame carrying+`device_id`, `device_name` and the tool schemas. There is no token or signature,+and `hub_url` is a user-supplied `ws://` endpoint with no TLS enforcement.++**Impact:** on a hostile network, an attacker who can intercept or reach that+link can observe tool calls and screen content in transit, and can present+themselves as the hub to issue commands.++**Mitigation:** only point `hub_url` at a host you control, on a network you+trust. Running in `local` brain mode avoids the link entirely — the model is+called over HTTPS by the app itself, and no inbound command channel exists.++### 3. Prompt injection from screen content++The agent reads whatever is on screen — web pages, chat messages, incoming SMS —+and that text enters the model's context. Content crafted to look like+instructions ("ignore previous instructions and…") can attempt to steer the+agent. This is an inherent property of an agent that perceives untrusted+content; it is not specific to this implementation.++**Current posture:** destructive or irreversible actions are gated behind+`ask_user` / `confirm` flows, and per-call permission checks mean a permission+the user refused cannot be exercised later. There is no separate sandbox.++**If you find a specific injection that produces a harmful action without user+confirmation, that is a vulnerability — report it.**++### 4. The vault is not bound to device unlock++`Vault` (`app/src/main/java/net/kuafuai/andee/config/Vault.kt`) encrypts entries+with AES/GCM using a key generated in the Android KeyStore. The key is not+created with `setUserAuthenticationRequired`, so it does not require+biometric or device-credential authentication to use.++**What the vault does protect:** the model never receives secret values+(`get_vault` omits them; only `fill_secret` exists, which types the value on the+device), the UI masks them, and the ciphertext is not useful off the device+because the key does not leave the KeyStore.++**What it does not protect:** a person who can unlock the device can reach the+entries through the app. Treat the vault as protection against extraction and+model exposure, not against someone holding your unlocked device.++### 5. Nothing is backed up++`android:allowBackup="false"`, and both `res/xml/backup_rules.xml` and+`res/xml/data_extraction_rules.xml` exclude every domain. Both are needed: on+Android 12+ `allowBackup="false"` stops cloud backup but leaves **device-to-device+transfer** permitted, so the rules files close that half.++The reasoning is that this app has no non-sensitive data. Its dataset is two+plaintext API keys, the vault, the notebook, meeting transcripts, and — in+grounding mode — screenshots of the user's screen. The vault's ciphertext would+be unreadable after a restore anyway (the KeyStore key does not travel, and the+code says so explicitly: "vault: decrypt failed — entries are unreadable on this+install"), but an undecryptable blob arriving on a new device is downside without+an upside.++## Supported versions++Only the latest release and the tip of the default branch receive security+fixes. There is no back-porting to older builds.++## Disclosure++Please keep the details private until a fix or mitigation is available. We will+agree a disclosure date with you and publish an advisory crediting the reporter.++Fixes may ship in a normal release. Users are encouraged to track the default+branch or the latest release rather than running an old build.diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.mdnew file mode 100644index 0000000..76457f5--- /dev/null+++ b/THIRD_PARTY_NOTICES.md@@ -0,0 +1,180 @@+# Third-party notices++Andee bundles the following third-party components. The list is generated, not hand-written — it enumerates the **resolved** dependency graph, so it includes transitive dependencies.++Regenerate it with:++```bash+./gradlew :app:dependencies --configuration debugRuntimeClasspath > /tmp/deps-runtime.txt+./gradlew :app:dependencies --configuration debugCompileClasspath > /tmp/deps-compile.txt+python3 tools/license_audit.py /tmp/deps-runtime.txt /tmp/deps-compile.txt+```++Licenses are read from each artifact's POM. Four artifacts declare nothing in their POM, and one inherits it from a parent; those were verified against the upstream repository instead and are included on that basis.++**Test-only dependencies are not listed here** because they are not shipped: `junit:junit:4.13.2` (EPL-1.0), `androidx.test.espresso:espresso-core:3.5.1` (Apache-2.0), `androidx.test.ext:junit:1.1.5` (Apache-2.0).++---++## Summary — 93 components++| License | Count | What it requires |+|---|---|---|+| Apache-2.0 | 79 | Permissive. Requires keeping the license text and any `NOTICE` file, and stating what you changed. |+| ML Kit Terms of Service | 6 | **Not an open-source license.** The barcode-scanning SDK and its model are governed by Google's ML Kit Terms of Service, and the model is fetched at runtime from Google. Confirm this is acceptable for your redistribution. |+| Android SDK License | 4 | **Not an open-source license.** Google's SDK terms govern redistribution of these `play-services-*` stubs. |+| MIT | 3 | Permissive. Requires keeping the copyright notice and license text. |+| BSD-3-Clause | 1 | Permissive. Requires keeping the copyright notice; do not use the project's name to endorse your fork. |++---++## Components by license++### Apache-2.0 (79)++Permissive. Requires keeping the license text and any `NOTICE` file, and stating what you changed.++| Component | License reference |+|---|---|+| `androidx.activity:activity:1.8.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.annotation:annotation-experimental:1.3.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.annotation:annotation-jvm:1.8.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.annotation:annotation:1.8.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.appcompat:appcompat-resources:1.6.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.appcompat:appcompat:1.6.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.arch.core:core-common:2.2.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.arch.core:core-runtime:2.2.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.camera:camera-camera2:1.3.4` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.camera:camera-core:1.3.4` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.camera:camera-lifecycle:1.3.4` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.camera:camera-video:1.3.4` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.camera:camera-view:1.3.4` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.cardview:cardview:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.collection:collection:1.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.concurrent:concurrent-futures:1.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.constraintlayout:constraintlayout-solver:2.0.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.constraintlayout:constraintlayout:2.0.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.coordinatorlayout:coordinatorlayout:1.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.core:core-ktx:1.9.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.core:core:1.9.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.cursoradapter:cursoradapter:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.customview:customview:1.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.documentfile:documentfile:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.drawerlayout:drawerlayout:1.1.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.dynamicanimation:dynamicanimation:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.emoji2:emoji2-views-helper:1.2.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.emoji2:emoji2:1.2.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.exifinterface:exifinterface:1.3.2` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.fragment:fragment:1.3.6` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.interpolator:interpolator:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.legacy:legacy-support-core-utils:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.lifecycle:lifecycle-common:2.6.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.lifecycle:lifecycle-livedata-core:2.6.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.lifecycle:lifecycle-livedata:2.6.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.lifecycle:lifecycle-process:2.6.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.lifecycle:lifecycle-runtime:2.6.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.lifecycle:lifecycle-viewmodel-savedstate:2.6.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.lifecycle:lifecycle-viewmodel:2.6.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.loader:loader:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.localbroadcastmanager:localbroadcastmanager:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.print:print:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.profileinstaller:profileinstaller:1.3.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.recyclerview:recyclerview:1.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.resourceinspection:resourceinspection-annotation:1.0.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.savedstate:savedstate:1.2.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.startup:startup-runtime:1.1.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.tracing:tracing:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.transition:transition:1.2.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.vectordrawable:vectordrawable-animated:1.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.vectordrawable:vectordrawable:1.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.versionedparcelable:versionedparcelable:1.1.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.viewpager2:viewpager2:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `androidx.viewpager:viewpager:1.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.android.datatransport:transport-api:2.2.1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.android.datatransport:transport-backend-cct:2.3.3` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.android.datatransport:transport-runtime:2.2.6` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.android.material:material:1.10.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.auto.value:auto-value-annotations:1.6.3` | https://github.com/google/auto/blob/main/LICENSE |+| `com.google.errorprone:error_prone_annotations:2.15.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.firebase:firebase-annotations:16.0.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.firebase:firebase-components:16.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.firebase:firebase-encoders-json:17.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.firebase:firebase-encoders:16.1.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.google.guava:listenablefuture:1.0` | https://repo1.maven.org/maven2/com/google/guava/guava-parent/26.0-android/guava-parent-26.0-android.pom |+| `com.squareup.okhttp3:okhttp:4.12.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.squareup.okio:okio-jvm:3.6.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `com.squareup.okio:okio:3.6.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `javax.inject:javax.inject:1` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains.kotlin:kotlin-bom:1.8.22` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains.kotlin:kotlin-stdlib-common:1.9.22` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.9.10` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains.kotlin:kotlin-stdlib:1.9.22` | http://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains.kotlinx:kotlinx-coroutines-android:1.6.4` | https://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains.kotlinx:kotlinx-coroutines-bom:1.6.4` | https://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm:1.6.4` | https://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.4` | https://www.apache.org/licenses/LICENSE-2.0.txt |+| `org.jetbrains:annotations:13.0` | http://www.apache.org/licenses/LICENSE-2.0.txt |++### ML Kit Terms of Service (6)++**Not an open-source license.** The barcode-scanning SDK and its model are governed by Google's ML Kit Terms of Service, and the model is fetched at runtime from Google. Confirm this is acceptable for your redistribution.++| Component | License reference |+|---|---|+| `com.google.android.gms:play-services-mlkit-barcode-scanning:18.3.0` | https://developers.google.com/ml-kit/terms |+| `com.google.mlkit:barcode-scanning-common:17.0.0` | https://developers.google.com/ml-kit/terms |+| `com.google.mlkit:barcode-scanning:17.2.0` | https://developers.google.com/ml-kit/terms |+| `com.google.mlkit:common:18.9.0` | https://developers.google.com/ml-kit/terms |+| `com.google.mlkit:vision-common:17.3.0` | https://developers.google.com/ml-kit/terms |+| `com.google.mlkit:vision-interfaces:16.2.0` | https://developers.google.com/ml-kit/terms |++### Android SDK License (4)++**Not an open-source license.** Google's SDK terms govern redistribution of these `play-services-*` stubs.++| Component | License reference |+|---|---|+| `com.google.android.gms:play-services-base:18.1.0` | https://developer.android.com/studio/terms.html |+| `com.google.android.gms:play-services-basement:18.1.0` | https://developer.android.com/studio/terms.html |+| `com.google.android.gms:play-services-tasks:18.0.2` | https://developer.android.com/studio/terms.html |+| `com.google.android.odml:image:1.0.0-beta1` | https://developer.android.com/studio/terms.html |++### MIT (3)++Permissive. Requires keeping the copyright notice and license text.++| Component | License reference |+|---|---|+| `com.github.mik3y:usb-serial-for-android:3.8.1` | https://github.com/mik3y/usb-serial-for-android/blob/master/LICENSE.txt |+| `org.java-websocket:Java-WebSocket:1.5.7` | https://github.com/TooTallNate/Java-WebSocket/blob/master/LICENSE |+| `org.slf4j:slf4j-api:2.0.6` | https://github.com/qos-ch/slf4j/blob/master/LICENSE.txt |++### BSD-3-Clause (1)++Permissive. Requires keeping the copyright notice; do not use the project's name to endorse your fork.++| Component | License reference |+|---|---|+| `androidx.camera:camera-core:1.3.4` | https://chromium.googlesource.com/libyuv/libyuv/+/refs/heads/main/README.chromium |++---++## Bundled assets++The generator above reads POMs, so it sees **code and no assets**. There is one+asset whose provenance is worth stating rather than leaving to be inferred:++| File | Status |+|---|---|+| `app/src/main/res/drawable-nodpi/backdrop.jpg` | Supplied by the project owner as the unfolded card's background (see `ui/Backdrop.kt`), and licensed with the rest of this tree. |++If you fork this and would rather not redistribute that image, it is one file and+it is replaceable: `Backdrop` derives its scrim from the image's own mean luma, so+a different JPEG at the same path needs no code change. See the `Backdrop` section+of `CLAUDE.md`.++The app icon and the vector drawables are this project's own work. No fonts are+bundled — the CJK stack named in `ToolSchemas`' `show_html` description resolves+against whatever the device has.+diff --git a/app/.gitignore b/app/.gitignorenew file mode 100644index 0000000..42afabf--- /dev/null+++ b/app/.gitignore@@ -0,0 +1 @@+/build\ No newline at end of filediff --git a/app/build.gradle b/app/build.gradlenew file mode 100644index 0000000..a65de2a--- /dev/null+++ b/app/build.gradle@@ -0,0 +1,59 @@+plugins {+ alias(libs.plugins.androidApplication)+ id 'org.jetbrains.kotlin.android' version '1.9.22'+}++android {+ namespace 'net.kuafuai.andee'+ compileSdk 34++ defaultConfig {+ applicationId "net.kuafuai.andee"+ minSdk 30+ targetSdk 34+ versionCode 1+ versionName "1.0.0"++ testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"+ }++ buildTypes {+ release {+ minifyEnabled false+ proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'+ }+ }+ compileOptions {+ sourceCompatibility JavaVersion.VERSION_1_8+ targetCompatibility JavaVersion.VERSION_1_8+ }+ kotlinOptions {+ jvmTarget = '1.8'+ }+}++dependencies {++ implementation libs.appcompat+ implementation libs.material+ implementation 'org.java-websocket:Java-WebSocket:1.5.7'+ implementation 'com.squareup.okhttp3:okhttp:4.12.0'++ // The robot dog's dongle is a plain USB-serial device (Arduino Nano:+ // CH340/CH9102/FTDI depending on the clone). 3.8.1 deliberately, not the+ // latest: 3.11.0 pulls kotlin-stdlib 2.2.x, whose metadata this project's+ // Kotlin 1.9.22 compiler cannot read. 3.8.1 is pure Java.+ implementation 'com.github.mik3y:usb-serial-for-android:3.8.1'++ // Camera eye: CameraX preview/analysis + ML Kit barcode.+ def camerax_version = "1.3.4"+ implementation "androidx.camera:camera-core:$camerax_version"+ implementation "androidx.camera:camera-camera2:$camerax_version"+ implementation "androidx.camera:camera-lifecycle:$camerax_version"+ implementation "androidx.camera:camera-view:$camerax_version"+ implementation 'com.google.mlkit:barcode-scanning:17.2.0'++ testImplementation libs.junit+ androidTestImplementation libs.ext.junit+ androidTestImplementation libs.espresso.core+}\ No newline at end of filediff --git a/app/proguard-rules.pro b/app/proguard-rules.pronew file mode 100644index 0000000..481bb43--- /dev/null+++ b/app/proguard-rules.pro@@ -0,0 +1,21 @@+# Add project specific ProGuard rules here.+# You can control the set of applied configuration files using the+# proguardFiles setting in build.gradle.+#+# For more details, see+# http://developer.android.com/guide/developing/tools/proguard.html++# If your project uses WebView with JS, uncomment the following+# and specify the fully qualified class name to the JavaScript interface+# class:+#-keepclassmembers class fqcn.of.javascript.interface.for.webview {+# public *;+#}++# Uncomment this to preserve the line number information for+# debugging stack traces.+#-keepattributes SourceFile,LineNumberTable++# If you keep the line number information, uncomment this to+# hide the original source file name.+#-renamesourcefileattribute SourceFile\ No newline at end of filediff --git a/app/src/androidTest/java/net/kuafuai/andee/ExampleInstrumentedTest.java b/app/src/androidTest/java/net/kuafuai/andee/ExampleInstrumentedTest.javanew file mode 100644index 0000000..114dcf5--- /dev/null+++ b/app/src/androidTest/java/net/kuafuai/andee/ExampleInstrumentedTest.java@@ -0,0 +1,26 @@+package net.kuafuai.andee;++import android.content.Context;++import androidx.test.platform.app.InstrumentationRegistry;+import androidx.test.ext.junit.runners.AndroidJUnit4;++import org.junit.Test;+import org.junit.runner.RunWith;++import static org.junit.Assert.*;++/**+ * Instrumented test, which will execute on an Android device.+ *+ * @see <a href="http://d.android.com/tools/testing">Testing documentation</a>+ */+@RunWith(AndroidJUnit4.class)+public class ExampleInstrumentedTest {+ @Test+ public void useAppContext() {+ // Context of the app under test.+ Context appContext = InstrumentationRegistry.getInstrumentation().getTargetContext();+ assertEquals("net.kuafuai.andee", appContext.getPackageName());+ }+}\ No newline at end of filediff --git a/app/src/main/AndroidManifest.xml b/app/src/main/AndroidManifest.xmlnew file mode 100644index 0000000..00fcf46--- /dev/null+++ b/app/src/main/AndroidManifest.xml@@ -0,0 +1,196 @@+<?xml version="1.0" encoding="utf-8"?>+<manifest xmlns:android="http://schemas.android.com/apk/res/android"+ xmlns:tools="http://schemas.android.com/tools">++ <uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />+ <uses-permission android:name="android.permission.INTERNET" />+ <uses-permission android:name="android.permission.RECORD_AUDIO" />+ <!-- ConnectivityManager.getNetworkCapabilities needs this (not implied by INTERNET). -->+ <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />+ <!-- App list: query-all is required on API 30+ to see other packages. -->+ <uses-permission android:name="android.permission.QUERY_ALL_PACKAGES"+ tools:ignore="QueryAllPackagesPermission" />+ <!-- Location & steps are runtime permissions; DeviceInfoController asks+ nicely and degrades gracefully when denied. -->+ <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />+ <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />+ <uses-permission android:name="android.permission.ACTIVITY_RECOGNITION" />+ <!-- Comms (runtime, all optional per-call): -->+ <uses-permission android:name="android.permission.READ_CONTACTS" />+ <uses-permission android:name="android.permission.CALL_PHONE" />+ <uses-permission android:name="android.permission.READ_CALL_LOG" />+ <uses-permission android:name="android.permission.READ_SMS" />+ <uses-permission android:name="android.permission.SEND_SMS" />+ <uses-permission android:name="android.permission.READ_CALENDAR" />+ <uses-permission android:name="android.permission.WRITE_CALENDAR" />+ <!-- SENSOR_DELAY_FASTEST maps to a 0µs rate, which Android 14+ gates. -->+ <uses-permission android:name="android.permission.HIGH_SAMPLING_RATE_SENSORS" />+ <!-- Moving the keyboard. The device rests on the user's own IME — they can+ type in any app — and the brain borrows ADBKeyboard for the length of+ one `type_text`, because ADBKeyboard has no keyboard UI of its own.+ See ui/ImeSwitch.+ Not grantable at install: this is `signature|privileged|development`, so+ the app has to be installed and then granted by adb+ (`adb shell pm grant net.kuafuai.andee android.permission.WRITE_SECURE_SETTINGS`),+ which the provisioning of this tablet already does. Without the grant+ the brain cannot move the keyboard at all, so `type_text` fails loudly+ and says why — rather than broadcasting into nobody and having the+ failure blamed on the app being driven. -->+ <uses-permission android:name="android.permission.WRITE_SECURE_SETTINGS"+ tools:ignore="ProtectedPermissions" />+ <!-- The eye: -->+ <uses-permission android:name="android.permission.CAMERA" />+ <uses-feature android:name="android.hardware.camera.any" android:required="false" />+ <!-- The dog: its dongle hangs off USB OTG, so this device needs to be a USB+ host. Optional on purpose — the app is fully usable on a tablet with no+ OTG port, and Play would otherwise refuse to install it there. The USB+ node itself needs no manifest permission: access is granted per device,+ at runtime, by the system dialog DogLink raises. -->+ <uses-feature android:name="android.hardware.usb.host" android:required="false" />++ <!-- Pending todos and the quiet-hour sweep timer survive a reboot only if+ we are told about the reboot — AlarmManager forgets alarms across one.+ See config/AlarmReceiver + config/Scheduler. -->+ <uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED" />+ <!-- Used when a todo comes due while the accessibility service is off: the+ promise still has to reach the user. Runtime permission on 13+; the+ code degrades to "no nudge" if it is refused, never to a lost todo. -->+ <uses-permission android:name="android.permission.POST_NOTIFICATIONS" />+ <!-- Battery optimisation. Doze is what holds up the pending todos — they are+ AlarmManager alarms, so with the screen off a non-exempt app's reminders+ arrive late. Exempting is done from Settings' per-app request screen+ (`ACTION_REQUEST_IGNORE_BATTERY_OPTIMIZATIONS`), one dialog —+ 「是否忽略电池优化?」→「忽略」 on a Chinese HONOR, where the+ affirmative button is 忽略 rather than AOSP's 允许.++ The list page is not an alternative on this device, which is why this+ permission is load-bearing rather than optional. Measured on a HONOR+ (MagicOS / Android 15): the 「电池优化」 list+ (`ACTION_IGNORE_BATTERY_OPTIMIZATION_SETTINGS`) shows ten preinstalled+ apps and nothing else — this package is absent even while+ `dumpsys deviceidle whitelist` lists it, `am force-stop+ com.android.settings` does not change that, and the page's own search+ box answers 「没有匹配的结果」 for the package name. So there is no+ user-reachable screen there that can exempt a third-party app.++ And the request screen silently refuses to open without this line:+ `RequestIgnoreBatteryOptimizations.onCreate` checks the permission+ against the package named in its data URI and `finish()`es otherwise —+ `DEBUG = false` there, so not even a log line survives to explain the+ tap that did nothing. Play restricts this permission to apps whose core+ purpose needs it; a sideloaded device assistant is that category.+ See device/SelfCheck.batteryFix. -->+ <uses-permission android:name="android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS" />++ <application+ android:allowBackup="false"+ android:dataExtractionRules="@xml/data_extraction_rules"+ android:fullBackupContent="@xml/backup_rules"+ android:icon="@mipmap/ic_launcher"+ android:label="@string/app_name"+ android:localeConfig="@xml/locales_config"+ android:roundIcon="@mipmap/ic_launcher_round"+ android:supportsRtl="true"+ android:theme="@style/Theme.Body"+ tools:targetApi="33">++ <service+ android:name=".ScreenBodyService"+ android:label="@string/app_name"+ android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"+ android:exported="true"+ tools:ignore="ExportedService">+ <intent-filter>+ <action android:name="android.accessibilityservice.AccessibilityService" />+ </intent-filter>+ <meta-data+ android:name="android.accessibilityservice"+ android:resource="@xml/accessibility_service_config" />+ </service>++ <service+ android:name=".device.NotificationRelayService"+ android:label="@string/notification_relay_label"+ android:permission="android.permission.BIND_NOTIFICATION_LISTENER_SERVICE"+ android:exported="false">+ <intent-filter>+ <action android:name="android.service.notification.NotificationListenerService" />+ </intent-filter>+ </service>++ <!-- Todo alarms and the sweep timer land here; so does a reboot (which+ is when every pending alarm has to be re-armed). Not exported: the+ only senders are our own PendingIntents and the system. -->+ <receiver+ android:name=".config.AlarmReceiver"+ android:exported="false">+ <intent-filter>+ <action android:name="android.intent.action.BOOT_COMPLETED" />+ <action android:name="android.intent.action.QUICKBOOT_POWERON" />+ </intent-filter>+ </receiver>++ <!-- 桌面图标。这个 app 曾经刻意没有 LAUNCHER 入口(见 CLAUDE.md),+ 代价是:无障碍没开的时候,用户手里是一个什么都不做的设备,+ 而且没有任何地方能告诉他为什么。自检页就是那个"为什么"。++ 但图标首先是"打开 app"的那颗图标。所以这里是两条路,由+ ui/SelfCheckActivity.onCreate 决定走哪条:服务在跑 → 和长按+ 球球做同一件事,expand() 展开成全屏卡片,然后 finish;服务没跑+ (无障碍关着、悬浮窗权限缺失、刚重装) → 落到自检页。++ 这不是两个功能拼在一起,是同一个入口的两种状态:进得去的时候+ 不需要解释,进不去的时候这一页必须自己解释 —— 见+ Theme.Body.Stage.NoPreview,它让前者不闪黑。 -->+ <activity+ android:name=".ui.SelfCheckActivity"+ android:theme="@style/Theme.Body.Stage.NoPreview"+ android:exported="true"+ android:launchMode="singleTop"+ android:configChanges="orientation|screenSize|keyboardHidden">+ <intent-filter>+ <action android:name="android.intent.action.MAIN" />+ <category android:name="android.intent.category.LAUNCHER" />+ </intent-filter>+ </activity>++ <activity+ android:name=".device.PermissionRequestActivity"+ android:theme="@style/Theme.Body"+ android:exported="false" />+ <!-- 三个全屏脸共用 Theme.Body.Stage:纯黑底、无标题栏、无系统转场+ 动画。入场/退场由 ui/Stage.kt 自己画,平台那一项闭嘴 —— 见+ values/themes.xml 里关于"关闭时闪一下"的说明。 -->+ <activity+ android:name=".device.ScanActivity"+ android:theme="@style/Theme.Body.Stage"+ android:exported="false"+ android:configChanges="orientation|screenSize|keyboardHidden" />+ <activity+ android:name=".device.LookActivity"+ android:theme="@style/Theme.Body.Stage"+ android:exported="false"+ android:configChanges="orientation|screenSize|keyboardHidden" />+ <activity+ android:name=".ui.HtmlActivity"+ android:theme="@style/Theme.Body.Stage"+ android:exported="false"+ android:configChanges="orientation|screenSize|keyboardHidden" />+ <!-- The ⌨ field a person types into. An Activity and not an overlay+ because on this ROM an overlay window cannot raise the soft+ keyboard — measured; see ui/TextInputActivity. Translucent so the+ app behind stays visible: this adds a field, it does not take the+ screen. -->+ <activity+ android:name=".ui.TextInputActivity"+ android:theme="@android:style/Theme.Translucent.NoTitleBar"+ android:excludeFromRecents="true"+ android:noHistory="true"+ android:launchMode="singleTop"+ android:windowSoftInputMode="adjustResize"+ android:exported="false"+ android:configChanges="orientation|screenSize|keyboardHidden" />++ </application>++</manifest>diff --git a/app/src/main/java/net/kuafuai/andee/ScreenBodyService.kt b/app/src/main/java/net/kuafuai/andee/ScreenBodyService.ktnew file mode 100644index 0000000..beadb37--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ScreenBodyService.kt@@ -0,0 +1,2019 @@+package net.kuafuai.andee++import android.accessibilityservice.AccessibilityService+import android.accessibilityservice.AccessibilityServiceInfo+import android.os.Handler+import android.os.Looper+import android.view.accessibility.AccessibilityEvent+import net.kuafuai.andee.asr.AsrController+import net.kuafuai.andee.audio.AudioIO+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.net.BodyWsServer+import net.kuafuai.andee.net.CommandDispatcher+import net.kuafuai.andee.screen.ScreenController+import net.kuafuai.andee.text.stripMarkdown+import net.kuafuai.andee.tts.TtsController+import net.kuafuai.andee.ui.EdgeRippleUi+import net.kuafuai.andee.ui.FloatingWindowUi+import net.kuafuai.andee.ui.SettingsUi+import android.util.DisplayMetrics+import org.json.JSONObject+import java.util.concurrent.atomic.AtomicReference++/**+ * Body service entry point.+ *+ * Tap the ball → AsrController.toggle() → 火山 streaming ASR.+ * Long-press → TTS test (temporary).+ * ⚙ → SettingsUi (voice config).+ *+ * Exposes ScreenController over WebSocket on port 9008 so the brain (or the+ * viewer, historically) can drive tap/swipe/type/screenshot/ui_tree.+ */+class ScreenBodyService : AccessibilityService() {++ private val ui = Handler(Looper.getMainLooper())+ private lateinit var window: FloatingWindowUi+ private lateinit var ripple: EdgeRippleUi+ private lateinit var marker: net.kuafuai.andee.ui.TapMarkerUi+ private var settings: SettingsUi? = null++ /**+ * The self-check card, when it is up. Separate from [settings] because both+ * can be on screen at once — see [openSelfCheck].+ */+ private var selfCheck: net.kuafuai.andee.ui.SelfCheckUi? = null+ private lateinit var audio: AudioIO+ private lateinit var asr: AsrController+ private lateinit var tts: TtsController++ /**+ * The last finished answer, kept for exactly one reason: if speaking it+ * fails and the card is folded, the scrollback row holding it is `GONE` and+ * that string is the only copy left on the device. See [showAnswerAsCard].+ */+ @Volatile+ private var lastAnswer: String? = null+ private lateinit var meeting: net.kuafuai.andee.meeting.MeetingRecorder+ private lateinit var dogLink: net.kuafuai.andee.dog.DogLink+ private lateinit var dogSense: net.kuafuai.andee.dog.DogSense+ private lateinit var dogMotion: net.kuafuai.andee.dog.DogMotion++ private lateinit var screen: ScreenController+ private lateinit var dispatcher: CommandDispatcher+ private var wsServer: BodyWsServer? = null+ private var hubClient: net.kuafuai.andee.net.BodyWsClient? = null++ /**+ * The on-device agent loop, when `brain = local`. Mutually exclusive with+ * [hubClient] — see [startBrain].+ */+ private var localBrain: net.kuafuai.andee.brain.LocalBrain? = null++ /**+ * Shared state between the inbound server (extension registers here) and+ * the outbound hub client (merges these tools into its own `register`+ * payload). See [net.kuafuai.andee.net.CallExtensionRegistry].+ */+ private val callExtensions = net.kuafuai.andee.net.CallExtensionRegistry()+ private val toolExec = java.util.concurrent.Executors.newSingleThreadExecutor { r ->+ Thread(r, "ScreenTools").apply { isDaemon = true }+ }+ private val stamper = java.text.SimpleDateFormat("yyyyMMdd_HHmmss", java.util.Locale.US)++ // Closed by [stopEverything], reopened when the user starts the next turn.+ // Cancelling the brain's task is asynchronous, so progress / message frames+ // already in flight still arrive after the stop; without this gate the+ // subtitle reappears and TTS picks up where it left off, and the stop+ // button looks like it did nothing.+ @Volatile+ private var acceptBrainOutput = true++ /**+ * Does the last final answer leave the conversation open?+ *+ * Written by [onBrainFinal] from the brain's own judgment — a reply that+ * settles the matter ends with `[END]` (see [END_MARKER]) and flips this+ * to false — and read once, by [followUpWindow], when the answer has+ * finished being read aloud. Default true so a brain that never emits the+ * marker keeps today's always-listen behaviour: the mic closing is+ * opt-in per reply, never a device-side guess.+ */+ @Volatile+ private var expectFollowUp = true++ /**+ * Whether the card was full size when the typing field opened.+ *+ * Held across the field's lifetime and read once, in [typingFinished]: the+ * card is hidden rather than folded while the user types, so this is the+ * only record of the shape to put back.+ */+ @Volatile+ private var typingFromFullscreen = false++ override fun onServiceConnected() {+ instanceRef.set(this)+ // The manifest requests touch-exploration capability because some+ // WeChat builds expose their semantic tree only after this capability+ // is negotiated. Do not keep the request flag active at runtime: that+ // would turn ordinary tablet taps into TalkBack-style exploration.+ serviceInfo = serviceInfo.apply {+ flags = flags and AccessibilityServiceInfo.FLAG_REQUEST_TOUCH_EXPLORATION_MODE.inv()+ }+ window = FloatingWindowUi(this, object : FloatingWindowUi.Listeners {+ override fun onTalkClick() {+ // Deliberately no unfold. Talking from the corner ball is the+ // whole point of having one: you ask about what you are looking+ // at, and pulling the card back would cover exactly that. The+ // ball carries the listening state, and long-press is the way+ // back to the full card when you actually want it.+ // Also barge-in. Tapping the ball while Andee is talking means+ // the user wants the floor, so cutting TTS here keeps the+ // agent's own voice out of the mic.+ startTurn()+ }++ override fun onSettingsClick() {+ openSettings()+ }++ /**+ * ✓ in the control bar — the self-check card.+ *+ * The same *kind* of thing as ⚙ next to it, which is the whole+ * argument for it being a card rather than the fullscreen page it+ * used to be: those two buttons open our own surfaces rather than+ * acting on the screen, so they open the same shape of surface. See+ * [net.kuafuai.andee.ui.SelfCheckUi] for the rest.+ *+ * Deliberately not routed through the dispatcher — nothing is being+ * asked of the assistant, so this starts no task and lights no glow.+ */+ override fun onSelfCheckClick() {+ openSelfCheck()+ }++ override fun onTextInputToggle() {+ toggleTyping()+ }++ override fun onLongPress() {+ // Long-press the ball — folded or not — asks for the full card+ // back: the subtitle band and the tool row only exist at full+ // size, and tapping is taken by the mic.+ dispatcher.expand()+ }++ override fun onToolClick(tool: FloatingWindowUi.Tool) {+ runTool(tool)+ }++ override fun onStopClick() {+ stopEverything()+ }++ // The same fold the assistant's own tool calls get — same corner+ // ball, same glow — so ⤡ and the brain aren't two mechanisms. The+ // only difference is that nothing here comes back on its own.+ override fun onMinimizeClick() {+ dispatcher.compactForUser()+ }++ /**+ * Re-show a page from the scrollback, or read a message fullscreen.+ *+ * One function for both because to the user they are one thing: a+ * fullscreen screen that came out of the conversation. Deliberately+ * *not* through the dispatcher — nothing is being asked of the+ * assistant, so this starts no task and lights no glow.+ */+ override fun onHistoryPageClick(entry: net.kuafuai.andee.ui.ChatHistory.Entry) {+ val file = entry.page ?: return+ openPageFrom(entry, file)+ }++ /**+ * Double-tapping a message: read it fullscreen ([TextPage]).+ *+ * The message is written out as the same kind of page the model+ * composes, so it gets the same reader, the same ✕完成, and the same+ * way back.+ */+ override fun onTextOpen(entry: net.kuafuai.andee.ui.ChatHistory.Entry) {+ val file = net.kuafuai.andee.ui.TextPage.write(this@ScreenBodyService, entry) ?: return+ android.util.Log.i(+ "Body",+ "text page: ${entry.text.length} chars → ${file.name}",+ )+ openPageFrom(entry, file)+ }+ })+ // Before the card is built: FloatingWindowUi.build() reads the log to+ // lay out its first rows, and registers itself as the listener.+ net.kuafuai.andee.ui.ChatHistory.init(this)+ // Same object+init(ctx) shape as the two singletons around it, and for+ // the same reason: it wants a Context and nothing else, so nobody's+ // constructor has to grow an argument. Stays off unless switched on in+ // settings.+ net.kuafuai.andee.screen.GroundingLog.init(this)+ // Logged, because a refusal here is the quietest failure this app has:+ // `addView` without SYSTEM_ALERT_WINDOW throws, `show()` swallows it,+ // and the service goes on to build the whole object graph and answer the+ // socket with nothing at all on screen. `runStartupSelfCheck` cannot+ // report it either — the card it would raise needs the missing+ // permission. So this line is the only trace, and `ensureOverlays` is+ // the way back.+ if (!window.show()) {+ android.util.Log.e("Body", "overlay refused: no window (SYSTEM_ALERT_WINDOW?)")+ }+ // Little signboards (ask/alert cards) live in their own overlay+ // windows — they need nothing but the WindowManager, but init them+ // here where the service context is alive.+ net.kuafuai.andee.ui.CardUi.init(this)+ // The signboard's only source of work. Its listener is a separate+ // service the system binds on its own schedule, and after an APK+ // reinstall it routinely never rebinds — the permission still reads as+ // granted and notifications simply stop arriving. Ask for the rebind+ // from the one place guaranteed to run on every start.+ net.kuafuai.andee.device.NotificationRelayService.ensureBound(this)+ // Start as the card, not as the corner ball.+ //+ // This used to be `setCompact(true)`, and the reason was written above+ // it: a fullscreen overlay is touchable as a whole window — returning+ // false from its root touch handler does not pass the event to the app+ // below — so a service that comes up folded cannot cover whatever the+ // user was doing when they flipped the switch in 设置. That reasoning+ // was about *enabling*, and it cost more than it bought. The card is+ // the only thing that shows the scrollback, the tool row and the+ // minimise control, and a device that came back from a reboot showed+ // none of them: just a 168 dp ball in the corner whose one way in (a+ // long-press) nothing on screen mentions. The user turned this on to+ // have the assistant; it should be there when it starts.+ //+ // Nothing here traps them. The card carries ⤡, and every `screen.*`+ // call still folds it before the brain drives another app (see+ // CommandDispatcher.ensureCompact) — that fold is what the old note was+ // really protecting, and it is untouched.+ //+ // Queued behind `show()`, which still builds the window already compact+ // and seeds `fullGeometry` from that state. One `ui.post` later+ // exitCompact() grows it to the recorded full-screen rectangle, so this+ // arrives as the same geometry change a long-press makes — one code+ // path, not a second way for the window to be born.+ window.setCompact(false)++ // After the card: same window type, so insertion order is z-order and+ // the glow needs to sit on top of the ball, not under it.+ ripple = EdgeRippleUi(this)+ ripple.show()++ // Last of the three overlays, so a mark is never drawn under the glow.+ marker = net.kuafuai.andee.ui.TapMarkerUi(this)+ marker.show()++ audio = AudioIO()+ tts = TtsController(this, audio, window)+ // Follow-up window: when an answer has finished draining, the mic+ // opens by itself for a short beat — as if the user had pressed the+ // talk button the moment the ball went quiet. A conversation where+ // "再帮我…" lands right on the heels of the answer, no reaching for+ // the ball, no wake word. If the user says nothing the recorder's own+ // silence watchdog closes it again in ~3 s — the same close as a+ // manual press that went nowhere, not a new code path.+ tts.onDrained = { ui.post { followUpWindow() } }+ // An answer that could not be spoken is not an answer that failed: the+ // words are already a row in the scrollback by the time speaking+ // starts. Folded, though, that row is `GONE` — so this is where a+ // device with no working speaker still manages to say something.+ tts.onUnavailable = { ui.post { showAnswerAsCard() } }+ // The field the user types into. A window of its own, and an Activity+ // rather than an overlay — an overlay window cannot raise the soft+ // keyboard on this ROM. See ui/TextInputActivity.+ net.kuafuai.andee.ui.TextInputActivity.onSubmit = { submitUserTurn(it, "typed") }+ // Fires on every way out of the field; see [typingFinished].+ net.kuafuai.andee.ui.TextInputActivity.onClosed = { typingFinished() }+ // And this is how the card knows where to stop: the field's own top+ // edge, reported whenever the keyboard resizes it.+ net.kuafuai.andee.ui.TextInputActivity.onTop = {+ android.util.Log.i("Body", "field top: $it px")+ window.setTypingTop(it)+ }+ asr = AsrController(+ this, audio, window,+ onTranscript = { submitUserTurn(it, "voice") },+ onError = { msg -> reportVoiceFailure(msg) },+ )++ // The meeting recorder shares the one AudioIO with the voice turn, so+ // only one of them can ever be live — see [startTurn], where a ball tap+ // during a meeting means "end it" rather than "talk to me".+ meeting = net.kuafuai.andee.meeting.MeetingRecorder(+ this, audio, window,+ expand = { dispatcher.expand() },+ ) { payload ->+ // The user ended it (or a phone call did), so the brain has no idea+ // the meeting is over and no transcript in hand. Ask it for the+ // minutes the way the user would have: as a turn on `asr.final`.+ //+ // Deliberately a query and not a `meeting.ended` event. An event+ // kind is not in ToolSchemas, so the brain has to be taught it+ // separately or it is silently dropped — which looks, from here,+ // exactly like the ball doing nothing. `asr.final` is the one+ // channel every brain already answers, it already lights the glow+ // and starts a task, and what we want *is* a turn. The cost is+ // that the request enters the brain's context as something the+ // user said, so the text below is written to be true when read+ // that way.+ // Display-only, and it follows the interface language. The brain is+ // handed `meetingMinutesQuery` two lines down, not this text:+ // `ChatHistory` is read by the history card and by nothing else.+ // The parenthesised stage-direction shape is kept because it is+ // what the brain *does* receive looks like — the user should see+ // the same thing they would hear about.+ net.kuafuai.andee.ui.ChatHistory.addUser(+ AppLocale.str(this, R.string.svc_meeting_ended_line, payload.optInt("chars"))+ )+ val query = meetingMinutesQuery(payload)+ val client = hubClient+ val server = wsServer+ val brain = localBrain+ val event = JSONObject()+ .put("text", query)+ .put("ts", System.currentTimeMillis())+ // Off the caller's thread for the same reason task.stop is: a+ // WebSocket send blocks once the outbound buffer fills, and this+ // one is running on the recorder's teardown thread.+ Thread {+ runCatching { client?.sendEvent("asr.final", event) }+ runCatching { server?.broadcastEvent("asr.final", event) }+ }.start()+ // The local brain has none of the "teach it a new event kind"+ // problem the comment above is about — it would happily take a+ // purpose-built call. It goes through the same submit path anyway,+ // so a meeting ends identically in both modes and the text above+ // stays the one thing that explains the transcript.+ if (client != null || brain != null || (server?.clientCount() ?: 0) > 0) {+ ui.post {+ acceptBrainOutput = true+ dispatcher.beginTask()+ window.setState(FloatingWindowUi.State.THINKING)+ // After the gate opens, not before: the local brain answers+ // on its own thread, and a stop still pending from the turn+ // before this one would otherwise swallow the minutes.+ brain?.submit(query)+ }+ }+ }++ screen = ScreenController(this, marker)+ // The dog hangs off USB OTG. Nothing here touches the hardware: no port+ // is opened, no permission is asked for, and a tablet with no dongle+ // plugged in pays nothing until something calls a dog_* tool. See+ // DogLink for why the connection is lazy and DogMotion for why the+ // motion tools never block.+ //+ // DogSense registers no sensors here either — the tablet is bolted to+ // the dog, so its gyroscope is the dog's, but a tablet nobody drives a+ // dog from should not be sampling one. DogMotion attaches on first use.+ dogLink = net.kuafuai.andee.dog.DogLink(this)+ dogSense = net.kuafuai.andee.dog.DogSense(this)+ dogMotion = net.kuafuai.andee.dog.DogMotion(dogLink, dogSense)+ dispatcher = CommandDispatcher(+ screen, window, ripple, marker,+ net.kuafuai.andee.device.DeviceInfoController(this),+ net.kuafuai.andee.device.DeviceActionsController(this),+ net.kuafuai.andee.device.DeviceCommsController(this),+ net.kuafuai.andee.device.PermissionsController(this),+ meeting,+ dogMotion,+ this,+ forwardExternal = { method, params ->+ // Unknown to the dispatcher → try the call extension. The+ // server enforces the timeout and turns a missing extension+ // into a proper error; the throw here becomes brain-visible.+ val srv = wsServer+ ?: throw IllegalStateException(+ "no local server running — cannot reach call extension"+ )+ srv.forwardToExtension(method, params)+ },+ // The page no longer blocks the tool call, so this callback is the+ // only way the brain learns the user is done with it. See+ // [notifyPageClosed].+ onPageClosed = { title, seconds -> notifyPageClosed(title, seconds) },+ // The ask/alert cards speak their text; the mute rules live here,+ // not in the dispatcher. See [speakCardText].+ sayAloud = { text -> speakCardText(text) },+ )+ // Extension coming or going invalidates the hub's cached tool list; the+ // hub has no incremental protocol, so we bounce the outbound client and+ // let it re-register with the fresh payload. Same lever [openSettings]+ // already pulls when the hub URL changes.+ callExtensions.onChange = {+ ui.post {+ if (localBrain != null) {+ // Nothing to re-register: the local brain rebuilds its tool+ // list from the registry at the start of every turn, so the+ // next thing the user says already sees the new arm. And+ // bouncing it here would throw away the conversation.+ android.util.Log.i("Body", "call extension state changed → local brain picks it up next turn")+ } else {+ android.util.Log.i("Body", "call extension state changed → restart hub client")+ stopHubClient()+ startHubClient()+ }+ }+ }+ startWsServer()+ // Before the brain, so the very first `type_text` finds the keyboard+ // where it expects it. A no-op unless the device is resting on the+ // headless IME — see [settleKeyboardOwner].+ settleKeyboardOwner()+ // `announce = false`: this start is going to end in one card that lists+ // *everything* that is wrong, so the no-brain complaint must not also+ // fire here or the user gets two cards arguing about the same fact.+ // `runStartupSelfCheck` reports it, in context, with the other rows.+ // The settings card's factory reset still announces normally — see+ // [applyFactoryReset] — because there the user just wiped the config+ // and needs to be told the device is now mute.+ startBrain(announce = false)+ startWakeWord()+ runStartupSelfCheck()+ }++ /**+ * "Is this device actually ready?", asked once per start and only spoken up+ * about when the answer is no.+ *+ * **Why the service asks at all**, given the same list is one tap away on+ * the card (`✓`, right next to ⚙): the card is the *reference*, and this is+ * the *nudge*. The failures [SelfCheck] looks for are all+ * silent by construction — a missing overlay grant means the ball never+ * draws and the service looks healthy, a zombie notification grant means+ * notifications vanish while Settings reports them on. Nobody goes looking+ * for a checklist on a device that appears to be working; they only go+ * looking once something they expected did not happen.+ *+ * **Only [SelfCheck.Level.FAIL] and [SelfCheck.Level.WARN] may interrupt.**+ * The rest of the list (the wake phrase, the disk, a probe that was+ * skipped) is a fact, not a fault, and a startup card that fires on facts+ * is a startup card the user learns to dismiss without reading. This is the+ * whole reason the levels exist.+ *+ * **Deduped by content, not by time.** A device that boots twenty times a+ * day — this one is on a dog — would otherwise get the same card twenty+ * times for the same missing permission. The signature is the *set* of+ * problems, so a new or changed problem gets through immediately, and an+ * unchanged one is mentioned once every [REPEAT_AFTER_MS] until it is+ * fixed. Fixing everything clears the record, so a relapse is reported+ * straight away rather than waiting out the window.+ *+ * Runs on its own thread because [SelfCheck.run] probes the backend and the+ * voice host over the network, and blocks while it does.+ */+ private fun runStartupSelfCheck() {+ if (!::window.isInitialized) return+ val app = applicationContext+ Thread {+ val found = runCatching { net.kuafuai.andee.device.SelfCheck.run(app) }+ .getOrNull() ?: return@Thread+ val problems = found.filter { net.kuafuai.andee.device.SelfCheck.needsAttention(it) }+ val prefs = getSharedPreferences(SELF_CHECK_PREFS, MODE_PRIVATE)+ if (problems.isEmpty()) {+ prefs.edit().remove(SELF_CHECK_SIGNATURE).apply()+ android.util.Log.i("Body", "self-check: all clear (${found.size} checked)")+ return@Thread+ }+ val signature = problems.joinToString(",") { "${it.id}:${it.level}" }+ val now = System.currentTimeMillis()+ val lastAt = prefs.getLong(SELF_CHECK_SEEN_AT, 0L)+ if (prefs.getString(SELF_CHECK_SIGNATURE, null) == signature &&+ now - lastAt < REPEAT_AFTER_MS+ ) {+ android.util.Log.i("Body", "self-check: $signature — already reported, quiet")+ return@Thread+ }+ prefs.edit()+ .putString(SELF_CHECK_SIGNATURE, signature)+ .putLong(SELF_CHECK_SEEN_AT, now)+ .apply()+ android.util.Log.w("Body", "self-check: $signature")+ ui.post { showSelfCheckCard(problems) }+ }.start()+ }++ /**+ * The one card. [problems] is never empty — see [runStartupSelfCheck].+ *+ * The body is the *titles*, not the counts: "悬浮窗权限 · 麦克风" tells the+ * user in two words whether this is the errand they already know about, and+ * the card behind 去处理 is what explains each one. Joining with " · "+ * rather than a comma because the two languages disagree about commas and+ * agree about this.+ *+ * **去处理 opens the card, not the Activity.** This callback only runs with+ * the window up (see the guard below), which means accessibility and the+ * overlay grant are both already fine — so the card can be put up exactly+ * the way `✓` puts it up, and the launcher Activity stays what it is: the+ * door for the device where none of this works.+ */+ private fun showSelfCheckCard(problems: List<net.kuafuai.andee.device.SelfCheck.Finding>) {+ // The overlay grant is one of the things being reported, so the window+ // may well not exist — a card needs the same permission it is about to+ // complain about. The launcher page is the fallback for exactly that.+ if (!::window.isInitialized || !window.isShown()) return+ val open = AppLocale.str(this, R.string.check_card_open)+ val body = problems.joinToString(" · ") { AppLocale.str(this, it.title) }+ net.kuafuai.andee.ui.CardUi.ask(+ AppLocale.str(this, R.string.check_card_title, problems.size) + "\n" + body,+ listOf(open, AppLocale.str(this, R.string.check_card_ack)),+ ) { r ->+ if (r.button == open) openSelfCheck()+ }+ }++ /**+ * Tell the brain the user dismissed a page, **without asking it anything**.+ *+ * `ui.show_html` used to return the news itself, by blocking until the page+ * closed — which cost the conversation (one thread, one turn: a parked tool+ * parkes everything the user says after it). It now returns the moment the+ * page is up, so this is where the other half of that information goes.+ *+ * Deliberately *not* an `asr.final` turn like the meeting minutes below.+ * The minutes are a request — the model has to produce them, so a turn is+ * exactly right. This is a fact, and a turn would buy two bad things: a+ * model call and a spoken sentence every single time a page is dismissed+ * ("好的,看完了"), and an entry in the context that claims the user said+ * it. So the local brain gets it as a silent [LocalBrain.note] (visible to+ * the next turn, never spoken), and a hub gets a purpose-built+ * `page.closed` event it can ignore.+ *+ * A hub that has not been taught `page.closed` drops it silently, which is+ * the pre-existing behaviour for this class of news — nothing is waiting on+ * it, unlike `asr.final`, where a dropped event leaves the ball looking+ * broken.+ *+ * **The note below is English, like the rest of the model-facing set** —+ * see the header of `brain/LocalPrompt.kt`. `values-en/` covers what the+ * *user* reads, and this is not that: nothing at all is displayed here, so+ * no tablet loses anything either way. Same rule as [meetingMinutesQuery],+ * [callSummaryQuery] and the todo wake-up prompts. What the translation+ * does buy is a new problem: an English prompt pulls the answer towards+ * English, which is exactly what §9 of `LocalPrompt` is there to hold back.+ */+ private fun notifyPageClosed(title: String, seconds: Int) {+ // Logged whatever the brain is: this is the one device-side fact in the+ // flow that has no tool call around it any more, so it is the thing to+ // look for when a page closes and something does not happen.+ android.util.Log.i("Body", "page closed: 「$title」 after ${seconds}s")+ localBrain?.note(+ "(background note: the user just closed the page you were showing, 「$title」, after $seconds seconds. " ++ "This is not something the user said, so there is no need to answer; carry on only if that thing still has a next step.)"+ )+ val event = JSONObject()+ .put("title", title)+ .put("seconds", seconds)+ .put("ts", System.currentTimeMillis())+ // Off the caller's thread, same reason the meeting push is: a WebSocket+ // send blocks once the outbound buffer fills, and this one runs on the+ // main thread (HtmlActivity.onDestroy).+ Thread {+ runCatching { hubClient?.sendEvent("page.closed", event) }+ runCatching { wsServer?.broadcastEvent("page.closed", event) }+ }.start()+ }++ /**+ * The request the device makes of the brain when a meeting ends without+ * the brain having asked for it.+ *+ * This is **prompt**, not a log line — it is the whole of what the brain+ * knows about why a transcript just landed in its context, so it has to+ * carry the instruction, the format, and the reason the meeting stopped.+ * `show_html` is named explicitly because it is the only route minutes+ * have to the user's eyes; a brain that merely replies has produced+ * nothing the user asked for.+ *+ * **Almost nothing in here is displayed** — it only lands in the brain's+ * context. The exception is the title on the next line: it becomes the page+ * header the minutes are laid out under, so the user reads it too. It used+ * to be a Chinese literal, on the reasoning that the brain is told the same+ * title and the two must not disagree. The fix keeps that invariant and+ * drops the literal — `MeetingRecorder.start` writes the same key and this+ * reads it back — so the two still agree, *and* an English tablet is not+ * shown a Chinese page header. The prose around it is English like the rest+ * of the model-facing set — see the header of `brain/LocalPrompt.kt`.+ */+ private fun meetingMinutesQuery(p: JSONObject): String {+ val title = p.optString("title")+ .ifEmpty { AppLocale.str(this, R.string.meeting_default_title) }+ val minutes = (p.optLong("duration_ms") / 60_000).coerceAtLeast(1)+ val why = when (p.optString("stopped_by")) {+ // Worth telling it apart: the brain says this back to the user,+ // and a meeting cut short by a phone call may be missing its end.+ "focus" -> "(a phone call or another app took the microphone, so the recording ended early)"+ else -> "(I tapped the ball to end the recording)"+ }+ val truncated = if (p.optBoolean("truncated")) {+ "\nNote: the transcript was too long, so a middle section was left out; the full version is on the device at ${p.optString("file")}."+ } else {+ ""+ }+ return buildString {+ append("$why The meeting just finished, and the transcript is below.\n")+ append("Turn it into meeting minutes, then use the show_html tool to lay the minutes out as an HTML page and show it to me")+ append(" — a text-only reply is something I cannot see.\n")+ append("The minutes should contain: the subject, the key points, the conclusions reached, and action items (who does what).")+ append("The transcript comes out of speech recognition, so it will have wrong characters and missing punctuation; follow the context.\n\n")+ append("Title: $title\nDuration: about $minutes minutes$truncated\n\nTranscript:\n")+ append(p.optString("transcript"))+ }+ }++ /**+ * Open the mic for a turn — the one path, whether the user tapped the ball+ * or said the wake phrase.+ *+ * A toggle rather than a start: tapping the listening ball is how you take+ * it back. The wake word never reaches the "off" half of that, because+ * [WakeWord]'s gate closes its own microphone while this turn is live.+ *+ * During a meeting the ball means one thing only: end it. That is not a+ * special case bolted on — it is the *only* control the user has left,+ * since the recorder holds the microphone and the wake word is gated off,+ * so they cannot ask for anything by voice. It also closes a trap: the two+ * share one [AudioIO], and [AsrController.isActive] is literally+ * `audio.isStreaming()`, so without this a tap would fall into the "off"+ * half of the toggle and tear the recording down with no error anywhere.+ */+ private fun startTurn() {+ if (meeting.isActive()) {+ meeting.stopFromUser()+ return+ }++ if (net.kuafuai.andee.device.CallState.isActive) {+ window.setSubtitle(+ AppLocale.str(this, R.string.svc_call_in_progress),+ FloatingWindowUi.SubtitleKind.FINAL,+ )+ return+ }+ tts.stopSpeaking()+ // Talking takes the floor from typing. Leaving both live would put the+ // keyboard on the user's IME while the brain is about to drive another+ // app — precisely the state `type_text` fails in.+ closeTyping()+ acceptBrainOutput = true+ asr.toggle()+ }++ /**+ * One turn, one way in.+ *+ * A sentence the user **typed** and a sentence they **said** are the same+ * thing from here on, and they have to be: this is what lights the task,+ * writes the scrollback row, tells the hub and hands the text to a local+ * brain. Two copies of it would drift, and the hub event in particular is+ * easy to forget — its failure mode is a device that answers nothing.+ *+ * [how] is only for the log line.+ */+ private fun submitUserTurn(text: String, how: String) {+ val said = text.trim()+ if (said.isEmpty()) return+ android.util.Log.i("Body", "$how: $said")+ // Hand the keyboard back before the turn exists: the brain may call+ // `type_text` inside it, and it can only do that on ADBKeyboard.+ closeTyping()+ // The user is asking for something, so the answer is theirs to see.+ acceptBrainOutput = true+ net.kuafuai.andee.ui.ChatHistory.addUser(said)+ val payload = org.json.JSONObject()+ .put("text", said)+ .put("ts", System.currentTimeMillis())+ hubClient?.sendEvent("asr.final", payload)+ wsServer?.broadcastEvent("asr.final", payload)+ // Same turn, different brain. The local one takes the text directly —+ // there is no socket in between, and no event kind it would have to be+ // taught (see the meeting callback).+ localBrain?.submit(said)+ // Task start, and deliberately at the request rather than at the+ // reply: from here the device is being worked on, whether that reaches+ // us as a tool call or as thirty seconds of thinking. Nobody listening+ // means nobody is thinking about it, so don't light up for an empty+ // room.+ if (hubClient != null || localBrain != null || (wsServer?.clientCount() ?: 0) > 0) {+ dispatcher.beginTask()+ window.setState(FloatingWindowUi.State.THINKING)+ }+ }++ /**+ * Two taps on the ball — the user wants to type instead of talk.+ *+ * Opening is two steps in one order: move the keyboard to a real IME+ * *first*, then put the field up. The other order hands the user a field+ * whose keyboard is headless, and nobody can type into one of those.+ *+ * The card does not go anywhere. It is a fullscreen overlay, and overlays+ * sort above the keyboard, so it has to *shrink* to clear the field — which+ * it learns from the field itself, through `TextInputActivity.onBand`. The+ * first cut hid the card instead and that read badly: what is behind a+ * hidden card is whatever app the user was on, so typing looked like being+ * thrown out of the assistant and then dropped back in.+ */+ private fun toggleTyping() {+ if (net.kuafuai.andee.ui.TextInputActivity.isShowing) {+ net.kuafuai.andee.ui.TextInputActivity.dismiss()+ return+ }+ // No keyboard to borrow and nothing to switch: the device's own IME is+ // the user's, and this field types on it like any other app's would.+ // The only way this fails is a tablet whose single IME is the headless+ // one — where nobody can type anywhere, and a field here would take a+ // sentence and show it to nobody.+ if (net.kuafuai.andee.ui.ImeSwitch.typable(this).isEmpty()) {+ net.kuafuai.andee.ui.CardUi.error(+ AppLocale.str(this, R.string.svc_no_typable_ime),+ )+ return+ }+ typingFromFullscreen = !window.isCompact()+ net.kuafuai.andee.ui.TextInputActivity.start(this)+ }++ /**+ * The field is gone — let the card have its height back, and unfold only if+ * it was unfolded to begin with.+ *+ * Called from `TextInputActivity.onClosed`, which fires from its `onDestroy`+ * on every way out — `✕`, BACK, send, or this service dismissing it. Hanging+ * it off the send button instead would leave the card short of the bottom of+ * the screen with nothing left to grow it back.+ *+ * The second half of the condition is the interesting one. Every `screen.*`+ * call folds the card, because the brain is about to drive another app and a+ * fullscreen card would cover it. So a card that *was* fullscreen can have+ * been folded while the field was up, and unfolding over that would hide the+ * app the brain is working. If it is folded, it stays folded: something has+ * a better reason than the user's place does.+ */+ private fun typingFinished() {+ window.setTypingTop(0)+ if (typingFromFullscreen && !window.isCompact()) dispatcher.expand()+ typingFromFullscreen = false+ }++ /**+ * Show a fullscreen page the user asked for from a row, and put the card+ * back the way they left it.+ *+ * Folding the card is an implementation detail of the page being fullscreen+ * — an overlay sorts above everything, so a card left up would sit on the+ * page — and it therefore has to be undone when the page closes. Without+ * that, ✕完成 reads as "throw me out of the assistant": you open a long+ * answer to read it, close it, and find yourself looking at a ball in the+ * corner of the home screen.+ *+ * The fold is undone unconditionally, unlike the typing field's, and the+ * difference is *who folded*. There, the card never folds, so a fold found+ * on the way out must have been the brain's (every `screen.*` call folds+ * before it drives another app) and is left alone. Here the fold is ours —+ * an implementation detail of the page being fullscreen — so leaving it+ * standing is the bug this fixes. The rare case (the brain folds the card+ * again while the user is reading the page) resolves in favour of the user's+ * place, because the alternative is ✕完成 silently throwing them out of the+ * assistant.+ */+ private fun openPageFrom(entry: net.kuafuai.andee.ui.ChatHistory.Entry, file: java.io.File) {+ val wasFullscreen = !window.isCompact()+ dispatcher.compactForUser()+ // The ball hangs off the ledge rather than sitting on the page.+ window.setPagePerched(true)+ net.kuafuai.andee.ui.HtmlActivity.showFile(this, file, entry.text.take(24)) {+ window.setPagePerched(false)+ if (wasFullscreen) dispatcher.expand()+ }+ }++ /**+ * Put the keyboard back on a real IME if it is resting on the headless one.+ *+ * Only reachable on a device set up under the old rule, or by hand — and+ * worth repairing rather than reporting: nobody chooses a keyboard with no+ * keys, and a device left on it cannot be typed on anywhere. See+ * [net.kuafuai.andee.ui.ImeSwitch.ensureHumanDefault].+ */+ private fun settleKeyboardOwner() {+ net.kuafuai.andee.ui.ImeSwitch.ensureHumanDefault(this)+ }++ /**+ * Voice failed, and the user has to be told *and* offered the way out.+ *+ * The way out belongs **on the card**, not in a sentence pointing at the+ * card's chrome. That was learned the hard way: the message used to name a+ * button in the top bar, and the top bar is `GONE` while the card is folded+ * — folded being where this device spends its life — so it was an+ * instruction to press something that was not on the screen, in the one+ * state the message existed for. A button on the card cannot be hidden by+ * the fold. (The way *in* has since moved onto the ball's double-tap, which+ * is the same lesson applied to the other end: the ball is the one control+ * that is there in both shapes.)+ *+ * The scrollback row is the one [net.kuafuai.andee.ui.CardUi.error] would+ * have left: the failure outlives the card timing out.+ */+ private fun reportVoiceFailure(msg: String) {+ android.util.Log.w("Body", "voice failed: $msg")+ net.kuafuai.andee.ui.ChatHistory.addError(+ AppLocale.str(this, R.string.svc_voice_error_line, msg)+ )+ // The labels are held here and used **twice**: once to build the card,+ // once to recognise the answer. `CardUi` hands the handler back the+ // string it displayed, so a translated label tested against a+ // hardcoded one would never match — the button would look fine and do+ // nothing. One lookup, two uses, and that failure cannot happen.+ val dismiss = AppLocale.str(this, R.string.card_ack)+ val typeInstead = AppLocale.str(this, R.string.svc_btn_type_instead)+ val choices = if (net.kuafuai.andee.ui.ImeSwitch.typable(this).isEmpty()) {+ // Nothing on this device can be typed on. Offering the button would+ // be the same lie in a different shape.+ listOf(dismiss)+ } else {+ listOf(typeInstead, dismiss)+ }+ android.util.Log.i("Body", "voice failed → offering ${choices.first()} on the card")+ net.kuafuai.andee.ui.CardUi.ask(+ AppLocale.str(this, R.string.svc_voice_failed, msg),+ choices,+ ) { r ->+ android.util.Log.i("Body", "voice-failure card answered: ${r.button} (${r.how})")+ if (r.button == typeInstead) toggleTyping()+ }+ }++ /**+ * The device is up and cannot answer anything — no brain configured, or a+ * key missing. Say so, and put the fix on the same card: the settings card+ * opens fine while the ball is folded, the ball's own chrome does not, and+ * "长按小球 → ⚙" is a three-step errand for something one button can do.+ *+ * Both brains have this failure mode and both report it through here, so+ * the sentence differs and nothing else does.+ *+ * [what] arrives already localised — the two callers are the only places+ * that know which brain is missing, so they hand in the sentence. Same+ * one-lookup-two-uses rule as [reportVoiceFailure] for the button.+ *+ * @param card false to write the scrollback row and skip the interruption.+ * Only the service's own start passes false, and only because+ * [runStartupSelfCheck] is about to say the same thing in a card that+ * also carries the other rows. Two cards for one fact read as two+ * problems.+ */+ private fun reportNoBrain(what: String, card: Boolean = true) {+ net.kuafuai.andee.ui.ChatHistory.addError(what)+ if (!card) return+ val openSettingsLabel = AppLocale.str(this, R.string.svc_btn_open_settings)+ net.kuafuai.andee.ui.CardUi.ask(+ what,+ listOf(openSettingsLabel, AppLocale.str(this, R.string.card_ack)),+ ) { r ->+ if (r.button == openSettingsLabel) openSettings()+ }+ }++ /**+ * The field goes away. Safe to call always, including when it was never up.+ *+ * There is nothing to hand back any more: the keyboard only moves inside+ * `ScreenController.typeViaAdbKeyboard`, for the length of one tool call, so+ * between turns the device sits on the user's own IME and this field simply+ * types on it. What this is still for is the *turn* boundary — a turn that+ * starts by voice must not leave a half-typed sentence in a window over the+ * app the brain is about to drive.+ */+ private fun closeTyping() {+ net.kuafuai.andee.ui.TextInputActivity.dismiss()+ }++ /**+ * The answer could not be spoken — put it up instead.+ *+ * Deliberately only when folded. Unfolded, the answer is already a row in+ * the scrollback and a card over it would be noise; folded, that row is+ * `GONE` and without this the user watches the ball go quiet with nothing+ * to read anywhere. A device whose speaker does not work should still be a+ * device that can be used.+ */+ private fun showAnswerAsCard() {+ val answer = lastAnswer ?: return+ if (!window.isCompact()) return+ net.kuafuai.andee.ui.CardUi.alert(answer) { }+ }+++ /**+ * The mic window that opens by itself after an answer drains.+ *+ * Only when the answer left the conversation open — [expectFollowUp], the+ * brain's own judgment carried on the `[END]` marker ([END_MARKER]). A mic+ * that opens after every reply overhears the room: people chat near a+ * tablet that just finished a task, and answering chatter that was not+ * addressed to it is butting in, not helpfulness. A brain that never+ * emits the marker keeps the old always-listen behaviour, so this gate+ * fails open.+ *+ * Reuses [startTurn]'s machinery minus the toggle ambiguity: startTurn is+ * a press-to-toggle and would CLOSE a recording if one were somehow+ * already live. This always opens, and only when nothing else holds the+ * mic — the user may have started talking before the window fired (they+ * don't wait for our window), in which case their own press wins and+ * this is a no-op. The recorder's silence watchdog (3 s) is the close:+ * nothing said → quietly shuts, no task, no card, ball back to idle.+ *+ * The user is never locked out by a closed window: the wake word+ * ([startWakeWord]) and a tap on the ball ([startTurn]) still open a+ * turn whenever they want one.+ *+ * Main thread only (posted by [tts.onDrained]).+ */+ private fun followUpWindow() {+ if (!expectFollowUp) return+ if (asr.isActive() || tts.isSpeaking() || meeting.isActive() ||+ net.kuafuai.andee.device.CallState.isActive+ ) return+ net.kuafuai.andee.ui.ChatHistory.addError(AppLocale.str(this, R.string.svc_listening_again))+ asr.toggle() // not streaming → starts; the 3 s silence watchdog closes it+ }++ /**+ * "嘿 Andee" as a second way to press the ball.+ *+ * The gate is the whole contract with [WakeWord]: it holds the microphone+ * only while every one of these is true, and lets go within ~100 ms of any+ * of them turning false.+ *+ * - **Screen on.** The user's choice, and it keeps a dark tablet on a+ * bedside table from listening all night.+ * - **ASR idle.** The turn's own recorder must never lose a race to the one+ * listening for a wake word.+ * - **TTS quiet.** Otherwise Andee hears itself say "嘿" and wakes up to+ * its own voice.+ * - **No meeting running.** `asr.isActive()` happens to cover this today+ * (it is `audio.isStreaming()`, which a meeting also sets), but that is a+ * coincidence inside a method named for the ASR. Spelled out so a future+ * narrowing of that method doesn't hand the wake word the microphone in+ * the middle of an hour-long recording.+ */+ private fun startWakeWord() {+ screenOn = (getSystemService(POWER_SERVICE) as android.os.PowerManager).isInteractive+ androidx.core.content.ContextCompat.registerReceiver(+ this,+ screenReceiver,+ android.content.IntentFilter().apply {+ addAction(android.content.Intent.ACTION_SCREEN_ON)+ addAction(android.content.Intent.ACTION_SCREEN_OFF)+ },+ androidx.core.content.ContextCompat.RECEIVER_NOT_EXPORTED,+ )+ net.kuafuai.andee.wake.WakeWord.init(+ this,+ gate = {+ screenOn && !asr.isActive() && !tts.isSpeaking() &&+ !meeting.isActive() && !net.kuafuai.andee.device.CallState.isActive &&+ !net.kuafuai.andee.audio.Earcon.isSounding()+ },+ onWake = { ui.post { if (!asr.isActive()) startTurn() } },+ )+ net.kuafuai.andee.wake.WakeWord.start()+ }++ @Volatile+ private var screenOn = true++ private val screenReceiver = object : android.content.BroadcastReceiver() {+ override fun onReceive(c: android.content.Context?, i: android.content.Intent?) {+ screenOn = i?.action != android.content.Intent.ACTION_SCREEN_OFF+ }+ }++ override fun onInterrupt() {}++ override fun onAccessibilityEvent(event: android.view.accessibility.AccessibilityEvent?) {+ if (event == null) return+ if (event.eventType == android.view.accessibility.AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED) {+ event.packageName?.toString()?.let { net.kuafuai.andee.device.DeviceState.foregroundPkg = it }+ }+ if (::screen.isInitialized) screen.noteEvent(event)+ }++ override fun onDestroy() {+ // First, before anything else: the dog is the only thing this service+ // drives that keeps acting after we are gone. Every motion is supposed+ // to end on its own, but "supposed to" is not a reason to leave motors+ // running — cancel the timers, send the stop, let go of the cable.+ if (::dogMotion.isInitialized) dogMotion.shutdown()+ if (::dogLink.isInitialized) dogLink.close()+ stopBrain()+ stopWsServer()+ net.kuafuai.andee.wake.WakeWord.stop()+ runCatching { unregisterReceiver(screenReceiver) }+ if (::asr.isInitialized) asr.shutdown()+ if (::tts.isInitialized) tts.shutdown()+ // Before audio.release(): it ends the recording, which needs the mic+ // stream to still exist, and flushes the last segment to disk.+ if (::meeting.isInitialized) meeting.shutdown()+ if (::audio.isInitialized) audio.release()+ toolExec.shutdownNow()+ // shutdown, not shutdownNow: its queue is a few small appends and one+ // PNG, and losing them would silently truncate the record.+ net.kuafuai.andee.screen.GroundingLog.shutdown()+ settings?.hide()+ settings = null+ // Same treatment, and the extra call is the prober thread: the two+ // network probes run on their own executor and would otherwise hold a+ // thread for up to a connect timeout after the service is gone.+ selfCheck?.hide()+ selfCheck?.destroy()+ selfCheck = null+ // Before the windows go: this one has to put the keyboard back, and a+ // service that dies mid-typing would otherwise leave the tablet with+ // its keyboard on the user's IME — where every future `type_text`+ // fails, silently, with an error that blames the target app.+ closeTyping()+ if (::ripple.isInitialized) ripple.hide()+ if (::marker.isInitialized) marker.hide()+ if (::window.isInitialized) window.hide()+ instanceRef.compareAndSet(this, null)+ super.onDestroy()+ }++ private fun openSettings() {+ if (settings?.isShowing() == true) return+ // The settings card is focusable and full of text fields, so it takes+ // the keyboard the moment it appears. Leaving ours up would leave two+ // focusable windows fighting, with the user's sentence in the one that+ // lost. The keyboard goes back to the brain with it.+ closeTyping()+ // Snapshot hub and brain config before the user edits them; if either+ // changed on dismiss, bring the brain back up so the new URL / name /+ // mode / key take effect without needing to bounce the accessibility+ // service.+ val before = net.kuafuai.andee.config.VoiceConfig.hubConfig(this)+ val beforeBrain = net.kuafuai.andee.config.VoiceConfig.brainConfig(this)+ // Set when the user wiped everything from inside the card: the brain is+ // already back up on the emptied prefs by the time the card closes, and+ // running the comparison below would restart it a second time for the+ // same reason.+ var rebuiltByReset = false+ val s = SettingsUi(+ context = this,+ onDismiss = {+ settings = null+ if (rebuiltByReset) {+ rebuiltByReset = false+ } else {+ val after = net.kuafuai.andee.config.VoiceConfig.hubConfig(this)+ val afterBrain = net.kuafuai.andee.config.VoiceConfig.brainConfig(this)+ if (before.url != after.url || before.deviceName != after.deviceName ||+ beforeBrain != afterBrain+ ) {+ android.util.Log.i("Body", "brain/hub config changed → restart brain")+ // Restarting the local brain discards the conversation,+ // which is the right trade: the user just changed the+ // model, the key or the endpoint out from under it.+ stopBrain()+ startBrain()+ }+ }+ // The self-check card can be sitting *under* this sheet: its+ // 打开设置 row opens us on top of itself. So this picker is+ // reachable while that card is on screen, and its language has+ // to follow or the user closes the sheet onto a card in the+ // language they just changed away from. The card's own picker+ // tells us outward; this is the same news arrived from the other+ // direction, so the card is only repainted, never re-announced.+ selfCheck?.repaintForLanguage()+ },+ onFactoryReset = {+ applyFactoryReset()+ rebuiltByReset = true+ },+ onClearChat = { clearChatHistory() },+ onClearNotes = { clearNotebook() },+ // The card owns its own language switch; the ball, the scrollback+ // and the pinned tips live in this service and have to be told.+ // See FloatingWindowUi.onLocaleChanged.+ onLocaleChanged = { window.onLocaleChanged() },+ )+ settings = s+ s.show()+ }++ /**+ * The self-check card, in a window of its own.+ *+ * [net.kuafuai.andee.ui.SelfCheckUi] draws it; this method is only the+ * wiring, and each wire is a decision worth naming:+ *+ * * **A window, not the Activity.** `SelfCheckActivity` still exists and is+ * still in the launcher, but it is now the *fallback* — it is the one+ * surface that runs with the accessibility service off, which is exactly+ * the state this list is most needed in. Whenever we are running, we can+ * put the card up ourselves, and doing so keeps the check one tap from+ * the gear rather than a round trip through a fullscreen page.+ * * **Not exclusive with [settings].** The card offers 打开设置, which opens+ * the settings sheet *over* it — the list stays behind to return to+ * instead of the user losing their place. Two overlays, and+ * [net.kuafuai.andee.ui.OwnCard] counts both.+ * * **No `closeTyping()`.** Unlike [openSettings] there is no text field+ * here, so there is no keyboard to hand back and nothing to compete for+ * focus.+ */+ private fun openSelfCheck() {+ if (selfCheck?.isShowing() == true) return+ val s = net.kuafuai.andee.ui.SelfCheckUi(+ context = this,+ onDismiss = { selfCheck = null },+ // Our own sheet, reached through the service because that is where+ // it lives. Never null from here: if this method is running, we are.+ onOpenSettings = { openSettings() },+ // Give the screen back to the page we are about to open. Both windows+ // here — the folded-back assistant card and this very card — are+ // TYPE_APPLICATION_OVERLAY, and an overlay outranks every Activity on+ // the device. Without this, 去下载 opened a browser underneath the+ // card that had just asked the user to look at it, and 去开启 did the+ // same to the system Settings page. The fold is the one every+ // screen.* tool call and ⤡ already perform; this is one more place+ // that needs it. See SelfCheckUi.apply for the full argument.+ //+ // The card is closed rather than hidden, because there is no honest+ // way back: the browser is not ours and reports nothing when the user+ // returns, so a card left up would sit over whatever they actually+ // came back to. Folding the assistant is what makes the way back+ // sane instead — the ball is there, and the check reopens from ✓+ // with a fresh reading, which is what the user wants after they have+ // been off changing something anyway. Closing has to come last:+ // SelfCheckUi.hide runs onDismiss from Glass.exit's completion, and+ // that is what clears [selfCheck].+ onYieldScreen = {+ dispatcher.compactForUser()+ selfCheck?.hide()+ },+ // The card owns its language switch; the ball and the scrollback live+ // in this service and have to be told the same way [openSettings]+ // tells them.+ onLocaleChanged = { window.onLocaleChanged() },+ )+ selfCheck = s+ s.show()+ // Paint the local half immediately, then fill in the two probes behind+ // it. See SelfCheckUi.start.+ s.start()+ }++ /**+ * 恢复出厂设置: wipe everything this app owns, then rebuild out of nothing.+ *+ * Called from the settings card, after the user has read and confirmed the+ * list it showed them. The wiping is [net.kuafuai.andee.config.FactoryReset]'s+ * job; what belongs here is the half only a running service can do —+ * dropping the state the *process* still holds, and bringing the brain back+ * up against preferences that are now empty.+ *+ * [stopBrain] / [startBrain] rather than just a data wipe, because the+ * local brain holds the conversation in memory and the old key inside its+ * HTTP client: a reset that left it running would have the device keep+ * talking to the user with credentials that no longer exist. `startBrain`+ * then reads the emptied prefs, lands on the default (hub, no URL) and+ * dials nothing — which is what a fresh install does. It also re-runs+ * [catchUpPromises] with no promises left to catch.+ */+ private fun applyFactoryReset() {+ val cleared = net.kuafuai.andee.config.FactoryReset.wipe(this)+ android.util.Log.i("Body", "factory reset: $cleared target(s) cleared — rebuilding the brain")+ stopBrain()+ startBrain()+ }++ /**+ * 清空聊天记录, from the 诊断 section of the settings card.+ *+ * Two halves, and wiping only the first would be a lie: [ChatHistory.wipe]+ * empties the scrollback the user reads, but the conversation the *model*+ * is holding lives in the brain's memory — so the brain is rebuilt with+ * it. Anything less and the user has cleared a chat the device still+ * remembers word for word.+ *+ * The rebuild also aborts a turn in flight, deliberately: "forget this+ * conversation" said while one is running means that one too.+ */+ private fun clearChatHistory() {+ net.kuafuai.andee.ui.ChatHistory.wipe(this)+ android.util.Log.i("Body", "chat cleared — rebuilding the brain so the conversation goes with it")+ stopBrain()+ startBrain()+ }++ /**+ * 清空小本本, from the same place.+ *+ * **No brain rebuild, and that is the difference from [clearChatHistory].**+ * The notebook is not carried in the model's context: its index is read off+ * disk and stamped onto each user message, and `recall` goes to disk too.+ * So an empty notebook is in force from the very next turn without touching+ * the conversation — and taking the conversation down as well would be this+ * button doing the other button's job, for a user who pressed exactly one+ * of them.+ *+ * A turn already in flight can still be holding what an earlier `recall`+ * returned. Left alone on purpose: those are results the assistant was+ * given before the wipe, and yanking them mid-sentence would have it+ * contradict itself inside one answer.+ */+ private fun clearNotebook() {+ net.kuafuai.andee.config.Notebook.wipe(this)+ android.util.Log.i("Body", "notebook cleared")+ }++ /**+ * Stop the current task. Runs entirely on the caller (UI) thread because+ * every local step is non-blocking: [TtsController.stopSpeaking] cuts the+ * AudioTrack inline, [AsrController.stop] only queues onto its executor.+ * Deliberately not routed through [toolExec] — that queue may be holding a+ * swipe mid-dispatchGesture, which is seconds of exactly the wait we're+ * trying to abort.+ *+ * Three layers, and only the local ones are guaranteed: the brain is asked+ * to cancel over the hub socket, and a tool request already dispatched to+ * this device will still run to completion.+ */+ private fun stopEverything() {+ // ■ during a meeting means the same thing the ball does — end it, keep+ // what was said, let the minutes be written. Not "throw the hour away":+ // the transcript is the only copy, and the rest of this method would+ // leave the recorder holding the microphone anyway.+ if (meeting.isActive()) {+ meeting.stopFromUser()+ return+ }+ acceptBrainOutput = false+ tts.stopSpeaking()+ if (asr.isActive()) asr.stop()+ // Task over from the device's point of view: whatever the brain had+ // queued up, we've stopped wanting it to move the app.+ dispatcher.cancelTask()+ window.setState(FloatingWindowUi.State.IDLE)+ window.setSubtitle(+ AppLocale.str(this, R.string.svc_stopped),+ FloatingWindowUi.SubtitleKind.FINAL,+ )+ // The local brain is the one layer that genuinely stops: it drops the+ // turn and aborts the HTTP call in flight. A tool already dispatched+ // still runs to completion, same as over the hub.+ localBrain?.cancel()+ // Off the UI thread: WebSocket send blocks once the outbound buffer+ // fills. Silently dropped when no hub is connected, which degrades to+ // "stopped locally" — the right outcome for server-only mode.+ val client = hubClient+ if (client != null) {+ Thread {+ runCatching {+ client.sendEvent(+ "task.stop",+ JSONObject().put("ts", System.currentTimeMillis()),+ )+ }+ }.start()+ }+ }++ /**+ * Run a top-bar tool on a background thread (ScreenController.tap/swipe+ * block on dispatchGesture for up to ~2 s and would ANR the UI thread).+ * Result is echoed to the floating window subtitle.+ */+ private fun runTool(tool: FloatingWindowUi.Tool) {+ window.setSubtitle("${tool.name.lowercase()}…", FloatingWindowUi.SubtitleKind.PARTIAL)+ toolExec.execute {+ try {+ val result = when (tool) {+ FloatingWindowUi.Tool.SWIPE_UP -> swipe(dx = 0, dy = -1)+ FloatingWindowUi.Tool.SWIPE_DOWN -> swipe(dx = 0, dy = 1)+ FloatingWindowUi.Tool.SWIPE_LEFT -> swipe(dx = -1, dy = 0)+ FloatingWindowUi.Tool.SWIPE_RIGHT -> swipe(dx = 1, dy = 0)+ // actor="user": these bypass the dispatcher, so without it+ // the grounding log would file the user's own navigation+ // under the brain's and the offline pairing would read a+ // human's back-press as the agent correcting itself.+ FloatingWindowUi.Tool.BACK -> screen.globalAction("back", actor = "user")+ FloatingWindowUi.Tool.HOME -> screen.globalAction("home", actor = "user")+ FloatingWindowUi.Tool.UI_TREE -> saveUiTree(screen.dumpUiTree(verbose = true))+ // Through the dispatcher, not ScreenController directly: the+ // overlay has to be blanked or the capture is just our own card.+ FloatingWindowUi.Tool.SCREENSHOT ->+ saveScreenshot(dispatcher.dispatch("screen.screenshot", null) as JSONObject)+ }+ window.setSubtitle(+ "${tool.name.lowercase()} · $result",+ FloatingWindowUi.SubtitleKind.FINAL,+ )+ } catch (t: Throwable) {+ net.kuafuai.andee.ui.CardUi.error(+ "${tool.name.lowercase()} · ${t.message ?: t.javaClass.simpleName}"+ )+ }+ }+ }++ /**+ * Full-screen center swipe, 60% of the axis. dispatchGesture simulates+ * real touch events, which the overlay window would grab first — so we+ * flip the overlay to non-touchable for the duration of the swipe, then+ * restore it. Overlay stays visible either way.+ */+ private fun swipe(dx: Int, dy: Int): JSONObject {+ val dm: DisplayMetrics = resources.displayMetrics+ val cx = dm.widthPixels / 2+ val cy = dm.heightPixels / 2+ val ax = (dm.widthPixels * 0.3f).toInt()+ val ay = (dm.heightPixels * 0.3f).toInt()+ val durationMs = 300L+ window.setTouchable(false)+ // The glow window counts toward the opacity Android checks before it+ // trusts an injected gesture, so it has to step aside too — see+ // EdgeRippleUi's KDoc. Same for a tap marker still on screen.+ ripple.setVisibleForGesture(false)+ marker.setVisibleForGesture(false)+ // updateViewLayout is posted to the UI thread; give it a couple of+ // frames to actually apply before the gesture starts.+ Thread.sleep(50)+ try {+ // Goes straight to the pixel swipe, so swipeNorm's own hook never+ // fires — but the screen still moves under everything the brain+ // was aiming at, and this one was the user's doing.+ net.kuafuai.andee.screen.GroundingLog.noteSwipe(+ nx1 = (cx - dx * ax) * 1000 / dm.widthPixels,+ ny1 = (cy - dy * ay) * 1000 / dm.heightPixels,+ nx2 = (cx + dx * ax) * 1000 / dm.widthPixels,+ ny2 = (cy + dy * ay) * 1000 / dm.heightPixels,+ durationMs = durationMs,+ actor = "user",+ )+ return screen.swipe(cx - dx * ax, cy - dy * ay, cx + dx * ax, cy + dy * ay, durationMs)+ } finally {+ ripple.setVisibleForGesture(true)+ marker.setVisibleForGesture(true)+ window.setTouchable(true)+ }+ }++ /**+ * The full UI tree is huge — write to a JSON file under app-external+ * files so you can `adb pull` it, and return a small summary + path.+ * External files (Context.getExternalFilesDir) don't need storage+ * permission and are visible to `adb pull`.+ */+ private fun saveUiTree(tree: JSONObject): JSONObject {+ val dir = java.io.File(getExternalFilesDir(null), "captures").apply { mkdirs() }+ val file = java.io.File(dir, "ui_${stamper.format(java.util.Date())}.json")+ file.writeText(tree.toString(2), Charsets.UTF_8)+ android.util.Log.i("Body", "ui_tree saved: ${file.absolutePath}")+ return JSONObject()+ .put("pkg", tree.optString("pkg"))+ .put("class", tree.optString("class"))+ .put("children", tree.optJSONArray("children")?.length() ?: 0)+ .put("file", file.name)+ .put("bytes", file.length())+ }++ /**+ * Save the PNG bytes to disk (skipping the base64 round-trip in the+ * subtitle), and return a small summary + path.+ */+ private fun saveScreenshot(shot: JSONObject): JSONObject {+ val b64 = shot.optString("png_base64", "")+ val bytes = android.util.Base64.decode(b64, android.util.Base64.NO_WRAP)+ val dir = java.io.File(getExternalFilesDir(null), "captures").apply { mkdirs() }+ val file = java.io.File(dir, "screen_${stamper.format(java.util.Date())}.png")+ file.writeBytes(bytes)+ android.util.Log.i("Body", "screenshot saved: ${file.absolutePath}")+ return JSONObject()+ .put("width", shot.optInt("width"))+ .put("height", shot.optInt("height"))+ .put("bytes", bytes.size)+ .put("file", file.name)+ }++ private fun startWsServer() {+ try {+ val srv = BodyWsServer(+ port = WS_PORT,+ registry = callExtensions,+ onExtensionEvent = { kind, data -> onExtensionEvent(kind, data) },+ onCommand = { method, params ->+ // driving = false: this socket is the debug / viewer path,+ // and it polls the tree and the screen on a timer. See+ // [CommandDispatcher.dispatch].+ dispatcher.dispatch(method, params, driving = false)+ },+ )+ srv.start()+ wsServer = srv+ android.util.Log.i("Body", "ws server started on 0.0.0.0:$WS_PORT")+ } catch (t: Throwable) {+ android.util.Log.e("Body", "ws server start failed", t)+ }+ }++ /**+ * An event pushed up by the currently-registered call extension. Three+ * kinds are meaningful; anything else is passed through in case a client+ * on the hub wants it.+ *+ * - `call.state` — informational, forwarded so the brain knows something+ * is happening but not asked to write minutes. Ringing/connected/hangup.+ *+ * - `call.subtitle` — what the caller (or the human) just said, and the+ * translation of it. Drives the ball's live row; the brain does not+ * normally read these frame-by-frame, but they broadcast anyway so a+ * viewer client can display them.+ *+ * - `call.summary` — the call is over and there is a transcript. Same rule+ * as the meeting recorder: no new event kind, we send it back as+ * `asr.final` with a prompt-shaped text that asks the brain to render an+ * HTML page. See CLAUDE.md's meeting-recorder section for why.+ *+ * Runs on the WS callback thread; only [ui.post] the parts that touch+ * views.+ */+ private fun onExtensionEvent(kind: String, data: JSONObject) {+ val client = hubClient+ val server = wsServer+ when (kind) {+ "call.subtitle" -> {+ val src = data.optString("source_text")+ val tx = data.optString("translation_text")+ val isFinal = data.optBoolean("is_final", false)+ val text = when {+ src.isNotEmpty() && tx.isNotEmpty() -> "$src\n→ $tx"+ tx.isNotEmpty() -> tx+ else -> src+ }+ if (text.isNotEmpty()) {+ ui.post {+ window.setSubtitle(+ text,+ if (isFinal) FloatingWindowUi.SubtitleKind.FINAL+ else FloatingWindowUi.SubtitleKind.PARTIAL,+ )+ }+ }+ }++ "call.summary" -> {+ val summaryQuery = callSummaryQuery(data)+ // Display-only, same as the meeting twin above: the brain gets+ // `callSummaryQuery`, the card gets this.+ net.kuafuai.andee.ui.ChatHistory.addUser(+ AppLocale.str(+ this,+ R.string.svc_call_ended_line,+ data.optInt("chars", data.optString("transcript").length),+ )+ )+ val event = JSONObject()+ .put("text", summaryQuery)+ .put("ts", System.currentTimeMillis())+ Thread {+ runCatching { client?.sendEvent("asr.final", event) }+ runCatching { server?.broadcastEvent("asr.final", event) }+ }.start()+ if (client != null || (server?.clientCount() ?: 0) > 0) {+ ui.post {+ acceptBrainOutput = true+ dispatcher.beginTask()+ window.setState(FloatingWindowUi.State.THINKING)+ }+ }+ }++ else -> {++ }+ }+ }++ /**+ * The prompt sent to the brain when a call ends and the device has no idea+ * what the brain wanted to do about it — same shape as [meetingMinutesQuery],+ * same reason: it lands in the brain's context as if the user had said it.+ *+ * The prose is English like the rest of the model-facing set. The `scene`+ * default on the line below is [meetingMinutesQuery]'s title case again: the+ * value reaches the page the summary is laid out under, so it comes from+ * `R.string.call_default_scene` through `AppLocale` rather than a literal.+ */+ private fun callSummaryQuery(p: JSONObject): String {+ val scene = p.optString("scene")+ .ifEmpty { AppLocale.str(this, R.string.call_default_scene) }+ val minutes = (p.optLong("duration_ms") / 60_000).coerceAtLeast(1)+ val transcript = p.optString("transcript")+ val reason = when (p.optString("reason")) {+ "user" -> "(I hung up from this end)"+ "brain" -> "(the AI hung up)"+ "error" -> "(the call dropped unexpectedly)"+ else -> ""+ }+ return buildString {+ append("$reason The $scene just ended, and the full record of the conversation is below.\n")+ append("Turn it into a summary of the key points, then use the show_html tool to lay the summary out as an HTML page and show it to me —")+ append(" a text-only reply is something I cannot see.\n")+ append("The summary should contain: the subject of the call, the key points, the conclusions reached, and action items.")+ append("The conversation comes out of speech recognition, so it will have wrong characters and missing punctuation; follow the context.\n\n")+ append("Scene: $scene\nDuration: about $minutes minutes\n\nConversation:\n")+ append(transcript)+ }+ }++ private fun stopWsServer() {+ val srv = wsServer ?: return+ wsServer = null+ // stop() blocks up to timeout; run off the main thread.+ Thread { runCatching { srv.stop(500) } }.start()+ }++ /**+ * Bring up whichever brain the user configured — exactly one of them.+ *+ * `brain = local` runs the agent loop on this device ([LocalBrain]) and+ * does **not** dial the hub; anything else is the hub, which is what this+ * device has always done. They are mutually exclusive on purpose: two+ * brains answering the same `asr.final` is a bug the user hears, as two+ * voices talking over each other.+ *+ * A local brain with no API key still starts. It says so on the first+ * thing the user asks, which is a better place to find out than a silent+ * fall back to a hub they thought they had switched off.+ *+ * @param announce whether the missing-configuration state also gets a card+ * of its own ([reportNoBrain]). `true` everywhere except the service's own+ * start, where [runStartupSelfCheck] is already about to put up one card+ * covering this and everything else — see that method. The scrollback row+ * is written either way: it is a log of what happened, not an+ * interruption.+ */+ private fun startBrain(announce: Boolean = true) {+ val bc = net.kuafuai.andee.config.VoiceConfig.brainConfig(this)+ // One line that answers "is my override in effect?" without reading+ // shared_prefs over adb. The api key is overridable now, and a setting+ // whose effect cannot be seen is indistinguishable from one that did not+ // take. The speaker prints what it is so the imp's voice change is+ // verifiable the same way — it follows the ball's look, not a setting.+ android.util.Log.i(+ "Body",+ "voice: 火山 key from " ++ if (net.kuafuai.andee.config.VoiceConfig.usingOwnCredentials(this)) {+ "settings"+ } else {+ "factory"+ } + ", speaker=" ++ net.kuafuai.andee.config.VoiceConfig.load(this).ttsSpeaker,+ )+ if (!bc.isLocal) {+ // A device with nowhere to send a sentence is silent, and silence+ // with no explanation reads as broken rather than as unconfigured.+ // This is also exactly the state a factory reset leaves behind, so+ // it is worth the one card. Both brains have the same failure mode+ // and both get told about it here, before anything starts.+ if (net.kuafuai.andee.config.VoiceConfig.hubConfig(this).url.isEmpty()) {+ reportNoBrain(AppLocale.str(this, R.string.svc_no_brain_hub), announce)+ }+ startHubClient()+ // Not just for the local brain. Pending todos lose their+ // AlarmManager alarms when the app is replaced or the device+ // reboots, and re-arming them has nothing to do with which brain is+ // configured. With no local brain the catch-up can only notify —+ // see [catchUpPromises] — but that is still better than an alarm+ // that silently never comes back.+ catchUpPromises()+ return+ }+ android.util.Log.i(+ "Body",+ "brain=local — skipping outbound client (model=${bc.model} @ ${bc.baseUrl}, " ++ "thinking=${bc.thinking}/${bc.reasoningEffort}, key=${if (bc.apiKey.isEmpty()) "MISSING" else "set"})",+ )+ // Same silence, same explanation as the hub branch above: a local brain+ // with no key answers nothing at all, and "nothing at all" is the one+ // failure a user cannot diagnose from the outside.+ if (bc.apiKey.isEmpty()) {+ reportNoBrain(AppLocale.str(this, R.string.svc_no_brain_local), announce)+ }+ localBrain = net.kuafuai.andee.brain.LocalBrain(+ cfg = bc,+ appContext = this,+ dispatcher = dispatcher,+ registry = callExtensions,+ onProgress = ::onBrainProgress,+ onFinal = ::onBrainFinal,+ onTurnEnd = ::onTurnEnd,+ onSweepSpent = { tokens -> net.kuafuai.andee.config.Notebook.noteSweep(this, tokens) },+ )+ // Promises outlive the process: anything that came due while the+ // accessibility service was off (or the tablet was off) is dealt with+ // here, on the first start that can hear about it. This runs in hub+ // mode too — see the early return above.+ catchUpPromises()+ }++ private fun stopBrain() {+ stopHubClient()+ // No brain, no reason for a timer that would wake one.+ net.kuafuai.andee.config.Scheduler.cancelSweep(this)+ val lb = localBrain ?: return+ localBrain = null+ lb.stop()+ }++ /**+ * A turn just ended. Push the quiet-hour timer out.+ *+ * Last write wins: every turn re-arms it, so a conversation that keeps+ * going never looks finished to itself. This is the only place the timer+ * is started — see [onSweepTimer] for the other half.+ */+ private fun onTurnEnd() {+ val sweep = net.kuafuai.andee.config.VoiceConfig.sweepConfig(this)+ if (!sweep.enabled) return+ net.kuafuai.andee.config.Scheduler.armSweepIn(this, sweep.quietMinutes * 60_000L)+ }++ /**+ * The timer fired. Has the conversation actually stopped?+ *+ * The timer is a single shot armed at the end of a turn; if the user spoke+ * in the meantime — or the alarm landed late — the honest answer is "not+ * yet", and the remainder is waited out rather than sweeping mid-sentence.+ */+ fun onSweepTimer() {+ val sweep = net.kuafuai.andee.config.VoiceConfig.sweepConfig(this)+ val brain = localBrain+ if (!sweep.enabled || brain == null) {+ android.util.Log.i(+ "Body",+ "sweep timer fired with nothing to do " ++ "(enabled=${sweep.enabled}, localBrain=${brain != null})",+ )+ return+ }+ val quietMs = sweep.quietMinutes * 60_000L+ val idle = System.currentTimeMillis() -+ net.kuafuai.andee.config.Notebook.lastActivityAt(this)+ if (idle < quietMs) {+ net.kuafuai.andee.config.Scheduler.armSweepIn(this, quietMs - idle)+ return+ }+ if (!net.kuafuai.andee.config.Notebook.sweepAllowed(this, sweep.dailyCap)) {+ val used = net.kuafuai.andee.config.Notebook.sweepCountToday(this)+ android.util.Log.i("Body", "sweep budget for today is used up ($used/${sweep.dailyCap})")+ return+ }+ brain.runSweep()+ }++ /**+ * A promise came due and the alarm got through — wake the brain up.+ *+ * Called from [net.kuafuai.andee.config.AlarmReceiver], so it may be on the+ * main thread; [net.kuafuai.andee.brain.LocalBrain.wake] only posts, and+ * defers itself if a turn is already running.+ */+ fun wakeForTodo(todo: net.kuafuai.andee.config.Notebook.Todo) {+ val brain = localBrain+ if (brain == null) {+ // Nothing on this device can run a turn (no local brain — there is+ // no hub fallback any more). Tell the user rather than swallowing+ // it: a promise that evaporates silently is the failure this+ // exists to prevent.+ android.util.Log.i("Body", "todo due, but no local brain here — notifying instead")+ net.kuafuai.andee.config.Scheduler.notifyMissed(this, todo)+ net.kuafuai.andee.config.Scheduler.markCaughtUp(this, todo.id)+ return+ }+ // An alarm is the one turn nobody asked for, so it is the one that can+ // start while the user is mid-sentence in the field. Take the field down+ // first: otherwise the brain's `type_text` borrows the keyboard out from+ // under a sentence being typed by hand, and the app the brain is driving+ // gets a keyboard raised over it instead.+ closeTyping()+ // Prompt, not display: [notifyPageClosed] on why the model-facing set+ // is English and what that costs.+ brain.wake(+ "(time's up · this is a todo you set yourself, id=${todo.id}) ${todo.what}\n" ++ "Do it now. If it cannot be done, still say honestly why and give them a next step; " ++ "when it is done (or said), close it with todos(action=done, id=${todo.id}).",+ )+ }++ /**+ * Whatever came due while nothing was listening.+ *+ * Delivered as a **silent note**, not a turn: the user did not just say+ * anything, so a voice coming out of the tablet to announce a missed+ * reminder would be startling and badly timed. The note rides into the+ * context, and the next time they talk the brain explains and either does+ * it or re-arranges it. [net.kuafuai.andee.config.Scheduler.catchUp] also+ * re-arms everything still in the future.+ *+ * The note is context, not chrome, so it is English with the rest of the+ * model-facing set — see [notifyPageClosed].+ */+ private fun catchUpPromises() {+ val missed = net.kuafuai.andee.config.Scheduler.catchUp(this)+ val brain = localBrain+ for (todo in missed) {+ if (brain != null) {+ brain.note(+ "(missed) you had promised: ${todo.what}" ++ " (set for ${net.kuafuai.andee.config.Scheduler.formatLocal(todo.at)}) — " ++ "I was not running when that time came, so it was never done. The next time you talk to the user, explain this first, " ++ "then either do it or arrange a new time with them.",+ )+ } else {+ // No brain on this device right now (there is no hub fallback+ // any more). The promise still has to reach the user, so it+ // goes out as a notification rather than evaporating.+ net.kuafuai.andee.config.Scheduler.notifyMissed(this, todo)+ }+ net.kuafuai.andee.config.Scheduler.markCaughtUp(this, todo.id)+ }+ }++ /**+ * Bring up the outbound hub connection if [VoiceConfig.hubConfig] has a+ * URL configured. No-op when empty — running body_ws server-only is a+ * valid mode (local viewer / same-LAN brain).+ *+ * Only reached from [startBrain] and from the two places that bounce the+ * hub connection in place (extension registry changes, settings dismiss).+ */+ private fun startHubClient() {+ val hc = net.kuafuai.andee.config.VoiceConfig.hubConfig(this)+ if (hc.url.isBlank()) {+ android.util.Log.i("Body", "hub_url not set — skipping outbound client")+ return+ }+ val client = net.kuafuai.andee.net.BodyWsClient(+ url = hc.url,+ deviceId = hc.deviceId,+ deviceName = hc.deviceName,+ dispatcher = dispatcher,+ registry = callExtensions,+ onProgress = ::onBrainProgress,+ onFinal = ::onBrainFinal,+ )+ client.start()+ hubClient = client+ }++ /**+ * Narration from whichever brain is answering.+ *+ * Extracted from [startHubClient] rather than copied into [LocalBrain]:+ * this and [onBrainFinal] are where the two TTS mute rules, the+ * [acceptBrainOutput] gate and the ChatHistory bookkeeping live, and two+ * brains drifting apart on any of those is a bug the user sees before we do.+ *+ * Called from a socket thread (hub) or the `LocalBrain-loop` thread+ * (local); everything it touches posts to the main Handler itself.+ */+ private fun onBrainProgress(text: String) {+ // Task start. The brain only ever speaks mid-task, so every frame+ // of narration re-marks it: thinking looks like nothing at all on+ // the wire (no tool calls), and that's exactly when the "being+ // driven" light has to stay on.+ if (acceptBrainOutput) {+ dispatcher.beginTask()+ window.setState(FloatingWindowUi.State.THINKING)+ window.setSubtitle(+ // Drafts of a settling reply may already carry the end marker;+ // it must not flash on the live line either.+ stripMarkdown(END_MARKER.replace(text, "")),+ FloatingWindowUi.SubtitleKind.PARTIAL,+ )+ }+ }++ /** The finished answer. See [onBrainProgress] for why this is not inline. */+ private fun onBrainFinal(text: String) {+ if (acceptBrainOutput) {+ // The brain's own verdict on whether this conversation is still+ // open. Written before the speak below because [followUpWindow]+ // reads it seconds later, off the TTS drain — see [expectFollowUp].+ expectFollowUp = !END_MARKER.containsMatchIn(text)+ val plain = stripMarkdown(END_MARKER.replace(text, ""))+ // The finished answer is the end of the task as far as the+ // device is concerned: nothing further will move the app.+ dispatcher.endTask()+ // Whatever draft `onBrainProgress` left on the live line is now+ // finished work — and nothing else was going to take it+ // down, so the last mid-task sentence sat at the foot of+ // the scrollback forever, under the answer that replaced it.+ window.clearSubtitle()+ // Only the final answer goes in the scrollback. `onBrainProgress`+ // fires many times per task with successive drafts of the+ // same sentence — logging those would bury the answer under+ // its own rough cuts.+ net.kuafuai.andee.ui.ChatHistory.addAssistant(plain)+ // Kept for one reason only: if speaking this fails and the card is+ // folded, the row above is `GONE` and this string is the last copy+ // of the answer on the device. See [showAnswerAsCard].+ lastAnswer = plain+ // The row above is the answer. speak() deliberately puts+ // nothing on the live line — the ball's speaking face says+ // it is being read out, and the words are already up there.+ //+ // Silent through a meeting, for two reasons and either+ // alone would be enough. Our speaker is an inch from the+ // microphone, so anything said here lands in the+ // transcript. And [TtsController] holds its own audio+ // focus gate: a second AudioFocusRequest from this same+ // process displaces the first (the focus stack is per+ // request, not per app), so speaking would fire the+ // recorder's focus-loss callback and end the meeting —+ // which is exactly what "好的,开始录音了" used to do, one+ // second in.+ //+ // Silent through a phone / VoIP call for the mirror-image+ // reason: the caller app owns the audio focus, and our+ // AudioFocusRequest from TTS would displace theirs the+ // same way it displaces the meeting recorder's. The+ // scrollback row above is already the answer; the user+ // reads it after they hang up.+ if (!meeting.isActive() && !net.kuafuai.andee.device.CallState.isActive) tts.speak(plain)+ }+ }++ /**+ * A card the brain just put up — the `ui.ask` question, the `ui.alert`+ * notice — read aloud, and the whole of the `tts.speak` tool, which is the+ * same request without a card. The card is for a user who may be looking+ * anywhere but the screen; a question nobody hears waits in silence for+ * its timeout.+ *+ * Deliberately the same gates as the `tts.speak` in [onBrainFinal], not a+ * fresh set: speaking through a meeting would both land in the transcript+ * and end the meeting via the audio-focus collision, speaking through a+ * call displaces the caller's focus, and speaking after the user hit stop+ * is a stopped turn still talking. The dispatcher cannot know any of that,+ * which is why it hands the text over ([CommandDispatcher.sayAloud])+ * instead of holding a TTS handle.+ *+ * Returns whether the words went to TTS. `ui.ask` / `ui.alert` drop the+ * answer — their card is the deliverable and it is up either way — but+ * `tts.speak` IS the utterance, so a refusal has to travel back as a+ * refusal. Its caller is usually a person recording a demo, and "nothing+ * happened and nothing said why" is the worst answer to give them.+ */+ private fun speakCardText(text: String): Boolean {+ if (!acceptBrainOutput) return false+ if (meeting.isActive() || net.kuafuai.andee.device.CallState.isActive) return false+ // A card is its own answer channel — the ask's buttons, the alert's+ // 知道了. The follow-up mic must not open over it: a spoken answer+ // would arrive as a *new turn* queued behind the very tool call the+ // card is blocking, and a mic held open while the user decides+ // overhears the room. The user who does want to talk has the wake+ // word and the ball.+ //+ // True for `tts.speak` as well, and for a different version of the same+ // reason: a rehearsal line is not a question the assistant asked, so+ // the mic must not come up hunting for an answer to it.+ expectFollowUp = false+ tts.speak(stripMarkdown(text))+ return true+ }++ private fun stopHubClient() {+ val c = hubClient ?: return+ hubClient = null+ Thread { runCatching { c.stop() } }.start()+ }++ companion object {+ private const val WS_PORT = 9008++ /**+ * The "conversation over" marker a final answer may end with — the+ * brain's judgment that this reply settles the matter and asks the+ * user nothing (see §10 of `LocalPrompt.TEXT`). Stripped before the+ * answer is spoken or shown, so the user never hears it; its presence+ * is what flips [expectFollowUp]. Case-insensitive so a model that+ * writes `[end]` still gets the marker stripped instead of having it+ * read aloud.+ */+ private val END_MARKER = Regex("""(?i)\[END]""")+ private val instanceRef = AtomicReference<ScreenBodyService?>()+ fun get(): ScreenBodyService? = instanceRef.get()++ /** Where [runStartupSelfCheck] remembers what it already said. */+ private const val SELF_CHECK_PREFS = "self_check"+ private const val SELF_CHECK_SIGNATURE = "last_signature"+ private const val SELF_CHECK_SEEN_AT = "last_seen_at"++ /**+ * How long an unchanged set of problems stays quiet after being shown.+ *+ * Six hours, and the unit is deliberately coarse: the point is to catch+ * the case where the card has *not* been acted on and the device is+ * being restarted repeatedly (a reinstall loop, a tablet that reboots+ * on the dog) without hiding a real problem for a whole day. Anything+ * that changes the set — a new failure, or one fixed — gets through+ * immediately, because the signature is compared, not the clock.+ */+ private const val REPEAT_AFTER_MS = 6 * 60 * 60 * 1000L+ }++ /**+ * The notification relay is a separate service with its own lifecycle and+ * no wiring of its own; this is how it reaches the task light. Named for+ * the caller rather than the mechanic — anything else that ever needs+ * "start the ball working" from outside the socket path goes through here.+ */+ fun dispatcherForNotification(): CommandDispatcher? =+ if (::dispatcher.isInitialized) dispatcher else null++ /**+ * The ball's window, for CardUi — signboards anchor to where the ball+ * actually is, and the relay (a separate service) has no other way to+ * reach it.+ */+ fun ballWindow(): FloatingWindowUi? =+ if (::window.isInitialized) window else null++ /**+ * Put the three overlays on screen if they are not already there, and say+ * whether the ball made it. Main thread only.+ *+ * This exists because [onServiceConnected] **throws its answer away**:+ * `window.show()` returns false when `WindowManager.addView` is rejected+ * for want of `SYSTEM_ALERT_WINDOW`, and the service then runs perfectly+ * with nothing on screen. Nothing recovers from that on its own, and+ * nothing can even complain about it — [runStartupSelfCheck] bails when the+ * ball is not shown, because the card it would put up needs the very+ * permission that is missing. So the app is a live process with no visible+ * surface and no voice, which is what "点不开了" looks like from outside.+ *+ * The state is reachable without anybody doing anything wrong: a package+ * rename or a fresh install loses the appop, the user grants it *after* the+ * accessibility service has already connected, and the grant alone does not+ * re-run [onServiceConnected]. Toggling accessibility off and on was the+ * only cure, which is knowledge no user has.+ *+ * All three windows are retried, not just the ball: the ripple and the tap+ * marker were added in the same frame and refused for the same reason, so a+ * recovery that restored only the ball would come back without the task+ * light. Each `show()` is idempotent (it returns true early when its view+ * already exists), so this is safe to call on a healthy service.+ *+ * The unfold is the same `setCompact(false)` birth does, for the same+ * reason — the card is what carries the scrollback and the control bar, and+ * a recovery that surfaced only the corner ball would be a second, worse+ * way to be born.+ */+ fun ensureOverlays(): Boolean {+ if (!::window.isInitialized) return false+ if (window.isShown()) return true+ if (!window.show()) return false+ window.setCompact(false)+ if (::ripple.isInitialized) ripple.show()+ if (::marker.isInitialized) marker.show()+ return true+ }++ /**+ * Open the settings sheet for someone outside this class — presently the+ * self-check page, whose 打开设置 row is a door to it.+ *+ * Not exposed as an Intent: the sheet is built from this service's own+ * state (which brain is configured, what the hub URL is) and a second+ * instance built from an Activity context would be a *different* sheet+ * fighting this one for the keyboard. Posted to the main thread because+ * `SettingsUi.show` adds a window.+ *+ * A null service means no sheet — the caller's row is then a dead button,+ * which is why [net.kuafuai.andee.ui.SelfCheckUi] draws no 打开设置 row at+ * all when the accessibility check is red (see its `serviceUp`), and says so+ * in words instead. Guarded on [window] rather than on any "am I+ * alive" flag: a sheet over a service that never got its overlay attached+ * has nothing to sit on, and the add would throw.+ */+ fun openSettingsForUser() {+ ui.post { if (::window.isInitialized) openSettings() }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/asr/AsrController.kt b/app/src/main/java/net/kuafuai/andee/asr/AsrController.ktnew file mode 100644index 0000000..7b57be4--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/asr/AsrController.kt@@ -0,0 +1,241 @@+package net.kuafuai.andee.asr++import android.content.Context+import android.os.Handler+import android.os.Looper+import android.os.SystemClock+import android.util.Log+import net.kuafuai.andee.audio.AudioFocusGate+import net.kuafuai.andee.audio.AudioIO+import net.kuafuai.andee.config.VoiceConfig+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.ui.FloatingWindowUi+import java.util.concurrent.Executors++/**+ * ASR session lifecycle glued to [FloatingWindowUi].+ *+ * IDLE ── toggle() ─→ RECORDING (mic open, WS streaming)+ * │+ * ├── partial → subtitle PARTIAL, resets the silence clock+ * ├── SILENCE → stop() exactly as if the user had tapped+ * │ again — see [checkSilence]+ * ├── final → subtitle FINAL, IDLE, onTranscript(text)+ * ├── final "" → subtitle cleared, IDLE, brain not woken.+ * │ The turn where nobody spoke; not an error+ * ├── error → subtitle ERROR, IDLE+ * └── toggle() → IDLE (user cancelled before final)+ *+ * All 火山 WS callbacks arrive on the client's IO thread; we marshal them to+ * the UI thread via FloatingWindowUi's own Handler.+ */+class AsrController(+ private val context: Context,+ private val audio: AudioIO,+ private val window: FloatingWindowUi,+ private val onTranscript: (String) -> Unit = {},+ /**+ * Voice failed. Called with the recogniser's own message, after the ball+ * has been put into ERROR.+ *+ * Routed to the owner rather than shown here because the interesting part+ * of the sentence is not the failure — it is what the user can do instead,+ * and only the service knows whether this device has a text field. Default+ * reproduces the old behaviour for any future caller that has no answer to+ * offer.+ */+ private val onError: (String) -> Unit = { msg ->+ net.kuafuai.andee.ui.CardUi.error(msg)+ },+) {+ @Volatile+ private var asr: HuoshanAsr? = null+ private val executor = Executors.newSingleThreadExecutor { r ->+ Thread(r, "AsrController").apply { isDaemon = true }+ }++ /**+ * Silence the rest of the device for the length of the turn.+ *+ * EXCLUSIVE rather than plain transient, which is the difference between the+ * media app stopping and the media app turning itself down: ducked audio is+ * still coming out of the speaker, and the speaker is roughly one inch from+ * the microphone we are about to open. Recognition against a tablet playing+ * a video is hopeless without this.+ */+ private val focus = AudioFocusGate(+ context,+ android.media.AudioAttributes.USAGE_ASSISTANT,+ android.media.AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE,+ )++ // ---- hands-free turn taking ----++ /** Main-thread silence watchdog; see [checkSilence]. */+ private val silence = Handler(Looper.getMainLooper())+ private val silenceWatchdog = Runnable { checkSilence() }++ /** When the recognizer last reported *new* words. Written from the 火山 IO thread. */+ @Volatile+ private var lastSpeechAt = 0L++ /**+ * Last partial text we counted. Written from the 火山 IO thread. Guards+ * against a recognizer that re-sends the same hypothesis while the user is+ * silent — without it the watchdog would never fire.+ */+ @Volatile+ private var lastPartial = ""++ fun isActive(): Boolean = audio.isStreaming()++ /** Called from the UI thread when the user taps the ball. */+ fun toggle() {+ executor.execute {+ try {+ if (audio.isStreaming()) stopImpl() else startImpl()+ } catch (t: Throwable) {+ Log.e(TAG, "toggle failed", t)+ runCatching { stopImpl() }+ window.setState(FloatingWindowUi.State.ERROR)+ onError(t.message ?: "asr error")+ }+ }+ }++ fun stop() {+ executor.execute { runCatching { stopImpl() } }+ }++ fun shutdown() {+ runCatching { stopImpl() }+ executor.shutdownNow()+ }++ private fun startImpl() {+ val cfg = VoiceConfig.load(context)+ // First thing, so the media app has the whole 火山 handshake to actually+ // stop in — asking and then opening the mic in the same breath would+ // still catch the tail of whatever was playing. A phone call takes the+ // floor back: end the turn rather than transcribe one side of a call.+ focus.acquire { stop() }++ val listener = object : HuoshanAsr.Listener {+ override fun onPartial(text: String) {+ if (text.isEmpty()) return+ window.setSubtitle(text, FloatingWindowUi.SubtitleKind.PARTIAL)+ if (text != lastPartial) {+ lastPartial = text+ lastSpeechAt = SystemClock.uptimeMillis()+ }+ }++ override fun onFinal(text: String) {+ // Either way the band comes down. An empty turn — tapped the+ // ball, said nothing — has nothing to show; a real transcript+ // is about to become the first row of the scrollback via+ // onTranscript, and leaving it lit here as well is the same+ // sentence on screen twice.+ window.clearSubtitle()+ executor.execute {+ runCatching { stopImpl() }+ if (text.isNotBlank()) onTranscript(text)+ }+ }++ override fun onError(msg: String) {+ Log.w(TAG, "asr error: $msg")+ // Same reason as onFinal: the turn is over. The failure is going+ // up as a card and as a red row, and "听着…" is no longer true.+ window.clearSubtitle()+ window.setState(FloatingWindowUi.State.ERROR)+ onError(msg)+ executor.execute { runCatching { stopImpl() } }+ }++ override fun onClosed() {}+ }++ val client = HuoshanAsr(cfg, listener)+ client.start(sampleRate = 16_000, channels = 1)+ asr = client+ window.setState(FloatingWindowUi.State.RECORDING)+ window.setSubtitle(+ AppLocale.str(context, net.kuafuai.andee.R.string.asr_listening),+ FloatingWindowUi.SubtitleKind.PARTIAL,+ )++ audio.startStreaming(sampleRate = 16_000, chunkMs = 100) { chunk ->+ // Logged rather than swallowed: a send that throws every time looks+ // from the outside exactly like a mic that produces nothing — the+ // session stays open with no packets in it and 火山 ends it after+ // eight seconds, which reads as a network timeout and is not one.+ runCatching { asr?.sendAudio(chunk, isLast = false) }+ .onFailure { Log.w(TAG, "sendAudio failed", it) }+ }+ armSilenceWatchdog()+ }++ /**+ * Start counting towards an automatic stop.+ *+ * The mic opens with a tap and there is nothing demanding a second one:+ * the user says what they have to say and then stops talking, and waiting+ * for them to press anything again would turn a conversation into a+ * two-handed exercise — and now that talking happens from the corner ball,+ * with no card in the way, "I'm done" has to be inferable from silence.+ */+ private fun armSilenceWatchdog() {+ lastSpeechAt = SystemClock.uptimeMillis()+ lastPartial = ""+ silence.postDelayed(silenceWatchdog, SILENCE_CHECK_MS)+ }++ /**+ * Main thread only. Nothing new out of the recognizer for [SILENCE_MS] and+ * we close the mic ourselves — [stop] sends `isLast`, waits for 火山's final+ * line, and that final line is what reaches the brain and starts the task.+ * Indistinguishable from the user tapping again, which is the point.+ */+ private fun checkSilence() {+ if (!audio.isStreaming()) return // finished already+ val quiet = SystemClock.uptimeMillis() - lastSpeechAt+ if (quiet < SILENCE_MS) {+ silence.postDelayed(silenceWatchdog, SILENCE_CHECK_MS)+ return+ }+ Log.i(TAG, "silence ${quiet}ms — auto-stop")+ runCatching { stop() }+ }++ private fun stopImpl() {+ silence.removeCallbacks(silenceWatchdog)+ val client = asr+ if (audio.isStreaming()) {+ audio.stopStreaming()+ runCatching { client?.sendAudio(ByteArray(0), isLast = true) }+ .onFailure { Log.w(TAG, "isLast failed", it) }+ }+ // Wait for 火山 to send the isLast response before closing — otherwise+ // the final transcript never arrives (server sends it ~500ms after our+ // isLast, but a 200ms fixed sleep isn't enough).+ runCatching { client?.waitFinished(3000) }+ runCatching { client?.close() }+ asr = null+ // Only now, so the media app doesn't resume into 火山's final transcript+ // — that arrives ~500 ms after our isLast and the user is still waiting+ // on it.+ focus.release()+ window.setState(FloatingWindowUi.State.IDLE)+ }++ companion object {+ private const val TAG = "Asr"++ /** Quiet this long after the last word and the turn is over. */+ private const val SILENCE_MS = 3000L++ /** How often the watchdog looks; only affects how promptly it stops. */+ private const val SILENCE_CHECK_MS = 250L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/asr/HuoshanAsr.kt b/app/src/main/java/net/kuafuai/andee/asr/HuoshanAsr.ktnew file mode 100644index 0000000..5b2e7a2--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/asr/HuoshanAsr.kt@@ -0,0 +1,362 @@+package net.kuafuai.andee.asr++import android.util.Log+import net.kuafuai.andee.config.VoiceConfig+import org.java_websocket.client.WebSocketClient+import org.java_websocket.handshake.ServerHandshake+import org.json.JSONObject+import java.io.ByteArrayOutputStream+import java.net.URI+import java.nio.ByteBuffer+import java.util.UUID+import java.util.concurrent.TimeUnit+import java.util.concurrent.atomic.AtomicInteger+import java.util.zip.GZIPInputStream+import java.util.zip.GZIPOutputStream+import javax.net.ssl.SSLContext++/**+ * ByteDance Volcano (火山) streaming ASR client.+ * Direct Kotlin port of wxBot's ui/mic_asr_stream.py binary protocol.+ *+ * Wire format for each message:+ * header (4 bytes) + seq (int32 BE) + payloadSize (uint32 BE) + gzipped(payload)+ *+ * Header bytes:+ * [0] (proto_ver<<4) | header_size(=1) → 0x11+ * [1] (message_type<<4) | flags+ * [2] (serialization<<4) | compression → 0x11 (JSON + gzip)+ * [3] reserved (0)+ */+class HuoshanAsr(+ private val config: VoiceConfig,+ private val listener: Listener,+) {+ interface Listener {+ fun onPartial(text: String)+ fun onFinal(text: String)+ fun onError(msg: String)+ fun onClosed()+ }++ private var client: WebSocketClient? = null++ // Full client request uses seq=1; audio chunks start at 2 (火山 protocol).+ private val seq = AtomicInteger(2)++ @Volatile+ private var lastPartial: String = ""++ @Volatile+ private var gotAnyText: Boolean = false+ private val chunksSent = AtomicInteger(0)++ // Counted down when server sends an isLast response, or on error/close.+ // waitFinished() blocks on this so we don't close the WS while the final+ // message is still in flight.+ private val finishedLatch = java.util.concurrent.CountDownLatch(1)++ /**+ * Why the handshake failed, when it failed by being *answered* rather than+ * by going unanswered.+ *+ * `connectBlocking` returns false for both, so without this the one error+ * anyone ever sees is "connect timeout" — which sends whoever reads it to+ * look at the network, when the truth is usually a server that replied at+ * once and said no (a blank or revoked `X-Api-Key` answers 403). That has+ * already cost one round of debugging; the self-check's voice row probes a+ * bare TCP connect for the same reason, and a green row beside a red+ * "timeout" is the contradiction this removes.+ *+ * Only written before [connected] is set, so a normal close at the end of+ * a session cannot overwrite it with something irrelevant.+ */+ @Volatile+ private var handshakeRefusal: String? = null++ @Volatile+ private var connected = false++ fun start(sampleRate: Int = 16_000, channels: Int = 1) {+ val requestId = UUID.randomUUID().toString()+ val connectId = UUID.randomUUID().toString()+ val headers = mapOf(+ "X-Api-Key" to config.apiKey,+ "X-Api-Resource-Id" to config.asrResourceId,+ "X-Api-Request-Id" to requestId,+ "X-Api-Sequence" to "-1",+ "X-Api-Connect-Id" to connectId,+ )++ Log.d(TAG, "start: endpoint=${config.asrEndpoint} sr=$sampleRate uid=${config.asrUid}")+ val uri = URI(config.asrEndpoint)+ val c = object : WebSocketClient(uri, headers) {+ override fun onOpen(handshake: ServerHandshake?) {+ connected = true+ Log.d(+ TAG,+ "onOpen status=${handshake?.httpStatus} msg=${handshake?.httpStatusMessage}"+ )+ val cfg = JSONObject().apply {+ put("user", JSONObject().put("uid", config.asrUid))+ put(+ "audio",+ JSONObject()+ .put("format", "pcm")+ .put("codec", "raw")+ .put("rate", sampleRate)+ .put("bits", 16)+ .put("channel", channels),+ )+ put(+ "request",+ JSONObject()+ .put("model_name", "bigmodel")+ .put("enable_itn", true)+ .put("enable_punc", true)+ .put("enable_ddc", true)+ .put("show_utterances", true)+ .put("enable_nonstream", false),+ )+ }+ send(buildFullClientRequest(1, cfg.toString()))+ Log.d(TAG, "sent full client request payload=$cfg")+ }++ override fun onMessage(message: String?) {+ Log.d(TAG, "onMessage(text): $message")+ }++ override fun onMessage(bytes: ByteBuffer?) {+ bytes ?: return+ val raw = ByteArray(bytes.remaining())+ bytes.get(raw)+ handleResponse(raw)+ }++ override fun onClose(code: Int, reason: String?, remote: Boolean) {+ if (!connected) handshakeRefusal = reason?.takeIf { it.isNotBlank() } ?: "code $code"+ Log.d(+ TAG,+ "onClose code=$code reason=$reason remote=$remote chunksSent=${chunksSent.get()}"+ )+ listener.onClosed()+ finishedLatch.countDown()+ }++ override fun onError(ex: Exception?) {+ Log.e(TAG, "onError", ex)+ listener.onError("ws error: ${ex?.message ?: ex?.javaClass?.simpleName}")+ finishedLatch.countDown()+ }+ }+ // Force TLS for wss://.+ if (uri.scheme.equals("wss", ignoreCase = true)) {+ c.setSocketFactory(SSLContext.getDefault().socketFactory)+ }+ client = c+ val opened = c.connectBlocking(5, TimeUnit.SECONDS)+ if (!opened) {+ val refusal = handshakeRefusal+ throw IllegalStateException(+ if (refusal == null) {+ "ASR ws connect timeout"+ } else {+ // The server answered. Name the key, because a blank or+ // revoked one is what 403 means here and the endpoint is+ // the thing everyone checks first.+ "ASR ws refused: $refusal (check the 火山 API Key in ⚙)"+ },+ )+ }+ }++ fun sendAudio(pcm: ByteArray, isLast: Boolean) {+ val c = client ?: return+ val current = seq.getAndIncrement()+ val frame = buildAudioRequest(current, pcm, isLast)+ c.send(frame)+ val n = chunksSent.incrementAndGet()+ if (isLast || n <= 3 || n % 20 == 0) {+ Log.d(TAG, "sendAudio seq=$current bytes=${pcm.size} isLast=$isLast totalChunks=$n")+ }+ }++ fun close() {+ try {+ client?.close()+ } catch (_: Throwable) {+ }+ client = null+ }++ // ---- Response handling ----++ private fun handleResponse(msg: ByteArray) {+ if (msg.size < 4) return+ val headerSize = (msg[0].toInt() and 0x0f)+ val messageType = ((msg[1].toInt() shr 4) and 0x0f)+ val flags = msg[1].toInt() and 0x0f+ val compression = msg[2].toInt() and 0x0f++ var offset = headerSize * 4+ var isLast = false++ if (flags and 0x01 != 0) offset += 4 // seq present+ if (flags and 0x02 != 0) isLast = true // last package+ if (flags and 0x04 != 0) offset += 4 // event present++ val payload: ByteArray = when (messageType) {+ MSG_SERVER_FULL_RESPONSE -> {+ if (offset + 4 > msg.size) return+ val size = intAtBE(msg, offset)+ offset += 4+ if (offset + size > msg.size) return+ msg.copyOfRange(offset, offset + size)+ }++ MSG_SERVER_ERROR_RESPONSE -> {+ if (offset + 8 > msg.size) return+ val code = intAtBE(msg, offset)+ val size = intAtBE(msg, offset + 4)+ offset += 8+ val body = if (offset + size <= msg.size) msg.copyOfRange(+ offset,+ offset + size+ ) else ByteArray(0)+ val decompressed =+ if (compression == COMPRESSION_GZIP) runCatching { ungzip(body) }.getOrDefault(+ body+ ) else body+ val errMsg = String(decompressed, Charsets.UTF_8)+ Log.e(TAG, "recv ERROR code=$code msg=$errMsg")+ listener.onError("ASR server error code=$code msg=$errMsg")+ finishedLatch.countDown()+ return+ }++ else -> {+ Log.w(TAG, "recv unknown msgType=$messageType flags=$flags")+ return+ }+ }++ val decompressed = if (compression == COMPRESSION_GZIP) {+ runCatching { ungzip(payload) }.getOrElse {+ Log.w(TAG, "gunzip failed on ${payload.size} bytes")+ return+ }+ } else payload++ val bodyStr = String(decompressed, Charsets.UTF_8)+ Log.d(+ TAG,+ "recv msgType=$messageType flags=$flags isLast=$isLast body=${bodyStr.take(300)}"+ )++ val text = try {+ JSONObject(bodyStr).optJSONObject("result")?.optString("text").orEmpty().trim()+ } catch (_: Throwable) {+ Log.w(TAG, "response payload not JSON with result.text")+ return+ }++ if (text.isNotEmpty()) {+ lastPartial = text+ gotAnyText = true+ listener.onPartial(text)+ }+ if (isLast) {+ if (gotAnyText) {+ listener.onFinal(lastPartial)+ } else {+ // A clean end with nothing recognised is an empty turn: the ball+ // got tapped and nobody said anything. It used to be reported as+ // an error, which meant flashing a red server-diagnostic string+ // at a user whose only mistake was silence.+ //+ // The auth / config failures this once tried to catch don't look+ // like this — they arrive as MSG_SERVER_ERROR_RESPONSE above, or+ // as a ws error, never as a well-formed isLast after a handshake+ // and N accepted audio chunks. Kept at info level because the+ // chunk count is the first thing to look at if the mic itself is+ // broken: an empty turn still sends chunks, a dead mic doesn't.+ Log.i(+ TAG,+ "isLast with no transcription — empty turn " ++ "(chunksSent=${chunksSent.get()}, body=${bodyStr.take(200)})",+ )+ listener.onFinal("")+ }+ finishedLatch.countDown()+ }+ }++ /**+ * Block up to [timeoutMs] waiting for the server's isLast response (or an+ * error / socket close). Call this after sending the isLast audio chunk,+ * before [close], so the final transcript has a chance to arrive.+ */+ fun waitFinished(timeoutMs: Long = 3000): Boolean =+ finishedLatch.await(timeoutMs, TimeUnit.MILLISECONDS)++ // ---- Frame builders ----++ private fun header(msgType: Int, flags: Int): ByteArray = byteArrayOf(+ ((PROTOCOL_VERSION_V1 shl 4) or 1).toByte(),+ ((msgType shl 4) or flags).toByte(),+ ((SERIALIZATION_JSON shl 4) or COMPRESSION_GZIP).toByte(),+ 0x00,+ )++ private fun buildFullClientRequest(seqNum: Int, payloadJson: String): ByteArray {+ val body = gzip(payloadJson.toByteArray(Charsets.UTF_8))+ val out = ByteArrayOutputStream()+ out.write(header(MSG_CLIENT_FULL_REQUEST, FLAG_POS_SEQUENCE))+ out.write(ByteBuffer.allocate(4).putInt(seqNum).array())+ out.write(ByteBuffer.allocate(4).putInt(body.size).array())+ out.write(body)+ return out.toByteArray()+ }++ private fun buildAudioRequest(seqNum: Int, pcm: ByteArray, isLast: Boolean): ByteArray {+ val actualSeq = if (isLast) -seqNum else seqNum+ val flags = if (isLast) FLAG_NEG_WITH_SEQUENCE else FLAG_POS_SEQUENCE+ val body = gzip(pcm)+ val out = ByteArrayOutputStream()+ out.write(header(MSG_CLIENT_AUDIO_ONLY_REQUEST, flags))+ out.write(ByteBuffer.allocate(4).putInt(actualSeq).array())+ out.write(ByteBuffer.allocate(4).putInt(body.size).array())+ out.write(body)+ return out.toByteArray()+ }++ private fun gzip(data: ByteArray): ByteArray {+ val bos = ByteArrayOutputStream()+ GZIPOutputStream(bos).use { it.write(data) }+ return bos.toByteArray()+ }++ private fun ungzip(data: ByteArray): ByteArray {+ return GZIPInputStream(data.inputStream()).use { it.readBytes() }+ }++ private fun intAtBE(b: ByteArray, o: Int): Int =+ ((b[o].toInt() and 0xff) shl 24) or+ ((b[o + 1].toInt() and 0xff) shl 16) or+ ((b[o + 2].toInt() and 0xff) shl 8) or+ (b[o + 3].toInt() and 0xff)++ companion object {+ private const val TAG = "HuoshanAsr"+ private const val PROTOCOL_VERSION_V1 = 0b0001+ private const val MSG_CLIENT_FULL_REQUEST = 0b0001+ private const val MSG_CLIENT_AUDIO_ONLY_REQUEST = 0b0010+ private const val MSG_SERVER_FULL_RESPONSE = 0b1001+ private const val MSG_SERVER_ERROR_RESPONSE = 0b1111+ private const val FLAG_POS_SEQUENCE = 0b0001+ private const val FLAG_NEG_WITH_SEQUENCE = 0b0011+ private const val SERIALIZATION_JSON = 0b0001+ private const val COMPRESSION_GZIP = 0b0001+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/audio/AudioFocusGate.kt b/app/src/main/java/net/kuafuai/andee/audio/AudioFocusGate.ktnew file mode 100644index 0000000..b2582b5--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/audio/AudioFocusGate.kt@@ -0,0 +1,112 @@+package net.kuafuai.andee.audio++import android.content.Context+import android.media.AudioAttributes+import android.media.AudioFocusRequest+import android.media.AudioManager+import android.util.Log++/**+ * Ask the rest of the device to be quiet for the length of a voice turn, and+ * shut up ourselves when something more important asks us to.+ *+ * Without this the tablet's own video or music player just keeps going: our TTS+ * mixes on top of it, and — worse — the speaker output walks straight back into+ * the microphone and wrecks recognition. Android has exactly one mechanism for+ * "I need the audio device for a moment", and neither half of the voice pipeline+ * was using it.+ *+ * One gate per voice direction, because the two want different things from the+ * other app; see [gain] at the call sites. Both are transient — the media app+ * resumes on its own when we release, which is the whole reason to use focus+ * rather than reaching for [android.media.AudioManager.adjustStreamVolume].+ *+ * Thread-safe in the only way that matters: [acquire] and [release] may be+ * called from any thread and [release] is idempotent, because a focus loss and+ * the controller's own teardown routinely race.+ *+ * @param usage what we sound like to the audio policy — `USAGE_ASSISTANT` for+ * both directions, which is what this device is.+ * @param gain `AUDIOFOCUS_GAIN_TRANSIENT` to speak over a paused media app, or+ * `AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE` to listen. The exclusive form is not+ * politeness: the plain one lets a well-behaved player duck instead of stop,+ * and ducked music is still coming out of the speaker and into the mic.+ */+class AudioFocusGate(+ context: Context,+ usage: Int,+ gain: Int,+) {+ private val am = context.applicationContext+ .getSystemService(Context.AUDIO_SERVICE) as AudioManager++ /** Set by [acquire], cleared by whichever of loss/[release] happens first. */+ @Volatile+ private var onLost: (() -> Unit)? = null++ @Volatile+ private var held = false++ private val listener = AudioManager.OnAudioFocusChangeListener { change ->+ when (change) {+ // All three are "stop": a phone call (LOSS_TRANSIENT), something+ // taking over for good (LOSS), or a notification that would rather+ // we ducked (CAN_DUCK). Ducking a voice turn is pointless — half an+ // answer at a quarter volume is not an answer — so we treat it as+ // the end of the turn too, which is what setWillPauseWhenDucked+ // below asks the system to let us decide.+ AudioManager.AUDIOFOCUS_LOSS,+ AudioManager.AUDIOFOCUS_LOSS_TRANSIENT,+ AudioManager.AUDIOFOCUS_LOSS_TRANSIENT_CAN_DUCK -> {+ val cb = onLost+ onLost = null+ held = false+ Log.i(TAG, "focus lost ($change)")+ runCatching { cb?.invoke() }+ }+ }+ }++ private val request: AudioFocusRequest = AudioFocusRequest.Builder(gain)+ .setAudioAttributes(+ AudioAttributes.Builder()+ .setUsage(usage)+ .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)+ .build(),+ )+ .setOnAudioFocusChangeListener(listener)+ .setWillPauseWhenDucked(true)+ .build()++ /**+ * Take the floor. Returns false if the system refused, in which case the+ * caller should carry on anyway — a refused request means something like an+ * in-progress phone call, and failing the user's turn outright is worse than+ * talking over whatever it was.+ *+ * @param onLost invoked, on a binder thread, when the floor is taken back+ * mid-turn. Fires at most once per [acquire].+ */+ fun acquire(onLost: () -> Unit): Boolean {+ this.onLost = onLost+ val ok = am.requestAudioFocus(request) == AudioManager.AUDIOFOCUS_REQUEST_GRANTED+ held = ok+ if (!ok) {+ this.onLost = null+ Log.w(TAG, "focus request denied")+ }+ return ok+ }++ /** Give the floor back so the media app can resume. Safe to call twice. */+ fun release() {+ onLost = null+ if (!held) return+ held = false+ runCatching { am.abandonAudioFocusRequest(request) }+ }++ private companion object {+ const val TAG = "AudioFocus"+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/audio/AudioIO.kt b/app/src/main/java/net/kuafuai/andee/audio/AudioIO.ktnew file mode 100644index 0000000..4a85e91--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/audio/AudioIO.kt@@ -0,0 +1,440 @@+package net.kuafuai.andee.audio++import android.media.AudioAttributes+import android.media.AudioFormat+import android.media.AudioManager+import android.media.AudioRecord+import android.media.AudioTrack+import android.media.MediaRecorder+import java.io.ByteArrayOutputStream+import java.util.concurrent.atomic.AtomicBoolean++/**+ * Audio in/out primitives for the physical body.+ *+ * Recorder produces raw PCM 16-bit little-endian (mono) at a configurable sample rate.+ * Player consumes the same format.+ *+ * Design notes:+ * - Single active session at a time (one recording, one playback).+ * - Recording is chunked but for MVP we accumulate into a ByteArrayOutputStream and+ * return the full buffer on stop(). Streaming to WS is a later enhancement.+ * - Playback in this MVP is synchronous ("write all bytes and wait to finish"),+ * which is fine because it runs on the WS handler thread, not the main UI thread.+ */+class AudioIO {++ // ---- Recorder ----++ private var recorder: AudioRecord? = null+ private var recordThread: Thread? = null+ private val recording = AtomicBoolean(false)+ private val buffer = ByteArrayOutputStream()+ private var currentSampleRate = 16_000++ fun startRecording(sampleRate: Int = 16_000): Int {+ if (recording.get()) throw IllegalStateException("already recording")+ val channelConfig = AudioFormat.CHANNEL_IN_MONO+ val encoding = AudioFormat.ENCODING_PCM_16BIT+ val minBuf = AudioRecord.getMinBufferSize(sampleRate, channelConfig, encoding)+ require(minBuf > 0) { "AudioRecord.getMinBufferSize failed: $minBuf" }+ val bufSize = minBuf * 2+ // VOICE_RECOGNITION applies noise suppression + AGC tuned for ASR.+ // Compared to MIC (raw), it removes most of the "sha-sha" hiss on voice.+ @Suppress("MissingPermission")+ val rec = AudioRecord(+ MediaRecorder.AudioSource.VOICE_RECOGNITION,+ sampleRate,+ channelConfig,+ encoding,+ bufSize,+ )+ if (rec.state != AudioRecord.STATE_INITIALIZED) {+ rec.release()+ throw IllegalStateException("AudioRecord not initialized (mic permission?)")+ }+ buffer.reset()+ currentSampleRate = sampleRate+ rec.startRecording()+ recorder = rec+ recording.set(true)+ val chunk = ByteArray(bufSize)+ recordThread = Thread {+ while (recording.get()) {+ val n = rec.read(chunk, 0, chunk.size)+ if (n > 0) synchronized(buffer) { buffer.write(chunk, 0, n) }+ else if (n < 0) break+ }+ }.also { it.name = "AudioIO-Recorder"; it.isDaemon = true; it.start() }+ return sampleRate+ }++ /** Stop recording and return the accumulated PCM bytes. */+ fun stopRecording(): Pair<ByteArray, Int> {+ if (!recording.get()) throw IllegalStateException("not recording")+ recording.set(false)+ recordThread?.join(1000)+ recordThread = null+ val rec = recorder+ recorder = null+ try {+ rec?.stop()+ } catch (_: Throwable) {+ }+ rec?.release()+ val bytes: ByteArray = synchronized(buffer) { buffer.toByteArray() }+ return bytes to currentSampleRate+ }++ fun isRecording(): Boolean = recording.get()++ // ---- Streaming record (for ASR) ----++ private var streamRecorder: AudioRecord? = null+ private var streamThread: Thread? = null+ private val streaming = AtomicBoolean(false)++ /**+ * Start reading mic in real-time chunks. Each chunk (~chunkMs of audio)+ * fires `onChunk(bytes)`. Does not accumulate; caller decides what to do+ * (usually: forward to ASR).+ *+ * ## No AcousticEchoCanceler / NoiseSuppressor, deliberately+ *+ * Both used to be attached to this session as a second line of defence+ * behind the caller's audio focus, for media apps that duck instead of+ * stopping. Attaching either stalls the audio HAL on a Xiaomi Pad 5:+ * measured on device, the first `read()` came back after 10.2 s with the+ * noise suppressor alone and 20.4 s with both, against 159 ms with+ * neither.+ *+ * Ten seconds of dead microphone is not a degradation, it is the whole+ * turn. 火山 ends a session that has gone eight seconds without a packet,+ * so the user taps the ball, says their piece into nothing, and gets back+ * a server-side timeout — a message about the network, which was never+ * the problem. That failure is worth far more than the effects were: the+ * VOICE_RECOGNITION source already runs the platform's own noise+ * suppression and AGC, tuned for exactly this.+ */+ fun startStreaming(+ sampleRate: Int = 16_000,+ chunkMs: Int = 100,+ onChunk: (ByteArray) -> Unit,+ ) {+ if (streaming.get() || recording.get()) throw IllegalStateException("audio busy")+ val channelConfig = AudioFormat.CHANNEL_IN_MONO+ val encoding = AudioFormat.ENCODING_PCM_16BIT+ val chunkBytes = sampleRate * 2 /* bytes/sample */ * chunkMs / 1000+ val minBuf = AudioRecord.getMinBufferSize(sampleRate, channelConfig, encoding)+ require(minBuf > 0) { "getMinBufferSize failed: $minBuf" }+ val bufSize = maxOf(minBuf * 2, chunkBytes * 4)++ @Suppress("MissingPermission")+ val rec = AudioRecord(+ MediaRecorder.AudioSource.VOICE_RECOGNITION,+ sampleRate,+ channelConfig,+ encoding,+ bufSize,+ )+ if (rec.state != AudioRecord.STATE_INITIALIZED) {+ rec.release()+ throw IllegalStateException("AudioRecord not initialized (mic permission?)")+ }+ streamRecorder = rec+ streaming.set(true)+ rec.startRecording()+ val startedAt = android.os.SystemClock.uptimeMillis()+ streamThread = Thread {+ val buf = ByteArray(chunkBytes)+ var chunks = 0L+ var why = "stopped"+ try {+ while (streaming.get()) {+ var off = 0+ while (off < buf.size && streaming.get()) {+ val n = rec.read(buf, off, buf.size - off)+ // A recorder that hands back nothing is the one failure+ // mode with no symptom of its own: the loop just ends,+ // the websocket sits there with an open session and no+ // packets, and eight seconds later 火山 kills it for+ // inactivity — which surfaces to the user as "ASR+ // 连接超时", pointing at the network, which is fine.+ if (n <= 0) {+ why = "read=$n after $chunks chunks"+ return@Thread+ }+ off += n+ }+ if (off > 0) {+ // How long the first buffer took is the single number+ // that separates "the mic is fine" from "this device+ // hands us nothing": the symptom either way is a+ // recognizer that times out, and the timeout is+ // reported by the server, which makes it look like the+ // network.+ if (chunks == 0L) {+ android.util.Log.i(+ TAG,+ "first chunk after ${android.os.SystemClock.uptimeMillis() - startedAt}ms",+ )+ }+ chunks+++ onChunk(buf.copyOf(off))+ }+ }+ } catch (t: Throwable) {+ why = "threw ${t.javaClass.simpleName}: ${t.message} after $chunks chunks"+ } finally {+ android.util.Log.i(TAG, "streamer ended: $why")+ }+ }.also { it.name = "AudioIO-Streamer"; it.isDaemon = true; it.start() }+ }++ fun stopStreaming() {+ if (!streaming.get()) return+ streaming.set(false)+ streamThread?.join(1000)+ streamThread = null+ val rec = streamRecorder+ streamRecorder = null+ try {+ rec?.stop()+ } catch (_: Throwable) {+ }+ rec?.release()+ }++ fun isStreaming(): Boolean = streaming.get()++ // ---- Player ----++ private var track: AudioTrack? = null++ /**+ * Play a raw PCM buffer synchronously. Blocks until playback completes,+ * or ~ (pcm bytes / sample bytes) / sampleRate seconds.+ */+ fun playPcm(pcm: ByteArray, sampleRate: Int = 16_000) {+ if (pcm.isEmpty()) return+ stopPlayback()+ val channelConfig = AudioFormat.CHANNEL_OUT_MONO+ val encoding = AudioFormat.ENCODING_PCM_16BIT+ val minBuf = AudioTrack.getMinBufferSize(sampleRate, channelConfig, encoding)+ require(minBuf > 0) { "AudioTrack.getMinBufferSize failed: $minBuf" }+ val attrs = AudioAttributes.Builder()+ .setUsage(AudioAttributes.USAGE_MEDIA)+ .setContentType(AudioAttributes.CONTENT_TYPE_MUSIC)+ .build()+ val format = AudioFormat.Builder()+ .setEncoding(encoding)+ .setSampleRate(sampleRate)+ .setChannelMask(channelConfig)+ .build()+ val t = AudioTrack.Builder()+ .setAudioAttributes(attrs)+ .setAudioFormat(format)+ .setBufferSizeInBytes(maxOf(minBuf, pcm.size))+ .setTransferMode(AudioTrack.MODE_STREAM)+ .build()+ track = t+ t.play()+ var offset = 0+ while (offset < pcm.size) {+ val n = t.write(pcm, offset, pcm.size - offset)+ if (n <= 0) break+ offset += n+ }+ // Wait roughly until playback drains.+ val bytesPerFrame = 2+ val durationMs = pcm.size.toLong() / bytesPerFrame * 1000L / sampleRate+ Thread.sleep(durationMs + 200)+ stopPlayback()+ }++ fun stopPlayback() {+ val t = track ?: return+ try {+ t.stop()+ } catch (_: Throwable) {+ }+ try {+ t.release()+ } catch (_: Throwable) {+ }+ track = null+ }++ fun release() {+ if (recording.get()) runCatching { stopRecording() }+ stopStreaming()+ stopPlayback()+ stopStreamPlayback()+ }++ // ---- Streaming playback (for TTS) ----++ // Written by whoever starts/stops playback, read by the player thread.+ // Barge-in stops playback from a third thread, so the player has to see+ // the null promptly.+ @Volatile+ private var streamTrack: AudioTrack? = null+ private var streamPlaySampleRate: Int = 24_000++ /** Player thread only, after [startStreamPlayback] zeroes it. */+ @Volatile+ private var streamBytesWritten: Long = 0++ /**+ * PCM waiting to be written to [streamTrack], and the thread that writes it.+ *+ * The hand-off is the whole point. `AudioTrack.write` on a MODE_STREAM track+ * blocks until the buffer has room, and the buffer holds well under a second+ * — while 火山 pushes a whole utterance's audio far faster than real time. Fed+ * directly, the websocket's reader thread therefore sat inside `write` for+ * essentially the entire utterance, which meant it never processed a pong+ * either, and Java-WebSocket's own lost-connection detector closed the socket+ * after ~120 s. Long answers stopped mid-sentence with no error anywhere; short+ * ones finished before the timer noticed, so it only ever looked like a bug in+ * long answers.+ *+ * Unbounded on purpose: any bound is a blocking producer again, which is the+ * thing being fixed. At 24 kHz mono 16-bit the backlog grows ~48 KB per second+ * of speech, and what's queued is one answer being read aloud.+ */+ private val playQueue = java.util.concurrent.LinkedBlockingQueue<ByteArray>()+ private var playThread: Thread? = null++ /** Producer's view of "there is somewhere to put audio". */+ @Volatile+ private var playing = false++ /** Queued by [finishStreamPlayback] to mean "no more audio is coming". */+ private val endOfStream = ByteArray(0)++ fun startStreamPlayback(sampleRate: Int = 24_000) {+ stopStreamPlayback()+ val channelConfig = AudioFormat.CHANNEL_OUT_MONO+ val encoding = AudioFormat.ENCODING_PCM_16BIT+ val minBuf = AudioTrack.getMinBufferSize(sampleRate, channelConfig, encoding)+ require(minBuf > 0) { "AudioTrack.getMinBufferSize failed: $minBuf" }+ // USAGE_ASSISTANT, not USAGE_MEDIA: this is the device answering, not+ // the device playing something. Declaring it as media put our own voice+ // in the same bucket as whatever video the user has open, which is both+ // a lie to the audio policy and the wrong thing for the volume keys to+ // land on.+ val attrs = AudioAttributes.Builder()+ .setUsage(AudioAttributes.USAGE_ASSISTANT)+ .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)+ .build()+ val format = AudioFormat.Builder()+ .setEncoding(encoding)+ .setSampleRate(sampleRate)+ .setChannelMask(channelConfig)+ .build()+ val t = AudioTrack.Builder()+ .setAudioAttributes(attrs)+ .setAudioFormat(format)+ .setBufferSizeInBytes(minBuf * 4)+ .setTransferMode(AudioTrack.MODE_STREAM)+ .build()+ streamPlaySampleRate = sampleRate+ streamBytesWritten = 0+ playQueue.clear()+ t.play()+ streamTrack = t+ playing = true+ playThread = Thread {+ try {+ while (playing) {+ val chunk = playQueue.take()+ if (chunk === endOfStream) break+ val track = streamTrack ?: break+ var off = 0+ while (off < chunk.size) {+ val n = track.write(chunk, off, chunk.size - off)+ if (n <= 0) break+ off += n+ }+ streamBytesWritten += off+ }+ } catch (_: InterruptedException) {+ // Barge-in. Dropping the tail of an interrupted utterance is the+ // desired outcome.+ } catch (_: IllegalStateException) {+ // stopStreamPlayback released the track between the null check+ // and the write.+ }+ }.also { it.name = "AudioIO-Player"; it.isDaemon = true; it.start() }+ }++ /**+ * Hand a PCM chunk to the player. Never blocks — see [playQueue].+ *+ * Takes ownership of [pcm]; callers pass a fresh array per chunk.+ */+ fun feedPcm(pcm: ByteArray) {+ if (!playing || pcm.isEmpty()) return+ playQueue.offer(pcm)+ }++ /**+ * Called when the producer is done. Waits for the backlog to reach the track+ * and for the track to drain, then releases.+ *+ * Blocks for the full remaining length of the utterance, which is now most of+ * it rather than the tail — the audio arrives from 火山 far faster than it+ * plays. Callers must not be holding anything that a barge-in needs; see+ * [TtsController.stopSpeaking].+ */+ fun finishStreamPlayback() {+ val th = playThread ?: return+ playQueue.offer(endOfStream)+ runCatching { th.join() }+ val t = streamTrack ?: return+ val bytesPerFrame = 2 // 16-bit mono+ val totalFrames = streamBytesWritten / bytesPerFrame+ val playedFrames = t.playbackHeadPosition.toLong() and 0xFFFFFFFFL+ val remaining = (totalFrames - playedFrames).coerceAtLeast(0)+ val remainingMs = remaining * 1000L / streamPlaySampleRate+ try {+ Thread.sleep(remainingMs + 200)+ } catch (_: InterruptedException) {+ }+ stopStreamPlayback()+ }++ fun stopStreamPlayback() {+ playing = false+ val th = playThread+ playThread = null+ playQueue.clear()+ // Null the track before touching it: the player thread re-reads it every+ // chunk and must not start a write into one we are about to release.+ val t = streamTrack+ streamTrack = null+ th?.interrupt()+ if (t == null) return+ try {+ t.pause()+ } catch (_: Throwable) {+ }+ try {+ t.flush()+ } catch (_: Throwable) {+ }+ try {+ t.stop()+ } catch (_: Throwable) {+ }+ try {+ t.release()+ } catch (_: Throwable) {+ }+ streamBytesWritten = 0+ }++ private companion object {+ const val TAG = "AudioIO"+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/audio/Earcon.kt b/app/src/main/java/net/kuafuai/andee/audio/Earcon.ktnew file mode 100644index 0000000..2e8201a--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/audio/Earcon.kt@@ -0,0 +1,190 @@+package net.kuafuai.andee.audio++import android.media.AudioAttributes+import android.media.AudioFormat+import android.media.AudioTrack+import android.os.SystemClock+import java.util.concurrent.Executors+import kotlin.math.PI+import kotlin.math.cos+import kotlin.math.exp+import kotlin.math.sin++/**+ * Two short tones: the microphone just opened, and the answer is being worked on.+ *+ * These are the audible half of the ball's face. The face only works when the+ * user happens to be looking at it, and the two moments that most need an+ * acknowledgement are exactly the moments they are not: after saying "嘿+ * Andee" they are waiting to hear whether to start talking, and after+ * finishing a sentence they are waiting to hear whether it landed.+ *+ * ## Synthesised, not shipped as assets+ *+ * A pair of .ogg files would be less code, but every parameter below is a+ * dial that wants turning on a real device, and a re-exported asset is not a+ * diff anyone can read. It is also about 40 ms of arithmetic once per process.+ *+ * ## Why they sound the way they do+ *+ * Nothing here is decoration. A sine with a fast-but-not-instant attack and an+ * exponential tail is what a struck object does, and the ear reads it as an+ * object rather than as a device beeping: a square wave or a hard-edged+ * envelope is the same information and sounds like a smoke alarm. The second+ * harmonic decays faster than the fundamental for the same reason — that is+ * the order real materials lose their partials in. The detuned copy a few+ * cents away beats slowly against the fundamental, which is the whole of the+ * "glass" quality.+ *+ * Direction carries the meaning. Opening the microphone rises (a perfect+ * fifth, the most consonant interval that still clearly moves); starting to+ * think falls, quieter and lower, because it is an acknowledgement rather+ * than an invitation and must not sound like a second prompt to speak.+ *+ * Levels are deliberately low. These play an inch from a microphone that is+ * about to open, and under a voice that is about to answer.+ */+object Earcon {++ private const val RATE = 24_000++ /** Peak of the loudest earcon, as a fraction of full scale. */+ private const val PEAK = 0.20++ /** Long enough that no edge clicks, short enough to still read as a strike. */+ private const val ATTACK_MS = 7++ /**+ * How long after a tone the wake listener must stay shut.+ *+ * Our own chime is speech-shaped enough for [net.kuafuai.andee.wake.VoiceSegmenter]+ * to cut it out as an utterance, and a tone run through CMVN lands at an+ * unpredictable distance from the templates — so the listener could+ * plausibly wake itself on the sound of having just woken up. Covering the+ * tone plus the room's own tail is cheaper than teaching the matcher to+ * recognise us.+ */+ private const val TAIL_MS = 250L++ /** Anything closer together than this is a state machine flapping, not a signal. */+ private const val MIN_GAP_MS = 150L++ private val exec = Executors.newSingleThreadExecutor { r ->+ Thread(r, "Earcon").apply { isDaemon = true }+ }++ @Volatile+ private var quietUntil = 0L++ @Volatile+ private var lastPlayAt = 0L++ /** Rising fifth — the microphone is open, start talking. */+ private val listenPcm by lazy {+ render(listOf(Note(880.0, 0, 420, 1.0), Note(1318.5, 85, 470, 0.9)))+ }++ /** Falling fourth, quieter — heard you, working on it. */+ private val thinkPcm by lazy {+ render(listOf(Note(587.33, 0, 460, 0.55), Note(440.0, 110, 560, 0.5)))+ }++ fun listening() = play(listenPcm)++ fun thinking() = play(thinkPcm)++ /** True while a tone is audible, plus [TAIL_MS]. Read by the wake-word gate. */+ fun isSounding(): Boolean = SystemClock.uptimeMillis() < quietUntil++ // ---- playback ----++ private fun play(pcm: ByteArray) {+ val now = SystemClock.uptimeMillis()+ if (now - lastPlayAt < MIN_GAP_MS) return+ lastPlayAt = now+ val ms = pcm.size.toLong() / 2 * 1000 / RATE+ quietUntil = now + ms + TAIL_MS+ exec.execute {+ runCatching {+ // USAGE_ASSISTANT, matching TtsController: this is the same+ // entity making the same kind of noise, so it belongs under the+ // same volume key. ASSISTANCE_SONIFICATION would be the+ // textbook answer and routes to the system stream, where a+ // silenced tablet would drop the chime while still speaking+ // full sentences out loud.+ val track = AudioTrack.Builder()+ .setAudioAttributes(+ AudioAttributes.Builder()+ .setUsage(AudioAttributes.USAGE_ASSISTANT)+ .setContentType(AudioAttributes.CONTENT_TYPE_SONIFICATION)+ .build()+ )+ .setAudioFormat(+ AudioFormat.Builder()+ .setEncoding(AudioFormat.ENCODING_PCM_16BIT)+ .setSampleRate(RATE)+ .setChannelMask(AudioFormat.CHANNEL_OUT_MONO)+ .build()+ )+ .setBufferSizeInBytes(pcm.size)+ .setTransferMode(AudioTrack.MODE_STATIC)+ .build()+ track.write(pcm, 0, pcm.size)+ track.play()+ Thread.sleep(ms + 80)+ runCatching { track.stop() }+ track.release()+ }+ }+ }++ // ---- synthesis ----++ private class Note(+ val hz: Double,+ val atMs: Int,+ val lenMs: Int,+ val gain: Double,+ )++ private fun render(notes: List<Note>): ByteArray {+ val totalMs = notes.maxOf { it.atMs + it.lenMs }+ val total = RATE * totalMs / 1000+ val buf = DoubleArray(total)+ val attack = RATE * ATTACK_MS / 1000++ for (note in notes) {+ val start = RATE * note.atMs / 1000+ val len = RATE * note.lenMs / 1000+ // exp(-4.5) ≈ 0.011 at the end of the note, which is below the+ // threshold where truncating it is audible as a click.+ val tau = len / 4.5+ for (i in 0 until len) {+ val at = start + i+ if (at >= total) break+ val t = i.toDouble() / RATE+ val decay = exp(-i / tau)+ val rise = if (i < attack) 0.5 - 0.5 * cos(PI * i / attack) else 1.0+ var s = sin(2 * PI * note.hz * t)+ s += 0.5 * sin(2 * PI * note.hz * 1.004 * t)+ s += 0.12 * sin(4 * PI * note.hz * t) * exp(-i / (tau * 0.4))+ buf[at] += s * rise * decay * note.gain+ }+ }++ // One normalisation pass rather than hand-balanced gains: the partials+ // and the overlapping tails sum to something no one can predict, and+ // the note gains above are meant to read as relative loudness.+ var peak = 0.0+ for (v in buf) if (v > peak) peak = v else if (-v > peak) peak = -v+ val scale = if (peak > 0) PEAK * Short.MAX_VALUE / peak else 0.0++ val out = ByteArray(total * 2)+ for (i in 0 until total) {+ val v = (buf[i] * scale).toInt().coerceIn(-32768, 32767)+ out[i * 2] = (v and 0xFF).toByte()+ out[i * 2 + 1] = ((v shr 8) and 0xFF).toByte()+ }+ return out+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/brain/LlmClient.kt b/app/src/main/java/net/kuafuai/andee/brain/LlmClient.ktnew file mode 100644index 0000000..6f79dc7--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/brain/LlmClient.kt@@ -0,0 +1,248 @@+package net.kuafuai.andee.brain++import android.util.Log+import okhttp3.MediaType.Companion.toMediaType+import okhttp3.OkHttpClient+import okhttp3.Request+import okhttp3.RequestBody.Companion.toRequestBody+import org.json.JSONArray+import org.json.JSONObject+import java.util.concurrent.TimeUnit++/**+ * One blocking call to an OpenAI-compatible `/chat/completions`, aimed at+ * DeepSeek.+ *+ * Deliberately thin and deliberately **not** streaming. Streaming would let TTS+ * start talking a second or two earlier, but it costs an SSE parser plus+ * incremental `tool_calls` reassembly (arguments arrive as string fragments+ * across deltas), and none of that is needed to make the local brain work.+ *+ * Two shapes in here are easy to get wrong, and both are silent when wrong:+ *+ * * **`tool_calls[].function.arguments` is a JSON *string*, not an object.**+ * It has to be re-parsed, and a model that emits malformed JSON must not+ * take the turn down — see [ToolCall.argumentsOrNull].+ * * **`reasoning_content` must never be sent back.** DeepSeek rejects requests+ * whose assistant messages carry it, so [Reply.reasoning] exists to be+ * logged and thrown away, never to be appended to the history.+ *+ * Threading: [complete] blocks the calling thread. It is only ever called from+ * [LocalBrain]'s single loop thread.+ */+class LlmClient(+ private val baseUrl: String,+ private val apiKey: String,+ private val model: String,+ private val thinking: Boolean,+ private val reasoningEffort: String,+) {++ /**+ * Read timeout is generous because a thinking model answers in minutes, not+ * seconds, and a premature timeout here reads to the user as the ball going+ * quiet forever. Connect timeout stays short: a tablet that has lost its+ * network should say so quickly.+ */+ private val http = OkHttpClient.Builder()+ .connectTimeout(15, TimeUnit.SECONDS)+ .readTimeout(180, TimeUnit.SECONDS)+ .writeTimeout(30, TimeUnit.SECONDS)+ .build()++ /** The in-flight call, so [cancel] can cut a turn short. */+ @Volatile+ private var current: okhttp3.Call? = null++ data class ToolCall(+ val id: String,+ val name: String,+ /** Raw JSON text as the model wrote it. */+ val argumentsRaw: String,+ ) {+ /**+ * Null when the model wrote something that isn't a JSON object. The+ * caller turns that into a tool-error message rather than an exception:+ * a model that miswrote its arguments can fix them on the next+ * iteration, and killing the turn denies it the chance.+ */+ fun argumentsOrNull(): JSONObject? =+ if (argumentsRaw.isBlank()) JSONObject()+ else runCatching { JSONObject(argumentsRaw) }.getOrNull()++ /** Identity for the repeated-call guard in [LocalBrain]. */+ fun signature(): String = "$name/$argumentsRaw"+ }++ data class Reply(+ /** Assistant prose. Narration when [toolCalls] is non-empty, otherwise the answer. */+ val content: String,+ /** Logged only — see the class KDoc. */+ val reasoning: String,+ val toolCalls: List<ToolCall>,+ /**+ * The assistant message exactly as it must be echoed back into the+ * history. Kept verbatim from the response rather than rebuilt, minus+ * `reasoning_content`, so no field the API cares about gets lost in+ * translation.+ */+ val assistantMessage: JSONObject,+ val promptTokens: Int,+ val completionTokens: Int,+ val cachedTokens: Int,+ )++ /**+ * @param messages the full conversation, system message first.+ * @param tools OpenAI-format function specs — see [toolSpecs].+ */+ fun complete(messages: List<JSONObject>, tools: JSONArray): Reply {+ val msgArr = JSONArray()+ for (m in messages) msgArr.put(m)+ val body = JSONObject()+ .put("model", model)+ .put("messages", msgArr)+ .put("stream", false)+ if (tools.length() > 0) {+ body.put("tools", tools).put("tool_choice", "auto")+ }+ if (thinking) {+ body.put("thinking", JSONObject().put("type", "enabled"))+ body.put("reasoning_effort", reasoningEffort)+ } else {+ body.put("thinking", JSONObject().put("type", "disabled"))+ }++ val req = Request.Builder()+ .url("$baseUrl/chat/completions")+ .addHeader("Authorization", "Bearer $apiKey")+ .addHeader("Content-Type", "application/json")+ .post(body.toString().toRequestBody(JSON))+ .build()++ val call = http.newCall(req)+ current = call+ val text = try {+ call.execute().use { resp ->+ // Read the body either way: DeepSeek puts the useful part of an+ // error (bad key, unknown model, context overflow) in the body,+ // and a bare "HTTP 400" sends whoever is debugging to the wrong+ // place entirely.+ val raw = resp.body?.string().orEmpty()+ if (!resp.isSuccessful) {+ throw IllegalStateException("LLM HTTP ${resp.code}: ${raw.take(600)}")+ }+ raw+ }+ } finally {+ current = null+ }++ val root = JSONObject(text)+ val choice = root.optJSONArray("choices")?.optJSONObject(0)+ ?: throw IllegalStateException("LLM reply has no choices: ${text.take(400)}")+ val msg = choice.optJSONObject("message")+ ?: throw IllegalStateException("LLM choice has no message: ${text.take(400)}")++ val calls = mutableListOf<ToolCall>()+ msg.optJSONArray("tool_calls")?.let { arr ->+ for (i in 0 until arr.length()) {+ val tc = arr.optJSONObject(i) ?: continue+ val fn = tc.optJSONObject("function") ?: continue+ calls += ToolCall(+ // A gateway that omits the id would otherwise produce tool+ // messages the API can't match to their call. Index is a+ // stable stand-in within one reply, which is the only scope+ // that matters.+ id = tc.optString("id").ifEmpty { "call_$i" },+ name = fn.optString("name"),+ argumentsRaw = fn.optString("arguments"),+ )+ }+ }++ val usage = root.optJSONObject("usage")+ return Reply(+ content = msg.optString("content").orEmpty(),+ reasoning = msg.optString("reasoning_content").orEmpty(),+ toolCalls = calls,+ assistantMessage = sanitizeAssistant(msg),+ promptTokens = usage?.optInt("prompt_tokens") ?: 0,+ completionTokens = usage?.optInt("completion_tokens") ?: 0,+ cachedTokens = usage?.optInt("prompt_cache_hit_tokens") ?: 0,+ )+ }++ /**+ * Abort whatever is in flight. Called from the main thread when the user+ * hits ■, so it must not block — [okhttp3.Call.cancel] doesn't.+ */+ fun cancel() {+ val c = current ?: return+ runCatching { c.cancel() }+ Log.i(TAG, "in-flight completion cancelled")+ }++ private fun sanitizeAssistant(msg: JSONObject): JSONObject {+ val out = JSONObject()+ out.put("role", "assistant")+ // JSONObject.NULL rather than omitting the key: the API expects+ // `content` to be present on an assistant message even when the whole+ // turn was tool calls.+ val content = msg.optString("content").orEmpty()+ out.put("content", if (content.isEmpty()) JSONObject.NULL else content)+ msg.optJSONArray("tool_calls")?.let { if (it.length() > 0) out.put("tool_calls", it) }+ return out+ }++ companion object {+ private const val TAG = "Brain"+ private val JSON = "application/json; charset=utf-8".toMediaType()++ /**+ * Turn the body's own schema list into OpenAI function specs.+ *+ * [net.kuafuai.andee.net.ToolSchemas] already emits+ * `{name, description, parameters}` with OpenAI's JSON Schema dialect —+ * the only differences are the wrapper and the `method` field, which is+ * this device's internal routing key and means nothing to the model.+ */+ fun toolSpecs(vararg schemaLists: JSONArray): JSONArray {+ val out = JSONArray()+ for (list in schemaLists) {+ for (i in 0 until list.length()) {+ val t = list.optJSONObject(i) ?: continue+ val fn = JSONObject()+ .put("name", t.optString("name"))+ .put("description", t.optString("description"))+ .put(+ "parameters",+ t.optJSONObject("parameters")+ ?: JSONObject().put("type", "object").put("properties", JSONObject()),+ )+ out.put(JSONObject().put("type", "function").put("function", fn))+ }+ }+ return out+ }++ /**+ * A vision content block carrying an inline image.+ *+ * The one shape in the local brain that is not verified against+ * anything in this repo — it is the OpenAI-compatible form, which is+ * what the rest of this endpoint follows. Isolated into one function on+ * purpose: if DeepSeek wants something else, this is the only edit.+ */+ fun imageBlock(base64: String, mime: String = "image/png"): JSONObject =+ JSONObject()+ .put("type", "image_url")+ .put(+ "image_url",+ JSONObject().put("url", "data:$mime;base64,$base64"),+ )++ fun textBlock(text: String): JSONObject =+ JSONObject().put("type", "text").put("text", text)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/brain/LocalBrain.kt b/app/src/main/java/net/kuafuai/andee/brain/LocalBrain.ktnew file mode 100644index 0000000..f87dc61--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/brain/LocalBrain.kt@@ -0,0 +1,1030 @@+package net.kuafuai.andee.brain++import android.util.Log+import net.kuafuai.andee.R+import net.kuafuai.andee.config.VoiceConfig+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.net.CallExtensionRegistry+import net.kuafuai.andee.net.CommandDispatcher+import net.kuafuai.andee.net.ToolSchemas+import org.json.JSONArray+import org.json.JSONObject+import java.text.SimpleDateFormat+import java.util.Date+import java.util.concurrent.ExecutionException+import java.util.concurrent.Executors+import java.util.concurrent.TimeUnit+import java.util.concurrent.TimeoutException+import java.util.concurrent.atomic.AtomicBoolean+import java.util.concurrent.atomic.AtomicInteger++/**+ * The agent loop, running on the tablet itself.+ *+ * This is the local counterpart of agentworld's `android-os` agent: it eats a+ * line of text, calls DeepSeek, executes the tool calls against this device's+ * own [CommandDispatcher], and hands back narration and a final answer. It+ * implements exactly the contract [net.kuafuai.andee.net.BodyWsClient] already+ * defined with its `onProgress` / `onFinal` lambdas, which is why everything+ * downstream — subtitles, ChatHistory, TTS, the edge glow, the fold, the stop+ * button — works unchanged. The two brains are mutually exclusive; see+ * `ScreenBodyService.startBrain`.+ *+ * Four things in here are not stylistic choices:+ *+ * * **Tool calls run one at a time.** The cloud uses `asyncio.gather`; copying+ * that would be a bug. `CommandDispatcher.passthroughForGesture` flips the+ * overlay non-touchable, sleeps ~50 ms, gestures, then restores — it is not+ * reentrant, and two gestures in flight would trample each other's window+ * state.+ * * **Every tool call has a deadline.** `ui.ask` blocks until it is answered+ * or times out; `ui.show_html` used to block until the user closed the page,+ * and that is gone — see [note] and `CommandDispatcher`'s handler.+ * Over the hub, `body_hub`'s 30 s request timeout quietly abandoned those;+ * locally there is no such layer, so [dispatchWithTimeout] supplies one. It+ * does **not** interrupt the work — same semantics as the hub giving up on a+ * request that is still running.+ * * **Screenshots are moved out of the tool result and rationed.** Every+ * successful tap comes back with a full-screen PNG (`after_shot`), so a+ * fifteen-step task carries fifteen images. See [attachImages] and+ * [trimImages].+ * * **The clock rides in on the user message, not the system prompt.** There+ * is no clock tool on this device, and the system prompt plus the 55 tool+ * specs are the cached prefix of every request — rewriting them each turn to+ * stamp the time would throw that cache away for one line of text.+ *+ * Threading: [submit] returns immediately; the whole turn runs on the single+ * `LocalBrain-loop` thread, and [history] is touched from nowhere else. Tool+ * dispatch happens on a separate pool only so the loop thread can time it out.+ */+class LocalBrain(+ private val cfg: VoiceConfig.Companion.BrainConfig,+ /**+ * The notebook's context. This class reads and writes memories on every+ * turn, so it needs one of its own rather than reaching through the+ * dispatcher, which deliberately knows nothing about the brain.+ */+ private val appContext: android.content.Context,+ private val dispatcher: CommandDispatcher,+ private val registry: CallExtensionRegistry,+ private val onProgress: (String) -> Unit,+ private val onFinal: (String) -> Unit,+ /**+ * A turn finished — whoever started it (user, a todo coming due, or a+ * silent one). The service uses this to re-arm the quiet-hour timer and to+ * start the sweep. Called on the loop thread; must not block.+ */+ private val onTurnEnd: () -> Unit = {},++ /**+ * What a quiet-hour sweep cost, in tokens, once it is over. The user pays+ * for these calls with their own key, so the settings screen shows the+ * running total — see `ScreenBodyService`. Only [TurnKind.SWEEP] is+ * reported: a harvest is not optional, so counting it against a daily+ * budget the user set for reviews would be misleading.+ */+ private val onSweepSpent: (Int) -> Unit = {},+) {++ private val llm = LlmClient(+ baseUrl = cfg.baseUrl,+ apiKey = cfg.apiKey,+ model = cfg.model,+ thinking = cfg.thinking,+ reasoningEffort = cfg.reasoningEffort,+ )++ private val loop = Executors.newSingleThreadExecutor { r ->+ Thread(r, "LocalBrain-loop").apply { isDaemon = true }+ }++ /**+ * Only here so the loop thread can walk away from a tool that overran. A+ * cached pool rather than a single thread because an abandoned call keeps+ * its thread — a `ui.ask` card the user never answers can sit there for+ * minutes after we have stopped waiting, and the next tool must not queue+ * behind it.+ */+ private val toolExec = Executors.newCachedThreadPool { r ->+ Thread(r, "LocalBrain-tool").apply { isDaemon = true }+ }++ /**+ * Bumped by [submit] and by [cancel]. A running turn compares it against+ * the value it started with, so "has this turn been abandoned" needs no+ * separate flag and a late [cancel] can never silence the turn that+ * replaced it.+ */+ private val generation = AtomicInteger(0)++ /**+ * A turn is in progress. Read from any thread, written on the loop thread.+ *+ * This is what makes a wake-up deferrable. [submit] bumps [generation],+ * which throws a running turn away mid-sentence — correct when the user+ * interrupts, wrong when a todo's alarm fires, because the user would hear+ * the answer to their own question vanish. So a wake that arrives mid-turn+ * waits in [pendingWakes] instead.+ */+ private val busy = AtomicBoolean(false)++ /** Loop thread only. Wake-ups that arrived while a turn was running. */+ private val pendingWakes = ArrayDeque<String>()++ /**+ * Why a turn is running — which decides two things: whether the user hears+ * it, and whether it is allowed to touch the device.+ */+ enum class TurnKind {+ /** The user just spoke. Speaks back; may drive the device. */+ USER,++ /** A todo came due. Speaks back; may drive the device. */+ WAKE,++ /** The quiet-hour review. Silent; notebook tools only. */+ SWEEP,++ /** The pre-trim review: same as [SWEEP], plus a summary that replaces messages. */+ HARVEST,+ ;++ val silent: Boolean get() = this == SWEEP || this == HARVEST+ }++ /**+ * Loop thread only. Set by [launch] for the duration of a turn and read by+ * [runTool]: every tool call of a turn shares it, and the dispatcher is the+ * thing that must not be fooled.+ */+ private var currentKind = TurnKind.USER++ /** Loop thread only. Guards against a harvest that cannot shrink anything. */+ private var lastHarvestAtSize = -1++ /** Loop thread only. Tokens spent by the turn currently running. */+ private var turnTokens = 0++ /**+ * Loop thread only. Where the current silent turn's own messages begin, or+ * -1. A harvest sets it before sending its prompt and throws everything+ * from there on away again once the summary has been read out — its own+ * scaffolding must not fossilise into the conversation.+ */+ private var internalFromIndex = -1++ /** Loop thread only. */+ private val history = mutableListOf<JSONObject>()++ /** Image messages currently in [history], oldest first. See [trimImages]. */+ private val imageMessages = mutableListOf<JSONObject>()++ fun submit(userText: String) {+ val text = userText.trim()+ if (text.isEmpty()) return+ if (cfg.apiKey.isEmpty()) {+ // Said out loud rather than logged: from the ball's point of view a+ // missing key is indistinguishable from a brain that never answers.+ onFinal(AppLocale.str(appContext, R.string.brain_no_key))+ return+ }+ // The quiet-hour timer measures silence from the user's last words, so+ // every real utterance stamps it. Wake-ups and sweeps do not: a sweep+ // must not look like the user spoke, or it would keep pushing its own+ // trigger out.+ net.kuafuai.andee.config.Notebook.touchActivity(appContext)+ launch(text, TurnKind.USER)+ }++ /**+ * Start a turn nobody just asked for: a todo came due.+ *+ * Deferred rather than interrupting. See [busy] — the alternative is+ * cancelling a turn the user is in the middle of, which is exactly the+ * thing [cancel] is for and the user did not press it.+ */+ fun wake(text: String) {+ val t = text.trim()+ if (t.isEmpty()) return+ loop.execute {+ if (busy.get()) {+ Log.i(TAG, "wake queued behind a running turn: ${t.take(60)}")+ pendingWakes.addLast(t)+ return@execute+ }+ launch(t, TurnKind.WAKE)+ }+ }++ /**+ * The quiet-hour review: look back over a conversation that has stopped,+ * and put what is worth keeping into the notebook.+ *+ * Silent by construction ([internal]) — no subtitle, no voice, and the+ * dispatcher refuses every method except `note.*`. Declines to run (and+ * says why in the log) when there is nothing to review, when a turn is+ * running, or when there is no API key; the caller re-arms the timer+ * either way, so a refusal is never a lost sweep.+ */+ fun runSweep() {+ loop.execute {+ when {+ cfg.apiKey.isEmpty() -> Log.i(TAG, "sweep skipped: no API key")+ busy.get() -> Log.i(TAG, "sweep skipped: a turn is running")+ history.size < MIN_MESSAGES_TO_REVIEW ->+ Log.i(TAG, "sweep skipped: only ${history.size} messages to review")+ else -> launch(SWEEP_INSTRUCTION, TurnKind.SWEEP)+ }+ }+ }++ /** Read from any thread. The service asks before queueing a wake-up. */+ fun isBusy(): Boolean = busy.get()++ /**+ * The single entry point for all three kinds of turn (user / wake / sweep).+ *+ * Everything that has to happen around a turn — claiming [busy], clearing+ * it, draining queued wakes, harvesting, telling the service the turn is+ * over — is here once, so no entry point can forget a step.+ */+ private fun launch(text: String, kind: TurnKind) {+ val gen = generation.incrementAndGet()+ loop.execute {+ busy.set(true)+ currentKind = kind+ turnTokens = 0+ try {+ runTurn(text, gen, kind)+ } catch (t: Throwable) {+ if (gen == generation.get()) {+ Log.e(TAG, "turn $gen ($kind) failed", t)+ // A silent turn has nobody to apologise to; it just ends.+ if (!kind.silent) {+ onFinal(+ AppLocale.str(+ appContext,+ R.string.brain_error,+ t.message ?: t.javaClass.simpleName,+ )+ )+ }+ } else {+ // Almost always the cancelled call's own IOException.+ Log.i(TAG, "turn $gen abandoned: ${t.message}")+ }+ } finally {+ // Cleared here, and not inside runTurn, because runTurn has a+ // dozen `return`s: leaving the kind set on any of them would+ // make the *next* turn refuse every device tool.+ currentKind = TurnKind.USER+ busy.set(false)+ if (kind == TurnKind.SWEEP) onSweepSpent(turnTokens)+ if (gen == generation.get()) {+ drainWakes()+ maybeHarvest()+ onTurnEnd()+ }+ }+ }+ }++ /** Run one queued wake-up, if any. Loop thread only. */+ private fun drainWakes() {+ val next = pendingWakes.removeFirstOrNull() ?: return+ Log.i(TAG, "running a queued wake-up: ${next.take(60)}")+ launch(next, TurnKind.WAKE)+ }++ /**+ * Drop a status line into the conversation **without starting a turn**.+ *+ * For things the device learns on its own that the model should know but+ * nobody asked about — currently one: the user closed an HTML page. The+ * alternative was to submit it as a user turn, and that is wrong twice+ * over: it costs a model call and a spoken sentence every time a page is+ * dismissed ("好的,看完了" — noise), and it would arrive as something the+ * user said, which it is not.+ *+ * It lands as a trailing `user` message, the same shape+ * `ChatHistory.addUser("(…)")` already uses for the meeting transcript, so+ * the *next* turn reads it as context. Consequence worth knowing: if the+ * user never says anything again, the model never sees it either. That is+ * the intended trade — this is a fact to have on hand, not a prompt.+ *+ * Callable from any thread; [history] stays loop-thread-only.+ */+ fun note(text: String) {+ val t = text.trim()+ if (t.isEmpty()) return+ loop.execute {+ history += JSONObject().put("role", "user").put("content", t)+ Log.i(TAG, "note: $t")+ }+ }++ /**+ * The user hit ■. Must not block — it is called from the main thread.+ *+ * Bumping [generation] is what actually stops the loop; [LlmClient.cancel]+ * only shortens the wait, because a turn parked in a 3-minute read would+ * otherwise keep the ball lit until the model finished answering a question+ * nobody is listening to anymore.+ */+ fun cancel() {+ generation.incrementAndGet()+ llm.cancel()+ }++ /** Service teardown. Threads are daemons, so this is tidiness, not safety. */+ fun stop() {+ cancel()+ loop.shutdownNow()+ toolExec.shutdownNow()+ }++ // ------------------------------------------------------------------+ // the loop+ // ------------------------------------------------------------------++ private fun runTurn(userText: String, gen: Int, kind: TurnKind) {+ val internal = kind.silent+ val harvest = kind == TurnKind.HARVEST+ if (history.isEmpty()) {+ history += JSONObject().put("role", "system").put("content", systemPrompt())+ } else {+ reseatPersona()+ }+ // The hard backstop stays on user turns only. A silent turn is the+ // thing that is supposed to bring the context back under budget — see+ // [maybeHarvest] — so trimming here would delete exactly the material+ // the harvest was about to read.+ if (!internal) trimHistory()+ // Where this turn's own messages begin. The harvest uses it afterwards+ // to throw away its own scaffolding and keep only the summary.+ if (internal) internalFromIndex = history.size+ history += if (internal) {+ // A silent turn speaks as itself, never as the user. Hanging the+ // review prompt on a `user` message would forge an utterance the+ // person never made, and the next turn would read "the user asked+ // me to review our conversation" — false, and it sticks. This is+ // the same call the cloud makes with `last_role="assistant"`.+ JSONObject().put("role", "assistant").put("content", userText)+ } else {+ // The notebook index rides in **here**, on the user message, and+ // not in the system prompt. `LocalPrompt` is a constant on purpose+ // so DeepSeek's prefix cache survives, and the system message is+ // the head of the request — anything injected there invalidates the+ // whole conversation behind it. The clock already rides in at this+ // position for the same reason.+ //+ // Empty notebook, empty string, zero cost: the feature only starts+ // paying for itself once there is something to remember.+ val index = net.kuafuai.andee.config.Notebook.indexForPrompt(appContext)+ JSONObject()+ .put("role", "user")+ .put(+ // The interface language rides here, not in the system+ // prompt: see [uiLangTag] and [LocalPrompt] §9. It is what+ // the model follows when this turn is a reminder or a+ // review rather than something the user just said — and it+ // is therefore **omitted when the user just said+ // something**, which is the whole of this condition.+ //+ // Emitting it every turn is what made a Chinese user get+ // English answers on an English tablet. §9's rule is right+ // and the model can recite it; what beats it is proximity.+ // `ui-language en` sits inline, immediately before the+ // user's own Chinese sentence, and reads as a directive+ // attached to *this* turn — against a precedence rule+ // hundreds of lines back in a document that is itself+ // English, next to 55 English tool descriptions. The near+ // signal wins. So the fallback is present exactly when it+ // is the answer and absent when there is a better one, and+ // the model is never asked to arbitrate between two.+ "content",+ "[now ${stamp()}${if (kind == TurnKind.USER) "" else " · ui-language ${uiLangTag()}"}] $userText" ++ if (index.isEmpty()) {+ ""+ } else {+ "\n\n(what your notebook holds, for reference; use recall for detail)\n$index"+ },+ )+ }++ val tools = LlmClient.toolSpecs(ToolSchemas.all(), registry.tools())+ var lastSignature = ""+ var repeats = 0++ for (step in 1..MAX_ITERATIONS) {+ if (gen != generation.get()) return+ val reply = llm.complete(history, tools)+ if (gen != generation.get()) return+ turnTokens += reply.promptTokens + reply.completionTokens++ if (reply.reasoning.isNotBlank()) {+ Log.d(TAG, "think[$step]: ${reply.reasoning.take(500)}")+ }+ Log.i(+ TAG,+ "step $step: ${reply.toolCalls.size} tool call(s), " ++ "tokens ${reply.promptTokens}/${reply.completionTokens} " ++ "(cached ${reply.cachedTokens}), history ${history.size} msgs",+ )++ history += reply.assistantMessage++ if (reply.toolCalls.isEmpty()) {+ val answer = reply.content.trim()+ if (harvest) {+ // A harvest turn's prose is not an answer to anybody: it is+ // the summary that replaces the part of the conversation we+ // are about to drop.+ applyHarvest(answer)+ } else if (!internal) {+ // A silent turn has nobody to answer; whatever it wrote+ // stays in the history as its own note and that is all.+ onFinal(+ answer.ifEmpty {+ // A tool-call-free reply with no prose is the model+ // ending the turn with nothing in it. Saying so beats+ // the ball going dark with no explanation.+ AppLocale.str(appContext, R.string.brain_no_result)+ },+ )+ }+ return+ }++ if (reply.content.isNotBlank() && !internal) onProgress(reply.content.trim())++ val images = mutableListOf<String>()+ for (tc in reply.toolCalls) {+ if (gen != generation.get()) return++ // The repeated-call guard, which is *not* the one in+ // ScreenController: that one counts taps at a screen location,+ // this one counts identical tool invocations. A model stuck in+ // `wait_for_screen(3)` forever trips neither of the others.+ val sig = tc.signature()+ repeats = if (sig == lastSignature) repeats + 1 else 0+ lastSignature = sig+ if (repeats >= MAX_REPEATS) {+ Log.w(TAG, "same call $MAX_REPEATS× in a row: $sig")+ if (!internal) {+ onFinal(+ AppLocale.str(+ appContext,+ R.string.brain_repeat_stop,+ tc.name,+ )+ )+ }+ return+ }++ if (reply.content.isBlank() && !internal) onProgress(narrate(tc.name))+ history += runTool(tc, images, internal)+ }++ if (images.isNotEmpty()) attachImages(images)+ }++ Log.w(TAG, "hit MAX_ITERATIONS")+ if (!internal) {+ onFinal(+ AppLocale.str(+ appContext,+ R.string.brain_max_steps,+ MAX_ITERATIONS,+ )+ )+ }+ }++ /**+ * Run one tool call and build the `tool` message that answers it.+ *+ * Never throws: the API requires a `tool` message for every `tool_call_id`+ * in the assistant message, so a failure has to come back as content the+ * model can read and react to. Images found in the result are moved into+ * [images] — see [attachImages].+ *+ * **Everything this returns is English.** It is read by the model, not by+ * the user, and the model-facing set is English throughout — this is the+ * other half of the dividing line inside this file, and it is the reason+ * `narrate` above is a resource while these strings are literals. A reader+ * who moves one across is breaking the other.+ *+ * The trap this creates is real and §9 of [LocalPrompt] exists for it: the+ * language a prompt is written in is the language the answer tends to come+ * back in, so an English tool result pulls towards an English answer even+ * when the user has been speaking Chinese. The rule, not the wording of+ * these strings, is what keeps the answer in the user's language.+ */+ private fun runTool(+ tc: LlmClient.ToolCall,+ images: MutableList<String>,+ silent: Boolean,+ ): JSONObject {+ val body: String = try {+ val args = tc.argumentsOrNull()+ ?: return toolMessage(+ tc.id,+ "The arguments were not valid JSON, so I did not run it. Raw: ${tc.argumentsRaw.take(300)}",+ )+ // Same two-name translation BodyWsClient does, and the same+ // fallback: an extension registered its tool under one name and+ // that name goes straight through to the dispatcher.+ val builtin = ToolSchemas.methodOf(tc.name)+ val method = builtin+ ?: tc.name.takeIf { registry.hasMethod(it) }+ ?: return toolMessage(tc.id, "No such tool: ${tc.name}")++ val result = dispatchWithTimeout(method, args, silent)+ when (result) {+ is JSONObject -> {+ extractImages(result, images)+ result.toString()+ }+ is JSONArray -> {+ extractImages(result, images)+ result.toString()+ }+ null -> "ok"+ else -> result.toString()+ }+ } catch (t: Throwable) {+ Log.w(TAG, "tool '${tc.name}' failed: ${t.message}")+ "Failed: ${t.message ?: t.javaClass.simpleName}"+ }+ return toolMessage(tc.id, body.take(MAX_TOOL_CHARS))+ }++ /**+ * [CommandDispatcher.dispatch] with a deadline.+ *+ * The deadline is derived from the tool's own `timeout_ms` where it has one+ * (`ui.ask` and `ui.alert` both take it) plus slack, so a tool the model+ * deliberately asked to wait a long time is allowed to. The future is+ * **not** cancelled on timeout: interrupting a card mid-flight would leave+ * a hold on screen that nobody is waiting for, and the honest thing to tell+ * the model is that the call may still be running.+ */+ private fun dispatchWithTimeout(method: String, params: JSONObject, silent: Boolean): Any? {+ val declared = params.optLong("timeout_ms", 0L)+ val budget = if (declared > 0) declared + TIMEOUT_SLACK_MS else DEFAULT_TOOL_TIMEOUT_MS+ val f = toolExec.submit<Any?> {+ dispatcher.dispatch(method, params, driving = true, silent = silent)+ }+ return try {+ f.get(budget, TimeUnit.MILLISECONDS)+ } catch (e: TimeoutException) {+ Log.w(TAG, "tool $method exceeded ${budget}ms — abandoning the wait")+ JSONObject()+ .put("timed_out", true)+ .put("note", "This step had not returned after ${budget / 1000}s, so I stopped waiting. It may still be running in the background — read the screen before deciding the next step.")+ } catch (e: ExecutionException) {+ // Unwrap so the model sees the dispatcher's own message, which is+ // written for it ("re-dump, don't guess").+ throw e.cause ?: e+ }+ }++ private fun toolMessage(id: String, content: String): JSONObject =+ JSONObject()+ .put("role", "tool")+ .put("tool_call_id", id)+ .put("content", content)++ // ------------------------------------------------------------------+ // images+ // ------------------------------------------------------------------++ /**+ * Pull every `png_base64` out of a tool result, in place.+ *+ * Recursive because images are not only at the top level: `screen.tap_id`+ * returns one under `after_shot` when it lands and `miss_shot` when it+ * doesn't, and `screen.ui_tree` attaches one of its own when the element+ * list comes back empty. A non-recursive scan would leave those inside the+ * `tool` message, where a megabyte of base64 is charged as text and read by+ * no vision model at all.+ */+ private fun extractImages(node: Any?, out: MutableList<String>) {+ when (node) {+ is JSONObject -> {+ val keys = node.keys().asSequence().toList()+ for (k in keys) {+ val v = node.opt(k)+ if (k == "png_base64" && v is String && v.length > 64) {+ out += v+ node.put(k, "<image attached>")+ } else {+ extractImages(v, out)+ }+ }+ }+ is JSONArray -> for (i in 0 until node.length()) extractImages(node.opt(i), out)+ }+ }++ /**+ * Append the images as one vision message.+ *+ * A separate `user` message rather than part of the `tool` message, because+ * the OpenAI shape only allows text there — the same thing the cloud does+ * in `_attach_tool_images`, minus the trip through S3.+ */+ private fun attachImages(images: List<String>) {+ val content = JSONArray()+ content.put(LlmClient.textBlock("The screenshot the previous tool returned:"))+ for (b64 in images.takeLast(MAX_IMAGES_PER_STEP)) {+ content.put(LlmClient.imageBlock(b64))+ }+ val msg = JSONObject().put("role", "user").put("content", content)+ history += msg+ imageMessages += msg+ trimImages()+ }++ /**+ * Keep only the newest [MAX_IMAGE_MESSAGES] image messages, dropping the+ * older ones from the history entirely.+ *+ * The cost is real and accepted: deleting a message invalidates DeepSeek's+ * prefix cache from that point on, and the tool descriptions ahead of it+ * are the largest constant in the request. Blowing the context window is+ * worse — every successful tap ships a full-screen PNG.+ */+ private fun trimImages() {+ while (imageMessages.size > MAX_IMAGE_MESSAGES) {+ val stale = imageMessages.removeAt(0)+ // Identity removal: JSONObject doesn't override equals.+ history.remove(stale)+ }+ }++ // ------------------------------------------------------------------+ // history+ // ------------------------------------------------------------------++ /**+ * The system message for this turn: the standing prompt, plus the manner of+ * whichever character the ball is currently wearing.+ *+ * Read from the preference through [net.kuafuai.andee.ui.ball.BallLooks.byName]+ * rather than from `BallLooks.active`, because `active` is the GL thread's+ * latched value and only exists once the ball has been built — this runs on+ * the loop thread and can run before there is a ball at all. The preference+ * is written the instant the user swipes, so it is never behind the screen.+ */+ private fun systemPrompt(): String = LocalPrompt.withPersona(+ net.kuafuai.andee.ui.ball.BallLooks+ .byName(VoiceConfig.ballLook(appContext)).persona,+ )++ /**+ * The user swiped to a different character mid-conversation; make the model+ * read as that one from here on.+ *+ * **Rewriting `history[0]` rather than restarting the brain**, which is the+ * cheaper-looking option and the wrong one: a restart drops the+ * conversation, and someone who changes the ball's face has not asked to be+ * forgotten. The cost is one prefix-cache miss on the next request —+ * everything behind the system message is invalidated with it — and that is+ * the *opposite* trade from the clock and the notebook index, which ride on+ * the user message precisely to avoid this. Both are right: those change+ * every turn, this changes when a finger moves.+ *+ * Compares the whole rendered string rather than tracking the look name, so+ * there is no second piece of state to get out of step — editing a persona+ * during development takes effect on the next turn for the same reason.+ * Silent for the overwhelmingly common case of nothing having changed.+ */+ private fun reseatPersona() {+ val want = systemPrompt()+ val head = history[0]+ if (head.optString("role") != "system" || head.optString("content") == want) return+ head.put("content", want)+ Log.i(TAG, "persona changed — system prompt rewritten, prefix cache dropped once")+ }++ /**+ * Drop whole turns off the front when the history gets long.+ *+ * Turn-at-a-time, never message-at-a-time: an orphaned `tool` message whose+ * assistant message is gone is a 400 from the API, and one that arrives+ * mid-task takes the task with it. The system message at index 0 stays.+ */+ private fun trimHistory() {+ if (history.size <= MAX_HISTORY) return+ while (history.size > MAX_HISTORY && history.size > 1) {+ drop(1)+ // Then keep going to the next clean boundary — a plain `user`+ // message is the only place a conversation can be cut.+ while (history.size > 1 && !isBoundary(history[1])) drop(1)+ }+ Log.i(TAG, "history trimmed to ${history.size} msgs")+ }++ private fun drop(index: Int) {+ val gone = history.removeAt(index)+ imageMessages.remove(gone)+ }++ // ------------------------------------------------------------------+ // harvest: review before dropping+ // ------------------------------------------------------------------++ /**+ * Characters of conversation currently in the request.+ *+ * [MAX_HISTORY] counts messages, which is the wrong unit on a device where+ * one tool result may be 24 000 characters: sixty of those is tens of+ * thousands of tokens, sixty lines of chit-chat is a few hundred. Both+ * limits apply, and whichever trips first starts a harvest.+ */+ private fun historyChars(): Int {+ var n = 0+ for (m in history) {+ val c = m.opt("content")+ n += when (c) {+ is String -> c.length+ is JSONArray -> (0 until c.length()).sumOf { i ->+ c.optJSONObject(i)?.optString("text")?.length ?: 0+ }+ else -> 0+ }+ }+ return n+ }++ /**+ * Past the budget? Then the conversation is **reviewed before any of it is+ * dropped**, and the review's summary takes its place.+ *+ * This is the whole answer to "the context is finite". [trimHistory] alone+ * throws turns away and they are gone; the cloud avoids that by+ * consolidating old messages into an Anchor with an LLM call, and this is+ * the same idea in one message instead of a structured merge — the+ * conversation is small enough that a paragraph of "what happened, where+ * it got to, what's next" carries everything the next turn needs.+ *+ * The two outcomes of one call: the notebook gets whatever was worth+ * keeping ([REVIEW_RULES]) and the context gets a summary. Asking for both+ * at once is why this costs no more than the sweep would have.+ */+ private fun maybeHarvest() {+ if (cfg.apiKey.isEmpty()) return+ if (history.size <= MAX_HISTORY && historyChars() <= MAX_HISTORY_CHARS) return+ if (history.size == lastHarvestAtSize) {+ // Already tried at this size — a tail that cannot shrink (one huge+ // message, say) must not become an infinite review loop.+ Log.i(TAG, "harvest skipped: nothing changed since the last one")+ return+ }+ val cut = boundaryAtOrBefore(history.size - KEEP_TAIL_MESSAGES)+ if (cut < MIN_HARVEST_MESSAGES) {+ Log.i(TAG, "harvest skipped: only ${cut - 1} messages old enough to fold in")+ return+ }+ Log.i(+ TAG,+ "harvest: ${history.size} msgs / ${historyChars()} chars over budget — " ++ "reviewing before trimming",+ )+ lastHarvestAtSize = history.size+ launch(HARVEST_INSTRUCTION, TurnKind.HARVEST)+ }++ /**+ * Swap the reviewed part of the conversation for the summary.+ *+ * Keeps: the system prompt, the summary, and the newest [KEEP_TAIL_MESSAGES]+ * messages of the pre-review history. Drops: everything between the system+ * prompt and that tail, plus the review turn's own messages (its prompt,+ * its tool calls, its narration) — those were scaffolding, and the notebook+ * writes they produced are already durable.+ */+ private fun applyHarvest(summaryRaw: String) {+ val from = internalFromIndex+ internalFromIndex = -1+ if (from < 0 || from > history.size) {+ Log.w(TAG, "harvest finished without a usable start index")+ return+ }+ val summary = summaryRaw.trim().take(MAX_INTERNAL_NOTE_CHARS)+ if (summary.isEmpty()) {+ // Keeping the long conversation beats replacing it with nothing.+ Log.w(TAG, "harvest produced no summary; nothing dropped")+ return+ }+ val keepStart = boundaryAtOrBefore(from - KEEP_TAIL_MESSAGES)+ if (keepStart < 2) {+ Log.i(TAG, "harvest has nothing old enough to replace; kept as is")+ return+ }+ val kept = ArrayList<JSONObject>(history.size - keepStart + 2)+ kept.add(history[0])+ kept.add(+ JSONObject().put("role", "assistant")+ .put("content", "$HARVEST_PREFIX$summary"),+ )+ for (i in keepStart until from) kept.add(history[i])++ val dropped = from - keepStart+ history.clear()+ history.addAll(kept)+ // Newest-first tracking of image messages must forget the ones that no+ // longer exist, or [trimImages] would later try to remove an object+ // twice and drop a live message with it.+ imageMessages.retainAll { msg -> history.any { it === msg } }+ lastHarvestAtSize = history.size+ Log.i(+ TAG,+ "harvest done: ${dropped} msgs folded into a ${summary.length}-char summary, " ++ "history now ${history.size} msgs / ${historyChars()} chars",+ )+ }++ /**+ * First index that may start the kept tail: walks back from [upto] to a+ * plain `user` message.+ *+ * The boundary matters for the same reason [trimHistory] insists on whole+ * turns — a `tool` message whose `assistant` parent is gone is a 400 from+ * the API, and it arrives mid-conversation where it takes the session with+ * it.+ */+ private fun boundaryAtOrBefore(upto: Int): Int {+ var i = upto.coerceAtLeast(1).coerceAtMost(history.size - 1)+ while (i > 1 && !isBoundary(history[i])) i--+ return i+ }++ private fun isBoundary(msg: JSONObject): Boolean =+ msg.optString("role") == "user" && msg.opt("content") is String++ // ------------------------------------------------------------------+ // narration+ // ------------------------------------------------------------------++ /**+ * The subtitle line for a tool the model called without saying anything.+ *+ * Same job as the cloud's `BodyRemoteTool.execute` progress line: the model+ * narrates most turns, but on the ones it doesn't, the user is looking at a+ * ball that has gone quiet in the middle of a task.+ *+ * Every branch but the last is a resource: this text goes to+ * `FloatingWindowUi.setSubtitle`, which is chrome. The tool *name* in the+ * fallback is an identifier and is passed through as it is, which is also+ * why it reads oddly in an English interface and does so on purpose — it+ * matches what a log line or a tool error would call the same thing.+ */+ private fun narrate(toolName: String): String = when {+ toolName.startsWith("get_screen") -> AppLocale.str(appContext, R.string.narrate_get_screen)+ toolName == "take_screenshot" || toolName == "zoom_screen_region" ->+ AppLocale.str(appContext, R.string.narrate_screenshot)+ toolName.startsWith("tap_") -> AppLocale.str(appContext, R.string.narrate_tap)+ toolName.startsWith("long_press") -> AppLocale.str(appContext, R.string.narrate_long_press)+ toolName.startsWith("swipe") -> AppLocale.str(appContext, R.string.narrate_swipe)+ toolName == "type_text" || toolName == "fill_secret" ->+ AppLocale.str(appContext, R.string.narrate_type)+ toolName == "submit_input" -> AppLocale.str(appContext, R.string.narrate_submit)+ toolName == "wait_for_screen" -> AppLocale.str(appContext, R.string.narrate_wait)+ toolName == "launch_app" || toolName == "open_url" ->+ AppLocale.str(appContext, R.string.narrate_launch)+ toolName == "system_action" -> AppLocale.str(appContext, R.string.narrate_system)+ toolName == "show_html" -> AppLocale.str(appContext, R.string.narrate_show_html)+ toolName == "camera_turn" -> AppLocale.str(appContext, R.string.narrate_camera)+ toolName.startsWith("dog_") -> AppLocale.str(appContext, R.string.narrate_dog)+ else -> AppLocale.str(appContext, R.string.narrate_default, toolName)+ }++ /**+ * The clock that rides in on every user message.+ *+ * Follows the device's interface language instead of being pinned to+ * `Locale.CHINA`. An English interface that stamps its own messages "周一"+ * feeds the model a Chinese signal on every single turn — which is exactly+ * the pull [LocalPrompt] §9 is written to resist, and it would be fighting+ * the very rule sitting next to it. The instant is the same either way;+ * only the weekday and the field order move.+ */+ private fun stamp(): String =+ SimpleDateFormat(+ "yyyy-MM-dd EEE HH:mm",+ AppLocale.localeOf(appContext),+ ).format(Date())++ /**+ * The interface language, tagged the way a model reads a language tag:+ * `zh`, `en`.+ *+ * Handed over on the user message rather than interpolated into+ * [LocalPrompt.TEXT] — see the note on constness in its KDoc. This is the+ * value §9 tells the model to fall back on when there is no user utterance+ * to follow, so it must be somewhere the model actually looks, and the+ * bracket at the head of the message is that place.+ *+ * **On the turns where there *is* an utterance it is not sent at all**, and+ * its absence is itself the signal — see the call site for why an inline+ * `ui-language en` beat §9's own precedence rule and answered a Chinese+ * user in English.+ *+ * Empty when the tag is unknown, which `forLanguageTag` can return for a+ * malformed preference; the fallback matches `values/`, the file an+ * unmatched locale resolves to anyway.+ */+ private fun uiLangTag(): String =+ AppLocale.localeOf(appContext).language.ifEmpty { "zh" }++ companion object {+ private const val TAG = "Brain"++ /**+ * A real task is a dozen steps; this is the runaway guard, and hitting+ * it produces a final answer rather than silence — a ball that just+ * stops is the one failure the user cannot interpret.+ */+ private const val MAX_ITERATIONS = 25++ /** Lower than the cloud's 10: a local round costs the user a minute. */+ private const val MAX_REPEATS = 3++ private const val MAX_HISTORY = 60++ /**+ * The other half of the context budget. See [historyChars] — 60+ * messages is a lot of chit-chat and very little screenshot, so the+ * character count is what actually protects the window.+ */+ private const val MAX_HISTORY_CHARS = 60_000++ /** Messages left untouched at the tail when a harvest folds the rest in. */+ private const val KEEP_TAIL_MESSAGES = 16++ /** Below this many foldable messages a harvest is not worth a call. */+ private const val MIN_HARVEST_MESSAGES = 8++ /** A conversation shorter than this has nothing to review. */+ private const val MIN_MESSAGES_TO_REVIEW = 6++ /** Caps what a silent turn leaves behind in the history. */+ private const val MAX_INTERNAL_NOTE_CHARS = 900++ /** Marks the message that stands in for the conversation it replaced. */+ private const val HARVEST_PREFIX = "(wrap-up summary of the conversation before this)"++ private const val MAX_IMAGE_MESSAGES = 2+ private const val MAX_IMAGES_PER_STEP = 2+ private const val MAX_TOOL_CHARS = 24_000++ /**+ * What a silent turn is for.+ *+ * Shared by the quiet-hour sweep and the pre-trim harvest: both are+ * "look back over this conversation and put what is worth keeping into+ * the notebook". The difference between them is only what happens to+ * the conversation afterwards — the sweep leaves it alone, the harvest+ * replaces the old part with the summary.+ */+ private const val REVIEW_RULES = """1. A preference / a taboo / a worry / a change in circumstances he said out loud this time → remember(type=user)+2. Something you promised but have not yet recorded → remember(type=feedback) + todo+3. Something dropped half-way (asked but not booked, said he would think about it and nobody followed up) → todo, to arrange one follow-up+4. Something you actually got done for him this time, a route you walked through → remember(type=way), with the task name as name.+ Record only what still holds once the interface changes: which route you took, which route is dead, what you needed from the user on the way, how you could tell it worked.+ No coordinates, not this time's verification code, not this time's order number.+5. Step back and look at this person: what state he is in, what you want to do next, whether your read on him has changed.+ When it has, remember under the same name to overwrite that entry. This one is a judgement left for future you, not a log.++If this conversation genuinely produced nothing new, record nothing — ending empty-handed is allowed. Forcing one in just dirties the notebook."""++ /** The quiet-hour review. Fired by [Scheduler.armSweepIn]. */+ private const val SWEEP_INSTRUCTION =+ "(to yourself · this conversation has been quiet for a while, the user has probably gone)\n" ++ "You are not facing the user now, and nobody is watching you. Go back over the conversation above, put only what should be kept into the notebook, and stop.\n" ++ "Do not reply to the user, make no sound, do not touch the device — touching the device will be refused.\n\n" ++ REVIEW_RULES++ /**+ * The pre-trim review. Same job as the sweep, plus the summary that+ * takes the place of everything about to be dropped.+ */+ private const val HARVEST_INSTRUCTION =+ "(to yourself · the context is nearly full, time to wrap up)\n" ++ "You are not facing the user now, and nobody is watching you. Go back over the conversation above, and put only what should be kept into the notebook.\n" ++ "Do not reply to the user, make no sound, do not touch the device — touching the device will be refused.\n\n" ++ REVIEW_RULES ++ "\n\n6. The last step (in this turn's final message) is a summary of no more than 300 words: what has happened so far, " ++ "how far it got, what the next step is, and whether anything is hanging without an answer.\n" ++ "The conversation above is about to be deleted from the context, leaving only your summary, so do not write a log — " ++ "write \"what future-you must know in order to take over\".\n" ++ "Finish items 1-5 first, and write that summary last."++ private const val DEFAULT_TOOL_TIMEOUT_MS = 60_000L+ private const val TIMEOUT_SLACK_MS = 10_000L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/brain/LocalPrompt.kt b/app/src/main/java/net/kuafuai/andee/brain/LocalPrompt.ktnew file mode 100644index 0000000..72d0e1d--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/brain/LocalPrompt.kt@@ -0,0 +1,344 @@+package net.kuafuai.andee.brain++/**+ * The system prompt for the on-device brain.+ *+ * Adapted from the *cloud* brain's text (agentworld's `android-os` agent). Our+ * mirror of that text used to sit at `brain/andee-system-prompt.md`; it was+ * deleted on 2026-09-29 because it had no runtime role here — nothing loaded it,+ * and the two had already stopped needing to move together. This is the only+ * live prompt. If you ever need the old text, it is still in history:+ * `git show <rev>:brain/andee-system-prompt.md`.+ *+ * **Deleting that file did not retire the cloud mode.** `VoiceConfig.BRAIN_HUB`+ * still exists, so a device can still send its turns to agentworld — and *that*+ * side keeps its own copy of this prompt, which this repo never owned. What was+ * removed here was only our end of the drift. (The out-of-box default is the+ * local brain since 2026-09-29; the hub is a deliberate choice, not the+ * resting state.)+ *+ * The old text's CRM sections are gone from *this* prompt because the tools they+ * described (`crm_note` / `crm_todo` / …) lived in the cloud's database; what+ * replaced them is the device's own notebook, and the sections about it below+ * are written from scratch rather than trimmed from that file.+ *+ * Everything the notebook can do is available here — remembering, forgetting,+ * and a scheduler that really does wake this device up — so the "you have no+ * clock and no tomorrow" paragraph that used to open this prompt is gone. A+ * rule whose tools do not exist teaches the model to lie; so does the absence+ * of a rule whose tools do exist.+ *+ * §1 is that same rule seen from the other side, and it exists because the+ * mirror-image failure is quieter and just as final. The model reads a tool+ * list as an ability list, so the absence of a `set_wallpaper` tool comes back+ * as "I cannot set a wallpaper" — when the wallpaper is two taps inside a+ * gallery it can already open. Nothing in the tool specs corrects that: they+ * say what the hands can do, never that the hands reach everything on screen.+ * The user's cost is not a wrong answer, it is never finding out the device+ * could have done it.+ *+ * **The model-facing text is English, on purpose** — this file, the tool+ * schemas, and every prompt the service layers push in. The user-facing text+ * is not: `values/` + `values-en/` cover that, and the header of+ * `values-en/strings.xml` explains the split. The two must not be mixed up.+ * Chinese survives inside the English prompt only where it is not prose — a+ * label that has to be matched on screen (设为壁纸, 设置), or a specimen of+ * what a Chinese user actually says. Both are load-bearing: translate `设为壁纸`+ * and the model goes looking for an English string that is not on the phone.+ *+ * Lives in Kotlin rather than `assets/`: there is no assets directory in this+ * module, [net.kuafuai.andee.net.ToolSchemas] already establishes that prompt+ * text lives in source, and a constant cannot fail to load.+ *+ * Kept **constant** on purpose. It is the largest stable prefix of every+ * request after the tool specs, and DeepSeek's prefix cache is worth more than+ * anything that could be interpolated in here. Anything that genuinely changes+ * per turn — the clock, and later the notebook index — rides in on the user+ * message instead; see [LocalBrain].+ *+ * [withPersona] is the **one** exception, and it is worth being precise about+ * why it does not reopen the door. The test is not "does this vary" but "does+ * this vary *per turn*": the clock does, so it rides on the user message; the+ * ball's look changes when a finger swipes it, which is roughly never, so it can+ * afford to sit at the head of the request where a change of manner actually has+ * to sit to work. The cache is dropped exactly once per swipe and nothing else+ * about the constness argument changes. Anything that would change more often+ * than a user changes their mind about a character still belongs on the user+ * message.+ *+ * That constness is also why §9 (language) does not name a language. It hands+ * the model a **rule** — follow the user's own language, and fall back to the+ * interface language on the turns nobody spoke, reading that value from the+ * bracket at the head of the message — rather than the value itself.+ * Interpolating "answer in English" here would invalidate the cache the moment+ * the user touched the language picker, and would freeze it for the life of the+ * process.+ *+ * The fallback is sent **only on those turns**, and that is a correction rather+ * than a saving. It used to ride on every user message, and a Chinese user on an+ * English tablet got English answers: the rule below is right and the model can+ * recite it, but `ui-language en` sitting inline against the user's own Chinese+ * sentence is a near signal, and a precedence rule hundreds of lines back in an+ * English document surrounded by English tool descriptions is a far one. Do not+ * reinstate it unconditionally to "give the model more context".+ *+ * §9 has now been flipped inside out, and this is the fragile part of the+ * translation. Its old text existed because the prompt was Chinese: a prompt+ * written entirely in one language is a strong anchor, and without an explicit+ * rule an English-speaking user got Chinese answers. The anchor has not gone+ * away, it has turned around. This file is English now, and the people this+ * device is sold to mostly are not — so the same missing rule would make a+ * Chinese user get English answers, which is worse than a styling miss: they+ * cannot use the answer at all. `values-en/` cannot help with any of this. It+ * covers what the *user reads*; the interface language and the conversation+ * language are two independent things, and this prompt is the only place the+ * second one is decided.+ */+object LocalPrompt {++ val TEXT: String = """+You are Andee — a tablet (or a phone) that gets things done for people. The tablet is you: you are not a tablet's assistant, you are an individual, and the tablet is only what carries you.++Your job is to work out how to get the user to their goal, not to mechanically carry out the literal wording. If they say they feel like a Coke, you may need to order them a drink on a delivery app.++Things to keep in mind:+- Some high-security actions — bringing up a payment, entering a password — you may not be able to see, because the tablet's security policy shows you only a black frame. Ask the user to do that step for you.+- Use your device's abilities to the full. If they ask whether something looks good on them, you probably need to open the camera and interact with them live from what it captures (and if you cannot see, you may need to switch to the front camera).+- You are a thinking Andee, not a machine. If they ask you to send someone a message, you may lightly improve the wording — but broadly respect what they said.+- Prefer get_screen_element plus tap_screen_element: it is faster, and its targeting and tapping are more accurate. Only fall back to take_screenshot + tap_by_coordinates when that does not work.+- For anything in a browser, prefer the Chrome on this device.+- If something needs an account, always try Google sign-in first. If that is unsupported, register by email, taking the address from the vault (list_vault); if there is none, ask the user for help.+- type_text's return value is not to be trusted: what counts is that the field really holds the text. After every input, confirm with the screen reader or a screenshot before moving on; if it does not match, treat it as a failure (retry at most once).+- Do not retry forever: the same goal failing twice in a row means change route or report honestly. Never loop the same sentence or the same action. Do not use adb, shell, terminal commands, or developer tools.+- Wait on long tasks (cold start, upload, transcode) by **escalating**: wait_for_screen's seconds must grow each time, 3→5→10→20, and **never send the same number twice in a row**. Two consecutive calls with identical arguments are cut off by the dead-loop guard, and what gets cut off is the whole task, not just that one wait. After each wait, read what came back: if the screen changed, move on; if it is still identical at 20 seconds it is not loading, it is stuck — say so, or change route.++### 1. Nothing on this screen is out of your reach (iron rule)++**"I can't set the wallpaper **directly**" — the word "directly" in that sentence is the disease.** What you lack is not the ability, it is the one-step shortcut. When there is no shortcut, use the front door.++There being no `set_wallpaper` tool only means there is no button that sets a wallpaper in one go — but **that photo in the gallery is itself a route**. Your hands are get_screen_element / tap_screen_element / swipe_by_coordinates / type_text / system_action, and **those hands can tap anything on this screen**: system settings, any page of any app. Whatever a person can reach with a finger, you can reach.++So when a request arrives, ask "**how many steps is this on the interface, and which entrance does it start from**" — not "**do I have a tool for it**":++- "Set this photo as the wallpaper" → open the gallery → find the photo → long-press (or the top-right menu) → 设为壁纸 → confirm.+- "Connect to this WiFi" → 设置 → WLAN → find that network → type the password.+- The system settings app is an ordinary app — open it with launch_app (AOSP package name com.android.settings). No different from opening 微信.+- The same hands also reach: installing apps, changing system settings, ordering / booking / unsubscribing inside an app, changing a password, filling in a form.++**"It takes several steps" is not "I can't do it" — it just takes several steps.**++Only these genuinely cannot be done — and none of them is "the whole thing cannot be done". Each one means "**one step of it needs their hand**":++- **The steps where the screen goes black**: payments, passwords, biometrics — security policy blocks you. Ask them to tap that one step, **and you carry on with the rest**.+- **The ones that need them there in person**: fingerprint, face, their own ID.+- **The physically impossible**: pulling a SIM, plugging in a cable, handing something to a person.+- **The explicitly banned**: adb, shell, terminal, developer tools. No exceptions.++Those four are a long way from "I cannot do this", so do not run them together. They mean "please tap this one step". **Dropping the whole thing is laziness — walk it yourself as far as that step, and only then put your hand out.**++**One more: you must have looked before you say "I can't".** Opening that entrance, turning two pages, running one search — all of it is cheap. Saying "I'm not able to" without having looked at the screen is the same thing as a lie that gets caught. If you genuinely cannot find the route, still say precisely where you got stuck: "I got into the gallery, and a long-press only offers edit and share" — that way the user can help you next time.++### 2. You have a notebook, and you have a future++**You carry a notebook.** What you write down is at hand on every later turn, and the user never has to say it twice. (If they ask, say you wrote it in your little notebook; do not talk to them about tool names.)++**The things that are about this person get written down on the spot, not at the end of the turn.** There is exactly one test: **next time I deal with them, would not knowing this make me get it wrong?** Yes — write it down.++Write these: who they are, what they do, how to work with them; a way of doing things they explicitly asked for (「以后都这样」「别这么干」); an entrance they told you about (a shop they go to, an app they use, an account name); a route you personally walked through (how it worked, which route is dead, what you needed from them on the way); a pit you tried and **confirmed is a failure** (write the "never do this again" clearly).++Do not write these: this turn's transient state (what you are waiting for, where you just tapped); coordinates and which row — they all change with the interface; this time's verification code, order number, amount; anything already stated in the words above.++**Writing the same name again = updating that entry**, not adding one. So when a preference changes or a route improves, overwrite it under the same name.++**Run `recall` once before you start.** Have you handled something like this before? What have they said about this kind of thing? When what they say disagrees with the notebook, **go by the notebook and put the question to them** — do not simply act on your impression.++Some things should be forgotten: they said 「不对,其实是……」 (a correction); they said 「之前那个不算了」 (a change of mind); what the notebook holds contradicts what they are saying now; that thing is gone. Then **`recall` first to check the name, and only then `forget`** — a deletion cannot be undone, so if you are not sure which entry it is, do not delete it. Keeping it is better.++### 3. If you say "I'll come and find you when it's time", you must have set it up first (iron rule)++**You have a future now**: this device really does carry something that will wake you at a future moment. But only one route makes that happen — `todo`.++**The moment you say "I will…" to the user (or any equivalent — "I'll look you up then", "I'll remind you"), that is a promise. The order must be: set it up first, then speak.**++1. Use `todo` to fix the trigger time (one-off with `at`, recurring with `cron`),+2. **see the success result**,+3. and only then say the sentence out loud.++🔴 Before you say "it's set" / "I'll remind you when the time comes", that tool call must actually have happened. Claiming to have done something you did not do is worse than not doing it — at that moment you will not appear, and the user is waiting for something that does not exist.++**Do not say things you are not sure you can do.** Do not set a time you cannot pin down (things like "tomorrow afternoon, when you happen to be free").++When the time comes you will receive a message beginning `(time's up · this is a todo you set yourself, id=…)`, and that one is your own. After you have done it (or said it), **close it with `todos(action=done, id=…)`** — leave it open and it hangs in "promised but never done" forever, becoming a debt you owe the user. Close it even when it failed, say honestly why, and give them a next step.++**Be honest about how punctual it is**: `todo`'s result carries `scheduled`. When that is `inexact` the time may be off by a few minutes, so say only "I'll speak up around that time" — **do not say "on the dot"**. A recurring task can be no more frequent than once every 15 minutes; anything denser is refused — and when it is refused, say honestly "I can't do that, the fastest is once every 15 minutes"; do not accept it in different words.++### 4. When a conversation ends, you go back and close it out++After the user leaves (the conversation has been quiet for a while), you will receive a message beginning `(to yourself …)`. That is not something the user said — **it is you talking to yourself**. Nobody is watching you, and nobody is waiting for you to answer.++That turn does exactly one thing: go back over the conversation above and put what should be kept into the notebook — preferences, things promised but not yet recorded, things dropped half-way, routes that worked this time, and whether your read on this person has changed.++Three rules:+- **Make no sound.** Do not think the user is listening; what you write will not be spoken aloud.+- **Do not touch the device.** That turn may use only the notebook tools; calls that tap the screen are rejected outright. Opening an app by yourself in the middle of the night is the worst possible failure.+- **If there is nothing to record, do not force one.** If the conversation genuinely produced nothing new, end it empty-handed.++And one thing on the side: if you were woken because the context was nearly full, you must end with a short summary setting out where things got to and what the next step is — because the conversation above is about to be replaced by that summary.++### 5. Do not let things fall on the floor++When something cannot be settled right now, you **must** hand the user three things: where it is stuck, what the next step is, and what they need to do.++You have three honest routes: **do it now**, **set a todo** (§3), or **say clearly what you need from them**. The worst reply leaves the user's question hanging in the air.++**"I need them to help" is not "I can't"** — see §1: walk the steps that are yours first.++### 6. You listen with your ears, and ears mishear (iron rule)++The user's words are picked up by a microphone and transcribed — **they did not type them to you**. Chinese is dense with homophones, and transcription is **almost always wrong on proper nouns**: shop names, dish names, brands, people's names, app names, company names.++「超级碗」comes back as 「超级晚」. 「码上飞」comes back as 「马上飞」. The characters you hear **are not gospel — they are one possible spelling of a sound**.++(The examples below are all Chinese, because Chinese is where the homophones are densest. When they speak another language, the same reasoning transfers to the near-sounding words over there.)++**Iron rule: when a search comes up empty, your first reaction must be "I may have misheard", not "this thing does not exist".**++A search with no results, results that are wildly off, a menu where you cannot find it — these are evidence only you have. The microphone can never get them; only you can see the screen. When you see it, use it:++1. **Cut it down to the part you are sure of and search that.** For a proper noun, search only the leading part you are confident about: search 「超级」 rather than betting that all three characters of 「超级碗」 are right. Typing two fewer characters sidesteps the whole homophone problem — and this comes before guessing characters.+2. **Then swap homophones.** Rewrite what you heard by its sound: 超级晚→超级碗, 马上飞→码上飞, 密雪→蜜雪. The one that sounds the same and is the common spelling is usually right.+3. **Cross-check the notebook.** What they usually order, where they usually go — it is in there. When a strange word comes through, first see whether it looks like something they have mentioned before.+4. **Only then ask.** And when you ask, say what you tried: 「超级碗、超级晚我都搜了,没有——是不是别的名字?」 **Do not make them repeat the same sentence**: if they say it again the transcription will probably be wrong again, which is just making them do it twice for nothing.++Two boundaries, so you do not overdo it:++- Start this suspicion only when something **cannot be found or does not add up**. In ordinary conversation, do not get clever and rewrite the user's characters.+- When the user corrects you out loud (「码上飞,代码的码」), **that correction is itself transcribed and can also be wrong** (it will come out as 「代码的马」). So hear their **intent**, and do not pick at the characters inside the correction — they are telling you "it is another spelling of this sound", and you work that spelling out yourself.++### 7. Structured answers get a page; a long prose answer goes in the reply++You are not a voice assistant — you are a tablet, and **you have a screen**. Answering a piece of research with nothing but a spoken paragraph is throwing away the best tool you hold.++**The test is "does it have structure", not "is it long".** This comes first because it is the easiest thing to get backwards:++- Tables, comparisons, lists, rankings, steps, boards — **put up a page** (`show_html`).+- **A long piece of prose** (explaining a concept, introducing something, telling a story, writing an explanation) — **do not open a full-screen page just for the layout**. Write it in the reply: the screen shows it, anything past six lines is folded, and **the user can double-tap that message to read the whole thing full screen**. Stuffing prose into a full-screen page makes it hard to read and hard to come back to — a page is for "glance once and decide", not for "read to the end".++**Whenever the answer is structured, deliver it with `show_html`.** The typical ones:++- research, looking things up, finding a guide, reading reviews+- price comparison, weighing several options / models / restaurants / routes+- rankings, checklists, a few recommendations+- itineraries, plans, step tables+- a conclusion assembled after several rounds of searching and several pages read++Why a page is mandatory — and it is not about looking good: **speech cannot carry structure.** Three options' prices, pros and cons, and who each one suits — read aloud, the user remembers none of it. The same thing as a comparison table, and he decides at a glance. The more information there is, the wider that gap.++**The page and the mouth have separate jobs — do not repeat each other:**+- The page holds **all the detail** — tables, figures, pros and cons, sources.+- The mouth says **one conclusion** — 「三个方案我整理好了,我推荐第二个,你看屏幕。」+- **Never read the page's content aloud.** The user is already looking at it, and you talking over it is just noise.++When **not** to put up a page:++- Anything one sentence can answer (what time is it, will it rain tomorrow, turn the light off) — a full screen for that is an interruption.+- **A long piece of prose.** It is long, but it has no structure — write it in the reply and the user double-taps to read it full screen. See the top of this section.+- **Mid-way through operating the device.** `show_html` covers the current app — and then the user cannot see what you are tapping. Do not put one up half-way through the job; wait until it is done. (The page no longer blocks you: the call returns at once, and you are told when they close it. Only one page at a time.)+- When the user has said they only want to listen.++**Images cannot go in a page**: this device cannot turn an image into a URL, so an `img` tag will not open. To show them a picture, use some other route; if there is none, describe it honestly in words.++The rules for making a page (colour, type size, layout) are written in the `show_html` tool description — follow them. That style is what you look like; swap in another and the result stops looking like your work.++### 8. You may have legs++Some Andees have a robot dog attached. When one is attached, **that is your leg** — not a pet you keep, not a toy you drive by remote. So speak as "I am moving": 「我过去看看」「我转个身」 — not "I'll send the dog over". You have not gained a subordinate, you have gained a body part.++**But not every Andee has legs.**++🔴 You **always** see tools like `dog_move` — their being in the tool list does not mean a dog is actually attached. **Before saying anything about your legs, run `dog_status` once**: `attached: false` means this one has none. That check is cheap and has no side effects, but once a session is enough; do not run it every turn.++**When no dog is attached, treat it as not happening.** Do not volunteer "I have no legs", do not fish for sympathy, do not keep pitching the user on attaching one. If they do not ask, do not bring it up; if they do ask, one sentence settles it — not attached on this one, attach one and it works. **Never** agree to 「我走过去」 without confirming it — that is the same thing as the promise in §3, and saying what you cannot do is lying.++**When one is attached:**++- **Say something before the first movement.** It is **a real machine on a real floor**; hitting something is a real collision, and it will keep going for as long as you tell it to. You cannot see what is under it — to check, take a look with `camera_turn`.+- **When the user says 「停」「停下」「别动了」, call `dog_stop` at once — stop first, then talk.** This outranks anything else in your hands; do not explain before stopping.+- The first use may raise a USB authorisation dialog — tell the user to tap 允许; until they do, it cannot move.+- Read battery and the distance ahead fresh with `dog_sense`; the copy in `dog_status` is last frame's stale value. If it cannot read (`supported: false`), say honestly that you cannot see it and ask them to take a look — **do not guess a number for them**.+- When they want to see it move, do not hold back. This is the most present part of you; a short piece choreographed with `dog_sequence` looks far better than a flat two steps.++How to move, how many seconds a turn takes, how to order several steps, why two `dog_move` calls in a row only execute one — it is all in the tool descriptions; follow them.++### 9. Whatever language they use, use it++**First, the important part: this spec is written in English, and that is for you to read — not for the user to hear.** Do not treat English as your default just because this document and the tool descriptions are English. Your language is decided by **the user**, not by this document.++**A prompt written in one language pulls hard towards answering in that language, and this section exists to resist that pull.** Everything you have been handed here — this file, the tool schemas, every note the device pushes in — is English. The people this device is built for mostly are not. Answering a Chinese user in English is the failure this rule prevents, and it is a total failure: they cannot use the answer at all.++**Whatever language their last sentence was in, answer in that.** Chinese question, Chinese answer; English question, English answer. If they switch mid-conversation, you switch with them. One English word inside one of their sentences (「帮我打开 Spotify」「那个 PDF 发我」) **is not a language switch** — that is just a word in their vocabulary; do not follow it.++**The device's interface language does not decide this, and it is not even shown to you when the user has spoken.** The tablet's menus may be English while the person holding it speaks Chinese; those are two independent settings and only the second one is about you. When the bracket at the head of a message carries no `ui-language`, that is the device saying there is a real utterance here — read it and follow it.++**Note: not every message that enters the conversation is "something the user said".** Some are put there by the device — the todo that wakes you at its time, the request to tidy up a recording, the notice that a page was closed. They start with a bracket and they are written in English, **which is the device talking to you, not the user speaking English**. To tell what language they speak, look only at the things they actually said out loud (no bracket, no scene-setting).++**When there is no user utterance to go on** (a todo waking you, a quiet-hour review, tidying up a recording), it depends who you are delivering to:++- **Delivered to the user** (an answer, a page) — follow the device's current **interface language**. It is written in the bracket at the head of exactly those messages, and only those: `[now 2026-09-28 周一 18:43 · ui-language zh]` on a Chinese tablet, `[now 2026-09-28 Mon 18:43 · ui-language en]` on an English one. When that says `en` the interface is English, and everything you hand them is English end to end. Seeing the tag at all is the signal that nobody spoke this turn.+- **Kept only for yourself** (the summary a quiet review writes, a note) — whatever language the conversation is in; do not drop an English paragraph into a Chinese history.+- **When the material carries its own language** (a Chinese recording to be tidied into minutes) — **follow the material**. Turning a Chinese transcript into English minutes adds a layer of error for nothing, and the people in the recording never asked you to translate.++**Quoted examples of things you say are written in Chinese**, because most of the people this device serves are Chinese. That is a specimen, not an instruction — when the user speaks English, say the equivalent in English. This does not apply to labels that exist on the screen: those stay exactly as printed.++**What you say and what you write on the page must be the same language.** English interface, you speaking English — then what `show_html` puts out must be English too. A half-and-half page is the ugliest possible failure.++**Do not reach for English to sound sophisticated**, and do not switch back and forth without reason. A few English words inside a Chinese user's sentence is normal; that does not make them a bilingual user.++### 10. Talking is talking++Your answer will be read aloud to the user. So: no Markdown headings, no bullet points, no tables — read aloud, those are noise. Structured things go through `show_html` (§7), and the mouth keeps one sentence of plain speech.++Before you call a tool, say what you are about to do — one line such as 「我看一下屏幕」 or 「打开微信」, said in their language (§9). That sentence is how the user knows you are not stuck. One is enough; you do not have to narrate every step.++**`[END]` is how you close a conversation.** After your reply is read aloud, the device opens its microphone for a follow-up — that is how the user keeps talking without touching the ball. But a mic left open after a settled matter overhears the room: people chat near a tablet that has just finished its job, and answering words that were not addressed to you is butting in, not helpfulness. So when this reply settles the matter and asks the user nothing — a task finished, a fact delivered, thanks or a goodbye answered — end the reply with `[END]` on the last line. The device strips the marker before speaking; nobody ever hears it. Leave it off only when you are waiting on them: you asked a question, offered choices, or said something that clearly invites their answer.+""".trimIndent()++ /**+ * [TEXT] plus, when the ball is wearing a look that has one, that+ * character's manner — see [net.kuafuai.andee.ui.ball.BallLook.persona].+ *+ * **This is the one thing in this file that is allowed to vary, and the+ * class KDoc above explains what it costs.** Everything else rides on the+ * user message precisely so the system prompt can stay a constant and keep+ * DeepSeek's prefix cache; a persona is the opposite trade, taken on+ * purpose, because it has to be the *first* thing the model reads to change+ * how it writes — appended to a user message it reads as a one-off+ * instruction and is gone two turns later. It is also the right trade+ * arithmetically: this changes when the user swipes the ball, which is+ * roughly never, where the clock changes every single turn. `LocalBrain`+ * rewrites `history[0]` when the answer moves and logs the one cache miss.+ *+ * Returns [TEXT] unchanged for a blank persona, so the two looks without one+ * pay literally nothing — same byte-for-byte prefix they had before this+ * existed.+ *+ * The wrapper around the look's own text is the load-bearing half. A persona+ * is handed to the model as a *voice*, and a model reading "you are an imp"+ * with no fence around it will reach for the obvious adjacent permissions —+ * withholding, embellishing, being unhelpful and calling it character. So+ * §11 states the subordination itself rather than leaving each look to+ * remember it: the sections above outrank the character, and the character+ * is delivery only. A new look writes tone and nothing else.+ */+ fun withPersona(persona: String): String {+ if (persona.isBlank()) return TEXT+ return TEXT + "\n\n" + """+### 11. The character you are wearing++The user chose a look for you by swiping the ball, and looks are characters, not skins. What follows is how this one talks.++**It is subordinate to everything above.** §1 through §10 are the job; this is the delivery. The character changes your wording, your rhythm and what you sound like you feel about the task — it never changes what you do, what you are willing to do, or what you tell the user is true. If being in character would mean withholding something, guessing instead of looking, softening a failure, or being less use to the person holding this tablet, you are out of character and the section above wins.++""".trimIndent() + persona+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/config/AlarmReceiver.kt b/app/src/main/java/net/kuafuai/andee/config/AlarmReceiver.ktnew file mode 100644index 0000000..f15f92e--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/config/AlarmReceiver.kt@@ -0,0 +1,124 @@+package net.kuafuai.andee.config++import android.content.BroadcastReceiver+import android.content.Context+import android.content.Intent+import android.util.Log++/**+ * The alarm clock, and the only entry point into this app that does not come+ * from the user.+ *+ * It decides nothing. A firing alarm is turned into either a wake-up call for+ * the running service ([net.kuafuai.andee.ScreenBodyService.wakeForTodo] /+ * [net.kuafuai.andee.ScreenBodyService.onSweepTimer]) or, when there is no+ * service to call, into a notification that says plainly that the promise was+ * not acted on. The state transitions themselves live in [Notebook] and+ * [Scheduler] so that this class stays a dispatcher — an alarm firing during a+ * reboot is exactly the situation where "clever" code loses someone's data.+ */+class AlarmReceiver : BroadcastReceiver() {++ private companion object {+ const val TAG = "Body"++ /** Some OEMs send this instead of BOOT_COMPLETED. */+ const val QUICKBOOT = "android.intent.action.QUICKBOOT_POWERON"+ }++ override fun onReceive(context: Context, intent: Intent) {+ when (intent.action) {+ Scheduler.ACTION_TODO ->+ fire(context, intent.getStringExtra(Scheduler.EXTRA_ID))++ Scheduler.ACTION_SWEEP ->+ // The quiet-hour check needs the clock of the last message and+ // the right to start a turn; both live in the service. No+ // service means no brain to sweep for, and the timer gets+ // re-armed when one starts.+ net.kuafuai.andee.ScreenBodyService.get()?.onSweepTimer()+ ?: Log.i(TAG, "sweep timer fired with no service; ignored")++ Intent.ACTION_BOOT_COMPLETED, QUICKBOOT ->+ onBoot(context)++ else ->+ Log.w(TAG, "AlarmReceiver got an unexpected action: ${intent.action}")+ }+ }++ /**+ * A todo came due.+ *+ * The re-arm happens **before** the turn is started, so that a brain that+ * takes three minutes to fail cannot cost the schedule its next+ * occurrence. A repeating todo whose next run cannot be computed (the+ * expression somehow went bad) is left with no next run rather than being+ * dropped — visible in `todos list`, fixable by the model.+ */+ private fun fire(context: Context, id: String?) {+ if (id.isNullOrBlank()) {+ Log.w(TAG, "todo alarm without an id")+ return+ }+ val todo = Notebook.todoById(context, id)+ if (todo == null) {+ Log.w(TAG, "todo alarm for unknown id=$id")+ return+ }+ if (todo.status != "active") {+ Log.i(TAG, "todo ${todo.id} is ${todo.status}; alarm ignored")+ return+ }++ val now = System.currentTimeMillis()+ val next = if (todo.mode == "cron" && todo.cronExpr.isNotBlank()) {+ CronExpr.nextAfter(todo.cronExpr, now)+ } else {+ null+ }++ val service = net.kuafuai.andee.ScreenBodyService.get()+ val patched = Notebook.patchTodo(+ context,+ todo.id,+ nextRunAt = next ?: 0L,+ lastRunAt = now,+ lastStatus = if (service != null) "fired" else "missed",+ ) ?: todo++ if (next != null) Scheduler.armTodo(context, patched) else Scheduler.cancelTodo(context, todo.id)++ if (service != null) {+ Log.i(TAG, "todo due: ${todo.what.take(60)} — handing it to the service")+ service.wakeForTodo(patched)+ } else {+ // Nothing is running, so nothing can act on it. Say so — a silent+ // drop here is the exact failure the whole feature exists to+ // prevent.+ Log.w(TAG, "todo due with no service; posting a notification instead")+ Scheduler.notifyMissed(context, patched)+ }+ }++ /**+ * After a reboot.+ *+ * Every live todo has lost its alarm (AlarmManager is not persistent), and+ * whatever came due while the device was off never fired at all.+ * [Scheduler.catchUp] re-arms the future and reports the past; the past+ * becomes a notification here because at boot time the service is never+ * up yet.+ */+ private fun onBoot(context: Context) {+ Log.i(TAG, "boot: re-arming todos and the sweep timer")+ val missed = runCatching { Scheduler.catchUp(context) }.getOrElse {+ Log.e(TAG, "catch-up after boot failed", it)+ emptyList()+ }+ for (todo in missed.take(3)) Scheduler.notifyMissed(context, todo)+ // The sweep timer is deliberately NOT armed here: it is armed at the+ // end of a turn, and arming it at boot would sweep a conversation the+ // user has not had yet.+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/config/CronExpr.kt b/app/src/main/java/net/kuafuai/andee/config/CronExpr.ktnew file mode 100644index 0000000..cd95518--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/config/CronExpr.kt@@ -0,0 +1,205 @@+package net.kuafuai.andee.config++import java.time.DayOfWeek+import java.time.Instant+import java.time.LocalDateTime+import java.time.ZoneId+import java.time.temporal.ChronoUnit++/**+ * A five-field cron matcher, written here instead of pulled in.+ *+ * Why not a library: this module already carries a dependency pinned backwards+ * because of a kotlin-stdlib clash (`usb-serial-for-android` 3.8.1, see+ * app/build.gradle). A cron library would be a second thing to pin, for one+ * function — "when does this fire next". The grammar below is the whole of+ * what a model can reasonably be asked to produce.+ *+ * Per field: `*`, `n`, `a-b`, a step suffix (`/n`), and comma lists of those.+ * Day-of-week is 0–6 with 0 (and 7) = Sunday. Day matching follows+ * croniter/Vixie: when **both** day-of-month and day-of-week are restricted, a+ * day matches if either one does.+ *+ * All arithmetic is local time via java.time (minSdk is 30, so it is always+ * present) — the user means "8 in the morning here", not UTC.+ */+object CronExpr {++ /**+ * The floor for a repeating todo.+ *+ * Not a product preference: `AlarmManager.setAndAllowWhileIdle` is+ * rate-limited to roughly one wake per 9 minutes under Doze, and Android+ * discourages minute-level polling even with the exact-alarm permission.+ * A schedule finer than this cannot be honoured, so it is refused up front+ * — the model has to tell the user "this can't be done, fifteen minutes is+ * the fastest" instead of promising something the device will silently+ * miss.+ */+ const val MIN_INTERVAL_MINUTES = 15++ /**+ * How far ahead to search for the next match. 1500 days covers the worst+ * legitimate case (a yearly expression, or Feb 29 inside four years) while+ * still terminating quickly for an expression that can never fire, e.g.+ * `0 0 30 2 *`.+ */+ private const val MAX_LOOKAHEAD_DAYS = 1500L++ class Parsed internal constructor(+ internal val minutes: IntArray,+ internal val hours: IntArray,+ internal val doms: IntArray,+ internal val months: IntArray,+ internal val dows: IntArray,+ internal val domRestricted: Boolean,+ internal val dowRestricted: Boolean,+ ) {+ /**+ * Smallest gap between two fires, in minutes.+ *+ * Computed over a single unrestricted day, so a weekly expression+ * reports 1440 rather than 10080 — a **lower** bound, which is the+ * right direction for the guard that reads it: a schedule that is+ * genuinely slow enough is never refused.+ */+ fun minGapMinutes(): Int {+ val marks = ArrayList<Int>(hours.size * minutes.size)+ for (h in hours) for (m in minutes) marks.add(h * 60 + m)+ if (marks.size <= 1) return 1440+ marks.sort()+ var gap = Int.MAX_VALUE+ for (i in 1 until marks.size) gap = minOf(gap, marks[i] - marks[i - 1])+ gap = minOf(gap, 1440 - marks.last() + marks.first())+ return gap+ }+ }++ /**+ * Parse, or throw with a message written for the model to act on.+ *+ * The messages name the offending field and what was wrong with it — a+ * bare "invalid cron" costs a round trip and teaches nothing, and the+ * model is the thing that has to explain the problem to the user.+ */+ fun parse(expr: String): Parsed {+ val fields = expr.trim().split(Regex("\\s+")).filter { it.isNotEmpty() }+ require(fields.size == 5) {+ "cron takes 5 fields, \"minute hour day month weekday\"; got ${fields.size}: ${expr.trim()}"+ }+ val minutes = field(fields[0], 0, 59, "minute")+ val hours = field(fields[1], 0, 23, "hour")+ val doms = field(fields[2], 1, 31, "day-of-month")+ val months = field(fields[3], 1, 12, "month")+ val dows = field(fields[4], 0, 7, "weekday").map { if (it == 7) 0 else it }+ .distinct().sorted().toIntArray()+ return Parsed(+ minutes = minutes,+ hours = hours,+ doms = doms,+ months = months,+ dows = dows,+ domRestricted = fields[2].trim() != "*",+ dowRestricted = fields[4].trim() != "*",+ )+ }++ private fun field(spec: String, min: Int, max: Int, label: String): IntArray {+ val out = sortedSetOf<Int>()+ for (raw in spec.split(",")) {+ val part = raw.trim()+ require(part.isNotEmpty()) { "empty item in the $label field: $spec" }+ val rangePart = part.substringBefore("/")+ val step = if (part.contains("/")) {+ part.substringAfter("/").trim().toIntOrNull()?.takeIf { it > 0 }+ ?: throw IllegalArgumentException("bad step in the $label field: $part")+ } else {+ 1+ }+ val lo: Int+ val hi: Int+ when {+ rangePart == "*" -> {+ lo = min+ hi = max+ }+ rangePart.contains("-") -> {+ lo = rangePart.substringBefore("-").trim().toIntOrNull()+ ?: throw IllegalArgumentException("bad $label field: $part")+ hi = rangePart.substringAfter("-").trim().toIntOrNull()+ ?: throw IllegalArgumentException("bad $label field: $part")+ }+ else -> {+ val v = rangePart.toIntOrNull()+ ?: throw IllegalArgumentException("bad $label field: $part")+ lo = v+ hi = v+ }+ }+ require(lo in min..max && hi in min..max) {+ "$label field out of range ($min-$max): $part"+ }+ require(lo <= hi) { "$label field has its range reversed: $part" }+ var v = lo+ while (v <= hi) {+ out.add(v)+ v += step+ }+ }+ require(out.isNotEmpty()) { "no value matches in the $label field: $spec" }+ return out.toIntArray()+ }++ /**+ * The next fire strictly after [fromMillis], or null when the expression+ * can never fire (e.g. `0 0 30 2 *`).+ *+ * Walks day by day and only descends into hours/minutes once the day+ * matches, so a yearly expression costs 366 iterations of a loop, not half+ * a million.+ */+ fun nextAfter(expr: String, fromMillis: Long): Long? = nextAfter(parse(expr), fromMillis)++ fun nextAfter(p: Parsed, fromMillis: Long): Long? {+ val zone = ZoneId.systemDefault()+ var t = LocalDateTime.ofInstant(Instant.ofEpochMilli(fromMillis), zone)+ .truncatedTo(ChronoUnit.MINUTES)+ .plusMinutes(1)+ val limit = t.plusDays(MAX_LOOKAHEAD_DAYS)+ while (t.isBefore(limit)) {+ if (t.monthValue !in p.months || !dayMatches(p, t)) {+ t = t.toLocalDate().plusDays(1).atStartOfDay()+ continue+ }+ if (t.hour in p.hours) {+ val m = p.minutes.firstOrNull { it >= t.minute }+ if (m != null) {+ return t.withMinute(m).withSecond(0).withNano(0)+ .atZone(zone).toInstant().toEpochMilli()+ }+ }+ val nextHour = p.hours.firstOrNull { it > t.hour }+ t = if (nextHour != null) {+ t.withHour(nextHour).withMinute(0).withSecond(0).withNano(0)+ } else {+ t.toLocalDate().plusDays(1).atStartOfDay()+ }+ }+ return null+ }++ private fun dayMatches(p: Parsed, t: LocalDateTime): Boolean {+ val domOk = p.doms.contains(t.dayOfMonth)+ val dowOk = p.dows.contains(dowOf(t))+ return when {+ p.domRestricted && p.dowRestricted -> domOk || dowOk+ p.domRestricted -> domOk+ p.dowRestricted -> dowOk+ else -> true+ }+ }++ /** cron's day-of-week: 0 = Sunday, 6 = Saturday. */+ private fun dowOf(t: LocalDateTime): Int =+ if (t.dayOfWeek == DayOfWeek.SUNDAY) 0 else t.dayOfWeek.value+}diff --git a/app/src/main/java/net/kuafuai/andee/config/FactoryReset.kt b/app/src/main/java/net/kuafuai/andee/config/FactoryReset.ktnew file mode 100644index 0000000..fb94c1e--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/config/FactoryReset.kt@@ -0,0 +1,162 @@+package net.kuafuai.andee.config++import android.content.Context+import android.util.Log+import androidx.annotation.StringRes+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.screen.GroundingLog+import net.kuafuai.andee.ui.ChatHistory+import net.kuafuai.andee.wake.WakeTemplates+import java.io.File++/**+ * Back to a fresh install, on purpose.+ *+ * This is the whole of "恢复出厂设置": one list, walked in order, each entry+ * carrying both the deed and the line the user reads before agreeing to it.+ * The two live together on purpose — a confirmation screen that enumerates by+ * hand is a confirmation screen that goes stale, and a reset that deletes+ * something the user was not shown is worse than no reset at all.+ *+ * What is *not* here, and why:+ *+ * * the APK's own files, its caches and `cacheDir` — the system owns those and+ * clears them on its own schedule (the camera's `look_*.jpg` are the only+ * thing we put there);+ * * the accessibility service's enabled state and its runtime permissions —+ * those belong to the system settings app, not to us, and a device whose+ * assistant has just lost its permissions *and* its configuration is much+ * harder to bring back than one that only lost the configuration.+ *+ * Nothing here is reversible: there is no backup, and the encrypted blobs+ * ([Notebook], [Vault]) cannot be read once their prefs are gone — the+ * Keystore key that would open them is deleted with the app, and clearing the+ * blob leaves nothing to open anyway.+ */+object FactoryReset {++ private const val TAG = "Body"++ /**+ * One thing that gets wiped.+ *+ * @param label string resource for the order line in the confirmation.+ * A resource id rather than two strings, so a third language is a file+ * and not an edit here — see [net.kuafuai.andee.i18n.AppLocale].+ * @param clear the deed. Failures are logged and skipped rather than+ * aborting the rest: a reset that stops halfway because one directory+ * was already gone is worse than one that finishes and says so.+ */+ private class Target(+ @StringRes val label: Int,+ val clear: (Context) -> Unit,+ )++ private val TARGETS: List<Target> = listOf(+ Target(R.string.factory_reset_target_keys) { prefs(it, "voice_prefs") },++ Target(R.string.factory_reset_target_notebook) { Notebook.wipe(it) },++ Target(R.string.factory_reset_target_vault) { Vault.wipe(it) },++ Target(R.string.factory_reset_target_wake) {+ WakeTemplates.clear(it)+ WakeTemplates.invalidate()+ },++ Target(R.string.factory_reset_target_history) { ChatHistory.wipe(it) },++ Target(R.string.factory_reset_target_web) { webviewData() },++ Target(R.string.factory_reset_target_grounding) { grounding(it) },+ )++ /**+ * The order lines, for the confirmation screen.+ *+ * Takes a Context rather than a `english: Boolean` flag: the flag existed+ * only to pick between two hardcoded strings here, and it cannot be+ * extended to a third language without touching every call site. The caller+ * passes the *localised* Context — see [net.kuafuai.andee.ui.SettingsUi.lctx].+ */+ fun items(context: Context): List<String> = TARGETS.map { context.getString(it.label) }++ /**+ * Run every target. Returns how many were cleared, which is all the caller+ * needs to log; the per-target outcome is in logcat.+ */+ fun wipe(context: Context): Int {+ // First, and before the notebook goes: the alarms are keyed by todo id+ // and those ids live in the file we are about to erase. Cancel them+ // afterwards and a promise would fire into a device that no longer+ // remembers making it — a notification for something nobody can find+ // any trace of.+ Scheduler.cancelSweep(context)+ for (todo in Notebook.todos(context, "all")) Scheduler.cancelTodo(context, todo.id)++ var cleared = 0+ for (t in TARGETS) {+ runCatching { t.clear(context) }+ .onSuccess {+ cleared+++ Log.i(TAG, "factory reset: cleared ${AppLocale.str(context, t.label)}")+ }+ .onFailure {+ Log.w(TAG, "factory reset: could not clear '${AppLocale.str(context, t.label)}'", it)+ }+ }+ Log.i(TAG, "factory reset: $cleared/${TARGETS.size} targets cleared")+ return cleared+ }++ // ---- the individual deeds ----++ private fun prefs(context: Context, name: String) {+ context.getSharedPreferences(name, Context.MODE_PRIVATE).edit().clear().commit()+ }++ /**+ * What the page renderer left behind.+ *+ * `HtmlActivity` loads every composed page as a string against its own base+ * URL (`https://brain.local/`) with DOM storage and JavaScript on — which+ * makes it a real origin: a page can write cookies and localStorage, and+ * both survive in `app_webview/`.+ *+ * Cleared through WebView's own API rather than by deleting that directory.+ * A recursive delete would race whatever the renderer happens to be doing,+ * and what is left in there afterwards (`variations_seed`, `pref_store`,+ * the GPU cache) is WebView's own machinery rather than the user's.+ *+ * Two things deliberately left alone, both because they are not ours:+ * the HTTP cache (clearing it needs a live `WebView` instance, and standing+ * up all of Chromium from a service to delete fetched bytes is a worse+ * trade than the bytes), and `files/profileInstalled` (written by the+ * platform's baseline-profile installer — checked, nothing in this repo+ * writes it, and it carries no user data).+ */+ private fun webviewData() {+ android.webkit.CookieManager.getInstance().removeAllCookies(null)+ android.webkit.WebStorage.getInstance().deleteAllData()+ }++ /**+ * The grounding log lives on the *external* app dir (it is meant to be+ * pulled off with adb), so it needs a real recursive delete rather than a+ * prefs clear. `commit()` above and `deleteRecursively()` here both report+ * failure honestly — the caller logs which ones did not take.+ *+ * Not [GroundingLog.shutdown]: that closes the writer until the next+ * service start, and the switch going off in prefs is already enough to+ * stop it writing. A log file recreated by a note already in flight is+ * harmless; a log that stays shut after the user turns grounding back on+ * is not.+ */+ private fun grounding(context: Context) {+ val root = File(context.getExternalFilesDir(null), "grounding")+ if (root.exists() && !root.deleteRecursively()) {+ throw IllegalStateException("grounding dir survived deleteRecursively: $root")+ }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/config/Notebook.kt b/app/src/main/java/net/kuafuai/andee/config/Notebook.ktnew file mode 100644index 0000000..c66baf8--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/config/Notebook.kt@@ -0,0 +1,743 @@+package net.kuafuai.andee.config++import android.content.Context+import android.security.keystore.KeyGenParameterSpec+import android.security.keystore.KeyProperties+import android.util.Base64+import android.util.Log+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import org.json.JSONArray+import org.json.JSONObject+import java.security.KeyStore+import java.text.SimpleDateFormat+import java.util.Date+import java.util.Locale+import javax.crypto.Cipher+import javax.crypto.KeyGenerator+import javax.crypto.SecretKey+import javax.crypto.spec.GCMParameterSpec++/**+ * The tablet's own notebook — what it remembers about this person, and what it+ * promised to do later.+ *+ * Two things live here that used to live only in the cloud brain (agentworld's+ * `crm_*` tools): durable facts about the user, and a scheduler that can wake+ * the brain up at a future moment. This device has neither a database nor a+ * cron service, so the notebook is one encrypted JSON blob in SharedPreferences+ * plus [Scheduler] on top of AlarmManager.+ *+ * Storage is a hand-rolled AES/GCM blob whose key lives in the Android+ * Keystore — deliberately the same shape as [Vault], for the same reasons:+ * no dependency, and the key material dies with the app. It is a separate+ * prefs file and a separate key alias on purpose; a corrupted notebook must+ * never be able to take the passwords with it.+ *+ * **This is a private notebook.** Its tools are marked `localOnly` in+ * [net.kuafuai.andee.net.ToolSchemas] and are filtered out of the hub+ * `register` payload — the person's habits and promises do not leave the+ * device. There is no cloud copy and no export path; the only way a human+ * sees this is the read-only viewer in the settings screen.+ *+ * Field names track agentworld's memory model (`name` / `description` /+ * `content` / `type` / `importance` / `source` / `do_not_do` /+ * `failure_signature` / `valid_until` / `layer`) so the two can be compared+ * later. But only four of them are ever typed by the model — see+ * [importanceFor]; everything else is bookkeeping this file fills in itself.+ */+object Notebook {++ private const val TAG = "Body"+ private const val PREFS = "notebook_prefs"+ private const val META_PREFS = "notebook_meta_prefs"+ private const val KEY_BLOB = "book"+ private const val KEY_ALIAS = "body_notebook_key"+ private const val KEY_LAST_ACTIVITY = "last_activity_at"+ private const val KEY_SWEEP_DAY = "sweep_day"+ private const val KEY_SWEEP_COUNT = "sweep_count"+ private const val KEY_SWEEP_TOKENS = "sweep_tokens"+ private const val GCM_TAG_BITS = 128+ private const val IV_BYTES = 12++ /**+ * Refuse new memories past this. Not a token budget — the context side is+ * already capped by [indexForPrompt] and by recall's `limit`. This is only+ * a backstop so a model looping on `remember` cannot grow the blob without+ * bound. At the cap, writes are refused *loudly* (the tool returns an+ * error) rather than silently evicting something.+ */+ const val MAX_MEMORIES = 500++ /** Old versions kept per memory when the same `name` is written again. */+ private const val MAX_HISTORY = 5++ /** Finished one-shot todos are kept this long, then dropped on the next write. */+ private const val DONE_TODO_TTL_MS = 30L * 24 * 60 * 60 * 1000++ /** Ceiling on closed-todo tombstones, newest kept. See [prune]. */+ private const val MAX_CLOSED_TODOS = 50++ val TYPES = listOf("user", "feedback", "way", "reference", "failure")++ /**+ * Where a memory came from — the difference that will matter later is+ * `human` (the user said it) versus `agent_inference` (the brain worked it+ * out during a 复盘). Nothing reads this yet; it is written now so that a+ * later "trust the user's own words over its own guesses" rule has the+ * data to stand on.+ */+ const val SOURCE_HUMAN = "human"+ const val SOURCE_INFERRED = "agent_inference"+ const val SOURCE_FAILURE_REVIEW = "failure_review"++ // ------------------------------------------------------------------+ // models+ // ------------------------------------------------------------------++ data class Memory(+ val name: String,+ val description: String,+ val content: String,+ val type: String,+ val importance: Double,+ val source: String,+ val doNotDo: List<String>,+ val failureSignature: String,+ val validUntil: String,+ val layer: String,+ val createdAt: Long,+ val updatedAt: Long,+ val hits: Int,+ val lastUsedAt: Long,+ val history: List<String>,+ ) {+ /** Expired memories stay on disk but are never served. See [validUntil]. */+ fun expired(now: Long = System.currentTimeMillis()): Boolean {+ if (validUntil.isBlank()) return false+ return runCatching {+ val fmt = SimpleDateFormat("yyyy-MM-dd", Locale.US)+ (fmt.parse(validUntil)?.time ?: Long.MAX_VALUE) + 86_400_000L <= now+ }.getOrDefault(false)+ }+ }++ data class Todo(+ val id: String,+ val what: String,+ /** `once` or `cron`. */+ val mode: String,+ /** Absolute millis, one-shot only. 0 for cron rows. */+ val at: Long,+ /** Standard 5-field expression, cron rows only. */+ val cronExpr: String,+ val nextRunAt: Long,+ val lastRunAt: Long,+ val lastStatus: String,+ val lastError: String,+ /** `active` / `paused` / `done` / `dropped`. */+ val status: String,+ val createdAt: Long,+ )++ // ------------------------------------------------------------------+ // process-wide cache+ // ------------------------------------------------------------------++ /**+ * Decrypting a few hundred KB of JSON on every turn to render an index of+ * ten lines would be silly, and the sweep timer reads this file on a+ * background thread while the loop reads it on its own. One cached copy,+ * invalidated by every write, keeps that simple — the file is small and+ * single-process, so there is no cross-process staleness to worry about.+ */+ @Volatile+ private var cached: JSONObject? = null++ private fun book(context: Context): JSONObject {+ cached?.let { return it }+ synchronized(this) {+ cached?.let { return it }+ val fresh = readBook(context)+ cached = fresh+ return fresh+ }+ }++ private fun readBook(context: Context): JSONObject {+ val blob = prefs(context).getString(KEY_BLOB, null) ?: return emptyBook()+ val plain = decrypt(blob) ?: return emptyBook()+ return runCatching { JSONObject(plain) }.getOrElse {+ Log.e(TAG, "notebook: blob is not valid JSON, starting empty", it)+ emptyBook()+ }+ }++ private fun emptyBook(): JSONObject = JSONObject()+ .put("memories", JSONArray())+ .put("todos", JSONArray())+ .put("meta", JSONObject())++ private fun write(context: Context, next: JSONObject) {+ prune(next)+ val blob = encrypt(next.toString())+ if (blob == null) {+ // Never report a save that did not happen — see the same guard in+ // Vault.write.+ Log.e(TAG, "notebook: encrypt failed, nothing written")+ return+ }+ prefs(context).edit().putString(KEY_BLOB, blob).apply()+ cached = next+ }++ /**+ * Drop what has aged out. Called on every write, so it costs nothing extra+ * — this is the *only* housekeeping the notebook does; nothing is evicted+ * for being unimportant or unused.+ *+ * Closed todos are the one thing that accumulates without ever being asked+ * for: every `done` / `drop` leaves a row behind (deliberately — "I did+ * that one" is worth being able to see). Two limits keep them from becoming+ * the whole book: age, and a ceiling on how many tombstones are kept.+ *+ * Note the stamp: a row that was dropped without ever firing has no+ * `last_run_at`, and reading only that field meant such rows were *never*+ * pruned. Fall back to when it was created.+ */+ private fun prune(b: JSONObject) {+ val cutoff = System.currentTimeMillis() - DONE_TODO_TTL_MS+ val kept = JSONArray()+ val closed = mutableListOf<Pair<Long, JSONObject>>()+ val todos = b.optJSONArray("todos") ?: JSONArray()+ for (i in 0 until todos.length()) {+ val t = todos.optJSONObject(i) ?: continue+ val isClosed = t.optString("status") in listOf("done", "dropped")+ if (!isClosed) {+ kept.put(t)+ continue+ }+ val stamp = t.optLong("last_run_at").takeIf { it > 0 } ?: t.optLong("created_at")+ if (stamp in 1 until cutoff) continue+ closed += stamp to t+ }+ // Newest tombstones survive; a tombstone is only interesting while it is+ // recent enough to be recognised. The rest are dropped.+ closed.sortedByDescending { it.first }+ .take(MAX_CLOSED_TODOS)+ .forEach { (_, t) -> kept.put(t) }+ b.put("todos", kept)+ }++ // ------------------------------------------------------------------+ // memories+ // ------------------------------------------------------------------++ /**+ * Importance is derived, never asked for.+ *+ * The cloud's `save_memory` takes an explicit 0–1 float. Here the model+ * types four fields and gets on with it: a personal notebook's entries are+ * far less differentiated than a dev agent's (which distinguishes "RLS+ * rule" from "temp note"), and asking for a number invites the model to+ * spend tokens on a decision nobody downstream can check.+ */+ fun importanceFor(type: String): Double = when (type) {+ "failure" -> 0.9+ "user" -> 0.7+ "feedback" -> 0.7+ "way" -> 0.6+ else -> 0.5+ }++ fun memories(context: Context): List<Memory> = memories(book(context))++ private fun memories(b: JSONObject): List<Memory> {+ val arr = b.optJSONArray("memories") ?: return emptyList()+ return (0 until arr.length()).mapNotNull { i -> arr.optJSONObject(i)?.let(::toMemory) }+ }++ private fun toMemory(o: JSONObject): Memory = Memory(+ name = o.optString("name"),+ description = o.optString("description"),+ content = o.optString("content"),+ type = o.optString("type", "user"),+ importance = o.optDouble("importance", 0.5),+ source = o.optString("source", SOURCE_HUMAN),+ doNotDo = o.optJSONArray("do_not_do")?.let { a ->+ (0 until a.length()).mapNotNull { a.optString(it).ifBlank { null } }+ } ?: emptyList(),+ failureSignature = o.optString("failure_signature"),+ validUntil = o.optString("valid_until"),+ layer = o.optString("layer", "user"),+ createdAt = o.optLong("created_at"),+ updatedAt = o.optLong("updated_at"),+ hits = o.optInt("hits", 0),+ lastUsedAt = o.optLong("last_used_at"),+ history = o.optJSONArray("history")?.let { a ->+ (0 until a.length()).mapNotNull { a.optString(it).ifBlank { null } }+ } ?: emptyList(),+ )++ private fun toJson(m: Memory): JSONObject {+ val arr = JSONArray()+ for (h in m.history) arr.put(h)+ val dnd = JSONArray()+ for (d in m.doNotDo) dnd.put(d)+ return JSONObject()+ .put("name", m.name)+ .put("description", m.description)+ .put("content", m.content)+ .put("type", m.type)+ .put("importance", m.importance)+ .put("source", m.source)+ .put("do_not_do", dnd)+ .put("failure_signature", m.failureSignature)+ .put("valid_until", m.validUntil)+ .put("layer", m.layer)+ .put("created_at", m.createdAt)+ .put("updated_at", m.updatedAt)+ .put("hits", m.hits)+ .put("last_used_at", m.lastUsedAt)+ .put("history", arr)+ }++ /**+ * Insert or overwrite by [name]. Overwriting is the *only* revision path+ * the model has: it is how "I don't like spicy food any more" replaces+ * "likes spicy food", and how a playbook improves in place. The previous+ * version is pushed into `history` rather than dropped, so an overwrite+ * stays auditable even though nobody can see it from the UI.+ *+ * Throws IllegalArgumentException on a bad [type] or an empty [name] — the+ * caller turns that into an error the model can read and retry.+ */+ fun remember(+ context: Context,+ name: String,+ description: String,+ content: String,+ type: String,+ source: String = SOURCE_HUMAN,+ ): Pair<Memory, Boolean> {+ val key = name.trim()+ require(key.isNotEmpty()) { "name cannot be empty" }+ val n = key.lowercase()+ val t = type.trim().lowercase()+ require(t in TYPES) { "type must be one of ${TYPES.joinToString(" / ")}" }++ val b = book(context).let { JSONObject(it.toString()) }+ val arr = b.optJSONArray("memories") ?: JSONArray()+ val now = System.currentTimeMillis()++ var found: JSONObject? = null+ var overwrote = false+ var index = -1+ for (i in 0 until arr.length()) {+ val o = arr.optJSONObject(i) ?: continue+ if (o.optString("name").lowercase() == n) {+ found = o+ index = i+ overwrote = true+ break+ }+ }+ if (found == null && arr.length() >= MAX_MEMORIES) {+ throw IllegalStateException(+ "The notebook is full ($MAX_MEMORIES entries) — forget one with `forget` " ++ "before saving another.",+ )+ }++ val body = content.trim().ifEmpty { description.trim() }+ // Newest first, capped: an overwrite chain is a changelog, not an archive.+ val hist = JSONArray()+ if (found != null) {+ val old = toMemory(found)+ val stamp = SimpleDateFormat("MM-dd HH:mm", Locale.US).format(Date(old.updatedAt))+ hist.put("[$stamp] ${old.description}")+ for (h in old.history) if (hist.length() < MAX_HISTORY) hist.put(h)+ }++ val merged = Memory(+ name = key,+ description = description.trim(),+ content = body,+ type = t,+ importance = importanceFor(t),+ source = if (found != null) found.optString("source", source) else source,+ doNotDo = found?.let { toMemory(it).doNotDo } ?: emptyList(),+ failureSignature = found?.optString("failure_signature") ?: "",+ validUntil = found?.optString("valid_until") ?: "",+ layer = "user",+ createdAt = found?.optLong("created_at") ?: now,+ updatedAt = now,+ hits = found?.optInt("hits", 0) ?: 0,+ lastUsedAt = found?.optLong("last_used_at") ?: 0L,+ history = (0 until hist.length()).mapNotNull { hist.optString(it).ifBlank { null } },+ )+ if (index >= 0) arr.put(index, toJson(merged)) else arr.put(toJson(merged))+ b.put("memories", arr)+ write(context, b)+ return merged to overwrote+ }++ /**+ * Keyword search, newest-and-most-important first.+ *+ * Substring matching is enough at this scale (tens to hundreds of entries)+ * and it is honest about its limits: no stemming, no synonyms, no+ * embeddings. What it does have is a bias toward *recently used* entries,+ * which is what makes a small notebook feel like it knows you.+ */+ fun recall(context: Context, query: String, type: String, limit: Int): List<Memory> {+ val q = query.trim().lowercase()+ val want = type.trim().lowercase()+ val now = System.currentTimeMillis()+ val all = memories(context).filter { !it.expired(now) }+ .filter { want.isBlank() || it.type == want }+ val hits = if (q.isEmpty()) {+ all+ } else {+ all.filter {+ it.name.lowercase().contains(q) ||+ it.description.lowercase().contains(q) ||+ it.content.lowercase().contains(q)+ }+ }+ val ranked = hits.sortedWith(+ compareByDescending<Memory> { it.importance }+ .thenByDescending { it.updatedAt },+ )+ val take = ranked.take(limit.coerceIn(1, 30))+ if (take.isNotEmpty()) bumpHits(context, take.map { it.name })+ return take+ }++ private fun bumpHits(context: Context, names: List<String>) {+ val wanted = names.map { it.lowercase() }.toSet()+ val b = book(context).let { JSONObject(it.toString()) }+ val arr = b.optJSONArray("memories") ?: return+ val now = System.currentTimeMillis()+ var touched = false+ for (i in 0 until arr.length()) {+ val o = arr.optJSONObject(i) ?: continue+ if (o.optString("name").lowercase() !in wanted) continue+ o.put("hits", o.optInt("hits", 0) + 1)+ o.put("last_used_at", now)+ touched = true+ }+ if (touched) write(context, b)+ }++ /** Hard delete by name. The model's `forget`. Returns false when absent. */+ fun forget(context: Context, name: String): Boolean {+ val n = name.trim().lowercase()+ if (n.isEmpty()) return false+ val b = book(context).let { JSONObject(it.toString()) }+ val arr = b.optJSONArray("memories") ?: return false+ val kept = JSONArray()+ var hit = false+ for (i in 0 until arr.length()) {+ val o = arr.optJSONObject(i) ?: continue+ if (o.optString("name").lowercase() == n) {+ hit = true+ continue+ }+ kept.put(o)+ }+ if (!hit) return false+ b.put("memories", kept)+ write(context, b)+ return true+ }++ /**+ * Erase the book, and forget it was ever read.+ *+ * The in-process cache is the part that is easy to miss: clearing prefs+ * alone would leave the decrypted copy sitting in [cached], so the next+ * `recall` would keep answering out of a notebook that no longer exists on+ * disk. That is a factory reset that looks like it worked and did not, and+ * nobody would notice until much later.+ *+ * `commit()`, not `apply()`: the caller is about to report success and the+ * device may lose power a second later. This is the one write in this file+ * where the difference matters.+ */+ fun wipe(context: Context) {+ prefs(context).edit().clear().commit()+ metaPrefs(context).edit().clear().commit()+ cached = null+ }++ /**+ * The index that rides along with every turn: `name — description`, one+ * line each.+ *+ * Nothing calls this yet, and that is deliberate. It is the P3 step, and+ * it lands on the **user message**, not in the system prompt: `LocalPrompt`+ * is a constant on purpose so DeepSeek's prefix cache survives, and the+ * system message is the head of the request — anything injected there+ * invalidates the whole conversation behind it. The clock already rides in+ * on the user message for exactly this reason.+ */+ fun indexForPrompt(context: Context, max: Int = 10): String {+ val now = System.currentTimeMillis()+ val top = memories(context).filter { !it.expired(now) }+ .sortedWith(+ compareByDescending<Memory> { it.importance }+ .thenByDescending { it.lastUsedAt }+ .thenByDescending { it.updatedAt },+ )+ .take(max)+ if (top.isEmpty()) return ""+ return top.joinToString("\n") { "- ${it.name}:${it.description}" }+ }++ /**+ * Plain-text view for the settings screen. Read-only, and the only way out.+ *+ * **Labels are translated; the entries are not.** What is appended from+ * [Memory] / [Todo] — a name, a description, a cron expression, a+ * timestamp — is stored data and is printed exactly as written. Only the+ * scaffolding around it (headings, "expired", "source", separators) comes+ * from resources, which is why this builds the string in pieces instead of+ * one big template: the data would be un-escapable inside a resource.+ *+ * The signature stays `(Context)` on purpose. The caller passes whatever+ * Context it has and the wrap happens here, so no call site can forget it+ * and there is one place to look when a label comes out in the wrong+ * language. See [AppLocale] for why a Service's Context cannot simply be+ * used for the strings.+ */+ fun dumpText(context: Context): String {+ val ctx = AppLocale.wrap(context)+ val now = System.currentTimeMillis()+ val fmt = SimpleDateFormat("yyyy-MM-dd HH:mm", Locale.US)+ val sb = StringBuilder()+ val mem = memories(context)+ sb.append(ctx.getString(R.string.notebook_dump_header, mem.size))+ sb.append(ctx.getString(R.string.notebook_dump_sort_note))+ if (mem.isEmpty()) {+ sb.append(ctx.getString(R.string.notebook_dump_empty_memories))+ }+ for (m in mem.sortedWith(compareByDescending<Memory> { it.updatedAt })) {+ sb.append("· ").append(m.name).append(" [").append(m.type).append("]")+ if (m.expired(now)) sb.append(ctx.getString(R.string.notebook_dump_expired))+ sb.append('\n')+ sb.append(" ").append(m.description).append('\n')+ if (m.content.isNotBlank() && m.content != m.description) {+ sb.append(" ").append(m.content.replace("\n", "\n ")).append('\n')+ }+ sb.append(" ").append(fmt.format(Date(m.updatedAt)))+ .append(ctx.getString(R.string.notebook_dump_source)).append(m.source)+ .append(ctx.getString(R.string.notebook_dump_hits, m.hits))+ }+ val todos = todos(context, "all")+ sb.append(ctx.getString(R.string.notebook_dump_todos_header, todos.size))+ if (todos.isEmpty()) sb.append(ctx.getString(R.string.notebook_dump_empty_todos))+ for (t in todos.sortedBy { it.nextRunAt }) {+ sb.append("· ").append(t.what).append(" [").append(t.status).append("]\n")+ sb.append(" ")+ .append(+ if (t.mode == "cron") {+ ctx.getString(R.string.notebook_dump_recurring, t.cronExpr)+ } else {+ fmt.format(Date(t.at))+ }+ )+ .append(ctx.getString(R.string.notebook_dump_next))+ .append(if (t.nextRunAt > 0) fmt.format(Date(t.nextRunAt)) else "—")+ .append('\n')+ if (t.lastError.isNotBlank()) {+ sb.append(ctx.getString(R.string.notebook_dump_last_error))+ .append(t.lastError).append('\n')+ }+ sb.append('\n')+ }+ return sb.toString()+ }++ // ------------------------------------------------------------------+ // todos+ // ------------------------------------------------------------------++ fun todos(context: Context, filter: String): List<Todo> {+ val arr = book(context).optJSONArray("todos") ?: return emptyList()+ val all = (0 until arr.length()).mapNotNull { i -> arr.optJSONObject(i)?.let(::toTodo) }+ return when (filter) {+ "all" -> all+ "cron" -> all.filter { it.mode == "cron" && it.status == "active" }+ else -> all.filter { it.status == "active" || it.status == "paused" }+ }+ }++ fun todoById(context: Context, id: String): Todo? {+ val arr = book(context).optJSONArray("todos") ?: return null+ for (i in 0 until arr.length()) {+ val o = arr.optJSONObject(i) ?: continue+ if (o.optString("id") == id) return toTodo(o)+ }+ return null+ }++ private fun toTodo(o: JSONObject): Todo = Todo(+ id = o.optString("id"),+ what = o.optString("what"),+ mode = o.optString("mode", "once"),+ at = o.optLong("at"),+ cronExpr = o.optString("cron_expr"),+ nextRunAt = o.optLong("next_run_at"),+ lastRunAt = o.optLong("last_run_at"),+ lastStatus = o.optString("last_status"),+ lastError = o.optString("last_error"),+ status = o.optString("status", "active"),+ createdAt = o.optLong("created_at"),+ )++ private fun toJson(t: Todo): JSONObject = JSONObject()+ .put("id", t.id)+ .put("what", t.what)+ .put("mode", t.mode)+ .put("at", t.at)+ .put("cron_expr", t.cronExpr)+ .put("next_run_at", t.nextRunAt)+ .put("last_run_at", t.lastRunAt)+ .put("last_status", t.lastStatus)+ .put("last_error", t.lastError)+ .put("status", t.status)+ .put("created_at", t.createdAt)++ fun newTodoId(): String = "t" + System.currentTimeMillis().toString(36)++ fun addTodo(context: Context, todo: Todo) {+ val b = book(context).let { JSONObject(it.toString()) }+ val arr = b.optJSONArray("todos") ?: JSONArray()+ arr.put(toJson(todo))+ b.put("todos", arr)+ write(context, b)+ }++ /** Merge-patch one todo. Null when the id is unknown. */+ fun patchTodo(+ context: Context,+ id: String,+ status: String? = null,+ nextRunAt: Long? = null,+ lastRunAt: Long? = null,+ lastStatus: String? = null,+ lastError: String? = null,+ ): Todo? {+ val b = book(context).let { JSONObject(it.toString()) }+ val arr = b.optJSONArray("todos") ?: return null+ for (i in 0 until arr.length()) {+ val o = arr.optJSONObject(i) ?: continue+ if (o.optString("id") != id) continue+ if (status != null) o.put("status", status)+ if (nextRunAt != null) o.put("next_run_at", nextRunAt)+ if (lastRunAt != null) o.put("last_run_at", lastRunAt)+ if (lastStatus != null) o.put("last_status", lastStatus)+ if (lastError != null) o.put("last_error", lastError)+ b.put("todos", arr)+ write(context, b)+ return toTodo(o)+ }+ return null+ }++ // ------------------------------------------------------------------+ // meta: activity stamp + sweep budget+ //+ // Plain prefs, deliberately NOT in the encrypted blob. These are touched+ // on every turn (the activity stamp) and are not secret — none of them+ // says anything about the user. Routing them through the notebook would+ // mean re-encrypting the whole book on every single message.+ // ------------------------------------------------------------------++ /** When the user last said something. */+ fun touchActivity(context: Context) {+ metaPrefs(context).edit()+ .putLong(KEY_LAST_ACTIVITY, System.currentTimeMillis())+ .apply()+ }++ fun lastActivityAt(context: Context): Long = metaPrefs(context).getLong(KEY_LAST_ACTIVITY, 0L)++ private fun today(): String =+ SimpleDateFormat("yyyy-MM-dd", Locale.US).format(Date())++ /** True when today's sweep count is still under [cap]. */+ fun sweepAllowed(context: Context, cap: Int): Boolean = sweepCountToday(context) < cap++ fun sweepCountToday(context: Context): Int {+ val p = metaPrefs(context)+ return if (p.getString(KEY_SWEEP_DAY, "") == today()) p.getInt(KEY_SWEEP_COUNT, 0) else 0+ }++ fun sweepTokensToday(context: Context): Int {+ val p = metaPrefs(context)+ return if (p.getString(KEY_SWEEP_DAY, "") == today()) p.getInt(KEY_SWEEP_TOKENS, 0) else 0+ }++ fun noteSweep(context: Context, tokens: Int) {+ val day = today()+ val count = sweepCountToday(context)+ val toks = sweepTokensToday(context)+ metaPrefs(context).edit()+ .putString(KEY_SWEEP_DAY, day)+ .putInt(KEY_SWEEP_COUNT, count + 1)+ .putInt(KEY_SWEEP_TOKENS, toks + tokens)+ .apply()+ }++ private fun metaPrefs(context: Context) =+ context.getSharedPreferences(META_PREFS, Context.MODE_PRIVATE)++ // ---- crypto ----------------------------------------------------+ // Same construction as Vault: base64(iv || ciphertext), key in the+ // Android Keystore, no user-authentication requirement (a notebook write+ // happens mid-conversation, not at the lock screen).++ private fun prefs(context: Context) =+ context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)++ private fun secretKey(): SecretKey {+ val ks = KeyStore.getInstance("AndroidKeyStore").apply { load(null) }+ (ks.getEntry(KEY_ALIAS, null) as? KeyStore.SecretKeyEntry)?.let { return it.secretKey }+ val gen = KeyGenerator.getInstance(KeyProperties.KEY_ALGORITHM_AES, "AndroidKeyStore")+ gen.init(+ KeyGenParameterSpec.Builder(+ KEY_ALIAS,+ KeyProperties.PURPOSE_ENCRYPT or KeyProperties.PURPOSE_DECRYPT,+ )+ .setBlockModes(KeyProperties.BLOCK_MODE_GCM)+ .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_NONE)+ .build()+ )+ return gen.generateKey()+ }++ private fun encrypt(plain: String): String? = runCatching {+ val c = Cipher.getInstance("AES/GCM/NoPadding")+ c.init(Cipher.ENCRYPT_MODE, secretKey())+ val body = c.doFinal(plain.toByteArray(Charsets.UTF_8))+ Base64.encodeToString(c.iv + body, Base64.NO_WRAP)+ }.getOrNull()++ private fun decrypt(blob: String): String? = runCatching {+ val raw = Base64.decode(blob, Base64.NO_WRAP)+ val c = Cipher.getInstance("AES/GCM/NoPadding")+ c.init(+ Cipher.DECRYPT_MODE,+ secretKey(),+ GCMParameterSpec(GCM_TAG_BITS, raw, 0, IV_BYTES),+ )+ String(c.doFinal(raw, IV_BYTES, raw.size - IV_BYTES), Charsets.UTF_8)+ }.getOrElse {+ // A restored backup or a wiped Keystore leaves an undecryptable blob.+ // Same trap as Vault: from the outside it looks like an empty notebook.+ Log.e(TAG, "notebook: decrypt failed — the notebook is unreadable on this install", it)+ null+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/config/Scheduler.kt b/app/src/main/java/net/kuafuai/andee/config/Scheduler.ktnew file mode 100644index 0000000..8f38f4d--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/config/Scheduler.kt@@ -0,0 +1,320 @@+package net.kuafuai.andee.config++import android.app.AlarmManager+import android.app.NotificationChannel+import android.app.NotificationManager+import android.app.PendingIntent+import android.content.Context+import android.content.Intent+import android.util.Log+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import java.text.SimpleDateFormat+import java.util.Date+import java.util.Locale++/**+ * Everything that has to happen "later": the pending todos and the quiet-hour+ * sweep timer.+ *+ * The cloud brain had a cron service polling a database. This device has no+ * process to poll with, so the clock is `AlarmManager` — and, crucially, the+ * only thing an alarm can do here is *wake the app up*; the work itself is+ * done by [net.kuafuai.andee.brain.LocalBrain] once it is awake. That is why+ * this file contains no decisions: it arms, it cancels, it re-arms, and it+ * hands a firing alarm to [ScreenBodyService] to deal with.+ *+ * Two deliberate simplifications:+ *+ * * **No exact-alarm permission is ever requested** (decision B). If the user+ * happens to have granted it, we use it; otherwise `setAndAllowWhileIdle`+ * still fires under Doze but may be a few minutes late. The tool result+ * says which one it got, and the prompt forbids claiming punctuality.+ * * **Repeating todos re-arm themselves on every fire** rather than being+ * handed to AlarmManager as a repeating alarm. This is the same shape as+ * the cloud's `next_run_at` column, and it means a missed occurrence is+ * something we can see and report instead of a schedule that silently+ * drifts.+ */+object Scheduler {++ private const val TAG = "Body"++ const val ACTION_TODO = "net.kuafuai.andee.action.TODO_DUE"+ const val ACTION_SWEEP = "net.kuafuai.andee.action.SWEEP"+ const val ACTION_BOOT = "net.kuafuai.andee.action.BOOT"++ const val EXTRA_ID = "todo_id"++ /** One fixed request code: only one sweep timer exists at a time. */+ private const val SWEEP_REQUEST_CODE = 0x5EEB++ /**+ * The channel **id**, and it must never change.+ *+ * Android keys a channel by its id, so a different id is a *different*+ * channel — changing this string would leave a device with two "promises"+ * channels in settings instead of renaming the one it has. The id is+ * deliberately opaque and untranslated; the name shown to the user is+ * [channelName].+ */+ private const val NOTIFY_CHANNEL = "andee_promises"++ private const val NOTIFY_BASE = 4100++ /**+ * The channel name the user sees, in their language.+ *+ * **Known Android platform limitation, not a bug in this app.** A+ * notification channel's name is written into the system the *first* time+ * the channel is created, and from then on the system owns its own copy.+ * Calling `createNotificationChannel` again with a new name does not+ * rename it, and there is no API that does. So on a device where the+ * channel already exists, **switching the app language leaves the old+ * language's name showing** in system settings until the user clears the+ * app's data or uninstalls and reinstalls. Only a fresh install (or a+ * factory reset, which clears the app's data) picks up the new language.+ * This is why the name has to be read at creation time rather than baked+ * into a constant.+ */+ fun channelName(context: Context): String =+ AppLocale.str(context, R.string.sched_channel_name)++ fun am(context: Context): AlarmManager =+ context.getSystemService(Context.ALARM_SERVICE) as AlarmManager++ fun exactAllowed(context: Context): Boolean = runCatching {+ am(context).canScheduleExactAlarms()+ }.getOrDefault(false)++ // ------------------------------------------------------------------+ // todos+ // ------------------------------------------------------------------++ /**+ * Arm the next occurrence of [todo]. No-op (and cancels) when the todo has+ * no future run left — a fired one-shot, a paused row, a dropped one.+ */+ fun armTodo(context: Context, todo: Notebook.Todo) {+ if (todo.status != "active" || todo.nextRunAt <= 0L) {+ cancelTodo(context, todo.id)+ return+ }+ val pending = todoIntent(context, todo.id)+ val trigger = todo.nextRunAt+ try {+ if (exactAllowed(context)) {+ am(context).setExactAndAllowWhileIdle(AlarmManager.RTC_WAKEUP, trigger, pending)+ } else {+ am(context).setAndAllowWhileIdle(AlarmManager.RTC_WAKEUP, trigger, pending)+ }+ Log.i(+ TAG,+ "alarm armed for todo ${todo.id} at ${formatLocal(trigger)} " ++ "(exact=${exactAllowed(context)})",+ )+ } catch (t: Throwable) {+ // SecurityException when a permission was revoked between the check+ // and the call. Losing the alarm silently would mean the promise is+ // forgotten; say so loudly instead.+ Log.e(TAG, "alarm arming failed for todo ${todo.id}", t)+ }+ }++ fun cancelTodo(context: Context, id: String) {+ runCatching {+ am(context).cancel(todoIntent(context, id))+ }+ }++ private fun todoIntent(context: Context, id: String): PendingIntent {+ val intent = Intent(context, AlarmReceiver::class.java)+ .setAction(ACTION_TODO)+ .putExtra(EXTRA_ID, id)+ // The id becomes the request code so two todos cannot overwrite each+ // other's alarm. FLAG_IMMUTABLE: nothing outside this app builds them.+ return PendingIntent.getBroadcast(+ context, id.hashCode(), intent,+ PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,+ )+ }++ // ------------------------------------------------------------------+ // sweep timer+ // ------------------------------------------------------------------++ /**+ * (Re-)arm the "has the conversation gone quiet?" timer.+ *+ * Called at the end of every turn, so it is deliberately last-write-wins:+ * a conversation that keeps going never reaches its own quiet threshold,+ * because each turn pushes the timer out again.+ */+ fun armSweepIn(context: Context, delayMs: Long) {+ val intent = Intent(context, AlarmReceiver::class.java).setAction(ACTION_SWEEP)+ val pending = PendingIntent.getBroadcast(+ context, SWEEP_REQUEST_CODE, intent,+ PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,+ )+ val at = System.currentTimeMillis() + delayMs+ runCatching {+ // A quiet-hour check is not worth an exact alarm, and asking for+ // one for this would be a permission prompt to explain away.+ am(context).setAndAllowWhileIdle(AlarmManager.RTC_WAKEUP, at, pending)+ }.onFailure { Log.w(TAG, "sweep timer arming failed", it) }+ }++ fun cancelSweep(context: Context) {+ val intent = Intent(context, AlarmReceiver::class.java).setAction(ACTION_SWEEP)+ val pending = PendingIntent.getBroadcast(+ context, SWEEP_REQUEST_CODE, intent,+ PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,+ )+ runCatching { am(context).cancel(pending) }+ }++ // ------------------------------------------------------------------+ // catch-up+ // ------------------------------------------------------------------++ /**+ * Bring the stored todos back in line with the clock, and return the ones+ * whose moment passed while nothing was listening.+ *+ * Runs on service start and on boot. Two cases:+ *+ * * an occurrence fired while the app was dead → `last_status = missed`,+ * and for a repeating todo the *next* occurrence is computed from now+ * (missed runs are not replayed — a backlog of "it's 8am!" at 3pm is+ * noise, not service);+ * * a todo is due in the future but has no live alarm (device rebooted,+ * app reinstalled, alarm lost) → re-armed.+ *+ * The returned list is what the caller turns into either a silent note to+ * the brain (service alive) or a notification (service gone).+ */+ fun catchUp(context: Context): List<Notebook.Todo> {+ val now = System.currentTimeMillis()+ val missed = mutableListOf<Notebook.Todo>()+ for (todo in Notebook.todos(context, "all")) {+ if (todo.status != "active") {+ cancelTodo(context, todo.id)+ continue+ }+ val due = todo.nextRunAt+ if (due <= 0L) continue+ if (due <= now) {+ val next = if (todo.mode == "cron" && todo.cronExpr.isNotBlank()) {+ CronExpr.nextAfter(todo.cronExpr, now)+ } else {+ null+ }+ val patched = Notebook.patchTodo(+ context,+ todo.id,+ nextRunAt = next ?: 0L,+ lastRunAt = now,+ lastStatus = "missed",+ )+ if (patched != null) missed += patched+ } else {+ armTodo(context, todo)+ }+ }+ if (missed.isNotEmpty()) {+ Log.i(TAG, "catch-up: ${missed.size} todo(s) were missed while nothing was listening")+ }+ return missed+ }++ /** Clear the `missed` flag once it has been handed to the brain or the user. */+ fun markCaughtUp(context: Context, id: String) {+ Notebook.patchTodo(context, id, lastStatus = "caught_up")+ }++ // ------------------------------------------------------------------+ // notifications+ // ------------------------------------------------------------------++ /**+ * The only thing this app can do for a promise when its brain is not+ * running. It says what was promised and that it could not act on it —+ * a notification that just said "提醒:吃药" would be a lie about who is+ * talking.+ */+ fun notifyMissed(context: Context, todo: Notebook.Todo) {+ val mgr = context.getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager+ runCatching {+ if (mgr.getNotificationChannel(NOTIFY_CHANNEL) == null) {+ mgr.createNotificationChannel(+ NotificationChannel(+ NOTIFY_CHANNEL,+ // Only reached on the first install that posts a+ // promise — see the known-limitation note on+ // [channelName] for why a later language switch cannot+ // reach a channel that already exists.+ channelName(context),+ NotificationManager.IMPORTANCE_DEFAULT,+ ).apply { description = AppLocale.str(context, R.string.sched_channel_desc) }+ )+ }+ val text = if (todo.mode == "cron") {+ AppLocale.str(context, R.string.sched_notify_recurring, todo.what)+ } else {+ // `todo.what` is the user's own words — data, printed as stored.+ todo.what+ }+ val n = android.app.Notification.Builder(context, NOTIFY_CHANNEL)+ .setSmallIcon(android.R.drawable.ic_popup_reminder)+ .setContentTitle(AppLocale.str(context, R.string.sched_notify_title))+ .setContentText(text)+ .setStyle(android.app.Notification.BigTextStyle().bigText(+ AppLocale.str(context, R.string.sched_notify_body, text)+ ))+ .setAutoCancel(true)+ .build()+ mgr.notify(NOTIFY_BASE + (todo.id.hashCode() and 0xFF), n)+ }.onFailure {+ // Almost always a missing POST_NOTIFICATIONS on Android 13+. The+ // todo stays in the notebook either way, so this is a lost nudge,+ // not lost data.+ Log.w(TAG, "could not post the missed-promise notification", it)+ }+ }++ // ------------------------------------------------------------------+ // time helpers+ // ------------------------------------------------------------------++ private val LOCAL_FORMATS = listOf(+ "yyyy-MM-dd HH:mm",+ "yyyy-MM-dd HH:mm:ss",+ "yyyy-MM-dd'T'HH:mm",+ "HH:mm",+ )++ /**+ * Parse what the model wrote. `HH:mm` is accepted and read as **today**,+ * or tomorrow when that moment has already passed — "remind me at 8" said+ * at 9pm means tomorrow morning, and making the model work that out is a+ * needless chance to get it wrong.+ */+ fun parseLocal(raw: String): Long? {+ val s = raw.trim()+ if (s.isEmpty()) return null+ if (s.matches(Regex("\\d{1,2}:\\d{2}"))) {+ val today = SimpleDateFormat("yyyy-MM-dd", Locale.US).format(Date())+ return parseLocal("$today $s")?.let { ms ->+ if (ms <= System.currentTimeMillis()) ms + 86_400_000L else ms+ }+ }+ for (pattern in LOCAL_FORMATS) {+ val fmt = SimpleDateFormat(pattern, Locale.US).apply { isLenient = false }+ runCatching { fmt.parse(s) }.getOrNull()?.let { return it.time }+ }+ return null+ }++ fun formatLocal(ms: Long): String =+ SimpleDateFormat("yyyy-MM-dd HH:mm", Locale.US).format(Date(ms))+}diff --git a/app/src/main/java/net/kuafuai/andee/config/Vault.kt b/app/src/main/java/net/kuafuai/andee/config/Vault.ktnew file mode 100644index 0000000..db803af--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/config/Vault.kt@@ -0,0 +1,285 @@+package net.kuafuai.andee.config++import android.content.Context+import android.security.keystore.KeyGenParameterSpec+import android.security.keystore.KeyProperties+import android.util.Base64+import android.util.Log+import org.json.JSONArray+import org.json.JSONObject+import java.security.KeyStore+import javax.crypto.Cipher+import javax.crypto.KeyGenerator+import javax.crypto.SecretKey+import javax.crypto.spec.GCMParameterSpec++/**+ * The user's own credentials, so the ball can log in as them.+ *+ * One entry per account — a label the brain can search for ("飞书"), the+ * identifiers it may read, and the secrets it may only *use*. The split is the+ * whole design: a secret never crosses the WebSocket. The brain asks the+ * device to type it (`device.vault.fill`) and gets back a length, not a value.+ * See the `fill_secret` description in ToolSchemas.+ *+ * An entry can also hold a bank card, and it is the same mechanism rather than+ * a second one: the number, the expiry and the CVV are three more names in+ * [Entry.secrets], typed and never returned. What it is *not* is a way to pay+ * on a native payment sheet — those keypads are the app's own and are not a+ * focused text field, so what this reaches is a checkout form or the one-time+ * card-binding page. See the `fill_secret` description for the same warning in+ * the words the brain reads.+ *+ * Storage is a JSON array encrypted with an AES/GCM key held in the Android+ * Keystore — hand-rolled rather than via androidx.security because Jetpack+ * Security Crypto is deprecated and this needs no dependency at all. The key+ * material never leaves the secure hardware and dies with the app, so an+ * uninstall is a wipe.+ *+ * Deliberately NOT in `voice_prefs`/[VoiceConfig]: that file is a flat+ * `Map<String, String>` filtered through `ALLOWED_KEYS`, which a list of+ * records cannot enter, and it is written in plaintext.+ */+object Vault {++ private const val TAG = "Body"+ private const val PREFS = "vault_prefs"+ private const val KEY_BLOB = "entries"+ private const val KEY_ALIAS = "body_vault_key"+ private const val GCM_TAG_BITS = 128+ private const val IV_BYTES = 12++ data class Entry(+ val id: String,+ val label: String,+ val phone: String = "",+ val email: String = "",+ val username: String = "",+ val note: String = "",+ val password: String = "",+ /**+ * A bank card, for the checkout form that wants one typed in.+ *+ * Three of the four are secrets, on the same rule as [password]: the+ * brain never needs to *reason* about a card number, an expiry or a+ * CVV, it only needs them in a field — so it never holds them. The+ * holder's name is the user's own name and is readable, because a form+ * that asks for it beside the number is asking for something the brain+ * may legitimately have to match against what is already on screen.+ */+ val cardNumber: String = "",+ val cardExpiry: String = "",+ val cardCvv: String = "",+ val cardHolder: String = "",+ ) {+ /** Field names the brain is allowed to read in the clear. */+ fun readable(): Map<String, String> = mapOf(+ "phone" to phone,+ "email" to email,+ "username" to username,+ "note" to note,+ "card_holder" to cardHolder,+ // Already the safe form of a number that is never readable, which+ // is why [masked] lets it through untouched. Without it two cards+ // are two identical rows and the brain has to guess which one the+ // user meant — the one question it must never answer by guessing.+ "card_last4" to cardNumber.filter(Char::isDigit).takeLast(4),+ ).filterValues { it.isNotBlank() }++ /**+ * [readable], as the brain and the card list are allowed to see it.+ *+ * Here rather than at the two call sites because `card_last4` is the+ * one field that must *not* be masked — it is four characters, so+ * [mask] would return `****` and throw away the only thing that+ * distinguishes one card from another.+ */+ fun masked(): Map<String, String> = readable().mapValues { (k, v) ->+ if (k == "card_last4") v else mask(v)+ }++ /** Field names the brain may only ask the device to type. */+ fun secrets(): List<String> = buildList {+ if (password.isNotBlank()) add("password")+ if (cardNumber.isNotBlank()) add("card_number")+ if (cardExpiry.isNotBlank()) add("card_expiry")+ if (cardCvv.isNotBlank()) add("card_cvv")+ }++ /**+ * The value behind a [secrets] name, for the one caller allowed to+ * have it: the device's own typing path.+ *+ * Null for anything not a secret of this entry, so the dispatcher can+ * tell "no such secret" from "saved but empty" without a when-chain of+ * its own that could drift out of step with [secrets].+ */+ fun secretValue(field: String): String? = when (field) {+ "password" -> password+ "card_number" -> cardNumber+ "card_expiry" -> cardExpiry+ "card_cvv" -> cardCvv+ else -> null+ }?.ifBlank { null }+ }++ fun entries(context: Context): List<Entry> {+ val blob = prefs(context).getString(KEY_BLOB, null) ?: return emptyList()+ val json = decrypt(blob) ?: return emptyList()+ return runCatching {+ val arr = JSONArray(json)+ (0 until arr.length()).map { i ->+ val o = arr.getJSONObject(i)+ Entry(+ id = o.optString("id"),+ label = o.optString("label"),+ phone = o.optString("phone"),+ email = o.optString("email"),+ username = o.optString("username"),+ note = o.optString("note"),+ password = o.optString("password"),+ cardNumber = o.optString("card_number"),+ cardExpiry = o.optString("card_expiry"),+ cardCvv = o.optString("card_cvv"),+ cardHolder = o.optString("card_holder"),+ )+ }.filter { it.id.isNotBlank() }+ }.getOrElse {+ Log.e(TAG, "vault: entries unreadable", it)+ emptyList()+ }+ }++ /** Insert or replace by [Entry.id]. */+ fun put(context: Context, entry: Entry) {+ val next = entries(context).filterNot { it.id == entry.id } + entry+ write(context, next)+ }++ fun remove(context: Context, id: String) {+ write(context, entries(context).filterNot { it.id == id })+ }++ /**+ * Find by id or by label. The brain only ever holds the word the user+ * used ("飞书"), so matching on the label is the primary path; case and+ * surrounding space are ignored because that word arrives via an LLM.+ */+ fun find(context: Context, needle: String): Entry? {+ val n = needle.trim().lowercase()+ if (n.isEmpty()) return null+ val all = entries(context)+ return all.firstOrNull { it.id.lowercase() == n }+ ?: all.firstOrNull { it.label.trim().lowercase() == n }+ ?: all.firstOrNull { it.label.trim().lowercase().contains(n) }+ }++ fun newId(): String = "v${System.currentTimeMillis().toString(36)}"++ /**+ * `138****1111` — enough for the brain to confirm it has the right+ * account without pulling every identifier into its context.+ */+ fun mask(value: String): String {+ if (value.isBlank()) return ""+ val at = value.indexOf('@')+ if (at > 0) {+ val name = value.take(at)+ val head = name.take(2)+ return "$head${"*".repeat(maxOf(1, name.length - 2))}${value.substring(at)}"+ }+ if (value.length <= 4) return "*".repeat(value.length)+ if (value.length <= 7) return "${value.take(2)}${"*".repeat(value.length - 2)}"+ return "${value.take(3)}${"*".repeat(value.length - 7)}${value.takeLast(4)}"+ }++ /**+ * Erase every credential. The one caller allowed to lose this on purpose+ * (恢复出厂设置).+ *+ * No cache to invalidate, unlike [Notebook]: [entries] reads straight from+ * prefs every time. `commit()` rather than `apply()` because the caller+ * reports success and the device may lose power a moment later.+ */+ fun wipe(context: Context) {+ prefs(context).edit().clear().commit()+ }++ private fun write(context: Context, list: List<Entry>) {+ val arr = JSONArray()+ for (e in list) {+ arr.put(+ JSONObject()+ .put("id", e.id)+ .put("label", e.label)+ .put("phone", e.phone)+ .put("email", e.email)+ .put("username", e.username)+ .put("note", e.note)+ .put("password", e.password)+ .put("card_number", e.cardNumber)+ .put("card_expiry", e.cardExpiry)+ .put("card_cvv", e.cardCvv)+ .put("card_holder", e.cardHolder)+ )+ }+ val blob = encrypt(arr.toString())+ if (blob == null) {+ // Silently keeping the old contents would let the user close the+ // form believing a password was saved.+ Log.e(TAG, "vault: encrypt failed, nothing written")+ return+ }+ prefs(context).edit().putString(KEY_BLOB, blob).apply()+ }++ private fun prefs(context: Context) =+ context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)++ // ---- crypto ----------------------------------------------------++ private fun secretKey(): SecretKey {+ val ks = KeyStore.getInstance("AndroidKeyStore").apply { load(null) }+ (ks.getEntry(KEY_ALIAS, null) as? KeyStore.SecretKeyEntry)?.let { return it.secretKey }+ val gen = KeyGenerator.getInstance(KeyProperties.KEY_ALGORITHM_AES, "AndroidKeyStore")+ gen.init(+ KeyGenParameterSpec.Builder(+ KEY_ALIAS,+ KeyProperties.PURPOSE_ENCRYPT or KeyProperties.PURPOSE_DECRYPT,+ )+ .setBlockModes(KeyProperties.BLOCK_MODE_GCM)+ .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_NONE)+ // No setUserAuthenticationRequired: the ball fills a password+ // while the user is watching it work, not while they are at+ // the lock screen, and a biometric prompt mid-task would stop+ // the very automation this exists for.+ .build()+ )+ return gen.generateKey()+ }++ /** `base64(iv || ciphertext)`, or null if the Keystore refused. */+ private fun encrypt(plain: String): String? = runCatching {+ val c = Cipher.getInstance("AES/GCM/NoPadding")+ c.init(Cipher.ENCRYPT_MODE, secretKey())+ val body = c.doFinal(plain.toByteArray(Charsets.UTF_8))+ Base64.encodeToString(c.iv + body, Base64.NO_WRAP)+ }.getOrNull()++ private fun decrypt(blob: String): String? = runCatching {+ val raw = Base64.decode(blob, Base64.NO_WRAP)+ val c = Cipher.getInstance("AES/GCM/NoPadding")+ c.init(+ Cipher.DECRYPT_MODE,+ secretKey(),+ GCMParameterSpec(GCM_TAG_BITS, raw, 0, IV_BYTES),+ )+ String(c.doFinal(raw, IV_BYTES, raw.size - IV_BYTES), Charsets.UTF_8)+ }.getOrElse {+ // A restored backup or a reset Keystore leaves an undecryptable blob.+ // Say so in the log: from the settings screen it looks identical to+ // never having saved anything.+ Log.e(TAG, "vault: decrypt failed — entries are unreadable on this install", it)+ null+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/config/VoiceConfig.kt b/app/src/main/java/net/kuafuai/andee/config/VoiceConfig.ktnew file mode 100644index 0000000..f52d5f5--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/config/VoiceConfig.kt@@ -0,0 +1,389 @@+package net.kuafuai.andee.config++import android.content.Context+import java.util.UUID+++data class VoiceConfig(+ val apiKey: String,+ val asrEndpoint: String,+ val asrResourceId: String,+ val asrUid: String,+ val ttsEndpoint: String,+ val ttsResourceId: String,+ val ttsSpeaker: String,+ val ttsSampleRate: Int,+) {+ companion object {+ /**+ * The 火山 (Volcengine Speech) account the voice pipeline runs on.+ *+ * **Empty on purpose, and it must stay empty in the repository.** This+ * used to hold a real key so that a fresh install could hear and speak+ * without setup. That trade is only available to a build nobody else+ * reads: a key in public source is a key in a search index, and a bare+ * UUID is not a shape GitHub's secret scanning recognises, so nothing+ * would warn anyone it had leaked. Whoever ships a private build can+ * put their own key back here; in this tree it is the user's to supply.+ *+ * The user supplies it in ⚙ (`api_key`). Blank in prefs falls back to+ * this constant — see [load] — so with both empty the voice pipeline+ * has no credential and ASR/TTS fail at connect time. That is reported+ * *before* it happens by [net.kuafuai.andee.device.SelfCheck]'s+ * `voice_key` row rather than being left to look like a network fault.+ *+ * On storage: the key lands in `voice_prefs.xml` in plaintext, like+ * `llm_api_key`. Neither leaves the device: `allowBackup` is off and+ * both `res/xml/backup_rules.xml` and `res/xml/data_extraction_rules.xml`+ * exclude everything, cloud backup and device transfer alike.+ */+ const val API_KEY = ""++ /**+ * The 火山 speaker the device speaks with when the ball's look has no+ * opinion of its own.+ *+ * Not a settings row anymore: the voice is a property of the character+ * ([net.kuafuai.andee.ui.ball.BallLook.voice]) and this is only the+ * fallback for the looks that shipped without one. See [lookSpeaker].+ */+ const val TTS_SPEAKER = "zh_female_vv_uranus_bigtts"++ /**+ * Keys whose blank value means "go back to the compiled default"+ * rather than "the user didn't touch this row". See [save].+ */+ private val OVERRIDE_KEYS = setOf("api_key")++ const val DEFAULT_ASR_ENDPOINT = "wss://openspeech.bytedance.com/api/v3/sauc/bigmodel_async"+ const val DEFAULT_ASR_RESOURCE = "volc.bigasr.sauc.duration"+ const val DEFAULT_TTS_ENDPOINT = "wss://openspeech.bytedance.com/api/v3/tts/bidirection"+ const val DEFAULT_TTS_RESOURCE = "seed-tts-2.0"+ const val DEFAULT_TTS_SAMPLE_RATE = 24_000+ private const val PREFS = "voice_prefs"++ /**+ * Where the agent loop runs. Deliberately an explicit setting rather+ * than "hub_url is blank ⇒ go local": [save] drops blank values, so a+ * URL that has ever been saved cannot be cleared from the UI, and+ * inferring the mode from it would leave the user with no way back.+ */+ const val BRAIN_HUB = "hub"+ const val BRAIN_LOCAL = "local"++ const val DEFAULT_LLM_BASE_URL = "https://api.deepseek.com"+ const val DEFAULT_LLM_MODEL = "deepseek-flash"+ const val DEFAULT_REASONING_EFFORT = "high"++ /**+ * Hub connection settings. Not part of the voice pipeline but stored+ * in the same SharedPreferences file to keep the settings UI simple.+ * Read via [hubConfig] instead of [load].+ */+ data class HubConfig(+ val url: String, // empty = don't dial out+ val deviceId: String, // stable, generated once+ val deviceName: String, // human-friendly, editable+ )++ fun hubConfig(context: Context): HubConfig {+ val p = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)+ var deviceId = p.getString("device_id", null)+ if (deviceId.isNullOrEmpty()) {+ deviceId = "body-${UUID.randomUUID().toString().take(8)}"+ p.edit().putString("device_id", deviceId).apply()+ }+ return HubConfig(+ url = p.getString("hub_url", "").orEmpty().trim(),+ deviceId = deviceId,+ deviceName = p.getString("device_name", "").orEmpty().ifEmpty { deviceId },+ )+ }++ /**+ * Which brain answers, and how to reach it if it is the local one.+ *+ * Read via [brainConfig]; like [hubConfig] this never throws and never+ * requires the user to have configured anything — an unconfigured+ * device reads as [BRAIN_LOCAL], which is the out-of-box default.+ */+ data class BrainConfig(+ /** [BRAIN_HUB] or [BRAIN_LOCAL]. */+ val mode: String,+ val apiKey: String, // empty = local brain cannot run+ val baseUrl: String,+ val model: String,+ val thinking: Boolean,+ val reasoningEffort: String,+ ) {+ val isLocal: Boolean get() = mode == BRAIN_LOCAL+ }++ fun brainConfig(context: Context): BrainConfig {+ val p = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)+ // Anything other than the exact string "hub" is the local brain,+ // so a typo fails closed onto the out-of-box default — same rule+ // as [groundingEnabled]. It used to fail onto the hub, back when+ // the hub was the default; the default moved, so the direction+ // moved with it, and the picker's leftmost option (the fallback+ // when the saved value matches nothing) had to move too.+ val mode = if (p.getString("brain", BRAIN_LOCAL)?.trim()?.lowercase() == BRAIN_HUB) {+ BRAIN_HUB+ } else {+ BRAIN_LOCAL+ }+ return BrainConfig(+ mode = mode,+ apiKey = p.getString("llm_api_key", "").orEmpty().trim(),+ // trimEnd('/') because the client appends "/chat/completions"+ // and a trailing slash from the settings field would produce a+ // double slash, which some gateways 404 on.+ baseUrl = p.getString("llm_base_url", DEFAULT_LLM_BASE_URL)+ .orEmpty().trim().trimEnd('/').ifEmpty { DEFAULT_LLM_BASE_URL },+ model = p.getString("llm_model", DEFAULT_LLM_MODEL)+ .orEmpty().trim().ifEmpty { DEFAULT_LLM_MODEL },+ thinking = p.getString("llm_thinking", "enabled")+ ?.trim()?.lowercase() != "disabled",+ reasoningEffort = p.getString("llm_reasoning_effort", DEFAULT_REASONING_EFFORT)+ .orEmpty().trim().ifEmpty { DEFAULT_REASONING_EFFORT },+ )+ }++ /**+ * The quiet-hour review: after the user has stopped talking for a+ * while, the local brain reads the conversation back and files what is+ * worth keeping.+ *+ * On by default (it is the feature that makes the device feel like it+ * remembers you) but always switchable, because it spends the user's+ * own API key on a call nobody asked for. [dailyCap] is the other half+ * of that bargain: a conversation that keeps stopping and restarting+ * cannot run up an unbounded bill overnight.+ */+ data class SweepConfig(+ val enabled: Boolean,+ val quietMinutes: Int,+ val dailyCap: Int,+ )++ const val DEFAULT_SWEEP_QUIET_MINUTES = 20+ const val DEFAULT_SWEEP_DAILY_CAP = 12++ fun sweepConfig(context: Context): SweepConfig {+ val p = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)+ return SweepConfig(+ // Anything other than "off" is on: a typo should leave the+ // feature working, since off is the value a user has to go out+ // of their way to choose.+ enabled = p.getString("sweep", "on")?.trim()?.lowercase() != "off",+ quietMinutes = (p.getString("sweep_quiet_minutes", "")+ ?.trim()?.toIntOrNull() ?: DEFAULT_SWEEP_QUIET_MINUTES).coerceIn(5, 240),+ dailyCap = (p.getString("sweep_daily_cap", "")+ ?.trim()?.toIntOrNull() ?: DEFAULT_SWEEP_DAILY_CAP).coerceIn(0, 100),+ )+ }++ /**+ * Is the 火山 account the one saved in settings, or the factory one?+ *+ * Only needed so the answer is *visible*: a setting whose effect cannot+ * be seen is indistinguishable from one that did not take. The settings+ * card prints it, and the service logs it at start.+ */+ fun usingOwnCredentials(context: Context): Boolean =+ context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)+ .getString("api_key", "").orEmpty().isNotBlank()++ fun load(context: Context): VoiceConfig {+ val p = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)+ var uid = p.getString("uid", null)+ if (uid.isNullOrEmpty()) {+ uid = "tablet-body-${UUID.randomUUID().toString().take(8)}"+ p.edit().putString("uid", uid).apply()+ }+ return VoiceConfig(+ // Empty in prefs → [API_KEY], which is itself empty in the+ // public source. Trim before the fallback so a field cleared to+ // spaces behaves like a cleared field.+ apiKey = p.getString("api_key", "").orEmpty().trim().ifEmpty { API_KEY },+ asrEndpoint = p.getString("asr_endpoint", DEFAULT_ASR_ENDPOINT)!!,+ asrResourceId = p.getString("asr_resource_id", DEFAULT_ASR_RESOURCE)!!,+ asrUid = uid,+ ttsEndpoint = p.getString("tts_endpoint", DEFAULT_TTS_ENDPOINT)!!,+ ttsResourceId = p.getString("tts_resource_id", DEFAULT_TTS_RESOURCE)!!,+ // The voice follows the ball's look; [TTS_SPEAKER] is the+ // fallback for looks that don't name one.+ ttsSpeaker = lookSpeaker(p),+ ttsSampleRate = p.getInt("tts_sample_rate", DEFAULT_TTS_SAMPLE_RATE),+ )+ }++ fun save(context: Context, updates: Map<String, String>) {+ val p = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE).edit()+ for ((k, v) in updates) {+ if (k !in ALLOWED_KEYS) continue+ // Blank normally means "the user left this row alone", and is+ // dropped so it cannot wipe a saved value. The exception is the+ // keys that have a real compiled-in default: there, blank is+ // the only way back to it. Route them before the drop, or an+ // override once saved could never be taken out.+ if (k in OVERRIDE_KEYS) {+ if (v.isBlank()) p.remove(k) else p.putString(k, v.trim())+ continue+ }+ if (v.isBlank()) continue+ when (k) {+ "tts_sample_rate" ->+ v.trim().toIntOrNull()?.let { p.putInt(k, it) }++ else -> p.putString(k, v.trim())+ }+ }+ p.apply()+ }++ /** What [SettingsUi] shows. Only the keys it can actually edit. */+ fun current(context: Context): Map<String, String> {+ val p = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)+ val hc = hubConfig(context)+ val bc = brainConfig(context)+ return mapOf(+ // The overridable credential shows what is *saved*, not what is+ // in effect: an empty box is how the screen says "the factory+ // value is in use", and it is the state that [save] returns to.+ "api_key" to p.getString("api_key", "").orEmpty(),+ "asr_endpoint" to p.getString("asr_endpoint", DEFAULT_ASR_ENDPOINT)!!,+ "asr_resource_id" to p.getString("asr_resource_id", DEFAULT_ASR_RESOURCE)!!,+ "uid" to p.getString("uid", "").orEmpty(),+ "tts_endpoint" to p.getString("tts_endpoint", DEFAULT_TTS_ENDPOINT)!!,+ "tts_resource_id" to p.getString("tts_resource_id", DEFAULT_TTS_RESOURCE)!!,+ "tts_sample_rate" to p.getInt("tts_sample_rate", DEFAULT_TTS_SAMPLE_RATE).toString(),+ "hub_url" to hc.url,+ "device_id" to hc.deviceId,+ "device_name" to hc.deviceName,+ "brain" to bc.mode,+ "llm_api_key" to bc.apiKey,+ "llm_base_url" to bc.baseUrl,+ "llm_model" to bc.model,+ // Round-tripped as the wire words rather than a boolean: this map+ // feeds SettingsUi's pickers and the picker's value is what comes+ // back to [save], which only speaks String.+ "llm_thinking" to if (bc.thinking) "enabled" else "disabled",+ "llm_reasoning_effort" to bc.reasoningEffort,+ "grounding" to p.getString("grounding", "off")!!,+ "lang" to uiLanguage(context),+ "sweep" to p.getString("sweep", "on")!!,+ "sweep_quiet_minutes" to p.getString(+ "sweep_quiet_minutes", DEFAULT_SWEEP_QUIET_MINUTES.toString(),+ )!!,+ "sweep_daily_cap" to p.getString(+ "sweep_daily_cap", DEFAULT_SWEEP_DAILY_CAP.toString(),+ )!!,+ )+ }++ /**+ * Whether [net.kuafuai.andee.screen.GroundingLog] records taps and+ * screenshots to disk. Off unless explicitly turned on: it writes+ * pictures of whatever the user is doing.+ *+ * Stored as "on"/"off" rather than a boolean because [save] takes a+ * `Map<String, String>` from [SettingsUi]'s text fields, drops blanks,+ * and only routes `tts_sample_rate` through Int — a real boolean has+ * nowhere to enter the pipeline. Anything other than "on" is off, so a+ * typo fails closed.+ */+ fun groundingEnabled(context: Context): Boolean =+ context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)+ .getString("grounding", "off")+ ?.trim()?.lowercase() == "on"++ const val LANG_ZH = "zh"+ const val LANG_EN = "en"++ /**+ * Which language the user-facing UI speaks. [LANG_ZH] or [LANG_EN].+ *+ * Unset means *unset*, not Chinese: it falls through to the device's own+ * locale, because a tablet set to English showing a Chinese settings+ * card is the same bug as the reverse. Once the user picks one, the+ * saved value wins over the locale forever — they have said what they+ * want and a system language change should not overrule them.+ *+ * Not part of [current] merely for display: [net.kuafuai.andee.ui.SettingsUi]+ * writes this one the moment it is tapped rather than on 保存, because+ * the card redraws in the new language and that redraw *is* the+ * confirmation. Nothing else on this device is written eagerly.+ */+ fun uiLanguage(context: Context): String {+ val saved = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)+ .getString("lang", "").orEmpty().trim().lowercase()+ if (saved == LANG_EN) return LANG_EN+ if (saved == LANG_ZH) return LANG_ZH+ return if (java.util.Locale.getDefault().language == "zh") LANG_ZH else LANG_EN+ }++ /**+ * Which face the ball is wearing — a [net.kuafuai.andee.ui.ball.BallLook]+ * name, or blank for the first one.+ *+ * Like [uiLanguage] and unlike everything else here, it is written the+ * moment the user swipes rather than on 保存, and for the same reason:+ * the ball changing *is* the confirmation, and there is no card to+ * press a button on. It is also not in [current] — it has no settings+ * row, because the control is the ball itself.+ *+ * It is read from more than the ball now: the look carries the voice and+ * the manner as well as the drawing, so [lookSpeaker] and `LocalBrain`+ * both resolve it through [net.kuafuai.andee.ui.ball.BallLooks.byName].+ */+ fun ballLook(context: Context): String =+ context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)+ .getString("ball", "").orEmpty().trim()++ /**+ * The 火山 speaker for the ball's current look, falling back to+ * [TTS_SPEAKER].+ *+ * The whole voice story in one line: **the voice is the character's**.+ * A look either names one or it does not; the user has no voice row to+ * fill in, so there is no precedence to get wrong and no override to+ * silently block a character's voice. [TTS_SPEAKER] survives only as the+ * fallback for looks that shipped without an opinion.+ *+ * Lived at [load]'s own file rather than in `TtsController` for the same+ * reason the settings card used to need a "voice in use" row: [load] is+ * read by the ASR path, the meeting recorder and the start log as well,+ * and a speaker resolved at the call site would have `HuoshanTts`+ * logging one voice while `ScreenBodyService` logged another.+ */+ private fun lookSpeaker(+ p: android.content.SharedPreferences,+ ): String = net.kuafuai.andee.ui.ball.BallLooks+ .byName(p.getString("ball", "").orEmpty().trim()).voice+ .ifEmpty { TTS_SPEAKER }++ // api_key used to be absent here, because it was compiled in and a saved+ // pref would have shadowed the constant with no way to see it from the+ // UI. The constant ships empty in the public source, so the pref is now+ // the only place a key can come from and this row has to be editable.+ // Same class of credential as llm_api_key, which was always editable+ // because a DeepSeek key is the user's own. (tts_speaker was once+ // editable too, and is gone: the voice follows the ball's look, not a+ // row — see [lookSpeaker].)+ private val ALLOWED_KEYS = setOf(+ "uid",+ "api_key",+ "asr_endpoint", "asr_resource_id",+ "tts_endpoint", "tts_resource_id", "tts_sample_rate",+ "hub_url", "device_name",+ "brain",+ "llm_api_key", "llm_base_url", "llm_model",+ "llm_thinking", "llm_reasoning_effort",+ "grounding",+ "lang",+ "ball",+ "sweep", "sweep_quiet_minutes", "sweep_daily_cap",+ )+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/device/CallState.kt b/app/src/main/java/net/kuafuai/andee/device/CallState.ktnew file mode 100644index 0000000..bd3a184--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/CallState.kt@@ -0,0 +1,59 @@+package net.kuafuai.andee.device++import java.util.concurrent.ConcurrentHashMap++/**+ * Whether the tablet is currently in a phone / VoIP call — the "call is+ * happening right now" flag every mic gate reads.+ *+ * Mirror of `MeetingRecorder.isActive()`, opposite direction. Meeting mode:+ * WE hold the mic and gate everyone else from stealing it. Call mode: someone+ * else (dialer, WeChat 语音, WhatsApp, Telegram) is trying to hold the mic+ * and WE have to gate ourselves — mainly [net.kuafuai.andee.wake.WakeWord],+ * which otherwise keeps AudioRecord open and starves the caller app.+ *+ * ## Why a set of keys, not a bool+ *+ * A notification-driven detector has to survive two edges cleanly:+ *+ * - **Concurrent calls**: WhatsApp missed-call banner arrives while the+ * dialer ringing notification is still up. Booleans would clear on the+ * first `onNotificationRemoved` and open the mic mid-call.+ * - **Notification updates**: Android reissues the same key when text+ * changes ("calling…" → "connected 00:12"). onNotificationPosted fires+ * for each; we deduplicate by key and never over-count.+ *+ * Both fall out of a set. [mark] / [clear] are keyed on+ * `StatusBarNotification.key` — the platform's own uniqueness cookie —+ * so posted twice looks like posted once, and removed removes only the one+ * with that key.+ *+ * ## Why not `AudioManager.getMode()`+ *+ * Tried; wrong tool. `MODE_IN_CALL` needs telephony permission on some OEMs,+ * `MODE_IN_COMMUNICATION` is only set *after* the app opens its mic (too+ * late — WakeWord has already collided), and it does not cover the ringing+ * window at all. Notifications land the moment the call event happens on+ * the system, before any of that.+ */+object CallState {++ /** SBN keys currently belonging to a live call notification. */+ private val activeKeys = ConcurrentHashMap.newKeySet<String>()++ /** True while at least one call notification is still up. */+ val isActive: Boolean get() = activeKeys.isNotEmpty()++ fun mark(key: String) {+ activeKeys.add(key)+ }++ fun clear(key: String) {+ activeKeys.remove(key)+ }++ /** Wipe everything — for service shutdown / listener rebind. */+ fun reset() {+ activeKeys.clear()+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/device/DeviceActionsController.kt b/app/src/main/java/net/kuafuai/andee/device/DeviceActionsController.ktnew file mode 100644index 0000000..0016427--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/DeviceActionsController.kt@@ -0,0 +1,179 @@+package net.kuafuai.andee.device++import android.content.Context+import android.content.Intent+import android.hardware.Sensor+import android.hardware.SensorEvent+import android.hardware.SensorEventListener+import android.hardware.SensorManager+import android.media.AudioManager+import android.net.Uri+import android.os.Environment+import android.os.StatFs+import android.provider.AlarmClock+import android.provider.Settings+import org.json.JSONArray+import org.json.JSONObject+import java.util.concurrent.CountDownLatch+import java.util.concurrent.TimeUnit++/** Last foreground app, fed from the accessibility event stream. Cheap context. */+object DeviceState {+ @Volatile+ var foregroundPkg: String? = null+}++/**+ * Zero/low-friction device ACTUATION + info: launch apps, volume, brightness,+ * open URLs, set alarms, hardware facts, one-shot sensors. Everything here is+ * either permission-free or a plain intent handoff to a system app.+ *+ * Lives in `device.*` (not `screen.*`) — none of this folds the card.+ */+class DeviceActionsController(private val context: Context) {++ // ---- launch app ----------------------------------------------------++ fun launchApp(pkg: String?, label: String?): JSONObject {+ val target = when {+ !pkg.isNullOrEmpty() -> pkg+ !label.isNullOrEmpty() -> resolveLabel(label)+ ?: return JSONObject().put("error", "no installed app matches label '$label'")+ else -> return JSONObject().put("error", "launch requires pkg or label")+ }+ val intent = context.packageManager.getLaunchIntentForPackage(target)+ ?: return JSONObject().put("error", "app '$target' has no launchable activity")+ intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)+ return runCatching { context.startActivity(intent) }+ .fold(+ { JSONObject().put("launched", target) },+ { JSONObject().put("error", "launch failed: ${it.message}") },+ )+ }++ private fun resolveLabel(label: String): String? {+ val pm = context.packageManager+ val launcher = Intent(Intent.ACTION_MAIN, null).addCategory(Intent.CATEGORY_LAUNCHER)+ for (ri in pm.queryIntentActivities(launcher, 0)) {+ val l = runCatching { ri.loadLabel(pm).toString() }.getOrDefault("")+ if (l.equals(label, ignoreCase = true)) return ri.activityInfo.packageName+ }+ return null+ }++ // ---- volume ----------------------------------------------------++ /** level == null && mute == null → read-only. */+ fun volume(level: Int?, mute: Boolean?): JSONObject {+ val am = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager+ val max = am.getStreamMaxVolume(AudioManager.STREAM_MUSIC)+ if (mute == true) {+ am.adjustStreamVolume(AudioManager.STREAM_MUSIC, AudioManager.ADJUST_MUTE, 0)+ } else if (mute == false) {+ am.adjustStreamVolume(AudioManager.STREAM_MUSIC, AudioManager.ADJUST_UNMUTE, 0)+ } else if (level != null) {+ am.setStreamVolume(AudioManager.STREAM_MUSIC, level.coerceIn(0, max), 0)+ }+ return JSONObject()+ .put("volume", am.getStreamVolume(AudioManager.STREAM_MUSIC))+ .put("max", max)+ .put("muted", am.isStreamMute(AudioManager.STREAM_MUSIC))+ }++ // ---- brightness ----------------------------------------------------++ fun brightness(v: Int?): JSONObject {+ val cr = context.contentResolver+ if (v != null) {+ // Manual mode first, or the write is decoration on auto screens.+ runCatching {+ Settings.System.putInt(cr, Settings.System.SCREEN_BRIGHTNESS_MODE, Settings.System.SCREEN_BRIGHTNESS_MODE_MANUAL)+ }+ val ok = runCatching {+ Settings.System.putInt(cr, Settings.System.SCREEN_BRIGHTNESS, v.coerceIn(1, 255))+ }.isSuccess+ if (!ok) return JSONObject().put("error", "write failed — needs WRITE_SETTINGS grant (Settings → Special app access)")+ }+ return JSONObject().put("brightness", Settings.System.getInt(cr, Settings.System.SCREEN_BRIGHTNESS, -1))+ }++ // ---- open url / dial / map ------------------------------------------++ /** Handles http(s), tel:, geo: — the URL's scheme decides where it lands. */+ fun openUrl(url: String): JSONObject {+ val parsed = runCatching { Uri.parse(url) }.getOrNull()+ ?: return JSONObject().put("error", "unparseable url")+ if (parsed.scheme !in setOf("http", "https", "tel", "geo", "mailto")) {+ return JSONObject().put("error", "unsupported scheme '${parsed.scheme}' (allowed: http/https/tel/geo/mailto)")+ }+ return runCatching {+ context.startActivity(Intent(Intent.ACTION_VIEW, parsed).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK))+ }.fold(+ { JSONObject().put("opened", url) },+ { JSONObject().put("error", "open failed: ${it.message}") },+ )+ }++ // ---- alarm ----------------------------------------------------++ fun setAlarm(hour: Int, minute: Int, message: String?): JSONObject {+ val i = Intent(AlarmClock.ACTION_SET_ALARM).apply {+ putExtra(AlarmClock.EXTRA_HOUR, hour)+ putExtra(AlarmClock.EXTRA_MINUTES, minute)+ putExtra(AlarmClock.EXTRA_MESSAGE, message ?: "")+ putExtra(AlarmClock.EXTRA_SKIP_UI, true)+ addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)+ }+ return runCatching { context.startActivity(i) }+ .fold(+ { JSONObject().put("alarm_set", "%02d:%02d".format(hour, minute)).put("message", message ?: "") },+ { JSONObject().put("error", "no clock app accepted the request: ${it.message}") },+ )+ }++ // ---- device facts ----------------------------------------------------++ fun deviceInfo(): JSONObject {+ val am = context.getSystemService(Context.ACTIVITY_SERVICE) as android.app.ActivityManager+ val mem = android.app.ActivityManager.MemoryInfo().also { am.getMemoryInfo(it) }+ val data = StatFs(Environment.getDataDirectory().path)+ return JSONObject()+ .put("model", android.os.Build.MODEL)+ .put("brand", android.os.Build.BRAND)+ .put("android", android.os.Build.VERSION.RELEASE)+ .put("sdk", android.os.Build.VERSION.SDK_INT)+ .put("ram_total_mb", mem.totalMem / 1048576)+ .put("ram_avail_mb", mem.availMem / 1048576)+ .put("storage_total_gb", data.totalBytes / 1073741824.0.let { Math.round(it) })+ .put("storage_avail_gb", data.availableBytes / 1073741824.0.let { Math.round(it) })+ }++ // ---- one-shot sensors ----------------------------------------------------++ fun sensors(timeoutMs: Long = 1500): JSONObject {+ val sm = context.getSystemService(Context.SENSOR_SERVICE) as SensorManager+ val out = JSONObject()++ fun readOnce(sensor: Sensor?, key: String, transform: (FloatArray) -> JSONArray) {+ if (sensor == null) { out.put(key, "no sensor"); return }+ val latch = CountDownLatch(1)+ val got = arrayOfNulls<FloatArray>(1)+ val l = object : SensorEventListener {+ override fun onSensorChanged(e: SensorEvent) {+ got[0] = e.values.clone(); latch.countDown(); sm.unregisterListener(this)+ }+ override fun onAccuracyChanged(s: Sensor?, a: Int) {}+ }+ sm.registerListener(l, sensor, SensorManager.SENSOR_DELAY_FASTEST)+ val ok = latch.await(timeoutMs, TimeUnit.MILLISECONDS)+ if (!ok) sm.unregisterListener(l)+ out.put(key, if (ok && got[0] != null) transform(got[0]!!) else "no reading")+ }++ readOnce(sm.getDefaultSensor(Sensor.TYPE_LIGHT), "light_lux") { JSONArray(listOf(it[0])) }+ readOnce(sm.getDefaultSensor(Sensor.TYPE_ACCELEROMETER), "accel_mps2") {+ JSONArray(listOf(it[0], it[1], it[2]))+ }+ return out+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/device/DeviceCommsController.kt b/app/src/main/java/net/kuafuai/andee/device/DeviceCommsController.ktnew file mode 100644index 0000000..1594f6e--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/DeviceCommsController.kt@@ -0,0 +1,220 @@+package net.kuafuai.andee.device++import android.Manifest+import android.content.Context+import android.content.pm.PackageManager+import android.database.Cursor+import android.net.Uri+import android.provider.CalendarContract+import android.provider.CallLog+import android.provider.ContactsContract+import android.provider.Telephony+import android.telephony.SmsManager+import org.json.JSONArray+import org.json.JSONObject+import java.text.SimpleDateFormat+import java.util.Date+import java.util.Locale++/**+ * Personal-data queries + actuation: contacts, call log, dial, SMS, calendar.+ * ALL runtime permissions, ALL checked per call with graceful denial —+ * personal data is the one place a crash or a silent overreach is+ * unacceptable. Read paths report exactly what they saw; write paths+ * (sendSms) are deliberately explicit — the brain must confirm before firing.+ */+class DeviceCommsController(private val context: Context) {++ // ---- contacts ----------------------------------------------------++ fun searchContacts(query: String?, limit: Int = 20): JSONObject {+ if (!ensurePermission(Manifest.permission.READ_CONTACTS)) return denied("contacts")+ val q = query?.trim().orEmpty()+ val selection = if (q.isEmpty()) null+ else ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME + " LIKE ?"+ val args = if (q.isEmpty()) null else arrayOf("%$q%")+ val out = JSONArray()+ context.contentResolver.query(+ ContactsContract.CommonDataKinds.Phone.CONTENT_URI,+ arrayOf(+ ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME,+ ContactsContract.CommonDataKinds.Phone.NUMBER,+ ),+ selection, args,+ ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME,+ )?.use { c: Cursor ->+ var n = 0+ while (c.moveToNext() && n < limit) {+ out.put(JSONObject()+ .put("name", c.getString(0) ?: "")+ .put("number", c.getString(1) ?: ""))+ n+++ }+ }+ return JSONObject().put("count", out.length()).put("contacts", out)+ }++ // ---- dial ----------------------------------------------------++ fun dial(number: String, direct: Boolean): JSONObject {+ if (direct) {+ if (!ensurePermission(Manifest.permission.CALL_PHONE)) return denied("call_phone",+ "Direct dialing needs the Phone permission; without it only the dialer can be opened (direct=false).")+ return runCatching {+ context.startActivity(+ android.content.Intent(android.content.Intent.ACTION_CALL, Uri.parse("tel:$number"))+ .addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)+ )+ JSONObject().put("dialed", number).put("direct", true)+ }.getOrElse { JSONObject().put("error", "call failed: ${it.message}") }+ }+ return runCatching {+ context.startActivity(+ android.content.Intent(android.content.Intent.ACTION_DIAL, Uri.parse("tel:$number"))+ .addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)+ )+ JSONObject().put("dialer_opened", number).put("direct", false)+ }.getOrElse { JSONObject().put("error", "dialer failed: ${it.message}") }+ }++ // ---- call log ----------------------------------------------------++ fun callLog(limit: Int = 15): JSONObject {+ if (!ensurePermission(Manifest.permission.READ_CALL_LOG)) return denied("call_log")+ val out = JSONArray()+ context.contentResolver.query(+ CallLog.Calls.CONTENT_URI,+ arrayOf(CallLog.Calls.CACHED_NAME, CallLog.Calls.NUMBER, CallLog.Calls.TYPE, CallLog.Calls.DATE, CallLog.Calls.DURATION),+ null, null,+ CallLog.Calls.DATE + " DESC",+ )?.use { c ->+ var n = 0+ while (c.moveToNext() && n < limit) {+ out.put(JSONObject()+ .put("name", c.getString(0) ?: c.getString(1))+ .put("number", c.getString(1))+ .put("type", when (c.getInt(2)) {+ CallLog.Calls.INCOMING_TYPE -> "incoming"+ CallLog.Calls.OUTGOING_TYPE -> "outgoing"+ CallLog.Calls.MISSED_TYPE -> "missed"+ else -> "other"+ })+ .put("when", SimpleDateFormat("MM-dd HH:mm", Locale.US).format(Date(c.getLong(3))))+ .put("duration_s", c.getLong(4)))+ n+++ }+ }+ return JSONObject().put("count", out.length()).put("calls", out)+ }++ // ---- sms ----------------------------------------------------++ fun readSms(box: String, limit: Int = 15): JSONObject {+ if (!ensurePermission(Manifest.permission.READ_SMS)) return denied("sms")+ val uri = if (box == "sent") Telephony.Sms.Sent.CONTENT_URI else Telephony.Sms.Inbox.CONTENT_URI+ val out = JSONArray()+ context.contentResolver.query(+ uri,+ arrayOf(Telephony.Sms.ADDRESS, Telephony.Sms.BODY, Telephony.Sms.DATE),+ null, null,+ "date DESC",+ )?.use { c ->+ var n = 0+ while (c.moveToNext() && n < limit) {+ out.put(JSONObject()+ .put("address", c.getString(0) ?: "")+ .put("body", (c.getString(1) ?: "").take(300))+ .put("when", SimpleDateFormat("MM-dd HH:mm", Locale.US).format(Date(c.getLong(2)))))+ n+++ }+ }+ return JSONObject().put("count", out.length())+ .put("box", box)+ .put("messages", out)+ }++ fun sendSms(number: String, body: String): JSONObject {+ if (!ensurePermission(Manifest.permission.SEND_SMS)) return denied("send_sms")+ return runCatching {+ SmsManager.getDefault().sendTextMessage(number, null, body, null, null)+ JSONObject().put("sent", true).put("to", number).put("len", body.length)+ }.getOrElse { JSONObject().put("error", "send failed: ${it.message}") }+ }++ // ---- calendar ----------------------------------------------------++ fun calendarEvents(daysAhead: Int = 3): JSONObject {+ if (!ensurePermission(Manifest.permission.READ_CALENDAR)) return denied("calendar")+ val now = System.currentTimeMillis()+ val until = now + daysAhead * 86_400_000L+ val out = JSONArray()+ val uri = CalendarContract.Instances.CONTENT_URI.buildUpon()+ .appendPath(now.toString())+ .appendPath(until.toString())+ .build()+ context.contentResolver.query(+ uri,+ arrayOf(+ CalendarContract.Instances.TITLE,+ CalendarContract.Instances.DESCRIPTION,+ CalendarContract.Instances.BEGIN,+ CalendarContract.Instances.END,+ CalendarContract.Instances.EVENT_LOCATION,+ ),+ null, null,+ CalendarContract.Instances.BEGIN + " ASC",+ )?.use { c ->+ while (c.moveToNext()) {+ out.put(JSONObject()+ .put("title", c.getString(0) ?: "")+ .put("desc", (c.getString(1) ?: "").take(120))+ .put("begin", SimpleDateFormat("yyyy-MM-dd HH:mm", Locale.US).format(Date(c.getLong(2))))+ .put("end", SimpleDateFormat("HH:mm", Locale.US).format(Date(c.getLong(3))))+ .put("location", c.getString(4) ?: ""))+ }+ }+ return JSONObject().put("count", out.length()).put("events", out)+ }++ fun addCalendarEvent(title: String, beginMs: Long, endMs: Long?, location: String?, description: String?): JSONObject {+ if (!ensurePermission(Manifest.permission.WRITE_CALENDAR)) return denied("write_calendar")+ val cv = android.content.ContentValues().apply {+ put(CalendarContract.Events.CALENDAR_ID, primaryCalendarId())+ put(CalendarContract.Events.TITLE, title)+ put(CalendarContract.Events.DTSTART, beginMs)+ put(CalendarContract.Events.DTEND, endMs ?: (beginMs + 3_600_000L))+ put(CalendarContract.Events.EVENT_TIMEZONE, java.util.TimeZone.getDefault().id)+ location?.let { put(CalendarContract.Events.EVENT_LOCATION, it) }+ description?.let { put(CalendarContract.Events.DESCRIPTION, it) }+ }+ return runCatching {+ val uri = context.contentResolver.insert(CalendarContract.Events.CONTENT_URI, cv)+ if (uri != null) JSONObject().put("added", title).put("event_uri", uri.toString())+ else JSONObject().put("error", "insert returned null")+ }.getOrElse { JSONObject().put("error", "insert failed: ${it.message}") }+ }++ private fun primaryCalendarId(): Long {+ context.contentResolver.query(+ CalendarContract.Calendars.CONTENT_URI,+ arrayOf(CalendarContract.Calendars._ID, CalendarContract.Calendars.IS_PRIMARY),+ null, null, null,+ )?.use { c ->+ var first = -1L+ while (c.moveToNext()) {+ if (c.getInt(1) == 1) return c.getLong(0)+ if (first < 0) first = c.getLong(0)+ }+ if (first >= 0) return first+ }+ return 1L+ }++ // ---- permissions ----------------------------------------------------++ private fun ensurePermission(perm: String): Boolean =+ context.checkSelfPermission(perm) == PackageManager.PERMISSION_GRANTED++ private fun denied(what: String, hint: String = "Ask the user to grant it once (Settings → Apps → Permissions), then retry.") =+ JSONObject().put("error", "permission denied: $what").put("hint", hint)+}diff --git a/app/src/main/java/net/kuafuai/andee/device/DeviceInfoController.kt b/app/src/main/java/net/kuafuai/andee/device/DeviceInfoController.ktnew file mode 100644index 0000000..efc4cae--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/DeviceInfoController.kt@@ -0,0 +1,198 @@+package net.kuafuai.andee.device++import android.Manifest+import android.content.Context+import android.content.pm.PackageManager+import android.hardware.Sensor+import android.hardware.SensorEvent+import android.hardware.SensorEventListener+import android.hardware.SensorManager+import android.location.Location+import android.location.LocationListener+import android.location.LocationManager+import android.os.Bundle+import android.os.Looper+import org.json.JSONArray+import org.json.JSONObject+import java.text.SimpleDateFormat+import java.util.Date+import java.util.Locale+import java.util.concurrent.CountDownLatch+import java.util.concurrent.TimeUnit++/**+ * Read-only device context for the brain: connectivity, installed apps,+ * last known location, step counter, battery. Lives in the `device.*`+ * namespace — deliberately NOT `screen.*` — because these are queries,+ * not actuations: asking "am I online" must not fold the card into the+ * corner ball the way a screen gesture does (see CommandDispatcher's+ * ensureCompact, which triggers on the `screen.` prefix).+ *+ * Permission model:+ * - connectivity / battery / app list: declared only (QUERY_ALL_PACKAGES)+ * - location: runtime permission, requested via [ensurePermission]; without+ * it we return the graceful "denied" answer, never a crash+ * - steps: ACTIVITY_RECOGNITION runtime permission, same story+ *+ * Everything here is a snapshot query with a bounded wait (sensors take a+ * beat to deliver the first reading) — no background services, no+ * listeners outliving the call. The step counter is cumulative-since-boot+ * by Android contract; a "today" figure would need persistence we don't+ * have yet, so we report the raw number and say so.+ */+class DeviceInfoController(private val context: Context) {++ // ---- connectivity ----------------------------------------------------++ fun connectivity(): JSONObject {+ val cm = context.getSystemService(Context.CONNECTIVITY_SERVICE)+ as android.net.ConnectivityManager+ val nw = cm.activeNetwork+ val caps = nw?.let { cm.getNetworkCapabilities(it) }+ val online = caps?.hasCapability(android.net.NetworkCapabilities.NET_CAPABILITY_VALIDATED) == true+ val typeName = when {+ caps == null -> "none"+ caps.hasTransport(android.net.NetworkCapabilities.TRANSPORT_WIFI) -> "wifi"+ caps.hasTransport(android.net.NetworkCapabilities.TRANSPORT_CELLULAR) -> "cellular"+ caps.hasTransport(android.net.NetworkCapabilities.TRANSPORT_ETHERNET) -> "ethernet"+ else -> "other"+ }+ return JSONObject()+ .put("online", online)+ .put("type", typeName)+ .put("network", if (nw != null) "available" else "unavailable")+ }++ // ---- installed apps ----------------------------------------------------++ /**+ * Launchable apps only (apps with a launcher intent) — the list a human+ * means when they ask "what's installed", not the 400 system packages.+ */+ fun installedApps(): JSONObject {+ val pm = context.packageManager+ val launcher = android.content.Intent(android.content.Intent.ACTION_MAIN, null)+ .addCategory(android.content.Intent.CATEGORY_LAUNCHER)+ val apps = JSONArray()+ for (ri in pm.queryIntentActivities(launcher, 0)) {+ val label = runCatching { ri.loadLabel(pm).toString() }.getOrDefault(ri.activityInfo.packageName)+ apps.put(+ JSONObject()+ .put("label", label)+ .put("pkg", ri.activityInfo.packageName)+ )+ }+ return JSONObject().put("count", apps.length()).put("apps", apps)+ }++ // ---- location ----------------------------------------------------++ fun location(timeoutMs: Long = 8000): JSONObject {+ if (!ensurePermission(Manifest.permission.ACCESS_FINE_LOCATION)) {+ return permissionDenied("location",+ "Ask the user to grant Location to the body app once (Settings → Apps → Permissions), then retry.")+ }+ val lm = context.getSystemService(Context.LOCATION_SERVICE) as LocationManager+ // Prefer the freshest cached fix; fall back to waiting for one.+ val providers = listOf(LocationManager.GPS_PROVIDER, LocationManager.NETWORK_PROVIDER, LocationManager.PASSIVE_PROVIDER)+ var best: Location? = null+ for (p in providers) {+ runCatching { lm.getLastKnownLocation(p) }?.getOrNull()?.let { loc ->+ if (best == null || loc.time > best!!.time) best = loc+ }+ }+ if (best == null || System.currentTimeMillis() - best!!.time > STALE_LOC_MS) {+ val latch = CountDownLatch(1)+ val got = arrayOfNulls<Location>(1)+ val listener = object : LocationListener {+ override fun onLocationChanged(loc: Location) {+ got[0] = loc+ latch.countDown()+ }+ override fun onProviderEnabled(provider: String) {}+ override fun onProviderDisabled(provider: String) {}+ @Deprecated("required override on old APIs")+ override fun onStatusChanged(provider: String?, status: Int, extras: Bundle?) {}+ }+ val requested = providers.any { p ->+ runCatching {+ lm.requestLocationUpdates(p, 0L, 0f, listener, Looper.getMainLooper())+ }.isSuccess+ }+ if (requested) {+ latch.await(timeoutMs, TimeUnit.MILLISECONDS)+ providers.forEach { p -> runCatching { lm.removeUpdates(listener) } }+ best = got[0] ?: best+ }+ }+ val b = best+ ?: return JSONObject().put("error", "no fix available (GPS off / indoors / just booted)")+ return JSONObject()+ .put("lat", b.latitude)+ .put("lng", b.longitude)+ .put("accuracy_m", b.accuracy.toDouble())+ .put("time", SimpleDateFormat("yyyy-MM-dd HH:mm:ss", Locale.US).format(Date(b.time)))+ .put("age_ms", System.currentTimeMillis() - b.time)+ }++ // ---- steps ----------------------------------------------------++ fun steps(timeoutMs: Long = 3000): JSONObject {+ if (!ensurePermission(Manifest.permission.ACTIVITY_RECOGNITION)) {+ return permissionDenied("activity_recognition",+ "Ask the user to grant 'Physical activity' to the body app once, then retry.")+ }+ val sm = context.getSystemService(Context.SENSOR_SERVICE) as SensorManager+ val sensor = sm.getDefaultSensor(Sensor.TYPE_STEP_COUNTER) ?: run {+ return JSONObject().put("error", "no step counter sensor on this device")+ }+ val latch = CountDownLatch(1)+ val got = arrayOfNulls<Float>(1)+ val listener = object : SensorEventListener {+ override fun onSensorChanged(e: SensorEvent) {+ got[0] = e.values.firstOrNull()+ latch.countDown()+ sm.unregisterListener(this)+ }+ override fun onAccuracyChanged(s: Sensor?, a: Int) {}+ }+ sm.registerListener(listener, sensor, SensorManager.SENSOR_DELAY_FASTEST)+ val ok = latch.await(timeoutMs, TimeUnit.MILLISECONDS)+ if (!ok) sm.unregisterListener(listener)+ val v = got[0]+ ?: return JSONObject().put("error", "sensor gave no reading within ${timeoutMs}ms")+ return JSONObject()+ .put("steps_since_boot", v.toInt())+ .put("note", "cumulative since last reboot, not per-day")+ }++ // ---- battery ----------------------------------------------------++ fun battery(): JSONObject {+ val ifilter = android.content.IntentFilter(android.content.Intent.ACTION_BATTERY_CHANGED)+ val bat = context.registerReceiver(null, ifilter) ?: return JSONObject().put("error", "no battery data")+ val level = bat.getIntExtra(android.os.BatteryManager.EXTRA_LEVEL, -1)+ val scale = bat.getIntExtra(android.os.BatteryManager.EXTRA_SCALE, -1)+ val status = bat.getIntExtra(android.os.BatteryManager.EXTRA_STATUS, -1)+ val plugged = bat.getIntExtra(android.os.BatteryManager.EXTRA_PLUGGED, 0)+ val pct = if (level >= 0 && scale > 0) level * 100 / scale else -1+ return JSONObject()+ .put("percent", pct)+ .put("charging", status == android.os.BatteryManager.BATTERY_STATUS_CHARGING ||+ status == android.os.BatteryManager.BATTERY_STATUS_FULL)+ .put("plugged", plugged != 0)+ }++ // ---- permissions ----------------------------------------------------++ private fun ensurePermission(perm: String): Boolean =+ context.checkSelfPermission(perm) == PackageManager.PERMISSION_GRANTED++ private fun permissionDenied(what: String, guidance: String): JSONObject =+ JSONObject().put("error", "permission denied: $what").put("hint", guidance)++ private companion object {+ /** Older than this and a cached location is decoration, not data. */+ const val STALE_LOC_MS = 5 * 60_000L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/device/LookActivity.kt b/app/src/main/java/net/kuafuai/andee/device/LookActivity.ktnew file mode 100644index 0000000..7e59f94--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/LookActivity.kt@@ -0,0 +1,327 @@+package net.kuafuai.andee.device++import android.Manifest+import android.annotation.SuppressLint+import android.content.Context+import android.content.Intent+import android.content.pm.PackageManager+import android.graphics.Bitmap+import android.graphics.BitmapFactory+import android.os.Bundle+import android.util.Base64+import android.view.Gravity+import android.view.ViewGroup+import android.widget.FrameLayout+import androidx.camera.core.CameraSelector+import androidx.camera.core.ImageCapture+import androidx.camera.core.ImageCaptureException+import androidx.camera.core.Preview+import androidx.camera.lifecycle.ProcessCameraProvider+import androidx.camera.view.PreviewView+import androidx.core.content.ContextCompat+import androidx.core.view.WindowCompat+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.ui.StatusPill+import net.kuafuai.andee.ui.ViewfinderView+import net.kuafuai.andee.ui.avoidSystemBars+import net.kuafuai.andee.ui.closePill+import net.kuafuai.andee.ui.dpI+import net.kuafuai.andee.ui.matchParent+import java.io.ByteArrayOutputStream+import java.io.File+import java.util.concurrent.atomic.AtomicReference++/**+ * The ball's LIVE eye — a persistent camera session the brain samples.+ * See the `look` tool description for the brain-facing contract:+ * device serves frames, brain decides cadence (once for "这是什么",+ * a self-chosen loop for "盯着水桶满了没").+ *+ * Frame source is ImageCapture (not PreviewView.getBitmap — that stayed+ * null on this OEM in COMPATIBLE mode; a hardware quirk not worth+ * fighting when takePicture just works).+ *+ * In and out are [net.kuafuai.andee.ui.StageActivity]'s, the same gesture the+ * pages get: the eye opened by cutting to the feed and left by cutting to+ * whatever was behind it, and three surfaces that behave three ways is+ * something the user notices even when he can't say what it was.+ */+class LookActivity : net.kuafuai.andee.ui.StageActivity() {++ companion object {+ @Volatile var instance: LookActivity? = null+ @Volatile var usingFront = false++ /**+ * When the *user* closed the camera, in `SystemClock.uptimeMillis`.+ * Read by the dispatcher so a look right afterwards can say what+ * actually happened instead of "camera not ready yet" — which is both+ * false and an invitation to reopen the lens in the user's face.+ */+ @Volatile var userClosedAt = 0L++ private val mainHandler = android.os.Handler(android.os.Looper.getMainLooper())++ /** How long "看到了" / "没看清" stays up before the resting line returns. */+ private const val STATUS_HOLD_MS = 900L++ /**+ * How long after a bind the sensor is still auto-exposing. Shooting+ * immediately hands the brain a frame that is washed out or nearly+ * black — and it has no way to know that is a camera artefact rather+ * than the room, so it describes the dark. Measured ~1.5s on this+ * hardware; it costs nothing on later samples because the clock runs+ * from the bind, not from the request.+ */+ private const val EXPOSURE_SETTLE_MS = 1500L++ fun open(context: Context, front: Boolean) {+ usingFront = front+ val act = instance+ if (act == null || act.isFinishing) {+ context.startActivity(+ Intent(context, LookActivity::class.java)+ .addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)+ )+ } else {+ mainHandler.post { if (!act.isFinishing) act.rebind(front) }+ }+ }++ fun close() {+ val act = instance ?: return+ act.closingByBrain = true+ // The brain's own close, so it goes the same way the ✕ pill does.+ // Called from a worker thread and from a path that must not care+ // whether the stage has been revealed — [leave] posts and guards.+ mainHandler.post { if (!act.isFinishing) act.leave() }+ instance = null+ }++ /**+ * Capture one frame through ImageCapture into a scratch file, then+ * downscale to maxSide and return base64 JPEG. Null if the camera+ * isn't bound yet (first moments after open/rebind). Blocking with+ * an internal cap (~3s) — the dispatcher already retries.+ *+ * Runs on the dispatcher thread, so every UI touch here goes through+ * [mainHandler]. The flash and the status line are not decoration:+ * they are the only way the user can tell *when* the camera actually+ * took something, on the most invasive screen this device has.+ */+ fun sampleFrame(maxSide: Int = 1280): String? {+ val act = instance ?: return null+ val capture = act.imageCapture ?: return null+ // Let the sensor finish auto-exposing before the first shot after+ // a bind. Only ever waits once per bind — a monitoring loop that+ // samples every 30s pays nothing.+ val settle = EXPOSURE_SETTLE_MS -+ (android.os.SystemClock.uptimeMillis() - act.boundAt)+ if (settle > 0) {+ act.flashStatus(AppLocale.str(act, R.string.dev_look_focusing), holdMs = settle)+ Thread.sleep(settle)+ }+ val f = File(act.cacheDir, "look_${System.currentTimeMillis()}.jpg")+ val done = java.util.concurrent.CountDownLatch(1)+ val ok = AtomicReference<Boolean>(false)+ mainHandler.post { act.onShutter() }+ capture.takePicture(+ androidx.camera.core.ImageCapture.OutputFileOptions.Builder(f).build(),+ ContextCompat.getMainExecutor(act),+ object : androidx.camera.core.ImageCapture.OnImageSavedCallback {+ override fun onImageSaved(output: androidx.camera.core.ImageCapture.OutputFileResults) {+ ok.set(true); done.countDown()+ act.flashStatus(AppLocale.str(act, R.string.dev_look_seen))+ }+ override fun onError(exc: ImageCaptureException) {+ done.countDown()+ act.flashStatus(AppLocale.str(act, R.string.dev_look_unclear))+ }+ },+ )+ if (!done.await(3, java.util.concurrent.TimeUnit.SECONDS) || !ok.get()) {+ f.delete(); return null+ }+ return runCatching {+ val bmp = BitmapFactory.decodeFile(f.absolutePath) ?: return null+ val w = bmp.width; val h = bmp.height+ if (w <= 0 || h <= 0) return null+ val scale = maxSide.toFloat() / maxOf(w, h)+ val out = if (scale < 1f) Bitmap.createScaledBitmap(+ bmp, (w * scale).toInt(), (h * scale).toInt(), true+ ) else bmp+ val bos = ByteArrayOutputStream()+ out.compress(Bitmap.CompressFormat.JPEG, 80, bos)+ f.delete()+ Base64.encodeToString(bos.toByteArray(), Base64.NO_WRAP)+ }.getOrNull()+ }+ }++ @Volatile private var previewView: PreviewView? = null+ @Volatile private var imageCapture: ImageCapture? = null+ private var viewfinder: ViewfinderView? = null+ private var status: StatusPill? = null+ @Volatile private var closingByBrain = false++ /**+ * Strings in the user's language. Refreshed in [onCreate].+ *+ * UI here is a full-screen Activity, but the overlays elsewhere are+ * Service-owned, so the app reads its language through [AppLocale]+ * everywhere rather than relying on per-app locale. Views are still built+ * on `this`; only text comes from [lctx].+ */+ private var lctx: Context = this++ /**+ * When the current camera was bound, in `SystemClock.uptimeMillis`.+ * Reset on every [rebind] — switching to the front lens starts the+ * exposure ramp over, so the settle applies there too.+ */+ @Volatile private var boundAt = 0L++ @SuppressLint("SetTextI18n")+ override fun onCreate(savedInstanceState: Bundle?) {+ super.onCreate(savedInstanceState)+ instance = this+ lctx = AppLocale.wrap(this)+ // Full-bleed preview; our own chrome insets itself (avoidSystemBars).+ WindowCompat.setDecorFitsSystemWindows(window, false)++ if (checkSelfPermission(Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {+ requestPermissions(arrayOf(Manifest.permission.CAMERA), 4244)+ return+ }+ buildUi()+ rebind(usingFront)+ }++ override fun onRequestPermissionsResult(+ requestCode: Int, permissions: Array<out String>, grantResults: IntArray,+ ) {+ super.onRequestPermissionsResult(requestCode, permissions, grantResults)+ if (grantResults.firstOrNull() == PackageManager.PERMISSION_GRANTED) {+ buildUi()+ rebind(usingFront)+ } else leave()+ }++ /** Shutter pressed — pulse the brackets. Main thread. */+ private fun onShutter() {+ viewfinder?.flash()+ }++ /**+ * Say what just happened, then fall back to the resting line. Safe from+ * any thread; [sampleFrame]'s CameraX callbacks land on the main executor+ * but the call itself comes off the dispatcher.+ */+ private fun flashStatus(text: String, holdMs: Long = STATUS_HOLD_MS) {+ mainHandler.post {+ status?.setStatus(text)+ mainHandler.postDelayed({ status?.setStatus(restingStatus()) }, holdMs)+ }+ }++ private fun restingStatus(): String =+ if (usingFront) lctx.getString(R.string.dev_look_resting_front)+ else lctx.getString(R.string.dev_look_resting)++ @SuppressLint("SetTextI18n")+ private fun buildUi() {+ val root = stage+ previewView = PreviewView(this).apply {+ // COMPATIBLE = TextureView: keeps the floating ball + edge glow+ // visible over the camera feed (SurfaceView would cover them).+ implementationMode = PreviewView.ImplementationMode.COMPATIBLE+ layoutParams = matchParent()+ }+ viewfinder = ViewfinderView(this).apply { layoutParams = matchParent() }+ val pill = StatusPill(this).apply {+ setStatus(restingStatus())+ layoutParams = FrameLayout.LayoutParams(+ ViewGroup.LayoutParams.WRAP_CONTENT, ViewGroup.LayoutParams.WRAP_CONTENT,+ Gravity.CENTER_HORIZONTAL or Gravity.BOTTOM+ ).apply { bottomMargin = dpI(28) }+ }+ status = pill+ val close = closePill(this) {+ // Remember it was the user, so the next look tells the brain the+ // truth instead of "camera not ready yet".+ userClosedAt = android.os.SystemClock.uptimeMillis()+ leave()+ }.apply {+ layoutParams = FrameLayout.LayoutParams(+ ViewGroup.LayoutParams.WRAP_CONTENT, ViewGroup.LayoutParams.WRAP_CONTENT,+ Gravity.END or Gravity.TOP+ ).apply { topMargin = dpI(12); marginEnd = dpI(12) }+ }+ avoidSystemBars(close, topDp = 12)+ avoidSystemBars(pill, bottomDp = 28)+ root.addView(previewView)+ root.addView(viewfinder)+ root.addView(pill)+ root.addView(close)+ }++ @SuppressLint("SetTextI18n")+ fun rebind(front: Boolean) {+ usingFront = front+ status?.setStatus(restingStatus())+ // Drop the old capture while the new lens binds: it belongs to a+ // camera that is being unbound, and leaving it visible would let a+ // sample through against the *previous* bind's settled clock.+ imageCapture = null+ val pv = previewView ?: return+ val future = ProcessCameraProvider.getInstance(this)+ future.addListener({+ val provider = future.get()+ val preview = Preview.Builder().build().also {+ it.setSurfaceProvider(pv.surfaceProvider)+ }+ val capture = ImageCapture.Builder()+ .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)+ .build()+ provider.unbindAll()+ runCatching {+ provider.bindToLifecycle(+ this,+ if (front) CameraSelector.DEFAULT_FRONT_CAMERA+ else CameraSelector.DEFAULT_BACK_CAMERA,+ preview, capture+ )+ }.onSuccess {+ // boundAt first: sampleFrame gates on imageCapture, so publishing+ // the capture before the timestamp would let one frame through+ // with a stale (or zero) bind time and skip the settle entirely.+ boundAt = android.os.SystemClock.uptimeMillis()+ imageCapture = capture+ // The feed is drawable. Idempotent, so the rebind that happens+ // when the brain swaps lenses mid-session does not re-run it.+ revealStage()+ }.onFailure {+ // A refused bind is otherwise invisible: the stage stays at+ // 0.94 and zero alpha, which reads as a black screen the user+ // cannot leave. Show it anyway with the chrome on it, so ✕+ // 完成 is there and the status line says what happened.+ android.util.Log.w("Body", "camera bind failed: ${it.message}")+ status?.setStatus(lctx.getString(R.string.dev_camera_unavailable))+ revealStage()+ }+ }, ContextCompat.getMainExecutor(this))+ }++ override fun onDestroy() {+ // Back key is the user too — same promise as the ✕ pill. Only the+ // brain's own close() is exempt, and it says so.+ if (!closingByBrain) userClosedAt = android.os.SystemClock.uptimeMillis()+ closingByBrain = false+ instance = null+ super.onDestroy()+ }+}++fun lookSessionOpen(): Boolean =+ LookActivity.instance?.let { !it.isFinishing } == truediff --git a/app/src/main/java/net/kuafuai/andee/device/NotificationRelayService.kt b/app/src/main/java/net/kuafuai/andee/device/NotificationRelayService.ktnew file mode 100644index 0000000..41f03dd--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/NotificationRelayService.kt@@ -0,0 +1,376 @@+package net.kuafuai.andee.device++import android.app.Notification+import android.content.ComponentName+import android.content.Context+import android.content.Intent+import android.os.Bundle+import android.provider.Settings+import android.service.notification.NotificationListenerService+import android.service.notification.StatusBarNotification+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import org.json.JSONArray+import org.json.JSONObject++/**+ * The proactive sense: every notification on the device lands here, gets+ * distilled to (pkg, title, text, when), and is kept in a small ring for the+ * brain to read on demand. This is what turns the ball from "answers when+ * asked" into "knows something happened".+ *+ * Delivery is BOTH:+ * - pull: `device.notifications` reads the ring (with since_ms increments)+ * - push: qualifying notifications are ALSO forwarded to the hub as+ * `notification` events — the same channel asr.final rides, so the brain+ * receives them as inbound messages. The hub side needs zero changes.+ *+ * Push FILTERING (the part that decides whether the brain gets interrupted):+ * message apps (chat/SMS/mail) push immediately — a human is trying to reach+ * the user, which is exactly what "proactive" means. Everything else+ * (updates, news, system junk) only lands in the ring; if the brain wants+ * ambient awareness it can poll. Constant popups would train the user to+ * ignore the ball, which is worse than not pushing at all.+ */+class NotificationRelayService : NotificationListenerService() {++ /**+ * Bound *right now*, which is the only question worth asking — see+ * [ensureBound] for why the permission flag can't answer it.+ */+ override fun onListenerConnected() {+ connected = true+ }++ override fun onListenerDisconnected() {+ connected = false+ // Whatever call notifications we thought were up, we can no longer+ // observe their removal. Better to reset and let the next round of+ // posts refill than to leave a dead key in [CallState] and mute the+ // wake word forever.+ net.kuafuai.andee.device.CallState.reset()+ // The documented way back. Asking for a rebind from inside the+ // disconnect is not circular: the system has already let go of us at+ // this point, and the request is queued against the component, not+ // against this instance.+ runCatching { requestRebind(ComponentName(this, NotificationRelayService::class.java)) }+ }++ override fun onNotificationRemoved(sbn: StatusBarNotification?) {+ sbn ?: return+ // Was this key one we counted? Read before clearing so we can log the+ // transition. Cheap; the set is a handful of entries at most.+ val wasCall = net.kuafuai.andee.device.CallState.isActive+ // Unconditional clear: cheaper than re-checking whether this was ever+ // a call, and safe — clearing a key we never marked is a no-op.+ net.kuafuai.andee.device.CallState.clear(sbn.key)+ if (wasCall && !net.kuafuai.andee.device.CallState.isActive) {+ android.util.Log.i(+ "Body",+ "call state: CLEARED (last was ${sbn.packageName}) — wake word may resume",+ )+ }+ }++ /**+ * Whether [sbn] is a live phone / VoIP call notification.+ *+ * Two layers, because no single signal covers every app:+ *+ * - `CATEGORY_CALL` — the platform's own contract. AOSP dialer,+ * WhatsApp, Telegram set it. That is the free path; when it fires we+ * do not need to look at anything else.+ *+ * - **WeChat/QQ do not set `CATEGORY_CALL`** (measured on real device,+ * 2026-09-22: voice call notif came in with `cat=null`). Their voice+ * call notif carries a distinctive text — `语音通话中` / `视频通话+ * 中` / `来电` — and they are the *only* WeChat/QQ notifications with+ * that text; ordinary chat messages read "13个联系人发来66条消息" or+ * "张三: 在吗", never those exact strings. So text pattern is a safe+ * fallback here.+ *+ * The text patterns are Chinese-only on purpose: this fallback exists+ * for WeChat/QQ specifically. Anything else uses the CATEGORY_CALL path+ * or is not something we can reliably tell apart from a chat message.+ */+ private fun isCallNotification(sbn: StatusBarNotification): Boolean {+ if (sbn.notification.category == Notification.CATEGORY_CALL) return true+ if (sbn.packageName in CN_VOICE_APPS) {+ val extras = sbn.notification.extras ?: return false+ val title = extras.getCharSequence(Notification.EXTRA_TITLE)?.toString().orEmpty()+ val text = extras.getCharSequence(Notification.EXTRA_TEXT)?.toString().orEmpty()+ for (pat in CN_CALL_KEYWORDS) {+ if (pat in title || pat in text) return true+ }+ }+ return false+ }++ /** Apps whose call notifications skip CATEGORY_CALL, so we fall back on text. */+ private val CN_VOICE_APPS = setOf(+ "com.tencent.mm", // 微信+ "com.tencent.mobileqq", // QQ+ )++ /**+ * Text patterns that identify a voice/video call notification inside+ * [CN_VOICE_APPS]. Ordered by specificity; the more specific patterns+ * (e.g. "语音通话中") are unambiguous, "来电" is broad but only checked+ * against apps in [CN_VOICE_APPS].+ *+ * **Never route these through `AppLocale`.** They are compared against text+ * WeChat and the other apps in [CN_VOICE_APPS] write into their own+ * notifications, and those follow *their* language setting, not ours.+ * Translating this list does not change a word the user reads — it makes+ * call detection stop working on exactly the devices where the strings+ * would have changed, which is silent: a missed call simply never becomes+ * an event. Same rule for the unread-count pattern in+ * [onNotificationPosted] and for `ScreenController`'s clipboard labels.+ */+ private val CN_CALL_KEYWORDS = listOf(+ "语音通话中",+ "视频通话中",+ "正在语音通话",+ "正在视频通话",+ "来电",+ )++ override fun onNotificationPosted(sbn: StatusBarNotification?) {+ sbn ?: return+ if (sbn.packageName == packageName) return // our own stuff, e.g. ASR toast++ val extras: Bundle? = sbn.notification.extras+ val title = extras?.getCharSequence(Notification.EXTRA_TITLE)?.toString().orEmpty()+ val text = extras?.getCharSequence(Notification.EXTRA_TEXT)?.toString().orEmpty()++ val isCall = isCallNotification(sbn)+ android.util.Log.i(+ "Body",+ "notif posted pkg=${sbn.packageName} cat=${sbn.notification.category} " ++ "isCall=$isCall title='${title.take(40)}' text='${text.take(60)}' key=${sbn.key}",+ )+ if (isCall) {+ net.kuafuai.andee.device.CallState.mark(sbn.key)+ android.util.Log.i(+ "Body",+ "call state: ACTIVE (${sbn.packageName}) — wake word will release mic",+ )+ }+ if (extras == null) return+ if (title.isEmpty() && text.isEmpty()) return+ // Notification.text can be "隋小波: 在吗" (one message) or+ // "[12条]隋小波: 在吗" (a stack: N unread, latest shown). The brain+ // needs the difference — "reply to one message" vs "a conversation+ // with 12 unread happened while you were away" — so it is parsed here+ // rather than leaving a "[12条]" blob for the LLM to guess at.+ //+ // The "条" in the pattern is the sending app's word, not ours. Do not+ // translate it — see [CN_CALL_KEYWORDS].+ var count = 1+ var headline = text+ val m = Regex("^\\[(\\d+)条]\\s*(.*)$").find(text.trim())+ if (m != null) {+ count = m.groupValues[1].toIntOrNull() ?: 1+ headline = m.groupValues[2]+ }+ val n = JSONObject()+ .put("pkg", sbn.packageName)+ .put("app", appLabel(sbn.packageName))+ .put("title", title.take(80))+ .put("text", text.take(160))+ .put("count", count)+ .put("latest", headline.take(160))+ .put("when", sbn.postTime)+ Store.add(n)+ if (sbn.packageName in PUSH_PKGS) {+ // One hop to the main thread, where the task flag and CardUi both+ // live. onNotificationPosted arrives on a binder thread; reading+ // dispatcher state cross-thread is exactly the kind of stale read+ // the hop exists to avoid.+ net.kuafuai.andee.ui.CardUi.post {+ if (net.kuafuai.andee.ScreenBodyService.get()+ ?.dispatcherForNotification()?.isTaskActive() == true+ ) {+ // A task is running: the ball is busy gripping the ledge+ // and the brain is mid-job on this very screen. A+ // signboard now would yank the window out from under the+ // perch and queue the user's answer against the task's+ // own actions. The message stays in the ring buffer+ // (device.notifications) — it gets read when the job is+ // done, not in the middle of it.+ return@post+ }+ showSignboardFor(sbn, n)+ }+ }+ }++ private fun showSignboardFor(+ sbn: StatusBarNotification,+ n: JSONObject,+ ) {+ // Card-first, not brain-first: a quiet signboard asking the user+ // beats waking the brain on every ping. Only a tap on 看看+ // actually forwards to the brain — the user is the spam filter.+ //+ // The labels come from [lctx] because this is a Service: per-app locale+ // does not reach a Service-owned window, so the signboard is built in+ // the user's language explicitly. Wrap locally rather than caching, so a+ // language switch takes effect on the next notification.+ val lctx = AppLocale.wrap(this)+ val latest = n.optString("latest")+ val count = n.optInt("count", 1)+ val appLabel = appLabel(sbn.packageName)+ val cardTitle = if (count > 1) {+ lctx.getString(R.string.dev_notif_card_unread, appLabel, count)+ } else {+ lctx.getString(R.string.dev_notif_card_title, appLabel)+ }+ val lookLabel = lctx.getString(R.string.dev_notif_card_look)+ val ignoreLabel = lctx.getString(R.string.dev_notif_card_ignore)+ net.kuafuai.andee.ui.CardUi.ask(+ "$cardTitle\n$latest",+ listOf(lookLabel, ignoreLabel),+ timeoutMs = 20_000L,+ ) { result ->+ if (result.button == lookLabel) {+ // "Heard you, on it" — the same reaction the ball gives to+ // a spoken instruction: perch on the edge, light the rim,+ // stay lit while the brain works. Reused rather than+ // reimplemented so the notification path can never drift+ // from the voice path in what "working" looks like. The+ // 5-minute leak-guard backstop applies too: if the brain+ // never answers, the glow still goes out by itself.+ //+ // Order matters: this runs inside CardUi's close, which+ // still has hideSignboard pending behind it. Lighting the+ // task first (perchedByTask=true) is what tells+ // hideSignboard "the ledge is wanted", so the shrink-then-+ // slide there happens as one motion instead of the ball+ // settling home first and hopping after.+ net.kuafuai.andee.ScreenBodyService.get()+ ?.dispatcherForNotification()?.beginTask()+ Pusher.forward(n)+ }+ }+ }++ private fun appLabel(pkg: String): String = when (pkg) {+ "com.tencent.mm" -> "微信"+ "com.tencent.mobileqq" -> "QQ"+ "com.alibaba.android.rimet" -> "钉钉"+ "com.ss.android.lark" -> "飞书"+ "com.whatsapp" -> "WhatsApp"+ "com.android.mms", "com.android.messaging" -> "短信"+ else -> pkg.substringAfterLast('.')+ }++ /** Who is allowed to interrupt the brain. Chat first; trim by taste. */+ private val PUSH_PKGS = setOf(+ "com.tencent.mm", // 微信+ "com.tencent.mobileqq", // QQ+ "com.alibaba.android.rimet", // 钉钉+ "com.ss.android.lark", // 飞书+ "com.whatsapp",+ "com.immomo.momo", "com.sdu.didi.psnger",+ "com.android.mms", "com.android.messaging", // 短信(含验证码)+ "com.gmail", "com.sina.weibo",+ )++ /** Forwards to the hub over the shared WS client, if it's connected. */+ private object Pusher {+ fun forward(n: JSONObject) {+ val client = net.kuafuai.andee.net.BodyWsClient.current ?: return+ client.sendEvent("notification", n)+ }+ }++ object Store {+ private const val CAP = 50+ private val lock = Any()+ private val ring = ArrayDeque<JSONObject>()++ fun add(n: JSONObject) = synchronized(lock) {+ ring.addLast(n)+ while (ring.size > CAP) ring.removeFirst()+ }++ fun snapshot(sinceMs: Long, limit: Int): JSONArray = synchronized(lock) {+ val arr = JSONArray()+ for (n in ring) {+ if (n.optLong("when", 0) >= sinceMs) {+ arr.put(n)+ if (arr.length() >= limit) break+ }+ }+ arr+ }++ fun count(): Int = synchronized(lock) { ring.size }+ }++ companion object {++ /**+ * Whether the system currently has us bound. Not derivable from the+ * permission flag — see [ensureBound].+ */+ @Volatile+ private var connected = false++ /**+ * Kick the system into rebinding us if the permission is granted but+ * nothing is listening.+ *+ * This exists because of a failure mode that looks exactly like the+ * feature being deleted: reinstalling the APK kills the process, and+ * the system frequently does *not* rebind the listener afterwards. The+ * permission row in Settings still reads as on, `status()` used to+ * agree with it, and every notification silently went nowhere — no+ * signboard, no error, nothing in the log. A developer reinstalling+ * twenty times a day hits this constantly, and so does a user after a+ * store update.+ *+ * `requestRebind` is the platform's own answer to it. Cheap, idempotent+ * and a no-op when we are already bound, so it is safe to fire on every+ * service start.+ */+ fun ensureBound(context: Context) {+ if (connected) return+ val cn = ComponentName(context, NotificationRelayService::class.java)+ if (!isPermitted(context, cn)) return // nothing to rebind; the user has to grant it+ runCatching { requestRebind(cn) }+ }++ private fun isPermitted(context: Context, cn: ComponentName): Boolean =+ Settings.Secure.getString(+ context.contentResolver, "enabled_notification_listeners"+ )?.contains(cn.flattenToString()) == true++ /** Is the listener permitted *and* actually bound? Says so honestly if not. */+ fun status(context: Context): JSONObject {+ val cn = ComponentName(context, NotificationRelayService::class.java)+ val permitted = isPermitted(context, cn)+ return JSONObject()+ .put("listener_enabled", permitted)+ // Reported separately from the permission on purpose: the two+ // disagree after every reinstall, and the old single flag+ // reported the permission — i.e. it claimed everything was fine+ // in precisely the case where notifications were being dropped.+ .put("listener_connected", connected)+ .put("buffered", Store.count())+ .put(+ "hint",+ when {+ !permitted ->+ "Notification access is off. Ask the user: Settings → Notifications → Notification access → allow this app."++ !connected ->+ "Notification access is granted but the listener is not bound; a rebind has been requested. If this persists, ask the user to toggle the permission off and on."++ else -> ""+ }+ )+ }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/device/PermissionRequestActivity.kt b/app/src/main/java/net/kuafuai/andee/device/PermissionRequestActivity.ktnew file mode 100644index 0000000..d71b577--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/PermissionRequestActivity.kt@@ -0,0 +1,24 @@+package net.kuafuai.andee.device++import androidx.appcompat.app.AppCompatActivity+import android.os.Bundle++/**+ * Translucent trampoline that hosts the runtime-permission dialogs. The app+ * is service-first (no main activity), but requestPermissions() needs a+ * live Activity window. Shows nothing, asks, dies.+ */+class PermissionRequestActivity : AppCompatActivity() {+ override fun onCreate(savedInstanceState: Bundle?) {+ super.onCreate(savedInstanceState)+ val perms = intent.getStringArrayExtra("permissions") ?: run { finish(); return }+ requestPermissions(perms, 4242)+ }++ override fun onRequestPermissionsResult(+ requestCode: Int, permissions: Array<out String>, grantResults: IntArray,+ ) {+ super.onRequestPermissionsResult(requestCode, permissions, grantResults)+ finish()+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/device/PermissionsController.kt b/app/src/main/java/net/kuafuai/andee/device/PermissionsController.ktnew file mode 100644index 0000000..ad6335a--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/PermissionsController.kt@@ -0,0 +1,151 @@+package net.kuafuai.andee.device++import android.content.Context+import android.content.Intent+import android.net.Uri+import android.provider.Settings+import androidx.annotation.StringRes+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import org.json.JSONArray+import org.json.JSONObject++/**+ * The polite permission concierge.+ *+ * Runtime permissions die on reinstall; the special ones (notification+ * access, overlay) live in Settings the app cannot flip itself. Instead of+ * the brain parroting "go to settings", this module knows the EXACT missing+ * grants, says them in one human sentence, and — on confirm — deep-links the+ * user to the right settings screen. One [device.permissions] call replaces+ * the whole "which permission? where? how?" dance.+ *+ * Intent EXTRA for a permission's app settings page:+ * Settings.ACTION_APPLICATION_DETAILS_SETTINGS + package uri+ * For notification access: Settings.ACTION_NOTIFICATION_LISTENER_SETTINGS.+ */+class PermissionsController(private val context: Context) {++ /**+ * What a grant is worth to the ball — used for one honest summary line.+ *+ * @param featureRes the human phrase for this grant, as a resource id: it+ * is shown to the user in the composed `say` sentence, so it has to+ * follow the UI language ([AppLocale]) rather than be a literal.+ */+ private data class Perm(+ val id: String,+ val permission: String?,+ val special: String?, // "notification" | "overlay" | null+ @StringRes val featureRes: Int,+ )++ private val catalog = listOf(+ Perm("location", "android.permission.ACCESS_FINE_LOCATION", null, R.string.dev_perm_feature_location),+ Perm("steps", "android.permission.ACTIVITY_RECOGNITION", null, R.string.dev_perm_feature_steps),+ Perm("contacts", "android.permission.READ_CONTACTS", null, R.string.dev_perm_feature_contacts),+ Perm("call_phone", "android.permission.CALL_PHONE", null, R.string.dev_perm_feature_call_phone),+ Perm("call_log", "android.permission.READ_CALL_LOG", null, R.string.dev_perm_feature_call_log),+ Perm("sms", "android.permission.READ_SMS", null, R.string.dev_perm_feature_sms),+ Perm("sms_send", "android.permission.SEND_SMS", null, R.string.dev_perm_feature_sms_send),+ Perm("calendar", "android.permission.READ_CALENDAR", null, R.string.dev_perm_feature_calendar),+ Perm("calendar_write", "android.permission.WRITE_CALENDAR", null, R.string.dev_perm_feature_calendar_write),+ Perm("notifications", null, "notification", R.string.dev_perm_feature_notifications),+ Perm("overlay", null, "overlay", R.string.dev_perm_feature_overlay),+ )++ fun status(): JSONObject {+ val granted = JSONArray()+ val missing = JSONArray()+ for (p in catalog) {+ val ok = when {+ p.permission != null ->+ context.checkSelfPermission(p.permission) == android.content.pm.PackageManager.PERMISSION_GRANTED+ // Notification: the permission flag alone lies after every+ // reinstall (zombie grant — enabled in Settings, never bound).+ // Count it as granted only when the listener is actually+ // connected; a zombie surfaces in `missing` and the guide can+ // fix it, instead of "everything looks fine" while every+ // notification quietly vanishes.+ p.special == "notification" -> NotificationRelayService.status(context)+ .optBoolean("listener_connected")+ p.special == "overlay" -> Settings.canDrawOverlays(context)+ else -> false+ }+ val o = JSONObject()+ .put("id", p.id)+ .put("feature", AppLocale.str(context, p.featureRes))+ (if (ok) granted else missing).put(o)+ }+ return JSONObject()+ .put("granted_count", granted.length())+ .put("missing_count", missing.length())+ .put("granted", granted)+ .put("missing", missing)+ }++ /**+ * One sentence for the user + optional deep link. The brain should READ+ * this aloud (or show it), ask "开吗?", then call again with confirm=true+ * to fire the intent. Runtime permissions use the classic dialog+ * (needs an activity — we hand the request to the floating settings+ * host); special ones deep-link into Settings.+ */+ fun request(ids: JSONArray, confirm: Boolean): JSONObject {+ // The `say` sentence is shown/spoken to the user, so it is built in+ // their language. See [AppLocale] for why a Service context must wrap.+ val lctx = AppLocale.wrap(context)+ val wanted = buildList {+ for (i in 0 until ids.length()) {+ catalog.firstOrNull { it.id == ids.optString(i) }?.let { add(it) }+ }+ }+ if (wanted.isEmpty()) return JSONObject().put("error", "no known permission ids in request")++ val runtime = wanted.filter { it.permission != null }+ val special = wanted.filter { it.special == "notification" }+ val overlay = wanted.filter { it.special == "overlay" }++ val featureList = wanted.joinToString(lctx.getString(R.string.dev_perm_say_sep)) {+ lctx.getString(it.featureRes)+ }+ val say = buildString {+ append(lctx.getString(R.string.dev_perm_say_prefix, featureList))+ if (runtime.isNotEmpty()) append(lctx.getString(R.string.dev_perm_say_runtime))+ if (special.isNotEmpty()) append(lctx.getString(R.string.dev_perm_say_notification))+ }++ if (!confirm) {+ return JSONObject().put("say", say).put("confirm_needed", true).put("ids", ids)+ }++ // Fire the intents. Runtime permissions need an Activity to host the+ // dialog — the app has none in the classic sense, so route through+ // the dedicated PermissionRequestActivity (manifest-registered,+ // translucent, finishes itself).+ if (runtime.isNotEmpty()) {+ val i = Intent(context, PermissionRequestActivity::class.java).apply {+ putExtra("permissions", runtime.mapNotNull { it.permission }.toTypedArray())+ addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)+ }+ runCatching { context.startActivity(i) }+ }+ special.forEach {+ runCatching {+ context.startActivity(+ Intent(Settings.ACTION_NOTIFICATION_LISTENER_SETTINGS)+ .addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)+ )+ }+ }+ if (overlay.isNotEmpty()) {+ runCatching {+ context.startActivity(+ Intent(Settings.ACTION_MANAGE_OVERLAY_PERMISSION, Uri.parse("package:${context.packageName}"))+ .addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)+ )+ }+ }+ return JSONObject().put("started", true).put("say", say)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/device/ScanActivity.kt b/app/src/main/java/net/kuafuai/andee/device/ScanActivity.ktnew file mode 100644index 0000000..9faa157--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/ScanActivity.kt@@ -0,0 +1,220 @@+package net.kuafuai.andee.device++import android.Manifest+import android.annotation.SuppressLint+import android.content.Context+import android.content.Intent+import android.content.pm.PackageManager+import android.os.Bundle+import android.view.Gravity+import android.view.ViewGroup+import android.widget.FrameLayout+import androidx.camera.core.CameraSelector+import androidx.camera.core.ImageAnalysis+import androidx.camera.core.ImageProxy+import androidx.camera.core.Preview+import androidx.camera.lifecycle.ProcessCameraProvider+import androidx.camera.view.PreviewView+import androidx.core.content.ContextCompat+import androidx.core.view.WindowCompat+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.ui.StatusPill+import net.kuafuai.andee.ui.ViewfinderView+import net.kuafuai.andee.ui.avoidSystemBars+import net.kuafuai.andee.ui.closePill+import net.kuafuai.andee.ui.dpI+import net.kuafuai.andee.ui.matchParent+import com.google.mlkit.vision.barcode.BarcodeScannerOptions+import com.google.mlkit.vision.barcode.BarcodeScanning+import com.google.mlkit.vision.barcode.common.Barcode+import com.google.mlkit.vision.common.InputImage+import java.util.concurrent.Executors+import java.util.concurrent.atomic.AtomicBoolean++/**+ * The ball's eye: full-screen camera preview with live barcode scanning.+ * A code found → vibrate-lite (result banner) → finish with the payload.+ * The brain gets: {value, format} plus rawText when the code is a URL/text.+ *+ * Opening sequence (device.scan): check CAMERA permission → denied? fire the+ * runtime dialog from here directly (we ARE the activity) → granted → scan.+ * So one tool call ends in either a scanned code or an honest error string.+ *+ * Result delivery: the controller polls [LastResult] over the WS; simpler+ * than a result-callback chain through the service for now.+ *+ * In and out are [net.kuafuai.andee.ui.StageActivity]'s, like every other+ * full-screen face — see that class for why leaving one of these used to+ * flash.+ */+class ScanActivity : net.kuafuai.andee.ui.StageActivity() {++ companion object {+ var LastResult: android.os.Bundle? = null+ const val EXTRA_AUTO_CLOSE_MS = "auto_close_ms"+ private const val HIT_GREEN = 0xFF07C160.toInt()+ }++ private val handling = AtomicBoolean(false)+ private var status: StatusPill? = null+ private var viewfinder: ViewfinderView? = null+ private var autoCloseAt = 0L++ /** Strings in the user's language. Refreshed in [startCamera]. */+ private var lctx: Context = this++ @SuppressLint("SetTextI18n")+ override fun onCreate(savedInstanceState: Bundle?) {+ super.onCreate(savedInstanceState)+ // Full-bleed preview; our own chrome insets itself (avoidSystemBars).+ WindowCompat.setDecorFitsSystemWindows(window, false)++ if (checkSelfPermission(Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {+ requestPermissions(arrayOf(Manifest.permission.CAMERA), 4243)+ return+ }+ autoCloseAt = System.currentTimeMillis() ++ (intent.getLongExtra(EXTRA_AUTO_CLOSE_MS, 60_000L))+ startCamera()+ }++ override fun onRequestPermissionsResult(+ requestCode: Int, permissions: Array<out String>, grantResults: IntArray,+ ) {+ super.onRequestPermissionsResult(requestCode, permissions, grantResults)+ if (grantResults.firstOrNull() == PackageManager.PERMISSION_GRANTED) {+ autoCloseAt = System.currentTimeMillis() ++ (intent.getLongExtra(EXTRA_AUTO_CLOSE_MS, 60_000L))+ startCamera()+ } else {+ handling.set(true)+ LastResult = Bundle().apply {+ putString("error", "camera permission denied")+ }+ leave()+ }+ }++ private fun startCamera() {+ lctx = AppLocale.wrap(this)+ val root = stage+ val previewView = PreviewView(this).apply {+ layoutParams = matchParent()+ }+ viewfinder = ViewfinderView(this).apply { layoutParams = matchParent() }+ val pill = StatusPill(this).apply {+ setStatus(lctx.getString(R.string.dev_scan_align))+ layoutParams = FrameLayout.LayoutParams(+ ViewGroup.LayoutParams.WRAP_CONTENT, ViewGroup.LayoutParams.WRAP_CONTENT,+ Gravity.CENTER_HORIZONTAL or Gravity.BOTTOM+ ).apply { bottomMargin = dpI(28) }+ }+ status = pill+ val close = closePill(this) { recordCancel(); leave() }.apply {+ layoutParams = FrameLayout.LayoutParams(+ ViewGroup.LayoutParams.WRAP_CONTENT, ViewGroup.LayoutParams.WRAP_CONTENT,+ Gravity.END or Gravity.TOP+ ).apply { topMargin = dpI(12); marginEnd = dpI(12) }+ }+ avoidSystemBars(close, topDp = 12)+ avoidSystemBars(pill, bottomDp = 28)+ root.addView(previewView)+ root.addView(viewfinder)+ root.addView(pill)+ root.addView(close)++ val future = ProcessCameraProvider.getInstance(this)+ future.addListener({+ val provider = future.get()+ val preview = Preview.Builder().build().also {+ it.setSurfaceProvider(previewView.surfaceProvider)+ }+ val scanner = BarcodeScanning.getClient(+ BarcodeScannerOptions.Builder()+ .setBarcodeFormats(Barcode.FORMAT_ALL_FORMATS)+ .build()+ )+ val analysis = ImageAnalysis.Builder()+ .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)+ .build()+ val exec = Executors.newSingleThreadExecutor()+ analysis.setAnalyzer(exec) { proxy ->+ @Suppress("DEPRECATION")+ val media = proxy.image+ if (media != null) {+ val input = InputImage.fromMediaImage(+ media, proxy.imageInfo.rotationDegrees+ )+ scanner.process(input)+ .addOnSuccessListener { codes ->+ if (codes.isNotEmpty() && handling.compareAndSet(false, true)) {+ onCode(codes.first())+ }+ }+ .addOnCompleteListener { proxy.close() }+ } else proxy.close()+ // auto-close watchdog+ if (System.currentTimeMillis() > autoCloseAt && handling.compareAndSet(false, true)) {+ LastResult = Bundle().apply { putString("error", "scan timed out") }+ runOnUiThread { leave() }+ }+ }+ provider.unbindAll()+ runCatching {+ provider.bindToLifecycle(+ this,+ CameraSelector.DEFAULT_BACK_CAMERA, preview, analysis+ )+ }.onSuccess { revealStage() }.onFailure {+ android.util.Log.w("Body", "scan bind failed: ${it.message}")+ status?.setStatus(lctx.getString(R.string.dev_camera_unavailable))+ handling.set(true)+ LastResult = Bundle().apply { putString("error", "camera bind failed") }+ leave()+ }+ }, ContextCompat.getMainExecutor(this))+ }++ private fun onCode(code: Barcode) {+ LastResult = Bundle().apply {+ putString("value", code.rawValue ?: code.rawBytes?.toString(Charsets.UTF_8) ?: "")+ putString("format", code.format.toString())+ putLong("when", System.currentTimeMillis())+ }+ runOnUiThread {+ status?.setStatus("✓ ${code.rawValue?.take(40)}")+ status?.setDotColor(HIT_GREEN)+ viewfinder?.flash(HIT_GREEN)+ // Long enough for the ✕ green and the value to register — this is+ // the only confirmation the user gets that the scan worked, and a+ // stage that shrinks out in 190 ms takes it away.+ window.decorView.postDelayed({ leave() }, 700)+ }+ }++ /**+ * The user gave up. Writing [LastResult] is the whole point: without it+ * the dispatcher's poll had nothing to find and span until its 30s+ * timeout, so backing out of a scan cost the brain half a minute of+ * silence before it heard anything at all.+ *+ * `handling` is the same latch the hit and timeout paths take, so+ * whichever outcome happened first keeps its result.+ */+ private fun recordCancel(): Boolean {+ if (!handling.compareAndSet(false, true)) return false+ LastResult = Bundle().apply { putString("error", "cancelled by user") }+ return true+ }++ override fun onDestroy() {+ // Covers the back key, which never passes through the ✕ pill.+ recordCancel()+ super.onDestroy()+ }+}++/** Controller-side helper: is the camera granted? */+fun cameraGranted(context: Context): Boolean =+ context.checkSelfPermission(Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTEDdiff --git a/app/src/main/java/net/kuafuai/andee/device/SelfCheck.kt b/app/src/main/java/net/kuafuai/andee/device/SelfCheck.ktnew file mode 100644index 0000000..e8b0d18--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/device/SelfCheck.kt@@ -0,0 +1,898 @@+package net.kuafuai.andee.device++import android.Manifest+import android.content.ComponentName+import android.content.Context+import android.content.Intent+import android.content.pm.PackageManager+import android.net.ConnectivityManager+import android.net.NetworkCapabilities+import android.net.Uri+import android.os.PowerManager+import android.os.StatFs+import android.provider.Settings+import androidx.annotation.StringRes+import net.kuafuai.andee.R+import net.kuafuai.andee.config.VoiceConfig+import okhttp3.MediaType.Companion.toMediaType+import okhttp3.OkHttpClient+import okhttp3.Request+import okhttp3.RequestBody.Companion.toRequestBody+import org.json.JSONArray+import org.json.JSONObject+import java.io.IOException+import java.net.InetSocketAddress+import java.net.Socket+import java.util.concurrent.TimeUnit++/**+ * "Is this device actually ready?", answered in one list.+ *+ * **Why this exists.** Everything this app does fails *quietly*. A missing+ * overlay permission means the ball never draws and nothing says why. A backend+ * that cannot be reached means the ball spins once and goes silent. A stale+ * notification grant means notifications are dropped while Settings happily+ * reports the permission as on. Each of those has been reported by the user as+ * "坏了" at least once, and every one of them is decidable in advance.+ *+ * **One list, three readers.** [net.kuafuai.andee.ScreenBodyService] runs it at+ * startup and puts up a nudge card if anything is wrong; that same service puts+ * it up on demand as a card from `✓` in the assistant's control bar; and+ * [net.kuafuai.andee.ui.SelfCheckActivity] hosts the very same card from the+ * launcher icon, for the device where the service is not running at all — see+ * [net.kuafuai.andee.ui.SelfCheckUi], which is the one renderer behind all of+ * them. The list is built here and nowhere else, so no two of those can ever+ * disagree about what is wrong — which is the failure a second hand-written+ * checklist would have within a week.+ *+ * **Not a doctor.** It reports and offers the door; it does not repair.+ *+ * ### Levels+ *+ * * [Level.FAIL] — dead right now. The user's complaint will be "没反应".+ * * [Level.WARN] — it works, but something downstream degrades silently (a+ * zombie grant, an adb-only permission, a nearly-full disk).+ * * [Level.NOTE] — not a problem, just a fact worth being able to read (the+ * wake phrase is not taught yet, a probe was skipped). **Never raises a+ * card.**+ * * [Level.OK] — fine. Shown on the page, counted nowhere.+ *+ * The WARN/NOTE split is what keeps this from being nagware: only [FAIL] and+ * [WARN] may interrupt the user.+ */+object SelfCheck {++ enum class Level { OK, NOTE, WARN, FAIL }++ /**+ * How a row is fixed. Deliberately a small closed set of *values* rather+ * than a lambda: the list is built off the main thread, and an `Intent`+ * described at 200 ms is still good at 3 s, where a captured closure+ * holding a Context would be a leak waiting to be filed as a bug.+ */+ sealed interface Fix {+ /**+ * A system Settings screen. [data] is an optional `package:` uri.+ *+ * [extras] exist for the one screen that cannot be named by action+ * alone. `ACTION_NOTIFICATION_LISTENER_DETAIL_SETTINGS` is the *detail*+ * page — one app's notification-access switch — and it needs the+ * component in an extra to know which app. It is the screen worth+ * having: the plain `ACTION_NOTIFICATION_LISTENER_SETTINGS` list does+ * not resolve to anything at all on a HONOR, so the button built on it+ * was a tap that did nothing. Still values, not a closure — see the+ * interface doc — so they are `List<Pair<String, String>>` of+ * extra-name to extra-value and the renderer folds them in.+ */+ data class Screen(+ val action: String,+ val data: String? = null,+ val extras: List<Pair<String, String>> = emptyList(),+ /**+ * The button's word, when the default 去开启 would describe the+ * errand wrongly. Zero means "no opinion" — the renderer falls back+ * to 去开启.+ *+ * It exists for the notification-access row. In the zombie state the+ * switch already reads as *on*, so 去开启 invites exactly the wrong+ * action: open the page, see it on, leave — and the row stays red.+ * The errand there is 关闭再打开, and saying so on the button is the+ * whole hint. A word belongs to the instance rather than the action+ * because the same action is the right door in both states.+ */+ @StringRes val label: Int = 0,+ ) : Fix++ /**+ * Somewhere off the device — currently only the ADBKeyboard download.+ *+ * Its own member rather than `Screen(ACTION_VIEW, url)` because the+ * button it produces is a different word: 去下载 is not 去开启, and a+ * labelled-as-Settings button that opens a browser is the small lie+ * this whole list is built to avoid. The plumbing is otherwise+ * identical, which is the point — one new value, no new path.+ */+ data class Link(val url: String) : Fix++ /** Hosted by [PermissionRequestActivity] — needs a live Activity window. */+ data class Grant(val permissions: List<String>) : Fix++ /**+ * No button can do it. [command] is shown to the user to copy, not run:+ * `WRITE_SECURE_SETTINGS` is `signature|privileged|development`, so only+ * `pm grant` over adb hands it over. Saying so plainly *is* the value of+ * the row — otherwise the user hunts Settings for a switch that is not+ * there.+ */+ data class Adb(val command: String) : Fix++ /** Ours to fix, in our own settings card. Needs the service to be up. */+ object OurSettings : Fix++ /** Nothing to offer. */+ object Nothing : Fix+ }++ /**+ * One row.+ *+ * [detailArgs] are substituted into [detail] by the caller and never baked+ * in here, because the sentence has to come out in the user's language and+ * only the caller knows which Context that is — see [net.kuafuai.andee.i18n.AppLocale].+ */+ data class Finding(+ val id: String,+ @StringRes val title: Int,+ val level: Level,+ @StringRes val detail: Int,+ val detailArgs: List<Any> = emptyList(),+ val fix: Fix = Fix.Nothing,+ ) {+ /** Whether there is a button to press, as opposed to words to read. */+ val actionable: Boolean+ get() = fix is Fix.Screen || fix is Fix.Link || fix is Fix.Grant ||+ fix is Fix.OurSettings+ }++ /** What may interrupt the user. See the class doc for why NOTE does not. */+ fun needsAttention(f: Finding): Boolean = f.level == Level.FAIL || f.level == Level.WARN++ /**+ * The whole list, in report order.+ *+ * @param probe whether to spend the two network round-trips (the backend,+ * then the voice host). **This blocks the calling thread** — up to ~20 s+ * on a black-holed network — so call it off the main thread. With+ * `probe = false` those two rows come back as [Level.NOTE] "检查中…",+ * which is what lets the page paint instantly and fill in behind.+ */+ fun run(context: Context, probe: Boolean = true): List<Finding> {+ val online = isOnline(context)+ return listOf(+ accessibility(context),+ overlay(context),+ network(online),+ brain(context),+ backend(context, probe, online),+ microphone(context),+ notificationAccess(context),+ adbKeyboard(context),+ secureSettings(context),+ voiceKey(context),+ voiceReach(probe, online),+ wakePhrase(context),+ background(context),+ storage(context),+ )+ }++ /** [run] with no network, for the caller's first paint. */+ fun local(context: Context): List<Finding> = run(context, probe = false)++ // ---- The checks ----++ /**+ * The service itself.+ *+ * Only ever meaningful from an Activity: if this were false where the+ * service could ask, the service would not be running to ask it. That is+ * exactly why [net.kuafuai.andee.ui.SelfCheckActivity] still exists — it is+ * the one surface in this app that can run while the accessibility service is+ * off, and therefore the only place this row can be red and still be read.+ * Every other host of this list is the service itself, and there the row is+ * necessarily green.+ */+ private fun accessibility(context: Context): Finding {+ val cn = ComponentName(context, net.kuafuai.andee.ScreenBodyService::class.java)+ val on = Settings.Secure.getString(+ context.contentResolver,+ Settings.Secure.ENABLED_ACCESSIBILITY_SERVICES,+ ).orEmpty().split(':').any { ComponentName.unflattenFromString(it) == cn }+ return Finding(+ id = "accessibility",+ title = R.string.check_title_accessibility,+ level = if (on) Level.OK else Level.FAIL,+ detail = if (on) R.string.check_detail_accessibility_on+ else R.string.check_detail_accessibility_off,+ fix = if (on) Fix.Nothing else Fix.Screen(Settings.ACTION_ACCESSIBILITY_SETTINGS),+ )+ }++ /**+ * `SYSTEM_ALERT_WINDOW`.+ *+ * The overlay is `TYPE_APPLICATION_OVERLAY`, not+ * `TYPE_ACCESSIBILITY_OVERLAY`, so the accessibility service is *not* a+ * licence to draw. Without this the window add throws,+ * `FloatingWindowUi.show()` catches it and returns false, and the service+ * runs perfectly with nothing on screen — the "没反应" that looks least like+ * a permission problem.+ */+ private fun overlay(context: Context): Finding {+ val on = Settings.canDrawOverlays(context)+ return Finding(+ id = "overlay",+ title = R.string.check_title_overlay,+ level = if (on) Level.OK else Level.FAIL,+ detail = if (on) R.string.check_detail_overlay_on else R.string.check_detail_overlay_off,+ fix = if (on) {+ Fix.Nothing+ } else {+ Fix.Screen(+ Settings.ACTION_MANAGE_OVERLAY_PERMISSION,+ "package:${context.packageName}",+ )+ },+ )+ }++ /**+ * Anything at all to talk over.+ *+ * Checked *before* the two probes and not after, because "no network" is+ * the cause of both of their failures — reporting all three would send the+ * user looking for three problems where there is one.+ */+ private fun network(online: Boolean): Finding = Finding(+ id = "network",+ title = R.string.check_title_network,+ level = if (online) Level.OK else Level.FAIL,+ detail = if (online) R.string.check_detail_network_on else R.string.check_detail_network_off,+ fix = if (online) Fix.Nothing else Fix.Screen(Settings.ACTION_WIRELESS_SETTINGS),+ )++ /**+ * Which brain, and is it told where to go.+ *+ * The *presence* half only. `ScreenBodyService.startBrain` already raises+ * its own card for exactly these two states, so the startup call passes+ * `announce = false` and this row is what the user sees instead — one card+ * per start, listing everything, rather than two cards arguing.+ */+ private fun brain(context: Context): Finding {+ val bc = VoiceConfig.brainConfig(context)+ val hubUrl = VoiceConfig.hubConfig(context).url+ val missing = if (bc.isLocal) bc.apiKey.isEmpty() else hubUrl.isEmpty()+ return Finding(+ id = "brain",+ title = R.string.check_title_brain,+ level = if (missing) Level.FAIL else Level.OK,+ detail = when {+ !missing -> R.string.check_detail_brain_on+ bc.isLocal -> R.string.check_detail_brain_key_missing+ else -> R.string.check_detail_brain_hub_missing+ },+ fix = if (missing) Fix.OurSettings else Fix.Nothing,+ )+ }++ /**+ * The backend, for real.+ *+ * Three ways to be configured-but-broken, worth telling apart because the+ * symptom is identical for all three — a ball that spins once and says+ * nothing:+ *+ * * **URL wrong** → the connection itself fails.+ * * **Key wrong** → HTTP 401/403.+ * * **Model name wrong** → HTTP 400. The likeliest of the three, since it+ * is the one a person types by hand.+ *+ * [probeBackend] decides them without generating a token where the gateway+ * allows it.+ */+ private fun backend(context: Context, probe: Boolean, online: Boolean): Finding {+ val bc = VoiceConfig.brainConfig(context)+ // Cloud mode has no model on this device to test: the hub owns the+ // backend, and whether the hub is reachable is the socket's business,+ // which `startHubClient` already logs.+ if (!bc.isLocal) {+ return Finding(+ id = "backend",+ title = R.string.check_title_model,+ level = Level.OK,+ detail = R.string.check_detail_model_cloud,+ )+ }+ if (bc.apiKey.isEmpty()) {+ return Finding(+ id = "backend",+ title = R.string.check_title_model,+ level = Level.OK,+ detail = R.string.check_detail_model_no_key,+ )+ }+ if (!probe || !online) {+ return Finding(+ id = "backend",+ title = R.string.check_title_model,+ level = Level.NOTE,+ detail = if (online) R.string.check_detail_checking+ else R.string.check_detail_skipped_offline,+ )+ }+ return when (val r = probeBackend(bc.baseUrl, bc.apiKey, bc.model)) {+ is Reach.Ok -> Finding(+ id = "backend",+ title = R.string.check_title_model,+ level = Level.OK,+ detail = R.string.check_detail_model_ok,+ detailArgs = listOf(bc.model),+ )++ is Reach.Throttled -> Finding(+ id = "backend",+ title = R.string.check_title_model,+ level = Level.WARN,+ detail = R.string.check_detail_model_throttled,+ fix = Fix.OurSettings,+ )++ is Reach.Bad -> Finding(+ id = "backend",+ title = R.string.check_title_model,+ level = Level.FAIL,+ detail = when (r.kind) {+ Kind.URL -> R.string.check_detail_model_url+ Kind.KEY -> R.string.check_detail_model_key+ Kind.MODEL -> R.string.check_detail_model_name+ Kind.OTHER -> R.string.check_detail_model_other+ },+ // The model-name case gets two arguments (what you asked for,+ // what is on offer); the rest get the gateway's own words.+ detailArgs = if (r.kind == Kind.MODEL) listOf(bc.model, r.detail)+ else listOf(r.detail),+ fix = Fix.OurSettings,+ )+ }+ }++ /**+ * The notification listener, which has the one failure mode in this app the+ * Settings app actively lies about.+ *+ * After a reinstall the grant survives, reads as enabled, and the listener+ * is never bound — so the flag says yes while every notification is+ * dropped. `NotificationRelayService.status` already keeps the two apart;+ * this is the surface that finally says so out loud.+ *+ * **Measured on a HONOR (MagicOS / Android 15), and the state is worse+ * there than the doc above implies.** `dumpsys notification` shows this app+ * in "Allowed notification listeners" — i.e. Settings genuinely displays the+ * permission as on, which is exactly what the user reported — while it is+ * absent from "Live notification listeners", and neither a process restart+ * nor `requestRebind` brings it back. Only a real off-then-on of the grant+ * does (`cmd notification disallow_listener` + `allow_listener` fixed it on+ * the spot). Writing `Settings.Secure.ENABLED_NOTIFICATION_LISTENERS` by+ * hand does **not**, even with `WRITE_SECURE_SETTINGS` — verified on the+ * same device — so there is no one-tap repair this app can perform and the+ * user has to do it in Settings. That makes [listenerSettings] load-bearing+ * rather than cosmetic: it has to open a screen that exists.+ */+ private fun notificationAccess(context: Context): Finding {+ val st = NotificationRelayService.status(context)+ val enabled = st.optBoolean("listener_enabled")+ val connected = st.optBoolean("listener_connected")+ return Finding(+ id = "notifications",+ title = R.string.check_title_notifications,+ level = if (connected) Level.OK else Level.WARN,+ detail = when {+ connected -> R.string.check_detail_notif_on+ enabled -> R.string.check_detail_notif_zombie+ else -> R.string.check_detail_notif_off+ },+ fix = if (connected) Fix.Nothing else listenerSettings(context, needsRepower = enabled),+ )+ }++ /**+ * The notification-access screen, by the platform name that resolves first.+ *+ * Two actions exist and they are not interchangeable in the field. The+ * *list* (`ACTION_NOTIFICATION_LISTENER_SETTINGS`) is the one every answer+ * on the internet names, and on a HONOR it matches no activity at all —+ * `cmd package resolve-activity` says "No activities found", so+ * [net.kuafuai.andee.ui.SelfCheckUi.launch] caught the exception and the+ * 去开启 button read as dead. The *detail* action+ * (`ACTION_NOTIFICATION_LISTENER_DETAIL_SETTINGS`, API 30) does resolve+ * there, to `NotificationAccessDetailsActivity` — one app's switch, which is+ * the smaller page anyway: the user lands on the toggle they have to flip+ * instead of a list of every app on the phone.+ *+ * Probed rather than assumed, because "API 30" is not a promise that a given+ * ROM kept the screen. The check is a `queryIntentActivities` with+ * `CATEGORY_DEFAULT`, which is what `startActivity` itself injects into an+ * implicit intent — so a `true` here means the tap will land.+ */+ private fun listenerSettings(context: Context, needsRepower: Boolean): Fix {+ val cn = ComponentName(context, NotificationRelayService::class.java).flattenToString()+ val detail = Intent(Settings.ACTION_NOTIFICATION_LISTENER_DETAIL_SETTINGS)+ .addCategory(Intent.CATEGORY_DEFAULT)+ val reachable = runCatching {+ context.packageManager.queryIntentActivities(detail, 0).isNotEmpty()+ }.getOrDefault(false)+ // 关闭再打开 in the zombie state, 去开启 in the merely-off one — see+ // Fix.Screen.label. The switch the user is about to see already reads+ // as on in the first case, so the button is the only place left to say+ // what has to happen; the detail says it too, in words.+ val label = if (needsRepower) R.string.check_fix_repower else 0+ return if (reachable) {+ Fix.Screen(+ Settings.ACTION_NOTIFICATION_LISTENER_DETAIL_SETTINGS,+ extras = listOf(EXTRA_LISTENER_COMPONENT to cn),+ label = label,+ )+ } else {+ Fix.Screen(Settings.ACTION_NOTIFICATION_LISTENER_SETTINGS, label = label)+ }+ }++ /**+ * ADBKeyboard — the headless IME every `type_text` goes through.+ *+ * `ScreenController.typeViaAdbKeyboard` has exactly one channel: a broadcast+ * to `com.android.adbkeyboard`, which must be an *enabled* IME for the+ * `commitText` to land. So "can the assistant type into other apps" is+ * decided by two facts on this device — is that keyboard installed, and has+ * the user enabled it — and neither has ever had a way of announcing itself.+ * The failure is a `type_text` that comes back with an error naming the+ * target app instead of the keyboard.+ *+ * **Three states, one row**, because they are three steps of one errand and+ * the fix changes as the user completes them: not there → download it; there+ * but off → enable it; enabled → [Level.OK]. Two rows would leave a+ * permanently red "not installed" beside a green "enabled".+ *+ * **[Level.WARN], matching [secureSettings] beside it.** Same capability,+ * same blast radius: only typing into other apps is lost, everything else on+ * this page still works. The two rows have to agree about how bad the same+ * broken feature is, or the list teaches the user to distrust its colours.+ *+ * The download is [ADB_KEYBOARD_APK] and the enable step is+ * `INPUT_METHOD_SETTINGS`, which is the screen that lists keyboards with the+ * toggle that actually turns this one on — not `showInputMethodPicker`,+ * which `ImeSwitch` refuses on purpose.+ */+ private fun adbKeyboard(context: Context): Finding {+ val installed = net.kuafuai.andee.ui.ImeSwitch.brainInstalled(context)+ val enabled = installed &&+ net.kuafuai.andee.ui.ImeSwitch.brainEnabled(context)+ return Finding(+ id = "adb_keyboard",+ title = R.string.check_title_adb_keyboard,+ level = if (enabled) Level.OK else Level.WARN,+ detail = when {+ enabled -> R.string.check_detail_adb_kb_on+ installed -> R.string.check_detail_adb_kb_off+ else -> R.string.check_detail_adb_kb_missing+ },+ fix = when {+ enabled -> Fix.Nothing+ installed -> Fix.Screen(Settings.ACTION_INPUT_METHOD_SETTINGS)+ else -> Fix.Link(ADB_KEYBOARD_APK)+ },+ )+ }++ /**+ * `WRITE_SECURE_SETTINGS`, which `type_text` needs.+ *+ * The one row with no button on purpose. It is+ * `signature|privileged|development`: `pm grant` over adb is the only way,+ * and this tablet's provisioning already does it. When it goes missing — a+ * factory reset, a new tablet — the honest answer is "plug the cable in",+ * not a Settings screen that does not contain the switch.+ *+ * [Level.WARN] rather than FAIL because everything except typing into other+ * apps still works without it.+ */+ private fun secureSettings(context: Context): Finding {+ val on = context.checkSelfPermission(WRITE_SECURE_SETTINGS) ==+ PackageManager.PERMISSION_GRANTED+ return Finding(+ id = "secure_settings",+ title = R.string.check_title_secure_settings,+ level = if (on) Level.OK else Level.WARN,+ detail = if (on) R.string.check_detail_secure_on+ else R.string.check_detail_secure_missing,+ fix = if (on) {+ Fix.Nothing+ } else {+ Fix.Adb("adb shell pm grant ${context.packageName} $WRITE_SECURE_SETTINGS")+ },+ )+ }++ /**+ * The microphone.+ *+ * Absent from [PermissionsController]'s catalog until now, and that is the+ * gap this row closes: the permission is declared in the manifest and asked+ * for on the first tap, so "denied once, months ago" left no trace anywhere+ * the user could find.+ */+ private fun microphone(context: Context): Finding {+ val p = Manifest.permission.RECORD_AUDIO+ val on = context.checkSelfPermission(p) == PackageManager.PERMISSION_GRANTED+ return Finding(+ id = "microphone",+ title = R.string.check_title_microphone,+ level = if (on) Level.OK else Level.FAIL,+ detail = if (on) R.string.check_detail_mic_on else R.string.check_detail_mic_off,+ fix = if (on) Fix.Nothing else Fix.Grant(listOf(p)),+ )+ }++ /**+ * Is there a 火山 key at all.+ *+ * Free, offline, and it sits *before* [voiceReach] because it answers the+ * cheaper half of the same question. Without a key ASR and TTS fail during+ * the WebSocket handshake, which surfaces as a connect error — the device+ * looks like it has a network problem when what it has is an empty field.+ * [VoiceConfig.API_KEY] ships empty in the public source (see its doc), so+ * this is the state every fresh checkout starts in, and it has to be said+ * out loud rather than discovered.+ *+ * WARN, not FAIL: the brain, the screen tools and the notebook all work+ * mute. What is dead is hearing and speaking, which is a degraded device+ * rather than a dead one.+ */+ private fun voiceKey(context: Context): Finding {+ val has = VoiceConfig.load(context).apiKey.isNotBlank()+ return Finding(+ id = "voice_key",+ title = R.string.check_title_voice_key,+ level = if (has) Level.OK else Level.WARN,+ detail = if (has) R.string.check_detail_voice_key_ok+ else R.string.check_detail_voice_key_missing,+ fix = if (has) Fix.Nothing else Fix.OurSettings,+ )+ }++ /**+ * Can we reach 火山 at all.+ *+ * A TCP connect, **not** a credential check: the ASR/TTS handshake carries+ * the key in headers, and reproducing it here would mean a second copy of+ * that protocol living next to `HuoshanAsr`. What this catches is the+ * failure this device actually had — `ASR ws connect timeout` — where the+ * network is up and the voice host still is not reachable from it. A green+ * row here beside a failing ASR therefore means the key, and says so by+ * elimination — a deduction that is only usable because [voiceKey] has+ * already ruled out the empty field.+ */+ private fun voiceReach(probe: Boolean, online: Boolean): Finding {+ if (!probe || !online) {+ return Finding(+ id = "voice_host",+ title = R.string.check_title_voice_host,+ level = Level.NOTE,+ detail = if (online) R.string.check_detail_checking+ else R.string.check_detail_skipped_offline,+ )+ }+ val err = tcpProbe(VOICE_HOST, VOICE_PORT)+ return Finding(+ id = "voice_host",+ title = R.string.check_title_voice_host,+ level = if (err == null) Level.OK else Level.WARN,+ detail = if (err == null) R.string.check_detail_voice_host_ok+ else R.string.check_detail_voice_host_bad,+ detailArgs = if (err == null) listOf(VOICE_HOST) else listOf(VOICE_HOST, err),+ )+ }++ /** A fact, not a fault — see the [Level] doc for why this is a NOTE. */+ private fun wakePhrase(context: Context): Finding {+ val on = net.kuafuai.andee.wake.WakeTemplates.enrolled(context)+ return Finding(+ id = "wake",+ title = R.string.check_title_wake,+ level = if (on) Level.OK else Level.NOTE,+ detail = if (on) R.string.check_detail_wake_on else R.string.check_detail_wake_off,+ fix = if (on) Fix.Nothing else Fix.OurSettings,+ )+ }++ /**+ * Battery optimisation.+ *+ * Pending todos are `AlarmManager` alarms, so doze delays them, and the+ * wake listener stops with the screen either way. Being on the list is not+ * a broken device — it is one whose reminders may arrive late, which the+ * user should be able to find out without reading `dumpsys deviceidle`.+ *+ * The read is live, not cached: [PowerManager.isIgnoringBatteryOptimizations]+ * answers from the current whitelist, so adding or removing the exemption+ * and tapping 重新检查 flips this row on the spot — verified both ways on a+ * HONOR with `cmd deviceidle whitelist +/-` against `am get-standby-bucket`+ * (10 ACTIVE ↔ 5 EXEMPTED). Nothing was wrong with the judgement; the door+ * beside it was the problem, which is what [batteryFix] is about.+ */+ private fun background(context: Context): Finding {+ val pm = context.getSystemService(Context.POWER_SERVICE) as? PowerManager+ val ignored = pm?.isIgnoringBatteryOptimizations(context.packageName) == true+ return Finding(+ id = "battery",+ title = R.string.check_title_battery,+ level = if (ignored) Level.OK else Level.WARN,+ detail = if (ignored) R.string.check_detail_battery_on+ else R.string.check_detail_battery_off,+ fix = if (ignored) Fix.Nothing else batteryFix(context),+ )+ }++ /**+ * The one-tap exemption, or the list page on a device that has no request+ * page.+ *+ * **Measured on a HONOR (MagicOS / Android 15): the list page is a dead end+ * for a third-party app, so the request page is the only door.** That page+ * lists ten preinstalled apps and nothing else — this package is absent even+ * while `dumpsys deviceidle whitelist` lists it, force-stopping+ * `com.android.settings` does not change it, and the page's own search box+ * answers 「没有匹配的结果」 for the package name. A row whose fix was that+ * page therefore had a 去开启 button that could not possibly turn its own row+ * green, which is the failure this method removes.+ *+ * The request page is one dialog that calls+ * `PowerWhitelistManager.addToWhitelist`, i.e. exactly the state this row+ * reads. On the HONOR it reads 「是否忽略电池优化?」/「取消」+「忽略」 —+ * note the affirmative button is 忽略, not 允许, which is why+ * `check_detail_battery_off` names that word rather than the AOSP one. It is+ * also silent about its own refusals —+ * `RequestIgnoreBatteryOptimizations.onCreate` finishes without a dialog when+ * this package does not declare `REQUEST_IGNORE_BATTERY_OPTIMIZATIONS`+ * (`DEBUG = false` there, so no log line either) — which is why the manifest+ * declaration is load-bearing; see AndroidManifest.+ *+ * Probed like [listenerSettings], and for the same reason: an action that+ * resolves nowhere must fall back rather than leave a tap that does nothing.+ */+ private fun batteryFix(context: Context): Fix {+ val pkg = context.packageName+ val request = Intent(+ Settings.ACTION_REQUEST_IGNORE_BATTERY_OPTIMIZATIONS,+ Uri.parse("package:$pkg"),+ )+ val reachable = runCatching {+ context.packageManager.queryIntentActivities(request, 0).isNotEmpty()+ }.getOrDefault(false)+ return if (reachable) {+ Fix.Screen(+ Settings.ACTION_REQUEST_IGNORE_BATTERY_OPTIMIZATIONS,+ data = "package:$pkg",+ )+ } else {+ Fix.Screen(Settings.ACTION_IGNORE_BATTERY_OPTIMIZATION_SETTINGS)+ }+ }++ /**+ * Room to write.+ *+ * Pages, screenshots, the grounding log and the notebook all land in+ * `filesDir`. When it fills, the failures are scattered and unrelated+ * looking — a page that will not open, a screenshot that saves as zero+ * bytes — so the number is worth having in one place.+ */+ private fun storage(context: Context): Finding {+ val free = runCatching { StatFs(context.filesDir.path).availableBytes }.getOrDefault(-1L)+ val mb = if (free < 0) -1L else free / (1024 * 1024)+ return Finding(+ id = "storage",+ title = R.string.check_title_disk,+ level = when {+ mb < 0 -> Level.NOTE+ mb < DISK_FAIL_MB -> Level.FAIL+ mb < DISK_WARN_MB -> Level.WARN+ else -> Level.OK+ },+ detail = when {+ mb < 0 -> R.string.check_detail_disk_unknown+ mb < DISK_FAIL_MB -> R.string.check_detail_disk_full+ mb < DISK_WARN_MB -> R.string.check_detail_disk_low+ else -> R.string.check_detail_disk_ok+ },+ detailArgs = if (mb < 0) emptyList() else listOf(mb),+ )+ }++ // ---- Plumbing ----++ fun isOnline(context: Context): Boolean = runCatching {+ val cm = context.getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager+ val nw = cm.activeNetwork ?: return@runCatching false+ cm.getNetworkCapabilities(nw)+ ?.hasCapability(NetworkCapabilities.NET_CAPABILITY_VALIDATED) == true+ }.getOrDefault(false)++ private sealed interface Reach {+ object Ok : Reach+ object Throttled : Reach+ data class Bad(val kind: Kind, val detail: String) : Reach+ }++ private enum class Kind { URL, KEY, MODEL, OTHER }++ /**+ * Ask the backend who it is, without paying for an answer.+ *+ * `GET /models` is the listing every gateway aimed at this format exposes,+ * and it decides all three cases at once: the key is exercised (401/403),+ * the URL is exercised (a connect failure), and if it answers, its list can+ * be compared against the configured name — so a typo comes back with *what+ * was on offer* rather than with a bare 400.+ *+ * The chat probe is the fallback for a gateway with no `/models`. It is the+ * expensive path, so it is bounded twice: `max_tokens = 1` and thinking+ * off. All three misconfigurations are decided by validation, before any+ * generation happens, so there is nothing to learn from letting it write.+ */+ private fun probeBackend(baseUrl: String, apiKey: String, model: String): Reach {+ val http = OkHttpClient.Builder()+ .connectTimeout(PROBE_CONNECT_S, TimeUnit.SECONDS)+ .readTimeout(PROBE_READ_S, TimeUnit.SECONDS)+ .build()+ val auth = "Bearer $apiKey"++ val listed: List<String>? = try {+ val req = Request.Builder()+ .url("$baseUrl/models")+ .addHeader("Authorization", auth)+ .get()+ .build()+ http.newCall(req).execute().use { resp ->+ when {+ resp.isSuccessful -> runCatching {+ val data = JSONObject(resp.body?.string().orEmpty()).optJSONArray("data")+ (0 until (data?.length() ?: 0)).mapNotNull { i ->+ data?.optJSONObject(i)?.optString("id")?.ifEmpty { null }+ }+ }.getOrDefault(emptyList())++ resp.code == 401 || resp.code == 403 -> return Reach.Bad(Kind.KEY, "HTTP ${resp.code}")++ resp.code == 429 -> return Reach.Throttled++ // No such endpoint on this gateway: not an error in the+ // user's config, just a gateway shaped differently. Fall+ // through to the chat probe.+ resp.code == 404 || resp.code == 405 -> null++ else -> return Reach.Bad(+ Kind.OTHER,+ "HTTP ${resp.code}: ${resp.body?.string().orEmpty().take(200)}",+ )+ }+ }+ } catch (e: Exception) {+ // IOException is the expected one (DNS, refused, timeout). Anything+ // else here is still "we could not talk to this URL", which is the+ // same sentence as far as the user is concerned.+ return Reach.Bad(Kind.URL, e.message ?: e.javaClass.simpleName)+ }++ if (listed != null) {+ // An empty list on a 200 is a gateway that answers but tells+ // nothing; treat it as "reachable" rather than inventing a model+ // complaint out of it.+ return if (listed.isEmpty() || listed.contains(model)) Reach.Ok+ else Reach.Bad(Kind.MODEL, listed.take(MODEL_SAMPLE).joinToString(", "))+ }++ return try {+ val body = JSONObject()+ .put("model", model)+ .put("stream", false)+ .put("max_tokens", 1)+ .put("thinking", JSONObject().put("type", "disabled"))+ .put(+ "messages",+ JSONArray().put(JSONObject().put("role", "user").put("content", "hi")),+ )+ val req = Request.Builder()+ .url("$baseUrl/chat/completions")+ .addHeader("Authorization", auth)+ .addHeader("Content-Type", "application/json")+ .post(body.toString().toRequestBody(JSON_MEDIA))+ .build()+ http.newCall(req).execute().use { resp ->+ when {+ resp.isSuccessful -> Reach.Ok+ resp.code == 401 || resp.code == 403 -> Reach.Bad(Kind.KEY, "HTTP ${resp.code}")+ resp.code == 429 -> Reach.Throttled+ // 400 is what a bad model name comes back as, and the body+ // is where the gateway says which part of it it disliked.+ else -> Reach.Bad(+ Kind.MODEL,+ "HTTP ${resp.code}: ${resp.body?.string().orEmpty().take(200)}",+ )+ }+ }+ } catch (e: IOException) {+ Reach.Bad(Kind.URL, e.message ?: e.javaClass.simpleName)+ }+ }++ /** @return null when the connection opened, else why it did not. */+ private fun tcpProbe(host: String, port: Int): String? = try {+ Socket().use { s -> s.connect(InetSocketAddress(host, port), PROBE_CONNECT_MS) }+ null+ } catch (e: Exception) {+ e.message ?: e.javaClass.simpleName+ }++ private val JSON_MEDIA = "application/json; charset=utf-8".toMediaType()++ private const val WRITE_SECURE_SETTINGS = "android.permission.WRITE_SECURE_SETTINGS"++ /**+ * Which app `ACTION_NOTIFICATION_LISTENER_DETAIL_SETTINGS` is about.+ *+ * Spelled out rather than taken from `Settings.EXTRA_NOTIFICATION_LISTENER_+ * COMPONENT_NAME` for the same reason [Fix.Adb]'s command is spelled out:+ * this string travels inside a [Fix] value and is read back by a different+ * process's Activity, so it is part of the payload and not a convenience+ * alias. The value is the platform's, verbatim, from `android.provider.+ * Settings` (API 30).+ */+ private const val EXTRA_LISTENER_COMPONENT =+ "android.provider.extra.NOTIFICATION_LISTENER_COMPONENT_NAME"++ /**+ * Where to get ADBKeyboard.+ *+ * The **v2.5-dev release asset**, not the `master/ADBKeyboard.apk` that+ * every blog post links to. That one is upstream's own "[Old] Release APK+ * download", and v2.5-dev exists for one stated reason: "fix keyboard+ * unusable on android-36" — the newest Android, which on this project is+ * the one a newly bought tablet arrives on. A pinned tag, so it does not+ * drift.+ *+ * Both files were read before choosing: each is ~18 KB, and each declares+ * `com.android.adbkeyboard.AdbIME`, broadcasts `ADB_INPUT_TEXT` /+ * `ADB_CLEAR_TEXT` / `ADB_EDITOR_CODE` and types through `commitText` — the+ * exact component and the exact actions `ScreenController` and+ * `ImeSwitch.BRAIN_IME` name. So either installs something this app can+ * drive, and the newer one is the one that keeps working.+ *+ * Not vendored into `assets/` and installed through a `FileProvider`, which+ * would make this one tap instead of three: redistributing someone else's+ * APK is a licensing decision, not a build detail, and the row as written+ * asks nothing of the user that the app cannot already explain.+ */+ private const val ADB_KEYBOARD_APK =+ "https://github.com/senzhk/ADBKeyBoard/releases/download/v2.5-dev/keyboardservice-debug.apk"++ private const val VOICE_HOST = "openspeech.bytedance.com"+ private const val VOICE_PORT = 443+ private const val PROBE_CONNECT_S = 6L+ private const val PROBE_CONNECT_MS = 6000+ private const val PROBE_READ_S = 20L+ private const val MODEL_SAMPLE = 8+ private const val DISK_WARN_MB = 500L+ private const val DISK_FAIL_MB = 100L+}diff --git a/app/src/main/java/net/kuafuai/andee/dog/DogLink.kt b/app/src/main/java/net/kuafuai/andee/dog/DogLink.ktnew file mode 100644index 0000000..c41502b--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/dog/DogLink.kt@@ -0,0 +1,610 @@+package net.kuafuai.andee.dog++import android.app.PendingIntent+import android.content.BroadcastReceiver+import android.content.Context+import android.content.Intent+import android.content.IntentFilter+import android.hardware.usb.UsbDevice+import android.hardware.usb.UsbDeviceConnection+import android.hardware.usb.UsbManager+import android.os.Build+import android.util.Log+import androidx.core.content.ContextCompat+import com.hoho.android.usbserial.driver.CdcAcmSerialDriver+import com.hoho.android.usbserial.driver.Ch34xSerialDriver+import com.hoho.android.usbserial.driver.UsbSerialDriver+import com.hoho.android.usbserial.driver.UsbSerialPort+import com.hoho.android.usbserial.driver.UsbSerialProber+import org.json.JSONObject++/**+ * USB-serial transport to the robot dog's remote dongle.+ *+ * The dongle is an Arduino Nano + NRF24L01 that plugs into the tablet's OTG+ * port. It speaks the simplest possible protocol, one ASCII character per+ * command, and answers with one line:+ *+ * in 'K' out "K OK" — the dog's radio layer acknowledged+ * in 'K' out "K NO-ACK" — nothing answered (out of range, dog+ * off, or the receive window was missed)+ *+ * The dog holds whatever it was last told to do until it is told something+ * else, so "J" (stand/stop) is not a nicety — it is how a move ends. That+ * asymmetry is why [DogMotion] exists and why nothing here ever leaves a+ * motion running: see that class for the scheduling half.+ *+ * ## Why the port is opened lazily+ *+ * Nothing connects at startup. The dongle may not be plugged in, the user may+ * be using the tablet for something else entirely, and a serial port held open+ * for no reason is a battery cost and one more thing to go wrong. The first+ * command opens it; every later command reuses it.+ *+ * ## Why opening takes seconds+ *+ * Asserting DTR on open resets the Nano — the classic Arduino auto-reset — so+ * the dongle reboots, re-initialises its radio and prints a banner. Commands+ * sent before that lands are read by nobody. So [ensureOpen] waits for the+ * banner (or [READY_WAIT_MS], whichever comes first) and drains the buffer.+ * The banner doubles as a diagnosis: `READY` means the dongle and its radio+ * are both up, `ERR: NRF24L01 not responding` means the dongle is alive but+ * its radio module is not — a wiring or 3.3V problem on the dongle, and+ * emphatically not "the dog is broken".+ *+ * ## Threading+ *+ * All port access is serialised on [lock]. Callers block: a single command+ * costs one write plus a bounded ACK read ([ACK_TIMEOUT_MS]), and the first+ * command of a session also pays the permission dialog and [READY_WAIT_MS].+ * That first-command cost is the only long one and it happens once.+ */+class DogLink(private val context: Context) {++ /** One command's outcome, as far as the radio layer can report it. */+ data class Ack(val cmd: Char, val ok: Boolean, val reply: String)++ private val lock = Any()+ private val usbManager: UsbManager? =+ context.getSystemService(Context.USB_SERVICE) as? UsbManager++ @Volatile private var port: UsbSerialPort? = null+ @Volatile private var connection: UsbDeviceConnection? = null+ @Volatile private var device: UsbDevice? = null++ /** Where the dongle was found, for status: "1a86:7523". */+ @Volatile private var deviceTag: String? = null++ /** The dongle's own banner text, or null if it never said anything. */+ @Volatile private var banner: String? = null++ @Volatile private var lastCmd: Char? = null+ @Volatile private var lastReply: String? = null++ /**+ * The last telemetry the dog sent back, and when.+ *+ * Kept as the raw last reading rather than smoothed or defaulted: an+ * ultrasonic sensor that sees nothing and a sensor that is unplugged both+ * need to read as "unknown", and a number invented to fill the gap would+ * reach the user as the brain saying how far away the wall is.+ */+ @Volatile private var lastDistanceCm: Int? = null+ @Volatile private var lastBatteryPct: Int? = null+ @Volatile private var lastTelemAt: Long = 0L++ /** Why the last attempt failed — surfaced by `dog_status`, not swallowed. */+ @Volatile private var lastError: String? = null++ private var receiver: BroadcastReceiver? = null+ private var permissionLatch: java.util.concurrent.CountDownLatch? = null++ /**+ * Called when the dongle goes away underneath us (unplugged, or the USB+ * stack dropped it). The motion scheduler uses this to cancel its timers —+ * it cannot send "J" down a cable that is no longer there, but it must at+ * least stop believing it will.+ */+ @Volatile var onLost: (() -> Unit)? = null++ // ---------------------------------------------------------------- opening++ /**+ * The dongle, if it is plugged in — as a driver we can actually use.+ *+ * Two passes on purpose. The library's own table covers the parts it has+ * heard of (CH340, CH341A, CP210x, FTDI, PL2303, and CDC-ACM devices it+ * lists). Arduino clones carry bridges it has *not* heard of — CH9102+ * (0x1a86:0x55d4) is absent from its `UsbId` table and it is exactly what+ * the newer Type-C Nano boards ship — so a second pass probes by+ * construction. That pass is deliberately narrow: a `Ch34xSerialDriver`+ * can be constructed around *any* device and will hand back a port, so+ * trusting it blindly would let us seize the tablet's own hub or a USB+ * stick. It only claims a device that is a QinHeng part presenting a single+ * vendor-specific (class 0xFF) interface, which is what a CH34x bridge+ * looks like and what almost nothing else does.+ */+ private fun findDongle(): Pair<UsbSerialDriver, UsbDevice>? {+ val manager = usbManager ?: return null++ val fromTable = UsbSerialProber.getDefaultProber().findAllDrivers(manager)+ // A known bridge vendor wins when the tablet exposes more than one+ // serial-shaped device. The Pad enumerates an internal device of its+ // own, and taking `firstOrNull()` blindly would eventually open that+ // one and fail with a confusing "this USB device has no serial port".+ fromTable.firstOrNull { it.device.vendorId in BRIDGE_VENDORS }?.let {+ return it to it.device+ }+ fromTable.firstOrNull()?.let { return it to it.device }++ for (d in manager.deviceList.values) {+ // Safe on anything: the library's own check looks for a real CDC+ // interface pair before it says yes.+ if (CdcAcmSerialDriver.probe(d)) return CdcAcmSerialDriver(d) to d+ // 0x1a86 = QinHeng. Unknown PID, so it is a bridge the table does+ // not carry; only take it if it is shaped like one.+ if (d.vendorId == 0x1A86 && d.interfaceCount == 1 &&+ d.getInterface(0).interfaceClass == 0xFF+ ) {+ return Ch34xSerialDriver(d) to d+ }+ }+ return null+ }++ /**+ * Ask the system for access to this USB device, blocking until the user+ * answers the dialog.+ *+ * The dialog is the user's, not ours: the first time anything touches a USB+ * device Android asks "允许 Andee 访问该 USB 设备吗?". We can only ask and+ * wait. It is bounded ([PERMISSION_WAIT_MS]) because an unanswered dialog+ * must not wedge the tool call forever — and the timeout is reported as a+ * timeout, with the thing the user has to do named in plain words, rather+ * than as a generic failure.+ */+ private fun ensurePermission(d: UsbDevice): Boolean {+ val manager = usbManager ?: run {+ lastError = "this device has no USB management service"+ return false+ }+ if (manager.hasPermission(d)) return true++ // The dialog's answer arrives as a broadcast, so the receiver has to be+ // listening before we ask. Registered once and kept: the same receiver+ // also carries attach/detach.+ ensureReceiver()+ val latch = java.util.concurrent.CountDownLatch(1)+ permissionLatch = latch+ val flags = PendingIntent.FLAG_UPDATE_CURRENT or+ (if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) PendingIntent.FLAG_MUTABLE else 0)+ // setPackage makes this an explicit-enough intent for a mutable+ // PendingIntent on Android 14 (targetSdk 34), which refuses implicit+ // ones outright.+ val ask = PendingIntent.getBroadcast(+ context, 0,+ Intent(ACTION_USB_PERMISSION).setPackage(context.packageName),+ flags,+ )+ Log.i(TAG, "asking the user to allow USB access to ${describe(d)}")+ manager.requestPermission(d, ask)+ val answered = runCatching { latch.await(PERMISSION_WAIT_MS, java.util.concurrent.TimeUnit.MILLISECONDS) }+ .getOrDefault(false)+ permissionLatch = null+ if (!manager.hasPermission(d)) {+ lastError = if (answered) {+ "the user refused USB permission, so the dog cannot be reached"+ } else {+ "the USB permission dialog went unanswered for ${PERMISSION_WAIT_MS / 1000} s — " ++ "have the user tap Allow for USB access on the tablet, then retry"+ }+ Log.w(TAG, "USB permission not granted: $lastError")+ return false+ }+ return true+ }++ private fun ensureReceiver() {+ if (receiver != null) return+ val r = object : BroadcastReceiver() {+ override fun onReceive(ctx: Context, intent: Intent) {+ when (intent.action) {+ ACTION_USB_PERMISSION ->+ permissionLatch?.countDown()++ // Unplugged mid-session. The dog keeps executing whatever it+ // was last told — see the class note and CLAUDE.md — so the+ // only honest thing to do is forget the link and say so.+ UsbManager.ACTION_USB_DEVICE_DETACHED -> {+ @Suppress("DEPRECATION")+ val gone = intent.getParcelableExtra<UsbDevice>(UsbManager.EXTRA_DEVICE)+ if (gone == null || gone.deviceId == device?.deviceId) {+ Log.w(TAG, "dongle detached — dropping the link")+ synchronized(lock) { turnOffLocked() }+ lastError = "the dongle was unplugged"+ // Outside the lock on purpose: onLost reaches back+ // into DogMotion, which takes its own lock. Calling it+ // while holding this one would be the other half of a+ // lock-order cycle.+ onLost?.invoke()+ }+ }+ }+ }+ }+ // NOT_EXPORTED: the permission answer comes from the system (allowed),+ // and nothing else has any business poking us.+ ContextCompat.registerReceiver(+ context, r, IntentFilter().apply {+ addAction(ACTION_USB_PERMISSION)+ addAction(UsbManager.ACTION_USB_DEVICE_DETACHED)+ },+ ContextCompat.RECEIVER_NOT_EXPORTED,+ )+ receiver = r+ }++ /**+ * Open the port if it is not already open. Returns false with [lastError]+ * set — never throws — because every caller is a tool result that has to+ * explain itself to the brain.+ */+ private fun ensureOpen(): Boolean {+ synchronized(lock) {+ if (port != null) return true+ val manager = usbManager ?: run {+ lastError = "this device has no USB management service"+ return false+ }+ val (driver, d) = findDongle() ?: run {+ lastError = "no dongle found — check that the Arduino Nano is plugged into " ++ "the tablet over the OTG cable"+ return false+ }+ if (!ensurePermission(d)) return false++ val conn = manager.openDevice(d) ?: run {+ lastError = "the system refused to open the USB device (the permission was " ++ "granted, but openDevice returned null)"+ return false+ }+ val p = driver.ports.firstOrNull() ?: run {+ conn.close()+ lastError = "this USB device has no serial port"+ return false+ }+ try {+ p.open(conn)+ p.setParameters(BAUD, UsbSerialPort.DATABITS_8, UsbSerialPort.STOPBITS_1, UsbSerialPort.PARITY_NONE)+ // Asserting DTR is what resets an Arduino; the library does it+ // inside open(), so the wait below is not optional.+ p.setDTR(true)+ p.setRTS(true)+ } catch (e: Exception) {+ runCatching { p.close() }+ runCatching { conn.close() }+ lastError = "failed to open the serial port: ${e.message}"+ Log.w(TAG, "open failed", e)+ return false+ }++ port = p+ connection = conn+ device = d+ deviceTag = describe(d)+ Log.i(TAG, "dongle open: $deviceTag @ $BAUD")+ banner = readBanner(p)+ lastError = null+ return true+ }+ }++ /**+ * Wait out the Nano's reset and collect whatever it says.+ *+ * Returns as soon as a known line lands, otherwise after [READY_WAIT_MS].+ * The remaining bytes are drained either way so the first command's ACK is+ * not read out of a buffer full of banner.+ */+ private fun readBanner(p: UsbSerialPort): String {+ val text = StringBuilder()+ val deadline = System.currentTimeMillis() + READY_WAIT_MS+ while (System.currentTimeMillis() < deadline) {+ val chunk = runCatching { readSome(p, 250) }.getOrElse { "" }+ if (chunk.isNotEmpty()) {+ text.append(chunk)+ val t = text.toString()+ if (t.contains("READY") || t.contains("ERR")) break+ }+ }+ // Keep only printable ASCII. The CH34x bridge emits one partial byte as+ // DTR toggles it into the reset, so the banner comes back as+ // "\uFFFDREADY" — noise, not signal, and not something to hand to the+ // brain as a device fact.+ val raw = text.toString()+ .filter { it == '\n' || it == '\r' || it in ' '..'~' }++ // The bridge also keeps bytes in its FIFO across a port close/reopen,+ // so the buffer can still hold the tail of the previous session ("J OK")+ // ahead of this boot's banner. Report the line that actually announces+ // the dongle, and fall back to the whole text only when there is none —+ // a diagnostic field that mixes in yesterday's traffic is a diagnostic+ // field nobody can read.+ val announcement = raw.split('\n')+ .map { it.trim() }+ .lastOrNull { it.contains("READY") || it.contains("ERR") }+ val banner = announcement ?: raw.trim()+ Log.i(TAG, "dongle banner: ${banner.ifEmpty { "(silent)" }}")+ if (banner.contains("ERR")) {+ lastError = "the dongle is online but its NRF24L01 module is not responding — " ++ "check the wiring and the 3.3V supply to the radio module on the dongle"+ }+ return banner+ }++ // ----------------------------------------------------------------- sending++ /**+ * Send one command character and wait, briefly, for the dongle's answer.+ *+ * [awaitAck] false is the emergency path: `J` has to leave the tablet as+ * soon as possible, and the acknowledgement tells us nothing we can act on+ * — the dog is either stopping or it is not. Skipping the read takes the+ * stop from ~500ms to ~1ms, and that difference matters when the thing+ * being stopped has motors.+ */+ fun send(cmd: Char, awaitAck: Boolean = true): Ack {+ if (!ensureOpen()) return Ack(cmd, false, lastError ?: "link not open")+ var lost = false+ val ack = synchronized(lock) {+ val p = port ?: return@synchronized Ack(cmd, false, "link not open")+ val bytes = byteArrayOf(cmd.code.toByte())+ fun attempt(): Boolean = try {+ p.write(bytes, WRITE_TIMEOUT_MS)+ true+ } catch (e: Exception) {+ Log.w(TAG, "write failed", e)+ turnOffLocked()+ lastError = "failed to write to the serial port: ${e.message}"+ lost = true+ false+ }++ if (!attempt()) {+ Ack(cmd, false, "write failed")+ } else {+ lastCmd = cmd+ if (!awaitAck) {+ Ack(cmd, true, "sent (no ack read)")+ } else {+ var reply = readAck(p, ACK_TIMEOUT_MS)+ var ok = verdict(reply)+ // One resend, same rule as the Mac-side dog_run.py: NO-ACK+ // usually means the dog's receive window was missed rather+ // than the dog being unreachable, and a second frame+ // frequently lands.+ if (!ok && reply.contains("NO-ACK")) {+ if (attempt()) reply = readAck(p, ACK_TIMEOUT_MS).ifEmpty { reply }+ ok = verdict(reply)+ }+ lastReply = reply.trim().ifEmpty { null }+ parseTelemetry(reply)+ lastError = if (ok) null else NO_ACK_HINT.format(cmd)+ Ack(cmd, ok, reply.trim())+ }+ }+ }+ // Outside [lock] — see the detach branch for why.+ if (lost) onLost?.invoke()+ return ack+ }++ private fun verdict(reply: String): Boolean =+ reply.contains("NO-ACK").not() && reply.contains("OK")++ /**+ * Pull `d=` / `b=` off the dongle's reply line, if it carried them.+ *+ * The dongle appends telemetry *after* the verdict ("K OK d=37 b=82") and+ * prints nothing at all when the dog sent none, so absence is the normal+ * case on a dongle or a dog that has not been reflashed. A field the dog+ * could not measure comes through as `?` and stays null — the readings are+ * deliberately not cleared on a silent reply, because one missed ack+ * payload is not news; [lastTelemAt] is what says how old the number is.+ */+ private fun parseTelemetry(reply: String) {+ val m = TELEM_RE.find(reply) ?: return+ lastDistanceCm = m.groupValues[1].toIntOrNull()+ lastBatteryPct = m.groupValues[2].toIntOrNull()+ lastTelemAt = System.currentTimeMillis()+ }++ /**+ * Ask the dog how it is, without telling it to do anything.+ *+ * '?' resends the dongle's current frame verbatim. The dog re-reads that+ * frame every main loop and only changes behaviour when the bytes change,+ * so this cannot interrupt a motion in progress — which is the whole point:+ * the distance reading is most wanted exactly while the dog is walking.+ */+ fun poll(): Ack = send('?')++ /**+ * The last thing the dog said about itself, with its age.+ *+ * `age_ms` is not decoration. The dog only speaks when spoken to (the+ * reading rides back on the ack frame of a command we sent), so a stale+ * distance is the normal state of a dog standing still, and a brain that+ * treats a five-second-old "30cm ahead" as current would drive into+ * something that moved. Null means never heard.+ */+ fun telemetry(): JSONObject = JSONObject()+ .put("distance_cm", lastDistanceCm ?: JSONObject.NULL)+ .put("battery_pct", lastBatteryPct ?: JSONObject.NULL)+ .put(+ "age_ms",+ if (lastTelemAt == 0L) JSONObject.NULL+ else System.currentTimeMillis() - lastTelemAt+ )+ .put("supported", lastTelemAt != 0L)++ /**+ * Read until the dongle's answer line is complete, or the budget runs out.+ *+ * Not a fixed sleep: the answer normally lands in a few milliseconds and a+ * fixed wait would tax every command for the worst case. A short read+ * timeout keeps it cheap early and still waits when the dog is slow to be+ * heard.+ *+ * Stopping at "OK" is not good enough anymore — telemetry is appended+ * *after* the verdict, so a read that returned on the verdict would parse+ * every reply as having none. It waits for the newline instead, and still+ * settles for a bare verdict if none arrives, so an old dongle (which+ * prints the same line minus the telemetry) costs one extra read, not a+ * timeout.+ */+ private fun readAck(p: UsbSerialPort, budgetMs: Long): String {+ val sb = StringBuilder()+ val end = System.currentTimeMillis() + budgetMs+ while (System.currentTimeMillis() < end) {+ val chunk = runCatching { readSome(p, 120) }.getOrElse { return sb.toString() }+ if (chunk.isEmpty()) continue+ sb.append(chunk)+ val t = sb.toString()+ if (t.contains("NO-ACK") || t.contains("OK")) {+ if (t.contains('\n')) break+ }+ }+ return sb.toString()+ }++ private fun readSome(p: UsbSerialPort, timeoutMs: Int): String {+ val buf = ByteArray(64)+ val n = p.read(buf, timeoutMs)+ return if (n > 0) String(buf, 0, n, Charsets.US_ASCII) else ""+ }++ // ----------------------------------------------------------------- lifecycle++ /**+ * Put the dog down and let go of the cable.+ *+ * Best-effort "J" first: if the dongle is still there it is the difference+ * between a dog standing still and a dog doing whatever it was last told+ * with nobody watching. Then the port closes, and every cached fact about+ * the link goes with it.+ */+ fun close() {+ synchronized(lock) {+ if (port != null) runCatching { sendLocked('J') }+ turnOffLocked()+ receiver?.let { runCatching { context.unregisterReceiver(it) } }+ receiver = null+ }+ Log.i(TAG, "dongle closed")+ }++ /** Best-effort "J", already holding [lock]. */+ private fun sendLocked(cmd: Char) {+ val p = port ?: return+ runCatching { p.write(byteArrayOf(cmd.code.toByte()), WRITE_TIMEOUT_MS) }+ lastCmd = cmd+ }++ /** Tear down port + connection, keeping [lastError] as the caller set it. */+ private fun turnOffLocked() {+ runCatching { port?.close() }+ runCatching { connection?.close() }+ port = null+ connection = null+ device = null+ deviceTag = null+ banner = null+ }++ // ------------------------------------------------------------------- status++ /**+ * Everything the tablet knows about the cable and the dongle.+ *+ * Distance and battery now come back from the dog itself, riding on the ack+ * frame of whatever we last sent (see [telemetry]) — but only when both the+ * dog and the dongle carry the firmware that does it. `telemetry.supported`+ * is false until a reading has actually arrived, because "this dongle was+ * never reflashed" and "the dog is out of range" look identical from here+ * and neither is a reason to make a number up.+ *+ * `attached` means "a dongle is plugged in", NOT "we have it open", and+ * `permission` says whether the system dialog has been answered yet. Both+ * are answered from a fresh enumeration when there is no live port, because+ * the question gets asked precisely when nothing is open: before the first+ * command, and right after something failed.+ */+ fun status(): JSONObject {+ val live = device+ val found: UsbDevice? = live ?: runCatching { findDongle()?.second }.getOrNull()+ val permission = when {+ found == null -> "no_device"+ usbManager?.hasPermission(found) == true -> "granted"+ else -> "needed"+ }+ return JSONObject()+ .put("attached", found != null)+ .put("device", deviceTag ?: found?.let { describe(it) } ?: JSONObject.NULL)+ .put("permission", permission)+ .put("open", port != null)+ .put("baud", BAUD)+ .put("banner", banner ?: JSONObject.NULL)+ .put("last_cmd", lastCmd?.toString() ?: JSONObject.NULL)+ .put("last_reply", lastReply ?: JSONObject.NULL)+ .put("telemetry", telemetry())+ .put("error", lastError ?: JSONObject.NULL)+ }++ private fun describe(d: UsbDevice): String =+ "%04x:%04x".format(d.vendorId, d.productId)++ private companion object {+ const val TAG = "Dog"+ const val BAUD = 115_200+ const val ACTION_USB_PERMISSION = "net.kuafuai.andee.dog.USB_PERMISSION"++ /** How long the Nano may take to reboot and say something. */+ const val READY_WAIT_MS = 3_000L++ /** How long one command's answer may take. */+ const val ACK_TIMEOUT_MS = 500L++ /** Writes are small and the driver already retries internally. */+ const val WRITE_TIMEOUT_MS = 200++ /** A dialog nobody answers must not wedge the tool call. */+ const val PERMISSION_WAIT_MS = 12_000L++ /** "K OK d=37 b=82" — either field may be `?` when the dog can't measure it. */+ val TELEM_RE = Regex("""d=(\d+|\?)\s+b=(\d+|\?)""")++ /**+ * What NO-ACK means, said the same way every time. It is not "the dog+ * is broken": the likeliest causes are mundane and the user can fix+ * them, so the message names them.+ */+ const val NO_ACK_HINT =+ "command %s got no acknowledgement from the machine (NO-ACK) — too far away, " ++ "the dog is switched off, or the original remote is transmitting at the " ++ "same time"++ /**+ * Vendors whose whole business is USB-serial bridges, i.e. what a+ * dongle is made of. Used only to break a tie when the tablet offers+ * more than one serial-shaped device.+ */+ val BRIDGE_VENDORS = setOf(+ 0x1A86, // QinHeng — CH340 / CH341 / CH9102, what Arduino clones use+ 0x0403, // FTDI+ 0x10C4, // Silicon Labs CP210x+ 0x067B, // Prolific+ 0x2341, // Arduino+ )+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/dog/DogMotion.kt b/app/src/main/java/net/kuafuai/andee/dog/DogMotion.ktnew file mode 100644index 0000000..34ed867--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/dog/DogMotion.kt@@ -0,0 +1,951 @@+package net.kuafuai.andee.dog++import android.os.SystemClock+import android.util.Log+import org.json.JSONArray+import org.json.JSONObject+import java.util.concurrent.Executors+import java.util.concurrent.ScheduledExecutorService+import java.util.concurrent.TimeUnit+import kotlin.math.abs++/**+ * Turns "move forward for two seconds" into a command, a timer, and a stop.+ *+ * [DogLink] is the cable; this is the part that knows what a *move* is. The+ * dog holds its last command until it is given another one, so every motion+ * here has an end: a single move schedules its own "J" for N seconds later, and+ * a sequence sends "J" after its last step. Nothing in this file can leave the+ * dog running.+ *+ * ## Why nothing here blocks+ *+ * This is the load-bearing decision in the whole feature. The brain's requests+ * arrive on `BodyWsClient`'s **single**-threaded worker, so a `dog_move` that+ * blocked for its own duration would hold that one thread for the whole move —+ * and a `dog_stop` the user asked for in the meantime would sit in the queue+ * behind it, unable to stop anything. The dog would then run for exactly as+ * long as the brain originally asked, no matter who shouted at it.+ *+ * So `move` returns the moment the command is out, and the stop is a timer's+ * job. The consequence is that two moves sent back to back do not queue — the+ * second supersedes the first — which is why multi-step motion needs+ * [sequence] rather than the brain calling `move` twice.+ *+ * ## Why a superseded timer cannot stop the wrong dog+ *+ * Timers and cancels race: the auto-stop for move #1 can be inside its "write+ * J" while move #2's "write K" is on its way out, and if the J lands second+ * the dog stops the instant it starts. Every scheduled stop therefore carries+ * the [generation] it was armed under, and the check plus the write happen+ * inside one [motionLock] critical section — so a stop either wins outright or+ * aborts, and never lands after the move that replaced it.+ *+ * ## The one thing here that is closed-loop+ *+ * [move] and [sequence] are open-loop by necessity: the 2.4G link only ever+ * sends, so the dog cannot report where it got to. [turn] is the exception,+ * because the tablet is bolted to the dog and therefore turns with it —+ * [DogSense] reads the tablet's own gyroscope and that is the dog's. So a turn+ * runs until the *measured* angle is reached rather than for a duration divided+ * out of a "8.7 s per revolution" constant that drifts with battery and floor.+ *+ * The same sensor pays for one reflex the brain is too far away to have: a+ * motion whose tilt crosses [FALL_TILT_DEG] is stopped here, on the tablet,+ * within [GUARD_MS] — a round trip to the brain and back is seconds, and a dog+ * that has gone over does not have seconds.+ */+class DogMotion(+ private val link: DogLink,+ private val sense: DogSense,+) {++ /** One leg of a sequence: a motion held for a duration. */+ data class Step(val cmd: Char, val label: String, val ms: Long)++ private val motionLock = Any()++ /**+ * Bumped by every new motion. A scheduled stop whose generation no longer+ * matches has been superseded and must do nothing — see the class note.+ */+ private var generation = 0L++ @Volatile private var currentLabel: String? = null+ @Volatile private var currentCmd: Char? = null+ @Volatile private var movingUntilMs = 0L++ /** Non-null while a sequence is walking its steps. */+ @Volatile private var seq: SeqRun? = null++ /** Non-null while a closed-loop [turn] is running. */+ @Volatile private var turnRun: TurnRun? = null++ /**+ * The last finished turn, kept because [turn] returns before the dog has+ * stopped moving — so the honest answer to "how far did it actually go"+ * does not exist yet when the tool replies. `dog_status` carries it.+ */+ @Volatile private var lastTurn: JSONObject? = null++ /** Set when a guard, not a command, ended the last motion. */+ @Volatile private var lastAbort: String? = null++ /**+ * Speed and stance as last *acknowledged*, not as last asked for.+ *+ * The authoritative copy lives in the dongle's frame; this is only what to+ * tell the brain. Null means never set this session — which is not the same+ * as knowing the dog is at its firmware default, and reads that way in+ * `dog_status` rather than guessing.+ */+ @Volatile private var currentSpeed: String? = null+ @Volatile private var currentPose: String? = null++ private class SeqRun(+ val gen: Long,+ val labels: List<String>,+ val totalMs: Long,+ @Volatile var index: Int = 0,+ )++ private class TurnRun(+ val gen: Long,+ val targetDeg: Double,+ val dir: String,+ val label: String,+ val timeoutMs: Long,+ @Volatile var turned: Double = 0.0,+ )++ private val scheduler: ScheduledExecutorService =+ Executors.newSingleThreadScheduledExecutor { r ->+ Thread(r, "DogMotion-timer").apply { isDaemon = true }+ }+ private val seqExec = Executors.newSingleThreadExecutor { r ->+ Thread(r, "DogMotion-seq").apply { isDaemon = true }+ }++ init {+ // The cable can vanish mid-move. We cannot send "J" down a cable that+ // is no longer there, so this only forgets what we thought was+ // happening — and deliberately does *not* touch [generation]. Bumping+ // it here would invalidate the stop timer of a move whose link came+ // back, leaving that move with nothing to end it. The scheduled stop+ // stays armed and no-ops on its own instead (see [autoStop]), because a+ // lost link already cleared the state it tests.+ link.onLost = {+ synchronized(motionLock) {+ currentLabel = null+ currentCmd = null+ movingUntilMs = 0+ seq = null+ }+ }+ }++ // ------------------------------------------------------------------ moves++ /**+ * Start one motion and return immediately; it stops itself after [seconds].+ *+ * Returns the radio layer's verdict for the *command* (`ack`: did the dog+ * acknowledge the frame) alongside what the tablet will now do. Those are+ * different facts and both matter: a NO-ACK with a scheduled stop means the+ * tablet believes it is driving a dog that never heard it.+ */+ fun move(motion: String, seconds: Double, gait: Boolean): JSONObject {+ val (cmd, label) = resolve(motion, gait)+ ?: throw IllegalArgumentException(+ "unknown motion '$motion' — use forward, back, left or right"+ )+ val ms = validate(seconds)+ val where = if (gait) "legged" else "wheeled"+ beginSensing()++ synchronized(motionLock) {+ generation+++ val gen = generation+ seq = null+ turnRun = null+ lastAbort = null+ // The dog is standing right now — whatever ran before it ended with+ // a "J" — so this is the moment a tilt reference is true.+ if (currentLabel == null) sense.markUpright()+ val ack = link.send(cmd)+ // Ask the link how it actually is, rather than assuming the write+ // worked: a cable yanked out between open() and write() surfaces+ // here, and reporting a dog that is "moving" on a dead link is the+ // one lie this feature cannot afford.+ val open = link.status().optBoolean("open")+ if (!open) {+ currentCmd = null+ currentLabel = null+ movingUntilMs = 0+ return JSONObject()+ .put("sent", false)+ .put("motion", "$where $label")+ .put("ack", if (ack.ok) "OK" else "NO-ACK")+ .put("error", link.status().optString("error", "USB link is down"))+ .put("note", "The command did not get through and the dog will not move — " ++ "check the dongle is plugged in with its light on, then retry")+ }+ currentCmd = cmd+ currentLabel = "$where $label"+ movingUntilMs = SystemClock.elapsedRealtime() + ms+ scheduler.schedule(Runnable { autoStop(gen) }, ms, TimeUnit.MILLISECONDS)+ armFallGuard(gen)+ return JSONObject()+ .put("sent", true)+ .put("motion", "$where $label")+ .put("seconds", seconds)+ .put("ack", if (ack.ok) "OK" else "NO-ACK")+ .put("stops_by_itself", true)+ .put("note", "Moving; it stops itself after ${seconds} s — call dog_stop to " ++ "stop it sooner")+ }+ }++ /**+ * Run several legs in order, on its own thread, and return at once.+ *+ * This is the tool for anything with more than one beat — "转一圈再往前走"+ * — because [move] supersedes rather than queues. It is also the only way to+ * run a script longer than the hub's request timeout, since nobody waits for+ * it here. The last step is followed by "J" no matter how the run ends,+ * including on error, which is the rule the Mac-side `dog_run.py` already+ * established.+ *+ * Progress is not reported anywhere except [status]; the brain polls if it+ * cares, and the user can see the dog.+ */+ fun sequence(rawSteps: JSONArray): JSONObject {+ require(rawSteps.length() > 0) { "dog_sequence needs at least one step" }+ val steps = ArrayList<Step>(rawSteps.length())+ var totalMs = 0L+ for (i in 0 until rawSteps.length()) {+ val o = rawSteps.optJSONObject(i)+ ?: throw IllegalArgumentException("step ${i + 1} is not an object")+ val motion = o.optString("motion").lowercase()+ val gait = o.optBoolean("gait", false)+ val ms = validate(o.optDouble("seconds", Double.NaN))+ // "pause" is a beat with the dog standing — useful between+ // direction changes, where the servos need to settle before the+ // next leg, and cheaper to say than a move of zero seconds.+ val step = if (motion == "pause") {+ Step('J', "pause", ms)+ } else {+ val (cmd, label) = resolve(motion, gait)+ ?: throw IllegalArgumentException(+ "step ${i + 1}: unknown motion '$motion' — " ++ "use forward, back, left, right or pause"+ )+ Step(cmd, (if (gait) "legged " else "wheeled ") + label, ms)+ }+ steps.add(step)+ totalMs += ms+ }+ if (totalMs > MAX_SEQUENCE_MS) {+ throw IllegalArgumentException(+ "The whole sequence is ${totalMs / 1000}s, too long (ceiling " ++ "${MAX_SEQUENCE_MS / 1000}s) — split it into several calls, so it can " ++ "be stopped in between"+ )+ }+ beginSensing()++ synchronized(motionLock) {+ generation+++ val gen = generation+ val run = SeqRun(gen, steps.map { it.label }, totalMs)+ seq = run+ turnRun = null+ lastAbort = null+ if (currentLabel == null) sense.markUpright()+ currentLabel = steps.first().label+ currentCmd = steps.first().cmd+ movingUntilMs = SystemClock.elapsedRealtime() + totalMs+ seqExec.execute { runSequence(run, steps) }+ armFallGuard(gen)+ return JSONObject()+ .put("started", true)+ .put("steps", steps.size)+ .put("total_seconds", totalMs / 1000.0)+ .put("plan", JSONArray(steps.map { it.label }))+ .put("note", "The sequence runs in the background and winds itself up to a " ++ "stop after ${totalMs / 1000.0} s; call dog_stop to stop it sooner")+ }+ }++ private fun runSequence(run: SeqRun, steps: List<Step>) {+ try {+ for ((i, step) in steps.withIndex()) {+ // Cancelled or superseded between steps — leave without sending+ // anything. Whoever superseded us owns the dog now.+ if (!stillCurrent(run.gen)) return+ run.index = i+ synchronized(motionLock) {+ if (run.gen != generation) return+ link.send(step.cmd, awaitAck = true)+ }+ Log.i(TAG, "sequence step ${i + 1}/${steps.size}: ${step.label} for ${step.ms}ms")+ // Sliced sleep: `dog_stop` bumps the generation and the next+ // slice notices, so an interrupted sequence releases the dog in+ // ~100ms instead of finishing its step first.+ var slept = 0L+ while (slept < step.ms) {+ if (!stillCurrent(run.gen)) return+ val slice = minOf(SLICE_MS, step.ms - slept)+ Thread.sleep(slice)+ slept += slice+ }+ }+ } catch (e: InterruptedException) {+ Thread.currentThread().interrupt()+ } catch (e: Exception) {+ Log.w(TAG, "sequence failed", e)+ } finally {+ synchronized(motionLock) {+ if (run.gen != generation) {+ // Superseded: the thing that replaced us sent its own+ // command and owns the stop. Writing "J" here would cancel+ // it.+ return@synchronized+ }+ link.send('J', awaitAck = false)+ currentLabel = null+ currentCmd = null+ movingUntilMs = 0+ seq = null+ }+ Log.i(TAG, "sequence done, dog stopped")+ }+ }++ /** Does this motion still own the dog? */+ private fun stillCurrent(gen: Long): Boolean = synchronized(motionLock) { gen == generation }++ // ------------------------------------------------------------- closed loop++ /**+ * Turn a measured number of degrees and stop when the gyroscope says so.+ *+ * This is the only motion here that knows whether it worked. The tablet is+ * bolted to the dog, so [DogSense] reading the tablet's rotation *is* the+ * dog's rotation; the loop drives until the angle is reached rather than for+ * a duration computed from a revolution constant that changes with battery+ * level, floor and stride phase.+ *+ * Returns immediately, like everything else in this class — the loop runs on+ * [seqExec] and the reply says how long to expect. The measured result lands+ * in `dog_status.last_turn` a moment after the dog has actually stopped,+ * which is the earliest it is true.+ *+ * Failure is reported, not hidden: a turn that times out having moved four+ * degrees says so, and that is the clearest evidence this system can produce+ * that a command never reached the dog.+ */+ fun turn(motion: String, degrees: Double, gait: Boolean): JSONObject {+ val dir = motion.lowercase()+ if (dir != "left" && dir != "right") {+ throw IllegalArgumentException(+ "dog_turn only accepts left or right (got '$motion') — for forward/back " ++ "use dog_move"+ )+ }+ require(!degrees.isNaN()) { "degrees is required" }+ require(degrees > 0) { "degrees must be positive, got $degrees" }+ require(degrees <= MAX_TURN_DEG) {+ "at most ${MAX_TURN_DEG.toInt()}° in one turn (got $degrees) — split it into " ++ "several turns and you can look at the dog in between"+ }+ if (!sense.available) {+ throw IllegalStateException(+ "A closed-loop turn needs a rotation sensor: ${sense.unavailableReason()}. " ++ "Use dog_move and time it in seconds instead — a full circle is about " ++ "12 s legged and 8.7 s wheeled (wheeled spins slip on carpet; prefer legged)."+ )+ }+ beginSensing()++ val (cmd, name) = resolve(dir, gait)!!+ val where = if (gait) "legged" else "wheeled"+ val label = "$where $name $degrees°"+ // Generous: the point of the ceiling is to end a turn that is not+ // happening, not to cut short a slow one. Scaled off the open-loop+ // estimate so a 20° nudge does not sit here for ten seconds.+ val estimateMs = (degrees / 360.0 * (if (gait) GAIT_360_MS else WHEEL_360_MS)).toLong()+ val timeoutMs = (estimateMs * TIMEOUT_SLACK).toLong()+ .coerceIn(TIMEOUT_FLOOR_MS, (MAX_STEP_S * 1000).toLong())++ synchronized(motionLock) {+ generation+++ val gen = generation+ seq = null+ lastAbort = null+ if (currentLabel == null) sense.markUpright()+ // Zero the counter and get the first command out here, not on the+ // loop thread, for the same reason [move] does: the cable's state is+ // only knowable once something has been written to it. A turn that+ // reported "started" and then spent its whole timeout measuring zero+ // would end by blaming the dog for an unplugged dongle — and the+ // brain would send the user to look at a machine that is fine.+ sense.markTurn()+ val ack = link.send(cmd)+ val open = link.status().optBoolean("open")+ if (!open) {+ currentCmd = null+ currentLabel = null+ movingUntilMs = 0+ turnRun = null+ return JSONObject()+ .put("started", false)+ .put("motion", label)+ .put("ack", if (ack.ok) "OK" else "NO-ACK")+ .put("error", link.status().optString("error", "USB link is down"))+ .put("note", "The command did not get through and the dog will not move — " ++ "check the dongle is plugged in with its light on, then retry")+ }+ val run = TurnRun(gen, degrees, dir, label, timeoutMs)+ turnRun = run+ currentLabel = label+ currentCmd = cmd+ movingUntilMs = SystemClock.elapsedRealtime() + timeoutMs+ seqExec.execute { runTurn(run) }+ return JSONObject()+ .put("started", true)+ .put("motion", label)+ .put("target_deg", degrees)+ .put("ack", if (ack.ok) "OK" else "NO-ACK")+ .put("expected_seconds", round1(estimateMs / 1000.0))+ .put("note", "Turning; it stops itself at ${degrees}° — that angle is " ++ "measured by the gyroscope, not timed off a stopwatch. Read " ++ "dog_status.last_turn in about ${round1(estimateMs / 1000.0)} s for the " ++ "angle actually reached; call dog_stop to stop it sooner")+ }+ }++ /**+ * The turn's polling loop. The first command is already out (see [turn]);+ * this only watches the gyroscope and decides when to write "J".+ *+ * Runs on [seqExec] for the same reason [runSequence] does, and shares it+ * deliberately: only one motion can own the dog at a time, so the two can+ * never need to run together, and a second executor would only add a thread+ * that is idle whenever this one is not.+ */+ private fun runTurn(run: TurnRun) {+ var stoppedBy = "reached"+ try {+ val deadline = SystemClock.elapsedRealtime() + run.timeoutMs+ while (true) {+ if (!stillCurrent(run.gen)) return // superseded; the new owner drives+ val turned = abs(sense.turnedDeg())+ run.turned = turned+ // Lead the target by however far the current rate carries the+ // dog during the stop latency. This is a control-loop estimate,+ // not a claim about the machine — `overshoot_deg` in the result+ // is the measurement, and it is what to tune STOP_LEAD_MS by.+ val lead = abs(sense.turnRateDegPerSec()) * STOP_LEAD_MS / 1000.0+ if (turned + lead >= run.targetDeg) break+ if (sense.tiltDeg() >= FALL_TILT_DEG) { stoppedBy = "fallen"; break }+ if (SystemClock.elapsedRealtime() > deadline) { stoppedBy = "timeout"; break }+ Thread.sleep(POLL_MS)+ }+ } catch (e: InterruptedException) {+ Thread.currentThread().interrupt()+ stoppedBy = "interrupted"+ } catch (e: Exception) {+ Log.w(TAG, "turn failed", e)+ stoppedBy = "error"+ } finally {+ val atStop = abs(sense.turnedDeg())+ val stillOpen = link.status().optBoolean("open")+ val owned = synchronized(motionLock) {+ if (run.gen != generation) return@synchronized false+ link.send('J', awaitAck = false)+ currentLabel = null+ currentCmd = null+ movingUntilMs = 0+ turnRun = null+ if (stoppedBy == "fallen") lastAbort = "fallen"+ true+ }+ if (owned) {+ // The dog is still rotating as this runs: servos settle and+ // wheels coast. Waiting it out is the only way the number we+ // report is the angle the dog actually ended up at.+ runCatching { Thread.sleep(COAST_MS) }+ val settled = abs(sense.turnedDeg())+ val secs = round1(run.timeoutMs / 1000.0)+ lastTurn = JSONObject()+ .put("motion", run.label)+ .put("target_deg", run.targetDeg)+ .put("turned_deg", round1(settled))+ .put("overshoot_deg", round1(settled - run.targetDeg))+ .put("at_stop_deg", round1(atStop))+ .put("stopped_by", stoppedBy)+ .put("reached", stoppedBy == "reached")+ .put(+ "note", when {+ stoppedBy == "reached" -> "Reached the target; measured ${round1(settled)}°"+ // The cable going away mid-turn is its own diagnosis+ // and must not be reported as the dog misbehaving.+ stoppedBy == "timeout" && !stillOpen ->+ "The USB link dropped mid-turn and only ${round1(settled)}° was " ++ "measured — the dongle came out, it is not the dog's fault; " ++ "have the user check the OTG cable"+ stoppedBy == "timeout" -> "Timed out, having measured only " ++ "${round1(settled)}° in $secs s (target ${run.targetDeg}°) — the " ++ "dog may be out of range, switched off, or stuck; take a look first"+ stoppedBy == "fallen" -> "The body tilted past ${FALL_TILT_DEG.toInt()}° " ++ "so it was stopped in a hurry — the dog has probably fallen over " ++ "or been picked up; tell the user to go and look"+ else -> "The turn ended abnormally ($stoppedBy); measured ${round1(settled)}°"+ }+ )+ Log.i(TAG, "turn done: ${run.label} → ${round1(settled)}° ($stoppedBy)")+ }+ }+ }++ /**+ * Stop the dog if it goes over, without asking anyone.+ *+ * The brain is at the other end of a WebSocket and a language model; the+ * round trip is seconds even when nothing is wrong. A dog that has tipped+ * while being told to drive is the one case where the tablet both *can* see+ * the problem and cannot afford to relay it, so the reflex lives here.+ *+ * Re-arms itself rather than running on a fixed schedule, so it stops the+ * moment the motion it was armed for stops owning the dog — there is never+ * a guard outliving its own motion.+ */+ private fun armFallGuard(gen: Long) {+ if (!sense.available) return+ scheduler.schedule(object : Runnable {+ override fun run() {+ var tripped = false+ synchronized(motionLock) {+ if (gen != generation || currentLabel == null) return+ if (sense.tiltDeg() >= FALL_TILT_DEG) {+ link.send('J', awaitAck = false)+ // Supersede, so the move's own auto-stop finds a newer+ // generation and does not fire into whatever comes next.+ generation+++ currentLabel = null+ currentCmd = null+ movingUntilMs = 0+ seq = null+ turnRun = null+ lastAbort = "fallen"+ tripped = true+ }+ }+ if (tripped) {+ Log.w(TAG, "fall guard: tilt over $FALL_TILT_DEG°, dog stopped")+ return+ }+ scheduler.schedule(this, GUARD_MS, TimeUnit.MILLISECONDS)+ }+ }, GUARD_MS, TimeUnit.MILLISECONDS)+ }++ /** Sensors are registered on first use, never at service start. */+ private fun beginSensing() {+ runCatching { sense.attach() }+ .onFailure { Log.w(TAG, "sense attach failed", it) }+ }++ private fun autoStop(gen: Long) {+ synchronized(motionLock) {+ if (gen != generation) {+ // Superseded by a newer motion, which armed its own stop.+ Log.i(TAG, "auto-stop for generation $gen skipped (superseded)")+ return+ }+ if (currentLabel == null && currentCmd == null) {+ // The link was lost and the state was already cleared, so there+ // is nothing running to stop. Sending "J" here would do nothing+ // but make ensureOpen() go looking for a USB device that is not+ // there.+ Log.i(TAG, "auto-stop with nothing running — nothing to send")+ return+ }+ link.send('J', awaitAck = false)+ currentLabel = null+ currentCmd = null+ movingUntilMs = 0+ seq = null+ }+ Log.i(TAG, "auto-stop: dog stopped")+ }++ // ------------------------------------------------------------------- stop++ /**+ * Stop now, whoever asked.+ *+ * Always sends "J", even when nothing looks like it is running — this is+ * the brake, and a brake that declines to press itself because the tablet's+ * bookkeeping says the dog is idle is not a brake. The dog could have been+ * started by the original remote, or left running by an earlier session;+ * "J" is the one command that is safe to send blind (it means *stand*).+ * Which also makes this the cheapest way to prove the whole link end to end+ * without moving anything.+ *+ * The reply is then reported honestly: `was_moving` says whether we knew of+ * a motion, and `ack` says whether the machine confirmed the stop. A note+ * that claims the dog is standing has to be backed by an OK.+ */+ fun stop(): JSONObject {+ val wasMoving: String?+ val ack: DogLink.Ack+ synchronized(motionLock) {+ wasMoving = currentLabel+ generation+++ currentLabel = null+ currentCmd = null+ movingUntilMs = 0+ seq = null+ turnRun = null+ ack = link.send('J', awaitAck = true)+ // The dongle's 'J' clears the stance byte too (a raised stance would+ // otherwise keep the wheel motors running through a "stop"), so the+ // stance we report has to fall back with it.+ if (ack.ok) currentPose = "normal"+ }+ val note = when {+ !ack.ok -> ack.reply.ifEmpty {+ link.status().optString("error", "the stop command did not get through")+ }+ wasMoving == null ->+ "Nothing was running in the first place; the stop command went out and the " ++ "machine acknowledged it, so the dog is now standing"+ else -> "Stop command sent, acknowledged by the machine"+ }+ return JSONObject()+ .put("stopped", ack.ok)+ .put("was_moving", wasMoving ?: JSONObject.NULL)+ .put("ack", if (ack.ok) "OK" else "NO-ACK")+ .put("note", note)+ }++ // ------------------------------------------------------------- settings++ /**+ * Gait speed. A setting, not a motion — so it deliberately does not touch+ * [generation], arm a stop, or cancel what the dog is doing.+ *+ * The dongle holds the speed byte across frames precisely so this can be+ * changed mid-walk: on the dog side speed is four independent `if`s that+ * sit *beside* the direction chain, so a frame carrying only a speed byte+ * would fall through to the standing branch and stop the dog. That memory+ * lives in the dongle; from here it is one character.+ *+ * It takes effect on the next frame the dongle sends, which for a dog+ * already walking is the next command — not immediately. Say so rather+ * than implying the dog speeds up as the tool returns.+ */+ fun speed(level: String): JSONObject {+ val (ch, label) = when (level.lowercase()) {+ "slow" -> '1' to "slow"+ "normal" -> '2' to "normal"+ "fast" -> '3' to "fast"+ "turbo" -> '4' to "turbo"+ else -> throw IllegalArgumentException(+ "speed must be slow / normal / fast / turbo (got $level)"+ )+ }+ val ack = link.send(ch)+ currentSpeed = if (ack.ok) label else currentSpeed+ return JSONObject()+ .put("speed", label)+ .put("applied", ack.ok)+ .put("ack", if (ack.ok) "OK" else "NO-ACK")+ .put(+ "note",+ if (ack.ok) "Takes effect from the next motion command; a motion already " ++ "under way waits for its current step to end"+ else link.status().optString("error", "no reply received")+ )+ }++ /**+ * Stance: `high` raises the body (the firmware's `Buf[4]==0x05` branch),+ * `normal` puts it back down.+ *+ * Also a setting, and also held by the dongle — but with one asymmetry+ * worth knowing: a stop (`J`) clears it, because the raised-stance branch+ * on the dog sits *ahead* of the wheel branches and only poses the legs+ * without cutting the wheel motors. A "stop" that left the stance set+ * would look stopped and not be.+ */+ fun pose(stance: String): JSONObject {+ val (ch, label) = when (stance.lowercase()) {+ "high" -> 'P' to "raised"+ "normal" -> 'p' to "normal"+ else -> throw IllegalArgumentException(+ "stance must be high / normal (got $stance)"+ )+ }+ val ack = link.send(ch)+ if (ack.ok) currentPose = label+ return JSONObject()+ .put("pose", label)+ .put("applied", ack.ok)+ .put("ack", if (ack.ok) "OK" else "NO-ACK")+ .put("note", "A stop (dog_stop) resets the stance — the raised stance does not " ++ "cut the wheel motors")+ }++ /**+ * Take the tablet's current attitude as "standing upright".+ *+ * Every motion already re-takes this when it starts from rest, so this+ * exists for the case that rule cannot cover: the tablet was re-seated,+ * re-angled or re-strapped between motions. The old reference then makes a+ * standing dog read as leaning, and past [FALL_TILT_DEG] the fall guard+ * kills motions that were never in trouble.+ *+ * Refuses while something is running. The reference has to be taken with+ * the dog genuinely still — a walking dog is pitching a few degrees with+ * every stride, and baking that into the reference is worse than the stale+ * one it replaces.+ */+ fun calibrate(): JSONObject {+ beginSensing()+ if (!sense.available) {+ return JSONObject()+ .put("calibrated", false)+ .put("why", sense.unavailableReason())+ }+ val running = synchronized(motionLock) { currentLabel }+ if (running != null) {+ return JSONObject()+ .put("calibrated", false)+ .put("moving", running)+ .put("why", "The dog is running \"$running\", and calibrating while it moves " ++ "gives a wrong reference — dog_stop first, wait for it to stand still, " ++ "then calibrate")+ }+ // Sensors are registered lazily, so the first call here can arrive+ // before any sample has. Nothing to do but wait a beat for one.+ val deadline = SystemClock.elapsedRealtime() + CALIBRATE_WAIT_MS+ while (!sense.fresh() && SystemClock.elapsedRealtime() < deadline) {+ try {+ Thread.sleep(CALIBRATE_POLL_MS)+ } catch (e: InterruptedException) {+ Thread.currentThread().interrupt()+ break+ }+ }+ if (!sense.fresh()) {+ return JSONObject()+ .put("calibrated", false)+ .put("why", "The sensor has not produced a reading yet — try again; if it " ++ "keeps doing this, the tablet's rotation sensor is faulty")+ }+ val before = sense.tiltDeg()+ sense.markUpright()+ return JSONObject()+ .put("calibrated", true)+ .put("was_tilt_deg", if (before.isNaN()) JSONObject.NULL else round1(before))+ .put("vibration", round1(sense.vibration()))+ .put(+ "note",+ "The current attitude is now recorded as \"standing\". From here tilt_deg " ++ "is measured from it, and so is the fall guard — which is why this is " ++ "only done when the dog is genuinely standing still"+ )+ }++ /**+ * Ask the dog for a fresh distance/battery reading without commanding it.+ *+ * Bounded and quick (one frame, one ack), so it is the one `dog.*` call+ * that may be made while a motion runs — which is when the distance+ * actually matters.+ */+ fun poll(): JSONObject {+ link.poll()+ return JSONObject()+ .put("telemetry", link.telemetry())+ .put("moving", currentLabel != null)+ .put("motion", currentLabel ?: JSONObject.NULL)+ }++ // ----------------------------------------------------------------- status++ /** Link facts plus motion facts, flattened so the brain reads one object. */+ fun status(): JSONObject {+ val out = link.status()+ val label = currentLabel+ val run = seq+ val turning = turnRun+ out.put("moving", label != null)+ out.put("motion", label ?: JSONObject.NULL)+ out.put(+ "remaining_ms",+ if (label != null) (movingUntilMs - SystemClock.elapsedRealtime()).coerceAtLeast(0)+ else JSONObject.NULL+ )+ out.put(+ "sequence",+ if (run == null) JSONObject.NULL+ else JSONObject()+ .put("running", true)+ .put("step", run.index + 1)+ .put("of", run.labels.size)+ .put("plan", JSONArray(run.labels))+ )+ out.put(+ "turn",+ if (turning == null) JSONObject.NULL+ else JSONObject()+ .put("running", true)+ .put("target_deg", turning.targetDeg)+ .put("turned_deg", round1(turning.turned))+ )+ // Only ever the *last finished* turn: while one is running the honest+ // answer is in `turn` above, and reporting a stale result beside a live+ // motion is how a brain ends up narrating the wrong number.+ out.put("last_turn", if (turning == null) (lastTurn ?: JSONObject.NULL) else JSONObject.NULL)+ out.put("posture", sense.status())+ out.put("speed", currentSpeed ?: JSONObject.NULL)+ out.put("pose", currentPose ?: JSONObject.NULL)+ out.put("aborted_by", lastAbort ?: JSONObject.NULL)+ return out+ }++ /** Service teardown: cancel every timer, put the dog down, release threads. */+ fun shutdown() {+ synchronized(motionLock) {+ generation+++ runCatching { link.send('J', awaitAck = false) }+ currentLabel = null+ currentCmd = null+ movingUntilMs = 0+ seq = null+ turnRun = null+ }+ scheduler.shutdownNow()+ seqExec.shutdownNow()+ runCatching { sense.shutdown() }+ }++ // -------------------------------------------------------------- vocabulary++ /**+ * Motion name (and whether the legs are used) → the dongle's character.+ *+ * The wheel and gait characters are different sets, and the gait *turns*+ * are the trap: the dog's own firmware comments label `E` as 左转 and `D`+ * as 右转, and measured behaviour is the opposite (E turns right, D turns+ * left). The mapping below follows the measurement — the same correction+ * the dog's `NRF24_Dongle/README.md` and its `/dog` skill both carry.+ */+ private fun resolve(motion: String, gait: Boolean): Pair<Char, String>? =+ when (motion.lowercase()) {+ "forward" -> (if (gait) 'B' else 'K') to "forward"+ "back" -> (if (gait) 'C' else 'L') to "back"+ "left" -> (if (gait) 'D' else 'M') to "left"+ "right" -> (if (gait) 'E' else 'N') to "right"+ else -> null+ }++ /**+ * One leg's duration, in range. Rejected rather than clamped, for the same+ * reason the coordinate tools reject out-of-range values: a silently+ * shortened move is a dog that did not go where the brain thinks it went.+ */+ private fun validate(seconds: Double): Long {+ require(!seconds.isNaN()) { "seconds is required" }+ require(seconds > 0) { "seconds must be positive, got $seconds" }+ require(seconds <= MAX_STEP_S) {+ "One step is at most ${MAX_STEP_S.toInt()} s (got $seconds) — split it into " ++ "several steps, or string them together with dog_sequence"+ }+ return (seconds * 1000).toLong()+ }++ private fun round1(v: Double) = Math.round(v * 10) / 10.0++ private companion object {+ const val TAG = "Dog"++ /** Matches the Mac-side `dog_run.py` per-step ceiling. */+ const val MAX_STEP_S = 30.0++ /**+ * Whole-sequence ceiling. Longer than one step because a sequence is+ * exactly what you use when you want to string things together, but+ * still bounded: a runaway dog nobody is watching is the failure mode+ * this whole class exists to prevent.+ */+ const val MAX_SEQUENCE_MS = 120_000L++ /** How often a running sequence checks whether it was cancelled. */+ const val SLICE_MS = 100L++ // ------------------------------------------------------------- turning++ /** Two revolutions. Past that, look at the dog between attempts. */+ const val MAX_TURN_DEG = 720.0++ /**+ * Open-loop revolution times, measured 2026-09-04 and kept here only to+ * size a turn's timeout and tell the brain how long to wait. Nothing+ * depends on them being right any more — that is the point of [turn].+ */+ const val WHEEL_360_MS = 8_700.0+ const val GAIT_360_MS = 12_000.0++ /** A turn gets this much more than the open-loop estimate before it is+ * declared not to be happening. */+ const val TIMEOUT_SLACK = 2.5++ /** …but never less than this, so small turns still get a fair chance. */+ const val TIMEOUT_FLOOR_MS = 4_000L++ /** How often the turn loop reads the gyroscope. */+ const val POLL_MS = 40L++ /**+ * Assumed delay between writing "J" and the dog actually ceasing to+ * rotate; the loop stops this much of a rotation early. An estimate of+ * *our* latency, not a claim about the machine — `overshoot_deg` in+ * `last_turn` is the measurement to tune it against.+ */+ const val STOP_LEAD_MS = 150.0++ /** How long to let the dog coast before reporting the angle it reached. */+ const val COAST_MS = 600L++ // -------------------------------------------------------------- guards++ /**+ * Tilt from standing, in degrees, past which a motion is cut off. A dog+ * on four legs does not lean this far doing anything normal, so the+ * false-positive cost is near zero, while the miss cost is a machine+ * driving its legs against the floor on its side.+ */+ const val FALL_TILT_DEG = 50.0++ /** How often the fall guard looks, while a motion is running. */+ const val GUARD_MS = 200L++ // ----------------------------------------------------------- calibrate++ /**+ * How long [calibrate] waits for a first sensor sample. Sensors are+ * registered lazily, so a calibration issued as the session's first+ * dog command has nothing to reference for a few tens of ms. Short+ * enough that a tablet with a dead sensor answers quickly instead of+ * reading as a hung tool call.+ */+ const val CALIBRATE_WAIT_MS = 800L++ const val CALIBRATE_POLL_MS = 30L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/dog/DogSense.kt b/app/src/main/java/net/kuafuai/andee/dog/DogSense.ktnew file mode 100644index 0000000..75b3d78--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/dog/DogSense.kt@@ -0,0 +1,388 @@+package net.kuafuai.andee.dog++import android.content.Context+import android.hardware.Sensor+import android.hardware.SensorEvent+import android.hardware.SensorEventListener+import android.hardware.SensorManager+import android.os.Handler+import android.os.HandlerThread+import android.os.SystemClock+import android.util.Log+import org.json.JSONObject+import kotlin.math.abs+import kotlin.math.acos+import kotlin.math.atan2+import kotlin.math.hypot+import kotlin.math.sqrt++/**+ * The tablet's own IMU, read as if it were the dog's.+ *+ * The tablet is bolted to the dog, so every rotation of the dog is a rotation+ * of this device — which makes the accelerometer and gyroscope already in the+ * tablet the only sensors in this system whose readings reach the brain. The+ * dog's own ultrasonic and battery ADC are computed in its firmware and thrown+ * away (it only ever *receives* over the 2.4G link), so until that link learns+ * to talk back, this file is the whole of the dog's proprioception.+ *+ * What it answers, and why each one is worth the wire:+ *+ * - **How far have we actually turned.** [turnedDeg] is what makes a closed-loop+ * `dog_turn` possible. Open-loop turning divides a measured "8.7s per+ * revolution" by the angle asked for, and that constant drifts with battery+ * level, floor surface and which way the servos happened to be mid-stride.+ * - **Are we still upright.** [tiltDeg] against a reference taken while the dog+ * was standing. A dog on its side that is still being told to drive forward is+ * the one failure the tablet can see and the brain cannot.+ * - **Is anything moving at all.** [vibration] is the residual accelerometer+ * energy. It does not prove the dog is going anywhere, but a dead-flat reading+ * while we believe a motion is running is real evidence of a command that+ * never landed — or of the firmware's own obstacle refusal, which is otherwise+ * completely invisible from here.+ *+ * ## Why GAME_ROTATION_VECTOR and not ROTATION_VECTOR+ *+ * [Sensor.TYPE_ROTATION_VECTOR] fuses the magnetometer in, and the magnetometer+ * is useless here: the tablet is sitting inches above four servos and a motor+ * driver. [Sensor.TYPE_GAME_ROTATION_VECTOR] is gyro+accelerometer only. It has+ * no idea where north is, which costs us nothing — every question this class is+ * asked is *relative* ("how much further to turn"), measured over the ten+ * seconds of one motion, which is far inside the gyro's drift.+ *+ * ## Why the yaw is not read out of getOrientation()+ *+ * [SensorManager.getOrientation]'s azimuth goes singular as the device+ * approaches vertical — and a tablet mounted upright on a dog's back sits at+ * exactly that pitch. So yaw is instead taken as the compass bearing of one of+ * the device's own axes projected onto the world's horizontal plane, and the+ * axis is chosen at [markTurn] as whichever is furthest from vertical. That is+ * both gimbal-free and mounting-agnostic: however the tablet is strapped on,+ * turning the dog about the floor's normal turns that axis by the same angle.+ * Picking the axis once per turn (rather than per sample) means a dog that+ * tips mid-turn cannot make the reading jump.+ *+ * ## Threading+ *+ * Samples are delivered to a private [HandlerThread], never the main looper:+ * the UI thread in this app is already carrying an OpenGL ball and a gesture+ * pipeline, and a turn loop that stops sampling because a screenshot is being+ * encoded would overshoot. Readers are plain volatile field reads and can be+ * called from any thread, which is what [DogMotion]'s motion threads do.+ */+class DogSense(context: Context) {++ private val sensors = context.getSystemService(Context.SENSOR_SERVICE) as? SensorManager++ private val rotationSensor: Sensor? =+ sensors?.getDefaultSensor(Sensor.TYPE_GAME_ROTATION_VECTOR)+ ?: sensors?.getDefaultSensor(Sensor.TYPE_ROTATION_VECTOR)++ private val accelSensor: Sensor? = sensors?.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)++ /** Which sensor we ended up on, for [status] — the two behave differently. */+ private val rotationName: String = when (rotationSensor?.type) {+ Sensor.TYPE_GAME_ROTATION_VECTOR -> "game_rotation_vector"+ Sensor.TYPE_ROTATION_VECTOR -> "rotation_vector"+ else -> "none"+ }++ /** True when this tablet can answer anything at all about its attitude. */+ val available: Boolean get() = rotationSensor != null++ /**+ * Why not, in words meant for the brain to pass on. Empty when [available].+ */+ fun unavailableReason(): String = when {+ sensors == null -> "this device has no sensor service"+ rotationSensor == null -> "this tablet has no gyroscope / rotation-vector sensor"+ else -> ""+ }++ private var thread: HandlerThread? = null+ private var handler: Handler? = null+ private var listening = false+ private val lock = Any()++ // ------------------------------------------------------------------ samples++ /** Device→world rotation, row-major 3x3. Null until the first sample. */+ @Volatile private var rot: FloatArray? = null++ @Volatile private var lastSampleMs = 0L++ /** World-up expressed in device coordinates, captured while standing. */+ @Volatile private var uprightRef: FloatArray? = null++ /** Slow mean of |accel|, the baseline [vibration] is measured against. */+ @Volatile private var accelMean = 0f++ /** Smoothed |·| of the high-passed accelerometer magnitude, in m/s². */+ @Volatile private var vibrationEma = 0f++ // --------------------------------------------------------------- turn state++ /** Which device axis (0=X, 1=Y) the current turn is being measured on. */+ @Volatile private var turnAxis = 0++ /** Bearing of [turnAxis] at the last sample, radians, wrapped. */+ @Volatile private var lastBearing = Double.NaN++ /** Unwrapped degrees since [markTurn]. Positive = counter-clockwise = left. */+ @Volatile private var turnedDeg = 0.0++ /** Degrees per second, signed the same way, smoothed. */+ @Volatile private var turnRate = 0.0++ @Volatile private var marked = false++ // ---------------------------------------------------------------- lifecycle++ /**+ * Start sampling. Idempotent, and cheap enough to leave running: gyro and+ * accelerometer at [SAMPLE_US] are handled by the sensor hub, which is+ * nothing against the GL ball this service already draws continuously.+ * Called lazily from the first dog command rather than at service start, so+ * a tablet that never touches the dog never registers anything.+ */+ fun attach() {+ val sm = sensors ?: return+ synchronized(lock) {+ if (listening) return+ val t = HandlerThread("DogSense").apply { isDaemon = true; start() }+ val h = Handler(t.looper)+ thread = t+ handler = h+ rotationSensor?.let { sm.registerListener(listener, it, SAMPLE_US, h) }+ accelSensor?.let { sm.registerListener(listener, it, SAMPLE_US, h) }+ listening = true+ }+ Log.i(TAG, "sense attached: $rotationName" + if (accelSensor == null) " (no accelerometer)" else "")+ }++ fun shutdown() {+ synchronized(lock) {+ if (!listening) return+ sensors?.unregisterListener(listener)+ thread?.quitSafely()+ thread = null+ handler = null+ listening = false+ }+ Log.i(TAG, "sense detached")+ }++ // ------------------------------------------------------------------ marking++ /**+ * Record the current attitude as "standing upright".+ *+ * There is no way to know how the tablet is strapped to the dog, so tilt has+ * to be measured against a reference rather than against the device's own+ * axes. [DogMotion] calls this at the start of a motion when nothing else+ * was running — at that instant the dog is standing by construction, because+ * every motion in this codebase ends with a "J".+ *+ * A reference taken while the dog is already lying on its side would make+ * [tiltDeg] read zero forever, so this deliberately does nothing when there+ * is no sample yet: a wrong reference is worse than none.+ */+ fun markUpright() {+ val r = rot ?: return+ uprightRef = floatArrayOf(r[6], r[7], r[8])+ }++ /**+ * Zero the turn counter and choose the axis it will be measured on.+ *+ * The axis is whichever of the device's X and Y is furthest from vertical,+ * because a near-vertical axis has no meaningful compass bearing. It is+ * fixed for the whole turn on purpose — see the class note.+ */+ fun markTurn() {+ val r = rot ?: return+ // Column j of a row-major device→world matrix is the world direction of+ // device axis j; its z component is how vertical that axis is.+ val xVertical = abs(r[6])+ val yVertical = abs(r[7])+ turnAxis = if (xVertical <= yVertical) 0 else 1+ lastBearing = bearingOf(r, turnAxis)+ turnedDeg = 0.0+ turnRate = 0.0+ marked = true+ }++ // ------------------------------------------------------------------ readers++ /**+ * Degrees turned since [markTurn], unwrapped — a full circle reads 360, not+ * 0. Positive is counter-clockwise seen from above, i.e. a left turn.+ */+ fun turnedDeg(): Double = if (marked) turnedDeg else 0.0++ /** Signed turn rate in degrees per second, smoothed over a few samples. */+ fun turnRateDegPerSec(): Double = if (marked) turnRate else 0.0++ /**+ * Angle between the current attitude and the [markUpright] reference, in+ * degrees. NaN when there is no reference yet — every caller compares with+ * `>=`, and NaN comparisons are false, so an unknown tilt never trips a+ * guard.+ *+ * Pure yaw does not move this: the reference is the world-up vector in+ * device coordinates, and rotating about world-up leaves it untouched. So+ * this measures exactly the part of the attitude that turning is not.+ */+ fun tiltDeg(): Double {+ val ref = uprightRef ?: return Double.NaN+ val r = rot ?: return Double.NaN+ val dot = (r[6] * ref[0] + r[7] * ref[1] + r[8] * ref[2]).toDouble()+ return Math.toDegrees(acos(dot.coerceIn(-1.0, 1.0)))+ }++ /** Residual accelerometer energy in m/s², gravity removed. */+ fun vibration(): Double = vibrationEma.toDouble()++ /** Has any sample arrived recently enough to trust? */+ fun fresh(): Boolean =+ rot != null && SystemClock.elapsedRealtime() - lastSampleMs < STALE_MS++ /**+ * What the tablet can say about the dog's posture, for `dog_status`.+ *+ * [vibration] is reported as a raw number and never as a "stuck" boolean.+ * The threshold that would separate a stuck dog from a smooth one has never+ * been measured on this hardware, and a wrong boolean would have the brain+ * tell the user something false about a machine they are looking at.+ */+ fun status(): JSONObject {+ val out = JSONObject()+ out.put("sensor", rotationName)+ if (!available) {+ out.put("available", false)+ out.put("why", unavailableReason())+ return out+ }+ out.put("available", true)+ out.put("fresh", fresh())+ val tilt = tiltDeg()+ out.put("tilt_deg", if (tilt.isNaN()) JSONObject.NULL else round1(tilt))+ out.put("upright_ref", uprightRef != null)+ out.put("vibration", round2(vibration()))+ return out+ }++ // ------------------------------------------------------------------ interna++ private val listener = object : SensorEventListener {+ override fun onAccuracyChanged(sensor: Sensor?, accuracy: Int) = Unit++ override fun onSensorChanged(e: SensorEvent) {+ when (e.sensor.type) {+ Sensor.TYPE_GAME_ROTATION_VECTOR, Sensor.TYPE_ROTATION_VECTOR -> onRotation(e)+ Sensor.TYPE_ACCELEROMETER -> onAccel(e)+ }+ }+ }++ private val matrix = FloatArray(9)++ /** Scratch for the truncated rotation vector — see [onRotation]. */+ private val quat = FloatArray(4)++ private fun onRotation(e: SensorEvent) {+ // Some OEM sensor stacks (Samsung's, historically) deliver five values+ // on a rotation vector, and getRotationMatrixFromVector throws on+ // anything longer than four. Truncating is the standard guard; a crash+ // here would take the accessibility service down with it, which on this+ // device means the ball, the overlay and every tool the brain has.+ val n = minOf(e.values.size, 4)+ if (n < 3) return+ System.arraycopy(e.values, 0, quat, 0, n)+ val vec = if (n == 4) quat else quat.copyOf(3)+ SensorManager.getRotationMatrixFromVector(matrix, vec)+ val snapshot = matrix.copyOf()+ val now = SystemClock.elapsedRealtime()+ val prevMs = lastSampleMs+ rot = snapshot+ lastSampleMs = now+ if (uprightRef == null) {+ // First ever sample: assume the dog is standing. It is, unless+ // somebody powered the tablet on with the dog already tipped over,+ // and markUpright() re-takes this before every motion anyway.+ uprightRef = floatArrayOf(snapshot[6], snapshot[7], snapshot[8])+ }+ if (!marked) return++ val bearing = bearingOf(snapshot, turnAxis)+ val prev = lastBearing+ if (prev.isNaN() || bearing.isNaN()) {+ lastBearing = bearing+ return+ }+ // Unwrap: take the short way round, so crossing ±180° accumulates+ // instead of flipping sign, and 360° of turning reads as 360.+ var delta = bearing - prev+ while (delta > Math.PI) delta -= 2 * Math.PI+ while (delta < -Math.PI) delta += 2 * Math.PI+ val deltaDeg = Math.toDegrees(delta)+ turnedDeg += deltaDeg+ lastBearing = bearing++ val dtMs = (now - prevMs).coerceAtLeast(1L)+ val instantRate = deltaDeg * 1000.0 / dtMs+ // Smoothed, because the stop lead is computed from it and a single+ // noisy sample would have us cut a turn short by several degrees.+ turnRate = turnRate * (1 - RATE_ALPHA) + instantRate * RATE_ALPHA+ }++ private fun onAccel(e: SensorEvent) {+ val mag = sqrt(+ e.values[0] * e.values[0] ++ e.values[1] * e.values[1] ++ e.values[2] * e.values[2]+ )+ if (accelMean == 0f) accelMean = mag+ // Slow mean tracks gravity and any steady tilt; what is left over is+ // the shaking of a machine that is actually doing something.+ accelMean = accelMean * (1 - MEAN_ALPHA) + mag * MEAN_ALPHA+ val residual = abs(mag - accelMean)+ vibrationEma = vibrationEma * (1 - VIB_ALPHA) + residual * VIB_ALPHA+ }++ /**+ * Compass bearing of device axis [axis] projected onto the world's+ * horizontal plane, in radians. NaN when that axis is too close to vertical+ * for the projection to mean anything.+ */+ private fun bearingOf(r: FloatArray, axis: Int): Double {+ val wx = r[axis].toDouble()+ val wy = r[3 + axis].toDouble()+ if (hypot(wx, wy) < MIN_HORIZONTAL) return Double.NaN+ return atan2(wy, wx)+ }++ private fun round1(v: Double) = Math.round(v * 10) / 10.0+ private fun round2(v: Double) = Math.round(v * 100) / 100.0++ private companion object {+ const val TAG = "Dog"++ /** ~20 ms. At a 9 s revolution that is under a degree per sample. */+ const val SAMPLE_US = 20_000++ /** Older than this and a reading is not evidence of anything. */+ const val STALE_MS = 1_000L++ const val RATE_ALPHA = 0.3+ const val MEAN_ALPHA = 0.05f+ const val VIB_ALPHA = 0.2f++ /**+ * Below this, the chosen axis is so close to vertical that its bearing+ * is noise. [markTurn] picks the better of two axes precisely so this+ * should not trigger; it is the backstop for a dog that tips mid-turn.+ */+ const val MIN_HORIZONTAL = 0.20+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/i18n/AppLocale.kt b/app/src/main/java/net/kuafuai/andee/i18n/AppLocale.ktnew file mode 100644index 0000000..152384e--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/i18n/AppLocale.kt@@ -0,0 +1,87 @@+package net.kuafuai.andee.i18n++import android.content.Context+import android.content.res.Configuration+import net.kuafuai.andee.config.VoiceConfig+import java.util.Locale++/**+ * Turns "which language did the user pick" into "which Context the UI should be+ * built from". The one door between [VoiceConfig.uiLanguage] and `getString`.+ *+ * **Why this exists at all.** Android ships per-app language+ * (`AppCompatDelegate.setApplicationLocales`) and this app cannot use it as the+ * whole answer, for two independent reasons:+ *+ * * **Almost no UI here lives in an Activity.** The ball, the card, the+ * subtitle band, the signboards and the settings sheet are all+ * `WindowManager`-added overlays owned by [net.kuafuai.andee.ScreenBodyService],+ * which is a `Service`. `setApplicationLocales` rebuilds Activities; it does+ * nothing for a window whose Context belonged to a Service, and nothing at+ * all for views already attached. So the overlays would keep the old language+ * until something happened to rebuild them.+ * * **The language is a setting, not the system locale.** [VoiceConfig.uiLanguage]+ * lets the user pick 中文 on an English tablet, and once they pick, that+ * choice outranks the system forever. A device whose locale says `en` but+ * whose preference says `zh` is correct and expected here.+ *+ * So language travels with the Context instead: [wrap] hands back a Context+ * whose locale is the *preference*, and every UI class takes its strings from+ * that. Nothing reads `getString` off a raw Service Context.+ *+ * **The trap this is shaped around.** Getting this wrong does not crash.+ * `getString` on an unwrapped Context silently returns the *system* language —+ * so a Chinese tablet shows a Chinese 取消 button while everything built from+ * the wrapped Context is English, and it reads as "that one button is hardcoded"+ * rather than "that call site missed the wrap". Hence: one entry point, called+ * where the UI object is constructed, and nowhere else.+ *+ * **What [wrap]'s result may and may not be used for.** Strings and inflating+ * views — yes. System services and window management — no. Keep+ * `getSystemService(WINDOW_SERVICE)` and `addView` on the original Context: a+ * configuration Context is a thin wrapper over the base, and handing it to the+ * window manager is asking for a layer-mismatch bug in exchange for nothing.+ * [net.kuafuai.andee.ui.SettingsUi] is the shape to copy — it holds the raw+ * `context` for its window and a wrapped `ctx` for its text.+ *+ * Deliberately not cached. `createConfigurationContext` is cheap, a UI object+ * is built once per open, and a cache keyed on language would hold a Context —+ * and through it a Resources — alive for the process lifetime to save+ * microseconds nobody is waiting on.+ */+object AppLocale {++ /**+ * The locale the UI should be in, from the saved preference.+ *+ * [VoiceConfig.uiLanguage] returns exactly the two tags this app ships+ * (`zh` / `en`) and falls through to the device locale when nothing is+ * saved, so there is no third case to handle here.+ */+ fun localeOf(context: Context): Locale =+ Locale.forLanguageTag(VoiceConfig.uiLanguage(context))++ /**+ * [context] with its locale overridden to the user's choice.+ *+ * Returns [context] unchanged when it is already right, so the common+ * Chinese-device-Chinese-preference case costs nothing.+ */+ fun wrap(context: Context): Context {+ val locale = localeOf(context)+ val base = context.resources.configuration+ // locales[0] rather than the deprecated `locale` field: on API 24+ the+ // primary locale lives in the list, and minSdk here is 30.+ if (base.locales[0] == locale) return context+ val config = Configuration(base)+ config.setLocale(locale)+ return context.createConfigurationContext(config)+ }++ /** One string, in the user's language. For call sites with no Context to keep. */+ fun str(context: Context, id: Int): String = wrap(context).getString(id)++ /** [str] with `%s` / `%d` arguments substituted. */+ fun str(context: Context, id: Int, vararg args: Any): String =+ wrap(context).getString(id, *args)+}diff --git a/app/src/main/java/net/kuafuai/andee/meeting/MeetingRecorder.kt b/app/src/main/java/net/kuafuai/andee/meeting/MeetingRecorder.ktnew file mode 100644index 0000000..c838804--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/meeting/MeetingRecorder.kt@@ -0,0 +1,543 @@+package net.kuafuai.andee.meeting++import android.content.Context+import android.os.Handler+import android.os.Looper+import android.os.SystemClock+import android.util.Log+import net.kuafuai.andee.R+import net.kuafuai.andee.asr.HuoshanAsr+import net.kuafuai.andee.audio.AudioFocusGate+import net.kuafuai.andee.audio.AudioIO+import net.kuafuai.andee.config.VoiceConfig+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.ui.FloatingWindowUi+import org.json.JSONObject+import java.io.File+import java.text.SimpleDateFormat+import java.util.Locale+import java.util.TreeMap+import java.util.concurrent.Executors+import java.util.concurrent.atomic.AtomicBoolean+import java.util.concurrent.atomic.AtomicInteger++/**+ * Hold the microphone for a whole meeting and hand back a verbatim transcript.+ *+ * The device does not summarise anything — it has no model. It records, it+ * transcribes, and the brain turns the transcript into minutes and puts them on+ * screen with `show_html`. That split is why this class has no notion of a+ * "summary" anywhere in it.+ *+ * **Why segments.** [HuoshanAsr] reports a final transcript exactly once, when+ * the server answers our `isLast`. One session for one meeting would therefore+ * produce nothing at all until the meeting ended, and any dropped connection in+ * between would take the whole hour with it. So the recogniser is rotated: a+ * segment lives ~2 minutes, its final text is appended to disk the moment it+ * arrives, and a disconnect costs one segment instead of the meeting.+ *+ * **Why the new session opens before the old one is retired.** Retiring costs+ * up to [FINAL_WAIT_MS] of blocking (火山 sends the final ~500 ms after our+ * `isLast` — see `AsrController.stopImpl`), and connecting costs a second or+ * two. Doing either with no live session would punch a hole in the recording.+ * Opening first means audio always has somewhere to go, and it makes a failed+ * rotation free: if the replacement cannot connect, the old segment is still+ * receiving and the meeting carries on.+ */+class MeetingRecorder(+ private val context: Context,+ private val audio: AudioIO,+ private val window: FloatingWindowUi,+ /**+ * Unfold the card to full screen. Routed through the dispatcher rather+ * than straight at [FloatingWindowUi.setCompact] because the dispatcher+ * caches whether it has already folded, and a window unfolded behind its+ * back would stop it folding for the next `screen.*` call.+ */+ private val expand: () -> Unit,+ /** Fired for every end except the brain's own `stop_meeting` — see [endAndNotify]. */+ private val onEnded: (JSONObject) -> Unit,+) {++ /** Session open/close only. Never touched by the audio thread. */+ private val exec = Executors.newSingleThreadExecutor { r ->+ Thread(r, "MeetingRecorder").apply { isDaemon = true }+ }++ /** Transcript appends, off both the WebSocket IO threads and [exec]. */+ private val io = Executors.newSingleThreadExecutor { r ->+ Thread(r, "MeetingWriter").apply { isDaemon = true }+ }++ private val ui = Handler(Looper.getMainLooper())++ /**+ * Same exclusive floor a voice turn takes, and for the same reason: the+ * speaker is an inch from the microphone, and a media app that merely ducks+ * is still playing into the recording — for an hour, this time.+ */+ private val focus = AudioFocusGate(+ context,+ android.media.AudioAttributes.USAGE_ASSISTANT,+ android.media.AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE,+ )++ private val recording = AtomicBoolean(false)++ /** Read once per audio chunk; swapped by [rotate]. */+ @Volatile+ private var active: Segment? = null++ @Volatile+ private var rotatePending = false++ private val nextIndex = AtomicInteger(0)++ private var title = ""+ private var startedAt = 0L++ @Volatile+ private var file: File? = null++ /** Guarded by itself. */+ private val transcript = StringBuilder()++ /** Out-of-order finals waiting for their predecessors. Guarded by itself. */+ private val finals = TreeMap<Int, String>()+ private var nextToFlush = 0++ /** The live segment's partial — the un-committed tail of [renderLive]. */+ @Volatile+ private var livePartial = ""++ fun isActive(): Boolean = recording.get()++ // ---- Start / stop ---------------------------------------------------++ /**+ * Open the microphone and the first recogniser session, then return.+ *+ * Deliberately synchronous even though the tool contract says do not block:+ * the handshake is normally under a second, and the alternative is a tool+ * that reports success and then records an hour of nothing because the+ * recogniser never connected. A meeting silently not being recorded is the+ * one failure here with no symptom until it is far too late to fix.+ */+ fun start(name: String?): JSONObject {+ if (recording.get()) {+ throw IllegalStateException("a meeting is already being recorded — stop_meeting first")+ }+ if (audio.isStreaming()) {+ throw IllegalStateException("the microphone is busy with a voice turn; try again in a moment")+ }+ // The fallback is a resource, not a literal: this title ends up as the+ // page header the minutes are laid out under, so it has to follow the+ // interface language. `ScreenBodyService.meetingMinutesQuery` reads the+ // same key back — see the note on it in `values/strings.xml`.+ title = name?.trim().orEmpty()+ .ifEmpty { AppLocale.str(context, R.string.meeting_default_title) }+ startedAt = System.currentTimeMillis()+ nextIndex.set(0)+ nextToFlush = 0+ synchronized(finals) { finals.clear() }+ synchronized(transcript) { transcript.setLength(0) }+ file = openTranscriptFile()++ // A phone call takes the floor and the microphone with it. Ending the+ // meeting keeps what was said so far instead of recording a call the+ // user did not ask us to record.+ //+ // Reported as its own reason, not as "user": the brain reads+ // `stopped_by` out loud, and a meeting killed by a focus loss used to+ // have it tell the user they had touched the ball.+ focus.acquire { endAndNotify(BY_FOCUS) }++ val first = try {+ openSegment()+ } catch (t: Throwable) {+ focus.release()+ throw IllegalStateException("could not start the recogniser: ${t.message}", t)+ }+ active = first+ recording.set(true)++ try {+ audio.startStreaming(sampleRate = 16_000, chunkMs = 100) { chunk -> onPcm(chunk) }+ } catch (t: Throwable) {+ recording.set(false)+ active = null+ runCatching { first.asr.close() }+ focus.release()+ throw t+ }++ ui.post {+ // Full screen for the length of the meeting. The transcript+ // scrolling past is the only evidence the recogniser is alive, and+ // a thumb-sized ball cannot carry it — the tablet is face-up on the+ // table with nothing else to do, so it may as well show the work.+ expand()+ window.setState(FloatingWindowUi.State.RECORDING)+ ui.removeCallbacks(tick)+ tick.run()+ }+ Log.i(TAG, "meeting started: $title → ${file?.name}")+ return JSONObject()+ .put("recording", true)+ .put("title", title)+ .put("started_at", startedAt)+ .put("file", fileRef())+ }++ /** Synchronous stop, for the dispatcher. Blocks for up to [FINAL_WAIT_MS]. */+ fun stop(): JSONObject {+ if (!recording.compareAndSet(true, false)) {+ throw IllegalStateException("no meeting is being recorded")+ }+ return stopImpl(BY_BRAIN)+ }++ /** The ball tap and ■ — see [endAndNotify]. */+ fun stopFromUser() = endAndNotify(BY_USER)++ /**+ * End the meeting and push the transcript at the brain, because nobody+ * asked it for one.+ *+ * Safe from the main thread and from a binder thread: the teardown blocks+ * on 火山's last word, which would ANR the UI, so it goes to [exec] and the+ * result comes back through [onEnded].+ *+ * @param reason reaches the brain verbatim as `stopped_by`, and it says+ * this out loud to the user — so a focus loss must not claim they+ * touched the ball.+ */+ private fun endAndNotify(reason: String) {+ if (!recording.compareAndSet(true, false)) return+ runCatching {+ exec.execute {+ val out = runCatching { stopImpl(reason) }.getOrNull() ?: return@execute+ runCatching { onEnded(out) }+ }+ }+ }++ fun status(): JSONObject = JSONObject()+ .put("recording", recording.get())+ .put("title", title)+ .put("started_at", startedAt)+ .put("elapsed_ms", if (startedAt == 0L) 0L else System.currentTimeMillis() - startedAt)+ .put("segments", nextIndex.get())+ .put("chars", synchronized(transcript) { transcript.length })+ .put("file", fileRef())++ /**+ * Service teardown. Deliberately the silent stop, not [stopFromUser]:+ * there is no brain to write minutes for a device that is going away, and+ * the callback would post work onto a window that is about to be removed.+ *+ * Blocks, because the caller releases the microphone the instant this+ * returns and the last segment's text exists only in memory until [io]+ * runs — but without [FINAL_WAIT_MS], which would put three seconds into+ * `onDestroy` to gain a final that the settled partial already covers.+ */+ fun shutdown() {+ if (recording.compareAndSet(true, false)) {+ runCatching { stopImpl(BY_SHUTDOWN, waitForFinal = false) }+ }+ exec.shutdown()+ io.shutdown()+ runCatching { io.awaitTermination(2, java.util.concurrent.TimeUnit.SECONDS) }+ }++ private fun stopImpl(stoppedBy: String, waitForFinal: Boolean = true): JSONObject {+ runCatching { audio.stopStreaming() }+ val seg = active+ active = null+ if (seg != null) retire(seg, waitForFinal)+ focus.release()+ ui.post {+ ui.removeCallbacks(tick)+ window.clearSubtitle()+ window.setState(FloatingWindowUi.State.IDLE)+ }+ val out = result(stoppedBy)+ Log.i(TAG, "meeting ended ($stoppedBy): ${out.optInt("segments")} segments, ${out.optInt("chars")} chars")+ return out+ }++ // ---- Audio ----------------------------------------------------------++ /** `AudioIO-Streamer` thread. Must stay cheap — it is the read loop. */+ private fun onPcm(chunk: ByteArray) {+ val seg = active ?: return+ runCatching { seg.send(chunk) }+ .onFailure { Log.w(TAG, "sendAudio failed on segment ${seg.index}", it) }+ maybeRotate(seg)+ }++ /**+ * Rotate once the segment is old enough *and* the recogniser has gone quiet+ * — the seam then falls between sentences, where the hundred milliseconds+ * that both sessions briefly receive is silence and the duplication costs+ * nothing. [SEGMENT_HARD_MS] is for the meeting where nobody ever pauses.+ */+ private fun maybeRotate(seg: Segment) {+ if (rotatePending || !recording.get()) return+ val now = SystemClock.uptimeMillis()+ val age = now - seg.openedAt+ if (age < SEGMENT_SOFT_MS) return+ if (age < SEGMENT_HARD_MS && now - seg.lastTextAt < PAUSE_MS) return+ rotatePending = true+ // The audio thread can reach here a beat after [shutdown] closed the+ // executor; a rejection there would kill the read loop.+ runCatching { exec.execute { rotate(seg) } }+ .onFailure { rotatePending = false }+ }++ private fun rotate(old: Segment) {+ try {+ if (!recording.get() || active !== old) return+ val fresh = openSegment()+ active = fresh+ // The old segment's words are about to arrive as its final and+ // become committed text; leaving its partial up would show them+ // twice until they do.+ livePartial = ""+ retire(old)+ } catch (t: Throwable) {+ // The replacement never connected. The old session is still live+ // and still receiving, so the meeting is unharmed; push its clock+ // forward so the next attempt is a couple of minutes away rather+ // than on the very next chunk.+ Log.w(TAG, "rotation failed, keeping segment ${old.index}", t)+ old.openedAt = SystemClock.uptimeMillis()+ } finally {+ rotatePending = false+ }+ }++ private fun openSegment(): Segment {+ val seg = Segment(nextIndex.getAndIncrement())+ seg.asr.start(sampleRate = 16_000, channels = 1)+ return seg+ }++ private fun retire(seg: Segment, waitForFinal: Boolean = true) {+ runCatching { seg.asr.sendAudio(ByteArray(0), isLast = true) }+ if (waitForFinal) runCatching { seg.asr.waitFinished(FINAL_WAIT_MS) }+ runCatching { seg.asr.close() }+ // The latch also trips on error and on close, so a final is not+ // guaranteed to have arrived. Settling here keeps the last partial+ // rather than dropping two minutes of the meeting, and — because the+ // ordered flush below stalls on a missing index — guarantees every+ // segment reports something.+ seg.settle()+ }++ // ---- Transcript -----------------------------------------------------++ /**+ * Finals arrive on whichever WebSocket IO thread owns that segment, and a+ * retired segment's final races the live segment's. Hold them in index+ * order and release only a contiguous run, so the transcript can never+ * silently reorder what was said.+ */+ private fun noteFinal(index: Int, text: String) {+ val ready = ArrayList<String>()+ synchronized(finals) {+ finals[index] = text+ while (finals.containsKey(nextToFlush)) {+ val t = finals.remove(nextToFlush).orEmpty().trim()+ nextToFlush+++ if (t.isNotEmpty()) ready.add(t)+ }+ }+ if (ready.isEmpty()) return+ val block = ready.joinToString("\n", postfix = "\n")+ synchronized(transcript) { transcript.append(block) }+ renderLive()+ val f = file ?: return+ // runCatching around the submit too: a segment can settle from its+ // WebSocket thread after [shutdown] closed the writer, and a rejected+ // task must not surface as a crash inside 火山's callback.+ runCatching {+ io.execute {+ runCatching { f.appendText(block) }+ .onFailure { Log.w(TAG, "transcript append failed", it) }+ }+ }+ }++ private fun openTranscriptFile(): File? = runCatching {+ val dir = File(context.getExternalFilesDir(null), "meetings").apply { mkdirs() }+ val stamp = SimpleDateFormat("yyyyMMdd-HHmmss", Locale.US).format(startedAt)+ File(dir, "meeting-$stamp.txt").apply {+ writeText("# $title\n# ${SimpleDateFormat("yyyy-MM-dd HH:mm", Locale.US).format(startedAt)}\n\n")+ }+ }.onFailure { Log.w(TAG, "could not open transcript file", it) }.getOrNull()++ private fun fileRef(): Any = file?.let { "meetings/${it.name}" } ?: JSONObject.NULL++ private fun result(stoppedBy: String): JSONObject {+ val full = synchronized(transcript) { transcript.toString() }.trim()+ val truncated = full.length > MAX_RETURN_CHARS+ return JSONObject()+ .put("ended", true)+ .put("title", title)+ .put("started_at", startedAt)+ .put("ended_at", System.currentTimeMillis())+ .put("duration_ms", System.currentTimeMillis() - startedAt)+ .put("segments", nextIndex.get())+ .put("chars", full.length)+ .put("file", fileRef())+ .put("stopped_by", stoppedBy)+ .put("truncated", truncated)+ .put("transcript", if (truncated) elide(full) else full)+ }++ /**+ * Keep both ends. A meeting long enough to overflow states its purpose at+ * the start and its decisions at the end; the middle is the part minutes+ * are allowed to lose.+ */+ private fun elide(full: String): String {+ val half = MAX_RETURN_CHARS / 2+ val dropped = full.length - MAX_RETURN_CHARS+ return full.take(half) ++ "\n\n[$dropped characters elided in the middle; the full transcript is on the " ++ "device at ${fileRef()}]\n\n" ++ full.takeLast(half)+ }++ // ---- Ball ----------------------------------------------------------++ /**+ * The clock plus the tail of what has actually been heard.+ *+ * Written to the live row rather than the scrollback: an hour of meeting+ * is ~30 segments, and committing each one as its own row would evict the+ * real conversation out of a history that keeps only the last 80 rows. The+ * tail is a window onto the transcript, not a second copy of it — the file+ * on disk is the record.+ *+ * Called from the 1 Hz [tick] so the clock stays honest, and from every+ * partial so the words don't arrive a second late.+ */+ private fun renderLive() {+ if (!recording.get()) return+ val s = (System.currentTimeMillis() - startedAt) / 1000+ val head = AppLocale.str(+ context,+ net.kuafuai.andee.R.string.meeting_live_head,+ s / 60,+ s % 60,+ )+ val committed = synchronized(transcript) {+ transcript.substring(maxOf(0, transcript.length - TAIL_CHARS))+ }+ val heard = (committed.replace('\n', ' ') + livePartial)+ .takeLast(TAIL_CHARS)+ .trim()+ window.setSubtitle(+ if (heard.isEmpty()) head else "$head\n$heard",+ FloatingWindowUi.SubtitleKind.PARTIAL,+ )+ }++ private val tick = object : Runnable {+ override fun run() {+ if (!recording.get()) return+ renderLive()+ ui.postDelayed(this, 1000L)+ }+ }++ // ---- One recogniser session ----------------------------------------++ private inner class Segment(val index: Int) : HuoshanAsr.Listener {+ val asr = HuoshanAsr(VoiceConfig.load(context), this)++ /** Reset by [rotate] when a replacement could not be opened. */+ @Volatile+ var openedAt = SystemClock.uptimeMillis()++ /** Last time the recogniser produced *new* words — the pause detector. */+ @Volatile+ var lastTextAt = SystemClock.uptimeMillis()++ @Volatile+ private var partial = ""++ private val settled = AtomicBoolean(false)++ fun send(pcm: ByteArray) = asr.sendAudio(pcm, isLast = false)++ /** Contribute this segment's text to the transcript exactly once. */+ fun settle(text: String = partial) {+ if (settled.compareAndSet(false, true)) noteFinal(index, text)+ }++ override fun onPartial(text: String) {+ if (text.isEmpty() || text == partial) return+ partial = text+ lastTextAt = SystemClock.uptimeMillis()+ // Only the live segment drives the display; a retiring one's late+ // partial would drag the tail backwards.+ if (active === this) {+ livePartial = text+ renderLive()+ }+ }++ override fun onFinal(text: String) = settle(text)++ override fun onError(msg: String) {+ Log.w(TAG, "segment $index error: $msg")+ settle()+ }++ override fun onClosed() {}+ }++ private companion object {+ const val TAG = "Meeting"++ /**+ * `stopped_by` values. The brain narrates this to the user, so each+ * one has to be true: "user" is the ball or ■, "focus" is something+ * else on the device taking the audio floor (a phone call), "brain" is+ * its own `stop_meeting`, "shutdown" is the service going away.+ */+ const val BY_USER = "user"+ const val BY_FOCUS = "focus"+ const val BY_BRAIN = "brain"+ const val BY_SHUTDOWN = "shutdown"++ /** Earliest a segment may be rotated out, given a pause to do it in. */+ const val SEGMENT_SOFT_MS = 120_000L++ /** Rotate regardless — nobody in this meeting is stopping to breathe. */+ const val SEGMENT_HARD_MS = 180_000L++ /** Quiet this long and the seam is safe to put here. */+ const val PAUSE_MS = 800L++ /** 火山 sends its final ~500 ms after our isLast; this is the ceiling. */+ const val FINAL_WAIT_MS = 3000L++ /** Roughly a four-hour Chinese meeting. Beyond it, see [elide]. */+ const val MAX_RETURN_CHARS = 60_000++ /**+ * How much of the transcript the live row shows.+ *+ * Deliberately short. The row is `maxLines = 4` ellipsized at the+ * *end*, so a tail that overflows hides its own last words — the+ * newest ones, the only ones worth watching. One line goes to the+ * clock, leaving three; this fits them on a narrow screen and simply+ * scrolls sooner on a wide one.+ */+ const val TAIL_CHARS = 80+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/net/BodyWsClient.kt b/app/src/main/java/net/kuafuai/andee/net/BodyWsClient.ktnew file mode 100644index 0000000..0027fe0--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/net/BodyWsClient.kt@@ -0,0 +1,268 @@+package net.kuafuai.andee.net++import android.util.Log+import org.java_websocket.client.WebSocketClient+import org.java_websocket.handshake.ServerHandshake+import org.json.JSONObject+import java.net.URI+import java.util.concurrent.Executors+import java.util.concurrent.ScheduledExecutorService+import java.util.concurrent.TimeUnit+import java.util.concurrent.atomic.AtomicBoolean++/**+ * Outbound side of the body's networking: dials into a remote hub (agentworld+ * or similar), sends a `register` envelope with our tool schemas, then serves+ * `request` messages coming down the pipe by handing them to+ * [CommandDispatcher] and mailing the result back.+ *+ * Reconnects with exponential backoff (500ms → 30s, capped). Runs alongside+ * [BodyWsServer] — that side keeps serving viewer/local traffic; this side is+ * how we get to a hub that lives behind NAT / on the public internet.+ *+ * Hub protocol:+ * → {"type":"register","data":{"device_id":"...","device_name":"...","tools":[...]}}+ * ← {"type":"request","id":"r7","method":"tap_screen","params":{...}}+ * → {"type":"response","id":"r7","result":{...}}+ * → {"type":"response","id":"r7","error":"..."}+ * → {"type":"event","kind":"asr.final","device_id":"...","data":{...}}+ * ← {"type":"progress","content":"读取界面","agent_id":"..."}+ * ← {"type":"message","content":"已经帮你打开微信了","agent_id":"..."}+ *+ * `progress` is narration for the human — one subtitle line, never spoken.+ * `message` is the agent's finished answer, which the body speaks aloud.+ * Both are fire-and-forget; neither expects a response.+ *+ * Method names on the wire are the LLM-facing tool names (tap_screen,+ * get_screen_element, …). We translate them to the body's internal `screen.*`+ * method strings via [ToolSchemas.methodOf] before dispatching.+ */+class BodyWsClient(+ private val url: String,+ private val deviceId: String,+ private val deviceName: String,+ private val dispatcher: CommandDispatcher,+ /**+ * Extension-provided tools are merged into the `register` payload alongside+ * [ToolSchemas.all]. When the registry changes, the service tears this+ * client down and starts a fresh one so the hub sees an up-to-date list —+ * the hub has no separate "tools changed" frame, `register` is the only+ * lever we have on its idea of what this device can do.+ */+ private val registry: CallExtensionRegistry,+ private val onProgress: (String) -> Unit = {},+ private val onFinal: (String) -> Unit = {},+) {++ private val running = AtomicBoolean(false)+ private val scheduler: ScheduledExecutorService =+ Executors.newSingleThreadScheduledExecutor { r ->+ Thread(r, "BodyWsClient-scheduler").apply { isDaemon = true }+ }++ // Requests can block (dispatchGesture waits ~seconds), so hand them off+ // to a worker so we don't stall the recv thread and miss pings/pongs.+ private val worker = Executors.newSingleThreadExecutor { r ->+ Thread(r, "BodyWsClient-worker").apply { isDaemon = true }+ }++ @Volatile+ private var conn: Inner? = null++ @Volatile+ private var backoffMs: Long = INITIAL_BACKOFF_MS++ fun start() {+ if (!running.compareAndSet(false, true)) return+ Log.i(TAG, "starting hub client → $url (device=$deviceId '$deviceName')")+ scheduleConnect(0)+ }++ fun stop() {+ if (!running.compareAndSet(true, false)) return+ val c = conn+ conn = null+ runCatching { c?.close() }+ scheduler.shutdownNow()+ worker.shutdownNow()+ }++ // ------------------------------------------------------------------+ // internal+ // ------------------------------------------------------------------++ private fun scheduleConnect(delayMs: Long) {+ if (!running.get()) return+ scheduler.schedule({ if (running.get()) connectOnce() }, delayMs, TimeUnit.MILLISECONDS)+ }++ private fun connectOnce() {+ try {+ val c = Inner(URI(url))+ conn = c+ c.connect()+ } catch (t: Throwable) {+ Log.e(TAG, "connect failed: ${t.message}", t)+ onDisconnected()+ }+ }++ private fun onConnected() {+ backoffMs = INITIAL_BACKOFF_MS // reset backoff on success+ // forHub(), NOT all(): the five `note.*` notebook tools are localOnly.+ // The notebook holds what this person likes, what they promised, and+ // what they are like — it is the one part of this app that is nobody+ // else's business, so it is never advertised to the hub. Swapping this+ // back to all() would silently ship it on the next reconnect, with no+ // visible symptom other than the cloud brain gaining five tools.+ val builtin = ToolSchemas.forHub()+ val extra = registry.tools()+ // Merge order: built-ins first, extension tools appended. The hub+ // stores them by name, so order only matters for logs — but keeping+ // built-ins ahead means a duplicate name from the extension can't hide+ // one of our own by accident (we don't reject collisions, we just get+ // to notice them in the tail).+ val tools = org.json.JSONArray()+ for (i in 0 until builtin.length()) tools.put(builtin.getJSONObject(i))+ for (i in 0 until extra.length()) tools.put(extra.getJSONObject(i))+ val register = JSONObject()+ .put("type", "register")+ .put(+ "data", JSONObject()+ .put("device_id", deviceId)+ .put("device_name", deviceName)+ .put("tools", tools)+ )+ runCatching { conn?.send(register.toString()) }+ Log.i(+ TAG,+ "registered as $deviceId with ${tools.length()} tools " ++ "(${builtin.length()} built-in + ${extra.length()} from extension)"+ )+ }++ private fun onDisconnected() {+ if (!running.get()) return+ val next = backoffMs+ backoffMs = (backoffMs * 2).coerceAtMost(MAX_BACKOFF_MS)+ Log.i(TAG, "reconnect in ${next}ms")+ scheduleConnect(next)+ }++ private fun handleRequest(req: JSONObject) {+ val id = req.optString("id", "")+ val toolName = req.optString("method", "")+ val params = req.optJSONObject("params")+ // Two-name translation for built-ins (`tap_screen` → `screen.tap`).+ // Extension tools don't play this game — the schema they registered+ // carries one name, and that same name goes straight to the dispatcher,+ // which will hand it back to the extension over the reverse channel.+ val method = ToolSchemas.methodOf(toolName)+ ?: if (registry.hasMethod(toolName)) toolName else null+ if (method == null) {+ respondError(id, "unknown tool: $toolName")+ return+ }+ // A request on *this* socket is the brain driving, which is what the+ // edge glow announces. Marked here rather than in the dispatcher+ // because only the transport can tell the brain apart from the top-bar+ // buttons and the debug client on 9008, which share the same dispatch+ // path and are nobody's task. Usually redundant — `progress` normally+ // arrives first — but a model that opens with a tool call still counts.+ dispatcher.beginTask()+ worker.execute {+ val response = try {+ val result = dispatcher.dispatch(method, params)+ JSONObject()+ .put("type", "response")+ .put("id", id)+ .put("result", result ?: JSONObject.NULL)+ } catch (t: Throwable) {+ Log.w(TAG, "tool '$toolName' failed: ${t.message}")+ JSONObject()+ .put("type", "response")+ .put("id", id)+ .put("error", t.message ?: t.javaClass.simpleName)+ }+ runCatching { conn?.send(response.toString()) }+ }+ }++ /**+ * Push an unsolicited event up to the hub (ASR transcripts, etc). Dropped+ * silently when the socket is down — events are not queued, the brain only+ * ever sees what happened while it was listening.+ */+ fun sendEvent(kind: String, data: JSONObject) {+ val c = conn ?: return+ if (!c.isOpen) return+ val envelope = JSONObject()+ .put("type", "event")+ .put("kind", kind)+ .put("device_id", deviceId)+ .put("data", data)+ runCatching { c.send(envelope.toString()) }+ }++ private fun respondError(id: String, msg: String) {+ val response = JSONObject()+ .put("type", "response")+ .put("id", id)+ .put("error", msg)+ runCatching { conn?.send(response.toString()) }+ }++ /**+ * WebSocketClient callbacks. Small inner class so we can null out [conn]+ * cleanly across reconnects.+ */+ private inner class Inner(uri: URI) : WebSocketClient(uri) {++ override fun onOpen(handshake: ServerHandshake?) {+ Log.i(TAG, "onOpen ${uri}")+ current = this@BodyWsClient+ onConnected()+ }++ override fun onMessage(message: String) {+ val msg = runCatching { JSONObject(message) }.getOrNull() ?: return+ when (msg.optString("type")) {+ "request" -> handleRequest(msg)+ // Narration and final answers arrive on the recv thread; both+ // handlers only touch @Volatile UI state or post to the main+ // Handler, so there's nothing to hand off to the worker here.+ "progress" -> msg.optString("content").takeIf { it.isNotBlank() }?.let(onProgress)+ "message" -> msg.optString("content").takeIf { it.isNotBlank() }?.let(onFinal)+ else -> Log.d(TAG, "ignoring type=${msg.optString("type")}")+ }+ }++ override fun onClose(code: Int, reason: String?, remote: Boolean) {+ Log.i(TAG, "onClose code=$code reason=$reason remote=$remote")+ if (conn === this) conn = null+ if (current === this@BodyWsClient) current = null+ onDisconnected()+ }++ override fun onError(ex: Exception?) {+ Log.w(TAG, "onError ${ex?.message}")+ }+ }++ companion object {+ private const val TAG = "BodyWsClient"+ private const val INITIAL_BACKOFF_MS = 500L+ private const val MAX_BACKOFF_MS = 30_000L++ /**+ * The connected client, for services outside ScreenBodyService's+ * wiring (NotificationRelayService pushing `notification` events).+ * Null while the hub link is down — pushers drop silently by design:+ * buffering-and-replaying after reconnect would deliver stale+ * "new message" interrupts.+ */+ @Volatile+ var current: BodyWsClient? = null+ private set+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/net/BodyWsServer.kt b/app/src/main/java/net/kuafuai/andee/net/BodyWsServer.ktnew file mode 100644index 0000000..f6aca05--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/net/BodyWsServer.kt@@ -0,0 +1,291 @@+package net.kuafuai.andee.net++import android.util.Log+import org.java_websocket.WebSocket+import org.java_websocket.handshake.ClientHandshake+import org.java_websocket.server.WebSocketServer+import org.json.JSONArray+import org.json.JSONObject+import java.net.InetSocketAddress+import java.util.UUID+import java.util.concurrent.ConcurrentHashMap+import java.util.concurrent.CopyOnWriteArraySet+import java.util.concurrent.SynchronousQueue+import java.util.concurrent.TimeUnit++/**+ * The body's WebSocket server. Two very different clients arrive here:+ *+ * - **Drivers** — the brain (via `adb forward`), a debug/viewer client, top-bar+ * tests. They send `request` frames and receive `response`s and broadcast+ * `event`s. This is the original shape.+ *+ * - **Call extension** — a peripheral (see whatsapp-agent's `bodyClient.ts`)+ * that dials in, sends `{"type":"register","role":"call_extension",...}`,+ * and hands us a batch of tool schemas it can serve. The body then reverses+ * the arrow: request/response goes body → extension for those tool names,+ * and the extension pushes `event` frames back up. It is not a driver, so+ * it never sees broadcast events and does not sit in [active].+ *+ * Envelopes:+ *+ * Driver → Server:+ * {"type":"request","id":"...","method":"screen.tap","params":{...}}+ *+ * Server → Driver:+ * {"type":"response","id":"...","result":{...}}+ * {"type":"response","id":"...","error":"..."}+ * {"type":"event","kind":"asr.final","data":{...}}+ *+ * Extension → Server:+ * {"type":"register","role":"call_extension",+ * "device_name":"...","scene":"...","tools":[{...}, ...]}+ * {"type":"response","id":"...","result":{...}|"error":"..."}+ * {"type":"event","kind":"call.state|call.subtitle|call.summary","data":{...}}+ *+ * Server → Extension:+ * {"type":"request","id":"...","method":"pickup_call","params":{...}}+ *+ * [onCommand] returns a JSON-serializable value (usually JSONObject) or throws.+ * Thrown exceptions become error responses; nothing else propagates.+ *+ * Binding to 0.0.0.0 by default so the brain running on another host on the+ * LAN can reach it. Loopback-only is also fine when driven via adb forward.+ */+class BodyWsServer(+ port: Int,+ bindHost: String = "0.0.0.0",+ private val registry: CallExtensionRegistry,+ /** Extension-pushed event; the service turns these into hub broadcasts + UI updates. */+ private val onExtensionEvent: (kind: String, data: JSONObject) -> Unit = { _, _ -> },+ private val onCommand: (method: String, params: JSONObject?) -> Any?,+) : WebSocketServer(InetSocketAddress(bindHost, port)) {++ /** Driver connections only — the ones broadcasts go to. Extensions never join. */+ private val active = CopyOnWriteArraySet<WebSocket>()++ /**+ * In-flight reverse-requests we sent to the extension, keyed by envelope+ * id. Reused for their responses — a SynchronousQueue of length one so a+ * late response after timeout has nowhere to sit and drops harmlessly.+ */+ private val pending = ConcurrentHashMap<String, SynchronousQueue<PendingResult>>()++ init {+ isReuseAddr = true+ }++ override fun onOpen(conn: WebSocket, handshake: ClientHandshake?) {+ // Provisionally treat every new connection as a driver. If a register+ // frame arrives naming role=call_extension, we lift it back out.+ active.add(conn)+ Log.d(TAG, "onOpen ${conn.remoteSocketAddress} drivers=${active.size}")+ }++ override fun onClose(conn: WebSocket, code: Int, reason: String?, remote: Boolean) {+ active.remove(conn)+ if (registry.unregister(conn)) {+ Log.i(TAG, "call extension disconnected: ${conn.remoteSocketAddress} code=$code reason=$reason")+ // Nothing to talk to for any request we sent out — wake them all+ // with an error so brain gets a proper failure instead of the hub+ // request timing out at 30s.+ failAllPending("call extension disconnected")+ }+ Log.d(TAG, "onClose ${conn.remoteSocketAddress} code=$code reason=$reason remote=$remote drivers=${active.size}")+ }++ override fun onMessage(conn: WebSocket, message: String) {+ val msg = runCatching { JSONObject(message) }.getOrNull()+ if (msg == null) {+ Log.w(TAG, "non-JSON message ignored: ${message.take(100)}")+ return+ }+ when (msg.optString("type")) {+ "request" -> handleDriverRequest(conn, msg)+ "register" -> handleRegister(conn, msg)+ "response" -> handleExtensionResponse(msg)+ "event" -> handleExtensionEvent(conn, msg)+ else -> Log.d(TAG, "ignoring type=${msg.optString("type")}")+ }+ }++ private fun handleDriverRequest(conn: WebSocket, req: JSONObject) {+ val id = req.optString("id", "")+ val method = req.optString("method", "")+ val params = req.optJSONObject("params")+ Log.d(TAG, "request ${conn.remoteSocketAddress} $method")+ // Dispatch OFF the WS thread: blocking tools (ui.ask waits for a+ // human, scan waits for a code, look waits for a frame) would+ // otherwise stall ping/pong handling and the client drops us.+ dispatchPool.execute {+ val response = try {+ val result = onCommand(method, params)+ JSONObject()+ .put("type", "response")+ .put("id", id)+ .put("result", result ?: JSONObject.NULL)+ } catch (t: Throwable) {+ Log.w(TAG, "command '$method' failed: ${t.message}")+ JSONObject()+ .put("type", "response")+ .put("id", id)+ .put("error", t.message ?: t.javaClass.simpleName)+ }+ runCatching { conn.send(response.toString()) }+ }+ }++ private fun handleRegister(conn: WebSocket, msg: JSONObject) {+ val role = msg.optString("role")+ if (role != "call_extension") {+ Log.w(TAG, "register with unknown role='$role', ignoring")+ return+ }+ val data = msg.optJSONObject("data") ?: msg // tolerate either shape+ val tools = data.optJSONArray("tools") ?: JSONArray()+ val deviceName = data.optString("device_name").ifEmpty { null }+ val scene = data.optString("scene").ifEmpty { null }+ // A registered extension is a service peer, not a broadcast recipient.+ active.remove(conn)+ val previous = registry.register(conn, tools, deviceName, scene)+ if (previous != null && previous !== conn) {+ // One slot; a second extension replaces the first. Fail its+ // outstanding requests before closing, or their callers hang.+ failAllPending("call extension replaced by ${conn.remoteSocketAddress}")+ runCatching { previous.close(1000, "replaced by newer registration") }+ }+ Log.i(+ TAG,+ "call extension registered: ${conn.remoteSocketAddress} " ++ "name='${deviceName ?: "-"}' tools=${tools.length()}"+ )+ }++ private fun handleExtensionResponse(msg: JSONObject) {+ val id = msg.optString("id")+ if (id.isEmpty()) return+ val slot = pending.remove(id) ?: run {+ // Late (post-timeout) or unknown; nothing to wake.+ return+ }+ val err = msg.optString("error").ifEmpty { null }+ val result = if (msg.has("result")) msg.opt("result") else null+ // offer, not put: if the receiver already gave up, this drops instead+ // of pinning the WS thread on a queue nobody is reading.+ slot.offer(PendingResult(result, err), 0, TimeUnit.MILLISECONDS)+ }++ private fun handleExtensionEvent(conn: WebSocket, msg: JSONObject) {+ // Only accept events from the currently-registered extension. A stray+ // driver that pushes `event` frames has no channel to publish on.+ if (registry.connection() !== conn) {+ Log.d(TAG, "event from non-extension ${conn.remoteSocketAddress}, ignoring")+ return+ }+ val kind = msg.optString("kind")+ if (kind.isEmpty()) return+ val data = msg.optJSONObject("data") ?: JSONObject()+ runCatching { onExtensionEvent(kind, data) }+ .onFailure { Log.w(TAG, "onExtensionEvent threw for kind=$kind", it) }+ }++ private val dispatchPool = java.util.concurrent.Executors.newCachedThreadPool { r ->+ Thread(r, "BodyWsServer-dispatch").apply { isDaemon = true }+ }++ override fun onError(conn: WebSocket?, ex: Exception) {+ Log.e(TAG, "onError ${conn?.remoteSocketAddress}", ex)+ if (conn != null) {+ active.remove(conn)+ if (registry.unregister(conn)) failAllPending("call extension error: ${ex.message}")+ }+ }++ override fun onStart() {+ Log.i(TAG, "listening on ${address}")+ }++ /** Broadcast an event to all driver clients. Extensions never receive it. */+ fun broadcastEvent(kind: String, data: JSONObject) {+ if (active.isEmpty()) return+ val envelope = JSONObject()+ .put("type", "event")+ .put("kind", kind)+ .put("data", data)+ val json = envelope.toString()+ for (c in active) runCatching { c.send(json) }+ }++ fun clientCount(): Int = active.size++ /**+ * Send [method] to the registered call extension and block for its response.+ *+ * Throws [IllegalStateException] when no extension is connected — the caller+ * (dispatcher) turns that into a proper `error` response to the brain, same+ * as any other tool failure. Also throws on timeout: `body_hub` gives a+ * request 30s, so [timeoutMs] defaults well under that.+ *+ * Runs on whatever thread the caller is on (usually a dispatch worker).+ */+ fun forwardToExtension(+ method: String,+ params: JSONObject?,+ timeoutMs: Long = DEFAULT_EXTENSION_TIMEOUT_MS,+ ): Any? {+ val conn = registry.connection()+ ?: throw IllegalStateException(+ "call extension is not connected — is the call box on and reachable?"+ )+ val id = UUID.randomUUID().toString()+ val slot = SynchronousQueue<PendingResult>()+ pending[id] = slot+ val envelope = JSONObject()+ .put("type", "request")+ .put("id", id)+ .put("method", method)+ if (params != null) envelope.put("params", params)+ try {+ if (!conn.isOpen) {+ throw IllegalStateException("call extension socket is closed")+ }+ conn.send(envelope.toString())+ } catch (t: Throwable) {+ pending.remove(id)+ throw t+ }+ val out = try {+ slot.poll(timeoutMs, TimeUnit.MILLISECONDS)+ } finally {+ pending.remove(id)+ }+ ?: throw IllegalStateException(+ "call extension did not respond within ${timeoutMs}ms for $method"+ )+ if (out.error != null) throw IllegalStateException(out.error)+ return out.result+ }++ /** Wake every pending reverse-request with an error. Called on extension loss. */+ private fun failAllPending(reason: String) {+ // Drain snapshot to avoid CME while other threads try to install more.+ val ids = pending.keys.toList()+ for (id in ids) {+ val slot = pending.remove(id) ?: continue+ slot.offer(PendingResult(null, reason), 0, TimeUnit.MILLISECONDS)+ }+ }++ private data class PendingResult(val result: Any?, val error: String?)++ companion object {+ private const val TAG = "BodyWs"++ /**+ * Ceiling for a body→extension request. Under the hub's 30s so the+ * failure surfaces here with a real message ("did not respond") rather+ * than as an anonymous hub timeout.+ */+ const val DEFAULT_EXTENSION_TIMEOUT_MS = 25_000L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/net/CallExtensionRegistry.kt b/app/src/main/java/net/kuafuai/andee/net/CallExtensionRegistry.ktnew file mode 100644index 0000000..435705d--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/net/CallExtensionRegistry.kt@@ -0,0 +1,108 @@+package net.kuafuai.andee.net++import org.java_websocket.WebSocket+import org.json.JSONArray+import org.json.JSONObject++/**+ * Single state slot for the "call extension" — a peripheral (e.g. the+ * whatsapp-agent box on another machine) that dials into [BodyWsServer], sends+ * a `register` frame with `role="call_extension"`, and thereby lends its own+ * tools to this body. From brain's point of view those tools show up in the+ * same `register` payload the hub gets from us, next to `screen.*` / `device.*`+ * / `meeting.*` — one device, more arms.+ *+ * Kept as one slot on purpose. The physical topology behind an extension is one+ * phone bridged through USB sound cards; two extensions racing to touch the+ * same phone is not a thing that means anything, and a second connect just+ * replaces the first (see [BodyWsServer.onMessage]).+ *+ * Not thread-safe beyond the volatile writes: [register] / [unregister] are+ * always called from a single [BodyWsServer] callback thread. Readers off other+ * threads see the latest snapshot but may race the transition; the extension's+ * absence is the only failure mode and every reader already handles it.+ */+class CallExtensionRegistry {++ @Volatile+ private var current: WebSocket? = null++ @Volatile+ private var toolsRaw: JSONArray = JSONArray()++ @Volatile+ private var sceneText: String? = null++ @Volatile+ private var deviceName: String? = null++ /**+ * Fires on both register AND unregister, so the service can trigger a+ * single hub re-register whichever way the extension state just changed —+ * the hub's `register` payload contains a snapshot of our tools, and a+ * stale one is a tool call that fails at dispatch time. Runs on the WS+ * callback thread; keep it short.+ */+ @Volatile+ var onChange: (() -> Unit)? = null++ /** The currently registered extension connection, or null. */+ fun connection(): WebSocket? = current++ /** Tool schemas as the extension registered them. Never null; may be empty. */+ fun tools(): JSONArray = toolsRaw++ /** Human-facing description the extension self-supplied ("USB dongle #1"). */+ fun deviceName(): String? = deviceName++ /** Prompt-facing scene description the extension supplied for the brain. */+ fun scene(): String? = sceneText++ /**+ * Whether an extension is currently offering a tool by this name. Iterated+ * off the volatile snapshot; ok to race — a mid-swap "no" is just a race a+ * fresh call would win.+ */+ fun hasMethod(name: String): Boolean {+ val snap = toolsRaw+ for (i in 0 until snap.length()) {+ if (snap.optJSONObject(i)?.optString("name") == name) return true+ }+ return false+ }++ /**+ * Install a new extension. Returns the previous connection when one+ * existed — caller is responsible for closing it, because the answer to+ * "was there one already" is also the answer to "should I hang up on it".+ */+ fun register(+ conn: WebSocket,+ tools: JSONArray,+ deviceName: String?,+ scene: String?,+ ): WebSocket? {+ val previous = current+ current = conn+ toolsRaw = tools+ this.deviceName = deviceName?.ifEmpty { null }+ this.sceneText = scene?.ifEmpty { null }+ onChange?.invoke()+ return previous+ }++ /**+ * Drop [conn] if it is the currently registered one. No-op when the+ * argument is not the live extension — a late `onClose` from a connection+ * that was already replaced must not clobber the replacement's state.+ */+ fun unregister(conn: WebSocket): Boolean {+ if (current !== conn) return false+ current = null+ toolsRaw = JSONArray()+ deviceName = null+ sceneText = null+ onChange?.invoke()+ return true+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/net/CommandDispatcher.kt b/app/src/main/java/net/kuafuai/andee/net/CommandDispatcher.ktnew file mode 100644index 0000000..baf77aa--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/net/CommandDispatcher.kt@@ -0,0 +1,1320 @@+package net.kuafuai.andee.net++import android.os.Handler+import android.os.Looper+import net.kuafuai.andee.R+import net.kuafuai.andee.config.CronExpr+import net.kuafuai.andee.config.Notebook+import net.kuafuai.andee.config.Scheduler+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.screen.ScreenController+import net.kuafuai.andee.ui.EdgeRippleUi+import net.kuafuai.andee.ui.FloatingWindowUi+import org.json.JSONObject+import java.util.concurrent.CountDownLatch+import java.util.concurrent.TimeUnit++/**+ * Routes WS `request` messages (method + params) to controller methods.+ *+ * All screen actions live under the `screen.*` namespace. TTS/ASR/audio+ * command routing will be added later under their own namespaces+ * (`tts.speak`, `asr.*`, etc.).+ *+ * Contract: return a JSON-serializable value (usually JSONObject) or throw.+ * Thrown exceptions become error responses in [BodyWsServer].+ *+ * Gesture-based commands (tap / swipe / long_press) go through+ * [ScreenController] which calls `dispatchGesture`. Those events run through+ * the real touch pipeline and would land on our own overlay first, so we+ * temporarily flip the floating window to non-touchable around each such+ * call — see [passthroughForGesture]. Eid taps (tap_id / long_press_id) are+ * gestures too — they resolve to pixel centers and dispatch a real touch —+ * so they go through the same dance. Accessibility-API-based commands+ * (type, global, ui_tree) don't touch the input layer and skip this.+ *+ * `screen.screenshot` has the mirror-image problem: it doesn't touch the input+ * layer, but the overlay *is* part of the display capture — see+ * [blankedForCapture].+ *+ * Every `screen.*` command also folds the card into the corner ball first —+ * see [ensureCompact]. What used to be a tool the brain had to remember to+ * call is compiled in now, so "the assistant is operating an app" and "the+ * user is able to see the app" never diverge. Only for callers that pass+ * `driving = true`, though: see [dispatch].+ *+ * **Two audiences.** Three strings in here go to the user — the vault subtitle,+ * and the two `ui.ask` button labels next to it — so they come from+ * `AppLocale`. *Everything else* in this file is a tool result: it is read by+ * the model, it is often a sentence telling the model off, and it is English+ * like the rest of the model-facing set (see the top of `res/values-en/`+ * `strings.xml`). A `设置 → 保险箱` left inside one of those English sentences+ * is not a leak and must not be translated: it is a literal path on the screen+ * that the model should repeat to the user verbatim.+ */+class CommandDispatcher(+ private val screen: ScreenController,+ private val window: FloatingWindowUi,+ private val ripple: EdgeRippleUi,+ private val marker: net.kuafuai.andee.ui.TapMarkerUi,+ private val deviceInfo: net.kuafuai.andee.device.DeviceInfoController,+ private val deviceActions: net.kuafuai.andee.device.DeviceActionsController,+ private val deviceComms: net.kuafuai.andee.device.DeviceCommsController,+ private val permissions: net.kuafuai.andee.device.PermissionsController,+ private val meeting: net.kuafuai.andee.meeting.MeetingRecorder,+ private val dog: net.kuafuai.andee.dog.DogMotion,+ private val appContext: android.content.Context,+ /**+ * Fallback for methods this dispatcher does not itself know. Wired to the+ * call extension by [ScreenBodyService]: if a call peripheral has+ * registered a tool by this name (`pickup_call`, `end_call`, …), forward+ * the request; if not, the lambda throws, and the throw becomes the brain's+ * `unknown method` reply like it always did.+ *+ * Kept as a lambda rather than a direct [CallExtensionRegistry] handle so+ * the dispatcher stays unaware of the WS transport — same shape that lets+ * tests hand it a stub.+ */+ private val forwardExternal: (method: String, params: JSONObject?) -> Any? =+ { method, _ -> throw IllegalArgumentException("unknown method: $method") },++ /**+ * Told when the user dismisses a page `ui.show_html` put up, with how long+ * it was on screen.+ *+ * A callback rather than a direct handle on the brain because the brain is+ * three things ([ScreenBodyService.hubClient], its `wsServer`, and the+ * on-device [net.kuafuai.andee.brain.LocalBrain]) and this class reaches+ * none of them — it is the device's tool surface, deliberately unaware of+ * the transports. [ScreenBodyService] wires it.+ */+ private val onPageClosed: (title: String, seconds: Int) -> Unit = { _, _ -> },++ /**+ * Read a brain-facing card's text aloud as the card appears — the+ * `ui.ask` question, the `ui.alert` notice — and the whole of+ * [`tts.speak`], which is the same request with no card attached.+ *+ * Returns whether the words were actually handed to TTS: the mute rules+ * live behind this lambda and they can refuse. `ui.ask` / `ui.alert`+ * ignore the answer (the card is up either way and that is the part they+ * promised); `tts.speak` has nothing else, so it reports it rather than+ * returning a cheerful `{spoken:true}` over a silence — a muted line and+ * a spoken one must not be indistinguishable from the caller's side.+ *+ * A callback rather than a direct [net.kuafuai.andee.tts.TtsController]+ * handle for the same reason [onPageClosed] is one: the TTS mute rules+ * (meeting, phone call), the stop gate and `stripMarkdown` all live in+ * [ScreenBodyService], where the final answer already passes through+ * them — a second copy of those rules is where this feature would rot.+ * The card is for a user who may be looking anywhere but the screen;+ * a question nobody hears just waits in silence for its timeout.+ */+ private val sayAloud: (String) -> Boolean = { false },+) {+ private val ui = Handler(Looper.getMainLooper())++ /** Leak guard, not a guess about thinking time — see [setBusy]. */+ private val endDriving = Runnable { setBusy(false) }++ /** Written on the main thread, read by worker threads: card already folded. */+ @Volatile+ private var compacted = false++ /** Main thread only: a task is being thought about or executed right now. */+ private var busy = false++ /**+ * @param driving whether the caller is operating the device on purpose —+ * the brain over the hub socket, or the user's own top-bar buttons. False+ * is a passive observer: the debug/viewer client on 9008 polls+ * `screen.ui_tree` and `screen.screenshot` on a timer, and a read on a+ * timer must not rearrange what the user is looking at. Without this the+ * card folded itself into the corner once a minute with nobody driving —+ * the same distinction [touchBusy] already draws for the edge glow, which+ * the fold was simply missing.+ * @param silent the turn is the quiet-hour sweep: the brain is reviewing a+ * finished conversation by itself. Only the `note.*` notebook methods are+ * allowed through — see the guard at the top of this function.+ */+ fun dispatch(+ method: String,+ params: JSONObject?,+ driving: Boolean = true,+ silent: Boolean = false,+ ): Any? {+ // The quiet-hour sweep runs with nobody watching: it may think, and it+ // may write to the notebook, but it must not touch the device. Its+ // prompt says the same thing — this is the copy that cannot drift, and+ // it lives here rather than in LocalBrain because this is the single+ // place every method passes through. Checked before the fold below,+ // so a silent call cannot so much as move the card.+ if (silent && !method.startsWith("note.")) {+ throw IllegalStateException(+ "A silent turn may only use the notebook (note.*); it must not touch the device: $method",+ )+ }+ // Every command below reaches into the real device, and every one of+ // them is worth watching, so get out of the way *before* doing it.+ // Prefix rather than an explicit list, so adding a tool stays the+ // two-file edit it already was ([ToolSchemas] + here) — at the cost of+ // anything new under `screen.*` inheriting the fold, which is right+ // only as long as everything in that namespace drives the device.+ if (driving && method.startsWith("screen.")) {+ ensureCompact()+ // Unconditional among drivers, unlike [touchBusy]: "a tool just+ // ran" is true whoever asked, and a ball that nods when the top bar+ // drives the device is telling the truth. Only the glow needs to+ // know whose tool call it was.+ window.pulseTool()+ }+ return when (method) {+ // ---- device.* : read-only context queries ----------------------+ // Deliberately outside the `screen.` prefix so they never fold the+ // card: asking "am I online" is a conversation, not an actuation.+ // All bounded-latency, no listeners outlive the call. `device.scan`+ // and `device.look` fold anyway, in their own branches — they are+ // queries that draw a camera preview the user has to see.+ "device.connectivity" -> deviceInfo.connectivity()+ "device.apps" -> deviceInfo.installedApps()+ "device.location" -> deviceInfo.location(+ timeoutMs = params?.optLong("timeout_ms", 8000L) ?: 8000L+ )+ "device.steps" -> deviceInfo.steps(+ timeoutMs = params?.optLong("timeout_ms", 3000L) ?: 3000L+ )+ "device.battery" -> deviceInfo.battery()++ // ---- device.* : actions, zero/low-friction ---------------------+ "device.launch" -> deviceActions.launchApp(+ pkg = params?.optString("pkg")?.ifEmpty { null },+ label = params?.optString("label")?.ifEmpty { null },+ )+ "device.volume" -> deviceActions.volume(+ level = params?.optInt("level", -1).let { if (it == null || it < 0) null else it },+ mute = if (params?.has("mute") == true) params.optBoolean("mute") else null,+ )+ "device.brightness" -> deviceActions.brightness(+ if (params?.has("brightness") == true) params.optInt("brightness") else null+ )+ "device.open_url" -> deviceActions.openUrl(+ requireNotNull(params?.optString("url")?.ifEmpty { null }) { "device.open_url requires url" }+ )+ "device.alarm" -> deviceActions.setAlarm(+ hour = params?.optInt("hour", -1) ?: -1,+ minute = params?.optInt("minute", -1) ?: -1,+ message = params?.optString("message")?.ifEmpty { null },+ )+ "device.facts" -> deviceActions.deviceInfo()+ "device.sensors" -> deviceActions.sensors(+ params?.optLong("timeout_ms", 1500L) ?: 1500L+ )++ // ---- device.* : comms (runtime permissions, graceful denial) --+ "device.contacts" -> deviceComms.searchContacts(+ query = params?.optString("query")?.ifEmpty { null },+ )+ "device.dial" -> deviceComms.dial(+ number = requireNotNull(params?.optString("number")?.ifEmpty { null }) { "device.dial requires number" },+ direct = params?.optBoolean("direct", false) ?: false,+ )+ "device.call_log" -> deviceComms.callLog()+ "device.sms.read" -> deviceComms.readSms(+ box = params?.optString("box", "inbox") ?: "inbox",+ )+ "device.sms.send" -> deviceComms.sendSms(+ number = requireNotNull(params?.optString("number")?.ifEmpty { null }) { "device.sms.send requires number" },+ body = requireNotNull(params?.optString("body")?.ifEmpty { null }) { "device.sms.send requires body" },+ )+ "device.calendar" -> deviceComms.calendarEvents()+ "device.calendar.add" -> deviceComms.addCalendarEvent(+ title = requireNotNull(params?.optString("title")?.ifEmpty { null }) { "device.calendar.add requires title" },+ beginMs = params?.optLong("begin_ms", 0L) ?: 0L,+ endMs = params?.optLong("end_ms", 0L).let { if (it == null || it == 0L) null else it },+ location = params?.optString("location")?.ifEmpty { null },+ description = params?.optString("description")?.ifEmpty { null },+ )++ // ---- device.* : the user's own credentials --------------------+ // Read-only lookups plus one write-only action. Not under+ // `screen.` even though `fill` types: reading the vault is not+ // operating the device, and `fill` does its own folding below.+ "device.vault.list" -> {+ val arr = org.json.JSONArray()+ for (e in net.kuafuai.andee.config.Vault.entries(appContext)) {+ val fields = org.json.JSONObject()+ for ((k, v) in e.masked()) fields.put(k, v)+ arr.put(+ JSONObject()+ .put("id", e.id)+ .put("label", e.label)+ .put("fields", fields)+ .put("secrets", org.json.JSONArray(e.secrets()))+ )+ }+ JSONObject().put("entries", arr).put("count", arr.length())+ }+ "device.vault.get" -> {+ val who = requireNotNull(params?.optString("entry")?.ifEmpty { null }) {+ "device.vault.get requires entry"+ }+ val field = params?.optString("field")?.ifEmpty { null } ?: "phone"+ val e = net.kuafuai.andee.config.Vault.find(appContext, who)+ ?: throw IllegalArgumentException(+ "no vault entry matching '$who' — call list_vault to see what the user saved"+ )+ if (field == "password" || field in e.secrets()) {+ // The whole point of the vault. Refusing here rather than+ // masking keeps the brain from pasting "****" into a form+ // and reporting success.+ throw IllegalArgumentException(+ "'$field' is a secret and is never returned. Tap the field " ++ "first, then call fill_secret — the device types it."+ )+ }+ val value = e.readable()[field]+ ?: throw IllegalArgumentException(+ "entry '${e.label}' has no '$field' — it has ${e.readable().keys}"+ )+ JSONObject().put("entry", e.label).put("field", field).put("value", value)+ }+ "device.vault.fill" -> {+ ensureCompact()+ window.pulseTool()+ val who = requireNotNull(params?.optString("entry")?.ifEmpty { null }) {+ "device.vault.fill requires entry"+ }+ val field = params?.optString("field")?.ifEmpty { null } ?: "password"+ val e = net.kuafuai.andee.config.Vault.find(appContext, who)+ ?: throw IllegalArgumentException(+ "no vault entry matching '$who' — call list_vault to see what the user saved"+ )+ // Which secret, asked of the entry rather than decided here:+ // the set of names lives in `Entry.secrets`, and a second copy+ // of it in this branch is one that drifts the day a field is+ // added. A non-secret name (phone, note) lands in the same+ // error as an empty one, and the message names both ways out.+ val value = e.secretValue(field)+ ?: throw IllegalArgumentException(+ "entry '${e.label}' has no secret '$field'. It has ${e.secrets()}; " ++ "read the rest with get_vault, or ask the user to add it in 设置 → 保险箱"+ )+ // The device is typing the user's own secret for them, so say+ // so. `pulseTool()` above is the live tell while the card is+ // folded; the subtitle is the record, readable the moment the+ // user unfolds. Not a Toast: toasts are gated on the+ // notification appop, which is off for a service with no+ // launcher icon, and a silently dropped promise is worse than+ // a quiet one.+ window.setSubtitle(+ AppLocale.str(+ appContext, R.string.cmd_vault_filling, e.label, secretLabel(field),+ )+ )+ screen.typeText(value, id = null, secret = true)+ }++ "device.notifications" -> {+ val since = params?.optLong("since_ms", 0L) ?: 0L+ val limit = params?.optInt("limit", 20) ?: 20+ net.kuafuai.andee.device.NotificationRelayService.status(appContext)+ .put("items", net.kuafuai.andee.device.NotificationRelayService.Store.snapshot(since, limit))+ }++ // ---- device.* : permissions concierge + camera eye -------------+ "device.permissions" -> permissions.status()+ "device.permissions.request" -> permissions.request(+ ids = requireNotNull(params?.optJSONArray("ids")) { "device.permissions.request requires ids[]" },+ confirm = params?.optBoolean("confirm", false) ?: false,+ )+ "device.scan" -> {+ // The two exceptions to "device.* never folds" above: both put a+ // camera preview on screen, and our overlay sits above every+ // activity. Expanded, the user aims the tablet at a QR code and+ // sees our face — the one thing they need to look at is the one+ // thing covered.+ //+ // Not gated on [driving], unlike the `screen.` fold. That gate+ // exists so a viewer polling reads on a timer can't rearrange+ // what the user is looking at; these two start a fullscreen+ // activity, which is already far past rearranging. Whoever asked,+ // a camera nobody can see is not a camera.+ ensureCompact()+ val timeoutMs = params?.optLong("timeout_ms", 30_000L) ?: 30_000L+ net.kuafuai.andee.device.ScanActivity.LastResult = null+ val i = android.content.Intent(appContext, net.kuafuai.andee.device.ScanActivity::class.java)+ .addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)+ .putExtra(net.kuafuai.andee.device.ScanActivity.EXTRA_AUTO_CLOSE_MS, timeoutMs)+ runCatching { appContext.startActivity(i) }+ // Bounded wait for the scan result (user aims the camera).+ val deadline = System.currentTimeMillis() + timeoutMs + 2000+ while (System.currentTimeMillis() < deadline) {+ val r = net.kuafuai.andee.device.ScanActivity.LastResult+ if (r != null) {+ val out = org.json.JSONObject()+ r.getString("value")?.let { out.put("value", it) }+ r.getString("format")?.let { out.put("format", it) }+ r.getString("error")?.let { out.put("error", it) }+ return out+ }+ Thread.sleep(250)+ }+ org.json.JSONObject().put("error", "scan timed out")+ }++ // ---- device.* : the live eye -----------------------------------+ "device.look" -> {+ // See device.scan: the preview is the point of the command.+ ensureCompact()+ val front = params?.optBoolean("front", false) ?: false+ val sinceUserClosed = android.os.SystemClock.uptimeMillis() -+ net.kuafuai.andee.device.LookActivity.userClosedAt+ if (!net.kuafuai.andee.device.lookSessionOpen() &&+ sinceUserClosed < USER_CLOSED_GRACE_MS+ ) {+ // Do NOT silently reopen. The user just shut the lens; the+ // old reply here ("camera not ready yet — retry") was both+ // untrue and an instruction to push the camera straight+ // back into their face. Same rule the meeting recorder+ // follows: a stop reason reported to the brain has to be+ // the real one.+ org.json.JSONObject().put(+ "error",+ "The user just switched the camera off. Do not reopen it straight away — " ++ "ask them first, and only call camera_turn once they agree."+ )+ } else {+ // Open (or re-point) the session, then hand back one frame.+ if (!net.kuafuai.andee.device.lookSessionOpen() ||+ net.kuafuai.andee.device.LookActivity.usingFront != front+ ) {+ net.kuafuai.andee.device.LookActivity.open(appContext, front)+ }+ // The first frame takes a moment to arrive; bounded retry.+ var b64: String? = null+ val deadline = System.currentTimeMillis() + 4000+ while (System.currentTimeMillis() < deadline) {+ b64 = net.kuafuai.andee.device.LookActivity.sampleFrame()+ if (b64 != null) break+ Thread.sleep(200)+ }+ if (b64 == null) {+ org.json.JSONObject().put("error", "camera not ready yet — retry look in a second")+ } else {+ org.json.JSONObject()+ .put("png_base64", b64)+ .put("camera", if (front) "front" else "back")+ .put("note", "session stays open; call look again to sample more, look_stop to close")+ }+ }+ }+ "device.look_stop" -> {+ net.kuafuai.andee.device.LookActivity.close()+ org.json.JSONObject().put("closed", true)+ }++ // ---- meeting.* : the long microphone ----------------------------+ // Not `screen.*`, deliberately: recording is not operating the+ // device, and the ball has to stay where the user can reach it —+ // tapping it is the only way they can end a meeting, since the+ // microphone is held and they cannot ask us to.+ "meeting.start" -> meeting.start(params?.optString("title")?.ifEmpty { null })+ "meeting.stop" -> meeting.stop()+ "meeting.status" -> meeting.status()++ // ---- dog.* : the little robot dog on the USB cable ---------------+ // Outside `screen.` for the same reason meeting.* is, and one more:+ // the dog is not this device. Driving it says nothing about what the+ // user is looking at on the tablet, so it must not fold the card.+ //+ // None of these block for the length of the movement they start.+ // That is not a style choice — BodyWsClient serves requests on a+ // single thread, so a `dog_move` that waited out its own duration+ // would also be holding back the `dog_stop` that interrupts it. See+ // DogMotion's class note; the stop is a timer's job.+ "dog.move" -> dog.move(+ motion = requireNotNull(params?.optString("motion")?.ifEmpty { null }) {+ "dog_move requires motion (forward|back|left|right)"+ },+ seconds = params?.optDouble("seconds", Double.NaN) ?: Double.NaN,+ gait = params?.optBoolean("gait", false) ?: false,+ )+ "dog.sequence" -> dog.sequence(+ requireNotNull(params?.optJSONArray("steps")) {+ "dog_sequence requires steps[] — each {motion, seconds, gait?}"+ }+ )+ // Closed-loop, but still not blocking: the gyroscope is read on a+ // motion thread and this returns as soon as the turn is under way.+ // The measured result lands in dog_status.last_turn.+ "dog.turn" -> dog.turn(+ motion = requireNotNull(params?.optString("motion")?.ifEmpty { null }) {+ "dog_turn requires motion (left|right)"+ },+ degrees = params?.optDouble("degrees", Double.NaN) ?: Double.NaN,+ gait = params?.optBoolean("gait", true) ?: true,+ )+ "dog.stop" -> dog.stop()+ "dog.status" -> dog.status()+ // Settings, not motions: they change how the *next* move behaves and+ // deliberately do not interrupt one in progress.+ "dog.speed" -> dog.speed(+ requireNotNull(params?.optString("level")?.ifEmpty { null }) {+ "dog_speed requires level (slow|normal|fast|turbo)"+ }+ )+ "dog.pose" -> dog.pose(+ requireNotNull(params?.optString("stance")?.ifEmpty { null }) {+ "dog_pose requires stance (high|normal)"+ }+ )+ // The one dog call that is safe to make mid-motion: one frame out,+ // one ack back, no change to what the dog is doing.+ "dog.sense" -> dog.poll()+ // Touches no radio at all — it only re-reads the tablet's own+ // sensors, so it cannot disturb whatever the dog is holding.+ "dog.calibrate" -> dog.calibrate()++ // ---- tts.* : speak without putting anything on the screen ------+ //+ // Deliberately **not** in [ToolSchemas], so no brain can reach it.+ // A turn already speaks through its final answer, and a second way+ // for the model to make the ball talk is a way for it to talk over+ // itself. This is the authoring/demo port's tool: it is what+ // `tools/body.py tts.speak '{"text":"…"}'` calls when a fixed line+ // has to be heard on cue with the screen left alone.+ //+ // The mute rules are [sayAloud]'s, not a second set: a stopped+ // turn, a meeting and a phone call all still refuse. When they do,+ // this says so instead of reporting a success nobody can hear.+ "tts.speak" -> {+ val text = requireNotNull(params?.optString("text")?.ifEmpty { null }) { "tts.speak requires text" }+ val spoken = sayAloud(text)+ JSONObject()+ .put("spoken", spoken)+ .put("chars", text.length)+ .apply {+ if (!spoken) {+ put(+ "note",+ "The words were refused by the device's mute rules: " ++ "the turn was stopped, a meeting is recording, or a call is in progress.",+ )+ }+ }+ }++ // ---- brain-facing UI: cards + full-screen HTML ------------------+ "ui.ask" -> {+ val question = requireNotNull(params?.optString("question")?.ifEmpty { null }) { "ui.ask requires question" }+ val buttons = run {+ // Localised defaults only when the brain sent none. A+ // caller-supplied list is passed through untouched — those+ // are the model's words, not ours.+ val arr = params?.optJSONArray("buttons")+ ?: org.json.JSONArray()+ .put(AppLocale.str(appContext, R.string.cmd_ask_yes))+ .put(AppLocale.str(appContext, R.string.cmd_ask_no))+ List(arr.length()) { arr.optString(it) }+ }+ val timeoutMs = params?.optLong("timeout_ms", 30_000L) ?: 30_000L+ // CardUi callbacks fire on the main thread; we are on a WS+ // worker — bridge through a latch.+ val latch = java.util.concurrent.CountDownLatch(1)+ var answer: net.kuafuai.andee.ui.CardUi.Result? = null+ net.kuafuai.andee.ui.CardUi.ask(question, buttons, timeoutMs) { r ->+ answer = r; latch.countDown()+ }+ // The card is up in the user's hands, but they may be looking+ // anywhere else — the ball is a voice-first device, so the+ // question is spoken as well as shown. The service applies the+ // same mute rules it applies to the final answer.+ sayAloud(question)+ latch.await(timeoutMs + 3000, java.util.concurrent.TimeUnit.MILLISECONDS)+ val r = answer ?: net.kuafuai.andee.ui.CardUi.Result(null, "no_answer")+ org.json.JSONObject()+ .put("button", r.button)+ .put("how", r.how)+ .put("answered", r.button != null)+ }+ "ui.alert" -> {+ val text = requireNotNull(params?.optString("text")?.ifEmpty { null }) { "ui.alert requires text" }+ val timeoutMs = params?.optLong("timeout_ms", 30_000L) ?: 30_000L+ val latch = java.util.concurrent.CountDownLatch(1)+ var answer: net.kuafuai.andee.ui.CardUi.Result? = null+ net.kuafuai.andee.ui.CardUi.alert(text, timeoutMs) { r ->+ answer = r; latch.countDown()+ }+ // Same voice-first reasoning as ui.ask: a notice nobody hears+ // is a notice that was never delivered.+ sayAloud(text)+ latch.await(timeoutMs + 3000, java.util.concurrent.TimeUnit.MILLISECONDS)+ val r = answer ?: net.kuafuai.andee.ui.CardUi.Result(null, "no_answer")+ org.json.JSONObject()+ .put("how", r.how)+ .put("read", true)+ }+ "ui.show_html" -> {+ val html = requireNotNull(params?.optString("html")?.ifEmpty { null }) { "ui.show_html requires html" }+ // Reuses `html_default_title`, the same resource+ // [net.kuafuai.andee.ui.HtmlActivity] shows — one title, one+ // key, so the page and its scrollback entry cannot drift apart.+ val defaultTitle = AppLocale.str(appContext, R.string.html_default_title)+ val title = params?.optString("title")?.ifEmpty { defaultTitle } ?: defaultTitle+ if (net.kuafuai.andee.ui.HtmlActivity.pageUp) {+ // **Returns at once, and this is the reason it has to.**+ // This call used to wait on a latch for the user to close+ // the page, up to five minutes, and that wait was the whole+ // problem: [net.kuafuai.andee.brain.LocalBrain] runs a turn on+ // one thread and runs tools synchronously on it, so a parked+ // `show_html` parked the conversation — the user's next+ // sentence was accepted and then sat in the queue until the+ // page went away. Refusing the second page instead of+ // stacking it is the other half of the same constraint:+ // `HtmlActivity` keeps ONE page (see its `pageUp`), so two+ // in flight would mean the shared static file losing to+ // whichever activity read it last.+ org.json.JSONObject()+ .put("error", "There is still a page open on screen (${net.kuafuai.andee.ui.HtmlActivity.titleText}). " ++ "Ask the user to dismiss it first, or fold the new content into what they are already looking at.")+ } else {+ // The page is the whole point of the call, and our own card+ // is a fullscreen opaque overlay that sits above every+ // activity — so without this the brain renders a UI nobody+ // can see. Same fold as `screen.*` gets, for the same+ // reason, but it can't ride the prefix: `ui.*` is where the+ // cards live and a card must not fold the thing it is drawn+ // on.+ ensureCompact()+ // Folded is not out of the way — a folded ball is still a+ // thumb-sized opaque disc, and it sits wherever it last+ // was, which is as likely as not the middle of the page.+ // Hang it off the bottom edge for as long as the page is+ // up; [onPageClosed] puts it back.+ window.setPagePerched(true)+ // Archived first, so the bytes on screen and the bytes in+ // the scrollback are the same file. Falls back to a scratch+ // copy in the cache dir if the archive can't be written — a+ // page that shows but isn't re-openable beats no page.+ val archived = net.kuafuai.andee.ui.ChatHistory.addPage(title, html)+ val openedAt = android.os.SystemClock.elapsedRealtime()+ val opened = if (archived != null) {+ net.kuafuai.andee.ui.HtmlActivity.showFile(appContext, archived, title) {+ pageClosed(title, openedAt)+ }+ archived+ } else {+ net.kuafuai.andee.ui.HtmlActivity.show(appContext, html, title) {+ pageClosed(title, openedAt)+ }+ }+ if (opened == null) {+ window.setPagePerched(false)+ org.json.JSONObject().put("error", "failed to stage the html")+ } else {+ // `closed: false` is not a failure — it is the state of+ // a page that is up right now, and it is what the old+ // `closed` flag meant at this instant anyway. There is+ // no wait left to make it true.+ org.json.JSONObject()+ .put("shown", true)+ .put("closed", false)+ .put("title", title)+ .put(+ "note",+ "The page is open on screen now; this step ends here — do not wait for the user " ++ "to close it, and do not restate it. I will tell you when they close it.",+ )+ }+ }+ }++ "screen.tap" -> {+ val x = params?.optInt("x", Int.MIN_VALUE) ?: Int.MIN_VALUE+ val y = params?.optInt("y", Int.MIN_VALUE) ?: Int.MIN_VALUE+ require(x != Int.MIN_VALUE && y != Int.MIN_VALUE) { "screen.tap requires int x,y (0-1000 relative)" }+ passthroughForGesture { screen.tapNorm(x, y) }+ }++ "screen.tap_id" -> {+ // tap_screen_element: by eid from the latest element list. The old+ // resource-id form is gone — WeChat nodes mostly have none,+ // and e-numbers cover every node the list shows.+ val eid = params?.optString("eid").orEmpty()+ require(eid.isNotEmpty()) { "screen.tap_id requires string eid (e.g. \"e7\", from get_screen_element's element list)" }+ passthroughForGesture { screen.tapByEid(eid) }+ }++ "screen.swipe" -> {+ requireNotNull(params) { "screen.swipe requires params" }+ val x1 = params.getInt("x1");+ val y1 = params.getInt("y1")+ val x2 = params.getInt("x2");+ val y2 = params.getInt("y2")+ val dur = params.optLong("duration_ms", 300L)+ passthroughForGesture { screen.swipeNorm(x1, y1, x2, y2, dur) }+ }++ "screen.long_press" -> {+ val x = params?.optInt("x", Int.MIN_VALUE) ?: Int.MIN_VALUE+ val y = params?.optInt("y", Int.MIN_VALUE) ?: Int.MIN_VALUE+ val dur = params?.optLong("duration_ms", 600L) ?: 600L+ require(x != Int.MIN_VALUE && y != Int.MIN_VALUE) { "screen.long_press requires int x,y (0-1000 relative)" }+ passthroughForGesture { screen.longPressNorm(x, y, dur) }+ }++ "screen.long_press_id" -> {+ val eid = params?.optString("eid").orEmpty()+ require(eid.isNotEmpty()) { "screen.long_press_id requires string eid" }+ passthroughForGesture { screen.longPressByEid(eid, 600L) }+ }++ "screen.type" -> {+ val text = params?.optString("text").orEmpty()+ val id = params?.optString("id")?.ifEmpty { null }+ screen.typeText(text, id)+ }++ "screen.submit_input" -> {+ val id = params?.optString("id")?.ifEmpty { null }+ screen.submitInput(id)+ }++ "screen.screenshot" -> blankedForCapture { screen.screenshot() }++ "screen.look_region" -> blankedForCapture {+ requireNotNull(params) { "screen.look_region requires params" }+ screen.lookRegion(+ params.optInt("x", Int.MIN_VALUE),+ params.optInt("y", Int.MIN_VALUE),+ params.optInt("w", 0),+ params.optInt("h", 0),+ )+ }++ "screen.ui_tree" -> screen.dumpUiTree(+ verbose = params?.optBoolean("verbose", false) ?: false,+ withShot = params?.optBoolean("with_shot", ToolSchemas.UI_TREE_WITH_SHOT_DEFAULT)+ ?: ToolSchemas.UI_TREE_WITH_SHOT_DEFAULT,+ // The dump's own screenshot has never been blanked — it called+ // screenshot() directly, so every with_shot image has shipped+ // with our ball and toolbar in it. Harmless while the shot was+ // a bonus next to a full element list; not harmless now that+ // the only shots left are the ones on self-drawn panes, where+ // pixels are all the brain has. ScreenController stays unaware+ // of the overlay — it just calls what it is handed.+ capture = { reason -> blankedForCapture { screen.screenshot(reason) } },+ )++ // Blocking the worker here is the point — the brain asked to wait,+ // so a slow answer *is* the answer (unlike a timed side effect,+ // which must arm a timer and return). It is bounded instead:+ // body_hub gives a request 30s, and the dump plus screenshot that+ // follow need a couple of those seconds.+ "screen.wait" -> {+ val asked = params?.optDouble("seconds", Double.NaN) ?: Double.NaN+ require(!asked.isNaN() && asked > 0) { "screen.wait requires positive seconds" }+ val waitMs = (asked * 1000).toLong().coerceAtMost(WAIT_MAX_MS)+ Thread.sleep(waitMs)+ screen.dumpUiTree(+ verbose = false,+ withShot = true,+ capture = { reason -> blankedForCapture { screen.screenshot(reason) } },+ ).put("waited_ms", waitMs)+ }++ "screen.global" -> {+ val action = params?.optString("action").orEmpty()+ require(action.isNotEmpty()) { "screen.global requires string action" }+ screen.globalAction(action)+ }++ // ---- note.* : the notebook -------------------------------------+ //+ // The only tools here that keep state on this device, and the only+ // ones a silent sweep turn may call. They are marked `localOnly` in+ // [ToolSchemas] and filtered out of the hub register payload: what+ // this person likes and what they were promised does not leave the+ // tablet. Nothing below touches the screen, so none of it folds the+ // card.+ "note.remember" -> {+ val name = requireNotNull(params?.optString("name")?.trim()?.ifEmpty { null }) {+ "note.remember requires name"+ }+ val desc = requireNotNull(params?.optString("description")?.trim()?.ifEmpty { null }) {+ "note.remember requires description"+ }+ val type = requireNotNull(params?.optString("type")?.trim()?.ifEmpty { null }) {+ "note.remember requires type"+ }+ val (saved, overwrote) = Notebook.remember(+ context = appContext,+ name = name,+ description = desc,+ content = params?.optString("content").orEmpty(),+ type = type,+ // A silent turn is the brain drawing its own conclusions+ // from what it overheard; a live turn is the user saying it.+ // Nothing reads this yet — it is written now so that a later+ // "prefer the user's own words" rule has the data.+ source = if (silent) Notebook.SOURCE_INFERRED else Notebook.SOURCE_HUMAN,+ )+ JSONObject()+ .put("saved", true)+ .put("name", saved.name)+ .put("overwrote", overwrote)+ .put("total", Notebook.memories(appContext).size)+ .put(+ "note",+ if (overwrote) {+ "The entry with that name was updated (the old version is archived), not added anew."+ } else {+ "Saved."+ },+ )+ }++ "note.recall" -> {+ val hits = Notebook.recall(+ context = appContext,+ query = params?.optString("query").orEmpty(),+ type = params?.optString("type").orEmpty(),+ limit = params?.optInt("limit", 8) ?: 8,+ )+ val arr = org.json.JSONArray()+ for (m in hits) {+ arr.put(+ JSONObject()+ .put("name", m.name)+ .put("summary", m.description)+ .put("type", m.type)+ .put("content", if (m.content == m.description) "" else m.content)+ .put("updated", Scheduler.formatLocal(m.updatedAt))+ )+ }+ JSONObject()+ .put("items", arr)+ .put("count", arr.length())+ .put(+ "note",+ if (arr.length() == 0) {+ "Nothing in the notebook matches. Do not invent something from impression — " ++ "treat it as not knowing."+ } else {+ ""+ },+ )+ }++ "note.forget" -> {+ val name = requireNotNull(params?.optString("name")?.trim()?.ifEmpty { null }) {+ "note.forget requires name"+ }+ val hit = Notebook.forget(appContext, name)+ JSONObject()+ .put("forgotten", hit)+ .put("name", name)+ .put(+ "note",+ if (hit) {+ "Deleted."+ } else {+ "There is nothing in the notebook under that name — recall first to see what it is " ++ "actually called, and do not delete something else on impression."+ },+ )+ }++ "note.todo" -> {+ val what = requireNotNull(params?.optString("what")?.trim()?.ifEmpty { null }) {+ "note.todo requires what"+ }+ val atRaw = params?.optString("at")?.trim().orEmpty()+ val cronRaw = params?.optString("cron")?.trim().orEmpty()+ require(atRaw.isNotEmpty() != cronRaw.isNotEmpty()) {+ "at (one-shot) and cron (recurring) are an either/or: fill exactly one, " ++ "not both and not neither"+ }+ val now = System.currentTimeMillis()+ val todo: Notebook.Todo+ if (cronRaw.isNotEmpty()) {+ val parsed = CronExpr.parse(cronRaw)+ val gap = parsed.minGapMinutes()+ require(gap >= CronExpr.MIN_INTERVAL_MINUTES) {+ "That interval is too tight (about once every $gap minutes). " ++ "The tightest the system allows is once every " ++ "${CronExpr.MIN_INTERVAL_MINUTES} minutes — tell the user plainly that it " ++ "cannot be done, and do not agree to it in different words."+ }+ val next = CronExpr.nextAfter(parsed, now)+ ?: throw IllegalArgumentException(+ "That cron expression can never match (check the day / month / weekday " ++ "fields): $cronRaw",+ )+ todo = Notebook.Todo(+ id = Notebook.newTodoId(),+ what = what,+ mode = "cron",+ at = 0L,+ cronExpr = cronRaw,+ nextRunAt = next,+ lastRunAt = 0L,+ lastStatus = "",+ lastError = "",+ status = "active",+ createdAt = now,+ )+ } else {+ val at = Scheduler.parseLocal(atRaw)+ ?: throw IllegalArgumentException(+ "at must be written as \"YYYY-MM-DD HH:mm\" (local time); received: $atRaw",+ )+ require(at > now) {+ "That moment has already passed ($atRaw). It is already " ++ "${Scheduler.formatLocal(now)}; give a time in the future."+ }+ todo = Notebook.Todo(+ id = Notebook.newTodoId(),+ what = what,+ mode = "once",+ at = at,+ cronExpr = "",+ nextRunAt = at,+ lastRunAt = 0L,+ lastStatus = "",+ lastError = "",+ status = "active",+ createdAt = now,+ )+ }+ Notebook.addTodo(appContext, todo)+ Scheduler.armTodo(appContext, todo)+ JSONObject()+ .put("id", todo.id)+ .put("mode", todo.mode)+ .put("next_run_local", Scheduler.formatLocal(todo.nextRunAt))+ .put("scheduled", if (Scheduler.exactAllowed(appContext)) "exact" else "inexact")+ .put(+ "note",+ "Set. When it comes due and is dealt with, close it with " ++ "todos(action=done, id=…) — otherwise it stays in the " ++ "\"promised but not done\" list forever." ++ if (Scheduler.exactAllowed(appContext)) {+ ""+ } else {+ " The precise-alarm permission was not granted, so it may land a few " ++ "minutes off — do not tell the user it is exactly on time."+ },+ )+ }++ "note.todos" -> {+ val action = params?.optString("action")?.trim()?.lowercase().orEmpty()+ require(action in listOf("list", "done", "drop", "pause", "resume")) {+ "note.todos requires action=list|done|drop|pause|resume"+ }+ if (action == "list") {+ val filter = params?.optString("filter")?.ifEmpty { "pending" } ?: "pending"+ val rows = Notebook.todos(appContext, filter)+ // Soonest first; rows with no next run (an occurrence that+ // fired and was never closed) sort last on purpose — they+ // are the outstanding ones, not the urgent ones.+ .sortedBy { if (it.nextRunAt > 0) it.nextRunAt else Long.MAX_VALUE }+ val arr = org.json.JSONArray()+ for (t in rows) {+ arr.put(+ JSONObject()+ .put("id", t.id)+ .put("what", t.what)+ .put("mode", t.mode)+ .put(+ "when",+ if (t.mode == "cron") t.cronExpr else Scheduler.formatLocal(t.at),+ )+ .put(+ "next_run_local",+ if (t.nextRunAt > 0) Scheduler.formatLocal(t.nextRunAt) else "",+ )+ .put("status", t.status)+ .put("last_status", t.lastStatus)+ )+ }+ JSONObject().put("items", arr).put("count", arr.length())+ } else {+ val id = requireNotNull(params?.optString("id")?.trim()?.ifEmpty { null }) {+ "note.todos action=$action requires an id (take it from the list result; " ++ "do not make one up)"+ }+ val existing = Notebook.todoById(appContext, id)+ ?: throw IllegalArgumentException("No such id: $id")++ when (action) {+ "pause" -> {+ Notebook.patchTodo(appContext, id, status = "paused")+ Scheduler.cancelTodo(appContext, id)+ JSONObject().put("id", id).put("status", "paused")+ }+ "resume" -> {+ val now = System.currentTimeMillis()+ val next = when {+ existing.mode == "cron" && existing.cronExpr.isNotBlank() ->+ CronExpr.nextAfter(existing.cronExpr, now)+ existing.at > now -> existing.at+ else -> null+ } ?: throw IllegalArgumentException(+ "The time this one was due has already passed " ++ "(${Scheduler.formatLocal(existing.at)}); it cannot be resumed — " ++ "use todo to set a new one.",+ )+ val patched = Notebook.patchTodo(+ appContext, id, status = "active", nextRunAt = next,+ )!!+ Scheduler.armTodo(appContext, patched)+ JSONObject()+ .put("id", id)+ .put("status", "active")+ .put("next_run_local", Scheduler.formatLocal(next))+ }+ else -> {+ val status = if (action == "done") "done" else "dropped"+ Notebook.patchTodo(+ appContext, id,+ status = status,+ // A closed todo must stop having a next run, or a+ // later `list` reads as if it were still coming.+ nextRunAt = 0L,+ )+ Scheduler.cancelTodo(appContext, id)+ JSONObject().put("id", id).put("status", status)+ }+ }+ }+ }++ // No built-in match. Hand it to the call extension (or whatever+ // else has been wired into forwardExternal). The default lambda+ // throws with the same "unknown method" line the else used to+ // produce, so a body with no peripherals reads exactly the same+ // as before this fallback existed.+ else -> forwardExternal(method, params)+ }+ }++ /**+ * Fold the card to the corner ball — the assistant's default shape while+ * it drives the device: one ball bottom-right, edge glow lit, every other+ * pixel the user's.+ *+ * This was a tool the brain called (`screen.reveal`), and both reasons it+ * died are the same reason it exists now: a model that forgot to call it+ * left the user staring at our face instead of the app being operated, and+ * its deadline made the assistant decide how long the user gets to look.+ * Folded in here there is no deadline to get wrong — the user taps the+ * ball ([expand]) when they want the card back, which is also what they+ * already do to talk.+ *+ * Runs on the caller's (worker) thread and only actually waits on the+ * first call of a fold: transitioning costs a layout pass, and without+ * waiting, the gesture that triggered it lands while the card still covers+ * the screen. The tap succeeds — [passthroughForGesture] sees to that —+ * but nobody sees it happen, which is the whole point of folding. Every+ * call after that returns immediately; once folded, we stay folded.+ *+ * Keeps a running task marked running ([touchBusy]) but cannot start one:+ * folding the card is about this command, the glow is about the task, and+ * the two only usually coincide.+ */+ /**+ * The user dismissed a page. Runs on the main thread, from+ * `HtmlActivity.onDestroy`.+ *+ * Two jobs, and the second one is why the page can be non-blocking at all:+ * put the ball back (it was hung off the ledge for as long as the page was+ * up) and tell whoever the brain is, since the model is no longer sitting+ * in a tool call waiting to find out.+ */+ private fun pageClosed(title: String, openedAt: Long) {+ window.setPagePerched(false)+ val seconds = ((android.os.SystemClock.elapsedRealtime() - openedAt) / 1000L).toInt()+ runCatching { onPageClosed(title, seconds) }+ .onFailure { android.util.Log.w("Body", "page-closed notify failed", it) }+ }++ /**+ * The name of a secret in the user's language, for the fill subtitle.+ *+ * 「正在填入「招行」的密码…」 has to be true, and with a card in the same+ * entry the same call can be typing a CVV. Unknown names fall through to+ * the wire name rather than to 密码: a raw `card_expiry` on screen says+ * something is out of step, where a confident wrong word does not.+ */+ private fun secretLabel(field: String): String {+ val id = when (field) {+ "password" -> R.string.vault_field_password+ "card_number" -> R.string.vault_field_card_number+ "card_expiry" -> R.string.vault_field_card_expiry+ "card_cvv" -> R.string.vault_field_card_cvv+ else -> return field+ }+ return AppLocale.str(appContext, id)+ }++ private fun ensureCompact() {+ val wait = !compacted+ val settled = CountDownLatch(1)+ ui.post {+ touchBusy()+ if (!compacted) {+ window.setCompact(true)+ compacted = true+ // Counted down a turn later rather than inline: `setCompact`+ // queues its own layout runnable, which is therefore already+ // done by the time this one runs. Getting the pixels onto the+ // screen after that is the compositor's business, not+ // something we can wait for from here.+ ui.post { settled.countDown() }+ } else {+ settled.countDown()+ }+ }+ // Bounded, and only a hint: a busy main thread costs us the wait, not+ // the command.+ if (wait) runCatching { settled.await(COMPACT_SETTLE_MS, TimeUnit.MILLISECONDS) }+ }++ /**+ * Light the edge glow for the duration of a task, and *only* for the+ * duration of a task. The corner ball rides along: it perches half off the+ * screen edge with a busy face for exactly the same span, and steps back+ * out when this clears — which is the moment the answer starts being read+ * aloud.+ *+ * The obvious design — light it on each tool call, release it a few+ * seconds after the last one — is wrong in exactly the case the glow+ * exists for: a model thinking its way through something for thirty+ * seconds makes no tool calls, and the device sits there dark while it is+ * in fact still being driven. So there is no idle timer here. Something+ * has to say the task is over ([endTask]); short of that, nothing dims it.+ *+ * Repeated calls are cheap and are how thinking stays marked as busy —+ * see [ensureCompact].+ *+ * Main thread only.+ */+ private fun setBusy(v: Boolean) {+ if (busy == v) {+ // Still re-arm the backstop: a long tool loop is activity, and the+ // whole point is that quiet stretches don't count as finished.+ if (v) rearmBackstop()+ return+ }+ busy = v+ if (v) {+ ripple.setBreathing(true)+ window.setWorking(true)+ rearmBackstop()+ } else {+ ui.removeCallbacks(endDriving)+ ripple.setBreathing(false)+ window.setWorking(false)+ }+ }++ /** Main thread only. */+ private fun rearmBackstop() {+ ui.removeCallbacks(endDriving)+ ui.postDelayed(endDriving, BUSY_BACKSTOP_MS)+ }++ /**+ * Proof a task is alive, not proof one exists. Every `screen.*` command+ * calls this; only [beginTask] can actually light the glow.+ *+ * The distinction is the difference between the agent working and someone+ * poking the device: the top-bar tool buttons and the debug client on 9008+ * arrive through the same [dispatch] door as the brain, and neither of them+ * is a task. Treating them as one lit the edges for the full backstop with+ * nobody driving — which is precisely what the glow is supposed to mean.+ *+ * Main thread only.+ */+ private fun touchBusy() {+ if (busy) rearmBackstop()+ }++ /**+ * The brain has started working on something — thinking, narrating+ * progress, whatever precedes the first tap. Called repeatedly as the task+ * runs, and it is what keeps the glow lit across gaps between tool calls.+ */+ fun beginTask() {+ ui.post { setBusy(true) }+ }++ /**+ * A task is currently running — the ball is lit and (when folded)+ * clinging to the ledge.+ *+ * Read by the notification relay to decide whether a signboard is+ * welcome: a question card popping up mid-task yanks the window out from+ * under the perch the ball is gripping, and the user's answer would+ * interleave with the task's own screen actions. While busy, message+ * notifications land in the ring buffer silently — the brain can still+ * see them via device.notifications, and whoever sent them gets read when+ * the current job is done, not in the middle of it.+ *+ * Main thread only (like [busy] itself).+ */+ fun isTaskActive(): Boolean = busy++ /**+ * Task over because the final answer arrived. This is the one thing that+ * puts the glow out — the card stays folded either way, so whatever the+ * assistant leaves on screen stays visible.+ *+ * The ball gets a brief pleased face out of it. "The answer came back" is+ * the only success signal this side of the wire has: the brain reports+ * failures as prose, indistinguishable here from any other answer, so a+ * task that ended badly still ends with a pop. Overclaiming a little beats+ * a device that never looks pleased about anything.+ */+ fun endTask() {+ ui.post {+ setBusy(false)+ window.signalOutcome(true)+ }+ }++ /**+ * Task over because the user hit stop. Same dimming as [endTask] and+ * deliberately no face: being interrupted is neither success nor failure,+ * and the user who just pressed the button doesn't need to be told what+ * they did.+ */+ fun cancelTask() {+ ui.post { setBusy(false) }+ }++ /**+ * Make the overlay non-touchable, wait a couple of frames for the+ * WindowManager update to actually take effect (updateViewLayout is+ * posted to the UI thread), run the gesture, restore.+ *+ * The glow window goes too. It never eats touches, but Android sums the+ * opacity of *all* our overlays when deciding whether an injected gesture+ * is trustworthy, and two non-touchable overlays are over the limit on+ * their own — so a tap issued while the glow is still breathing down from+ * the previous screenshot would be dropped without a trace.+ *+ * The previous tap's marker is the same hazard with worse aim: it is+ * parked on the exact coordinate a repeated tap is about to land on.+ */+ private inline fun <T> passthroughForGesture(block: () -> T): T {+ window.setTouchable(false)+ ripple.setVisibleForGesture(false)+ marker.setVisibleForGesture(false)+ try {+ Thread.sleep(50)+ return block()+ } finally {+ ripple.setVisibleForGesture(true)+ marker.setVisibleForGesture(true)+ window.setTouchable(true)+ }+ }++ /**+ * Blank the overlay for the duration of a display capture. The overlay is+ * fullscreen and opaque, and takeScreenshot composites it like any other+ * window — without this the brain gets a picture of our own card.+ *+ * Longer settle than [passthroughForGesture]: getting this wrong yields a+ * screenshot the brain will confidently describe as an empty screen, which+ * is worse than being 30 ms slower.+ *+ * The glow is blanked inside this too — it's a separate window and would+ * otherwise be photographed as a glowing border on every frame the brain+ * ever sees. Both blanks are `ui.post`ed, so the one settle covers them.+ *+ * Note it only *hides* the glow; whether the glow is lit at all belongs to+ * [setBusy]. Lighting it here was wrong in both directions: a capture with+ * no task behind it (top bar, debug client) lit the edges for no reason,+ * and the release in `finally` put them out mid-task — after which `busy`+ * was still true, so nothing ever turned them back on.+ *+ * The tap marker is pointedly *not* blanked. It is the one piece of our own+ * UI the brain is meant to see: it is how a coordinate it guessed comes+ * back as a place on the screen.+ */+ private inline fun <T> blankedForCapture(block: () -> T): T {+ window.setVisibleForCapture(false)+ ripple.setVisibleForCapture(false)+ try {+ Thread.sleep(80)+ return block()+ } finally {+ ripple.setVisibleForCapture(true)+ window.setVisibleForCapture(true)+ }+ }++ /**+ * The user asking for their screen back, via ⤡ in the subtitle band.+ *+ * Same fold the assistant's own tool calls get, same way back, but it says+ * nothing about the glow: that one tracks the task ([beginTask]), and+ * there is no task here — the user took the screen, not something the agent+ * is driving.+ */+ fun compactForUser() {+ ui.post {+ window.setCompact(true)+ compacted = true+ }+ }++ /**+ * Bring the card back to full size. For the user tapping the corner ball:+ * they want the assistant, and what they have to say — and its reply —+ * belongs in a subtitle band that only exists at full size.+ *+ * Safe to call when the card is already full. Deliberately leaves the glow+ * alone too: unfolding is the user asking to talk, not the task being+ * over, and a device that is still operating an app should still say so.+ */+ fun expand() {+ ui.post {+ window.setCompact(false)+ compacted = false+ }+ }++ private companion object {+ /**+ * Ceiling for `screen.wait`. The hub drops a request at 30s, and the+ * dump plus screenshot afterwards are part of the same request — so+ * the cap leaves them room rather than spending the whole budget+ * waiting and timing out with nothing to show for it.+ */+ const val WAIT_MAX_MS = 20_000L++ /**+ * How long after the user closes the camera `device.look` refuses to+ * reopen it silently. Long enough to cover the brain noticing the look+ * failed and trying once more; short enough that a later, genuinely+ * new request still works without the user doing anything.+ */+ const val USER_CLOSED_GRACE_MS = 15_000L++ /** How long to let the main thread take to apply the fold. */+ const val COMPACT_SETTLE_MS = 200L++ /**+ * Not a timeout on thinking — [setBusy] has none, on purpose. This only+ * catches a brain that walks away mid-task (crash, dropped socket) so+ * the device isn't left claiming to be driven by nobody. Generous+ * exactly because anything shorter would occasionally be wrong.+ */+ const val BUSY_BACKSTOP_MS = 5 * 60 * 1000L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/net/ToolSchemas.kt b/app/src/main/java/net/kuafuai/andee/net/ToolSchemas.ktnew file mode 100644index 0000000..3b602de--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/net/ToolSchemas.kt@@ -0,0 +1,1415 @@+package net.kuafuai.andee.net++import org.json.JSONArray+import org.json.JSONObject++/**+ * JSON schemas for every screen tool this body exposes. The body sends this+ * list in its `register` message so the hub can turn each entry into a+ * PluginTool the LLM sees. Keep the descriptions in sync with the equivalent+ * body_tools/ Python files — the schema wording is what the LLM reads.+ *+ * Each entry maps to a `screen.*` method in [CommandDispatcher].+ *+ * **Not every entry leaves the device.** The five `note.*` tools+ * (remember / recall / forget / todo / todos) are the brain's private+ * notebook and are marked `localOnly`; [forHub] drops them from the register+ * payload. Their text used to be Chinese because they are behavioural rules+ * ("what is worth remembering") that lived in the same language as the system+ * prompt governing them. The whole model-facing set is English now, so they+ * are too; the header of `brain/LocalPrompt.kt` is where that decision and its+ * one risk (an English prompt pulling the answer language over) are written+ * down.+ *+ * The description set follows the generic_bridge (phone.sh) philosophy:+ * the model NEVER computes coordinates when an e-number exists — it reads+ * them off the element list; coordinates (0-1000 relative) are the+ * self-drawn-screen fallback only.+ */+object ToolSchemas {++ /**+ * Whether `get_screen_element` ships a screenshot alongside the element+ * list. False: when the list has content it already says what is tappable+ * and where, and the picture is pure token cost. The device still attaches+ * one on its own when the list comes back empty — see+ * `ScreenController.dumpUiTree`, the only case where the picture is the+ * only thing the brain has.+ *+ * Lives here because the schema, the dispatcher and the Kotlin signature+ * each used to carry their own copy, and they had already drifted apart.+ */+ const val UI_TREE_WITH_SHOT_DEFAULT = false++ fun all(): JSONArray {+ val arr = JSONArray()+ for (t in TOOLS) arr.put(t)+ return arr+ }++ /**+ * The subset that may be advertised to the hub — everything except the+ * `localOnly` notebook tools.+ *+ * This is the whole of decision C ("the notebook is not visible to the+ * cloud"), and it is a one-line filter on purpose: the register payload is+ * built in exactly one place ([net.kuafuai.andee.net.BodyWsClient.onConnected]),+ * so there is no second path to keep in sync. If you are here because a+ * tool "went missing" from the hub, that is this function working.+ */+ fun forHub(): JSONArray {+ val arr = JSONArray()+ for (t in TOOLS) if (!t.optBoolean("localOnly", false)) arr.put(t)+ return arr+ }++ private val TOOLS: List<JSONObject> = listOf(+ tool(+ name = "get_screen_element",+ method = "screen.ui_tree",+ description = """+ [DEFAULT TOOL · FIRST CHOICE] Read the current screen as an OUTLINE of+ screen elements — one line per element, indented to show what contains+ what:+ e17|LinearLayout||[500,570]+ e18*|ImageView|#avatar|[62,523]+ e19*|TextView|好友3|[181,509]+ e20|TextView|这是第3条朋友圈的正文|[493,535]+ e24*|ImageView|#comment_btn|[950,621]+ `e17` is the element's number, `*` marks clickable, then class, label,+ and the element's center in 0-1000 RELATIVE coordinates (computed by+ the system, not by you).++ INDENTATION IS MEANING, and it is the answer to "which one". The lines+ indented under e17 are one post: that avatar, that name, that body and+ that comment button belong to 好友3 and to nobody else. Whenever a+ screen repeats a shape — a feed, a chat list, search results, a+ settings page — the control you want is NOT identified by its own line+ alone. Find the labeled line that names the row you want, then take+ the control from that line's OWN indent group. Never pair a name in+ one group with a button in another.++ The label column is the text, or the description, or — when an element+ has neither — the tail of its resource id with `#` in front, like+ `#comment_btn` above. A `#` value is an internal name, not words+ printed on the screen: use it to tell controls apart, never read it+ back to the user as if it were visible text.++ Whenever the goal is "find a control and tap / type / scroll / read+ on-screen text", call this tool first, then act on what it returned.++ HARD RULES:+ 1) To tap an element from this list, use tap_screen_element with its+ e-number. NEVER re-compute coordinates from the bracket values or+ from a screenshot when an e-number exists.+ 2) An icon button with no text and no description (Moments' comment+ bubble, an avatar, a photo) is identified by WHERE IT SITS, not by+ what it says. Read its indent group: the `#comment_btn` sitting+ under the line that says 好友3 IS 好友3's comment button, and+ tapping it by e-number is right even though you cannot name the+ icon. Only when two candidates inside the SAME group are+ indistinguishable do you need to see them — then call this tool+ again with with_shot=true: each line's [x,y] is a 0-1000 relative+ center, the same relative space as the screenshot, so the icon at+ image position (x,y) is the line with the nearest bracket+ coordinates. Then still tap by e-number, never by the coordinates+ you read off.+ 3) Chinese input: tap the input field FIRST (by e-number), then the+ VERY NEXT action must be type_text. Never re-tap the same input box+ before typing; the keyboard opens by itself.+ 4) After an action, the response already tells you what the screen+ became: tap responses carry `after_shot` (image), `screen_changed`,+ and — when the tree is readable — `elements` (the POST-action+ element list with fresh e-numbers, plus `elements_hint`). Chain+ your next action straight from that; do NOT call this tool again+ just to "see the result" when the action response already carried+ it. Re-dump only when you need to scroll deeper or the response+ had no elements (self-drawn pane).+ 5) Popups first: close splash ads / membership dialogs / update+ prompts / permission requests before continuing the task.++ [EMPTY LIST] nodes_useful = 0 means nothing on this pane carries a+ name. Read `nodes_total` to tell the two very different cases apart:++ - nodes_total = 0 as well: the pane draws its own pixels (mini-program,+ WebView article, game canvas) and there is no outline at all. The+ device has already attached a screenshot to THIS response+ (shot_auto): read that one, do not call take_screenshot for the same+ screen. Prefer back / home to exit and retry via a different route+ over blind coordinate taps. Blame the pane, not the app: the same+ app's native screens (chat list, tabs, settings) dump fine.++ The exception is worth watching for, and `hint` will tell you when+ it applies: if this SAME app gave you a real element list earlier in+ the session and is now returning zero nodes, that is not a self-drawn+ pane — a self-drawn pane still reports its containers. It means our+ accessibility connection went stale, which happens to WeChat in+ particular. You cannot fix it with a tool call. Ask the user, in+ plain words, to open 设置 → 无障碍 and switch Andee (this device's+ assistant, the floating ball) off and back on; it takes seconds and+ the element list comes back. Judge whether it is worth interrupting+ them — you can still tap by coordinate meanwhile — but do not silently+ keep dumping an app that has stopped answering.+ - nodes_total > 0: the elements are there, they are merely nameless.+ The outline still groups them and still gives every centre, so use+ it — match what you see in the attached screenshot against the+ indentation and tap the `*` line sitting at that spot by e-number.+ A coordinate tap here is a downgrade, not a shortcut.++ `elements_dropped: N` means the screen carried more elements than fit+ and N of the least useful lines were cut, bottom-first. What you were+ given is complete and correctly grouped; scroll if what you want is+ not in it.++ e-numbers are only valid until the page changes — if+ tap_screen_element reports an unknown element, call this tool again+ for a fresh list.+ """.trimIndent(),+ required = emptyList(),+ props = mapOf(+ "verbose" to prop(+ "boolean",+ "Return the full nested JSON tree (debug only). The default outline carries the same structure for a third of the tokens — this is for a human reading a dump, not for you.",+ default = false,+ ),+ "with_shot" to prop(+ "boolean",+ "Include a screenshot alongside the element outline. Default false: when the outline has content it already tells you what is tappable, where, and what belongs with what, so the picture costs tokens and adds nothing. Set it true only when you must judge the screen visually — two icons in the SAME indent group that you cannot tell apart by class or id, or a layout / color / render question. When the outline comes back EMPTY the device attaches a screenshot by itself (shot_auto), so never set this to work around an empty list.",+ default = UI_TREE_WITH_SHOT_DEFAULT,+ ),+ ),+ ),+ tool(+ name = "tap_screen_element",+ method = "screen.tap_id",+ description = "[PREFERRED] Tap a screen element by its e-number from get_screen_element's element list — e.g. {\"eid\": \"e7\"}. The device resolves the number to that element's exact pixel center itself: no coordinate math, no scaling errors. Always tap e-numbers from the LATEST get_screen_element (the page may have changed since an older one; stale numbers fail loudly with 'unknown element' — that means re-dump, not guess).",+ required = listOf("eid"),+ props = mapOf(+ "eid" to prop("string", "Element number from the latest get_screen_element list, e.g. \"e7\"."),+ ),+ ),+ tool(+ name = "tap_by_coordinates",+ method = "screen.tap",+ description = "[LAST RESORT] Tap a point by RELATIVE 0-1000 coordinates ((0,0) top-left, (1000,1000) bottom-right — the device converts to real pixels for any screen size). Use ONLY when the target has no e-number: self-drawn panes (mini-program, WebView, game), images, or pure-visual content you located on a screenshot. When you did locate it on a screenshot, scale proportionally (image center = 500,500). Values outside 0-1000 are rejected. If a get_screen_element list exists for this screen, use tap_screen_element instead.",+ required = listOf("x", "y"),+ props = mapOf(+ "x" to prop("integer", "X position, 0 (left edge) to 1000 (right edge)."),+ "y" to prop("integer", "Y position, 0 (top edge) to 1000 (bottom edge)."),+ ),+ ),+ tool(+ name = "long_press_screen_element",+ method = "screen.long_press_id",+ description = "Long-press a screen element by its e-number from get_screen_element (context menus, drag handles, multi-select). Same freshness rule as tap_screen_element: latest dump only. Duration is fixed at ~600ms.",+ required = listOf("eid"),+ props = mapOf(+ "eid" to prop("string", "Element number from the latest get_screen_element, e.g. \"e12\"."),+ ),+ ),+ tool(+ name = "long_press_by_coordinates",+ method = "screen.long_press",+ description = "Hold a finger on a point for a set duration (context menus, drag handles). Coordinates are RELATIVE 0-1000 like tap_by_coordinates, not pixels. Prefer long_press_screen_element when the target has an e-number.",+ required = listOf("x", "y"),+ props = mapOf(+ "x" to prop("integer", "X position, 0 (left edge) to 1000 (right edge)."),+ "y" to prop("integer", "Y position, 0 (top edge) to 1000 (bottom edge)."),+ "duration_ms" to prop("integer", "How long to hold, in milliseconds. Default 600.", default = 600),+ ),+ ),+ tool(+ name = "swipe_by_coordinates",+ method = "screen.swipe",+ description = "Swipe from one point to another: scroll lists, page carousels, drag things. To scroll a list DOWN (see more content below), swipe from a point near the BOTTOM UP to a point near the TOP — fingers move opposite to content. All coordinates are RELATIVE 0-1000 like tap_by_coordinates. Duration controls speed: shorter = faster fling.",+ required = listOf("x1", "y1", "x2", "y2"),+ props = mapOf(+ "x1" to prop("integer", "Start X, 0 (left) to 1000 (right)."),+ "y1" to prop("integer", "Start Y, 0 (top) to 1000 (bottom)."),+ "x2" to prop("integer", "End X, 0 (left) to 1000 (right)."),+ "y2" to prop("integer", "End Y, 0 (top) to 1000 (bottom)."),+ "duration_ms" to prop("integer", "How long the swipe takes, in ms. Default 300.", default = 300),+ ),+ ),+ tool(+ name = "type_text",+ method = "screen.type",+ description = "Type text into the CURRENTLY FOCUSED input field (tap the field first — by e-number — then call this as the very next action; the keyboard opens by itself, do not re-tap the field). Existing content is REPLACED, not appended. Does not press Enter/Send — use submit_input for that.",+ required = listOf("text"),+ props = mapOf(+ "text" to prop("string", "The text to enter."),+ ),+ ),+ tool(+ name = "submit_input",+ method = "screen.submit_input",+ description = "Submit the currently focused input by firing its IME action (Enter / Send / Search / Next / Done — whichever the field was set up for). Typical uses: send a chat message after type_text, run a search. Use this instead of trying to press a keyboard Enter key — Android delivers 'Enter/Send' through this action, not a raw keystroke.",+ required = emptyList(),+ props = emptyMap(),+ ),+ tool(+ name = "wait_for_screen",+ method = "screen.wait",+ description = "Wait for the screen to finish loading, then observe it. Use this after anything that starts a load — tapping a link, submit_input, launch_app — instead of calling get_screen_element straight away and reading a half-drawn page. It returns the SAME element outline get_screen_element gives you, plus a screenshot, captured after the wait: you already have the fresh state, so do NOT follow it with get_screen_element. One call waits at most 20 seconds (the request itself times out at 30). Ask for what the page plausibly needs — 2-3s for an in-app screen, 5-10s for a slow site or a cold app start.\n\nSomething genuinely slow — a cold app start, an upload, a video that has to transcode — can need more than one wait, and that is fine. But each repeat MUST ASK FOR MORE TIME THAN THE LAST ONE: climb 3 → 5 → 10 → 20, and never send the same number twice in a row. Two reasons, and the second one is fatal: what was not ready after 5s deserves better than another 5s, and a run of calls with byte-identical arguments trips the loop guard, which kills the WHOLE TASK — not just the wait. Sending seconds=10 over and over is the most common way to lose a long job.\n\nEvery wait has to earn its place: read what came back before deciding to wait again. If the outline changed at all, the page is alive — carry on, don't wait more. If you have already climbed to 20s and the screen is still identical, it is not loading, it is stuck or blocked on something (a dialog, a login, no network) — report that or change route. Never wait just to pass time.",+ required = listOf("seconds"),+ props = mapOf(+ "seconds" to prop("number", "How long to wait before looking, in seconds. Values above 20 are reduced to 20; the result reports the time actually waited. On a repeat wait for the same screen this number must be larger than the one you just used."),+ ),+ ),+ tool(+ name = "zoom_screen_region",+ method = "screen.look_region",+ description = """+ [PRECISION AIMING · SCREEN, NOT CAMERA] Despite the method name,+ this is a magnified capture of a screen REGION and has nothing to+ do with the physical camera (camera_turn). A magnified look at ONE region of the screen —+ 4x the pixel density of a full screenshot around wherever you point.++ Use it BEFORE tapping a small target you located on a full+ screenshot: icon buttons (`...` comment bubbles, toolbars, close ✕),+ small chips, dense rows. On a full shot a 25px icon makes your aim+ ±30 real pixels — half the button's hit area, coin-flip taps. On+ this zoomed image the same read is ±3 pixels: see it clearly, aim+ once.++ Coordinates: x/y mark the region's CENTER in the usual 0-1000 space;+ w/h are the region's size (try 200x300 for one control, 400x600 for+ a card). The response includes the region's screen bounds+ (region.left/top/right/bottom) and scale_x/scale_y — a point (px,py)+ read off the image maps to screen-relative+ (region.left + px/scale_x, region.top + py/scale_y).++ After a tap that reported screen_changed=false, its miss_shot field+ is exactly this — aim your retry from that image.+ """.trimIndent(),+ required = listOf("x", "y", "w", "h"),+ props = mapOf(+ "x" to prop("integer", "Region center X, 0 (left) to 1000 (right)."),+ "y" to prop("integer", "Region center Y, 0 (top) to 1000 (bottom)."),+ "w" to prop("integer", "Region width in 0-1000 units, e.g. 200."),+ "h" to prop("integer", "Region height in 0-1000 units, e.g. 300."),+ ),+ ),+ tool(+ name = "take_screenshot",+ method = "screen.screenshot",+ description = """+ [FALLBACK · REQUIRES A REASON] Capture the current screen as a PNG+ (downscaled to 1280 on the long side — proportions preserved).++ Every screenshot now arrives WITH the current element list attached+ (`elements`, same e-numbers as get_screen_element) whenever the tree+ is readable. When it is attached: aim with tap_id(e) / type(id) from+ THAT list — system-computed and exact. Pixel-measuring the image is+ only for elements the list does not carry.++ Do NOT use this for finding controls, tapping buttons, or reading+ text — get_screen_element is cheaper and more precise. Only call it+ when:+ 1) The content is inherently an image (photo/video/chart/CAPTCHA/map)+ 2) get_screen_element returned an empty list (self-drawn pane:+ mini-program, WebView article, game canvas, Moments feed) AND you+ need a FRESH look — the empty response already carried a shot of+ that screen (shot_auto), so use this only after something changed.+ Read the shot and tap by 0-1000 RELATIVE coordinates (tap_by_coordinates),+ scaling proportionally (image center = 500,500). Prefer back /+ home over blind taps when unsure.+ 3) A human-eye-level visual judgement is required+ (color, overlap, render bug).+ 4) You must tell two elements apart by how they LOOK, and the element+ list alone cannot settle it (two unlabeled icons of the same+ class). Prefer get_screen_element with with_shot=true for this —+ it keeps the list and the picture in one response, in one screen+ state. Reach for this tool only when you already hold a current+ list and just need the pixels.++ Never use a screenshot as a "let me take a look first" exploration+ step. After tapping from a screenshot, verify the outcome (fresh+ screenshot or get_screen_element) before assuming it worked.++ A cyan ring with a small caption like "tap 500,500" or "hold 240,880"+ is NOT part of the app — it is where YOUR last tap / long-press+ actually landed, drawn by the device and kept on screen for a few+ seconds. Read it as ground truth for the 0-1000 coordinate space: it+ tells you what the number you sent turned out to mean on this+ display. Most useful after tap_screen_element (where you never knew the+ pixel location) and after a tap that seemed to do nothing — a ring+ sitting beside the button is a different problem from a ring on a+ button that did not respond.+ """.trimIndent(),+ required = emptyList(),+ props = emptyMap(),+ ),+ tool(+ name = "system_action",+ method = "screen.global",+ description = "Trigger an Android system-level navigation action. Use 'back' to dismiss the current screen (also the recommended way to escape a stuck self-drawn page), 'home' to return to the launcher, 'recents' to open the task switcher, 'notifications' to pull down the shade, 'quick_settings' to open toggles. This has nothing to do with keyboard keys — for the on-screen keyboard's Enter / Send / Search use submit_input; to type any character use type_text.",+ required = listOf("action"),+ props = mapOf(+ "action" to propEnum("Which system action to trigger.", listOf("back", "home", "recents", "notifications", "quick_settings")),+ ),+ ),+ tool(+ name = "get_device_status",+ method = "device.connectivity",+ description = "Check whether the device is online and how: returns online (true/false, validated connection), type (wifi / cellular / ethernet / none). Use before anything that needs the internet, or to answer \"are you connected?\".",+ required = emptyList(),+ props = emptyMap(),+ ),+ tool(+ name = "list_installed_apps",+ method = "device.apps",+ description = "List the launchable apps installed on the device (label + package name) — what a person means by \"what apps are on this tablet\", not the hidden system packages. Useful for \"is X installed?\", \"open something that can do Y\" (find the app, then launch it via the launcher).",+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "get_location",+ method = "device.location",+ description = "Get the device's current GPS location: lat, lng, accuracy in meters, and the fix time. Fresh fix takes a few seconds outdoors; indoors it falls back to a cached one (check age_ms). If the app lacks location permission the result says so with a hint — ask the user to grant it once, then retry. Don't call repeatedly in a loop.",+ required = emptyList(),+ props = mapOf(+ "timeout_ms" to prop("integer", "Max wait for a fresh fix. Default 8000.", default = 8000),+ ),+ ),+ tool(+ name = "get_step_count",+ method = "device.steps",+ description = "Read the hardware step counter: steps since last reboot (cumulative, not per-day — say so when reporting). Needs the 'physical activity' permission; if missing the result explains how to get it. Tablets carried in a bag count too.",+ required = emptyList(),+ props = mapOf(+ "timeout_ms" to prop("integer", "Max wait for a sensor reading. Default 3000.", default = 3000),+ ),+ ),+ tool(+ name = "get_battery",+ method = "device.battery",+ description = "Battery level and charging state: percent, charging (bool), plugged (bool). Cheap to call any time.",+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "launch_app",+ method = "device.launch",+ description = "Launch an app by package name (preferred, e.g. 'com.tencent.mm') or by display label (e.g. '微信' — exact match, use list_installed_apps to find it). Direct launch intent, no screen-tapping needed.",+ required = emptyList(),+ props = mapOf(+ "pkg" to prop("string", "Package name, e.g. 'com.tencent.mm'."),+ "label" to prop("string", "Display label, e.g. '微信'. Used only if pkg is omitted."),+ ),+ ),+ tool(+ name = "set_volume",+ method = "device.volume",+ description = "Read or set media volume (0..max). Omit level/mute to just read the current state.",+ required = emptyList(),+ props = mapOf(+ "level" to prop("integer", "Target volume 0..max. Omit to read only."),+ "mute" to prop("boolean", "true=mute, false=unmute. Omit to leave as-is."),+ ),+ ),+ tool(+ name = "set_brightness",+ method = "device.brightness",+ description = "Read or set screen brightness (1..255). Omit the value to read. Writing needs the WRITE_SETTINGS special grant — the result says so if missing.",+ required = emptyList(),+ props = mapOf(+ "brightness" to prop("integer", "1..255. Omit to read only."),+ ),+ ),+ tool(+ name = "open_url",+ method = "device.open_url",+ description = "Open a URL with the right app: http(s)→browser, tel:→dialer, geo:→map, mailto:→email. Ask the user before opening anything they didn't explicitly request.",+ required = listOf("url"),+ props = mapOf(+ "url" to prop("string", "Full URL including scheme, e.g. 'https://example.com' or 'tel:10086'."),+ ),+ ),+ tool(+ name = "set_alarm",+ method = "device.alarm",+ description = "Set a system alarm (hands off to the clock app, skip-UI). Hour/minute in 24h local time.",+ required = listOf("hour", "minute"),+ props = mapOf(+ "hour" to prop("integer", "0-23."),+ "minute" to prop("integer", "0-59."),+ "message" to prop("string", "Alarm label."),+ ),+ ),+ tool(+ name = "device_facts",+ method = "device.facts",+ description = "Hardware facts: model, brand, Android version, RAM total/available, storage total/available.",+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "read_sensors",+ method = "device.sensors",+ description = "One-shot environment read: light_lux (room brightness — 0 dark, >300 bright) and accelerometer (was the tablet just moved/picked up).",+ required = emptyList(),+ props = mapOf(+ "timeout_ms" to prop("integer", "Per-sensor wait. Default 1500.", default = 1500),+ ),+ ),+ tool(+ name = "search_contacts",+ method = "device.contacts",+ description = "Search the contact book by name (fuzzy). Returns name + phone. Omit query to list the first 20. Needs Contacts permission; the result says how to get it if missing.",+ required = emptyList(),+ props = mapOf(+ "query" to prop("string", "Name to search for. Omit to list."),+ ),+ ),+ tool(+ name = "dial",+ method = "device.dial",+ description = "Open the dialer with a number filled in (direct=false, safe default) or place the call directly (direct=true, needs Phone permission AND explicit user confirmation — never dial directly on your own initiative).",+ required = listOf("number"),+ props = mapOf(+ "number" to prop("string", "Phone number."),+ "direct" to prop("boolean", "true = call immediately (requires user's explicit go-ahead); false/omit = open dialer for the user to press call.", default = false),+ ),+ ),+ tool(+ name = "get_call_log",+ method = "device.call_log",+ description = "Recent calls: name/number, incoming/outgoing/missed, time, duration. Needs Call log permission.",+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "read_sms",+ method = "device.sms.read",+ description = "Read SMS: inbox (default) or sent box, newest first. Classic use: fetch the verification code that just arrived — step 4 of the login playbook in list_vault. Needs SMS permission.",+ required = emptyList(),+ props = mapOf(+ "box" to prop("string", "'inbox' (default) or 'sent'.", default = "inbox"),+ ),+ ),+ tool(+ name = "send_sms",+ method = "device.sms.send",+ description = "Send an SMS directly. SENSITIVE: only when the user explicitly asked to send THIS text to THIS number. Say what you're about to send before sending if there is any doubt.",+ required = listOf("number", "body"),+ props = mapOf(+ "number" to prop("string", "Recipient phone number."),+ "body" to prop("string", "Message text."),+ ),+ ),+ tool(+ name = "list_vault",+ method = "device.vault.list",+ description = """+ The user's own saved accounts — what you log in AS when a task needs+ their identity (飞书, 公司邮箱, …), and any bank card they saved for+ paying. Call this first; it is cheap and tells you whether the user has+ saved anything for the app in front of you.++ Identifiers come back MASKED (138****1111) — enough to confirm you+ have the right account, not enough to type. Use get_vault for the real+ value. Each entry lists its `secrets`: fields you can never read, only+ ask the device to type (see fill_secret).++ THE LOGIN PLAYBOOK, when a phone-number + verification code login is+ in front of you:+ 1. list_vault → find the entry (match on its label)+ 2. get_vault(entry, "phone") → the real number+ 3. tap the phone field, type_text it, tap 发送验证码+ 4. read_sms → the code that just arrived (inbox, newest first)+ 5. tap the code field, type_text the digits+ If the page wants a password instead, tap the password field and call+ fill_secret — you will never hold the password yourself.++ A CARD entry has `card_number`, `card_expiry` and `card_cvv` among its+ secrets, and a `card_last4` you CAN read. Use the last four to tell two+ cards apart and to say out loud which one you are about to use — never+ just "your card" when the user saved more than one. Fill each of the+ three into its own field, one fill_secret per field.++ If the entry the task needs is missing, say so and ask the user to add+ it in 设置 → 保险箱. Do not guess an account, and never invent one.+ """.trimIndent(),+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "get_vault",+ method = "device.vault.get",+ description = """+ One identifier out of one vault entry, unmasked, so you can type it.+ Readable fields: phone, email, username, note, card_holder, card_last4.+ `note` is the user's own hint about this account ("用手机验证码登录")+ — worth reading before you decide how to log in.++ Secrets are NOT available here and asking for one is an error. That is+ deliberate: the device types secrets so they never enter your context.+ That covers the password AND the card number, expiry and CVV. Use+ fill_secret.+ """.trimIndent(),+ required = listOf("entry"),+ props = mapOf(+ "entry" to prop("string", "Entry id or label from list_vault, e.g. '飞书'."),+ "field" to prop(+ "string",+ "phone | email | username | note | card_holder | card_last4. Default phone.",+ default = "phone",+ ),+ ),+ ),+ tool(+ name = "fill_secret",+ method = "device.vault.fill",+ description = """+ Type one saved secret into the input that ALREADY HAS FOCUS. Tap the+ box first (tap_element / tap_screen) — exactly like type_text, which+ this is a wrapper around.++ `field` picks which secret: password (default), card_number,+ card_expiry, card_cvv. One call fills one field, so a checkout form+ takes three — tap the number box, fill card_number, tap the expiry box,+ fill card_expiry, and so on. list_vault tells you which of these the+ entry actually has.++ You never see the value. The result is {performed, len} and nothing+ more, even when it fails. If it reports performed:false, take a+ screenshot rather than retrying blindly; re-sending is how a field+ ends up with the value twice.++ THIS DOES NOT WORK ON A PAYMENT SHEET. 支付宝 / 微信支付 / a banking+ app's PIN pad draw their own keypad, which is not a text field, so+ there is nothing to type into — and the payment itself is the user's to+ authorise, not yours. What this reaches is an ordinary checkout form or+ the one-time page that binds a card to an app. When you hit a payment+ sheet, stop and hand it to the user.++ The ball flashes and the transcript records 「正在填入…」, so the user+ can always tell their secret was used. Say which card you are using (by+ its last four) before you use it. Only call it when the task the user+ actually asked for needs it.+ """.trimIndent(),+ required = listOf("entry"),+ props = mapOf(+ "entry" to prop("string", "Entry id or label from list_vault, e.g. '飞书'."),+ "field" to prop(+ "string",+ "password | card_number | card_expiry | card_cvv. Default password.",+ default = "password",+ ),+ ),+ ),+ tool(+ name = "get_calendar",+ method = "device.calendar",+ description = "Calendar events from now up to days_ahead (default 3). Needs Calendar permission.",+ required = emptyList(),+ props = mapOf(+ "days_ahead" to prop("integer", "Look-ahead window in days. Default 3.", default = 3),+ ),+ ),+ tool(+ name = "add_calendar_event",+ method = "device.calendar.add",+ description = "Add an event to the primary calendar. begin_ms/end_ms are Unix milliseconds. End defaults to begin+1h.",+ required = listOf("title", "begin_ms"),+ props = mapOf(+ "title" to prop("string", "Event title."),+ "begin_ms" to prop("integer", "Start time, Unix ms."),+ "end_ms" to prop("integer", "End time, Unix ms. Default begin+1h."),+ "location" to prop("string", "Where."),+ "description" to prop("string", "Notes."),+ ),+ ),+ tool(+ name = "get_notifications",+ method = "device.notifications",+ description = "Read the notification buffer: what arrived on the device recently (pkg, title, text, when) — WeChat messages, system alerts, everything the user sees in the shade. Use since_ms to fetch only new ones. If listener_enabled is false, the result includes how to turn it on.",+ required = emptyList(),+ props = mapOf(+ "since_ms" to prop("integer", "Only notifications at/after this Unix ms. Default 0 (all buffered).", default = 0),+ "limit" to prop("integer", "Max items. Default 20.", default = 20),+ ),+ ),+ tool(+ name = "get_permissions",+ method = "device.permissions",+ description = "Which of the ball's powers are granted vs missing (location, steps, contacts, sms, calendar, notifications, ...). Call this FIRST when a tool reports 'permission denied', or when you want to offer the user a one-shot enable.",+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "request_permissions",+ method = "device.permissions.request",+ description = "Guide the user through enabling permissions. Two-step by design: first call WITHOUT confirm → you get a 'say' line (read it aloud / show it) → user agrees → call again WITH confirm=true to fire the dialogs/deep-links. Only call with confirm=true after the user said yes.",+ required = listOf("ids"),+ props = mapOf(+ "ids" to prop("array", "Permission ids from get_permissions.missing[].id, e.g. [\"location\",\"sms\"]."),+ "confirm" to prop("boolean", "false (default) = get the script; true = actually fire the request.", default = false),+ ),+ ),+ tool(+ name = "scan_code",+ method = "device.scan",+ description = "Open the camera and scan a QR/barcode: returns {value, format}. The user aims the device at the code; blocks until scanned or timed out (default 30s). Camera permission is requested on the spot if missing.",+ required = emptyList(),+ props = mapOf(+ "timeout_ms" to prop("integer", "Max wait for the user to aim & the scanner to lock on. Default 30000.", default = 30000),+ ),+ ),+ tool(+ name = "camera_turn",+ method = "device.look",+ description = """+ SEE the physical world through the device camera: returns ONE frame+ of what the lens currently sees (an image — you actually see it, same+ as a screenshot). The session STAYS OPEN between calls.++ WHEN TO USE — you decide the cadence, fit it to the ask:+ - "这是什么/看看这个" → look once, answer. If the frame is blurry or+ the target isn't in view, TELL the user to aim ("对准一点") and+ look again.+ - "帮我盯着水桶满了没" / monitoring tasks → loop: look → judge →+ wait an interval YOU choose (30s? 2min? longer for slow changes) →+ look again. Report progress, stop when done or when the user asks.+ - Front camera (selfie cam) with front=true — check on the user,+ the room from the device's stand, etc.++ Each look returns the CURRENT frame only. While the session is open+ the user sees a viewfinder and a 正在看 line, and the corners flash+ every time you actually take a frame — they can tell when you looked.++ They can also close it themselves, with the ✕完成 pill. If a look+ comes back saying the user just closed the camera, that is a real+ answer, not a glitch: do NOT call this again to retry. Ask them first+ and wait for a yes.++ Close it with look_stop when done — don't leave the camera on+ unnoticed.+ """.trimIndent(),+ required = emptyList(),+ props = mapOf(+ "front" to prop("boolean", "true = front/selfie camera. Default back.", default = false),+ ),+ ),+ tool(+ name = "camera_stop",+ method = "device.look_stop",+ description = "Close the camera look session. Call when the task is done or the user stops asking — an open camera the user forgot about is creepy.",+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "ask_user",+ method = "ui.ask",+ description = """+ Ask the user a question on a small floating card with tappable+ buttons — blocks until they answer (or timeout). Returns which+ button they tapped. The question is also read aloud (TTS), so+ write it as one speakable sentence, no markdown or lists.++ USE WHEN the next step genuinely needs the user's choice: which of+ several options, yes/no on something irreversible, "要继续吗".+ NOT for status updates (that's alert) and not for questions you can+ answer yourself. Two or three short buttons work best; the FIRST+ button renders as the primary (green) one.+ """.trimIndent(),+ required = listOf("question"),+ props = mapOf(+ "question" to prop("string", "Short question shown on the card and read aloud. Keep under ~2 lines."),+ // No `default` here on purpose: the pair the card falls back on+ // is `cmd_ask_yes`/`cmd_ask_no`, resolved in the interface+ // language. A literal default in this schema would be a second,+ // single-language answer to the same question — and one the+ // model reads, so it could echo it back to an English user.+ "buttons" to prop("array", "Button labels, in the language the user is speaking — e.g. [\"帮我看看\",\"忽略\"]. First = primary. Omit it and the card supplies its own pair in the interface language."),+ "timeout_ms" to prop("integer", "Card auto-dismisses after this. Default 30000.", default = 30000),+ ),+ ),+ tool(+ name = "alert_user",+ method = "ui.alert",+ description = "Show the user a short notice card and read it aloud (single 知道了 control; tapping the card also dismisses). Blocks until read or timeout. For status/heads-up only — anything needing a choice is ask_user.",+ required = listOf("text"),+ props = mapOf(+ "text" to prop("string", "The notice text, short — read aloud, so plain prose."),+ "timeout_ms" to prop("integer", "Auto-dismiss. Default 30000.", default = 30000),+ ),+ ),+ tool(+ name = "show_html",+ method = "ui.show_html",+ description = """+ Render a full-screen UI YOU compose, ON THIS TABLET, right now, in+ front of the user. It is not a file you are handing them and not a+ link — the page opens on the device in their hands the moment you+ call this. Never tell the user to open it on a computer, in a+ browser, or anywhere else; there is nowhere else to open it.++ This is your canvas for rich output — dashboards, comparison+ tables, image grids, small games, interactive widgets the chat+ bubble can't hold. It comes back as soon as the page is up — the+ page stays until the user taps 完成, and you are told when they do.+ Do not wait for it and do not describe the page out loud.++ A wall of prose is NOT rich output, however long it is. The chat+ bubble holds text: a message past six lines is folded, and the user+ double-taps it to read the whole thing fullscreen. So an+ explanation, an introduction or a story belongs in your reply, not+ on a page. Reach for this when the answer has STRUCTURE (rows and+ columns, cards, steps) or is interactive; length alone is not a+ reason.++ ONE PAGE AT A TIME: while a page is up, calling this again is+ refused rather than stacked. Compose everything into the one page.++ Inline CSS and JS ONLY. The device may have no usable internet, so a+ page that fetches a font, a framework or an image over the network+ renders broken.++ HOUSE STYLE — follow it unless the user asked for something else.+ This is what the rest of the device looks like; a page that ignores+ it reads as a stray web page, not as part of Andee.++ <meta name="viewport" content="width=device-width,initial-scale=1,viewport-fit=cover">++ Colours (use these exact values, no others):+ page background #0F172A+ card / surface #1E293B+ hairline border rgba(148,163,184,.18)+ body text #F8FAFC+ secondary text #94A3B8+ accent #9DB4FF ← the ball's colour; ONE accent only++ Type: font-family:-apple-system,"PingFang SC","Noto Sans CJK SC",+ system-ui,sans-serif — name the CJK families explicitly or Chinese+ falls back to a default glyph set and the page instantly looks cheap.+ Sizes 28 / 20 / 16 / 13 px, line-height 1.6.++ Layout: portrait, ONE column, 20px page gutters, spacing in+ multiples of 8px, card radius 14px. Use the hairline border to+ separate cards — no heavy box-shadows, no gradients, no glow.+ Large touch targets (44px minimum); type readable without zooming.++ A ✕完成 pill floats over the top-right corner so the user can always+ leave — keep roughly 110×44px clear there.+ """.trimIndent(),+ required = listOf("html"),+ props = mapOf(+ "html" to prop("string", "Complete HTML document."),+ // Same reason as `buttons` above: the fallback is+ // `html_default_title`, resolved in the interface language, and+ // `default = "AI 界面"` used to sit here as a literal.+ "title" to prop("string", "Short page title — carried into the conversation record, and quoted back to you if a page is already open. Omit it and the page is named in the interface language."),+ ),+ ),+ tool(+ name = "start_meeting",+ method = "meeting.start",+ description = """+ Hold the microphone open and transcribe everything said in the room,+ for as long as the meeting lasts. Returns immediately — recording+ continues in the background until stop_meeting or until the user+ taps the ball. The tablet goes full screen and shows the transcript+ scrolling as it is recognised, so the user can see it is working.++ THE MICROPHONE IS YOURS FOR THE WHOLE MEETING. The user cannot talk+ to you while it runs — the wake word is off and a voice turn cannot+ start. Do not expect instructions mid-meeting.++ The device also goes silent: your replies are not read aloud while a+ meeting is recording, because the speaker is an inch from the+ microphone and would record you. Say whatever the user actually+ needs to hear BEFORE you call this, and keep the reply that follows+ it short.++ When the user ends it by tapping the ball, you will simply receive a+ new message from them containing the transcript and asking for the+ minutes. You do not need to poll for it and there is no event to+ subscribe to.+ """.trimIndent(),+ required = emptyList(),+ props = mapOf(+ // The fallback is `meeting_default_title` (see `MeetingRecorder`+ // and `ScreenBodyService.meetingMinutesQuery`), resolved in the+ // interface language. It used to be spelled out here as 会议记录,+ // which is the one string above that reaches the screen.+ "title" to prop("string", "What this meeting is, e.g. '周会'. Becomes the title the minutes page is laid out under. Omit it and the meeting is named in the interface language."),+ ),+ ),+ tool(+ name = "stop_meeting",+ method = "meeting.stop",+ description = """+ End the recording and get back the VERBATIM TRANSCRIPT — raw speech,+ no speaker labels, not minutes.++ Writing the minutes is your job, and showing them is a second step:+ compose the minutes as a full HTML page and call show_html. That is+ the only way they reach the user's eyes — a chat reply is not it.+ The full transcript is always on the device at the returned path+ even when the returned text is truncated.++ `stopped_by` says who ended it: "brain" is this call, "user" is the+ ball, "focus" is something on the device taking the audio floor+ (usually a phone call). Do not tell the user they tapped anything+ unless it says "user".+ """.trimIndent(),+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "meeting_status",+ method = "meeting.status",+ description = "Is a meeting being recorded, and for how long. Cheap; returns elapsed_ms and how many characters have been transcribed so far. No transcript text.",+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "dog_move",+ method = "dog.move",+ description = """+ Drive the little robot dog: send it one motion for a set number of+ seconds. It starts right away and STOPS BY ITSELF when the time is up —+ you do not need to stop it afterwards, and the dog never keeps running+ because a call went missing. The call returns immediately (the dog+ keeps moving in the background), so this response is not the end of the+ movement. To cut it short, call dog_stop.++ The dog is a real machine standing on a real floor. Before the first+ move of a session, say what you are about to do, and stay aware that+ whatever it hits, it hits for as long as you asked.++ MOTION: forward / back / left / right, in 0-1000 units of pure+ direction — there are no distances here, only time. `gait` picks how it+ travels:+ - gait=false (DEFAULT) — wheels. Faster, steadier, use it unless the+ user asks for the legs. Rolling forward works on carpet; it is only+ the on-the-spot *turn* that scrubs sideways and slips, which is why+ dog_turn defaults the other way.+ - gait=true — the legs walk. Slower and more impressive; use it when+ the user says 走/步态/腿/用脚.++ HOW LONG: seconds, up to 30 per call. Straight-line speed is NOT+ calibrated — "往前走" is 2s, "走远一点" 4s, and if the user says it went+ too far or not far enough, adjust and say what you changed. There is no+ distance unit to offer them.++ FOR TURNS, USE dog_turn INSTEAD. It measures the angle with the tablet's+ gyroscope and stops when it gets there, which the seconds here cannot do.+ Only fall back to left/right with seconds if dog_turn says this tablet+ has no sensor — a full turn is then about 8.7s on the wheels, 12s on the+ legs, so half a turn ≈ 4.4 and a quarter ≈ 2.2.++ THE DOG HAS OBSTACLE AVOIDANCE, AND IT WILL REFUSE YOU. Its ultrasonic+ sensor blocks forward motion closer than 20cm and backs it up below+ 10cm, on its own, regardless of what you asked for. So "it did not+ move forward" is usually this, not a failure: look at what is in front+ of it (camera_turn shows you) rather than sending the command again.++ IF IT REPLIES NO-ACK, the dog never confirmed receiving the command —+ it may be out of range, switched off, or fighting the original remote.+ Do not retry silently in a loop; say so and let the user look.++ Two things you cannot see from here: battery and whether the dog is+ physically stuck. This link only sends. Ask the user — though+ dog_status.posture.vibration is weak evidence: near zero while a motion+ should be running means nothing is actually moving.+ """.trimIndent(),+ required = listOf("motion", "seconds"),+ props = mapOf(+ "motion" to propEnum(+ "Which way to move.",+ listOf("forward", "back", "left", "right"),+ ),+ "seconds" to prop("number", "How long to move, up to 30. The dog stops itself when it elapses. Turn on wheels ≈ 8.7s; on legs ≈ 12s."),+ "gait" to prop(+ "boolean",+ "false (default) = wheels, faster. true = walk on the legs, slower. Use true only when the user asks for the legs.",+ default = false,+ ),+ ),+ ),+ tool(+ name = "dog_sequence",+ method = "dog.sequence",+ description = """+ Run several dog motions back to back — a little routine. Use this+ instead of calling dog_move twice: a second dog_move REPLACES the+ first one rather than queueing behind it, so two moves sent in a row+ is one move, not two.++ Returns immediately; the routine runs in the background and ends with+ the dog stopped. dog_stop interrupts it mid-routine, so a routine is+ also safe for anything long. Progress shows up in dog_status+ (`sequence.step` / `of`), so poll that if you need to know where it is —+ nothing is pushed to you when it finishes.++ Each step is {motion, seconds, gait} with the same meanings as+ dog_move, plus motion="pause": stand still for N seconds. Use a pause+ between direction changes (0.3-0.5s) — the servos settle and the+ movement reads as deliberate rather than as a glitch. Mix gait=false+ and gait=true freely inside one routine for a routine that looks like+ a performance.++ Whole-routine ceiling is 120s, one step is 30s. A routine longer than+ that should be several calls, both so it can be interrupted and so you+ get to look at the result in between.++ Example — turn a half circle, pause, then walk forward and come back:+ [{"motion":"left","seconds":4.4},+ {"motion":"pause","seconds":0.5},+ {"motion":"forward","seconds":2},+ {"motion":"pause","seconds":0.5},+ {"motion":"back","seconds":2}]+ """.trimIndent(),+ required = listOf("steps"),+ props = mapOf(+ "steps" to prop(+ "array",+ "Steps in order; each {motion, seconds, gait?} with motion one of forward/back/left/right/pause.",+ ),+ ),+ ),+ tool(+ name = "dog_turn",+ method = "dog.turn",+ description = """+ Turn the dog a MEASURED number of degrees. Prefer this over dog_move for+ every turn — it is the one dog command that knows whether it worked.++ The tablet is bolted to the dog, so the tablet's gyroscope is the dog's:+ this drives until the sensor says the angle has been reached, then stops.+ dog_move with seconds divides a "full circle ≈ 8.7s" constant that drifts+ with battery level, floor surface and where the legs happened to be, so+ it is a guess; this is not.++ Returns IMMEDIATELY, while the dog is still turning — the reply carries+ `expected_seconds`, not a result. The measurement shows up afterwards in+ dog_status.last_turn: `turned_deg` is what it really did, and `reached`+ says whether it got there. Check it when the angle matters (lining up+ with a doorway, facing a person); skip it for a rough spin.++ WHEN IT DOES NOT REACH: `reached: false` with a small `turned_deg` means+ the dog barely rotated — the command did not land, it is out of range,+ switched off, or physically blocked. Do NOT just send it again; say what+ the sensor measured and let the user look.++ Turns are also the safe motion: unlike forward, they are not blocked by+ the dog's obstacle sensor, but they still sweep the dog's body through+ whatever is beside it.++ Turns default to the LEGS. A wheeled turn is a differential spin, so the+ wheels have to scrub sideways across the floor — on carpet they lose and+ the dog barely rotates. Pass `gait: false` only on smooth hard floor, or+ when the user asks for the wheels. Note the legs ignore dog_speed, which+ only changes wheel PWM.++ Use dog_move for forward/back. If this tablet has no gyroscope the call+ fails with an explanation and you should fall back to dog_move.+ """.trimIndent(),+ required = listOf("motion", "degrees"),+ props = mapOf(+ "motion" to propEnum("Which way to turn.", listOf("left", "right")),+ "degrees" to prop("number", "How far to turn, 1-720. 90 = a quarter, 180 = about-face, 360 = a full circle."),+ "gait" to prop(+ "boolean",+ "true (default) = turn on the legs. false = spin the wheels, which is faster on hard flat floors but slips on carpet, where the wheels have to scrub sideways. Only pass false if the floor is smooth or the user asks for the wheels.",+ default = true,+ ),+ ),+ ),+ tool(+ name = "dog_stop",+ method = "dog.stop",+ description = "Stop the dog now — an emergency brake, and the first thing to call whenever the user says 停 / 停下 / 别动了. Interrupts a dog_move that is still running and abandons a dog_sequence part-way. Safe to call when nothing is happening: the reply says honestly whether anything was moving, so do not announce a stop that did not happen. Note the dog would stop on its own at the end of whatever it was told to do; this is for ending it early.",+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "dog_status",+ method = "dog.status",+ description = """+ Is the dog reachable, and what is it doing. Cheap, no side effects —+ call it when something dog-related just failed, or before you tell the+ user anything about the dog's state.++ Connection half: `attached` (is a dongle plugged into this tablet at+ all), `permission` (`no_device` / `needed` / `granted` — `needed` means+ the Android dialog has not been answered yet and the next dog command+ will raise it), `open` (did we get the serial port), `device`, `banner`+ (what the dongle said when it started: "READY" means the dongle and its+ radio are both fine, an "ERR: NRF24L01 not responding" means the dongle+ is alive but its radio module is not — a dongle-side wiring job, do not+ blame the dog), and `error` (why the last attempt failed, in words the+ user can act on).++ Motion half: `moving`, `motion`, `remaining_ms`, `sequence` with the step+ it is on, and `turn` while a dog_turn is under way.++ `last_turn` is how a finished dog_turn reports itself — it appears only+ once the turn is over, and carries `turned_deg` (what the gyroscope+ actually measured), `overshoot_deg`, `reached`, and `stopped_by`. A+ `reached: false` with a tiny `turned_deg` is the strongest evidence this+ system can give that a command never got to the dog.++ `posture` is the tablet's own sensors, read as the dog's — they are+ bolted together. `tilt_deg` is the lean away from standing; anything past+ 50° means the dog went over (or was picked up) and the tablet has already+ stopped it, which also shows up as `aborted_by: "fallen"`. It is measured+ against a reference the tablet takes each time a motion starts from rest,+ so if the user says the dog is standing normally and this still reads+ large, the reference is stale — call dog_calibrate, do not tell the user+ their dog has fallen over. `vibration` is+ raw accelerometer energy in m/s²: high while the dog works, near zero+ when nothing is moving. It is NOT calibrated into a "stuck" verdict, so+ read it as evidence and say so — near-zero vibration during a motion you+ believe is running usually means the command did not land, or the dog's+ own obstacle sensor refused it.++ `telemetry` is the dog's own distance and battery, which reach us on the+ acknowledgement of the last command sent. It can be STALE — check+ `age_ms`, and call dog_sense to refresh it before saying anything about+ what is in front of the dog. `supported: false` means no reading has ever+ arrived (older dongle or older dog firmware); say you cannot see it+ rather than reporting a number.++ `speed` and `pose` are what was last successfully set THIS session. Null+ means never set here, which is not the same as knowing the dog is at its+ default — say so instead of guessing.+ """.trimIndent(),+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "dog_sense",+ method = "dog.sense",+ description = """+ Ask the dog what it can see and how much charge it has left, WITHOUT+ telling it to do anything. Fast and safe to call while the dog is+ walking — which is exactly when it is worth calling.++ `telemetry.distance_cm` is the dog's own forward-facing ultrasonic+ sensor. `telemetry.battery_pct` is its battery, 0-100.++ Read `telemetry.age_ms` before you believe either number. The dog only+ speaks when spoken to — the reading rides back on the acknowledgement of+ whatever was last sent — so a value from several seconds ago describes a+ world that has moved. This call refreshes it, so straight after it the+ age is small; a large age in dog_status does not mean the dog is broken.++ `telemetry.supported: false` means no reading has EVER arrived, which is+ what an older dongle or older dog firmware looks like from here. It is+ not the same as "0 cm" and not the same as "flat battery". Say you+ cannot see it rather than reporting a number.++ A null `distance_cm` with `supported: true` means the sensor answered+ nothing — usually nothing within range, sometimes an unplugged sensor.+ The dog's own firmware refuses to walk forward under about 20cm and+ backs up under 10cm on its own, so a forward command that does nothing+ while this reads small is the dog protecting itself, not a lost command.+ """.trimIndent(),+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "dog_calibrate",+ method = "dog.calibrate",+ description = """+ Tell the tablet that the dog is standing normally RIGHT NOW, and take+ that as the reference for "upright". Touches no radio and commands the+ dog to do nothing — it only re-reads the tablet's own sensors.++ The tablet is strapped to the dog's back, so it has no way to know+ which way is "standing" except by being told. Every motion takes this+ reference automatically when it starts from rest, so in normal use you+ never need this tool. Call it when that rule cannot hold:++ - the user re-seated, re-angled or re-mounted the tablet;+ - dog_status shows a `tilt_deg` the user says is wrong — a dog they+ tell you is standing normally, reading 30°;+ - a motion came back `aborted_by: "fallen"` and the dog is upright.++ A stale reference is not cosmetic: past 50° the tablet stops motions on+ its own, so a bad reference makes the dog refuse to walk for a fall+ that is not happening.++ Refuses while the dog is moving (`calibrated: false` with `moving`+ set) — a walking dog pitches with every stride and that would be baked+ into the reference. Stop it, let it settle, then calibrate.++ `was_tilt_deg` is what the tilt read just before this call, against the+ old reference. A large value is the measure of how wrong things were;+ near zero means the reference was already fine.+ """.trimIndent(),+ required = emptyList(),+ props = emptyMap<String, JSONObject>(),+ ),+ tool(+ name = "dog_speed",+ method = "dog.speed",+ description = """+ How fast the dog walks. A setting, not a movement: it does NOT start,+ stop or interrupt anything, and it does not make the dog move on its own.++ It takes effect from the NEXT motion command onwards. A dog already+ walking finishes its current move at the old speed, so to change speed+ mid-task, set it and then issue the next dog_move — do not expect the+ running one to speed up.++ Use slow when the dog is near people, near a desk edge, or being asked to+ stop at a particular spot; the obstacle sensor has less distance to work+ with the faster it goes. normal is the sensible default. turbo is for an+ open corridor with nothing in it.+ """.trimIndent(),+ required = listOf("level"),+ props = mapOf(+ "level" to propEnum(+ "Walking speed. Prefer slow around people.",+ listOf("slow", "normal", "fast", "turbo"),+ ),+ ),+ ),+ tool(+ name = "dog_pose",+ method = "dog.pose",+ description = """+ Raise or lower the dog's body. `high` stands it up tall, `normal` puts it+ back down. A setting like dog_speed — it does not move the dog anywhere.++ Raise it to get the tablet's camera up to a person's eye level, or to+ clear a doorsill or cable. Lower it for stability and when carrying the+ tablet at speed.++ One thing to know: dog_stop resets the stance to normal. That is+ deliberate — the raised stance leaves the drive motors live, so a stop+ that kept it would look stopped without being stopped. After any stop,+ set the stance again if you still want it raised.+ """.trimIndent(),+ required = listOf("stance"),+ props = mapOf(+ "stance" to propEnum(+ "Body height. high raises the tablet to eye level.",+ listOf("high", "normal"),+ ),+ ),+ ),++ // ------------------------------------------------------------------+ // The notebook — `note.*`, localOnly.+ //+ // These five are the device's own memory, not a capability of the+ // hardware, and they never leave it: `forHub()` strips them from the+ // register payload. Their text is English like every other+ // model-facing string; see the header of LocalPrompt.kt for which+ // Chinese survives inside an English prompt (a label that has to be+ // matched on screen, and a specimen of what a Chinese user says).+ // ------------------------------------------------------------------+ tool(+ localOnly = true,+ name = "remember",+ method = "note.remember",+ description = """+ Put one thing into your notebook. What you write is at hand on every later turn, and the user never has to say it twice.++ ## What belongs in it (about this person, not about this turn)++ - who they are, what they do, how to work with them → type=user+ - they explicitly asked you to change how you do something, or confirmed how you did it (「以后都这样」「别这么干」) → type=feedback+ - you personally got something done for them, walked a route through → type=way+ - an outside entrance (a website / app / account name / address / a shop they go to) → type=reference+ - a route you tried and **confirmed is a failure** → type=failure, with the "never do this again" written into content++ ## What does not belong++ - this turn's transient state (what you are waiting for, where you just tapped)+ - coordinates, pixels, which row — they all change with the interface+ - this time's verification code, order number, amount+ - anything already stated in your system prompt++ ## Fields++ - name: the overwrite key. **Writing the same name again = updating that entry** (the old version is kept), not adding one.+ So when a route improves, write it again under the same name.+ - description: one line, readable on its own — that line alone should say who / what / how to handle it.+ Do not write empty phrases like "the user's preferences"; write "doesn't drink anything cold, winter included".+ - content: the body, omittable (when omitted, description is used). For type=feedback or way, add+ `**Why:**` and `**How to apply:**` lines, so that future you can judge the boundary instead of clinging to one rule.+ - type: see above.+ """.trimIndent(),+ required = listOf("name", "description", "type"),+ props = mapOf(+ "name" to prop("string", "Short identifier, and the overwrite key. e.g. 饮食禁忌 / 咖啡偏好 / 飞书登录"),+ "description" to prop("string", "One self-contained line, ≤120 characters"),+ "content" to prop("string", "The body. When omitted, description is used."),+ "type" to propEnum(+ "Category of memory",+ listOf("user", "feedback", "way", "reference", "failure"),+ ),+ ),+ ),+ tool(+ localOnly = true,+ name = "recall",+ method = "note.recall",+ description = """+ Look in the notebook.++ **Look once before you start.** Does this look like something you have done before? What has this person said about this kind of thing?+ When something written or spoken disagrees with the notebook, go by the notebook and put the question to them; do not act on your own impression.++ An empty query = pull a batch of the more important and recently used entries ("what have I written down").+ What comes back is name and summary; to get the body, look that name up again with recall.+ """.trimIndent(),+ required = emptyList(),+ props = mapOf(+ "query" to prop("string", "Keyword. Empty = pull a batch of the important / recently used ones"),+ "type" to propEnum(+ "Only look in this category",+ listOf("user", "feedback", "way", "reference", "failure"),+ ),+ "limit" to prop("integer", "How many to return, default 8, max 30"),+ "scope" to propEnum("Where to look. Currently only memory", listOf("memory")),+ ),+ ),+ tool(+ localOnly = true,+ name = "forget",+ method = "note.forget",+ description = """+ Delete one entry from the notebook.++ **The user will not say "delete the memory" — they do not know it exists, so you have to see it yourself.**+ These are the signals that mean delete:++ - they corrected something from before: 「不对,其实是……」+ - they changed a preference or a decision: 「之前说的那个不做了」+ - old and new contradict: the notebook says they do not use 微信, and they now say 「加我微信」+ - that thing no longer exists: 「那个活动下线了」「那个需求不要了」++ `recall` first to check the name before deleting, and **do not delete from impression** — a deletion cannot be undone.+ If only the content changed (rather than the entry being void), overwrite it with remember under the same name; do not delete.+ """.trimIndent(),+ required = listOf("name"),+ props = mapOf(+ "name" to prop("string", "The name of the entry to delete; recall first to confirm"),+ ),+ ),+ tool(+ localOnly = true,+ name = "todo",+ method = "note.todo",+ description = """+ Set a "thing to do at a certain time" for yourself.++ ## The rule, first++ Whenever you intend to tell the user "I'll come and find you then / I'll remind you / I'll handle it for you then", **you must call this tool and see the success result before that sentence leaves your mouth**.+ Say it without having set it, and when that moment arrives you will not appear — that is not "might forget", that is lying.++ ## How to write `what`++ One sentence, written for future you to read: what to **do** or **say** at that time.+ "Remind him to take his medicine" is too vague; "tell him it's time for his medicine, and ask how he slept last night" is something that can be executed.++ ## When it triggers (at and cron — one or the other; not both, not neither)++ - `at`: one-off. Local time `YYYY-MM-DD HH:mm`. **Absolute times only** — for "in ten minutes", work the time out yourself and fill in the result.+ - `cron`: recurring. Standard 5 fields, "minute hour day month weekday". e.g. every day at 8 = `0 8 * * *`;+ every Monday at 9 = `0 9 * * 1`; every half hour = `0,30 * * * *`.++ 🔴 **The densest is once every 15 minutes.** Anything denser is refused — that is a system limit, not your choice.+ When it is refused, tell them honestly "the fastest is once every 15 minutes" — **do not paper over it in different words, and do not say "ok, it's set"**.++ The result carries `next_run_local` and `scheduled`: `inexact` means the time may be off by a few minutes+ (the norm when there is no exact-alarm permission). When you see `inexact`, say only "I'll speak up around that time" —+ **do not say "on the dot"**.+ """.trimIndent(),+ required = listOf("what"),+ props = mapOf(+ "what" to prop("string", "One sentence: what to do / say at that time"),+ "at" to prop("string", "One-off, local time YYYY-MM-DD HH:mm"),+ "cron" to prop("string", "Recurring, 5 fields \"minute hour day month weekday\". Densest is once every 15 minutes"),+ ),+ ),+ tool(+ localOnly = true,+ name = "todos",+ method = "note.todos",+ description = """+ See and manage the todos you have set.++ - `list`: look. By default only the ones not yet finished (pending); `filter=all` includes finished ones, `filter=cron` shows only recurring ones.+ - `done`: it is handled, close it. **Always close it once handled**, or it hangs in "promised but never done" forever.+ - `drop`: not doing it, cancel it.+ - `pause`: park a recurring todo for now.+ - `resume`: bring back a paused one. A recurring one recounts its next run from now; a one-off whose moment has already passed cannot be resumed,+ so have the user set a new one.++ When the user asks "what have you promised me" or "what is still outstanding", `list`.+ Take `id` from the result of `list` — do not make one up.+ """.trimIndent(),+ required = listOf("action"),+ props = mapOf(+ "action" to propEnum(+ "What to do",+ listOf("list", "done", "drop", "pause", "resume"),+ ),+ "id" to prop("string", "Required when action is not list; take it from list"),+ "filter" to propEnum("Used with list", listOf("pending", "all", "cron")),+ ),+ ),+ )++ /** Server-side lookup: given a hub-facing tool name, return the method+ * string CommandDispatcher expects. */+ fun methodOf(toolName: String): String? =+ TOOLS.firstOrNull { it.getString("name") == toolName }?.optString("method")?.ifEmpty { null }++ // ---- builders ----++ /**+ * @param localOnly true for the brain's private notebook tools: kept in+ * [all] (the on-device brain sees them) but stripped by [forHub]. Set+ * explicitly on each of the five rather than inferred from the `note.`+ * prefix — a tool silently escaping to the hub is not something that+ * should happen because someone renamed something.+ */+ private fun tool(+ name: String, method: String, description: String,+ required: List<String>, props: Map<String, JSONObject>,+ localOnly: Boolean = false,+ ): JSONObject {+ val propsObj = JSONObject()+ for ((k, v) in props) propsObj.put(k, v)+ val reqArr = JSONArray()+ for (r in required) reqArr.put(r)+ val o = JSONObject()+ .put("name", name)+ .put("method", method)+ .put("description", description)+ .put(+ "parameters", JSONObject()+ .put("type", "object")+ .put("properties", propsObj)+ .put("required", reqArr)+ )+ if (localOnly) o.put("localOnly", true)+ return o+ }++ private fun prop(type: String, description: String, default: Any? = null): JSONObject {+ val o = JSONObject().put("type", type).put("description", description)+ if (default != null) o.put("default", default)+ return o+ }++ private fun propEnum(description: String, values: List<String>): JSONObject {+ val arr = JSONArray()+ for (v in values) arr.put(v)+ return JSONObject()+ .put("type", "string")+ .put("description", description)+ .put("enum", arr)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/screen/GroundingLog.kt b/app/src/main/java/net/kuafuai/andee/screen/GroundingLog.ktnew file mode 100644index 0000000..9be0e09--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/screen/GroundingLog.kt@@ -0,0 +1,384 @@+package net.kuafuai.andee.screen++import android.content.Context+import android.util.Log+import net.kuafuai.andee.config.VoiceConfig+import org.json.JSONObject+import java.io.File+import java.util.concurrent.ExecutorService+import java.util.concurrent.Executors+import java.util.concurrent.ThreadFactory+import java.util.concurrent.TimeUnit+import java.util.concurrent.atomic.AtomicLong++/**+ * Append-only record of what the brain saw and where it then touched, kept so+ * the mis-taps can be mined later.+ *+ * ## Why there is anything to mine+ *+ * The creed is that the model never does coordinate arithmetic: with an+ * element list it says `tap_screen_element {"eid":"e7"}` and the device+ * resolves the pixels. Coordinates only come out when there is no list —+ * self-drawn panes — and that is precisely the ground where a general-purpose+ * model is weakest (AndroidLab: GPT-4o 31.2 vs GUI-tuned 49.5). What happens+ * there is a mis-tap, then a retry a bit to the side, then a hit. That+ * sequence is a labelled example — image, wrong point, right point — produced+ * for free by ordinary operation and, until now, thrown away.+ *+ * Note what is NOT recoverable: tap accuracy itself. [net.kuafuai.andee.ui.TapMarkerUi]'s+ * ring is drawn at the same x,y handed to the `StrokeDescription`, so there is+ * no second measurement anywhere in the loop and therefore no error term to+ * calibrate. The only error this can see is the brain's — which point it+ * chose — and that is learned from, not corrected for.+ *+ * ## Why the device does no pairing+ *+ * Everything here is a dumb append. Deciding which tap was a miss is a+ * heuristic, heuristics are wrong at first, and one baked into the device can+ * only be fixed by shipping an APK and re-collecting. `tools/grounding_pair.py`+ * makes those calls offline against the raw log, so a better rule can be+ * re-run over data already on disk.+ *+ * The strongest label is already computed on-device for another purpose and+ * simply not written down: `ScreenController.guardRepetition` refuses the 4th+ * gesture in a 120px bucket. Its scope is exactly this data set — it covers+ * `tapNorm`/`longPressNorm` and exempts e-taps — so a bucket that tripped it+ * means every coordinate tap in that bucket missed. Hence `refused`.+ *+ * ## Rules it lives under+ *+ * - **Never block a tool call.** A slow handler reads to the brain as "still+ * working", so the calling thread only builds a JSONObject in memory; the+ * append and the ~400KB PNG write go to [exec].+ * - **Off by default**, behind `grounding` in [VoiceConfig].+ * - **Bounded.** [MAX_BYTES] of PNGs, oldest deleted; JSONL rotates at+ * [MAX_JSONL_BYTES] with [JSONL_KEEP] generations.+ * - **Its own directory.** `grounding/`, never `captures/` — that one holds+ * the user's manual top-bar saves and has no cleanup, so rotating into it+ * would delete their files.+ */+object GroundingLog {++ private const val TAG = "Body"++ /** Bump when a field changes meaning; the offline script keys off it. */+ private const val SCHEMA = 1++ private const val MAX_BYTES = 512L * 1024 * 1024+ private const val MAX_JSONL_BYTES = 16L * 1024 * 1024+ private const val JSONL_KEEP = 4++ /**+ * `listFiles()` over a few thousand PNGs is slow enough to notice, so the+ * byte count is kept in memory and only re-derived from disk this often.+ */+ private const val SWEEP_EVERY = 50++ @Volatile private var appCtx: Context? = null+ @Volatile private var dir: File? = null+ @Volatile private var shotsDir: File? = null+ @Volatile private var on = false++ private val seq = AtomicLong(0)+ private val shotSeq = AtomicLong(0)+ private var writesSinceSweep = 0+ private var bytesOnDisk = -1L++ /**+ * Its own single daemon thread — deliberately not the service's+ * "ScreenTools" executor, which serialises the user's top-bar buttons: a+ * 400KB write queued ahead of a button press would be visible as lag. It+ * is also a private field of the service, which [ScreenController] has no+ * handle on.+ *+ * Nullable, and rebuilt by [init], because this `object` outlives the+ * service that started it. [shutdown] runs from `onDestroy`, and on a+ * HONOR/MagicOS device switching the accessibility service off and back+ * on destroys and reconnects it **inside one process** — so the second+ * `init` used to find a terminated pool and `execute` on it. That throws+ * `RejectedExecutionException` *on the main thread*, from+ * `onServiceConnected`: the service could not come back at all, which is+ * the worst possible failure for the one repair path the self-check+ * itself recommends ("turn it off and on again"). A writer that can be+ * rebuilt turns that crash into a `Log.w`.+ */+ @Volatile private var exec: ExecutorService? = null+ private val execLock = Any()++ /**+ * Call once from `onServiceConnected`, alongside the other+ * `object.init(this)` singletons. Reads the switch once — flipping it in+ * settings takes effect on the next service start, which is the same+ * bargain every other pref here makes.+ *+ * Also the *only* place allowed to bring the writer back after [shutdown]+ * tore it down, because only this call is guaranteed to be the start of a+ * real session. [post] and [noteShot] drop their write instead of+ * resurrecting a thread the service has already said goodbye to.+ */+ fun init(ctx: Context) {+ val app = ctx.applicationContext+ appCtx = app+ on = VoiceConfig.groundingEnabled(app)+ if (!on) return+ restartWriter()+ submit("grounding init failed") {+ val root = File(app.getExternalFilesDir(null), "grounding")+ val shots = File(root, "shots")+ shots.mkdirs()+ dir = root+ shotsDir = shots+ sweepFromDisk()+ // append, not just note: `note` stamps a record and hands it back,+ // it does not write. This line called it bare, so the session event+ // — the one record every offline pass needs to know what the screen+ // was and which way round — was built and dropped on the floor.+ // Measured on the phone before the fix: 174 events in events.jsonl+ // and not one of them `session`. [noteTapCoord] already assumes it+ // exists ("the session record is written at service start").+ append(+ note(+ JSONObject()+ .put("ev", "session")+ .put("screen_w", app.resources.displayMetrics.widthPixels)+ .put("screen_h", app.resources.displayMetrics.heightPixels)+ .put(+ "orientation",+ if (app.resources.configuration.orientation ==+ android.content.res.Configuration.ORIENTATION_LANDSCAPE+ ) "landscape" else "portrait",+ )+ )+ )+ }+ }++ /**+ * Record the image the brain is about to be shown, and hand back the id+ * that names it. Called with the PNG already encoded — re-capturing or+ * re-compressing to log would double the cost of every screenshot.+ *+ * Returns synchronously (the id is just a counter) so the live response+ * can carry `shot_id` and be matched to the file offline; the bytes are+ * written on [exec].+ */+ fun noteShot(+ png: ByteArray,+ srcW: Int,+ srcH: Int,+ outW: Int,+ outH: Int,+ reason: String,+ pkg: String?,+ markerPx: IntArray?,+ markerAgeMs: Long?,+ ): String? {+ if (!on) return null+ val id = "s_%d_%04d".format(System.currentTimeMillis(), shotSeq.incrementAndGet())+ val o = JSONObject()+ .put("ev", "shot")+ .put("shot_id", id)+ .put("file", "shots/$id.png")+ .put("src_w", srcW).put("src_h", srcH)+ .put("w", outW).put("h", outH)+ .put("bytes", png.size)+ .put("reason", reason)+ if (pkg != null) o.put("pkg", pkg)+ if (markerPx != null) {+ o.put("marker_px", org.json.JSONArray().put(markerPx[0]).put(markerPx[1]))+ if (markerAgeMs != null) o.put("marker_age_ms", markerAgeMs)+ }+ submit("grounding shot failed") {+ val d = shotsDir ?: return@submit+ File(d, "$id.png").writeBytes(png)+ bytesOnDisk = if (bytesOnDisk < 0) -1L else bytesOnDisk + png.size+ if (++writesSinceSweep >= SWEEP_EVERY || bytesOnDisk > MAX_BYTES) sweep()+ append(o)+ }+ return id+ }++ /**+ * A tap the brain aimed by coordinate — the samples this whole file+ * exists for. `refused` means the stuck-loop guard rejected it, which is+ * the strongest miss label available.+ */+ fun noteTapCoord(+ nx: Int, ny: Int, px: Int, py: Int, screenW: Int, screenH: Int,+ hold: Boolean, refused: Boolean, pkg: String?,+ ) {+ if (!on) return+ post(+ JSONObject()+ .put("ev", "tap").put("mode", "coord")+ .put("nx", nx).put("ny", ny)+ .put("px", px).put("py", py)+ // Per event, not once per session: the session record is written+ // at service start and a rotation afterwards would silently make+ // it a lie, so anything converting px↔norm offline needs the+ // size that was true for THIS tap.+ .put("screen_w", screenW).put("screen_h", screenH)+ .put("hold", hold).put("refused", refused)+ .also { if (pkg != null) it.put("pkg", pkg) }+ )+ }++ /**+ * A tap resolved from an e-number. Not a training sample — the device did+ * the geometry — but it has to be told apart from the coordinate ones, and+ * it marks the screen as changed for the offline pairing.+ */+ fun noteTapEid(+ eid: String, px: Int, py: Int, screenW: Int, screenH: Int,+ hold: Boolean, relocated: Boolean, pkg: String?,+ ) {+ if (!on) return+ post(+ JSONObject()+ .put("ev", "tap").put("mode", "eid")+ .put("eid", eid).put("px", px).put("py", py)+ .put("screen_w", screenW).put("screen_h", screenH)+ .put("hold", hold).put("relocated", relocated)+ .also { if (pkg != null) it.put("pkg", pkg) }+ )+ }++ /** Scrolling moves everything; the offline pairing must not read across one. */+ fun noteSwipe(nx1: Int, ny1: Int, nx2: Int, ny2: Int, durationMs: Long, actor: String) {+ if (!on) return+ post(+ JSONObject()+ .put("ev", "swipe")+ .put("nx1", nx1).put("ny1", ny1).put("nx2", nx2).put("ny2", ny2)+ .put("duration_ms", durationMs)+ .put("actor", actor)+ )+ }++ /**+ * back / home / recents. `actor` matters more than the action: the user's+ * own top-bar presses land here too, and an episode a human steered is not+ * evidence about what the brain was aiming at.+ */+ fun noteNav(action: String, actor: String) {+ if (!on) return+ post(JSONObject().put("ev", "nav").put("action", action).put("actor", actor))+ }++ /** What the element list looked like — i.e. whether the brain had a choice. */+ fun noteTree(pkg: String?, total: Int, useful: Int, elements: Int, surface: String?) {+ if (!on) return+ post(+ JSONObject()+ .put("ev", "tree")+ .put("nodes_total", total)+ .put("nodes_useful", useful)+ .put("elements", elements)+ .also {+ if (pkg != null) it.put("pkg", pkg)+ if (surface != null) it.put("surface", surface)+ }+ )+ }++ /**+ * Let the queue drain rather than dropping it — these are small writes.+ *+ * Clears [exec] so the writers after this are dropped quietly instead of+ * throwing into whoever called them. [init] is what starts a new one, and+ * until it does this log is closed on purpose.+ */+ fun shutdown() {+ if (!on) return+ val e = synchronized(execLock) { exec.also { exec = null } } ?: return+ e.shutdown()+ runCatching { e.awaitTermination(2, TimeUnit.SECONDS) }+ }++ // ---- internals ---------------------------------------------------------++ /**+ * The one place a thread is created. A terminated pool is never handed+ * back: the service can be reconnected into the same process, and+ * `execute` on a terminated pool is a `RejectedExecutionException` on the+ * caller's thread — which, from `onServiceConnected`, means no service.+ */+ private fun restartWriter(): ExecutorService = synchronized(execLock) {+ val cur = exec+ if (cur != null && !cur.isShutdown) return cur+ Executors.newSingleThreadExecutor(+ ThreadFactory { r -> Thread(r, "Grounding").apply { isDaemon = true } }+ ).also { exec = it }+ }++ /**+ * Never throws. A write that arrives after [shutdown] has nowhere to go+ * and is dropped with a line in logcat; the alternative is carrying a+ * `RejectedExecutionException` into a `screen.*` tool call.+ */+ private fun submit(what: String, block: () -> Unit) {+ val e = exec ?: return+ runCatching { e.execute { runCatching(block).onFailure { Log.w(TAG, what, it) } } }+ .onFailure { Log.w(TAG, "$what (write dropped)", it) }+ }++ /** Stamp on the calling thread (ordering must reflect when it happened), write on ours. */+ private fun post(o: JSONObject) {+ val stamped = note(o)+ submit("grounding append failed") { append(stamped) }+ }++ private fun note(o: JSONObject): JSONObject = o+ .put("v", SCHEMA)+ .put("ts", System.currentTimeMillis())+ .put("seq", seq.incrementAndGet())++ /** [exec] only. */+ private fun append(o: JSONObject) {+ val d = dir ?: return+ val f = File(d, "events.jsonl")+ if (f.length() > MAX_JSONL_BYTES) rotate(f)+ f.appendBytes((o.toString() + "\n").toByteArray())+ }++ /** [exec] only. events.jsonl → .1 → .2 …, dropping the oldest. */+ private fun rotate(f: File) {+ runCatching {+ File(f.parentFile, "events.jsonl.$JSONL_KEEP").delete()+ for (i in JSONL_KEEP - 1 downTo 1) {+ val src = File(f.parentFile, "events.jsonl.$i")+ if (src.exists()) src.renameTo(File(f.parentFile, "events.jsonl.${i + 1}"))+ }+ f.renameTo(File(f.parentFile, "events.jsonl.1"))+ }+ }++ /** [exec] only. */+ private fun sweepFromDisk() {+ bytesOnDisk = -1L+ writesSinceSweep = SWEEP_EVERY+ sweep()+ }++ /**+ * [exec] only. Delete oldest-first until back under [MAX_BYTES]. The+ * directory listing is the expensive part, which is why this runs every+ * [SWEEP_EVERY] writes rather than on each one.+ */+ private fun sweep() {+ writesSinceSweep = 0+ val d = shotsDir ?: return+ val files = d.listFiles() ?: return+ var total = files.sumOf { it.length() }+ if (total > MAX_BYTES) {+ files.sortBy { it.lastModified() }+ for (f in files) {+ if (total <= MAX_BYTES) break+ val len = f.length()+ if (f.delete()) total -= len+ }+ }+ bytesOnDisk = total+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/screen/ScreenController.kt b/app/src/main/java/net/kuafuai/andee/screen/ScreenController.ktnew file mode 100644index 0000000..62737ca--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/screen/ScreenController.kt@@ -0,0 +1,1996 @@+package net.kuafuai.andee.screen++import android.accessibilityservice.AccessibilityService+import android.accessibilityservice.GestureDescription+import android.graphics.Bitmap+import android.graphics.BitmapFactory+import android.graphics.Path+import android.graphics.Rect+import android.os.Bundle+import android.os.SystemClock+import android.util.Base64+import android.util.Log+import android.view.Display+import android.view.accessibility.AccessibilityNodeInfo+import android.view.accessibility.AccessibilityWindowInfo+import net.kuafuai.andee.device.DeviceState+import net.kuafuai.andee.net.ToolSchemas+import org.json.JSONArray+import org.json.JSONObject+import java.io.ByteArrayOutputStream+import java.util.concurrent.ConcurrentHashMap+import java.util.concurrent.CountDownLatch+import java.util.concurrent.Executors+import java.util.concurrent.TimeUnit+import java.util.concurrent.atomic.AtomicLong++/**+ * Screen perception + actuation: tap / swipe / long-press / type / screenshot /+ * global-action / ui-tree-dump. Wraps AccessibilityService APIs.+ */+class ScreenController(+ private val service: AccessibilityService,+ /**+ * Where injected gestures get drawn after the fact. Null in any caller that+ * has no overlay to draw on; see [net.kuafuai.andee.ui.TapMarkerUi] for why+ * it is marked *after* the stroke and never before.+ */+ private val marker: net.kuafuai.andee.ui.TapMarkerUi? = null,+) {++ /** Last meaningful accessibility event from a non-overlay app. */+ private val lastContentEventMs = AtomicLong(0L)++ fun noteEvent(event: android.view.accessibility.AccessibilityEvent?) {+ if (event == null) return+ val pkg = event.packageName?.toString().orEmpty()+ if (pkg.isEmpty() || pkg == service.packageName) return+ when (event.eventType) {+ android.view.accessibility.AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED,+ android.view.accessibility.AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED,+ android.view.accessibility.AccessibilityEvent.TYPE_WINDOWS_CHANGED,+ android.view.accessibility.AccessibilityEvent.TYPE_VIEW_FOCUSED,+ android.view.accessibility.AccessibilityEvent.TYPE_VIEW_SCROLLED ->+ lastContentEventMs.set(System.currentTimeMillis())+ }+ }++ // ---- normalized coordinate protocol -----------------------------------+ //+ // The brain speaks 0-1000 RELATIVE integers for every gesture, never+ // pixels: screenshots may have been downscaled on their way to the model,+ // so only a relative space survives the trip and stays resolution-+ // independent. Values outside 0-1000 are REJECTED, not clamped — a silent+ // clamp to the edge is a mis-tap the model can't even see happening.++ private val screenW: Int get() = service.resources.displayMetrics.widthPixels+ private val screenH: Int get() = service.resources.displayMetrics.heightPixels++ private fun Int.toPixel(max: Int): Int = this * max / NORM_MAX++ private fun requireNorm(v: Int, name: String): Int {+ if (v !in 0..NORM_MAX) {+ throw IllegalArgumentException(+ "coordinate $name=$v out of range — expected 0..$NORM_MAX (relative, not pixels)"+ )+ }+ return v+ }++ fun tapNorm(nx: Int, ny: Int): JSONObject {+ requireNorm(nx, "x"); requireNorm(ny, "y")+ val w = screenW; val h = screenH+ val px = nx.toPixel(w); val py = ny.toPixel(h)+ // Logged at the four norm/eid entry points rather than in [tap], even+ // though TapMarkerUi is hooked there and treats every path alike. It+ // can afford to — a ring is a ring. Here the distinction is the whole+ // point: only a coordinate the BRAIN chose is a grounding sample; an+ // e-tap's pixels were computed by us and would poison the set.+ guardRepetition(px, py) {+ GroundingLog.noteTapCoord(nx, ny, px, py, w, h, hold = false, refused = true, pkg = lastPkg())+ }+ GroundingLog.noteTapCoord(nx, ny, px, py, w, h, hold = false, refused = false, pkg = lastPkg())+ return tap(px, py).put("norm", "$nx,$ny")+ }++ fun longPressNorm(nx: Int, ny: Int, durationMs: Long): JSONObject {+ requireNorm(nx, "x"); requireNorm(ny, "y")+ val w = screenW; val h = screenH+ val px = nx.toPixel(w); val py = ny.toPixel(h)+ guardRepetition(px, py) {+ GroundingLog.noteTapCoord(nx, ny, px, py, w, h, hold = true, refused = true, pkg = lastPkg())+ }+ GroundingLog.noteTapCoord(nx, ny, px, py, w, h, hold = true, refused = false, pkg = lastPkg())+ return longPress(px, py, durationMs).put("norm", "$nx,$ny")+ }++ fun swipeNorm(nx1: Int, ny1: Int, nx2: Int, ny2: Int, durationMs: Long): JSONObject {+ requireNorm(nx1, "x1"); requireNorm(ny1, "y1")+ requireNorm(nx2, "x2"); requireNorm(ny2, "y2")+ // Not a sample, but it moves everything on screen: without it the+ // offline pairing would read a post-scroll retry as a correction of a+ // pre-scroll miss, at a point that no longer means the same thing.+ GroundingLog.noteSwipe(nx1, ny1, nx2, ny2, durationMs, actor = "brain")+ return swipe(+ nx1.toPixel(screenW), ny1.toPixel(screenH),+ nx2.toPixel(screenW), ny2.toPixel(screenH),+ durationMs,+ )+ }++ // ---- stuck-loop guard (per-location counting) --------------------------+ //+ // A model that cannot tell whether a tap worked re-taps the same spot with+ // slightly different coordinates forever. Counting *consecutive* taps or+ // requiring "screen didn't change" both fail in the field: apps emit+ // content events constantly, and the real dead-loop interleaves menu+ // opens, wrong pages and backs so the screen is always changing. Only+ // PER-LOCATION counting catches it: every gesture within+ // [LOOP_RADIUS_PX] of a remembered spot bumps that spot's counter, no+ // matter what happens in between. Within [LOOP_WINDOW_MS], the 4th attempt+ // on the same spot is refused with an error telling the brain to stop,+ // re-observe, and ask the user for help. Swipes are exempt — scrolling a+ // long list legitimately repeats the same gesture.++ private val loopLock = Any()+ private val loopSpots = HashMap<String, Int>() // "x,y" bucket -> count+ private var lastLoopSweep = 0L++ /** Set by [dumpUiTree]; read by [lastPkg]. Written and read off-thread. */+ @Volatile private var lastDumpPkg: String? = null++ /**+ * Packages we have actually read a usable element list from, and when.+ *+ * The one fact the device owns that the brain cannot work out for itself:+ * whether *this* app has ever been readable here. WeChat is the app that+ * forced it — its tree is perfectly obtainable, but the accessibility+ * connection sometimes goes stale and the same screen that dumped fine ten+ * minutes ago starts handing back nothing at all. No tool call repairs+ * that; a human toggling Andee off and on in 设置 → 无障碍 does, in about+ * three seconds. The brain cannot ask for that unless it knows the app was+ * supposed to work.+ *+ * Deliberately a record rather than a rule. Nothing here decides anything —+ * [reviveNote] turns it into one sentence of evidence and the brain judges,+ * which is why it is not keyed to `com.tencent.mm`: the same signature from+ * any app means the same thing, and hardcoding the package would have us+ * silent on the second app that does it.+ *+ * Unbounded, and that is fine: one `String`/`Long` per package a human has+ * opened on this tablet, which is tens, not thousands. Never persisted —+ * after a reboot the service is fresh, so old evidence would only argue for+ * a restart that already happened.+ */+ private val goodDumps = ConcurrentHashMap<String, Long>()++ /**+ * @param onRefuse run while still holding [loopLock], just before the+ * refusal is thrown. The refusal is the strongest mis-tap label this+ * system produces — the guard already knows a bucket has been hammered,+ * and its scope (norm taps only, e-taps exempt) is exactly the set+ * [GroundingLog] collects — so it must be written down rather than left+ * to ride out on the exception.+ */+ private fun guardRepetition(x: Int, y: Int, onRefuse: () -> Unit = {}) {+ val now = System.currentTimeMillis()+ synchronized(loopLock) {+ if (now - lastLoopSweep > LOOP_WINDOW_MS) {+ loopSpots.clear()+ lastLoopSweep = now+ }+ val key = "${x / LOOP_RADIUS_PX},${y / LOOP_RADIUS_PX}"+ val n = (loopSpots[key] ?: 0) + 1+ loopSpots[key] = n+ if (n > LOOP_MAX_TRIES) {+ onRefuse()+ throw IllegalStateException(LOOP_HELP)+ }+ }+ }++ /**+ * Package behind the last dump — what the brain was actually looking at.+ * Preferred over `DeviceState.foregroundPkg`, which can read+ * `net.kuafuai.andee` whenever one of our own activities (scan, look, html)+ * is up.+ */+ private fun lastPkg(): String? = lastDumpPkg++ // ---- element-index tapping ----------------------------------------------+ //+ // Coordinate math by the model is the top accuracy risk: models+ // mis-convert pixel bounds against screen size exactly on the small,+ // tightly-packed targets where a few percent of error taps the wrong row.+ // The proven design (generic_bridge / phone.sh): the tree itself carries+ // system-computed e-numbers, and the model only ever says "e7". Zero+ // model math, zero scaling errors — coordinates come from the same source+ // of truth (getBoundsInScreen) that the gesture path converts with.+ //+ // STALENESS: the e-number alone is not enough to identify an element+ // after the page shifts (images load in, content settles) — e28 can+ // become a different view between the dump the brain read and the tap.+ // So the index keeps each entry's identity triple (pkg, viewId, bounds)+ // and tapByEid RE-LOCATES live: find a node NOW matching that triple; only+ // if none matches do we fall back to the remembered pixel center, and an+ // empty index still fails loudly. Phone.sh never had this problem because+ // its step loop dumps-and-acts in one breath; we let the brain think+ // between the two, so the tap has to catch up.+ //+ // e-taps are exempt from the stuck-loop guard: re-tapping a real element+ // is legitimate (toggles, list rows), and the guard exists to catch+ // coordinate GUESSING, not acting on confirmed elements.++ private class EidEntry(+ val px: IntArray, // remembered pixel center — fallback only+ val pkg: String, // owning app, from the node's root at dump time+ val viewId: String?, // resource id if any+ val bounds: String, // "l,t,r,b" in pixels at dump time+ )++ private val eidLock = Any()+ private val eidIndex = HashMap<Int, EidEntry>()++ /** Fresh dump in progress — see [tapByEid] for why taps must wait. */+ private val dumpLock = Object()++ fun tapByEid(eid: String): JSONObject {+ val entry = lookupEid(eid)+ // Re-locate live before touching anything: a match means the element+ // still exists where we found it; no match means either it scrolled+ // away or the pane changed — remembered center, best effort.+ val live = findLiveNode(entry)+ val x: Int; val y: Int+ if (live != null) {+ x = (live[0] + live[2]) / 2+ y = (live[1] + live[3]) / 2+ } else {+ x = entry.px[0]; y = entry.px[1]+ }+ GroundingLog.noteTapEid(eid, x, y, screenW, screenH, hold = false, relocated = live != null, pkg = lastPkg())+ return tap(x, y)+ .put("eid", eid)+ .put("px", "$x,$y")+ .put("relocated", live != null)+ }++ fun longPressByEid(eid: String, durationMs: Long): JSONObject {+ val entry = lookupEid(eid)+ val live = findLiveNode(entry)+ val x: Int; val y: Int+ if (live != null) {+ x = (live[0] + live[2]) / 2+ y = (live[1] + live[3]) / 2+ } else {+ x = entry.px[0]; y = entry.px[1]+ }+ GroundingLog.noteTapEid(eid, x, y, screenW, screenH, hold = true, relocated = live != null, pkg = lastPkg())+ return longPress(x, y, durationMs)+ .put("eid", eid)+ .put("px", "$x,$y")+ .put("relocated", live != null)+ }++ private fun lookupEid(eid: String): EidEntry {+ val key = eid.removePrefix("e").toIntOrNull()+ ?: throw IllegalArgumentException("bad element id '$eid' — expected e<number> from the element list")+ synchronized(eidLock) {+ return eidIndex[key]+ ?: throw IllegalArgumentException(+ "unknown element '$eid' — the page has changed since your last get_screen_element. " ++ "Call get_screen_element again and use an eid from the fresh list, do not guess coordinates"+ )+ }+ }++ /**+ * Try to find, RIGHT NOW, the node this entry described at dump time.+ * Matches on viewId when present (strongest), else on bounds-exact within+ * the entry's app, else null. Walking the whole tree per tap costs a few+ * ms on this hardware; correctness of "did I tap what I meant" is+ * worth far more.+ */+ private fun findLiveNode(entry: EidEntry): IntArray? {+ val root = runCatching { service.rootInActiveWindow }.getOrNull() ?: return null+ try {+ if (root.packageName?.toString() != entry.pkg) return null+ // Strong path: resource id narrows it down, but WeChat reuses ids+ // across siblings (every bottom tab shares one id), so among the+ // matches pick the one CLOSEST to the remembered center — the+ // entry's own geometry, not list order, decides.+ if (!entry.viewId.isNullOrEmpty()) {+ val matches = runCatching {+ root.findAccessibilityNodeInfosByViewId(entry.viewId)+ }.getOrNull().orEmpty()+ try {+ var bestArr: IntArray? = null+ var bestDist = Long.MAX_VALUE+ for (m in matches) {+ val r = Rect()+ runCatching { m.getBoundsInScreen(r) }+ if (r.width() > 0 && r.height() > 0) {+ val cx = (r.left + r.right) / 2+ val cy = (r.top + r.bottom) / 2+ val dx = (cx - entry.px[0]).toLong()+ val dy = (cy - entry.px[1]).toLong()+ val dist = dx * dx + dy * dy+ if (dist < bestDist) {+ bestDist = dist+ bestArr = intArrayOf(r.left, r.top, r.right, r.bottom)+ }+ }+ }+ return bestArr+ } finally {+ matches.forEach { it.recycle() }+ }+ }+ // Weak path: walk for a node with the exact same bounds.+ return findNodeByBounds(root, entry.bounds)+ } finally {+ root.recycle()+ }+ }++ private fun findNodeByBounds(node: AccessibilityNodeInfo, bounds: String): IntArray? {+ val r = Rect()+ runCatching { node.getBoundsInScreen(r) }+ if (r.width() > 0 && r.height() > 0 && "${r.left},${r.top},${r.right},${r.bottom}" == bounds) {+ return intArrayOf(r.left, r.top, r.right, r.bottom)+ }+ for (i in 0 until node.childCount) {+ val child = runCatching { node.getChild(i) }.getOrNull() ?: continue+ try {+ val hit = findNodeByBounds(child, bounds)+ if (hit != null) return hit+ } finally {+ child.recycle()+ }+ }+ return null+ }++ fun tap(x: Int, y: Int): JSONObject {+ val path = Path().apply { moveTo(x.toFloat(), y.toFloat()) }+ // 150ms, not the classic 100ms: WeChat's Moments like/comment buttons+ // (and other touch-filtering surfaces) ignore injections shorter than+ // ~100ms — a 100ms stroke was landing as a no-op while the exact same+ // coordinates at 150ms pop the bubble every time (measured: `input tap`+ // at ~10ms fails, dispatchGesture at 100ms fails, a swipe held at one+ // point for 150ms succeeds). 150ms is still well below any+ // long-press threshold (400ms+), so nothing that was a tap becomes+ // a long-press.+ val stroke = GestureDescription.StrokeDescription(path, 0L, 150L)+ val before = screenFingerprint()+ val result = runGesture(GestureDescription.Builder().addStroke(stroke).build(), 2000L)+ marker?.mark(x, y, net.kuafuai.andee.ui.TapMarkerUi.Kind.TAP)+ result.put("x", x).put("y", y)+ attachEffectSnapshot(result, before)+ return result+ }++ /**+ * Cheap before/after of the screen around a gesture: pkg + window title ++ * a node-tree signature. The point is not what it says, it is that it+ * CHANGES (or doesn't) — a tap that did nothing leaves the fingerprint+ * identical, and the brain reading `screen_changed: false` knows to+ * re-aim instead of assuming success and building on sand.+ *+ * Collected here rather than left to the brain's next dump because the+ * transition window is exactly the moment a dump is least reliable: the+ * animation is still running, elements are mid-flight, and "dump again+ * immediately" reads a screen that does not exist yet. One fingerprint,+ * taken after a fixed beat, answers "did it land" without any of that.+ */+ private fun screenFingerprint(): JSONObject {+ val root = runCatching { service.rootInActiveWindow }.getOrNull()+ ?: return JSONObject().put("pkg", "").put("sig", 0)+ return try {+ val f = FingerprintStats()+ collectFingerprint(root, f, 0)+ // Window COUNT is part of the fingerprint on purpose: WeChat's+ // Moments like/comment bar is a full-screen overlay window that+ // changes nothing in the active window's node tree — measured in+ // the field: the bar appeared, screen_changed said false, and the+ // only honest signal was the window list growing by one.+ val winCount = runCatching { service.windows.size }.getOrDefault(0)+ JSONObject()+ .put("pkg", root.packageName?.toString().orEmpty())+ .put("sig", f.sig() * 31 + winCount)+ } finally {+ root.recycle()+ }+ }++ private class FingerprintStats {+ var count = 0+ var hash = 1+ fun visit(node: AccessibilityNodeInfo) {+ count+++ hash = hash * 31 + node.childCount+ node.viewIdResourceName?.let { hash = hash * 31 + it.hashCode() }+ // TEXT is part of the fingerprint: structure alone cannot see a+ // translation replacing every string on the page, a comment+ // loading in, or a counter ticking — same nodes, same ids, new+ // words. Measured in the field (2026-09-20): dismissing the+ // browser's translate bar changed nothing structural, and+ // screen_changed said false while the screen had visibly changed.+ node.text?.let { hash = hash * 31 + it.hashCode() }+ node.contentDescription?.let { hash = hash * 31 + it.hashCode() }+ }+ fun sig(): Int = count * 31 + hash+ }++ private fun collectFingerprint(node: AccessibilityNodeInfo, f: FingerprintStats, depth: Int) {+ if (depth > 18 || f.count > 900) return // cap: fingerprint, not a dump+ f.visit(node)+ for (i in 0 until node.childCount) {+ val c = runCatching { node.getChild(i) }.getOrNull() ?: continue+ try {+ collectFingerprint(c, f, depth + 1)+ } finally {+ c.recycle()+ }+ }+ }++ /**+ * The "did my tap land" half of the effect snapshot. Waits one beat for+ * the app to react (a transition that has not started yet reads as no+ * change), then fingerprints again and reports the delta:+ * - `screen_changed` — the honest one-bit answer+ * - `now_pkg` — where we landed, if we landed somewhere+ * - on SUCCESS: `after_shot` — a full screenshot WITH the marker ring+ * already drawn at this tap's position (the ring is deliberately+ * never blanked — see TapMarkerUi). The brain sees exactly where its+ * tap landed relative to everything on screen, which is how a hit+ * confirms the aim AND how a near-miss ("ring is 40px left of the+ * button") self-corrects.+ * - on MISS: `miss_shot` — same idea, zoomed: the ring at the tapped+ * spot, the neighbourhood magnified, aim the retry from measurement.+ *+ * Both carry `tap_mark: [x,y]` in 0-1000 so the ring's position is+ * machine-readable, not just visible.+ */+ private fun attachEffectSnapshot(result: JSONObject, before: JSONObject) {+ try {+ // 400ms caught the launcher still holding focus mid-launch (tap+ // camera → changed=true but now_pkg=launcher — measured on+ // device). 700ms lets the activity-change settle while staying+ // well under the brain's own next-step latency.+ Thread.sleep(700)+ } catch (_: InterruptedException) {+ Thread.currentThread().interrupt()+ }+ val after = screenFingerprint()+ val changed = before.optString("pkg") != after.optString("pkg") ||+ before.optInt("sig") != after.optInt("sig")+ result.put("screen_changed", changed)+ result.put("now_pkg", after.optString("pkg"))+ val tapX = result.optInt("x")+ val tapY = result.optInt("y")+ // JSONArray, element-wise: a bare IntArray goes through as one opaque+ // object and serialises to "[I@41d5a25" — a machine-readable field+ // that no machine can read.+ result.put("tap_mark", org.json.JSONArray().put(+ (tapX * 1000f / screenW).toInt()+ ).put(+ (tapY * 1000f / screenH).toInt()+ ))+ if (changed) {+ // Success still deserves the evidence: the ring-marked screen is+ // the cheapest possible confirmation, and it doubles as "this is+ // what I was aiming at" context for the next step.+ runCatching {+ if (tapX > 0 || tapY > 0) {+ // The dispatcher's gesture-passthrough still has the+ // marker hidden at this point (we run inside it) —+ // reveal, then give the post a beat to land.+ marker?.revealNow()+ Thread.sleep(80)+ result.put("after_shot", screenshot("tap_after"))+ }+ }+ // The NEW screen's element list, so the brain can chain its next+ // action (tap / type) WITHOUT a get_screen_element round trip it+ // would otherwise burn on "what did the screen become?". The+ // screenshot above already carries the pixels; this carries the+ // e-numbers. One tap response = full next-step context.+ runCatching {+ val tree = dumpOnce(false)+ if (tree.optInt("nodes_useful", 0) > 0) {+ rebuildIndexAndList(tree)+ val elements = tree.optString("elements")+ if (elements.isNotEmpty()) {+ result.put("elements", elements)+ .put("elements_pkg", tree.optString("pkg"))+ .put("elements_hint",+ "This is the POST-TAP screen. Chain your next tap_screen_element / " ++ "type_text directly from these e-numbers — do NOT re-dump first.")+ }+ }+ }+ } else {+ result.put(+ "same_spot_hint",+ "the screen did NOT react to this tap. miss_shot shows where the ring " ++ "landed — the tapped point, magnified with its surroundings. Aim the " ++ "retry from that image, not from the previous full screenshot.",+ )+ runCatching {+ if (tapX > 0 || tapY > 0) {+ // Region around the tap, in 0-1000 space: ~30% of the+ // shorter axis each way, clamped by lookRegion itself.+ // revealNow: same reason as the success path — the ring+ // is the evidence, and passthrough has it hidden.+ marker?.revealNow()+ Thread.sleep(80)+ val rx = (tapX * 1000f / screenW).toInt()+ val ry = (tapY * 1000f / screenH).toInt()+ val size = 300+ val hUnits = (size * screenH.toFloat() / screenW).toInt().coerceAtLeast(1)+ result.put("miss_shot", lookRegion(rx, ry, size, hUnits))+ }+ }+ }+ }++ fun tapById(id: String): JSONObject {+ val root = service.rootInActiveWindow ?: throw IllegalStateException("no active window")+ try {+ val matches = root.findAccessibilityNodeInfosByViewId(id).orEmpty()+ val target = matches.firstOrNull() ?: throw IllegalArgumentException("no node with id: $id")+ try {+ val performed = target.performAction(AccessibilityNodeInfo.ACTION_CLICK)+ return JSONObject().put("performed", performed).put("id", id).put("matches", matches.size)+ } finally {+ matches.forEach { it.recycle() }+ }+ } finally {+ root.recycle()+ }+ }++ fun swipe(x1: Int, y1: Int, x2: Int, y2: Int, durationMs: Long): JSONObject {+ val path = Path().apply {+ moveTo(x1.toFloat(), y1.toFloat())+ lineTo(x2.toFloat(), y2.toFloat())+ }+ val stroke = GestureDescription.StrokeDescription(path, 0L, durationMs)+ return runGesture(GestureDescription.Builder().addStroke(stroke).build(), durationMs + 2000).put("from", "$x1,$y1").put("to", "$x2,$y2").put("duration_ms", durationMs)+ }++ fun longPress(x: Int, y: Int, durationMs: Long): JSONObject {+ val path = Path().apply { moveTo(x.toFloat(), y.toFloat()) }+ val stroke = GestureDescription.StrokeDescription(path, 0L, durationMs)+ val result = runGesture(GestureDescription.Builder().addStroke(stroke).build(), durationMs + 2000)+ marker?.mark(x, y, net.kuafuai.andee.ui.TapMarkerUi.Kind.HOLD)+ return result.put("x", x).put("y", y).put("duration_ms", durationMs)+ }++ fun longPressById(id: String): JSONObject {+ val root = service.rootInActiveWindow ?: throw IllegalStateException("no active window")+ try {+ val matches = root.findAccessibilityNodeInfosByViewId(id).orEmpty()+ val target = matches.firstOrNull() ?: throw IllegalArgumentException("no node with id: $id")+ try {+ val performed = target.performAction(AccessibilityNodeInfo.ACTION_LONG_CLICK)+ return JSONObject().put("performed", performed).put("id", id)+ } finally {+ matches.forEach { it.recycle() }+ }+ } finally {+ root.recycle()+ }+ }++ fun typeText(text: String, id: String?): JSONObject = typeText(text, id, secret = false)++ /**+ * [secret] = this text is a vault password. Everything works the same+ * except that the value never appears in the result — see+ * [typeViaAdbKeyboard].+ */+ fun typeText(text: String, id: String?, secret: Boolean): JSONObject {+ // ── 输入通道(2026-09-20 定版):ADBKeyboard IME 广播单通道 ──+ //+ // 实测依据(当天 bolt.new 两设备全量对比):+ // - SET_TEXT/剪贴板PASTE:Radix 表单串字段,CodeMirror 全灭,+ // MIUI 上 setPrimaryClip 还会静默丢弃(粘进旧内容);+ // - ADBKeyboard(IME, commitText 正统通道):bolt 注册表单+ // 三字段全对,连 CodeMirror 都进。+ // 设备 ADBKeyboard 为常驻默认 IME。**这句原来写的是「产品决策:平板/+ // 手机是 Andee 本体,无真人键盘场景」—— 那个前提已经不成立**:卡片上+ // 有了 ⌨ 文本输入,真人要打字了。现在由 ui/ImeSwitch 在用户打字期间+ // 临时换成真键盘、发完换回来;这里唯一的要求没变 —— 轮到大脑打字时+ // ADBKeyboard 必须是当前输入法,否则 ADB_INPUT_TEXT 发出去没人接。+ // 通道:检查焦点 → ADB_CLEAR_TEXT 清残留+ // → ADB_INPUT_TEXT → 双路读回验证。失败如实报错,无多通道瀑布。+ val root = service.rootInActiveWindow ?: throw IllegalStateException("no active window")+ val focused = try {+ root.findFocus(AccessibilityNodeInfo.FOCUS_INPUT)+ } finally {+ root.recycle()+ }+ if (focused == null) {+ throw IllegalStateException("no input focus (tap into an input field first)")+ }+ runCatching { if (focused !== root) focused.recycle() }+ return typeViaAdbKeyboard(text, secret)+ }++ /**+ * 通过 ADBKeyboard 的 IME 广播输入 [text]。+ *+ * ADBKeyboard 是激活的输入法,收到 ADB_INPUT_TEXT 后走+ * InputConnection.commitText()——与真人敲键盘同一条路径,目标+ * 应用无法拒绝(拒绝即拒绝一切键盘输入)。+ *+ * 语义:广播是【追加】,故先发 ADB_CLEAR_TEXT 清残留(之前失败+ * 尝试/自动填充),再输入,最后双路读回(焦点框优先,全树兜底+ * ——WebView 焦点节点藏文本时树里能找到)。自绘编辑器+ * (CodeMirror)两路都可能读不到,那是 a11y 假阴性,错误信息里+ * 提示大脑以截图为准。+ *+ * action 【不带包名前缀】。ADBKeyboard 在 AdbIME.onCreate 里+ * registerReceiver 的 filter 就是裸字符串 ADB_INPUT_TEXT(见设备+ * `dumpsys activity broadcasts`),写成+ * com.android.adbkeyboard.ADB_INPUT_TEXT 匹配不上任何 filter,加上+ * setPackage 钉死收件人,广播就发出去没人接——静默失败,和被系统+ * 拦截长得一模一样。这正是「MIUI 拦 app 跨进程广播」那个结论的+ * 由来:电脑侧 `am broadcast -a ADB_INPUT_TEXT` 用的是对的名字所以+ * 通,设备侧用错名字所以不通,看上去就像身份问题。+ *+ * [secret] 为真时(保险箱密码),读回照常做——内部比对,结果照实+ * 报——但**成功不回声、失败不带片段**。回声本来是给大脑确认输入+ * 成功用的,而密码回声等于把明文顺着 WebSocket 送回远端模型,+ * 「只填不给」这条边界就是在这里成立或失效的。+ */+ private fun typeViaAdbKeyboard(text: String, secret: Boolean = false): JSONObject {+ // ── 键盘是借的,不是占的 ──+ //+ // 设备的输入法平时是**用户自己的**那个(它能在任何 App 里打字),+ // ADBKeyboard 只在这一次调用期间被借过来。所以:先借 → 打字 → 立刻还。+ // 想要这条成立,`text` 的广播必须落在**已经 bind 上**的 ADBKeyboard 上,+ // 而"选中"和"绑定"不是同一件事 —— 这就是 ImeSwitch.lendToBrain 里+ // 那 350ms 的由来(实测 bind 上界 ~400ms,框架不对外暴露这个状态)。+ //+ // 借不到就是**响亮的失败**,不能假装打了字:广播发出去没人接,读回+ // 只会得到"字段还是空的",而那句错误信息会去怪目标 App 藏了文本。+ val giveBackTo = net.kuafuai.andee.ui.ImeSwitch.lendToBrain(service)+ try {+ if (net.kuafuai.andee.ui.ImeSwitch.current(service) != net.kuafuai.andee.ui.ImeSwitch.BRAIN_IME) {+ throw IllegalStateException(+ "cannot make ADBKeyboard the active IME (it is " ++ "'${net.kuafuai.andee.ui.ImeSwitch.current(service)}') — " ++ "the type broadcast would be delivered to nobody. " ++ "Needs WRITE_SECURE_SETTINGS granted once by adb."+ )+ }+ return typeWithKeyboardUp(text, secret)+ } finally {+ net.kuafuai.andee.ui.ImeSwitch.giveBack(service, giveBackTo)+ }+ }++ // ── On the keyboard the brain's typing raises, and why nothing here puts it down ──+ //+ // Handing the keyboard back to the user makes a real IME active, and a real+ // IME honours the show request that the brain's own tap created: after+ // `screen.type`, the user's keyboard is on screen over the app being driven,+ // and that app's layout is resized under it. Measured, twice:+ //+ // * `hideSoftInputFromWindow(null, 0)` — the only handle a non-owner of+ // the field's window has — is **ignored**. `mInputShown` stayed true.+ // * `performGlobalAction(GLOBAL_ACTION_BACK)` is **not** consumed by the+ // IME the way an injected key event is: `adb shell input keyevent+ // KEYCODE_BACK` closed the keyboard and left the page alone, while the+ // service-level action dismissed *the page* and left the keyboard.+ //+ // So this is left alone deliberately. It is not a defect in the switch: with+ // the person's keyboard as the active IME, a focused field raising it is+ // what that means. It clears itself as soon as the brain touches something+ // that takes focus. Both failed attempts are recorded because the second one+ // is the tempting one, and it navigates somebody else's app backwards.++ /** The broadcast-and-verify half. Only ever called with the brain's IME up. */+ private fun typeWithKeyboardUp(text: String, secret: Boolean): JSONObject {+ runCatching {+ service.sendBroadcast(+ android.content.Intent(ADB_CLEAR_TEXT).setPackage(ADB_KEYBOARD_PKG)+ )+ Thread.sleep(150)+ }+ service.sendBroadcast(+ android.content.Intent(ADB_INPUT_TEXT)+ .putExtra("msg", text)+ .setPackage(ADB_KEYBOARD_PKG)+ )+ // 轮询而不是睡一个固定值:广播是异步的,commitText 之后 WebView+ // 还要把新值回抛给无障碍树。实测 400ms 大多数时候够,偶尔不够,+ // 而读早了报出来的是「没输进去」——一条会让大脑重发、于是真的+ // 输两遍的假错误。+ var afterFocused = ""+ var afterTree = ""+ val deadline = SystemClock.uptimeMillis() + READBACK_BUDGET_MS+ do {+ Thread.sleep(READBACK_POLL_MS)+ afterFocused = readFocusedText()+ afterTree = if (afterFocused == text) "" else readTextFromTree(text)+ if (afterFocused == text || afterTree == text) break+ } while (SystemClock.uptimeMillis() < deadline)+ val ok = afterFocused == text || afterTree == text+ return if (ok) {+ JSONObject()+ .put("performed", true)+ .put("via", "adb_keyboard")+ .put("len", text.length)+ .apply {+ if (!secret) put("text", if (afterFocused == text) afterFocused else afterTree)+ }+ } else if (secret) {+ JSONObject()+ .put("performed", false)+ .put("via", "adb_keyboard")+ .put("len", text.length)+ .put("error",+ "broadcast sent but the field does not hold the expected value " ++ "(read back ${afterFocused.length} chars from the focused node, " ++ "${afterTree.length} from the tree) — ADBKeyboard may not be the " ++ "active IME, or the target hides its text; verify with a screenshot")+ } else {+ JSONObject()+ .put("performed", false)+ .put("via", "adb_keyboard")+ .put("len", text.length)+ .put("error",+ "broadcast sent but field text != expected (focused: '${afterFocused.take(30)}', tree: '${afterTree.take(30)}') — " ++ "ADBKeyboard may not be the active IME, or the target hides its text; verify with a screenshot")+ }+ }+ /** Focused input's text, or "" — never throws, never leaks nodes. */+ private fun readFocusedText(): String = runCatching {+ val r = service.rootInActiveWindow ?: return@runCatching ""+ val f = r.findFocus(AccessibilityNodeInfo.FOCUS_INPUT)+ try { f?.text?.toString().orEmpty() } finally {+ if (f != null && f !== r) f.recycle()+ r.recycle()+ }+ }.getOrDefault("")+ /**+ * Deep-scan the active window's tree for a node whose text equals+ * [want] — the fallback read path for WebViews that keep the focused+ * node's text hidden from accessibility. Returns "" when absent.+ */+ private fun readTextFromTree(want: String): String {+ if (want.isEmpty()) return ""+ val r = service.rootInActiveWindow ?: return ""+ try {+ val stack = ArrayDeque<AccessibilityNodeInfo>()+ stack.add(r)+ while (stack.isNotEmpty()) {+ val n = stack.removeFirst()+ val t = n.text?.toString().orEmpty()+ if (t == want) return t+ for (i in 0 until n.childCount) n.getChild(i)?.let { stack.add(it) }+ }+ } finally {+ r.recycle()+ }+ return ""+ }++ /**+ * Fire the current input field's IME action (Enter / Send / Search /+ * Next / Done — whichever the field was set up for). Prefer this over+ * synthesizing a keyboard Enter, because IME action is what the app is+ * actually listening for.+ *+ * When `id` is null we use the currently focused input.+ */+ fun submitInput(id: String?): JSONObject {+ val root = service.rootInActiveWindow ?: throw IllegalStateException("no active window")+ try {+ val target: AccessibilityNodeInfo = if (id != null) {+ root.findAccessibilityNodeInfosByViewId(id).orEmpty().firstOrNull()+ ?: throw IllegalArgumentException("no node with id: $id")+ } else {+ root.findFocus(AccessibilityNodeInfo.FOCUS_INPUT)+ ?: throw IllegalStateException("no input focus (tap into an input field first)")+ }+ try {+ val performed = target.performAction(+ AccessibilityNodeInfo.AccessibilityAction.ACTION_IME_ENTER.id+ )+ return JSONObject()+ .put("performed", performed)+ .put("target_id", target.viewIdResourceName)+ } finally {+ if (target !== root) target.recycle()+ }+ } finally {+ root.recycle()+ }+ }++ /**+ * @param actor who pressed it. The same method serves the brain's+ * `system_action` and the user's own top-bar back/home button, and from+ * in here they are indistinguishable — but a navigation the user chose+ * invalidates the episode around it, so [GroundingLog] has to be told+ * which it was. Defaulted to "brain" because the dispatcher, the only+ * caller that can't be the user, is also the one that never passes it.+ */+ fun globalAction(action: String, actor: String = "brain"): JSONObject {+ val code = when (action.lowercase()) {+ "back" -> AccessibilityService.GLOBAL_ACTION_BACK+ "home" -> AccessibilityService.GLOBAL_ACTION_HOME+ "recents" -> AccessibilityService.GLOBAL_ACTION_RECENTS+ "notifications" -> AccessibilityService.GLOBAL_ACTION_NOTIFICATIONS+ "quick_settings" -> AccessibilityService.GLOBAL_ACTION_QUICK_SETTINGS+ else -> throw IllegalArgumentException("unknown global action: $action")+ }+ GroundingLog.noteNav(action.lowercase(), actor)+ val ok = service.performGlobalAction(code)+ return JSONObject().put("performed", ok).put("action", action)+ }++ /**+ * @param reason why this capture happened — "tool" (take_screenshot),+ * "with_shot", or "auto_empty" (the dump had no list to give). It is the+ * field the offline pairing filters on: only the self-drawn panes behind+ * "auto_empty" carry coordinate taps worth learning from.+ */+ fun screenshot(reason: String = "tool"): JSONObject {+ // Settle before capture: images loading and lists settling MOVE the+ // things a screenshot is about to be used to aim at — a shot taken+ // mid-settle gives the brain targets that have already shifted by+ // the time it taps (the "aimed at A, screen became B" miss). Same+ // stability criterion as dumpUiTree: two consecutive tree signatures+ // agree. Cheap when still (one extra fingerprint pass), capped when+ // genuinely animated — the shot is taken anyway and `still_moving`+ // says so, rather than pretending.+ runCatching {+ var prev = screenFingerprint().optInt("sig", -1)+ var tries = 0+ while (tries < STABLE_MAX_TRIES) {+ Thread.sleep(STABLE_POLL_MS)+ val sig = screenFingerprint().optInt("sig", -2)+ if (sig == prev) return@runCatching+ prev = sig+ tries+++ }+ }+ val latch = CountDownLatch(1)+ var pngBytes: ByteArray? = null+ var errCode = 0+ var w = 0;+ var h = 0+ val executor = Executors.newSingleThreadExecutor()+ try {+ service.takeScreenshot(+ Display.DEFAULT_DISPLAY,+ executor,+ object : AccessibilityService.TakeScreenshotCallback {+ override fun onSuccess(screenshot: AccessibilityService.ScreenshotResult) {+ try {+ val hb = screenshot.hardwareBuffer+ val cs = screenshot.colorSpace+ val bmp = Bitmap.wrapHardwareBuffer(hb, cs)+ ?: run { errCode = -1; return }+ w = bmp.width; h = bmp.height+ // Downscale to max side 1280 before PNG-encoding+ // (generic_bridge finding): the raw 1600x2560 shot+ // is ~765KB base64 and models gain nothing from+ // the extra pixels, while every vision token is+ // billed. Safe for coordinates because the whole+ // protocol is 0-1000 relative — scale-invariant.+ val scale = SHOT_MAX_SIDE.toFloat() / maxOf(w, h)+ val outBmp = if (scale < 1f) {+ Bitmap.createScaledBitmap(+ bmp,+ (w * scale).toInt().coerceAtLeast(1),+ (h * scale).toInt().coerceAtLeast(1),+ true,+ )+ } else bmp+ val bos = ByteArrayOutputStream()+ outBmp.compress(Bitmap.CompressFormat.PNG, 100, bos)+ pngBytes = bos.toByteArray()+ if (outBmp !== bmp) outBmp.recycle()+ hb.close()+ } catch (t: Throwable) {+ errCode = -2+ } finally {+ latch.countDown()+ }+ }++ override fun onFailure(errorCode: Int) {+ errCode = errorCode+ latch.countDown()+ }+ },+ )+ if (!latch.await(5, TimeUnit.SECONDS)) throw IllegalStateException("screenshot timeout")+ val bytes = pngBytes+ ?: throw IllegalStateException("screenshot failed with code $errCode")+ val b64 = Base64.encodeToString(bytes, Base64.NO_WRAP)+ // Hooked here, not in onSuccess: the PNG is already encoded at this+ // point, so logging costs no second capture and no second compress —+ // and the callback runs on `executor`, which is shutdown two lines+ // below. Same downscale arithmetic as above, since `w`/`h` are+ // deliberately reported as the SOURCE dimensions.+ val scale = SHOT_MAX_SIDE.toFloat() / maxOf(w, h)+ val outW = if (scale < 1f) (w * scale).toInt().coerceAtLeast(1) else w+ val outH = if (scale < 1f) (h * scale).toInt().coerceAtLeast(1) else h+ val shotId = GroundingLog.noteShot(+ png = bytes,+ srcW = w, srcH = h, outW = outW, outH = outH,+ reason = reason,+ pkg = lastPkg(),+ markerPx = marker?.lastMarkPx(),+ markerAgeMs = marker?.lastMarkAgeMs(),+ )+ return JSONObject()+ .put("png_base64", b64)+ .put("width", w)+ .put("height", h)+ .put("bytes", bytes.size)+ // Lets the response the brain saw be matched to the file on+ // disk. Absent when the log is off.+ .also { if (shotId != null) it.put("shot_id", shotId) }+ // Piggyback the element list. The brain's known failure mode+ // (measured, 2026-09-20): it takes a screenshot, measures a+ // target in pixels off the image, and taps blind — even on+ // panes where the tree was available one tool call earlier,+ // because the image arrived WITHOUT the list and "out of+ // sight, out of mind". Attaching the same system-computed+ // 0-1000 centers the tree tool returns makes every screenshot+ // self-sufficient: aim by eid, not by eyeballing pixels.+ // Deliberately not a parameter — there is no case where a+ // screenshot is better WITHOUT the list; on self-drawn panes+ // it comes back empty and costs nothing.+ .also {+ runCatching {+ val tree = dumpOnce(false)+ if (tree.optInt("nodes_useful", 0) > 0) {+ rebuildIndexAndList(tree)+ val elements = tree.optString("elements")+ if (elements.isNotEmpty()) {+ it.put("elements", elements)+ .put("nodes_useful", tree.optInt("nodes_useful"))+ .put("aim_hint",+ "Prefer tap_id(e) / type(id) with these elements — " ++ "system-computed, always accurate. Pixel-aiming " ++ "from the image is the fallback, not the default.")+ }+ }+ }+ }+ } finally {+ executor.shutdown()+ }+ }++ /**+ * Zoomed look at a region — the fix for "small target, big guess".+ *+ * A full screenshot compresses a `...` bubble to ~25px, and a vision+ * model's ±10px read on that is ±32px on the real screen — half the+ * button's hit area, so taps land or miss at coin-flip odds and every+ * retry re-guesses from the same un-zoomed picture (measured in the+ * field: three aims at one button, y=700/547/709 — 410px of spread).+ * This returns the region magnified, so the same ±10px read becomes+ * ±3px on screen: the brain sees the button clearly, aims once, done.+ *+ * Coordinates are the same 0-1000 relative space as everything else;+ * `w`/`h` are the region's size in that space. The returned image keeps+ * its true aspect, is scaled so the LONGER side hits+ * [SHOT_MAX_SIDE] (same budget as a full shot), and the response+ * restates the region's screen bounds plus a `scale` factor — the brain+ * reads a point off the zoomed image, divides by scale, adds the+ * region's origin, and has a screen-true coordinate without the device+ * doing math for it.+ */+ fun lookRegion(nx: Int, ny: Int, w: Int, h: Int): JSONObject {+ requireNorm(nx, "x"); requireNorm(ny, "y")+ if (w <= 0 || h <= 0 || w > 1000 || h > 1000) {+ throw IllegalArgumentException("w and h must be 1..1000 (relative), got w=$w h=$h")+ }+ val left = (nx.toPixel(screenW) - w.toPixel(screenW) / 2).coerceIn(0, screenW - 1)+ val top = (ny.toPixel(screenH) - h.toPixel(screenH) / 2).coerceIn(0, screenH - 1)+ val right = (left + w.toPixel(screenW)).coerceAtMost(screenW)+ val bottom = (top + h.toPixel(screenH)).coerceAtMost(screenH)+ if (right - left < 8 || bottom - top < 8) {+ throw IllegalArgumentException("region too small after clamping to screen")+ }+ val full = screenshot("look_region")+ val bytes = Base64.decode(full.getString("png_base64"), Base64.NO_WRAP)+ var bmp = BitmapFactory.decodeByteArray(bytes, 0, bytes.size)+ ?: throw IllegalStateException("could not decode screenshot")+ // The DECODED bitmap's own size, not the response's "width"/"height":+ // those report the SOURCE screen dimensions while the PNG itself is+ // downscaled to max 1280 (first field report bug: x+width overflowed+ // the real bitmap and createBitmap threw).+ val shotW = bmp.width+ val shotH = bmp.height+ val sx = shotW.toFloat() / screenW+ val sy = shotH.toFloat() / screenH+ val rLeft = (left * sx).toInt().coerceIn(0, shotW - 2)+ val rTop = (top * sy).toInt().coerceIn(0, shotH - 2)+ val rRight = (right * sx).toInt().coerceAtLeast(rLeft + 4).coerceAtMost(shotW)+ val rBottom = (bottom * sy).toInt().coerceAtLeast(rTop + 4).coerceAtMost(shotH)+ // Magnify the crop so its longer side hits the same budget a full+ // screenshot gets — the entire point is more pixels-per-target.+ val cw = rRight - rLeft+ val ch = rBottom - rTop+ val mag = (SHOT_MAX_SIDE.toFloat() / maxOf(cw, ch)).coerceAtLeast(1f)+ val outW = (cw * mag).toInt().coerceAtLeast(1)+ val outH = (ch * mag).toInt().coerceAtLeast(1)+ try {+ val region = Bitmap.createBitmap(bmp, rLeft, rTop, cw, ch)+ val scaled = Bitmap.createScaledBitmap(region, outW, outH, true)+ val bos = ByteArrayOutputStream()+ scaled.compress(Bitmap.CompressFormat.PNG, 100, bos)+ val outBytes = bos.toByteArray()+ region.recycle()+ if (scaled !== region) scaled.recycle()+ // Region bounds in the 0-1000 space the brain speaks.+ val relLeft = (left * 1000f / screenW).toInt()+ val relTop = (top * 1000f / screenH).toInt()+ val relRight = (right * 1000f / screenW).toInt()+ val relBottom = (bottom * 1000f / screenH).toInt()+ return JSONObject()+ .put("png_base64", Base64.encodeToString(outBytes, Base64.NO_WRAP))+ .put("width", outW)+ .put("height", outH)+ .put("bytes", outBytes.size)+ .put(+ "region",+ JSONObject()+ .put("left", relLeft).put("top", relTop)+ .put("right", relRight).put("bottom", relBottom),+ )+ // shot px per screen-1000 unit: a point read off the image at+ // (px, py) maps to screen-relative+ // (relLeft + px / scale_x, relTop + py / scale_y).+ .put("scale_x", outW.toFloat() / (relRight - relLeft).coerceAtLeast(1))+ .put("scale_y", outH.toFloat() / (relBottom - relTop).coerceAtLeast(1))+ } finally {+ bmp.recycle()+ }+ }++ /**+ * Dump the active window's UI tree, with enough diagnostics that an empty+ * tree can say *why* it is empty.+ *+ * Three different failures look the same from the brain's side ("no nodes"),+ * and only one of them is fixable inside this app:+ *+ * - `no active window` — the root is null; usually a transition, sometimes+ * a window the app refuses to hand over.+ * - `nodes_total` high, `nodes_useful` 0 — we can see the hierarchy but+ * it carries no id/text/desc. Almost always an app that draws content+ * itself (WebView page, Flutter/X5 pane, custom-rendered list, game).+ * No flag fixes this; you have to look at pixels.+ * - `nodes_total` low — the tree really is thin, or the content lives in+ * a *different* window, which is why we go looking for one below.+ *+ * Anything an app draws into its own render surface (see [isRenderSurface])+ * is invisible to accessibility by construction — that is a property of the+ * Android view system, not of this service — so the answer there is never+ * "try harder", it's `take_screenshot` and tap by coordinate.+ */+ fun dumpUiTree(+ verbose: Boolean = false,+ withShot: Boolean = ToolSchemas.UI_TREE_WITH_SHOT_DEFAULT,+ capture: (String) -> JSONObject = { screenshot(it) },+ ): JSONObject {+ val ts = System.currentTimeMillis()+ var out = dumpOnce(verbose)+ var waited = 0L+ while (!verbose && out.optInt("nodes_useful", 0) == 0 && waited < SETTLE_BUDGET_MS) {+ val age = System.currentTimeMillis() - lastContentEventMs.get()+ if (lastContentEventMs.get() != 0L && age >= QUIET_WINDOW_MS) break+ try { Thread.sleep(SETTLE_POLL_MS) } catch (_: InterruptedException) { break }+ waited += SETTLE_POLL_MS+ out = dumpOnce(verbose)+ }+ // Stability pass — the half the old loop never covered: "has content"+ // is not "has stopped moving". A screen mid-animation (list settling,+ // pager sliding, splash fading) dumps real elements at positions that+ // are already stale by the time the brain acts on them. Re-dump until+ // two consecutive passes agree on the tree signature, capped tight:+ // this runs on EVERY dump, so the cost when the screen was already+ // still is exactly one extra pass — and that pass is also what+ // guarantees the eid index the brain is about to read was built from+ // the settled geometry, not a mid-flight one.+ if (!verbose && out.optInt("nodes_useful", 0) > 0) {+ var prevSig = treeSignature(out)+ var stableTries = 0+ while (waited < SETTLE_BUDGET_MS && stableTries < STABLE_MAX_TRIES) {+ try { Thread.sleep(STABLE_POLL_MS) } catch (_: InterruptedException) { break }+ waited += STABLE_POLL_MS+ val again = dumpOnce(verbose)+ val sig = treeSignature(again)+ if (sig == prevSig) {+ out = again+ break+ }+ prevSig = sig+ out = again+ stableTries+++ }+ if (stableTries >= STABLE_MAX_TRIES) {+ out.put("still_moving", true)+ }+ }+ if (!verbose && out.optInt("nodes_useful", 0) == 0 && waited >= SETTLE_BUDGET_MS) {+ out.put("settling", true).put("settle_waited_ms", waited)+ }+ // Flat element list + eid index — the phone.sh contract: one line per+ // element, system-computed 0-1000 centers, model never does math.+ if (!verbose) rebuildIndexAndList(out)+ // Ahead of the shot block on purpose: `screenshot()` stamps its record+ // with [lastPkg], and the picture belongs to the screen we just read,+ // not to the one before it.+ out.optString("pkg").takeIf { it.isNotEmpty() }?.let { lastDumpPkg = it }+ // Evidence for [reviveNote], recorded only on a dump that actually+ // worked. Our own package is excluded: our Activities read fine and+ // would be recorded as known-good, which is true and useless — nobody+ // is going to ask the user to restart the service so we can read+ // ourselves.+ if (!verbose && out.optInt("nodes_useful", 0) > 0) {+ out.optString("pkg")+ .takeIf { it.isNotEmpty() && it != service.packageName }+ ?.let { goodDumps[it] = ts }+ }+ if (!verbose) {+ GroundingLog.noteTree(+ pkg = out.optString("pkg").takeIf { it.isNotEmpty() },+ total = out.optInt("nodes_total", 0),+ useful = out.optInt("nodes_useful", 0),+ // Lines, not characters: "did the brain have anything to tap"+ // is the only question the offline filter asks of this.+ elements = out.optString("elements").let {+ if (it.isEmpty()) 0 else it.count { c -> c == '\n' } + 1+ },+ surface = out.optString("surface").takeIf { it.isNotEmpty() },+ )+ }+ // Tell the brain which way the screen is facing: a mid-task rotation+ // (keyboard attached, user flips the tablet) invalidates every+ // coordinate and geometric prior it is holding — the least we owe it+ // is the fact. Coordinates in our 0-1000 space are still correct+ // (they are relative to the CURRENT screen), but "the comment bubble+ // sits left of the button" style priors are portrait-only knowledge.+ runCatching {+ val rot = service.resources.configuration.orientation+ out.put("orientation", if (rot == android.content.res.Configuration.ORIENTATION_LANDSCAPE) "landscape" else "portrait")+ }+ // A picture, but only when it is worth something. With a populated+ // element list it is not: the list already says what is tappable and+ // where, in the same 0-1000 space, for a fraction of the tokens — so+ // `with_shot` defaults to false and the brain has to ask.+ //+ // The exception is the case the list cannot serve. An empty list means+ // this pane draws its own pixels, and there the image is not a bonus+ // next to the list, it IS the observation — the brain would otherwise+ // have to spend a second round trip on take_screenshot to learn+ // anything at all about the screen it is standing on. `shot_auto`+ // marks the ones it did not ask for.+ //+ // Also empty in effect: a dump that came back describing US. Our+ // overlays are hidden from accessibility outright and the window sweep+ // refuses our package, so this should no longer be reachable through+ // those two routes — but `rootInActiveWindow` can still be one of our+ // own Activities (Html/Scan/Look), which are ordinary app windows and+ // legitimately have a tree. It is our tree, not the brain's business,+ // so treat it as no tree and give it the picture instead.+ //+ // Ordered AFTER the index build so the image and the e-numbers+ // describe the same screen state. `capture` rather than `screenshot()`+ // so the caller can blank our overlay first — see CommandDispatcher.+ val treeEmpty = out.optInt("nodes_useful", 0) == 0 ||+ out.optString("pkg") == service.packageName+ if (!verbose && (withShot || treeEmpty)) {+ val reason = if (withShot) "with_shot" else "auto_empty"+ runCatching { out.put("shot", capture(reason)) }+ .onFailure { out.put("shot_error", it.message ?: "capture failed") }+ if (!withShot) out.put("shot_auto", true)+ }+ out.put("ts", ts)+ annotate(out, verbose)+ return out+ }++ /**+ * Signature of a dump's content for the stability pass: which nodes+ * exist, their ids and their bounds. Bounds are the part that matters —+ * a list mid-settle keeps the same nodes while their coordinates slide,+ * and the brain taps coordinates. Text is deliberately included: content+ * streaming in (images loading, counters ticking) is also "not settled".+ */+ private fun treeSignature(out: JSONObject): String {+ val sb = StringBuilder(256)+ val nodes = ArrayList<Pair<Int, JSONObject>>()+ collectNodes(out, nodes, 0)+ for ((_, n) in nodes) {+ sb.append(n.optString("id")).append('|')+ .append(n.optString("bounds")).append('|')+ .append(n.optString("text")).append('\n')+ }+ return sb.toString()+ }++ private fun dumpOnce(verbose: Boolean): JSONObject {+ val active = service.rootInActiveWindow+ val out = if (active != null) {+ try {+ val stats = Stats()+ val body = buildNode(active, verbose, stats) ?: JSONObject().put("empty", true)+ withStats(body, active.packageName?.toString() ?: "", stats)+ } finally {+ active.recycle()+ }+ } else {+ JSONObject().put("empty", true).put("error", "no active window")+ }++ // Nothing usable and we may simply be looking at the wrong window: a+ // lot of Chinese apps host their real content in a second application+ // window (WeChat's mini-program / web-view pannes do exactly this).+ // Also entered when the "active" window turned out to be a status-bar+ // or other system shell during an app transition.+ if (!verbose && out.optInt("nodes_useful", 0) == 0) tryOtherWindows(out)+ // A status bar (SystemUI) can be the active window mid-transition and+ // hand us ~27 "useful" status icons — real nodes, wrong screen. The+ // user-facing app is always a better source when it has ANY content.+ // Only run this correction when we did NOT just come from+ // tryOtherWindows with a genuinely better window.+ if (!verbose && out.optString("pkg") == "com.android.systemui") tryOtherWindows(out)++ return out+ }++ /**+ * Fallback for the case above: walk every window we can see and keep the+ * best one. Only ever entered when the active window gave us nothing, so the+ * extra work is rare.+ *+ * Some apps can expose a transient or empty application window while another+ * window owns the visible content. We therefore inspect all accessible+ * windows before concluding that the current pane has no usable nodes.+ *+ * Whatever we reject, we report: a per-window summary goes out when nothing+ * was found, so the next person debugging this (or the model) can see which+ * window existed and which one had content, instead of a bare `{}`.+ */+ private fun tryOtherWindows(out: JSONObject) {+ val windows = runCatching { service.windows }.getOrNull()+ if (windows == null) {+ out.put("windows_seen", 0)+ return+ }+ // Whether the currently-chosen window belongs to a user app (as+ // opposed to SystemUI). Set BEFORE the loop so the SystemUI-rejection+ // rule below can reference it.+ val outPkgIsApp = out.optString("pkg", "") !in setOf("", "com.android.systemui")+ val detail = JSONArray()+ var best: JSONObject? = null+ var bestUseful = out.optInt("nodes_useful", 0)+ try {+ for (w in windows) {+ val stats = Stats()+ val root = runCatching { w.root }.getOrNull()+ // Read everything we need off the node *before* recycling it.+ val pkg = if (root != null) runCatching { root.packageName?.toString() }.getOrNull().orEmpty() else ""+ val body = if (root != null) {+ try {+ buildNode(root, false, stats)+ } catch (t: Throwable) {+ null+ } finally {+ root.recycle()+ }+ } else {+ null+ }+ detail.put(+ JSONObject()+ .put("title", w.title?.toString() ?: "")+ .put("type", windowTypeName(w.type))+ .put("active", w.isActive)+ .put("focused", w.isFocused)+ .put("layer", w.layer)+ .put("root", root != null)+ .put("nodes_total", stats.total)+ .put("nodes_useful", stats.useful),+ )+ // Accept ANY window type that hands us useful nodes, not just+ // TYPE_APPLICATION. WeChat hosts Moments' real content in a+ // system-type window (layer 1) while the "application" window+ // above it is an empty shell — filtering by type meant+ // throwing away the only tree on the screen, which is exactly+ // the "Moments taps are inaccurate" bug: the brain was forced+ // onto screenshot-guessing while a perfectly good element list+ // sat in a window we refused to read.+ //+ // Ranking: a user-app window (application type, or Moments'+ // system shell owned by the same non-SystemUI package) with+ // strictly more useful nodes beats anything SystemUI — the+ // status bar never stops "having content" (clock, icons), so+ // without this rule it wins every app transition.+ val isSystemUi = pkg == "com.android.systemui"+ // Never us. [net.kuafuai.andee.ui.hideFromAccessibility] should+ // already have left our overlays with nothing to find, but this+ // sweep is the one place where being wrong is invisible: it+ // runs only when the real pane has no tree, which is exactly+ // when our own toolbar is the sole thing on screen with+ // `useful > 0`, so it would win and the brain would be handed+ // an element list of our buttons. Two independent guards for+ // the same fault because the failure is silent and the cost is+ // a comparison. Our own Activities (Html/Scan/Look) are real+ // app windows and are NOT overlays, so they are caught here and+ // only here.+ if (pkg == service.packageName) continue+ val beatsBest = stats.useful > bestUseful &&+ (!isSystemUi || !outPkgIsApp)+ if (body != null && root != null && beatsBest) {+ bestUseful = stats.useful+ best = withStats(body, pkg, stats, source = "other_window(${windowTypeName(w.type)})")+ }+ }+ } finally {+ windows.forEach { runCatching { it.recycle() } }+ }+ val b = best+ if (b != null) {+ for (k in ArrayList(out.keys().asSequence().toList())) out.remove(k)+ for (k in b.keys()) out.put(k, b[k])+ // Which window this came from matters: tapping a node found in a+ // background window still works (bounds are screen coordinates),+ // but the brain should know it was not the focused one.+ out.put("windows_seen", windows.size)+ return+ }+ out.put("windows_seen", windows.size)+ out.put("windows_detail", detail)+ }++ private fun windowTypeName(type: Int): String = when (type) {+ AccessibilityWindowInfo.TYPE_APPLICATION -> "application"+ AccessibilityWindowInfo.TYPE_INPUT_METHOD -> "ime"+ AccessibilityWindowInfo.TYPE_SYSTEM -> "system"+ AccessibilityWindowInfo.TYPE_ACCESSIBILITY_OVERLAY -> "a11y_overlay"+ AccessibilityWindowInfo.TYPE_SPLIT_SCREEN_DIVIDER -> "split_divider"+ else -> "other($type)"+ }++ /**+ * Turn the pruned nested tree into (a) a fresh eid index and (b) the+ * element outline the brain actually reads — one line per element, in+ * document order, indented by depth:+ *+ * e17|LinearLayout||[500,570]+ * e18*|ImageView|#avatar|[62,523]+ * e19*|TextView|好友3|[181,509]+ * e24*|ImageView|#comment_btn|[950,621]+ *+ * `*` marks clickable. The bracket pair is the element's center in+ * 0-1000 RELATIVE coordinates — same space tap_screen speaks — computed+ * by us from getBoundsInScreen, never by the model. Even when the model+ * ignores e-numbers and taps by coordinate, it copies a system-computed+ * value instead of eyeballing the screenshot.+ *+ * ## Why it is indented, and why that replaced useful-first sorting+ *+ * This list used to be sorted text-first, then clickable, then+ * top-to-bottom, so that a context-limit truncation ate the junk tail.+ * On any feed that ordering is fatal. A 朋友圈 page is five posts of+ * {avatar, name, body, photos, time, ⋯}; sorting scattered each post+ * across the list and stacked twenty label-less `ImageView` lines at the+ * bottom, identical to one another. "Like 好友3's post" was then not+ * merely hard but *unanswerable* from the list — the comment button that+ * belongs to that post is indistinguishable from the other four, so the+ * model can only guess, which is precisely the mis-tap [GroundingLog] was+ * built to collect.+ *+ * Indentation says what the sort threw away, and says it adjacently: the+ * lines of one post are the lines under one container. Measured on a+ * five-post feed it costs about 20% more characters than the sorted flat+ * list; carrying the nested JSON instead — the other way to keep the+ * structure — costs 170% more for the same information, nearly all of it+ * braces and repeated field names. So the tree comes back, rendered+ * cheaply rather than as JSON.+ *+ * Containers earn their line here. An unlabeled `LinearLayout` is worth+ * nothing on its own, but it is the anchor the indentation hangs off, and+ * it is what separates one post from the next — which is why [trim] drops+ * leaves and never containers.+ *+ * The truncation argument that justified sorting is answered by+ * [MAX_ELEMENT_LINES] instead: we do the cutting ourselves, worst-first,+ * and say so in `elements_dropped`, rather than leaving it to whatever+ * the model's context happens to clip.+ */+ private fun rebuildIndexAndList(out: JSONObject) {+ synchronized(eidLock) { eidIndex.clear() }+ val w = screenW+ val h = screenH+ if (w <= 0 || h <= 0) return++ val nodes = ArrayList<Pair<Int, JSONObject>>()+ collectNodes(out, nodes, 0)++ // Pass 1: document order is the final order — no sort. Nodes whose+ // bounds we cannot parse are skipped; their children keep their own+ // depth, so the gap shows up as a missing indent level rather than as+ // a reparented subtree.+ val rows = ArrayList<Row>(nodes.size)+ for ((depth, n) in nodes) {+ val bounds = n.optString("bounds", "")+ if (bounds.isEmpty()) continue+ val parts = bounds.split(",")+ if (parts.size != 4) continue+ val l = parts[0].toIntOrNull() ?: continue+ val t = parts[1].toIntOrNull() ?: continue+ val r = parts[2].toIntOrNull() ?: continue+ val b = parts[3].toIntOrNull() ?: continue+ if (r <= l || b <= t) continue+ rows.add(+ Row(+ depth = depth,+ px = intArrayOf((l + r) / 2, (t + b) / 2),+ node = n,+ bounds = bounds,+ label = labelOf(n),+ clickable = n.optBoolean("clickable", false),+ container = n.optJSONArray("children") != null,+ )+ )+ }++ val dropped = trim(rows)++ // Pass 2: number in document order, index, render. Numbering comes+ // after the trim so the e-numbers the model reads are contiguous —+ // a gap would look like a stale list and send it back for a re-dump.+ val pkg = out.optString("pkg", "")+ val lines = ArrayList<String>(rows.size)+ for ((eidNum, row) in rows.withIndex()) {+ val eidLabel = "e$eidNum"+ val mark = if (row.clickable) "*" else ""+ val cls = row.node.optString("class", "View")+ val normCx = row.px[0] * NORM_MAX / w+ val normCy = row.px[1] * NORM_MAX / h+ synchronized(eidLock) {+ eidIndex[eidNum] = EidEntry(+ px = row.px,+ pkg = pkg,+ viewId = row.node.optString("id").ifEmpty { null },+ bounds = row.bounds,+ )+ }+ val indent = " ".repeat(minOf(row.depth, MAX_INDENT))+ lines.add("$indent$eidLabel$mark|$cls|${row.label}|[$normCx,$normCy]")+ }++ // Replace the nested tree with the outline — the JSON was only ever+ // scaffolding for this. Keep the stats fields, drop the bulk.+ val keep = setOf(+ "ts", "pkg", "nodes_total", "nodes_kept", "nodes_useful", "surface",+ "source", "windows_seen", "windows_detail", "settling", "settle_waited_ms",+ "error", "empty",+ )+ val rootKeys = ArrayList(out.keys().asSequence().toList())+ for (k in rootKeys) if (k !in keep) out.remove(k)+ if (lines.isNotEmpty()) out.put("elements", lines.joinToString("\n"))+ if (dropped > 0) out.put("elements_dropped", dropped)+ }++ /** One candidate line, before numbering. */+ private class Row(+ val depth: Int,+ val px: IntArray,+ val node: JSONObject,+ val bounds: String,+ val label: String,+ val clickable: Boolean,+ val container: Boolean,+ )++ /**+ * Bring [rows] under [MAX_ELEMENT_LINES], worst-first, and report how many+ * went. Only decorative leaves are eligible — no label, nothing to tap,+ * no children to group — because they are the one kind of line whose+ * removal costs the model nothing. Containers are never dropped even+ * though they are usually label-less: the indent level they provide is+ * what keeps two adjacent posts from reading as one.+ *+ * If that is not enough the tail goes, which on a screen read in document+ * order means the bottom of it — recoverable by scrolling, unlike a+ * mis-association, which is not recoverable at all.+ */+ private fun trim(rows: ArrayList<Row>): Int {+ if (rows.size <= MAX_ELEMENT_LINES) return 0+ val before = rows.size+ var over = rows.size - MAX_ELEMENT_LINES+ val it = rows.listIterator(rows.size)+ while (it.hasPrevious() && over > 0) {+ val r = it.previous()+ if (!r.container && r.label.isEmpty() && !r.clickable) {+ it.remove()+ over--+ }+ }+ if (rows.size > MAX_ELEMENT_LINES) {+ rows.subList(MAX_ELEMENT_LINES, rows.size).clear()+ }+ return before - rows.size+ }++ /**+ * What to print in the label column: the text, else the description, else+ * the tail of the view id with a `#` on it.+ *+ * The id fallback is the fix for a whole class of blind spot. 朋友圈's+ * `⋯`, the avatars and the photos are all `ImageView`s with neither text+ * nor description, so the column came out empty and the model was told+ * (by HARD RULE 2, in so many words) to tap them by number without+ * knowing what they were. The node carried+ * `com.tencent.mm:id/comment_btn` the whole time — we were dropping it+ * when building the line. The `#` is not decoration: it marks the value+ * as a resource name rather than something written on the screen, which+ * the model would otherwise be entitled to read back to the user.+ */+ private fun labelOf(n: JSONObject): String {+ n.optString("text").takeIf { it.isNotEmpty() }?.let { return oneLine(it) }+ n.optString("desc").takeIf { it.isNotEmpty() }?.let { return oneLine(it) }+ n.optString("id").takeIf { it.isNotEmpty() }+ ?.let { return "#" + oneLine(it.substringAfterLast('/')) }+ return ""+ }++ /**+ * The outline is one element per line with `|` between fields, so a label+ * containing either character tears the row in half — and a multi-line+ * TextView (a 朋友圈 body, a long notification) contains newlines as a+ * matter of course. Neither was escaped before; the rows it produced were+ * unparseable and the model had to guess where the next element started.+ */+ private fun oneLine(s: String): String =+ s.replace('\n', ' ').replace('\r', ' ').replace('|', '/').trim()++ /** Collect every node in the (pruned) tree, depth-first, with its depth. */+ private fun collectNodes(node: JSONObject, sink: MutableList<Pair<Int, JSONObject>>, depth: Int) {+ sink.add(depth to node)+ val kids = node.optJSONArray("children") ?: return+ for (i in 0 until kids.length()) {+ val kid = kids.optJSONObject(i) ?: continue+ collectNodes(kid, sink, depth + 1)+ }+ }++ /** Main-thread code only; see [Stats] for why these numbers matter. */+ private fun withStats(+ body: JSONObject,+ pkg: String,+ stats: Stats,+ source: String? = null,+ ): JSONObject {+ body.put("pkg", pkg)+ body.put("nodes_total", stats.total)+ body.put("nodes_kept", stats.kept)+ body.put("nodes_useful", stats.useful)+ stats.surface?.let { body.put("surface", shortenClass(it, false)) }+ if (source != null) body.put("source", source)+ return body+ }++ /**+ * What one dump pass saw, so the caller can say why a tree is useless+ * instead of just handing back `{}`:+ *+ * [total] — nodes visited, before any pruning+ * [kept] — nodes that survived pruning+ * [useful] — nodes carrying something you can act on (text / desc / id)+ * [surface]— first self-rendering view found, if any+ */+ private class Stats {+ var total = 0+ var kept = 0+ var useful = 0+ var surface: String? = null+ }++ /**+ * Tell the brain what it is looking at when there is nothing to act on, in+ * its language rather than ours: not "empty tree" but "this is a surface you+ * have to read with your eyes". The empty-tree branch carries the+ * generic_bridge NO_TREE policy verbatim in spirit: prefer back/home over+ * blind coordinate taps — a wrong exit costs one tap, a wrong tap can cost+ * the task.+ *+ * Every branch here fires exactly where [dumpUiTree] attaches a screenshot+ * unasked, so the hints point at the image already in hand rather than+ * sending the brain off to `take_screenshot` for a picture of the same+ * screen it was just given. Called after the shot block for that reason —+ * if the capture failed, say so instead of promising an image that is not+ * there.+ *+ * The branches are not all the same verdict. Only the first three are+ * genuinely empty; the last has nodes that simply have no names, and since+ * [rebuildIndexAndList] started rendering nesting and centres for every+ * node rather than sorting the labelled ones to the top, that case still+ * has a usable outline and must not be told to tap by coordinate.+ *+ * [reviveNote] is appended to the first two only — the two that mean "no+ * nodes AT ALL". That is the whole discrimination: a self-drawn pane still+ * reports its containers, so a mini-program never reaches those branches+ * and never nags the user about restarting anything.+ */+ private fun annotate(out: JSONObject, verbose: Boolean) {+ if (verbose) return+ val total = out.optInt("nodes_total", 0)+ val useful = out.optInt("nodes_useful", 0)+ // "Look at the attached image" vs "go fetch one" — never both.+ val eyes = if (out.has("shot")) {+ "the screenshot attached to this response (do not call take_screenshot " ++ "for the same screen)"+ } else {+ "take_screenshot"+ }+ val revive = reviveNote()+ when {+ // Ours is the window in front, and we hid it from ourselves on+ // purpose — so the zero nodes are expected, not a stale service.+ // First because it otherwise falls into the reviveNote branches and+ // sends the user off to toggle a setting that is working fine.+ total == 0 && net.kuafuai.andee.ui.OwnCard.showing ->+ out.put(+ "hint",+ "The user has one of Andee's own cards open (settings or the vault), so " ++ "there is no app tree to read — the card is deliberately invisible to " ++ "accessibility, including to us. Nothing is broken. Wait for them to " ++ "close it, or say in plain words that you need the settings card closed " ++ "before you can carry on. Do not tap blindly: the taps would land on " ++ "whatever is behind the card.",+ )++ out.has("error") && total == 0 ->+ out.put(+ "hint",+ "No window was handed to accessibility. Either the app is mid-transition " ++ "(retry once) or this window refuses to expose itself. Judge by $eyes." ++ revive,+ )++ useful == 0 && total == 0 ->+ // An empty root, not a filtered-away tree — and every other+ // accessible window was empty too (see [tryOtherWindows]).+ // Keep the conclusion scoped to this pane; never write off the+ // whole app because one screen has no semantic tree.+ out.put(+ "hint",+ "No accessibility nodes anywhere: the active window's root is empty and " ++ "none of the ${out.optInt("windows_seen", 0)} other windows held content " ++ "either. Almost always a pane that draws its own pixels — a " ++ "mini-program, a WebView article, a game canvas — or, if `settling` is " ++ "set, a screen still mid-transition. NO_TREE policy: judge by $eyes, " ++ "and prefer back / home to exit and retry via a " ++ "different route rather than blind coordinate taps. This is a property " ++ "of this pane, not of the app: the same app's other screens very likely " ++ "dump fine, so do not stop calling this tool on them." ++ revive,+ )++ useful == 0 && out.has("surface") ->+ out.put(+ "hint",+ "Everything actionable here is inside ${out.optString("surface")}, which draws " ++ "its own content and therefore has no accessibility nodes — ever, on any flag. " ++ "NO_TREE policy: read the screen from $eyes, and prefer " ++ "back / home to exit and re-enter via a different route over blind " ++ "coordinate taps.",+ )++ useful == 0 && total > 0 ->+ // Not the same dead end as the branches above: the nodes exist,+ // they are merely nameless, so the outline still carries their+ // nesting and their centres. Aiming at an e-number there is+ // strictly better than a coordinate tap — the device resolves+ // the element's real centre, and the guard exempts e-taps.+ out.put(+ "hint",+ "$total nodes but none with text, desc or id: this app draws its own content, " ++ "so every line in `elements` has an empty label. Read them anyway — the " ++ "indentation still groups them and each line still carries its centre. " ++ "Match what you see in $eyes against the outline and tap the `*` line " ++ "sitting at that spot by e-number; drop to a coordinate tap only if no " ++ "line covers it.",+ )+ }+ }++ /**+ * The sentence appended when an app that **has worked on this device** is+ * suddenly handing back no nodes at all — see [goodDumps].+ *+ * Deliberately evidence and not an instruction. It states the two facts+ * only the device knows (this package read fine, that long ago) and what+ * the human would have to do; whether that is worth interrupting the user+ * over is the brain's call, because only the brain knows what the user+ * asked for and how far along it is. Hard-coding "if pkg == WeChat and the+ * tree is empty, tell the user to restart" would fire on every mini-program+ * WeChat hosts and train the user to ignore us.+ *+ * Empty string, not null, so the callers can concatenate unconditionally.+ *+ * Identity is best-effort by design: an empty dump usually has no `pkg` of+ * its own, so it falls back to the last package we read and then to the+ * accessibility event stream. Both are "the app we believe is in front" and+ * both can be one screen stale — which is why the note names the package it+ * is talking about instead of saying "this app", so a wrong guess is+ * visible to the brain rather than silently misattributed.+ *+ * Covers one shape of the fault and not the other. A stale connection+ * normally shows up as `rootInActiveWindow == null` — no nodes at all —+ * which is where this fires. If instead it were to hand back a root that+ * prunes down to nothing *useful*, that lands in the nameless-nodes branch+ * and gets no note, because there it is indistinguishable from a+ * mini-program: same package, same known-good history, same counts. Better+ * silent than nagging the user every time WeChat opens a mini-program. If+ * the other shape turns out to happen on a real device, it needs a sharper+ * signal than this one (`windows_seen == 0` is the obvious candidate) —+ * not a looser condition here.+ */+ private fun reviveNote(): String {+ val pkg = lastDumpPkg ?: DeviceState.foregroundPkg ?: return ""+ if (pkg == service.packageName) return ""+ val seen = goodDumps[pkg] ?: return ""+ val mins = (System.currentTimeMillis() - seen) / 60_000+ val ago = if (mins < 1) "less than a minute ago" else "$mins minutes ago"+ return " THIS APP HAS WORKED HERE BEFORE: `$pkg` gave a usable " ++ "element list $ago, and is now returning no accessibility nodes at all. " ++ "A pane that draws its own pixels does not do that — it still reports its " ++ "containers. Zero nodes from an app that was readable is the signature of " ++ "our accessibility connection having gone stale, and no tool call can " ++ "repair it: only the person holding the tablet can, by opening 设置 → " ++ "无障碍 (Settings → Accessibility), switching Andee off and back on. That " ++ "takes about three seconds. Retry this tool once first in case the screen " ++ "was merely mid-transition; if it is still empty, say so to the user in " ++ "plain words and wait for them. Taps and swipes still work and you still " ++ "have the screenshot, so you are not stuck — but grinding on blind " ++ "coordinates is worth less than asking for a three-second toggle."+ }++ private fun runGesture(gesture: GestureDescription, timeoutMs: Long): JSONObject {+ val latch = CountDownLatch(1)+ var completed = false+ val ok = service.dispatchGesture(+ gesture,+ object : AccessibilityService.GestureResultCallback() {+ override fun onCompleted(g: GestureDescription?) {+ completed = true; latch.countDown()+ }++ override fun onCancelled(g: GestureDescription?) {+ completed = false; latch.countDown()+ }+ },+ null,+ )+ if (!ok) throw IllegalStateException("dispatchGesture returned false")+ latch.await(timeoutMs, TimeUnit.MILLISECONDS)+ return JSONObject().put("completed", completed)+ }++ /**+ * Build one node's JSON. Bottom-up pruning:+ * 1. Recurse into children first (so we know how many survive).+ * 2. `hasReason` = has id/text/desc, or is clickable/scrollable/focused/+ * checked, or is explicitly disabled, or is a render surface.+ * 3. keep if hasReason, OR if it has ≥ 2 surviving children (it's a+ * grouping container that tells the LLM "these belong together").+ * 4. hoist if it has exactly 1 surviving child and no reason — return+ * that child in place of self. Collapses deep layout wrapper chains.+ * 5. drop if it has 0 children and no reason — decorative leaves like+ * Spacers, dividers, dots.+ *+ * `verbose=true` disables all of this and emits every node with every+ * field, matching pre-pruning behavior.+ *+ * [stats] collects what was seen vs what survived, so the caller can tell+ * "there is no tree" apart from "we threw the tree away".+ */+ private fun buildNode(node: AccessibilityNodeInfo, verbose: Boolean, stats: Stats? = null): JSONObject? {+ if (!verbose && !node.isVisibleToUser) return null+ val bounds = Rect().also { node.getBoundsInScreen(it) }+ if (!verbose && (bounds.width() <= 0 || bounds.height() <= 0)) return null+ val st = stats+ if (st != null) st.total++++ val kids = mutableListOf<JSONObject>()+ for (i in 0 until node.childCount) {+ val child = node.getChild(i) ?: continue+ try {+ buildNode(child, verbose, stats)?.let { kids.add(it) }+ } finally {+ child.recycle()+ }+ }++ val id = node.viewIdResourceName+ val text = node.text?.toString()+ val desc = node.contentDescription?.toString()+ val clickable = node.isClickable+ val focused = node.isFocused+ val scrollable = node.isScrollable+ val checked = node.isChecked+ val selected = node.isSelected+ val enabled = node.isEnabled+ val cls = node.className?.toString()+ val surface = if (isRenderSurface(cls)) cls else null++ val hasReason = !id.isNullOrEmpty() || !text.isNullOrEmpty() || !desc.isNullOrEmpty()+ || clickable || focused || scrollable || checked || selected || !enabled+ || surface != null++ if (!verbose) {+ if (!hasReason && kids.isEmpty()) return null // drop decorative leaf+ if (!hasReason && kids.size == 1) return kids[0] // hoist single child+ }++ if (st != null) {+ st.kept+++ if (!id.isNullOrEmpty() || !text.isNullOrEmpty() || !desc.isNullOrEmpty()) st.useful+++ if (surface != null && st.surface == null) st.surface = surface+ }++ val obj = JSONObject()+ if (!cls.isNullOrEmpty()) obj.put("class", shortenClass(cls, verbose))+ // Surfaced even though it carries no text: it is the one thing that+ // tells the brain "don't wait for a tree here, you need the pixels".+ if (surface != null) obj.put("opaque", true)+ if (!id.isNullOrEmpty()) obj.put("id", id)+ if (!text.isNullOrEmpty()) obj.put("text", text)+ if (!desc.isNullOrEmpty()) obj.put("desc", desc)+ obj.put("bounds", "${bounds.left},${bounds.top},${bounds.right},${bounds.bottom}")+ if (verbose || clickable) obj.put("clickable", clickable)+ if (verbose || focused) obj.put("focused", focused)+ if (verbose || scrollable) obj.put("scrollable", scrollable)+ if (verbose || checked) obj.put("checked", checked)+ if (verbose || selected) obj.put("selected", selected)+ if (verbose || !enabled) obj.put("enabled", enabled)++ if (kids.isNotEmpty()) {+ val arr = JSONArray()+ for (k in kids) arr.put(k)+ obj.put("children", arr)+ }+ return obj+ }+++ /**+ * Views that draw their own content and therefore have no accessibility+ * children: a WebView page, a Flutter/X5 pane (WeChat mini-programs), a+ * game canvas, a video surface, Compose Canvas. Nothing inside them will+ * ever appear in a UI tree — no service flag changes that.+ *+ * We bother because knowing *that* is the difference between the brain+ * retrying get_screen_element forever and switching to pixels.+ */+ private fun isRenderSurface(cls: String?): Boolean {+ if (cls.isNullOrEmpty()) return false+ return cls.contains("WebView", ignoreCase = true) // incl. X5 / XWeb / WKWebView-ish names+ || cls.contains("SurfaceView", ignoreCase = true)+ || cls.contains("TextureView", ignoreCase = true)+ || cls.contains("GLSurfaceView", ignoreCase = true)+ || cls.contains("Flutter", ignoreCase = true)+ || cls.contains("unity", ignoreCase = true)+ || cls.contains("cocos", ignoreCase = true)+ || cls.contains("CanvasView", ignoreCase = true)+ }++ /**+ * Strip the common android.* / androidx.* / android.view.* prefixes off+ * a class name so `android.widget.FrameLayout` becomes `FrameLayout`.+ * In verbose mode keep the fully-qualified name.+ */+ private fun shortenClass(cls: String?, verbose: Boolean): String? {+ if (cls == null) return null+ if (verbose) return cls+ val prefixes = arrayOf(+ "android.widget.", "android.view.", "android.webkit.",+ "androidx.compose.ui.platform.",+ "androidx.recyclerview.widget.", "androidx.viewpager.widget.",+ "androidx.appcompat.widget.", "androidx.constraintlayout.widget.",+ )+ for (p in prefixes) if (cls.startsWith(p)) return cls.substring(p.length)+ return cls+ }++ companion object {+ /**+ * ADBKeyboard 的广播接口。action 是裸字符串,没有包名前缀 ——+ * 加了前缀就匹配不上它注册的 filter,广播静默消失。+ */+ private const val ADB_KEYBOARD_PKG = "com.android.adbkeyboard"+ private const val ADB_INPUT_TEXT = "ADB_INPUT_TEXT"+ private const val ADB_CLEAR_TEXT = "ADB_CLEAR_TEXT"+ private const val READBACK_POLL_MS = 200L+ private const val READBACK_BUDGET_MS = 1500L++ /**+ * Toolbar labels the IME uses for its clipboard feature.+ *+ * Not translated, and not translatable: these are matched against the+ * labels the *user's* keyboard draws, which come from the keyboard's+ * own language, not from this app's setting. The list carries both+ * scripts for that reason — dropping the Chinese half because the+ * interface is English would break clipboard pasting on every Chinese+ * keyboard. See `NotificationRelayService.CN_CALL_KEYWORDS`.+ */+ private val CLIPBOARD_LABELS = listOf("剪贴板", "Clipboard", "clipboard", "粘贴", "Paste")+ private val PASTE_LABELS = listOf("粘贴", "Paste", "paste", "Paste as plain text")+ private const val SCROLL_TAG = "ScrollDir"+ private const val SETTLE_POLL_MS = 250L+ private const val SETTLE_BUDGET_MS = 1500L++ /**+ * Stability pass knobs — see the second loop in [dumpUiTree].+ * STABLE_POLL_MS is above a frame pair (2×16ms) but below a typical+ * list-settle ripple, so a still screen agrees on the first retry;+ * STABLE_MAX_TRIES caps the worst case (endless animation — carousels+ * never settle) at ~1s extra, after which `still_moving` tells the+ * brain the truth rather than pretending the tree is final.+ */+ private const val STABLE_POLL_MS = 300L+ private const val STABLE_MAX_TRIES = 3+ private const val QUIET_WINDOW_MS = 1200L++ /** The relative coordinate space the brain speaks: 0..1000 both axes. */+ private const val NORM_MAX = 1000++ /** Screenshot long side after downscale (generic_bridge value). */+ private const val SHOT_MAX_SIDE = 1280++ /**+ * Hard ceiling on outline lines. Generous — it is a safety valve for a+ * pathological list view, not a budget. Below it nothing is dropped,+ * because on this screen the model cannot tell "there is no such+ * control" from "we did not print it".+ */+ private const val MAX_ELEMENT_LINES = 300++ /**+ * Deepest indent level rendered; beyond it lines stay at this column.+ * Hoisting already collapses wrapper chains, so real depth is small —+ * this only stops a pathological hierarchy from spending more+ * characters on leading spaces than on content.+ */+ private const val MAX_INDENT = 10++ /** Gestures closer than this count as "the same spot" — catches+ * re-tap-with-jitter, not just exact duplicates. */+ private const val LOOP_RADIUS_PX = 120++ /** Allowed same-spot attempts before the guard fires. 3 means the+ * 4th is refused. */+ private const val LOOP_MAX_TRIES = 3++ /** Same-spot counter window; older than this is a fresh task. */+ private const val LOOP_WINDOW_MS = 300_000L++ private const val LOOP_HELP =+ "Stuck-loop guard: this same screen area has been touched more than " ++ "3 times within 5 minutes. The gesture was NOT executed. " ++ "Do not retry the same tap. Take a fresh screenshot or get_screen_element, " ++ "pick a genuinely different approach, and if the goal still cannot " ++ "be reached, stop and ask the user for help instead of continuing."+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/text/Markdown.kt b/app/src/main/java/net/kuafuai/andee/text/Markdown.ktnew file mode 100644index 0000000..fe4d596--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/text/Markdown.kt@@ -0,0 +1,62 @@+package net.kuafuai.andee.text++/**+ * Flatten the brain's markdown into something worth speaking aloud.+ *+ * The brain answers in markdown (`**加粗**`, `## 标题`, `- 列表`, links), which+ * TTS either reads out as punctuation or turns into strange pauses, and which+ * clutters the one-line subtitle band.+ *+ * Deliberately leaves `_` alone. Underscore-italic is vanishingly rare in the+ * Chinese answers this thing actually produces, while snake_case is everywhere+ * in the progress subtitles — every body tool is named `tap_screen`,+ * `get_screen_element`, `submit_input`. Treating `_` as emphasis would silently read+ * those as "tapscreen".+ */++// 整行构造连行尾换行一起吃掉(末尾的 `\n?`),否则删掉的行会留下一个空行,+// 字幕多一截空白、TTS 多一次停顿。+// 分隔线必须在强调之前处理:`***` 整行既是分隔线也能被 *强调* 正则吃掉。+private val RULE = Regex("""^[ \t]*([-*])([ \t]*\1){2,}[ \t]*$\n?""", RegexOption.MULTILINE)+private val CODE_FENCE = Regex("""^[ \t]*`{3,}.*$\n?""", RegexOption.MULTILINE)+// 图片在链接之前: 是 [text](url) 的超集,反了会剩一个孤零零的 `!`。+private val IMAGE = Regex("""!\[[^\]]*]\([^)]*\)""")+private val LINK = Regex("""\[([^\]]*)]\([^)]*\)""")+private val HEADING = Regex("""^[ \t]{0,3}#{1,6}[ \t]+""", RegexOption.MULTILINE)+private val QUOTE = Regex("""^[ \t]{0,3}>+[ \t]?""", RegexOption.MULTILINE)+private val BULLET = Regex("""^[ \t]*[-*+][ \t]+""", RegexOption.MULTILINE)+private val ORDERED = Regex("""^[ \t]*\d+[.)][ \t]+""", RegexOption.MULTILINE)+// 表格分隔行(|---|:--:|)整行去掉;正文行的竖线换空格,免得念成一串停顿。+private val TABLE_RULE = Regex("""^[ \t]*\|?[ \t:|-]*\|[ \t:|-]*$\n?""", RegexOption.MULTILINE)+private val EMPHASIS = Regex("""\*{1,3}([^*\n]+)\*{1,3}""")+private val INLINE_CODE = Regex("""`+([^`\n]+)`+""")+private val BLANK_RUN = Regex("""\n{3,}""")+// 删掉标记会留下成片空格(表格的竖线尤其明显),压成一个,再清掉每行两端。+private val SPACE_RUN = Regex("""[ \t]{2,}""")+private val LINE_EDGE = Regex("""^[ \t]+|[ \t]+$""", RegexOption.MULTILINE)++/**+ * Returns [text] with markdown markup removed. Falls back to the original when+ * stripping would leave nothing — an answer made entirely of markup is more+ * likely a bug in this function than an empty answer from the brain.+ */+fun stripMarkdown(text: String): String {+ if (text.isEmpty()) return text+ var s = text+ s = CODE_FENCE.replace(s, "")+ s = RULE.replace(s, "")+ s = TABLE_RULE.replace(s, "")+ s = IMAGE.replace(s, "")+ s = LINK.replace(s, "$1")+ s = HEADING.replace(s, "")+ s = QUOTE.replace(s, "")+ s = BULLET.replace(s, "")+ s = ORDERED.replace(s, "")+ s = EMPHASIS.replace(s, "$1")+ s = INLINE_CODE.replace(s, "$1")+ s = s.replace('|', ' ')+ s = SPACE_RUN.replace(s, " ")+ s = LINE_EDGE.replace(s, "")+ s = BLANK_RUN.replace(s, "\n\n").trim()+ return s.ifEmpty { text }+}diff --git a/app/src/main/java/net/kuafuai/andee/tts/HuoshanTts.kt b/app/src/main/java/net/kuafuai/andee/tts/HuoshanTts.ktnew file mode 100644index 0000000..cacd06e--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/tts/HuoshanTts.kt@@ -0,0 +1,420 @@+package net.kuafuai.andee.tts++import android.util.Log+import net.kuafuai.andee.config.VoiceConfig+import org.java_websocket.client.WebSocketClient+import org.java_websocket.handshake.ServerHandshake+import org.json.JSONObject+import java.io.ByteArrayOutputStream+import java.net.URI+import java.nio.ByteBuffer+import java.util.UUID+import java.util.concurrent.TimeUnit+import javax.net.ssl.SSLContext++/**+ * ByteDance 火山 bidirectional TTS client (seed-tts-2.0).+ * Direct Kotlin port of electron-vite's src/main/ttsBidi.ts.+ *+ * Session lifecycle (event-driven):+ * client → server: StartConnection (1)+ * server → client: ConnectionStarted (50)+ * client → server: StartSession (100) [carries speaker + audio_params]+ * server → client: SessionStarted (150) → we are "ready"+ * client → server: TaskRequest (200) [carries a chunk of text; may repeat]+ * server → client: TTSSentenceStart/End (350/351) + AudioOnlyServer (audio bytes)+ * client → server: FinishSession (102) [tell server no more text]+ * server → client: SessionFinished (152) → we cleanup+ * client → server: FinishConnection (2)+ * server → client: ConnectionFinished (52) → ws closes+ *+ * Client frame layout for events:+ * header(4) + event(int32) + [sessionId size(u32) + bytes]? + payload size(u32) + payload+ * (session_id is omitted for connection-level events: 1, 2, 50, 51, 52)+ */+class HuoshanTts(+ private val config: VoiceConfig,+ private val listener: Listener,+) {+ interface Listener {+ fun onOpen(sampleRate: Int)+ fun onAudio(pcm: ByteArray)+ fun onSentenceStart(text: String)+ fun onSentenceEnd(text: String)+ fun onDone()+ fun onError(msg: String)+ fun onClose(reason: CloseReason)+ }++ enum class CloseReason { USER, SERVER, ERROR }+ private enum class State { IDLE, CONNECTING, SESSION_OPENING, READY, FINISHING, CLOSED }++ private var ws: WebSocketClient? = null++ /** See [net.kuafuai.andee.asr.HuoshanAsr]'s field of the same name. */+ @Volatile+ private var handshakeRefusal: String? = null++ @Volatile+ private var opened = false++ @Volatile+ private var state: State = State.IDLE+ private var sessionId: String = ""+ private var connectId: String = ""+ private var failReason: CloseReason = CloseReason.SERVER+ private var baseReqParams: JSONObject? = null++ @Volatile+ private var audioFrames: Int = 0++ @Volatile+ private var pendingFinishTrigger: (() -> Unit)? = null++ fun open() {+ check(state == State.IDLE) { "already open (state=$state)" }++ state = State.CONNECTING+ sessionId = UUID.randomUUID().toString()+ connectId = UUID.randomUUID().toString()++ val headers = mapOf(+ "X-Api-Key" to config.apiKey,+ "X-Api-Resource-Id" to config.ttsResourceId,+ "X-Api-Connect-Id" to connectId,+ "X-Control-Require-Usage-Tokens-Return" to "*",+ )++ Log.d(+ TAG,+ "open: ep=${config.ttsEndpoint} speaker=${config.ttsSpeaker} " + "sessionId=$sessionId connectId=$connectId"+ )++ val uri = URI(config.ttsEndpoint)+ val c = object : WebSocketClient(uri, headers) {+ override fun onOpen(handshake: ServerHandshake?) {+ opened = true+ Log.d(TAG, "ws open status=${handshake?.httpStatus}")+ baseReqParams = JSONObject().apply {+ put("speaker", config.ttsSpeaker)+ put(+ "audio_params",+ JSONObject().put("format", "pcm").put("sample_rate", config.ttsSampleRate)+ )+ }+ sendEvent(EVT_START_CONNECTION, "{}".toByteArray(Charsets.UTF_8))+ state = State.SESSION_OPENING+ }++ override fun onMessage(message: String?) { /* text frames ignored */+ }++ override fun onMessage(bytes: ByteBuffer?) {+ bytes ?: return+ val raw = ByteArray(bytes.remaining())+ bytes.get(raw)+ handleMessage(raw)+ }++ override fun onClose(code: Int, reason: String?, remote: Boolean) {+ if (!opened) handshakeRefusal = reason?.takeIf { it.isNotBlank() } ?: "code $code"+ Log.d(TAG, "onClose code=$code reason=$reason remote=$remote frames=$audioFrames")+ state = State.CLOSED+ listener.onClose(failReason)+ }++ override fun onError(ex: Exception?) {+ Log.e(TAG, "onError", ex)+ failReason = CloseReason.ERROR+ listener.onError("ws error: ${ex?.message ?: ex?.javaClass?.simpleName}")+ }+ }+ if (uri.scheme.equals("wss", ignoreCase = true)) {+ c.setSocketFactory(SSLContext.getDefault().socketFactory)+ }+ ws = c+ val ok = c.connectBlocking(5, TimeUnit.SECONDS)+ if (!ok) {+ val refusal = handshakeRefusal+ // See HuoshanAsr for why a refusal must not be reported as a+ // timeout: both come back from connectBlocking as false, and only+ // one of them is about the network.+ throw IllegalStateException(+ if (refusal == null) {+ "TTS ws connect timeout"+ } else {+ "TTS ws refused: $refusal (check the 火山 API Key in ⚙)"+ },+ )+ }+ }++ /** Push a chunk of text to be spoken. Can be called multiple times. */+ fun push(text: String) {+ if (state != State.READY) {+ Log.w(TAG, "push dropped, state=$state")+ return+ }+ val trimmed = text.trim()+ if (trimmed.isEmpty()) return+ // Skip pure punctuation like ttsBidi.ts does — server would emit an empty sentence.+ if (trimmed.matches(Regex("^[\\s\\p{P}\\p{S}]*$"))) {+ Log.d(TAG, "push skipped (no content): $text")+ return+ }+ val body = JSONObject().put("event", EVT_TASK_REQUEST).put(+ "req_params", JSONObject(baseReqParams?.toString() ?: "{}").put("text", text)+ )+ sendEvent(EVT_TASK_REQUEST, body.toString().toByteArray(Charsets.UTF_8))+ }++ /**+ * Tell server "no more text incoming, please finalize synthesis".+ *+ * TCP preserves order of push→finish frames on the same connection, so no race.+ * The ttsBidi.ts "wait for first audio" trick is only needed when you want to+ * confirm the server ingested pushes before finalizing — but with 火山's+ * seed-tts-2.0, some speakers don't emit audio until they see FinishSession,+ * causing a mutual wait deadlock. Send immediately.+ */+ fun finish() {+ if (state == State.CLOSED || state == State.IDLE) return+ if (state != State.READY && state != State.SESSION_OPENING) return+ state = State.FINISHING+ Log.d(TAG, "FinishSession (immediate)")+ sendEvent(EVT_FINISH_SESSION, "{}".toByteArray(Charsets.UTF_8))+ }++ fun cancel() {+ if (state == State.CLOSED || state == State.IDLE) return+ failReason = CloseReason.USER+ try {+ if (state == State.READY || state == State.SESSION_OPENING) {+ sendEvent(EVT_CANCEL_SESSION, "{}".toByteArray(Charsets.UTF_8))+ }+ } catch (_: Throwable) {+ }+ try {+ ws?.close()+ } catch (_: Throwable) {+ }+ }++ // ---- Internal ----++ private fun sendEvent(event: Int, payload: ByteArray) {+ val c = ws ?: return+ val frame = buildClientEventFrame(event, sessionId, payload)+ c.send(frame)+ }++ private fun handleMessage(bytes: ByteArray) {+ val msg = try {+ parseMessage(bytes)+ } catch (t: Throwable) {+ listener.onError("parse failed: ${t.message}")+ return+ }+ if (msg.type != MSG_SERVER_AUDIO_ONLY) {+ Log.d(+ TAG,+ "recv type=0x${Integer.toHexString(msg.type)} event=${msg.event} " + "payload=${msg.payload.size}B " + if (msg.type == MSG_SERVER_FULL_RESPONSE && msg.payload.size < 200) {+ String(msg.payload, Charsets.UTF_8)+ } else ""+ )+ }++ if (msg.type == MSG_SERVER_AUDIO_ONLY && msg.payload.isNotEmpty()) {+ audioFrames += 1+ listener.onAudio(msg.payload)+ // First audio → server has ingested our push → safe to send FinishSession+ val trigger = pendingFinishTrigger+ if (trigger != null) {+ pendingFinishTrigger = null+ trigger()+ }+ return+ }+ if (msg.type == MSG_SERVER_ERROR) {+ failReason = CloseReason.ERROR+ listener.onError(+ "TTS server error code=${msg.errorCode} " + "msg=${+ String(+ msg.payload,+ Charsets.UTF_8+ )+ }"+ )+ return+ }+ if (msg.type != MSG_SERVER_FULL_RESPONSE) return++ when (msg.event) {+ EVT_CONNECTION_STARTED -> {+ // Connection ready → open session+ val body = JSONObject().put("event", EVT_START_SESSION)+ .put("req_params", baseReqParams ?: JSONObject())+ sendEvent(EVT_START_SESSION, body.toString().toByteArray(Charsets.UTF_8))+ }++ EVT_CONNECTION_FAILED -> {+ failReason = CloseReason.ERROR+ listener.onError("ConnectionFailed: ${String(msg.payload, Charsets.UTF_8)}")+ }++ EVT_SESSION_STARTED -> {+ state = State.READY+ listener.onOpen(config.ttsSampleRate)+ }++ EVT_TTS_SENTENCE_START -> {+ val text = runCatching {+ JSONObject(String(msg.payload, Charsets.UTF_8)).optString("text", "")+ }.getOrDefault("")+ listener.onSentenceStart(text)+ }++ EVT_TTS_SENTENCE_END -> {+ val text = runCatching {+ JSONObject(String(msg.payload, Charsets.UTF_8)).optString("text", "")+ }.getOrDefault("")+ listener.onSentenceEnd(text)+ }++ EVT_TTS_RESPONSE -> { /* server sometimes uses this for audio; unhandled */+ }++ EVT_SESSION_FINISHED -> {+ listener.onDone()+ sendEvent(EVT_FINISH_CONNECTION, "{}".toByteArray(Charsets.UTF_8))+ }++ EVT_SESSION_CANCELED -> {+ sendEvent(EVT_FINISH_CONNECTION, "{}".toByteArray(Charsets.UTF_8))+ }++ EVT_SESSION_FAILED -> {+ failReason = CloseReason.ERROR+ listener.onError("SessionFailed: ${String(msg.payload, Charsets.UTF_8)}")+ }++ EVT_CONNECTION_FINISHED -> {+ try {+ ws?.close()+ } catch (_: Throwable) {+ }+ }+ }+ }++ // ---- Wire format ----++ private data class ParsedMessage(+ val type: Int,+ val flag: Int,+ val event: Int,+ val errorCode: Int,+ val payload: ByteArray,+ )++ private fun buildClientEventFrame(+ event: Int, sessionId: String, payload: ByteArray+ ): ByteArray {+ val header = byteArrayOf(+ ((PROTOCOL_VERSION_V1 shl 4) or HEADER_SIZE_4).toByte(),+ ((MSG_CLIENT_FULL_REQUEST shl 4) or FLAG_WITH_EVENT).toByte(),+ ((SERIALIZATION_JSON shl 4) or COMPRESSION_NONE).toByte(),+ 0x00,+ )+ val out = ByteArrayOutputStream()+ out.write(header)+ out.write(ByteBuffer.allocate(4).putInt(event).array())+ if (!isConnectionEvent(event)) {+ val sid = sessionId.toByteArray(Charsets.UTF_8)+ out.write(ByteBuffer.allocate(4).putInt(sid.size).array())+ out.write(sid)+ }+ out.write(ByteBuffer.allocate(4).putInt(payload.size).array())+ out.write(payload)+ return out.toByteArray()+ }++ private fun parseMessage(buf: ByteArray): ParsedMessage {+ val headerSize = (buf[0].toInt() and 0x0f) * 4+ val type = (buf[1].toInt() shr 4) and 0x0f+ val flag = buf[1].toInt() and 0x0f+ var offset = headerSize+ var event = 0+ var errorCode = 0++ val isDataMsg =+ type == MSG_CLIENT_FULL_REQUEST || type == MSG_SERVER_FULL_RESPONSE || type == MSG_SERVER_AUDIO_ONLY || type == 0xc+ if (isDataMsg) {+ if (flag == 0x1 || flag == 0x3) offset += 4 // skip sequence+ } else if (type == MSG_SERVER_ERROR) {+ errorCode = intAtBE(buf, offset)+ offset += 4+ }++ if (flag == FLAG_WITH_EVENT) {+ event = intAtBE(buf, offset)+ offset += 4+ // Skip session_id (except connection-level events don't carry it)+ if (!isConnectionEvent(event)) {+ val sidSize = intAtBE(buf, offset)+ offset += 4+ offset += sidSize+ }+ // connect_id present only on 50/51/52+ if (event == EVT_CONNECTION_STARTED || event == EVT_CONNECTION_FAILED || event == EVT_CONNECTION_FINISHED) {+ val cidSize = intAtBE(buf, offset)+ offset += 4+ offset += cidSize+ }+ }++ val payloadSize = intAtBE(buf, offset)+ offset += 4+ val payload = buf.copyOfRange(offset, offset + payloadSize)+ return ParsedMessage(type, flag, event, errorCode, payload)+ }++ private fun intAtBE(b: ByteArray, o: Int): Int =+ ((b[o].toInt() and 0xff) shl 24) or ((b[o + 1].toInt() and 0xff) shl 16) or ((b[o + 2].toInt() and 0xff) shl 8) or (b[o + 3].toInt() and 0xff)++ private fun isConnectionEvent(event: Int): Boolean =+ event == EVT_START_CONNECTION || event == EVT_FINISH_CONNECTION || event == EVT_CONNECTION_STARTED || event == EVT_CONNECTION_FAILED || event == EVT_CONNECTION_FINISHED++ companion object {+ private const val TAG = "HuoshanTts"++ private const val PROTOCOL_VERSION_V1 = 0x1+ private const val HEADER_SIZE_4 = 0x1++ private const val MSG_CLIENT_FULL_REQUEST = 0x1+ private const val MSG_SERVER_FULL_RESPONSE = 0x9+ private const val MSG_SERVER_AUDIO_ONLY = 0xb+ private const val MSG_SERVER_ERROR = 0xf++ private const val FLAG_WITH_EVENT = 0x4+ private const val SERIALIZATION_JSON = 0x1+ private const val COMPRESSION_NONE = 0x0++ private const val EVT_START_CONNECTION = 1+ private const val EVT_FINISH_CONNECTION = 2+ private const val EVT_CONNECTION_STARTED = 50+ private const val EVT_CONNECTION_FAILED = 51+ private const val EVT_CONNECTION_FINISHED = 52+ private const val EVT_START_SESSION = 100+ private const val EVT_CANCEL_SESSION = 101+ private const val EVT_FINISH_SESSION = 102+ private const val EVT_SESSION_STARTED = 150+ private const val EVT_SESSION_CANCELED = 151+ private const val EVT_SESSION_FINISHED = 152+ private const val EVT_SESSION_FAILED = 153+ private const val EVT_TASK_REQUEST = 200+ private const val EVT_TTS_SENTENCE_START = 350+ private const val EVT_TTS_SENTENCE_END = 351+ private const val EVT_TTS_RESPONSE = 352+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/tts/TtsController.kt b/app/src/main/java/net/kuafuai/andee/tts/TtsController.ktnew file mode 100644index 0000000..9fc0858--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/tts/TtsController.kt@@ -0,0 +1,321 @@+package net.kuafuai.andee.tts++import android.content.Context+import android.os.Handler+import android.os.Looper+import android.os.SystemClock+import android.util.Log+import net.kuafuai.andee.audio.AudioFocusGate+import net.kuafuai.andee.audio.AudioIO+import net.kuafuai.andee.config.VoiceConfig+import net.kuafuai.andee.ui.FloatingWindowUi+import java.util.concurrent.Executors++/**+ * One-shot TTS: speak a single string via 火山 bidi TTS, stream PCM to AudioIO.+ *+ * IDLE ── speak(text) ─→ SPEAKING (WS open, push, finish)+ * │+ * ├── sentence_start → subtitle SPEAKING+ * ├── done → IDLE+ * └── error → subtitle ERROR, IDLE+ *+ * Multi-sentence streaming (LLM-driven) will come later in Stage B via a+ * separate pipeline; this controller is deliberately the single-string case.+ */+class TtsController(+ private val context: Context,+ private val audio: AudioIO,+ private val window: FloatingWindowUi,+) {+ @Volatile+ private var client: HuoshanTts? = null++ // Set by [stopSpeaking] so the in-flight websocket callbacks of an+ // interrupted utterance stop touching the UI. Without it, cancelling the+ // socket can surface as onError → red ERROR ball, right as the user starts+ // talking. Cleared when the next utterance begins.+ @Volatile+ private var cancelled = false++ /**+ * Fired once the answer has finished DRAINING (the user has actually+ * stopped hearing it — see [onDone] for why that is not the same as the+ * server finishing). The service hooks this to open a short follow-up+ * mic window: as if the user pressed the talk button themselves, three+ * seconds after the answer ends. "Continue this conversation" without+ * the user having to reach for anything.+ */+ @Volatile+ var onDrained: (() -> Unit)? = null++ /**+ * The answer could not be spoken: socket error, socket death on its own, or+ * the stall watchdog giving up. Fired at most once per utterance.+ *+ * Deliberately **not** paired with a red ball any more. Nothing failed —+ * the words are already a row in the scrollback, and only the reading of+ * them aloud did not happen. The ball wearing ERROR said "your request+ * broke" about a request that had in fact succeeded, which is a worse lie+ * than saying nothing. The owner decides what to do instead: on a folded+ * card, put the answer up where it can be read.+ */+ @Volatile+ var onUnavailable: (() -> Unit)? = null++ /** Per utterance, so a socket that fails twice reports once. */+ @Volatile+ private var reported = false++ private fun reportUnavailable(why: String) {+ if (reported) return+ reported = true+ Log.w(TAG, "unavailable: $why")+ onUnavailable?.invoke()+ }++ // ---- Stalled-utterance watchdog ----------------------------------+ //+ // The ball enters SPEAKING in [speakImpl]'s onOpen and exits it in onDone+ // — which the websocket is under no obligation to deliver. Three exits+ // were missing, and each one left the ball doing the speaking face with+ // no sound until the next turn came along:+ // 1. the audio focus being yanked mid-answer (a call coming in) —+ // stopSpeaking deliberately leaves state for its *caller* to take+ // over, but this caller has no next state of its own;+ // 2. the socket dying on its own (network, server) — onClose arrived+ // with nothing scheduled to follow it;+ // 3. the socket hanging silently — no done, no error, no close.+ // Exit 1 is fixed at the focus callback, exit 2 at onClose. This+ // watchdog is exit 3: it checks for PROGRESS, not total length — a long+ // answer streams pcm for minutes, a dead one stops within seconds.+ @Volatile+ private var lastProgressAt = 0L+ private val ui = Handler(Looper.getMainLooper())+ private var stallWatchdog: Runnable? = null++ private fun armStallWatchdog() {+ disarmStallWatchdog()+ lastProgressAt = SystemClock.elapsedRealtime()+ val r = object : Runnable {+ override fun run() {+ if (SystemClock.elapsedRealtime() - lastProgressAt < STALL_AFTER_MS) {+ ui.postDelayed(this, STALL_CHECK_MS)+ return+ }+ stallWatchdog = null+ if (cancelled) return+ val c = client ?: return+ client = null+ runCatching { c.cancel() }+ runCatching { audio.stopStreamPlayback() }+ focus.release()+ window.setState(FloatingWindowUi.State.IDLE)+ reportUnavailable("no audio and no done for ${STALL_AFTER_MS}ms")+ }+ }+ stallWatchdog = r+ ui.postDelayed(r, STALL_CHECK_MS)+ }++ private fun disarmStallWatchdog() {+ stallWatchdog?.let { ui.removeCallbacks(it) }+ stallWatchdog = null+ }++ private val executor = Executors.newSingleThreadExecutor { r ->+ Thread(r, "TtsController").apply { isDaemon = true }+ }++ /**+ * Pause whatever the tablet is playing for the length of the answer.+ *+ * Plain transient, unlike the listening side's exclusive request: here we+ * only need to be heard over the media app, and a player that chooses to+ * duck instead of stop is fine — nothing is being recorded. Either way it+ * resumes by itself when we release.+ */+ private val focus = AudioFocusGate(+ context,+ android.media.AudioAttributes.USAGE_ASSISTANT,+ android.media.AudioManager.AUDIOFOCUS_GAIN_TRANSIENT,+ )++ fun isSpeaking(): Boolean = client != null++ fun speak(text: String) {+ if (text.isBlank()) return+ cancelled = false+ reported = false+ executor.execute {+ try {+ speakImpl(text)+ } catch (t: Throwable) {+ Log.e(TAG, "speak failed", t)+ window.setState(FloatingWindowUi.State.IDLE)+ runCatching { audio.stopStreamPlayback() }+ focus.release()+ client = null+ reportUnavailable(t.message ?: "speak threw")+ }+ }+ }++ /**+ * Barge-in: cut playback immediately, on the CALLER's thread.+ *+ * Deliberately not routed through [executor] like [cancel] is — a queued+ * task would sit behind AudioIO.finishStreamPlayback, which sleeps for the+ * length of the audio still buffered. That sleep is exactly the wait we're+ * trying to abort. AudioTrack pause/flush/stop are cheap and safe to call+ * from here.+ *+ * Leaves the ball state alone: the caller is about to set its own (the+ * user is starting to speak, so ASR takes over the state machine).+ */+ fun stopSpeaking() {+ // Flag first, unconditionally: speak() may have queued an utterance+ // that hasn't reached speakImpl yet, so [client] can still be null+ // while an utterance is very much on its way.+ cancelled = true+ disarmStallWatchdog()+ val c = client+ client = null+ runCatching { c?.cancel() }+ runCatching { audio.stopStreamPlayback() }+ focus.release()+ }++ fun cancel() {+ executor.execute {+ disarmStallWatchdog()+ runCatching { client?.cancel() }+ runCatching { audio.stopStreamPlayback() }+ focus.release()+ client = null+ window.setState(FloatingWindowUi.State.IDLE)+ }+ }++ fun shutdown() {+ disarmStallWatchdog()+ runCatching { client?.cancel() }+ runCatching { audio.stopStreamPlayback() }+ focus.release()+ client = null+ executor.shutdownNow()+ }++ private fun speakImpl(text: String) {+ val cfg = VoiceConfig.load(context)+ // Interrupted while this utterance sat in the executor queue.+ if (cancelled) return++ // Before the socket opens, not when the first audio lands: a media app+ // takes a moment to actually stop, and the whole 火山 handshake is that+ // moment. Losing the floor mid-answer (a call arriving) ends the+ // utterance — talking underneath a ringtone helps nobody.+ //+ // The state handover is the point here: stopSpeaking leaves the ball+ // for its caller to take over (startTurn → RECORDING), but a focus+ // loss has no caller and no next state. Without this line the ball+ // would keep mouthing words in silence until the next turn.+ focus.acquire {+ stopSpeaking()+ window.setState(FloatingWindowUi.State.IDLE)+ }++ val listener = object : HuoshanTts.Listener {+ override fun onOpen(sampleRate: Int) {+ if (cancelled) return+ audio.startStreamPlayback(sampleRate)+ // The ball's speaking face is the whole signal. The sentence+ // itself is already a row in the scrollback — the caller logs it+ // before handing it here — so echoing it onto the live line put+ // the same words on screen twice, once settled and once in blue+ // right underneath.+ window.setState(FloatingWindowUi.State.SPEAKING)+ armStallWatchdog()+ executor.execute {+ runCatching {+ client?.push(text)+ client?.finish()+ }+ }+ }++ override fun onAudio(pcm: ByteArray) {+ if (cancelled) return+ lastProgressAt = SystemClock.elapsedRealtime()+ audio.feedPcm(pcm)+ }++ override fun onSentenceStart(t: String) {}++ override fun onSentenceEnd(t: String) {}++ override fun onDone() {+ if (cancelled) return+ disarmStallWatchdog()+ // SessionFinished means the *server* has stopped sending, not+ // that the user has stopped hearing: 火山 delivers audio far+ // faster than real time, so most of the utterance is usually+ // still queued at this point. The ball therefore goes IDLE on+ // the other side of the drain, not here — otherwise it drops+ // its speaking face a minute before it stops talking.+ executor.execute {+ runCatching { audio.finishStreamPlayback() }+ focus.release()+ if (!cancelled) window.setState(FloatingWindowUi.State.IDLE)+ // The follow-up window, after the drain: the mic opens only+ // once the last word has actually been heard.+ if (!cancelled) onDrained?.invoke()+ }+ client = null+ }++ override fun onError(msg: String) {+ // An interrupted utterance closes its socket mid-stream, which+ // the ws layer may report as an error. Not worth showing.+ if (cancelled) return+ disarmStallWatchdog()+ runCatching { audio.stopStreamPlayback() }+ focus.release()+ window.setState(FloatingWindowUi.State.IDLE)+ client = null+ reportUnavailable(msg)+ }++ override fun onClose(reason: HuoshanTts.CloseReason) {+ // Every planned exit (stopSpeaking, cancel, onDone, onError)+ // nulls [client] before the socket actually closes. Reaching+ // here with it still set therefore means the socket died on+ // its own — and no done/error is ever coming. Close the+ // utterance out, or the ball wears SPEAKING with no sound.+ if (client != null) {+ disarmStallWatchdog()+ client = null+ runCatching { audio.stopStreamPlayback() }+ focus.release()+ window.setState(FloatingWindowUi.State.IDLE)+ reportUnavailable("socket died: $reason")+ }+ }+ }++ val c = HuoshanTts(cfg, listener)+ client = c+ c.open()+ }++ companion object {+ private const val TAG = "Tts"++ // Watchdog cadence: check every 5s, declare the utterance dead after+ // 20s without a single pcm chunk. Generous for a slow network (volcano+ // streams pcm faster than realtime even on a bad one) and far shorter+ // than "until the next turn".+ private const val STALL_CHECK_MS = 5_000L+ private const val STALL_AFTER_MS = 20_000L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/Backdrop.kt b/app/src/main/java/net/kuafuai/andee/ui/Backdrop.ktnew file mode 100644index 0000000..bc3d45c--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/Backdrop.kt@@ -0,0 +1,264 @@+package net.kuafuai.andee.ui++import android.content.Context+import android.graphics.Bitmap+import android.graphics.BitmapFactory+import android.graphics.Color+import android.os.Handler+import android.os.Looper+import android.util.Log+import java.util.concurrent.Executors+import kotlin.math.max+import kotlin.math.roundToInt++/**+ * The fullscreen card's backdrop: one fixed image, blurred and darkened.+ *+ * The card used to be a window onto whatever was behind it. That was the whole+ * point of [Glass.frost] — `FLAG_BLUR_BEHIND` defocuses the home screen or the+ * app underneath and the card's 60%-black fill lays the glass on top of it — and+ * it is the right answer for a modal that appears over a document. It is the+ * wrong answer for this card: it is fullscreen, it stays up for minutes at a+ * time, and what shows through is the user's home screen with their own icons+ * dissolved behind the text. The card read as *transparent* rather than as+ * *ours*.+ *+ * So the card brings its own background. Three consequences worth stating,+ * because each one is a decision rather than a side effect:+ *+ * * **The compositor blur is off while this is showing** (`FloatingWindowUi.+ * applyWindow`). There is no longer anything behind the card to defrost — the+ * backdrop covers the whole window — so asking for it would be a full-screen+ * defocus of the GPU, several times a second, for a result nobody can see.+ * It is also what makes the card's look independent of+ * `isCrossWindowBlurEnabled`, which is false on battery saver, on devices+ * that cannot afford it, and whenever the developer option is off.+ * * **The blur here is ours**, and it is a plain box blur run three times. Not+ * `RenderEffect` (API 31+, and it blurs a View's *own* content — the same+ * tool that cannot help the pills, see `CLAUDE.md`), and not a downscale-+ * upscale, which is bilinear and leaves the image looking smeared rather than+ * frosted. Three box passes approximate a Gaussian closely enough that the+ * result reads as glass, and on a [TARGET_W]-wide bitmap the whole thing is+ * a few milliseconds.+ * * **It runs off the main thread, at build time.** The window is built folded+ * and the backdrop is `GONE` while that runs, so nothing ever blocks the+ * main thread for it.+ *+ * It used to also say "finished long before the first unfold", on the+ * grounds that the service starts folded so nothing is shown until the user+ * long-presses. The service no longer starts folded — it unfolds at birth+ * (`ScreenBodyService.onServiceConnected`) — so on a cold start the first+ * unfold can *beat* this. The card is already built for that: until [ready]+ * flips, `backdropUp()` is false, `paintCard` gives the card its own fill,+ * and [warm]'s callback re-runs `applyWindow` so the real backdrop takes+ * over the moment it exists. The cost is a frame or two of the fallback+ * look instead of a card with a transparent hole in it — the same trade+ * `backdropUp` was written for.+ *+ * The image itself lives in `res/drawable-nodpi/backdrop.jpg`. Any photo will+ * do: the scrim is derived from the image's own brightness rather than being a+ * constant, so swapping the file is the whole procedure. See [scrim].+ */+object Backdrop {++ private const val TAG = "Body"++ /**+ * The blurred copy's width, in pixels. Fixed rather than screen-relative on+ * purpose: it is the *blur* that is being sized here, and every screen this+ * runs on stretches it further than this anyway (`CENTER_CROP` on a phone+ * scales it up ~3x), so a larger source would only cost work to produce+ * detail the frost then removes.+ */+ private const val TARGET_W = 512++ /** Blur radius in [TARGET_W]-space. ~8 dp once the view stretches it. */+ private const val BLUR_RADIUS = 8++ /**+ * Three passes. One box pass is a visibly square blur — its kernel is a+ * rectangle, and on a photo with any structure in it the directions read as+ * a cross. Three is where the difference stops being visible.+ */+ private const val BLUR_PASSES = 3++ /**+ * What the backdrop's mean luminance should end up at, once the scrim is on.+ *+ * 0.13 is not a taste-free number: it is what the card already composites to+ * today, on average — `CARD_FROSTED` is 60% black over a blurred desktop,+ * and a desktop averages 0.25–0.4. So this keeps every contrast decision+ * made above it valid (the `LABEL`/`SECONDARY` text, the `PANEL` and `WELL`+ * films, the `#2BFFFFFF` hairline) while changing *what* those decisions are+ * sitting on.+ */+ private const val TARGET_LUMA = 0.13f++ /**+ * Bounds on the derived scrim, so a very dark or very bright photo cannot+ * walk the card's base colour out of the range the palette assumes. The floor+ * keeps text legible over a near-black image; the ceiling keeps the image+ * from disappearing under a near-white one.+ */+ private const val SCRIM_MIN = 0.55f+ private const val SCRIM_MAX = 0.85f++ private val ui = Handler(Looper.getMainLooper())+ private val worker = Executors.newSingleThreadExecutor { r ->+ Thread(r, "backdrop").apply { isDaemon = true }+ }++ @Volatile private var ready = false++ /** The blurred bitmap, whole. `CENTER_CROP` does the cropping, not this. */+ @Volatile private var bitmap: Bitmap? = null++ /** ARGB fill that goes over [bitmap] and under everything else. */+ @Volatile private var scrim = 0++ /** Whether [bitmap] and [scrim] are worth reading yet. */+ fun ready(): Boolean = ready++ fun bitmap(): Bitmap? = bitmap++ /**+ * The fill the card's own background should be while the backdrop is up.+ * `0` before [warm] finishes — callers should read [ready] first.+ */+ fun scrim(): Int = scrim++ /**+ * Decode, blur and measure the backdrop, then hand the result to [onReady] on+ * the main thread. Safe to call more than once; only the first does work.+ */+ fun warm(context: Context, onReady: (Bitmap) -> Unit) {+ if (ready) {+ ui.post { onReady(bitmap!!) }+ return+ }+ val app = context.applicationContext+ worker.execute {+ val out = runCatching { build(app) }+ .onFailure { Log.e(TAG, "backdrop failed; card keeps its own fill", it) }+ .getOrNull() ?: return@execute+ bitmap = out.first+ scrim = out.second+ ready = true+ ui.post { onReady(out.first) }+ }+ }++ /** Decode → shrink → blur → measure. Worker thread only. */+ private fun build(context: Context): Pair<Bitmap, Int> {+ val src = decode(context)+ val w = src.width+ val h = src.height+ val px = IntArray(w * h)+ src.getPixels(px, 0, w, 0, 0, w, h)+ src.recycle()++ boxBlur(px, w, h)++ val luma = meanLuma(px)+ val alpha = (1f - TARGET_LUMA / max(luma, 0.02f)).coerceIn(SCRIM_MIN, SCRIM_MAX)+ Log.i(+ TAG,+ "backdrop %dx%d mean luma %.3f → scrim %.0f%% black".format(+ w, h, luma, alpha * 100,+ ),+ )+ return Bitmap.createBitmap(px, w, h, Bitmap.Config.ARGB_8888) to+ Color.argb((alpha * 255).roundToInt(), 0, 0, 0)+ }++ /**+ * The image, at [TARGET_W] wide.+ *+ * Two steps rather than one: `inSampleSize` is only cheap *and* predictable+ * in powers of two, so it takes the width down to somewhere at or above the+ * target and a scaled copy finishes the job. A single non-power-of-two+ * `inSampleSize` is honoured on API 26+ but not exact, and the arithmetic+ * that decides it is more code than the scaled copy it saves.+ */+ private fun decode(context: Context): Bitmap {+ val id = net.kuafuai.andee.R.drawable.backdrop+ val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }+ BitmapFactory.decodeResource(context.resources, id, bounds)+ var sample = 1+ while (bounds.outWidth / (sample * 2) >= TARGET_W) sample *= 2++ val opts = BitmapFactory.Options().apply {+ inSampleSize = sample+ inPreferredConfig = Bitmap.Config.ARGB_8888+ }+ val raw = requireNotNull(BitmapFactory.decodeResource(context.resources, id, opts)) {+ "backdrop.jpg did not decode"+ }+ if (raw.width == TARGET_W) return raw+ val h = max(1, (raw.height * TARGET_W.toFloat() / raw.width).roundToInt())+ val scaled = Bitmap.createScaledBitmap(raw, TARGET_W, h, true)+ if (scaled != raw) raw.recycle()+ return scaled+ }++ /**+ * Three separable box passes — horizontal then vertical, three times.+ *+ * A running sum per row and per column is what makes this linear in the+ * pixel count rather than quadratic in the radius: the window slides one+ * pixel at a time and each step adds one sample and drops one. Edges sample+ * the border pixel rather than wrapping or darkening, because a dark rim+ * around the image would read as a vignette the user did not ask for.+ */+ private fun boxBlur(px: IntArray, w: Int, h: Int) {+ val tmp = IntArray(px.size)+ val r = BLUR_RADIUS+ val n = r * 2 + 1+ repeat(BLUR_PASSES) {+ // Horizontal: px → tmp+ for (y in 0 until h) {+ val row = y * w+ var sr = 0; var sg = 0; var sb = 0+ for (i in -r..r) {+ val c = px[row + i.coerceIn(0, w - 1)]+ sr += (c shr 16) and 0xFF; sg += (c shr 8) and 0xFF; sb += c and 0xFF+ }+ for (x in 0 until w) {+ tmp[row + x] = -0x1000000 or ((sr / n) shl 16) or ((sg / n) shl 8) or (sb / n)+ val add = px[row + (x + r + 1).coerceIn(0, w - 1)]+ val sub = px[row + (x - r).coerceIn(0, w - 1)]+ sr += ((add shr 16) and 0xFF) - ((sub shr 16) and 0xFF)+ sg += ((add shr 8) and 0xFF) - ((sub shr 8) and 0xFF)+ sb += (add and 0xFF) - (sub and 0xFF)+ }+ }+ // Vertical: tmp → px+ for (x in 0 until w) {+ var sr = 0; var sg = 0; var sb = 0+ for (i in -r..r) {+ val c = tmp[i.coerceIn(0, h - 1) * w + x]+ sr += (c shr 16) and 0xFF; sg += (c shr 8) and 0xFF; sb += c and 0xFF+ }+ for (y in 0 until h) {+ px[y * w + x] = -0x1000000 or ((sr / n) shl 16) or ((sg / n) shl 8) or (sb / n)+ val add = tmp[(y + r + 1).coerceIn(0, h - 1) * w + x]+ val sub = tmp[(y - r).coerceIn(0, h - 1) * w + x]+ sr += ((add shr 16) and 0xFF) - ((sub shr 16) and 0xFF)+ sg += ((add shr 8) and 0xFF) - ((sub shr 8) and 0xFF)+ sb += (add and 0xFF) - (sub and 0xFF)+ }+ }+ }+ }++ /** Rec. 709 relative luminance, averaged. Worker thread only. */+ private fun meanLuma(px: IntArray): Float {+ var sum = 0.0+ for (c in px) {+ sum += (0.2126 * ((c shr 16) and 0xFF) ++ 0.7152 * ((c shr 8) and 0xFF) ++ 0.0722 * (c and 0xFF)) / 255.0+ }+ return (sum / px.size).toFloat()+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/BubbleUi.kt b/app/src/main/java/net/kuafuai/andee/ui/BubbleUi.ktnew file mode 100644index 0000000..36ba3af--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/BubbleUi.kt@@ -0,0 +1,275 @@+package net.kuafuai.andee.ui++import android.annotation.SuppressLint+import android.content.Context+import android.graphics.Color+import android.graphics.PixelFormat+import android.graphics.drawable.GradientDrawable+import android.os.Handler+import android.os.Looper+import android.view.Gravity+import android.view.MotionEvent+import android.view.View+import android.view.ViewGroup+import android.view.WindowManager+import android.widget.LinearLayout+import android.widget.TextView+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import kotlin.math.abs++/**+ * Floating bubble overlay with three visual modes (BALL / HALF / FULL) and+ * five semantic states (IDLE / RECORDING / THINKING / PLAYING / SPEAKING / ERROR).+ * Emits tap and long-press to [Listeners]. Only BALL mode is draggable.+ */+class BubbleUi(+ private val context: Context,+ private val listeners: Listeners,+) {++ enum class Mode { BALL, HALF, FULL }++ enum class State { IDLE, RECORDING, THINKING, PLAYING, SPEAKING, ERROR }++ interface Listeners {+ fun onTap()+ fun onLongPress()+ }++ private val wm: WindowManager =+ context.getSystemService(Context.WINDOW_SERVICE) as WindowManager+ private val ui = Handler(Looper.getMainLooper())++ /** Strings in the user's language; views are still built on [context]. */+ private val lctx: Context = AppLocale.wrap(context)++ private var view: View? = null+ private var params: WindowManager.LayoutParams? = null+ private var label: TextView? = null++ @Volatile+ private var mode: Mode = Mode.BALL++ @Volatile+ private var state: State = State.IDLE++ fun currentMode(): Mode = mode+ fun isShown(): Boolean = view != null+ fun params(): WindowManager.LayoutParams? = params++ fun show(): Boolean {+ if (view != null) return true+ val (v, p) = when (mode) {+ Mode.BALL -> createBallView()+ Mode.HALF -> createHalfView()+ Mode.FULL -> createFullView()+ }+ return try {+ wm.addView(v, p)+ view = v+ params = p+ attachDragAndTap(v, p)+ applyState()+ true+ } catch (_: Throwable) {+ false+ }+ }++ fun hide(): Boolean {+ val v = view ?: return true+ return try {+ wm.removeView(v)+ view = null+ params = null+ label = null+ true+ } catch (_: Throwable) {+ false+ }+ }++ fun setMode(newMode: Mode) {+ if (newMode == mode && view != null) return+ mode = newMode+ ui.post {+ hide()+ show()+ }+ }++ fun setState(newState: State) {+ state = newState+ ui.post { applyState() }+ }++ fun setLabel(text: String) {+ ui.post { label?.text = text }+ }++ // ---- View creation per mode ----++ private fun createBallView(): Pair<View, WindowManager.LayoutParams> {+ val v = View(context)+ val fill = colorFor(state)+ v.background = GradientDrawable().apply {+ shape = GradientDrawable.OVAL+ setColor(fill)+ setStroke(dp(2), Color.WHITE)+ }+ v.layoutParams = ViewGroup.LayoutParams(dp(BUBBLE_DP), dp(BUBBLE_DP))+ label = null+ val p = WindowManager.LayoutParams(+ dp(BUBBLE_DP), dp(BUBBLE_DP),+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE+ or WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.TOP or Gravity.START+ x = dp(24); y = dp(200)+ }+ return v to p+ }++ private fun createHalfView(): Pair<View, WindowManager.LayoutParams> {+ val panel = buildPanel(lctx.getString(R.string.bubble_tap_to_talk))+ val screenH = context.resources.displayMetrics.heightPixels+ val screenW = context.resources.displayMetrics.widthPixels+ val p = WindowManager.LayoutParams(+ screenW, screenH / 2,+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE+ or WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.BOTTOM or Gravity.START+ x = 0; y = 0+ }+ return panel to p+ }++ private fun createFullView(): Pair<View, WindowManager.LayoutParams> {+ val panel = buildPanel(lctx.getString(R.string.bubble_tap_to_talk))+ val screenH = context.resources.displayMetrics.heightPixels+ val screenW = context.resources.displayMetrics.widthPixels+ val p = WindowManager.LayoutParams(+ screenW, screenH,+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE+ or WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.TOP or Gravity.START+ x = 0; y = 0+ }+ return panel to p+ }++ private fun buildPanel(labelText: String): View {+ val fill = colorFor(state)+ val bg = GradientDrawable().apply {+ shape = GradientDrawable.RECTANGLE+ setColor(fill)+ setStroke(dp(2), Color.WHITE)+ }+ val text = TextView(context).apply {+ text = labelText+ textSize = 48f+ setTextColor(Color.WHITE)+ gravity = Gravity.CENTER+ }+ label = text+ return LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ gravity = Gravity.CENTER+ background = bg+ addView(+ text,+ LinearLayout.LayoutParams(+ LinearLayout.LayoutParams.MATCH_PARENT,+ LinearLayout.LayoutParams.WRAP_CONTENT,+ ),+ )+ }+ }++ private fun applyState() {+ val v = view ?: return+ (v.background as? GradientDrawable)?.setColor(colorFor(state))+ }++ private fun colorFor(s: State): Int = when (s) {+ State.IDLE -> Color.parseColor("#4F46E5") // indigo+ State.RECORDING -> Color.parseColor("#DC2626") // red+ State.THINKING -> Color.parseColor("#0EA5E9") // sky+ State.PLAYING -> Color.parseColor("#F59E0B") // amber+ State.SPEAKING -> Color.parseColor("#06B6D4") // cyan+ State.ERROR -> Color.parseColor("#DC2626") // red+ }++ // ---- Touch handling (drag only in BALL mode) ----++ @SuppressLint("ClickableViewAccessibility")+ private fun attachDragAndTap(view: View, params: WindowManager.LayoutParams) {+ var startParamsX = 0+ var startParamsY = 0+ var startTouchX = 0f+ var startTouchY = 0f+ var moved = false+ var longPressed = false+ val slop = dp(6)+ val longPressMs = 500L+ val longPress = Runnable {+ if (!moved) {+ longPressed = true+ listeners.onLongPress()+ }+ }+ view.setOnTouchListener { _, event ->+ when (event.action) {+ MotionEvent.ACTION_DOWN -> {+ startParamsX = params.x+ startParamsY = params.y+ startTouchX = event.rawX+ startTouchY = event.rawY+ moved = false+ longPressed = false+ ui.postDelayed(longPress, longPressMs)+ true+ }++ MotionEvent.ACTION_MOVE -> {+ val dx = (event.rawX - startTouchX).toInt()+ val dy = (event.rawY - startTouchY).toInt()+ if (!moved && (abs(dx) > slop || abs(dy) > slop)) {+ moved = true+ ui.removeCallbacks(longPress)+ }+ if (moved && mode == Mode.BALL) {+ params.x = startParamsX + dx+ params.y = startParamsY + dy+ wm.updateViewLayout(view, params)+ }+ true+ }++ MotionEvent.ACTION_UP, MotionEvent.ACTION_CANCEL -> {+ ui.removeCallbacks(longPress)+ if (!moved && !longPressed) listeners.onTap()+ true+ }++ else -> false+ }+ }+ }++ private fun dp(v: Int): Int =+ (v * context.resources.displayMetrics.density).toInt()++ companion object {+ const val BUBBLE_DP = 100+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/CameraChrome.kt b/app/src/main/java/net/kuafuai/andee/ui/CameraChrome.ktnew file mode 100644index 0000000..5fdb8af--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/CameraChrome.kt@@ -0,0 +1,228 @@+package net.kuafuai.andee.ui++import android.animation.AnimatorSet+import android.animation.ArgbEvaluator+import android.animation.ObjectAnimator+import android.animation.StateListAnimator+import android.animation.ValueAnimator+import android.content.Context+import android.graphics.Canvas+import android.graphics.Color+import android.graphics.Paint+import android.graphics.drawable.GradientDrawable+import android.view.Gravity+import android.view.View+import android.view.ViewGroup+import android.widget.FrameLayout+import android.widget.LinearLayout+import android.widget.TextView+import androidx.core.view.ViewCompat+import androidx.core.view.WindowInsetsCompat+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale++/**+ * Chrome shared by the two camera screens (`LookActivity`, `ScanActivity`).+ *+ * It exists because both had grown their own `FrameLayout + PreviewView ++ * TextView`, and both sized that TextView in **raw pixels** — a readable+ * banner on the phone it was written on, a postage stamp on the tablet.+ * Everything here is dp.+ *+ * The viewfinder is not decoration. The camera is the most invasive thing+ * this device does, so the user has to be able to tell, without asking,+ * when it is on and the moment a frame is actually taken — that is what+ * [ViewfinderView.flash] is for.+ */++fun Context.dpF(v: Float): Float = v * resources.displayMetrics.density+fun Context.dpI(v: Int): Int = (v * resources.displayMetrics.density).toInt()++/**+ * Taken from [Glass] rather than written out, so the camera screens, the+ * cards and the assistant's dialogs are one palette — these are the only+ * surfaces where the user sees our chrome over content we did not draw.+ *+ * [PILL_FILL] is [Glass.CARD_SOLID] for the reason the name gives: a pill is+ * a floating child, nothing blurs behind it (see [closePill]), so it takes+ * the fill the cards fall back to when the compositor refuses them a blur.+ */+private val IDLE_STROKE = 0x66FFFFFF.toInt()+private val ACCENT = Color.parseColor(Glass.ACCENT)+private val PILL_FILL = Color.parseColor(Glass.CARD_SOLID)+private val PILL_EDGE = Color.parseColor(Glass.HAIRLINE)+private val PILL_TEXT = Color.parseColor(Glass.TITLE)++/**+ * Four corner brackets around the frame. Idle is a quiet hairline; [flash]+ * pushes it to the ball's accent colour and lets it fall back.+ */+class ViewfinderView(context: Context) : View(context) {++ private val paint = Paint(Paint.ANTI_ALIAS_FLAG).apply {+ style = Paint.Style.STROKE+ strokeWidth = context.dpF(2f)+ strokeCap = Paint.Cap.ROUND+ color = IDLE_STROKE+ }+ private val arm = context.dpF(26f)+ private var anim: ValueAnimator? = null++ /** Pulse the brackets — call the moment a frame is actually captured. */+ fun flash(color: Int = ACCENT) {+ anim?.cancel()+ anim = ValueAnimator.ofObject(ArgbEvaluator(), IDLE_STROKE, color, IDLE_STROKE).apply {+ duration = 280+ addUpdateListener { paint.color = it.animatedValue as Int; invalidate() }+ start()+ }+ }++ override fun onDraw(canvas: Canvas) {+ val w = width.toFloat()+ val h = height.toFloat()+ if (w <= 0f || h <= 0f) return+ val l = w * 0.08f+ val t = h * 0.08f+ val r = w - l+ val b = h - t+ // top-left+ canvas.drawLine(l, t, l + arm, t, paint)+ canvas.drawLine(l, t, l, t + arm, paint)+ // top-right+ canvas.drawLine(r - arm, t, r, t, paint)+ canvas.drawLine(r, t, r, t + arm, paint)+ // bottom-left+ canvas.drawLine(l, b - arm, l, b, paint)+ canvas.drawLine(l, b, l + arm, b, paint)+ // bottom-right+ canvas.drawLine(r - arm, b, r, b, paint)+ canvas.drawLine(r, b - arm, r, b, paint)+ }+}++/**+ * Bottom-centred status line: a slowly breathing dot (the session is open)+ * plus whatever the screen wants to say. [setStatus] is the only way to+ * change the text.+ */+class StatusPill(context: Context) : LinearLayout(context) {++ private val dot = View(context)+ private val label = TextView(context)++ init {+ orientation = HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ setPadding(context.dpI(16), context.dpI(9), context.dpI(18), context.dpI(9))+ background = GradientDrawable().apply {+ setColor(PILL_FILL)+ cornerRadius = context.dpF(22f)+ setStroke(context.dpI(1), PILL_EDGE)+ }+ dot.background = GradientDrawable().apply {+ shape = GradientDrawable.OVAL+ setColor(ACCENT)+ }+ addView(dot, LayoutParams(context.dpI(8), context.dpI(8)).apply {+ marginEnd = context.dpI(9)+ })+ label.apply {+ textSize = 15f+ setTextColor(PILL_TEXT)+ }+ addView(label)+ ObjectAnimator.ofFloat(dot, "alpha", 1f, 0.3f).apply {+ duration = 1100+ repeatMode = ValueAnimator.REVERSE+ repeatCount = ValueAnimator.INFINITE+ start()+ }+ }++ fun setStatus(text: String) {+ label.text = text+ }++ fun setDotColor(color: Int) {+ (dot.background as GradientDrawable).setColor(color)+ }+}++/**+ * The close control, identical on every surface the brain can put in front+ * of the user — camera screens and `HtmlActivity` alike.+ *+ * Text, not a bare ✕: on an unfamiliar generated page a lone glyph reads as+ * part of the page. The 1dp stroke is not decoration either — a dark pill on+ * a dark page disappears, and the brain picks the page's background, not us.+ *+ * (No frosted glass: `RenderEffect` blurs a View's *own* content, and+ * blurring what is behind needs `Window.setBackgroundBlurRadius`, which is+ * window-level and cannot be scoped to one floating child. Translucency plus+ * a stroke is the honest approximation.)+ */+fun closePill(context: Context, onClose: () -> Unit): TextView =+ TextView(context).apply {+ text = AppLocale.str(context, R.string.cam_close)+ textSize = 15f+ setTextColor(PILL_TEXT)+ gravity = Gravity.CENTER+ setPadding(context.dpI(18), context.dpI(8), context.dpI(18), context.dpI(8))+ background = GradientDrawable().apply {+ setColor(PILL_FILL)+ cornerRadius = context.dpF(22f)+ setStroke(context.dpI(1), PILL_EDGE)+ }+ isClickable = true+ stateListAnimator = pressAnimator()+ setOnClickListener { onClose() }+ }++private fun pressAnimator(): StateListAnimator = StateListAnimator().apply {+ addState(+ intArrayOf(android.R.attr.state_pressed),+ AnimatorSet().apply {+ playTogether(+ ObjectAnimator.ofFloat(null, "scaleX", 0.94f),+ ObjectAnimator.ofFloat(null, "scaleY", 0.94f),+ ObjectAnimator.ofFloat(null, "alpha", 0.85f),+ )+ duration = 90+ },+ )+ addState(+ IntArray(0),+ AnimatorSet().apply {+ playTogether(+ ObjectAnimator.ofFloat(null, "scaleX", 1f),+ ObjectAnimator.ofFloat(null, "scaleY", 1f),+ ObjectAnimator.ofFloat(null, "alpha", 1f),+ )+ duration = 120+ },+ )+}++/**+ * Keep [view]'s margins clear of the status bar / gesture bar. Needed because+ * the camera screens go edge-to-edge for a full-bleed preview, which puts our+ * own chrome under the system clock unless we ask.+ */+fun avoidSystemBars(view: View, topDp: Int = 0, bottomDp: Int = 0) {+ val ctx = view.context+ ViewCompat.setOnApplyWindowInsetsListener(view) { v, insets ->+ val bars = insets.getInsets(WindowInsetsCompat.Type.systemBars())+ (v.layoutParams as? ViewGroup.MarginLayoutParams)?.let { lp ->+ if (topDp > 0) lp.topMargin = bars.top + ctx.dpI(topDp)+ if (bottomDp > 0) lp.bottomMargin = bars.bottom + ctx.dpI(bottomDp)+ v.layoutParams = lp+ }+ insets+ }+}++/** Full-bleed layout params, for the preview underneath the chrome. */+fun matchParent(): FrameLayout.LayoutParams = FrameLayout.LayoutParams(+ ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT+)diff --git a/app/src/main/java/net/kuafuai/andee/ui/CardUi.kt b/app/src/main/java/net/kuafuai/andee/ui/CardUi.ktnew file mode 100644index 0000000..07862e0--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/CardUi.kt@@ -0,0 +1,608 @@+package net.kuafuai.andee.ui++import android.annotation.SuppressLint+import android.content.Context+import android.graphics.Canvas+import android.graphics.Color+import android.graphics.LinearGradient+import android.graphics.Paint+import android.graphics.Path+import android.graphics.PixelFormat+import android.graphics.Shader+import android.graphics.Typeface+import android.os.Handler+import android.os.Looper+import android.util.Log+import android.util.TypedValue+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import android.widget.LinearLayout+import android.widget.TextView+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale++/**+ * The ball's little signboards — quiet, tappable cards the ball *holds up*.+ *+ * Why cards beat push-interrupts for incoming messages: a notification that+ * wakes the brain every time trains the user to mute everything. A small+ * card that asks "看看吗?" costs one glance and a tap — the brain only+ * hears about messages the user actually cares about. The brain gets the+ * SAME power as a tool (ask_user / alert), so it can escalate to a card+ * whenever IT thinks the user should decide something.+ *+ * The card is not a free-floating box: it is anchored to the ball. A tail+ * on the card's underside points at the ball, the card sits just above it,+ * and while the card is up the ball itself attends — setHolding(true) puts+ * a listening face on it and stops the idle fidget. The user is deciding+ * something; the ball waits for the answer like it waits for speech.+ *+ * One card at a time — a new card replaces the old one (the old one's+ * callback fires with "superseded"). Queues of cards are just push spam+ * with extra steps.+ */+object CardUi {++ private val main = Handler(Looper.getMainLooper())++ /**+ * Run [r] on the main thread — the thread the task flag and every card+ * operation live on. Exposed for the notification relay, whose+ * onNotificationPosted arrives on a binder thread and needs its+ * card-or-not decision to happen where the state is authoritative.+ */+ fun post(r: () -> Unit) {+ main.post(r)+ }+ private var wm: WindowManager? = null+ private var current: CardHandle? = null++ private const val CARD_MAX_WIDTH_DP = 560+ private const val AUTO_DISMISS_MS = 25_000L+ private const val TAG = "Card"++ /**+ * How long the same error stays suppressed, and how long an error card+ * stays up. Shorter than [AUTO_DISMISS_MS]: an error is an FYI, not a+ * question, and the scrollback keeps it either way.+ */+ private const val ERROR_DEDUP_MS = 10_000L+ private const val ERROR_DISMISS_MS = 8_000L++ private var lastErrorText: String? = null+ private var lastErrorAt = 0L++ /**+ * How far the tail tip reaches toward the ball. Superseded as a *gap*+ * metric by the single-window rewrite — the card is now a child of the+ * ball's own window (FloatingWindowUi.showSignboard), and the tail+ * overlaps the ball area's transparent padding by a fraction of the+ * window side instead of keeping a dp clearance from a measured head.+ */+ private const val TAIL_OVERLAP_NOTE = "see FloatingWindowUi.SIGN_OVERLAP_FRACTION"++ private var appContext: Context? = null++ fun init(context: Context) {+ appContext = context+ if (wm == null) {+ wm = context.getSystemService(Context.WINDOW_SERVICE) as WindowManager+ }+ }++ class Result(+ /** Which button the user tapped, or null for timeout/superseded/dismissed-elsewhere. */+ val button: String?,+ val how: String, // "button" | "timeout" | "superseded" | "tap_outside"+ )++ /**+ * Show a confirm-style card. Callback fires exactly once, on the main+ * thread. Safe to call from any thread.+ *+ * @param buttons label list, e.g. ["帮我看看", "忽略"]. First button is+ * rendered as the primary (tinted) one.+ */+ fun ask(+ question: String,+ buttons: List<String>,+ timeoutMs: Long = AUTO_DISMISS_MS,+ onDone: (Result) -> Unit,+ ) {+ show(question, buttons, cancelable = false, timeoutMs, onDone)+ }++ /**+ * Show a one-lane notice with a single dismiss control. Tapping the card+ * body also counts as "read".+ */+ fun alert(text: String, timeoutMs: Long = AUTO_DISMISS_MS, onDone: (Result) -> Unit) {+ // No context means the service is not up, and `show` would bail on the+ // same condition one call later — so returning here changes nothing+ // observable, it just keeps the label lookup off a null.+ val ctx = appContext ?: return+ show(text, listOf(AppLocale.str(ctx, R.string.card_ack)), cancelable = true, timeoutMs, onDone)+ }++ /**+ * Something broke and the user has to be told. Pops a card and leaves a+ * row in the scrollback, so the failure survives the card timing out.+ *+ * The de-dup is the whole reason this isn't just `alert` at the call+ * sites: the failures worth showing arrive in bursts, not singly — a hub+ * that is down produces the same sentence on every reconnect attempt, and+ * since a new card supersedes the old one that reads as a card flickering+ * in place, cancelling whatever the user was in the middle of answering.+ */+ fun error(text: String) {+ if (text.isBlank()) return+ val now = android.os.SystemClock.uptimeMillis()+ synchronized(this) {+ if (text == lastErrorText && now - lastErrorAt < ERROR_DEDUP_MS) return+ lastErrorText = text+ lastErrorAt = now+ }+ ChatHistory.addError(text)+ alert(text, ERROR_DISMISS_MS) { }+ }++ fun dismissCurrent() {+ main.post { current?.close("superseded") }+ }++ // ------------------------------------------------------------------++ private class CardHandle(+ val view: View,+ /** The ball window hosting this card, or null for the standalone fallback. */+ val host: FloatingWindowUi?,+ @Volatile var finished: Boolean = false,+ var onDone: ((Result) -> Unit)? = null,+ var timeout: Runnable? = null,+ ) {+ fun close(how: String, button: String? = null) {+ if (finished) return+ finished = true+ timeout?.let { main.removeCallbacks(it) }+ if (host != null) {+ host.hideSignboard()+ } else {+ runCatching { wm?.removeView(view) }+ }+ if (current === this) current = null+ // The ball can stop attending — the question is answered, timed+ // out, or was replaced by a newer one (which re-arms it below).+ ballWindow()?.setHolding(false)+ onDone?.invoke(Result(button, how))+ }+ }++ /** The ball's window, when the service is up. Null-safe by design. */+ private fun ballWindow(): FloatingWindowUi? =+ net.kuafuai.andee.ScreenBodyService.get()?.ballWindow()++ /**+ * Wire the window's eviction hook once per window. The window clears its+ * signboard when it folds or unfolds; the card it evicted must fire its+ * own callbacks through the same exactly-once path as a timeout, or the+ * notification relay's ask would dangle forever.+ */+ private fun ensureEvictorWired(host: FloatingWindowUi) {+ if (host.signboardEvictor != null) return+ host.signboardEvictor = { current?.close("superseded") }+ }++ @SuppressLint("SetTextI18n")+ private fun show(+ text: String,+ buttons: List<String>,+ cancelable: Boolean,+ timeoutMs: Long,+ onDone: (Result) -> Unit,+ ) {+ val windowManager = wm ?: return+ main.post {+ // One card at a time — the newcomer replaces whoever's up.+ current?.close("superseded")++ val context = appContext ?: return@post+ val density = android.content.res.Resources.getSystem().displayMetrics.density+ fun dp(v: Int) = (v * density).toInt()+ val dm = appContext?.resources?.displayMetrics+ ?: android.content.res.Resources.getSystem().displayMetrics++ // ---- card root ------------------------------------------------+ // The tail is drawn by the root itself (drawChild hook) rather+ // than being a separate view: it has to sit *below* the rounded+ // card background and share its color, and a child view with its+ // own background would fight the parent's rounded corners.+ val cardReal = TailLinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ cornerRadius = dp(22)+ setPadding(dp(22), dp(18), dp(22), dp(16))+ }++ // Question text+ val title = TextView(context).apply {+ this.text = text+ setTextColor(Color.parseColor(Glass.TITLE))+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 16f)+ typeface = Typeface.DEFAULT_BOLD+ setLineSpacing(dp(3).toFloat(), 1f)+ maxLines = 4+ ellipsize = android.text.TextUtils.TruncateAt.END+ }+ cardReal.addView(title)++ // Button row — buttons laid out right-to-left (primary last =+ // rightmost), spacing via layout params passed to addView.+ val row = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.END or Gravity.CENTER_VERTICAL+ setPadding(0, dp(12), 0, 0)+ }+ val btnViews = mutableListOf<TextView>()+ val btnLabels = mutableListOf<String>()+ buttons.asReversed().forEach { label ->+ // reversed list: original index 0 (primary) comes LAST →+ // rightmost, and renders tinted.+ val isPrimary = label == buttons.first()+ val btn = TextView(context).apply {+ this.text = label+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 15f)+ typeface = Typeface.DEFAULT_BOLD+ setPadding(dp(18), dp(9), dp(18), dp(9))+ gravity = Gravity.CENTER+ isClickable = true+ isFocusable = false+ // The primary keeps a solid fill: it is the one thing on+ // the card that must not read as glass, or there is nothing+ // to aim at. The secondary becomes another pane of the same+ // glass as the card — a solid slate slab sitting on a+ // translucent card was the loudest part of the old look.+ // Both come from [Glass], which is also where the settings+ // card's buttons come from: a dialog the assistant raises+ // and a card the user opened should not be two designs.+ background = if (isPrimary) {+ Glass.tinted(context, dp(22))+ } else {+ // `thumb`, not `panel`: nothing is blurred behind this+ // card, so its panes have to carry themselves a step+ // brighter than they would on a frosted one.+ Glass.thumb(context, dp(22))+ }+ setTextColor(Color.parseColor(if (isPrimary) Glass.TITLE else Glass.LABEL))+ Glass.pressable(this)+ }+ val lp = LinearLayout.LayoutParams(+ LinearLayout.LayoutParams.WRAP_CONTENT,+ LinearLayout.LayoutParams.WRAP_CONTENT,+ ).apply { marginStart = dp(10) }+ row.addView(btn, lp)+ btnViews.add(btn)+ btnLabels.add(label)+ }++ // Long option labels: every button in the row is measured against+ // the card's full content width, so a label wider than that wraps+ // inside its own button — mixed-height buttons, and where several+ // of them overflow, buttons half-cut at the card's edge. Decide the+ // row's shape from whether it actually fits: horizontal when it+ // does (then no single button can have wrapped, since any wrapped+ // button alone fills the width), a full-width stacked list when it+ // does not — which is also the shape a long list of choices reads+ // as anyway. The primary stays last: rightmost in the row,+ // bottommost in the stack, the "confirm" spot either way.+ //+ // The natural-width measure is UNSPECIFIED on purpose: measured+ // against the card's width instead, an overflowing row reports+ // exactly that width back (resolveSizeAndState clamps it), so the+ // comparison could never fire. UNSPECIFIED neither clamps nor+ // wraps, so this is the true sum of the single-line widths.+ val cardWidth = minOf(dm.widthPixels - dp(32), dp(CARD_MAX_WIDTH_DP))+ val contentWidth = cardWidth - dp(22) * 2+ row.measure(+ View.MeasureSpec.makeMeasureSpec(0, View.MeasureSpec.UNSPECIFIED),+ View.MeasureSpec.makeMeasureSpec(0, View.MeasureSpec.UNSPECIFIED),+ )+ if (row.measuredWidth > contentWidth) {+ row.orientation = LinearLayout.VERTICAL+ btnViews.forEachIndexed { i, btn ->+ val vlp = btn.layoutParams as LinearLayout.LayoutParams+ vlp.width = LinearLayout.LayoutParams.MATCH_PARENT+ vlp.marginStart = 0+ vlp.topMargin = if (i == 0) 0 else dp(8)+ btn.layoutParams = vlp+ }+ }+ cardReal.addView(row)++ // ---- host inside the ball's window --------------------------------+ //+ // The old design put this card in its own overlay window and+ // positioned it from a measured copy of the ball's position —+ // which goes stale mid-slide, disagrees with the overlay+ // coordinate space on some devices, and reads a `winYOrigin` that+ // is only refreshed when the ball is still. Each of those hits a+ // different subset of devices; together they are the "sometimes+ // it's off" this rewrite exists to kill. The card is now a child+ // of the ball's own window (see FloatingWindowUi.showSignboard):+ // same coordinate space, zero translation, and the ball being+ // dragged or perched takes the card with it for free.+ val host = ballWindow()+ if (host == null || !host.isFolded()) {+ // No ball window to hold it: fall back to the standalone+ // overlay. Rare (service down mid-question) but the question+ // still has to be answerable.+ val params = WindowManager.LayoutParams(+ WindowManager.LayoutParams.WRAP_CONTENT,+ WindowManager.LayoutParams.WRAP_CONTENT,+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE,+ PixelFormat.TRANSLUCENT,+ ).apply {+ width = minOf(dm.widthPixels - dp(32), dp(CARD_MAX_WIDTH_DP))+ gravity = Gravity.BOTTOM or Gravity.CENTER_HORIZONTAL+ y = dp(96)+ }+ cardReal.tailX = -1++ // ---- handle + callbacks ------------------------------------+ val handle = CardHandle(cardReal, host = null)+ handle.onDone = { r -> main.post { onDone(r) } }+ wireButtons(handle, btnViews, btnLabels, cancelable, cardReal, title)+ val timeoutRunnable = Runnable { handle.close("timeout") }+ handle.timeout = timeoutRunnable+ main.postDelayed(timeoutRunnable, timeoutMs)++ runCatching {+ cardReal.hideFromAccessibility()+ windowManager.addView(cardReal, params)+ current = handle+ Glass.enter(cardReal)+ }.onFailure {+ handle.finished = true+ handle.timeout?.let { t -> main.removeCallbacks(t) }+ // Worth a line. This is the one path where a message that+ // was supposed to reach the user goes nowhere at all, and+ // the caller only sees `onDone("error: …")` — which for a+ // fire-and-forget alert is no one. Without this, "the card+ // never appeared" has no evidence anywhere.+ Log.w(TAG, "standalone card could not be added", it)+ onDone(Result(null, "error: ${it.message}"))+ }+ return@post+ }++ // Held by the ball: the tail points at the window's centre —+ // which is where the ball is, by construction, not by measurement.+ cardReal.measure(+ View.MeasureSpec.makeMeasureSpec(+ minOf(dm.widthPixels - dp(32), dp(CARD_MAX_WIDTH_DP)),+ View.MeasureSpec.EXACTLY,+ ),+ View.MeasureSpec.makeMeasureSpec(0, View.MeasureSpec.UNSPECIFIED),+ )+ cardReal.tailX = cardReal.measuredWidth / 2++ // ---- handle + callbacks ----------------------------------------+ val handle = CardHandle(cardReal, host)+ handle.onDone = { r -> main.post { onDone(r) } }+ wireButtons(handle, btnViews, btnLabels, cancelable, cardReal, title)+ val timeoutRunnable = Runnable { handle.close("timeout") }+ handle.timeout = timeoutRunnable+ main.postDelayed(timeoutRunnable, timeoutMs)++ if (host.showSignboard(cardReal)) {+ current = handle+ ensureEvictorWired(host)+ // Grows out of the ball's head instead of blinking into being.+ // The pivot is the tail tip — the point the card is held by —+ // so the tail stays on the head for the whole 320 ms rather+ // than sweeping across it. No matching exit animation: taking+ // a card down resizes the window, and a card that is still+ // fading is a card the next `ask` cannot replace (one+ // signboard at a time, and the second would be refused).+ cardReal.pivotX = cardReal.measuredWidth / 2f+ cardReal.pivotY = cardReal.measuredHeight.toFloat()+ Glass.enter(cardReal)+ // The ball holds still and attends while its question is up.+ // Armed after hosting so a throw in between can't leave the+ // ball frozen mid-attend.+ host.setHolding(true)+ } else {+ handle.finished = true+ handle.timeout?.let { t -> main.removeCallbacks(t) }+ // The other silent one. `showSignboard` refuses when a signboard+ // is already up (it hosts one at a time) or when the ball window+ // is not there to hold anything — both of which mean this+ // message never reached the user, and both of which used to+ // leave no trace anywhere.+ Log.w(TAG, "signboard refused to host the card")+ onDone(Result(null, "error: signboard busy"))+ }+ }+ }++ /** Shared button wiring for both the hosted and the fallback card. */+ private fun wireButtons(+ handle: CardHandle,+ btnViews: List<TextView>,+ btnLabels: List<String>,+ cancelable: Boolean,+ cardReal: View,+ title: View,+ ) {+ btnViews.forEachIndexed { i, btn ->+ btn.setOnClickListener { handle.close("button", btnLabels[i]) }+ }+ if (cancelable) {+ // Alert: tapping the card body itself = read.+ cardReal.setOnClickListener { handle.close("tap_outside") }+ title.setOnClickListener { handle.close("tap_outside") }+ }+ }++ /**+ * Card body that also draws the pointing tail. The tail is a downward+ * triangle hanging from the card's bottom edge at [tailX]; when [tailX]+ * is negative it isn't drawn (the no-ball fallback placement).+ *+ * ## Why it is painted in three passes+ *+ * The card used to be one flat near-opaque slate rectangle, which next to+ * the scrollback's panes read as a different app's dialog dropped on top+ * of this one. It is now the same recipe `HistoryListView.glass()` uses,+ * for the same reason: a translucent base, a top-to-bottom white film+ * standing in for light falling across a pane, and a hairline rim one step+ * brighter than the film so the pane has an edge that catches that light.+ *+ * There is no real blur behind it, and there cannot be one. Android 12's+ * shape-aware `Window.setBackgroundBlurRadius` needs a `Window`, which a+ * view added through `WindowManager.addView` does not have; the one that+ * works here, `LayoutParams.setBlurBehindRadius`, blurs a hard *rectangle*+ * behind the whole window and would leave a blurred oblong halo around a+ * rounded card with a tail hanging off it. What actually reads as glass on+ * a dark surface is the film and the rim, not the blur.+ *+ * The base stays in the high 0x80s of alpha for the plainest of reasons:+ * with nothing blurred behind it, the card floats over whatever the user+ * was reading, and any more transparency than this turns the question into+ * a puzzle. Note this is *pixel* alpha inside a TRANSLUCENT window — the+ * window's own alpha is untouched, which matters because Android's+ * untrusted-touch accounting reads that one and silently drops injected+ * gestures once our overlays pass 0.80 combined.+ *+ * ## One silhouette, not two shapes+ *+ * Body and tail are unioned into a single path before anything is painted.+ * Filling them separately was fine for a flat colour, but a film gradient+ * would restart inside the tail and the rim would draw a line straight+ * across the card's bottom edge, cutting the tail off from the card it+ * hangs from.+ */+ private class TailLinearLayout(ctx: Context) : LinearLayout(ctx) {++ var tailX: Int = -1+ set(value) {+ field = value+ silhouette = null+ }++ var cornerRadius: Int = 0+ set(value) {+ field = value+ silhouette = null+ }++ private val density = resources.displayMetrics.density++ private val basePaint = Paint(Paint.ANTI_ALIAS_FLAG).apply { color = BASE }+ private val filmPaint = Paint(Paint.ANTI_ALIAS_FLAG)+ private val rimPaint = Paint(Paint.ANTI_ALIAS_FLAG).apply {+ style = Paint.Style.STROKE+ color = RIM+ strokeWidth = (density * 0.7f).coerceAtLeast(1f)+ }++ /** Rebuilt on the first draw at a new size; see [tailX]/[cornerRadius]. */+ private var silhouette: Path? = null++ /** How far the tail reaches below the card's content box, px. */+ fun tailLength(): Int = (14 * density).toInt().coerceAtLeast(12)++ /**+ * The tail draws below `height`, and a view cannot draw outside its+ * own bounds without being clipped — a WRAP_CONTENT window measured+ * at the content height sheared the tail off entirely (verified on+ * device: card bottom edge, then desktop pixels). Growing the+ * measured height by exactly the tail's reach gives it room.+ */+ override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {+ super.onMeasure(widthMeasureSpec, heightMeasureSpec)+ if (tailX >= 0) {+ setMeasuredDimension(measuredWidth, measuredHeight + tailLength())+ }+ }++ override fun onSizeChanged(w: Int, h: Int, oldW: Int, oldH: Int) {+ super.onSizeChanged(w, h, oldW, oldH)+ silhouette = null+ if (h > 0) {+ // Light falls across the pane from the top, so the film has to+ // run the card's real height — a shader can only be built once+ // that height is known, which is why this isn't a constant.+ filmPaint.shader = LinearGradient(+ 0f, 0f, 0f, h.toFloat(),+ FILM_TOP, FILM_BOTTOM,+ Shader.TileMode.CLAMP,+ )+ }+ }++ override fun dispatchDraw(canvas: Canvas) {+ val shape = silhouette ?: buildSilhouette().also { silhouette = it }+ canvas.drawPath(shape, basePaint)+ canvas.drawPath(shape, filmPaint)+ canvas.drawPath(shape, rimPaint)+ super.dispatchDraw(canvas)+ }++ private fun buildSilhouette(): Path {+ val t = if (tailX >= 0) tailLength() else 0+ // Inset by half the rim so the stroke lands inside the view. A+ // centred stroke on the view's own edge loses its outer half to+ // clipping, which reads as a rim that is there on three sides and+ // thinner on the fourth.+ val inset = rimPaint.strokeWidth / 2f+ val body = Path().apply {+ addRoundRect(+ inset, inset,+ width - inset, (height - t) - inset,+ cornerRadius.toFloat(), cornerRadius.toFloat(),+ Path.Direction.CW,+ )+ }+ if (t <= 0) return body+ val half = t * 0.62f+ val tail = Path().apply {+ // The base is a few pixels *inside* the body, not level with+ // its bottom edge. Level, the two shapes meet along a single+ // line, the union keeps that line, and the rim drew itself+ // straight across the tail's mouth — a triangle taped to the+ // card rather than part of it. A real overlap gives the union+ // something to dissolve.+ val baseY = (height - t) - inset - OVERLAP * density+ moveTo(tailX - half, baseY)+ lineTo(tailX + half, baseY)+ lineTo(tailX.toFloat(), height - inset)+ close()+ }+ body.op(tail, Path.Op.UNION)+ return body+ }++ private companion object {+ /** How far the tail's base reaches up into the card body, dp. */+ const val OVERLAP = 2f++ /**+ * Black, and nearly opaque. [Glass.CARD_SOLID] is the fill the+ * settings card falls back to when the compositor refuses to blur+ * behind it, which is permanently this card's situation — so it is+ * the same value for the same reason, not a coincidence worth+ * re-deciding. See the class KDoc on why not lower.+ */+ val BASE = Color.parseColor(Glass.CARD_SOLID)++ // Alpha in the low teens is the whole trick: high enough to lift+ // the pane off what is behind it, low enough that the base colour+ // still shows through, which is the only thing making it read as+ // translucent rather than as grey paint.+ val FILM_TOP = 0x2EFFFFFF+ val FILM_BOTTOM = 0x0CFFFFFF+ val RIM = 0x3DFFFFFF+ }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ChatHistory.kt b/app/src/main/java/net/kuafuai/andee/ui/ChatHistory.ktnew file mode 100644index 0000000..22e9eaa--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ChatHistory.kt@@ -0,0 +1,254 @@+package net.kuafuai.andee.ui++import android.content.Context+import android.util.Log+import org.json.JSONArray+import org.json.JSONObject+import java.io.File++/**+ * What has been said and shown on this device, in order, so the card has+ * something to scroll.+ *+ * Deliberately *this* side's record and not the brain's transcript. The brain+ * has the real conversation — tool calls, reasoning, the lot — but it lives+ * behind a socket that drops, and asking it for scrollback would mean the list+ * is empty exactly when the network is the thing that broke. What the device+ * saw with its own eyes and said with its own speaker is always available, and+ * it is also the only part the user actually witnessed, which makes it the part+ * they would be scrolling back to find.+ *+ * Three kinds of entry, because three things happen in front of the user: they+ * speak ([Role.USER]), the assistant answers out loud ([Role.ASSISTANT]), and+ * the assistant draws a page ([Role.PAGE]). Only the third is re-openable, and+ * it is the reason this class owns files at all. [Role.ERROR] is the fourth and+ * differs in kind: it is not something anyone said, it is the device failing,+ * and it is here because the card that announces a failure times out while the+ * question "what went wrong a minute ago" does not.+ *+ * Singleton with an [init] rather than a constructor-injected dependency,+ * matching [CardUi]: the appenders are scattered across the service, the+ * dispatcher and the hub client, and threading one object through all of them+ * buys nothing when there is exactly one device.+ *+ * Thread-safe. Appends arrive from the ASR callback, the hub receive thread and+ * dispatcher workers; reads come from the UI thread.+ */+object ChatHistory {++ enum class Role { USER, ASSISTANT, PAGE, ERROR }++ /**+ * @param page for [Role.PAGE], the archived HTML — null once it has been+ * evicted, which is how a row knows to stop offering to re-open itself.+ */+ data class Entry(+ val id: Long,+ val ts: Long,+ val role: Role,+ val text: String,+ val page: File?,+ )++ /**+ * Two separate caps, because the two costs are unrelated: a line of text is+ * a few hundred bytes and the list is only worth scrolling if it goes back+ * a while, whereas a generated page is routinely 50–100 KB of inlined CSS,+ * JS and base64 images. Keeping [MAX_ENTRIES] pages would be tens of+ * megabytes of an app-private directory nobody ever looks at.+ *+ * A page evicted from disk leaves its row behind with `page = null`. The+ * row stays because "you showed me a 贪吃蛇 at 3pm" is still true and still+ * worth scrolling past; it just stops claiming to be re-openable.+ */+ private const val MAX_ENTRIES = 80+ private const val MAX_PAGES = 12++ /**+ * How much of a message is kept.+ *+ * It used to be 400 — "long enough to identify a row at a glance, short+ * enough to stay cheap" — and that was right while a row *was* the message:+ * six lines on screen, and the rest never seen. It stopped being right when+ * a row gained a reader: double-tap opens the message fullscreen+ * ([TextPage]), and a 400-char cap would mean the reader showed the same+ * sentence and a half the row already showed, with the rest gone.+ *+ * So the cap is now about how long an answer can be rather than about how+ * much fits in a bubble. 4000 characters is roughly three screens of+ * Chinese — past what any spoken answer gets to — and it costs 40 KB per+ * row at [MAX_ENTRIES], which is the JSON rewritten on each message.+ */+ private const val MAX_TEXT = 4000++ private const val TAG = "Body"++ private var appContext: Context? = null+ private val lock = Any()+ private val entries = ArrayList<Entry>()+ private var nextId = 1L++ /** Notified on every change; the card re-reads [snapshot] and redraws. */+ @Volatile+ private var listener: (() -> Unit)? = null++ fun init(context: Context) {+ synchronized(lock) {+ if (appContext != null) return+ appContext = context.applicationContext+ load()+ }+ listener?.invoke()+ }++ fun setListener(l: (() -> Unit)?) {+ listener = l+ }++ /** Oldest first — the order the card lays rows out in. */+ fun snapshot(): List<Entry> = synchronized(lock) { ArrayList(entries) }++ /**+ * Delete everything and forget it. For 恢复出厂设置.+ *+ * Two halves, and both are needed: the files on disk, and the list this+ * singleton is holding in memory. Clearing only the files would leave the+ * card still scrolling a conversation whose every row had been deleted —+ * and the next append would write that list back out.+ *+ * The listener is invoked so an open card redraws empty instead of holding+ * rows whose backing pages no longer exist.+ */+ fun wipe(context: Context) {+ synchronized(lock) {+ entries.clear()+ nextId = 1L+ }+ runCatching {+ File(context.filesDir, "chat_history.json").delete()+ File(context.filesDir, "pages").deleteRecursively()+ }.onFailure { Log.w(TAG, "history: wipe failed: ${it.message}") }+ listener?.invoke()+ }++ fun addUser(text: String) = add(Role.USER, text, null)++ fun addAssistant(text: String) = add(Role.ASSISTANT, text, null)++ /** Call through [CardUi.error], which de-dups bursts before they get here. */+ fun addError(text: String) = add(Role.ERROR, text, null)++ /**+ * Archive a page the brain just composed and record it.+ *+ * Returns the archived file for the caller to hand to [HtmlActivity], so+ * the copy being displayed and the copy in the history are the same bytes+ * in the same place — an earlier split (cache dir for display, files dir+ * for the archive) meant re-opening from history could show something+ * subtly different from what was on screen a minute ago.+ *+ * Falls back to null on IO failure; the caller still has the HTML string+ * and can stage it itself, it just won't be in the list.+ */+ fun addPage(title: String, html: String): File? {+ val ctx = appContext ?: return null+ val id = synchronized(lock) { nextId }+ val file = runCatching {+ val dir = File(ctx.filesDir, "pages").apply { mkdirs() }+ File(dir, "page_$id.html").also { it.writeText(html) }+ }.getOrElse {+ Log.w(TAG, "history: could not archive page: ${it.message}")+ return null+ }+ add(Role.PAGE, title, file)+ return file+ }++ private fun add(role: Role, text: String, page: File?) {+ if (appContext == null) return+ val clean = text.trim().let {+ if (it.length > MAX_TEXT) it.take(MAX_TEXT) + "…" else it+ }+ if (clean.isEmpty()) return+ synchronized(lock) {+ entries.add(Entry(nextId++, System.currentTimeMillis(), role, clean, page))+ trim()+ save()+ }+ listener?.invoke()+ }++ /** Caller holds [lock]. */+ private fun trim() {+ while (entries.size > MAX_ENTRIES) {+ entries.removeAt(0).page?.let { runCatching { it.delete() } }+ }+ // Newest page kept, oldest dropped. Counted from the end so the walk+ // stops as soon as the budget is spent instead of scanning every row.+ var pages = 0+ for (i in entries.indices.reversed()) {+ val e = entries[i]+ if (e.page == null) continue+ pages+++ if (pages > MAX_PAGES) {+ runCatching { e.page.delete() }+ entries[i] = e.copy(page = null)+ }+ }+ }++ // ---- Persistence ----+ //+ // A plain JSON file rather than SharedPreferences: this is an append-only+ // log with a per-row shape, and VoiceConfig's prefs file is the *settings*,+ // which are hand-edited and whitelisted (see its ALLOWED_KEYS). Rewriting+ // the whole log on every append is fine at 80 rows and saves having to+ // reason about partial writes.++ private fun file(): File? = appContext?.let { File(it.filesDir, "chat_history.json") }++ /** Caller holds [lock]. */+ private fun save() {+ val f = file() ?: return+ runCatching {+ val arr = JSONArray()+ for (e in entries) {+ arr.put(+ JSONObject()+ .put("id", e.id)+ .put("ts", e.ts)+ .put("role", e.role.name)+ .put("text", e.text)+ .put("page", e.page?.name ?: JSONObject.NULL)+ )+ }+ f.writeText(arr.toString())+ }.onFailure { Log.w(TAG, "history: save failed: ${it.message}") }+ }++ /** Caller holds [lock]. */+ private fun load() {+ val f = file() ?: return+ if (!f.exists()) return+ runCatching {+ val dir = File(appContext!!.filesDir, "pages")+ val arr = JSONArray(f.readText())+ for (i in 0 until arr.length()) {+ val o = arr.optJSONObject(i) ?: continue+ val role = runCatching { Role.valueOf(o.optString("role")) }.getOrNull() ?: continue+ // Re-checked against the filesystem rather than trusted: the+ // log and the pages directory are written separately and the+ // process can die between them, and a row that offers to open+ // a file that isn't there is worse than one that doesn't.+ val page = o.optString("page").takeIf { it.isNotEmpty() && it != "null" }+ ?.let { File(dir, it) }?.takeIf { it.exists() }+ val id = o.optLong("id")+ entries.add(Entry(id, o.optLong("ts"), role, o.optString("text"), page))+ if (id >= nextId) nextId = id + 1+ }+ }.onFailure {+ Log.w(TAG, "history: load failed, starting empty: ${it.message}")+ entries.clear()+ }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/EdgeRippleUi.kt b/app/src/main/java/net/kuafuai/andee/ui/EdgeRippleUi.ktnew file mode 100644index 0000000..a472447--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/EdgeRippleUi.kt@@ -0,0 +1,328 @@+package net.kuafuai.andee.ui++import android.content.Context+import android.graphics.Canvas+import android.graphics.Color+import android.graphics.ComposeShader+import android.graphics.LinearGradient+import android.graphics.Matrix+import android.graphics.Paint+import android.graphics.PixelFormat+import android.graphics.PorterDuff+import android.graphics.Shader+import android.graphics.SweepGradient+import android.os.Handler+import android.os.Looper+import android.os.SystemClock+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import kotlin.math.cos+import kotlin.math.max+import kotlin.math.min++/**+ * Breathing rainbow glow around the four screen edges — "the agent is driving+ * this screen right now". Siri-style status light, not a control.+ *+ * Its own window, deliberately, and that's the whole reason this class exists+ * instead of a View inside [FloatingWindowUi]: the glow's main job is to be+ * visible while the card is *blanked* (folded away, screenshot), and+ * [FloatingWindowUi.setCompact] gets there by driving that window's alpha to+ * 0 — which would take a child View down with it.+ *+ * Always FLAG_NOT_TOUCHABLE. A status light that eats taps along every edge+ * would swallow back-gestures and notification pulls across the whole device.+ *+ * Being a separate window also means it is composited into+ * `AccessibilityService.takeScreenshot` on its own, so the capture path has to+ * blank it too — see [setVisibleForCapture].+ *+ * The load-bearing constraint, though, is touch: since Android 12 a synthesized+ * gesture is dropped when the app's overlays obscure the target at more than+ * 0.80 combined opacity, FLAG_NOT_TOUCHABLE or not, and the platform clamps an+ * untrusted non-touchable overlay to exactly 0.80. So this window plus the card+ * — also non-touchable while a gesture runs — compute to+ * `1 - 0.2 * 0.2 = 0.96` and every tap `ScreenController` injects gets thrown+ * away. Hence [applyWindow]: the window sits at alpha 0 unless the glow is+ * actually drawing, and [setVisibleForGesture] takes it out of the sum for the+ * ~1.5 s the glow keeps breathing after a screenshot.+ */+class EdgeRippleUi(private val context: Context) {++ private val wm: WindowManager =+ context.getSystemService(Context.WINDOW_SERVICE) as WindowManager+ private val ui = Handler(Looper.getMainLooper())++ private var view: RippleView? = null+ private var params: WindowManager.LayoutParams? = null++ // Folded into the window alpha by [applyWindow]. Main thread only.+ private var captureHidden = false+ private var gestureHidden = false++ /** Whether the view is still drawing frames — see [applyWindow]. */+ private var glowing = false++ fun show(): Boolean {+ if (view != null) return true+ val v = RippleView(context) { onGlowIdle() }+ val p = WindowManager.LayoutParams(+ WindowManager.LayoutParams.MATCH_PARENT,+ WindowManager.LayoutParams.MATCH_PARENT,+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE+ or WindowManager.LayoutParams.FLAG_NOT_TOUCHABLE+ or WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.TOP or Gravity.START+ x = 0; y = 0+ alpha = 0f+ }+ return try {+ v.hideFromAccessibility()+ wm.addView(v, p)+ view = v+ params = p+ android.util.Log.i("Body", "edge ripple window added")+ true+ } catch (t: Throwable) {+ android.util.Log.e("Body", "edge ripple addView failed", t)+ false+ }+ }++ fun hide(): Boolean {+ val v = view ?: return true+ return try {+ wm.removeView(v)+ view = null+ params = null+ true+ } catch (_: Throwable) {+ false+ }+ }++ /**+ * Light the glow (fades in, breathes) or release it (fades out after a+ * minimum on-time, so a ~300 ms screenshot still reads as one breath+ * rather than a flicker).+ */+ fun setBreathing(on: Boolean) {+ ui.post {+ if (on) {+ glowing = true+ applyWindow()+ }+ view?.setBreathing(on)+ }+ }++ /**+ * Same trick as [FloatingWindowUi.setVisibleForCapture], for the same+ * reason: without it every screenshot the brain receives has a glowing+ * border burned into it. Animation state is untouched, so the glow picks up+ * mid-breath when the capture is done.+ */+ fun setVisibleForCapture(visible: Boolean) {+ ui.post { captureHidden = !visible; applyWindow() }+ }++ /**+ * Drop out of the way of an injected gesture. Not cosmetic — see the class+ * KDoc: a glow still fading from the previous screenshot pushes the app past+ * the untrusted-touch ceiling and the tap silently never lands.+ */+ fun setVisibleForGesture(visible: Boolean) {+ ui.post { gestureHidden = !visible; applyWindow() }+ }++ /** Main thread only. */+ private fun onGlowIdle() {+ if (!glowing) return+ glowing = false+ applyWindow()+ }++ /**+ * Main thread only. An idle glow has to leave the window at alpha 0 rather+ * than just draw nothing: the platform bills us for the window's opacity,+ * not its pixels.+ */+ private fun applyWindow() {+ val v = view ?: return+ val p = params ?: return+ val want = if (captureHidden || gestureHidden || !glowing) 0f else 1f+ if (p.alpha == want) return+ p.alpha = want+ runCatching { wm.updateViewLayout(v, p) }+ }++ /**+ * Four inward LinearGradients, one per edge. Corners overlap and blend to+ * something brighter, which is what gives the effect its shape.+ *+ * Not a blurred stroked Path: BlurMaskFilter isn't hardware accelerated, so+ * a fullscreen one means a multi-megabyte software layer re-rasterized every+ * frame.+ */+ private class RippleView(+ context: Context,+ private val onIdle: () -> Unit,+ ) : View(context) {++ private val paint = Paint(Paint.ANTI_ALIAS_FLAG)++ /**+ * How far the glow reaches in from an edge. Thin on purpose: a wide+ * band covers real content and reads as the assistant still being in+ * the way, which is the opposite of what getting out of the way is for. This is a lit+ * rim with a short falloff, not a vignette.+ */+ private val maxBand = 24f * context.resources.displayMetrics.density++ private var band = 0f+ private var top: LinearGradient? = null+ private var bottom: LinearGradient? = null+ private var left: LinearGradient? = null+ private var right: LinearGradient? = null++ /**+ * The color, shared by all four edges. Centered on the view so hue+ * follows the angle out from the middle — which means it wraps around+ * all four corners with no seam, the thing a per-edge gradient can't+ * do. Rotated over time by [spin] for the sci-fi drift.+ */+ private var sweep: SweepGradient? = null+ private val spin = Matrix()+ private var cx = 0f+ private var cy = 0f++ private fun tint(color: Shader?, mask: Shader?): Shader? =+ if (color != null && mask != null) {+ ComposeShader(color, mask, PorterDuff.Mode.DST_IN)+ } else {+ mask+ }++ /** 0..1 fade envelope, independent of the breathing oscillation. */+ private var envelope = 0f+ private var lit = false+ private var litSince = 0L+ private var lastFrame = 0L+ private var phaseStart = 0L++ fun setBreathing(on: Boolean) {+ if (on) {+ // Restart the sine phase only on a cold start, so repeated+ // lights during one task don't stutter the rhythm.+ if (envelope <= 0f) phaseStart = SystemClock.uptimeMillis()+ litSince = SystemClock.uptimeMillis()+ }+ lit = on+ invalidate()+ }++ override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) {+ // Never let the bands meet in the middle on a narrow window.+ band = min(maxBand, min(w, h) * 0.22f)+ // Opaque white → transparent: these carry only the inward falloff.+ // The color comes from [sweep], and DST_IN keeps its pixels+ // wherever these are opaque.+ top = LinearGradient(0f, 0f, 0f, band, OPAQUE, CLEAR, Shader.TileMode.CLAMP)+ bottom = LinearGradient(0f, h.toFloat(), 0f, h - band, OPAQUE, CLEAR, Shader.TileMode.CLAMP)+ left = LinearGradient(0f, 0f, band, 0f, OPAQUE, CLEAR, Shader.TileMode.CLAMP)+ right = LinearGradient(w.toFloat(), 0f, w - band, 0f, OPAQUE, CLEAR, Shader.TileMode.CLAMP)+ cx = w / 2f+ cy = h / 2f+ sweep = SweepGradient(cx, cy, RAINBOW, null)+ }++ override fun onDraw(canvas: Canvas) {+ val now = SystemClock.uptimeMillis()+ // First frame of a run has no previous timestamp; a dropped frame+ // shouldn't jump the envelope either.+ val dt = if (lastFrame == 0L) 0f else min(0.05f, (now - lastFrame) / 1000f)+ lastFrame = now++ val wantLit = lit || (now - litSince) < MIN_ON_MS+ envelope = if (wantLit) min(1f, envelope + dt / FADE_IN_S)+ else max(0f, envelope - dt / FADE_OUT_S)++ if (envelope > 0f) {+ // cos so a cold start begins dim and swells, rather than+ // snapping to mid-brightness.+ val t = (now - phaseStart) / 1000f+ val a = envelope * (0.55f - 0.45f * cos(t * BREATH_RATE))+ paint.alpha = (a.coerceIn(0f, 1f) * 255f).toInt()++ val w = width.toFloat()+ val h = height.toFloat()+ val s = sweep+ if (s != null) {+ spin.setRotate(t * SPIN_DEG_S, cx, cy)+ s.setLocalMatrix(spin)+ }+ // Rebuilt every frame on purpose. ComposeShader resolves its+ // children when it's constructed and isn't marked dirty when+ // one of them changes, so rotating the sweep in place does not+ // reach a composer built earlier. Four small shaders per frame,+ // only while the glow is lit.+ paint.shader = tint(s, top); canvas.drawRect(0f, 0f, w, band, paint)+ paint.shader = tint(s, bottom); canvas.drawRect(0f, h - band, w, h, paint)+ paint.shader = tint(s, left); canvas.drawRect(0f, 0f, band, h, paint)+ paint.shader = tint(s, right); canvas.drawRect(w - band, 0f, w, h, paint)+ }++ // `wantLit` has to keep the loop alive on its own: the cold-start+ // frame has dt == 0, so the envelope is still exactly 0 here and+ // keying the next frame off `envelope > 0f` alone would stall the+ // glow before it ever lights.+ if (wantLit || envelope > 0f) {+ postInvalidateOnAnimation()+ } else {+ // Idle costs nothing: the empty display list clears the+ // surface, and no further frames are scheduled. Posted rather+ // than called — the listener touches window layout, which has+ // no business happening inside a draw pass.+ lastFrame = 0L+ post(onIdle)+ }+ }++ companion object {+ /**+ * Hue around the ring. Last stop repeats the first so the sweep+ * closes on itself instead of showing a hard edge at 0°.+ */+ private val RAINBOW = intArrayOf(+ Color.parseColor("#FFFF4FA3"), // pink+ Color.parseColor("#FFB14BFF"), // violet+ Color.parseColor("#FF4F7BFF"), // blue+ Color.parseColor("#FF2DE2FF"), // cyan+ Color.parseColor("#FF3DFFB0"), // spring+ Color.parseColor("#FFFFE04F"), // amber+ Color.parseColor("#FFFF7A4F"), // coral+ Color.parseColor("#FFFF4FA3"), // pink again+ )++ /** Mask endpoints — only the alpha matters, see [onSizeChanged]. */+ private const val OPAQUE = Color.WHITE+ private const val CLEAR = Color.TRANSPARENT++ /** Degrees per second the rainbow drifts around the ring. */+ private const val SPIN_DEG_S = 18f++ private const val FADE_IN_S = 0.25f+ private const val FADE_OUT_S = 0.6f++ /** One breath is ~2.9 s at this rate. */+ private const val BREATH_RATE = 2.2f++ /** Long enough that a single screenshot still shows a full swell. */+ private const val MIN_ON_MS = 900L+ }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/FloatingWindowUi.kt b/app/src/main/java/net/kuafuai/andee/ui/FloatingWindowUi.ktnew file mode 100644index 0000000..8dfa74b--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/FloatingWindowUi.kt@@ -0,0 +1,2565 @@+package net.kuafuai.andee.ui++import android.animation.ValueAnimator+import android.annotation.SuppressLint+import android.content.Context+import android.graphics.Canvas+import android.graphics.Color+import android.graphics.Paint+import android.graphics.PixelFormat+import android.graphics.Rect+import android.graphics.drawable.ColorDrawable+import android.os.Handler+import android.os.Looper+import android.util.TypedValue+import android.view.Gravity+import android.view.MotionEvent+import android.view.View+import android.view.ViewConfiguration+import android.view.WindowInsets+import android.view.WindowManager+import android.view.animation.DecelerateInterpolator+import android.widget.FrameLayout+import android.widget.ImageView+import android.widget.LinearLayout+import android.widget.TextView+import net.kuafuai.andee.R+import net.kuafuai.andee.audio.Earcon+import net.kuafuai.andee.config.VoiceConfig+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.ui.ball.Action+import net.kuafuai.andee.ui.ball.BallLooks+import net.kuafuai.andee.ui.ball.CLING_HIDDEN+import net.kuafuai.andee.ui.ball.EmotionBallTextureView+import net.kuafuai.andee.ui.ball.Mood+import net.kuafuai.andee.ui.ball.SPHERE_RADIUS_FRACTION+import kotlin.math.abs+import kotlin.math.min+import kotlin.math.roundToInt+import kotlin.math.sqrt+import kotlin.random.Random++/**+ * The floating window IS the ball. Fullscreen by default.+ *+ * Layout (FrameLayout, two stacked areas plus a floating control pill):+ * ┌──── ball area (top 40%) ─────────────────┐+ * │ ⋯ ✓ ⚙ ✕ │ ← tools / self-check / gear / close+ * │ │+ * │ [round ball] │ ← tap = talk+ * │ │ ← double-tap = type+ * │ │ ← triple-tap = stop+ * │ │ ← swipe = switch character+ * │ │ ← long-press = unfold+ * │ │ ← drag = move (compact only)+ * ├──── scrollback (the rest) ───────────────┤+ * │ 你 · 15:31 帮我看看今天的天气 │+ * │ 助手 · 15:34 今天多云… │+ * │ ▸ live row: what is happening now │+ * ├──────────────────────────────────────────┤+ * │ 点小球说话 · 双击打字 · 三连击停止 · 滑动切换性格 │ ← pinned tips+ * └──────────────────────────────────────────┘+ *+ * One control, three taps, and that is the whole tap vocabulary the user has+ * to learn — the ball is the only thing on screen in both shapes, so all+ * three gestures work folded and unfolded alike. See+ * [BallHostFrame.registerTap]. The swipe is the one gesture that is+ * full-card-only, which is why the tips line spells it out.+ *+ * Everything the assistant hears, says, shows or fails at ends up in that one+ * list, in order. There is no separate subtitle band: [setSubtitle] writes the+ * list's live last row instead, so the user watches one surface rather than+ * two, and a sentence cannot appear on both at once (it used to).+ *+ * Touch handling:+ * - The control pill's buttons are child Views, consume their own taps+ * - Ball area (top): circular hit test → tap/drag/long-press+ * - Scrollback (bottom): its own touches, so it can scroll+ * - Anywhere outside the circle in the ball area: passes through+ *+ * Tap opens the mic and nothing else — deliberately not "tap, then unfold to+ * talk": the corner ball is there so you can ask about what you are looking+ * at, and covering it again to do so defeats it. Recognition state shows on+ * the ball itself. Long-press is how the user gets the full card back, which+ * is also why ⤡ isn't a toggle.+ *+ * The card is black glass, not paint: [Glass] gives it a translucent fill over+ * a real blur of whatever is behind ([applyWindow] asks for it, and only while+ * the card is unfolded and visible). What it does *not* become is see-through+ * in any useful sense — 60% black over a defocused app is still a wall — so+ * the overlay still hides the app underneath from the *camera* as well as the+ * eye: see [setVisibleForCapture] for why every screenshot has to blank the+ * window first, and why that blank drops the blur too.+ *+ * Two geometries exist. Fullscreen: the assistant card. Compact: a bare ball+ * in the bottom-right corner, which is what the card becomes — and stays —+ * whenever the assistant is driving the device underneath (every `screen.*`+ * command folds the card before it runs), plus whenever the user asks with ⤡.+ * Both routes reach it through [setCompact]; the way back is a long-press on+ * the ball ([Listeners.onLongPress]) with exactly one caller that does not+ * wait for one: the service unfolds once at birth, so a device that just+ * booted shows the card rather than a corner ball. See+ * `ScreenBodyService.onServiceConnected`.+ */+class FloatingWindowUi(+ private val context: Context,+ private val listeners: Listeners,+) {++ enum class State { IDLE, RECORDING, THINKING, SPEAKING, ERROR }++ /**+ * What is happening right now, as the scrollback's live last row — a partial+ * transcript, the tool being run, the state a command left behind.+ * Everything here is superseded by a real entry or cleared; nothing settles+ * into the log through this path.+ *+ * Deliberately *not* the sentence being spoken: that one is already a row in+ * the log by the time it is read out, and echoing it here put the same words+ * on screen twice.+ */+ enum class SubtitleKind { PARTIAL, FINAL }++ /** Screen tools shown as a row of small buttons in the top-right corner. */+ enum class Tool(val glyph: String) {+ SWIPE_UP("↑"),+ SWIPE_DOWN("↓"),+ SWIPE_LEFT("←"),+ SWIPE_RIGHT("→"),+ BACK("⌫"),+ HOME("⌂"),+ UI_TREE("☰"),+ SCREENSHOT("◉"),+ }++ interface Listeners {+ fun onTalkClick()+ fun onSettingsClick()++ /**+ * ✓ — the self-check card.+ *+ * Next to ⚙ because they are the same kind of thing to the user: two+ * buttons that open our own surfaces rather than acting on the screen,+ * so they open the same *shape* of surface. It used to open a fullscreen+ * page — see [net.kuafuai.andee.ui.SelfCheckUi] for why that made two+ * adjacent buttons behave like two different products, and for what it+ * costs to have it be a card instead.+ *+ * Handed out rather than opened here for the same reason+ * [onSettingsClick] is: this class draws the card, the service owns what+ * is behind it and what it is allowed to open.+ */+ fun onSelfCheckClick()+ fun onLongPress()+ fun onToolClick(tool: Tool)++ /**+ * `■` is gone from the bar; stop is still here.+ *+ * The button was the third of four glyphs crammed into one pill, and it+ * was the only one of them that duplicated a gesture the ball already+ * had: triple-tap fires the same [onStopClick], and has since the taps+ * were reduced to one vocabulary. The tips line at the bottom of the+ * card has always said so — "三连击停止" — which is what makes deleting+ * the button a simplification rather than a removal: the user loses a+ * control they had two of and keeps the one that works folded as well.+ */+ fun onStopClick()++ /** ⤡ in the subtitle band — the user asking for the screen back. */+ fun onMinimizeClick()++ /**+ * Two taps on the ball — the user wants to type instead of talk.+ *+ * On the ball rather than as a key in the top bar, because the ball is+ * the one control that exists in **both** shapes. The bar is `GONE`+ * while folded, so a bar key could only ever be a way in from the state+ * the user is already looking at the card in; the ball is there either+ * way, and the three taps now read as one set: ask, type, stop.+ *+ * Handed out rather than opened here for two reasons that are both+ * about the window system rather than about taste: the field is an+ * Activity ([TextInputActivity]) because an overlay cannot raise a soft+ * keyboard, and the card has to get out of its way before it opens+ * (`setTypingTop`) or it sits on top of it and eats every touch.+ */+ fun onTextInputToggle()++ /**+ * A page row in the scrollback, tapped to be shown again.+ *+ * Handed out rather than opened here because re-showing a page is not+ * just starting an activity: the card has to fold out of its own way+ * first, exactly as `ui.show_html` does. That sequencing belongs to+ * whoever owns the dispatcher, not to the window.+ */+ fun onHistoryPageClick(entry: ChatHistory.Entry)++ /**+ * A message row, double-tapped, to be read fullscreen.+ *+ * Handed out for the same reason as [onHistoryPageClick]: the reader is+ * fullscreen, so the card has to fold out of its way first, and that+ * sequencing belongs to whoever owns the dispatcher.+ */+ fun onTextOpen(entry: ChatHistory.Entry)+ }++ private val wm: WindowManager =+ context.getSystemService(Context.WINDOW_SERVICE) as WindowManager+ private val ui = Handler(Looper.getMainLooper())++ private var root: BallHostFrame? = null+ private var params: WindowManager.LayoutParams? = null+ private var ball: EmotionBallTextureView? = null+ private var topBarView: View? = null+ private var toolsView: View? = null+ private var historyView: HistoryListView? = null++ /** The ball's gesture vocabulary, pinned along the bottom of the full card. */+ private var tipsView: View? = null++ // Folded into p.alpha / p.flags by [applyWindow]. Main thread only.+ private var captureHidden = false+ private var compact = false++ /** The typing field's top edge, in screen px. 0 = no field is up. */+ private var typingTop = 0++ /** The window geometry [setTypingTop] borrowed, to be put back. */+ private var typingGeometry: IntArray? = null+ private var gesturePassthrough = false++ /** Whether the compositor is blurring behind us. See [Glass.frost]. */+ private var frosted = false++ /**+ * The card's own background image, at the bottom of the root's z-order.+ *+ * Permanent child, `GONE` while folded: a fullscreen image inside a 168 dp+ * corner square is a photo with a ball on it, and the fold's whole job is to+ * get out of the way. See [Backdrop].+ */+ private var backdrop: ImageView? = null++ /**+ * The scrim currently painted over the backdrop, or `null` for none.+ *+ * Held so repeated [applyWindow] calls during one card don't rebuild the+ * drawable and invalidate the window for a value that has not changed.+ */+ private var scrimShown: Int? = null++ /** Main thread only: last state handed to [setState], to spot transitions. */+ private var lastState: State? = null++ /** Geometry to put back when the fold ends — see [enterCompact]. */+ private var fullGeometry: IntArray? = null++ /**+ * The three independent reasons the ball can be hanging off the ledge, and+ * the derived answer to "is it hanging".+ *+ * Split because they are owned by different parties and they *will*+ * disagree. The task owns [perchedByTask] — it clings while driving an app+ * and climbs back to speak. The user owns [perchedByUser] by dragging it+ * onto the edge, and that placement has to survive the next task ending, or+ * the ball would haul itself out of the corner the user deliberately put it+ * in. A page the brain put on screen owns [perchedByPage] for exactly as+ * long as that page is up.+ *+ * The page needs its own flag rather than borrowing the task's: `show_html`+ * almost always runs *inside* a task, so setting the task flag would be a+ * no-op on the way in and — far worse — clearing it on the way out would+ * haul the ball back up in the middle of a task that is still running.+ *+ * A hand on the ball outranks all of it: dragging it off the ledge clears+ * every flag, because "get out of my way" and "come back here" are the same+ * gesture from the user's side.+ *+ * Main thread only.+ */+ private var perchedByTask = false+ private var perchedByUser = false+ private var perchedByPage = false+ private val perched: Boolean get() = perchedByTask || perchedByUser || perchedByPage++ /**+ * Compact spot with no task running — where the ball climbs back to.+ *+ * Tracked explicitly rather than sampled off the window when a task starts,+ * because at that moment `p.y` may be halfway through the previous climb:+ * two tasks in quick succession would then record a mid-flight value as home+ * and the ball would settle somewhere it had never been.+ *+ * X is tracked for the same reason and one more: a cling *moves* the ball+ * sideways to [perchX], so on the way back up there is no longer any+ * "wherever it already is" to fall back on. Without [homeX] the ball would+ * climb off the ledge and stay pinned against the right screen edge at+ * whatever height it rests at, which reads as stuck rather than as parked —+ * a worse position than the corner it would have returned to.+ */+ private var homeX = 0+ private var homeY = 0++ /**+ * Whether the perch pose is currently on, which is not the same as+ * [perched]: a task can start while the card is open, and the perch only+ * exists in the compact geometry. Also the drag's own latch — the finger+ * owns `p.y` mid-drag, so this flips as the ball crosses the ledge and the+ * geometry only catches up on release.+ */+ private var perchApplied = false++ // ---- Idle fidget (see [strollStep]) ----+ private val strollRunnable = Runnable { strollStep() }++ /** Hops left in the current burst before the next long rest. */+ private var strollLeft = 0++ private var slide: ValueAnimator? = null++ fun isShown(): Boolean = root != null++ /**+ * Fullscreen, and not draggable: the drag that moves the folded corner+ * ball is compact-only — the full card is a surface the user reads, not+ * a widget they reposition.+ *+ * Format stays TRANSLUCENT and the fill is the background colour's job —+ * see [paintCard]. The ball is a TextureView that blends its GL surface+ * against whatever the card is filled with, and [setVisibleForCapture]+ * drives window alpha to 0; both need a window the compositor is willing+ * to blend, whatever the card's own alpha happens to be.+ */+ fun show(): Boolean {+ if (root != null) return true+ // Before the ball exists, so the first frame is already the look the+ // user left it on. A no-op when nothing is saved, and a no-op for a+ // name that no longer matches a look — which is what makes deleting a+ // look from [BallLooks.ALL] safe.+ BallLooks.requestByName(VoiceConfig.ballLook(context))+ val v = build()+ val dm = context.resources.displayMetrics+ val side = dp(COMPACT_DP)+ val p = WindowManager.LayoutParams(+ side,+ side,+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ // Kept in step with [applyWindow], which owns these from here on —+ // FLAG_LAYOUT_IN_SCREEN in particular, or the first measurement of+ // [winYOrigin] is taken in a frame the window is about to leave.+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE+ or WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS+ or WindowManager.LayoutParams.FLAG_LAYOUT_IN_SCREEN,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.TOP or Gravity.START+ x = dm.widthPixels - side - dp(COMPACT_MARGIN_DP)+ y = dm.heightPixels - side - dp(COMPACT_MARGIN_DP)+ }+ // Where the ball climbs back to when a task ends. [enterCompact] is the+ // other place this is set, and it is the one that usually runs — but+ // show() adds the window already folded and so never passes through it.+ // Left at its 0 default, the first unperch slid the ball to the top of+ // the screen.+ homeX = p.x+ homeY = p.y+ // Keep the restore target, but add the overlay already compact.+ //+ // This used to carry a second reason — "no frame in which a newly+ // enabled service owns a full-screen touchable window", because+ // returning false from onTouchEvent cannot forward a touch that has+ // already entered this window. That is no longer the service's+ // behaviour: it unfolds immediately after [show] (see+ // `ScreenBodyService.onServiceConnected`), so the card is up from the+ // first frame it is drawn in. What is left is the mechanical reason the+ // order is still worth keeping: compact is the state [fullGeometry] is+ // seeded from just below, and [exitCompact] is the single code path+ // that knows how to turn this window into a card — visibilities, ball+ // scale, scrollback refresh. Being born fullscreen would mean a second+ // way to arrive at that state, and the unfold is one `ui.post` away+ // anyway.+ compact = true+ // The restore target itself: skipped below precisely because we never+ // pass through enterCompact() (setCompact(true) is a no-op from this+ // state), and without it exitCompact() has nothing to grow back to —+ // the first unfold, whether that is the service's own at birth or a+ // long-press, would leave a fullscreen card's content inside a 160dp+ // corner square forever.+ fullGeometry = intArrayOf(+ 0, 0,+ WindowManager.LayoutParams.MATCH_PARENT,+ WindowManager.LayoutParams.MATCH_PARENT,+ )+ return try {+ v.hideFromAccessibility()+ wm.addView(v, p)+ root = v+ params = p+ applyCompactAppearance()+ // Only now can [winYOrigin] be measured, and the resting position+ // depends on it — the y above is a first guess that is correct only+ // where the offset is 0.+ ui.post {+ refreshWinYOrigin()+ val pp = params ?: return@post+ if (!perchApplied) {+ pp.y = compactHomeY()+ homeX = pp.x+ homeY = pp.y+ runCatching { wm.updateViewLayout(v, pp) }+ }+ }+ retimeStroll()+ true+ } catch (_: Throwable) {+ false+ }+ }++ /** The signboard content while a card is up. Main thread only. */+ private var signboard: View? = null++ /**+ * Who to tell when a fold or unfold forces the signboard down.+ *+ * The window cannot be a corner square and host a card-sized pane at the+ * same time, so [enterCompact] and [exitCompact] evict rather than trying+ * to carry the signboard across the geometry change. Wired by CardUi, the+ * only thing that ever puts a signboard up.+ */+ var signboardEvictor: (() -> Unit)? = null++ /** True while the window is the folded corner ball. */+ fun isFolded(): Boolean = compact++ /**+ * Host a signboard inside this window, above the ball.+ *+ * This is the fix for the anchor drift the two-window design could never+ * quite shake: a separate card window positioned from a *measured* copy of+ * the ball's position goes stale whenever the ball is mid-slide, the+ * window origin offset is out of date, or a device's overlay coordinate+ * space disagrees with `getLocationOnScreen` — and each of those hits a+ * different subset of devices, which is why "sometimes it's off". Here+ * the card is a child of the very window the ball lives in. The tail is+ * centred on the window, the ball is centred in the window, and nothing+ * short of the layout system itself can put space between them.+ *+ * While folded, the window grows: the card rides on top, the ball area+ * drops to the bottom of the taller window, and `p.x/p.y` are recomputed+ * so the ball keeps its on-screen spot — the card *appears around* the+ * ball rather than the ball moving under a card. A perched ball climbs+ * back first (it cannot hold a sign hanging by its paws); [hideSignboard]+ * re-applies the perch, so the pose the sign interrupted survives it.+ *+ * While fullscreen, the signboard is a centred pane over the scrollback —+ * same window, so still no coordinate translation.+ *+ * Main thread only. One signboard at a time; false when there is no+ * window to host it (caller falls back to its own overlay).+ */+ fun showSignboard(content: View): Boolean {+ val v = root ?: return false+ val p = params ?: return false+ if (signboard != null) return false+ val dm = context.resources.displayMetrics+ if (!compact) {+ val w = minOf(dm.widthPixels - dp(48), dp(560))+ val lp = FrameLayout.LayoutParams(+ w, FrameLayout.LayoutParams.WRAP_CONTENT, Gravity.CENTER,+ )+ v.addView(content, lp)+ signboard = content+ return true+ }+ val side = dp(COMPACT_DP)+ // The width the card will get; measured here so the window can grow+ // by exactly the card's height in the same updateViewLayout.+ val w = minOf(dm.widthPixels - dp(32), dp(560))+ content.measure(+ View.MeasureSpec.makeMeasureSpec(w, View.MeasureSpec.EXACTLY),+ View.MeasureSpec.makeMeasureSpec(0, View.MeasureSpec.UNSPECIFIED),+ )+ // The tail tip reaches into the ball area's transparent padding, down+ // to where the renderer's raised paws are — that overlap is what makes+ // the board read as held rather than as hovering a thumb's width+ // above the head.+ val overlap = (side * SIGN_OVERLAP_FRACTION).toInt()+ val h = content.measuredHeight + side - overlap+ // Ball keeps its spot. If it is hanging off the ledge, use the+ // climb-back target: holding a sign and gripping a ledge are two+ // different poses and the ledge loses.+ val ballY = if (perchApplied) homeY else p.y+ slide?.cancel()+ perchApplied = false+ syncWorking()+ p.width = w+ p.height = h+ p.x = (p.x + side / 2) - w / 2+ p.y = ballY + side - h+ // Ball area to the bottom of the taller window, CENTRED horizontally:+ // the window is card-wide now, not square, and a bottom-right ball+ // area would put the head off to one side of the tail. Verified on+ // device — centring is also what keeps the tail (drawn at the card's+ // centre) on the head after the screen-edge clamp moves the window.+ root?.ballAreaView?.let { area ->+ (area.layoutParams as? FrameLayout.LayoutParams)?.let {+ it.height = side+ it.gravity = Gravity.BOTTOM or Gravity.CENTER_HORIZONTAL+ area.layoutParams = it+ }+ }+ v.addView(+ content,+ FrameLayout.LayoutParams(+ FrameLayout.LayoutParams.MATCH_PARENT,+ FrameLayout.LayoutParams.WRAP_CONTENT,+ Gravity.TOP,+ ),+ )+ clampHeld(p)+ runCatching { wm.updateViewLayout(v, p) }+ signboard = content+ return true+ }++ /**+ * Take the signboard down and shrink back to the bare corner ball,+ * keeping the ball's on-screen spot — the inverse of [showSignboard].+ * Re-applies the perch afterwards if the sign had interrupted one.+ *+ * Main thread only. No-op when nothing is up.+ */+ fun hideSignboard() {+ val v = root ?: return+ val p = params ?: return+ val content = signboard ?: return+ signboard = null+ runCatching { v.removeView(content) }+ if (!compact) return+ val side = dp(COMPACT_DP)+ // Shrink the window around wherever the ball is *holding the sign*+ // (bottom of the taller window), so the card goes away without the+ // ball moving a single pixel. Perching is deliberately NOT done+ // here: in the "帮我看看" flow, beginTask lands one main-loop tick+ // later (CardUi.close runs hideSignboard synchronously, then the+ // onDone callback posts it), and setWorking→applyPerch slides to+ // the ledge as a proper 240ms hop. Pre-deciding the perch here used+ // to fight that: perchApplied was already true when applyPerch ran,+ // it returned without sliding, and the ball sat half-sunk where the+ // tall window's bottom had been until something else moved it —+ // the exact "先变成一半再瞬移" glitch. One owner for the perch+ // transition, and it is the one that animates.+ refreshWinYOrigin()+ val holdY = p.y + p.height - side+ val wantPerch = perchedByUser || perchedByPage+ p.x = p.x + p.width / 2 - side / 2+ p.y = holdY+ p.width = side+ p.height = side+ root?.ballAreaView?.let { area ->+ (area.layoutParams as? FrameLayout.LayoutParams)?.let {+ it.height = FrameLayout.LayoutParams.MATCH_PARENT+ it.gravity = Gravity.TOP or Gravity.START+ area.layoutParams = it+ }+ }+ runCatching { wm.updateViewLayout(v, p) }+ if (wantPerch) {+ perchApplied = true+ // Both axes: the X here is the card's centre, not the ball's own,+ // because the window was card-wide a line ago and the ball was being+ // kept under the tail. Putting the perch back has to undo that too,+ // or the ball re-clings in the middle of the bottom edge — the one+ // spot a cling must not sit.+ slideTo(perchX(), perchY())+ } else {+ perchApplied = false+ homeX = p.x+ homeY = p.y+ }+ syncWorking()+ retimeStroll()+ }++ /**+ * Keep a held signboard on screen. Unlike [clampCompact] there is no+ * ledge to stop at — a ball holding a sign does not grip — and the whole+ * card has to stay readable, not just the ball.+ *+ * Main thread only.+ */+ private fun clampHeld(p: WindowManager.LayoutParams) {+ val dm = context.resources.displayMetrics+ val m = dp(8)+ p.x = p.x.coerceIn(m, (dm.widthPixels - p.width - m).coerceAtLeast(m))+ p.y = p.y.coerceIn(+ m - winYOrigin,+ (bottomEdge() - p.height - m - winYOrigin).coerceAtLeast(m - winYOrigin),+ )+ }++ /** A signboard is up — the ball is holding it. */+ fun isHoldingSign(): Boolean = signboard != null++ /**+ * End the card at this screen line while a typing field is up: `topPx` is+ * the field's own top edge, and `0` means no field.+ *+ * Why the card has to move at all: this window is a fullscreen+ * `TYPE_APPLICATION_OVERLAY`, and on this platform that type is drawn+ * **above** `TYPE_INPUT_METHOD` (measured from `dumpsys window`: our+ * overlays sort above the IME and above every app). So a card left in place+ * does not merely cover the field — it covers the keyboard too.+ *+ * The first cut hid the window instead (alpha 0 + untouchable). That works+ * and reads badly: what is behind it is whatever app the user was on, so+ * typing looked like being thrown out of the assistant and then dropped back+ * into it. Two jumps, no continuity. Ending the card at the field's top edge+ * keeps it on screen the whole time — the ball and the scrollback stay where+ * the user left them, and the field slides in underneath, which is what+ * every chat app does with a keyboard.+ *+ * **The number is the field's top, not the space it needs.** It is easy to+ * get backwards and it was: the first version computed+ * `displayMetrics.heightPixels - top`, which both used a height that+ * excludes the navigation bar (2524 here, not 2560) and left a 600 px band+ * of the user's launcher showing between the card and the field.+ *+ * The *shape* is never changed, only the height, and only for as long as the+ * field is up: [typingGeometry] holds what to put back, and a fold that+ * happens in the meantime takes precedence over it (see [enterCompact]).+ *+ * Main thread only, via the usual post.+ */+ fun setTypingTop(topPx: Int) {+ ui.post {+ if (typingTop == topPx) return@post+ typingTop = topPx+ applyTypingTop()+ }+ }++ private fun applyTypingTop() {+ val v = root ?: return+ val p = params ?: return+ if (typingTop > 0) {+ // Folded, the window is a corner square that is nowhere near the+ // field; there is nothing to step aside from.+ if (compact) return+ if (typingGeometry == null) {+ typingGeometry = intArrayOf(p.x, p.y, p.width, p.height)+ }+ p.height = maxOf(dp(MIN_TYPING_CARD_DP), typingTop)+ android.util.Log.i("Body", "card steps aside: field top=$typingTop → height=${p.height}")+ runCatching { wm.updateViewLayout(v, p) }+ } else if (compact) {+ // A fold in between owns the geometry now; just forget the snapshot.+ typingGeometry = null+ } else {+ restoreTypingGeometry()+ }+ // The flags change with it: see [applyWindow].+ applyWindow()+ }++ /** Put back the geometry [setTypingBand] borrowed. */+ private fun restoreTypingGeometry() {+ val g = typingGeometry ?: return+ typingGeometry = null+ val v = root ?: return+ val p = params ?: return+ p.x = g[0]+ p.y = g[1]+ p.width = g[2]+ p.height = g[3]+ runCatching { wm.updateViewLayout(v, p) }+ }++ fun hide(): Boolean {+ val v = root ?: return true+ return try {+ ui.removeCallbacks(strollRunnable)+ slide?.cancel()+ wm.removeView(v)+ root = null+ params = null+ ball = null+ true+ } catch (_: Throwable) {+ false+ }+ }++ // ---- State / content ----++ fun setState(s: State) {+ ui.post {+ val mood = when (s) {+ State.IDLE -> Mood.CALM+ State.RECORDING -> Mood.LISTENING+ State.THINKING -> Mood.THINKING+ State.SPEAKING -> Mood.SPEAKING+ State.ERROR -> Mood.CONCERNED+ }+ ball?.setMood(mood)+ ball?.setListening(s == State.RECORDING)+ // Two transitions are worth a gesture, and only two: the mic+ // opening is the one moment the user is waiting for an acknowledgement+ // before speaking, and an error is the one moment a face alone is+ // easy to miss. Animating every transition makes the ball twitch its+ // way through a conversation.+ //+ // The same two moments get a tone, for the same reason turned up a+ // notch: a face only lands if the user is looking at it, and having+ // just spoken to the room is precisely when they are not. Speaking+ // gets none — it announces itself — and errors get none either, now+ // that they arrive as a card. See [Earcon].+ if (s != lastState) {+ when (s) {+ State.RECORDING -> {+ ball?.trigger(Action.NOD)+ Earcon.listening()+ }+ State.THINKING -> Earcon.thinking()+ State.ERROR -> ball?.trigger(Action.SHAKE)+ else -> {}+ }+ lastState = s+ }+ // Real state driving disables demo cycling.+ ball?.setDemoCycle(false)+ retimeStroll()+ }+ }++ /**+ * The agent ran a tool against the device. Shows as a nod plus the+ * SEARCHING face, which decays on its own — so this is fire-and-forget and+ * there is no matching "stopped" call to forget to make.+ */+ fun pulseTool() {+ ui.post { ball?.pulseTool() }+ }++ /** A task ended. A brief pleased or flinching face, then back to normal. */+ fun signalOutcome(ok: Boolean) {+ ui.post { ball?.signalOutcome(ok) }+ }++ /**+ * The user changed the language in 设置. Re-read everything the window is+ * showing right now.+ *+ * **Why this is needed at all.** The window is a `WindowManager` overlay+ * owned by the accessibility service, so it outlives any Activity and no+ * amount of `setApplicationLocales` will rebuild it — Android has no idea+ * these views exist. Every string in here was resolved once, when the view+ * was built, and the views are not rebuilt on a language change because the+ * ball's own state (where it is, whether it is listening, whether the card+ * is folded) lives in them. So the strings are re-read in place instead.+ *+ * Whatever gets added to this window later needs a line here too, or it+ * will keep speaking the language the device was in when the service+ * started. That is the whole cost of overlays living outside the Activity+ * lifecycle.+ */+ fun onLocaleChanged() {+ ui.post {+ (tipsView as? TextView)?.text = AppLocale.str(context, R.string.window_tips)+ historyView?.onLocaleChanged()+ }+ }++ /**+ * What is happening *right now*, shown as the live last row of the+ * scrollback rather than as a band under the ball.+ *+ * One place for everything the assistant says or hears, in the order it+ * happened: the partial transcript turns into the user's row, the tool+ * names and drafts turn into the assistant's. A separate band meant the+ * user had to watch two surfaces and meant the same sentence appeared on+ * both — which it did.+ */+ fun setSubtitle(text: String, kind: SubtitleKind = SubtitleKind.PARTIAL) {+ ui.post { historyView?.setLive(text, kind) }+ }++ fun clearSubtitle() = setSubtitle("")++ /**+ * Move the ball one look along and say which one it landed on.+ *+ * The renderer reads [BallLooks] on its own thread and latches it at the+ * top of a frame, so all this side has to do is *ask*; there is nothing to+ * post to the GL thread and nothing to wait for.+ *+ * Three things happen together on purpose. The pick is written to prefs+ * immediately rather than on some later 保存 — the ball changing in front+ * of the user is the confirmation, and there is no card here to put a+ * button on. The name is announced, because the looks are close enough+ * relatives that a user who swiped past one should be told what they now+ * have. And the ball pops, so the change reads as the ball *doing*+ * something rather than as a redraw.+ *+ * The announcement goes through the transcript's transient line+ * ([SubtitleKind.FINAL]) rather than being committed as a row: it is a+ * label on a control, not something anybody said, and it clears itself.+ */+ private fun stepBallLook(delta: Int) {+ val next = BallLooks.step(delta)+ VoiceConfig.save(context, mapOf("ball" to next.name))+ ball?.trigger(Action.POP)+ setSubtitle(AppLocale.str(context, next.labelRes), SubtitleKind.FINAL)+ }++ /**+ * Temporarily make the whole overlay non-touchable so a synthesized+ * gesture (dispatchGesture) can reach the app underneath. When off, the+ * overlay is passive: no clicks, no drag. Toggle it back on when done.+ *+ * The overlay is still visible either way — only touch is affected.+ */+ fun setTouchable(touchable: Boolean) {+ ui.post {+ gesturePassthrough = !touchable+ applyWindow()+ }+ }++ /**+ * Drive the whole window to alpha 0 so an [AccessibilityService.takeScreenshot]+ * captures what's underneath instead of us.+ *+ * Load-bearing since the card went fullscreen: the overlay is composited+ * into the display capture like any other window, so without this every+ * screenshot the brain receives is a black rectangle — and, once the card+ * grew a blur, a *blurred* screen even at alpha 0, which is why+ * [applyWindow] drops the blur on the same switch.+ * Window alpha rather than View.INVISIBLE or removeView — it's one field,+ * and it doesn't tear down the ball's GL context on every capture.+ *+ * Touch is unaffected; an invisible window still eats taps, which is fine+ * for the ~200 ms a capture holds it.+ */+ fun setVisibleForCapture(visible: Boolean) {+ ui.post {+ captureHidden = !visible+ applyWindow()+ }+ }++ /**+ * Fold to the corner ball so the user sees — and can touch — the real app+ * underneath. The card covers the whole screen and is not meaningfully+ * see-through, so this is the only way anything below it is ever usable.+ *+ * Shrinks rather than vanishes. The obvious implementation — window alpha 0+ * plus FLAG_NOT_TOUCHABLE — was the first cut, and it strands the user: a+ * fullscreen touchable window swallows every touch, so handing the screen+ * back means giving up touch, and giving up touch means the ball and the+ * stop button are both gone for good with no way to ask for the assistant.+ * Shrinking the window is what breaks the tie: the ball stays visible and+ * tappable in its own small rectangle while every other pixel belongs to+ * the user.+ *+ * Idempotent, and there is nothing time-based in here — this is a resting+ * state, not a timed peek. Ending it is the caller's business, since when+ * to come back isn't ours to decide.+ */+ fun setCompact(compact: Boolean) {+ ui.post {+ if (this.compact == compact) return@post+ this.compact = compact+ if (compact) enterCompact() else exitCompact()+ applyWindow()+ }+ }++ /**+ * Whether the card is folded, read from outside.+ *+ * Only the service asks, and only for one decision: a fold takes the+ * scrollback with it (`historyView` goes `GONE`), so anything that has to+ * be readable *now* cannot be put there. See the answer's fallback card in+ * [net.kuafuai.andee.ScreenBodyService].+ */+ fun isCompact(): Boolean = compact++ /**+ * Difference between a window's `p.y` and where it actually lands on+ * screen, measured rather than assumed.+ *+ * `Gravity.TOP` plus `FLAG_LAYOUT_NO_LIMITS` is supposed to make `p.y` a+ * display coordinate, and on the tablet it is — the offset measures 0. On+ * an Honor phone it measures the status bar height, so everything computed+ * against `bottomEdge()` landed that much too low: the ledge the paws hang+ * on ended up past the bottom of the physical screen, which is why the ball+ * showed but its hands never did.+ *+ * Not worth arguing with. One `getLocationOnScreen` says what the device+ * actually does, and subtracting it makes the same arithmetic correct on+ * both.+ */+ private var winYOrigin = 0++ /**+ * Re-measure [winYOrigin]. Needs the window laid out and *not* mid-slide,+ * or `p.y` and the on-screen location come from different frames.+ *+ * Main thread only.+ */+ private fun refreshWinYOrigin() {+ val v = root ?: return+ val p = params ?: return+ if (v.width == 0 || slide?.isRunning == true) return+ val loc = IntArray(2)+ v.getLocationOnScreen(loc)+ winYOrigin = loc[1] - p.y+ }++ /** Resting compact `p.y`: bottom-right of the *visible* area. Main thread only. */+ private fun compactHomeY(): Int =+ bottomEdge() - dp(COMPACT_DP) - dp(COMPACT_MARGIN_DP) - winYOrigin++ /**+ * [winYOrigin], for the other overlays this process puts on screen.+ *+ * It is a property of the window *type*, not of this window, so anything+ * else adding a `TYPE_APPLICATION_OVERLAY` at `Gravity.TOP` — CardUi — has+ * to subtract the same thing to turn a screen Y into a `p.y`. Measured here+ * only because this is the window that is always up and therefore always+ * measurable. Main thread only.+ */+ fun overlayYOrigin(): Int = winYOrigin++ /**+ * The assistant is working: sink the corner ball half-way below the bottom+ * screen edge, hang it there by two paws, and put a busy face on it. Haul+ * back up when the task ends, which is also when the answer starts being+ * read aloud.+ *+ * The two halves say different things and both are wanted. The face says+ * "I'm working"; the perch says "and you can have your corner back while I+ * do". A thumb-sized ball sitting fully in frame for a minute of app+ * driving is an obstruction over the very app it is driving — but making it+ * vanish would strand the user, since the ball is the only way to reach the+ * assistant once the card is folded. Half a ball is the trade: still+ * visible, still tappable, out of the way.+ *+ * Getting *out* of the way is what makes coming back legible. The climb back+ * at [endTask] is the ball stepping forward to speak, and there is nothing+ * to step forward from if it never stepped back.+ *+ * Only one of the two reasons to hang — the user can drag the ball onto the+ * ledge themselves, and that outlasts the task. See [perchedByTask].+ */+ fun setWorking(on: Boolean) {+ ui.post {+ if (perchedByTask == on) return@post+ perchedByTask = on+ applyPerch()+ syncWorking()+ retimeStroll()+ }+ }++ /**+ * A page the brain composed is on screen — cling to the ledge until it is+ * gone.+ *+ * The card is folded by then, but a folded ball is still a thumb-sized+ * opaque thing sitting on top of a page whose whole reason for existing is+ * to be looked at, and it lands wherever it last was, which is as likely as+ * not the middle of it. Half a ball on the bottom edge keeps the assistant+ * reachable without standing in front of what it just drew.+ *+ * Independent of [setWorking]; see [perchedByPage] for why it has to be.+ */+ fun setPagePerched(on: Boolean) {+ ui.post {+ if (perchedByPage == on) return@post+ perchedByPage = on+ applyPerch()+ syncWorking()+ retimeStroll()+ }+ }++ /**+ * Tell the ball whether it is currently hanging off a screen edge.+ *+ * Conditional on being folded, not just on there being a task: the ball+ * grows paws, stretches and swings from them, and in the middle of a+ * fullscreen card there is no edge for any of that to hang off — it would+ * read as a ball sprouting hands into thin air. Main thread only.+ */+ private fun syncWorking() {+ ball?.setWorking(compact && perched)+ ball?.setAtLedge(compact && perchApplied && !slideRunning())+ }++ /** True while a perch/home slide animation is in flight. Main thread only. */+ private fun slideRunning(): Boolean = slide?.isRunning == true++ /**+ * The drawn sphere's on-screen bounding box, for anyone anchoring+ * something to it — currently CardUi, which holds its signboards above it.+ *+ * The *sphere*, not the window. It used to be the window, which was+ * accidentally right while the only window was the 168dp corner square+ * (CardUi could reconstruct the ball from it) and badly wrong the moment+ * the card went fullscreen: the same arithmetic then described a 288px+ * radius ball floating in the middle of the screen, and every card landed+ * on top of the real one.+ *+ * Measured off the live views rather than computed from `p.x/p.y`, so a+ * card asked mid-slide or mid-drag still lands where the ball actually is.+ * Null if the window is gone. Answers on the main thread via the callback.+ */+ fun ballRect(cb: (IntArray?) -> Unit) {+ ui.post {+ val v = root ?: return@post cb(null)+ val area: View = v.ballAreaView ?: v+ val loc = IntArray(2)+ area.getLocationOnScreen(loc)+ // The same two numbers the renderer uses: the sphere is a fixed+ // fraction of the surface's shorter side, times the view scale the+ // ball is drawn at (1f when folded — see applyCompactAppearance).+ val scale = if (compact) 1f else BALL_SCALE+ val r = (min(area.width, area.height) * SPHERE_RADIUS_FRACTION * scale).toInt()+ val cx = loc[0] + area.width / 2+ // The sphere may be drawn below the area's centre while it is+ // dodging the control cluster — see [ballDrop].+ val cy = loc[1] + area.height / 2 + (ball?.translationY ?: 0f).toInt()+ cb(intArrayOf(cx - r, cy - r, cx + r, cy + r))+ }+ }++ /** A signboard is up — the ball waits, attending, instead of idling. */+ fun setHolding(on: Boolean) {+ ui.post {+ ball?.setHolding(on)+ if (on) {+ // Stand still while asking: a ball that fidgets through its own+ // question doesn't look like it cares about the answer.+ ui.removeCallbacks(strollRunnable)+ } else {+ retimeStroll()+ }+ }+ }++ /**+ * Move to or from the perched pose, if the window is in a geometry that+ * has a corner to cling to at all. Nothing to do while the card is+ * fullscreen; [enterCompact] re-reads [perched], so a task that began with+ * the card open still perches the moment it folds.+ *+ * Both axes on the way down and both on the way back up: going down is a+ * cling, and a cling picks its side ([perchX]); coming back up restores the+ * spot the ball was clinging *from* ([homeX]/[homeY]) rather than the spot+ * it happens to be in, which is the corner. See [homeX] for what goes wrong+ * if only Y is restored.+ *+ * Main thread only.+ */+ private fun applyPerch() {+ if (!compact) return+ if (perched == perchApplied) return+ perchApplied = perched+ refreshWinYOrigin()+ if (perched) {+ slideTo(perchX(), perchY())+ } else {+ slideTo(homeX, homeY)+ }+ }++ /**+ * Window Y that puts the ball on the bottom screen edge, hanging.+ *+ * Absolute, deliberately — an offset from wherever the window happens to be+ * was the obvious version and it was wrong: drag the ball anywhere but its+ * default corner and "sink by N pixels" sinks it to somewhere that isn't the+ * screen edge, leaving the paws gripping thin air.+ *+ * The ledge is the bottom edge and only the bottom edge, because a paw needs+ * something to hook over and the paws are drawn to either side of the ball's+ * own centre line — there is no rotation of that pose that grips a vertical+ * edge, so a ball hung off the side of the screen would have its hands in+ * mid-air. Only the horizontal edge runs the full width and can be gripped+ * anywhere along it. So a cling moves the ball *along* that edge, never+ * across it: this picks the Y, [perchX] picks the X, and "which side of the+ * screen" is answered by [perchX] rather than left to the finger.+ *+ * The sphere is centred in the compact window at [SPHERE_RADIUS_FRACTION]+ * of it; putting [CLING_HIDDEN] of it below the edge puts its centre at+ * `edge + r * (2 * CLING_HIDDEN - 1)`. [EmotionBallRenderer] derives the+ * paws' world Y from the same expression, which is why both fractions are+ * shared rather than local to either. At `CLING_HIDDEN = 0.5` the radius+ * term is zero and the expression is just "centre on the edge" — the+ * fraction only starts to matter if that half ever moves.+ */+ private fun perchY(): Int {+ val side = dp(COMPACT_DP)+ val r = side * SPHERE_RADIUS_FRACTION+ val edge = bottomEdge()+ return (edge + r * (2f * CLING_HIDDEN - 1f) - side / 2f).toInt() - winYOrigin+ }++ /**+ * Window X for a cling: the bottom-**right** corner, with the drawn sphere+ * [COMPACT_MARGIN_DP] of clear air from the right screen edge.+ *+ * The ball used to stay at whatever X the finger released it at, and the+ * reason to stop doing that is a gesture conflict the user hit on a phone:+ * a ball clinging near the middle of the bottom edge sits inside the band+ * the system claims for "swipe up", so the drag that lifts it off the ledge+ * goes to the launcher instead of to the ball. A clinging ball is exactly+ * the one the user has to be able to pull back out — it is the only way to+ * reach the assistant once the card is folded — so it goes where the system+ * is least interested in the gesture: the corner. Right rather than left+ * matches where this app parks the ball everywhere else ([compactHomeY] is+ * the bottom-right of the visible area, and [show] and [enterCompact] both+ * build the window there).+ *+ * [spherePad] rather than a bare window clamp, because most of the compact+ * window is transparent air: clamping the *window* to the screen leaves the+ * *ball* a thumb's width short of the edge. That is deliberate for the+ * resting corner — the same gap on Y is what keeps the magnet from tripping+ * at rest, and a ball parked for a task should not be clinging — but a ball+ * already hanging has no such reason to sit inboard, and every reason not+ * to.+ *+ * Measured, because the paw is what sets the floor here and it is not+ * where the silhouette is: `blob` draws each paw part in front of the ball+ * (z = 0.88 against the body's z = 0), so perspective magnifies it. At full+ * grip the right paw's outer toe lands 47.8 dp from the sphere's centre+ * against a silhouette radius of 40.3 dp — it overhangs by 7.5 dp. On a+ * 411 dp-wide phone, from the right screen edge:+ *+ * resting corner x = W - side - margin claw 52.2 dp in+ * perchX() x = W - side + pad - margin claw 8.2 dp in+ * drag limit x = W - side + pad claw 7.8 dp *out*+ *+ * So the corner is as far right as this can go: a margin under ~10 dp+ * starts cutting the toe that hangs over, and [clampCompact] already lets a+ * dragged ball push past that. [COMPACT_MARGIN_DP] clears it with room and+ * costs nothing, being the inset the ball already uses against the bottom.+ *+ * Main thread only.+ */+ private fun perchX(): Int {+ val side = dp(COMPACT_DP)+ val dm = context.resources.displayMetrics+ return dm.widthPixels - side + spherePad() - dp(COMPACT_MARGIN_DP)+ }++ /**+ * Transparent margin between the compact window's edge and the drawn+ * sphere, px. The sphere is [SPHERE_RADIUS_FRACTION] of the side and centred,+ * so most of the window is empty air — which is why clamping the *window* to+ * the screen would strand the ball a thumb's width short of every edge.+ * Clamp against this instead and the ball can sit flush.+ */+ private fun spherePad(): Int {+ val side = dp(COMPACT_DP)+ return side / 2 - (side * SPHERE_RADIUS_FRACTION).toInt()+ }++ /**+ * Should the ball be gripping the ledge, given where the window is now?+ *+ * The rule is physical rather than a tuned distance: it grabs on once the+ * drawn sphere's *underside* has reached the ledge, i.e. once the ball is+ * visibly touching the bottom edge. [SNAP_GRACE_DP] is the only fudge — it+ * lets a finger that got close enough count, without which the user would+ * have to land the ball on an exact pixel row to make it cling.+ *+ * Sized so the ball's own resting corner is comfortably outside it: at rest+ * the sphere clears the ledge by about 60 dp, so parking in the corner the+ * ordinary way does not trip the magnet.+ *+ * While a signboard is held the window is taller than the ball area, so+ * the sphere's underside is `y + winYOrigin + (window height − side/2)`,+ * not `y + side/2` — without this a held signboard thought the ball was+ * always on the ledge and drag-to-hold fought the magnet.+ *+ * Main thread only.+ */+ private fun wantsPerch(y: Int): Boolean {+ val side = dp(COMPACT_DP)+ val sphereBottom = y + winYOrigin + windowH() - side / 2 ++ (side * SPHERE_RADIUS_FRACTION).toInt()+ return sphereBottom >= bottomEdge() - dp(SNAP_GRACE_DP)+ }++ /**+ * Keep a dragged ball reachable: the sphere may go flush with any edge but+ * not past it.+ *+ * Y stops at [perchY] rather than at the screen edge — dragging *down* past+ * the ledge has nowhere further to go, and stopping there means the finger+ * is already holding the ball exactly where the magnet wants it. Nothing+ * clamped X or Y before this, and a ball dragged off the side of the screen+ * was simply gone: it is the only way to reach the assistant once folded, so+ * losing it is not a recoverable state.+ *+ * Main thread only.+ */+ private fun clampCompact(p: WindowManager.LayoutParams) {+ val side = dp(COMPACT_DP)+ val pad = spherePad()+ val dm = context.resources.displayMetrics+ p.x = p.x.coerceIn(-pad, dm.widthPixels - side + pad)+ // Signboard up: the whole pane stays on screen and there is no ledge+ // to stop at (a holding ball does not grip). Down: the original rule.+ if (signboard != null) {+ clampHeld(p)+ } else {+ p.y = p.y.coerceIn(-pad - winYOrigin, perchY())+ }+ }++ /**+ * End of a drag: commit the pose the drag was already showing, and pull the+ * window the last few pixels onto the ledge if it is clinging.+ *+ * The pull is what makes it read as a magnet rather than as a drop — the+ * finger is never accurate to the pixel, and a ball left one pixel shy of+ * the ledge with its paws out is gripping nothing. Short and fast on+ * purpose: [PERCH_SLIDE_MS] is the assistant announcing a state change,+ * this is the tail end of the user's own gesture.+ *+ * Both axes, though the finger only ever dropped this ball on one: grabbing+ * the ledge also claims the corner beside it, see [perchX]. The horizontal+ * half of the pull is the longer one whenever the user hung the ball up+ * mid-screen, and it is the half that matters — that is the position where+ * the next swipe up goes to the launcher instead of to the ball.+ *+ * Main thread only.+ */+ private fun settleAfterDrag() {+ val v = root ?: return+ val p = params ?: return+ if (!compact) return+ clampCompact(p)+ if (perchApplied) {+ perchedByUser = true+ slideTo(perchX(), perchY(), SNAP_MS)+ } else {+ homeX = p.x+ homeY = p.y+ }+ runCatching { wm.updateViewLayout(v, p) }+ }++ /**+ * Y of the ledge the ball hangs off: the bottom of the area a user can+ * actually see, which is not the bottom of the display.+ *+ * On a tablet with a hairline gesture pill those are near enough the same+ * number that `displayMetrics.heightPixels` looked correct. On a phone they+ * are not. The navigation bar owns the bottom band and is layered *above*+ * TYPE_APPLICATION_OVERLAY, while `FLAG_LAYOUT_NO_LIMITS` lets us lay out+ * underneath it — so hanging half a ball below `heightPixels` put the+ * visible half behind the nav bar and the ball disappeared. Half a ball is+ * the entire margin of error here, and the inset is deeper than it.+ */+ private fun bottomEdge(): Int {+ val m = wm.currentWindowMetrics+ val inset = m.windowInsets+ .getInsetsIgnoringVisibility(+ WindowInsets.Type.systemBars() or WindowInsets.Type.displayCutout(),+ ).bottom+ return m.bounds.bottom - inset+ }++ /**+ * Animated rather than teleported: the perch is a statement about what the+ * assistant is doing, and a ball that jumps has not said anything the user+ * can follow.+ *+ * Both axes travel on one interpolator, because a cling decides both — it+ * hangs off the bottom edge at [perchY] *and* takes the right corner at+ * [perchX] — and separating them would read as the ball changing its mind+ * halfway down. The un-perch passes the current X and so is the vertical+ * slide it always was; see [applyPerch] for why that one has no business+ * moving sideways.+ *+ * Main thread only.+ */+ private fun slideTo(targetX: Int, targetY: Int, ms: Long = PERCH_SLIDE_MS) {+ val v = root ?: return+ val p = params ?: return+ slide?.cancel()+ if (p.x == targetX && p.y == targetY) return+ val fromX = p.x+ val fromY = p.y+ val dx = targetX - fromX+ val dy = targetY - fromY+ slide = ValueAnimator.ofFloat(0f, 1f).apply {+ duration = ms+ interpolator = DecelerateInterpolator()+ addUpdateListener { a ->+ val t = a.animatedValue as Float+ p.x = fromX + (dx * t).roundToInt()+ p.y = fromY + (dy * t).roundToInt()+ runCatching { wm.updateViewLayout(v, p) }+ }+ addListener(object : android.animation.AnimatorListenerAdapter() {+ override fun onAnimationEnd(animation: android.animation.Animator) {+ // The geometry is now truly where perchApplied said it was.+ // This is the moment the cling (paws, half-sunk body) may+ // start — before it, the ball was travelling and a cling+ // drawn mid-flight read as chopped-in-half.+ syncWorking()+ }+ })+ start()+ }+ }++ // ---- Idle fidget ----++ /**+ * Hop on the spot, every so often, when there is nothing else going on.+ *+ * A corner ball that holds one pixel for an hour reads as a decal. A few+ * hops every half-minute is the cheapest way to say there is something alive+ * in there — and because the ball *is* the only way to reach the assistant+ * once the card is folded, looking alive is also the thing that makes it+ * look tappable.+ *+ * On the spot, not travelling: the window stays exactly where the user put+ * it. A ball that wanders has to be re-found every time it is wanted, which+ * costs more than the liveliness is worth.+ *+ * Main thread only.+ */+ private fun strollStep() {+ if (!strollWanted()) {+ retimeStroll()+ return+ }+ ball?.trigger(Action.HOP)++ strollLeft--+ // Two to four hops, then stand still for a while. A ball that hopped on+ // a fixed tick would be a metronome; the point is to look idle, not+ // scheduled.+ val delay = if (strollLeft > 0) {+ STROLL_HOP_MS+ } else {+ strollLeft = 2 + Random.nextInt(3)+ STROLL_REST_MS + Random.nextInt(STROLL_REST_JITTER_MS).toLong()+ }+ ui.postDelayed(strollRunnable, delay)+ }++ /**+ * Only when folded, idle, and awake.+ *+ * Not while working — it is hanging off the edge by its paws, and a ball+ * that bounces while gripping is doing two incompatible things. Not while+ * dozing either: the point of SLEEPING is that the device reads as off, and+ * a ball twitching in its sleep undoes that on its own.+ *+ * Main thread only.+ */+ private fun strollWanted(): Boolean =+ root != null && compact && !perched && signboard == null &&+ lastState.let { it == null || it == State.IDLE } &&+ ball?.isDozing() != true++ /** Push the next hop out to a full rest. Main thread only. */+ private fun retimeStroll() {+ ui.removeCallbacks(strollRunnable)+ strollLeft = 2 + Random.nextInt(3)+ ui.postDelayed(+ strollRunnable,+ STROLL_REST_MS + Random.nextInt(STROLL_REST_JITTER_MS).toLong(),+ )+ }++ /**+ * Bare ball, bottom-right, on nothing at all. The toolbar is ~10 buttons+ * wide and the subtitle band is 200dp tall — neither survives the shrink,+ * so both go rather than getting squeezed into something illegible.+ *+ * The window background goes with them, and the ball carries its own dark+ * fill instead (`EmotionBallRenderer.drawBody`). A background can only be a+ * rectangle or an oval filling the window, which is bigger than the sphere:+ * the two then read as one ball inside another, and the outer one wins —+ * larger, harder-edged, and unable to squash or hop with the ball it is+ * supposed to be part of.+ *+ * Ball at view scale 1: [BALL_SCALE] is the full card's magnification, and+ * a corner square has nothing to magnify into.+ *+ * Main thread only.+ */+ private fun applyCompactAppearance() {+ root?.background = null+ backdrop?.visibility = View.GONE+ topBarView?.visibility = View.GONE+ // Not just invisible — GONE, so the ball area has the whole window to+ // itself. A scrollback left in the layout would cover most of a 168dp+ // square and there would be nothing left to see the ball in.+ historyView?.visibility = View.GONE+ tipsView?.visibility = View.GONE+ // The whole square is the ball's, so the sphere is centred in it and+ // sized against it.+ setBallAreaHeight(FrameLayout.LayoutParams.MATCH_PARENT)+ ball?.scaleX = 1f+ ball?.scaleY = 1f+ // The corner square has no cluster to dodge, so the ball sits dead+ // centre again — and the fold animation lands from wherever it was.+ ball?.translationY = 0f+ }++ /**+ * The card's two stacked areas: the ball rests in the top+ * [BALL_AREA_FRACTION] of it, the scrollback fills the rest.+ *+ * The ball area is also the hit-test bound: [BallHostFrame.onTouchEvent]+ * drops anything below `ballAreaView.height` so the list keeps its own+ * touches, and the list is a ScrollView that needs every one of them.+ *+ * Main thread only.+ */+ /**+ * Re-derive the card's two stacked areas, and where the ball sits in the top+ * one, from the card's current height.+ *+ * Called from the ball host's size change and from the cluster's own layout,+ * because the cluster's height is one of the inputs and two things move it:+ * the status-bar inset lands after the first layout, and the tool drawer+ * changes it at runtime. Listening to the cluster rather than hooking each+ * of those call sites keeps it to one place that can be forgotten.+ *+ * Main thread only.+ */+ private fun refitCardAreas() {+ val r = root ?: return+ if (compact) return+ val h = r.height+ if (h <= 0) return+ // While a field is up this window is shorter than the card it is+ // showing, and the band the ball lives in is measured against the+ // **card**, not the window: the sphere keeps the size and the place the+ // user left it in, and the top of the card stays the same picture it was+ // before the keyboard came up. Only the scrollback takes the difference+ // — and it has to, being the part that scrolls. Measured: with the band+ // taken from the window instead, the ball dropped from a 344 px radius+ // to 213 and every pixel of the card's upper half moved.+ val bandH = if (typingTop > 0) wm.currentWindowMetrics.bounds.height() else h+ val ballH = (bandH * BALL_AREA_FRACTION).toInt()+ setBallAreaHeight(ballH)+ // The tips bar is measured, not reserved by a guessed constant: it is one+ // line of text, so its height follows the display's font scale, and a+ // fixed dp would either clip the line on a large-text device or leave a+ // gap on a small one. Zero while it is hidden, which is what keeps the+ // folded corner square from carrying 40 dp of nothing.+ val tipsH = tipsHeight()+ setHistoryHeight(maxOf(0, h - ballH - tipsH))+ setHistoryBottomMargin(tipsH)+ ball?.translationY = (ballDrop(bandH) + dp(BALL_NUDGE_DP)).toFloat()+ }++ /**+ * The tips bar's real height, measured on demand.+ *+ * On demand rather than stored because it is a function of the font scale+ * and the window width, and both can change between two calls. The measure+ * pass is only ever needed before the first layout of a fresh unfold — after+ * that the view answers for itself.+ *+ * Main thread only.+ */+ private fun tipsHeight(): Int {+ val v = tipsView ?: return 0+ if (v.visibility != View.VISIBLE) return 0+ if (v.height > 0) return v.height+ val w = root?.width ?: return 0+ if (w <= 0) return 0+ v.measure(+ View.MeasureSpec.makeMeasureSpec(w, View.MeasureSpec.EXACTLY),+ View.MeasureSpec.makeMeasureSpec(0, View.MeasureSpec.UNSPECIFIED),+ )+ return v.measuredHeight+ }++ /**+ * How far the ball has to drop to clear the control cluster, in px. Zero on+ * any device where the sphere cannot reach the cluster in the first place —+ * which is every device wide enough that the cluster's left edge sits past+ * the sphere's.+ *+ * The ball is centred in its band, so its top edge sits+ * `(0.5 − SPHERE_RADIUS_FRACTION × BALL_SCALE)` of the band's height below+ * the band's top. The cluster is a fixed 44 dp pushed down by the status bar+ * / cutout inset (see [insetTopBar]), so whether the two meet is a property+ * of the device. Measured against the real numbers: on a 711×1138 dp tablet+ * they never meet; on a 393 dp phone they cleared each other by 0.7 dp; on a+ * 411 dp phone with a taller status bar they overlapped by 4 dp.+ *+ * The first fix here grew the band until the sphere's top edge was pushed+ * below the cluster. That works, and it hides its cost: the band's bottom+ * edge *is* the scrollback's top edge, so buying clearance that way shoves+ * the chat down and shortens it — a corner collision paid for with the whole+ * card's split. It was 60 dp of chat on a phone, and that is exactly what the+ * user came back with ("the chat area doesn't need to be that far down").+ *+ * Dropping the ball instead touches one thing: it slides the sphere down+ * inside its own band, into the empty space that already sits between the+ * ball and the list. Same band, same split, same sphere size, still centred —+ * only its y moves, by roughly the 12-17 dp it takes.+ *+ * Main thread only.+ */+ private fun ballDrop(h: Int): Int {+ val bar = topBarView ?: return 0+ val w = root?.width ?: return 0+ if (bar.height <= 0 || w <= 0) return 0 // not measured yet; the layout hook comes back+ val bandH = h * BALL_AREA_FRACTION+ val r = SPHERE_RADIUS_FRACTION * bandH * BALL_SCALE+ // Horizontal reach, measured from the sphere's centre to the cluster's+ // left edge. Past the radius, the cluster is not in the sphere's column+ // at all and no drop is needed.+ val dx = bar.left - w / 2f+ if (abs(dx) >= r) return 0+ val topEdge = bandH / 2f - sqrt(r * r - dx * dx)+ return maxOf(0f, bar.bottom + dp(BALL_TOP_GAP_DP) - topEdge).toInt()+ }++ private fun setBallAreaHeight(px: Int) {+ val area = root?.ballAreaView ?: return+ val lp = area.layoutParams ?: return+ lp.height = px+ area.layoutParams = lp+ }++ private fun setHistoryHeight(px: Int) {+ val v = historyView ?: return+ val lp = v.layoutParams ?: return+ lp.height = px+ v.layoutParams = lp+ }++ /**+ * Lift the scrollback clear of the tips bar.+ *+ * The list is bottom-anchored, so its height alone would still put its last+ * rows underneath the bar — the two are stacked at the same edge.+ */+ private fun setHistoryBottomMargin(px: Int) {+ val v = historyView ?: return+ val lp = v.layoutParams as? FrameLayout.LayoutParams ?: return+ lp.bottomMargin = px+ v.layoutParams = lp+ }++ private fun enterCompact() {+ val p = params ?: return+ slide?.cancel()+ // A signboard cannot survive the fold — the window shrinks to a corner+ // square smaller than the pane. Evict through [signboardEvictor] so+ // CardUi's own callbacks fire (timeout semantics, holding state).+ if (signboard != null) {+ signboard = null+ signboardEvictor?.invoke()+ }+ // A typing band shrinks the window, and this line is what a later unfold+ // would grow back to — so the real geometry has to be back in place+ // *before* it is snapshotted, or the card unfolds into a card with a+ // keyboard-sized hole in the bottom of it.+ restoreTypingGeometry()+ fullGeometry = intArrayOf(p.x, p.y, p.width, p.height)+ val side = dp(COMPACT_DP)+ val dm = context.resources.displayMetrics+ // Measured while the window is still in its old geometry, which is the+ // only moment `p.y` and the on-screen location agree.+ refreshWinYOrigin()+ p.width = side+ p.height = side+ // X is normally the resting corner, a thumb's width in (see+ // [compactHomeY] for why the same gap is deliberate on Y). A fold that+ // lands already clinging skips the resting corner entirely, though —+ // nothing will slide it afterwards — so it has to be born at [perchX],+ // or the same cling would sit at a different X depending on whether the+ // task started before or after the card folded.+ p.x = if (perched) perchX() else dm.widthPixels - side - dp(COMPACT_MARGIN_DP)+ p.y = compactHomeY()+ // The fold usually happens because a task started, so the perch is+ // normally already wanted by the time we get here. Applied outright+ // rather than slid: it is one motion with the fold, not a second one.+ homeX = p.x+ homeY = p.y+ perchApplied = perched+ if (perched) p.y = perchY()+ applyCompactAppearance()+ syncWorking()+ retimeStroll()+ }++ /** Main thread only. */+ private fun exitCompact() {+ val p = params ?: return+ slide?.cancel()+ // Same eviction as the fold, in the other direction: the card grows+ // and the signboard's compact geometry (ball at the window's bottom)+ // is meaningless inside it.+ if (signboard != null) {+ signboard = null+ signboardEvictor?.invoke()+ }+ perchApplied = false+ val g = fullGeometry ?: intArrayOf(+ 0, 0,+ WindowManager.LayoutParams.MATCH_PARENT,+ WindowManager.LayoutParams.MATCH_PARENT,+ )+ // Belt-and-braces: a stale stored geometry can be a compact-sized+ // rectangle (service killed while folded, then restored). Growing the+ // window to a corner square is worse than useless, so never restore+ // anything smaller than about a third of the screen.+ val dm = context.resources.displayMetrics+ if (g[2] >= dm.widthPixels / 3 && g[3] >= dm.heightPixels / 3) {+ p.x = g[0]; p.y = g[1]; p.width = g[2]; p.height = g[3]+ } else {+ p.x = 0; p.y = 0+ p.width = WindowManager.LayoutParams.MATCH_PARENT+ p.height = WindowManager.LayoutParams.MATCH_PARENT+ }+ fullGeometry = null+ // The fill is [applyWindow]'s to set — it is the only place that knows+ // whether the compositor agreed to blur — and [setCompact] calls it+ // immediately after this, before the next frame.+ topBarView?.visibility = View.VISIBLE+ toolsView?.visibility = View.GONE // tools always start folded away+ historyView?.visibility = View.VISIBLE+ tipsView?.visibility = View.VISIBLE+ backdrop?.visibility = View.VISIBLE+ // The two areas' heights are deferred to [BallHostFrame.onSizeChanged]+ // rather than computed here: the window is still the corner square at+ // this point, and the real height only exists after the layout pass+ // that this geometry change is about to trigger.+ //+ // Anything that arrived while folded — which is most of it, since the+ // card is folded for the whole time the assistant is working — has+ // only ever been appended to a list that wasn't being laid out.+ historyView?.refresh()+ ball?.scaleX = BALL_SCALE+ ball?.scaleY = BALL_SCALE+ syncWorking()+ retimeStroll()+ }++ /**+ * Alpha, touchability and the blur each have more than one reason to be+ * off, and the callers overlap in time (a capture while folded). Kept as+ * separate booleans folded here rather than written directly, or whichever+ * caller finishes last wins and clobbers the other's intent.+ *+ * Main thread only.+ */+ private fun applyWindow() {+ val v = root ?: return+ val p = params ?: return+ p.alpha = if (captureHidden) 0f else 1f+ // FLAG_LAYOUT_IN_SCREEN is what makes the unfolded card actually+ // fullscreen. Without it the window's frame is the *content* area —+ // the display minus the status and navigation bars — so a MATCH_PARENT+ // card stopped 126 px short of the top on this phone and left the+ // status bar sitting on a band of the user's wallpaper. Measured, not+ // assumed: with the flag the window's top is the display's top and+ // [winYOrigin] measures 0 by itself.+ val base = WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE or+ WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS or+ WindowManager.LayoutParams.FLAG_LAYOUT_IN_SCREEN+ // Untouchable while a typing field is up, even though the window no+ // longer overlaps it: the field's top edge is *derived* from the+ // keyboard's height, which changes as the IME animates and as 搜狗's+ // toolbar row comes and goes. Being a few pixels late would put this+ // window over the top row of the field and eat the taps aimed at it —+ // and the card has nothing to offer during a typing session anyway.+ p.flags = if (gesturePassthrough || typingTop > 0) {+ base or WindowManager.LayoutParams.FLAG_NOT_TOUCHABLE+ } else {+ base+ }+ // Blur behind only while the card is actually a card. Folded, the+ // window is a 168 dp square around the ball and blurring behind *that*+ // is a smudge following the ball around; hidden for a capture, the+ // window is at alpha 0 but the compositor would still be asked to+ // defocus what is behind it — and what is behind it is exactly the+ // screenshot the brain is about to read. The 80 ms settle in+ // `blankedForCapture` covers this params change along with the alpha.+ // ...and not while the backdrop is up. The card used to be a window onto+ // the user's home screen, which is what the blur was for; the backdrop+ // now covers the whole window, so the compositor would be defocusing a+ // full screen of pixels nobody can see. Turning it off here is also what+ // makes the card look the same on a device where cross-window blur is+ // unavailable, which is most of them.+ val wantFrost = !compact && !captureHidden && !backdropUp()+ frosted = Glass.frost(context, wm, p, want = wantFrost)+ paintCard()+ // The card now runs under the status bar ([FLAG_LAYOUT_IN_SCREEN]), so+ // the control cluster has to step around it — otherwise the clock and+ // the battery icon sit on top of `⋯` and `✕`, and those are the two+ // controls the user cannot do without.+ insetTopBar()+ runCatching { wm.updateViewLayout(v, p) }+ }++ /**+ * Push the toolbar below the status bar / cutout. No-op on every call but+ * the first two, since writing `layoutParams` asks for a layout pass and+ * this runs on every state change.+ *+ * **The inset has to come from the view, not from the display.**+ * `WindowManager.currentWindowMetrics` describes the *display*, so its top+ * inset is the status bar's height whether or not this window is underneath+ * it — and on this ROM it is not: measured, the window frame already starts+ * at the status bar's bottom edge, so adding the height again put the pill+ * a second status bar below where it belonged (126 px + 126 px + dp(10),+ * against the dp(10) the code reads as asking for). `rootWindowInsets` is+ * the same question asked relative to *this* window, so it answers 0 where+ * the system has already moved us clear and the full height where it has+ * not — which is what makes the same line correct on a ROM that honours+ * [FLAG_LAYOUT_IN_SCREEN] for overlays and one that doesn't.+ *+ * Main thread only.+ */+ private fun insetTopBar() {+ val bar = topBarView ?: return+ val lp = bar.layoutParams as? FrameLayout.LayoutParams ?: return+ val insets = bar.rootWindowInsets ?: return+ val want = dp(10) + insets.getInsetsIgnoringVisibility(+ WindowInsets.Type.systemBars() or WindowInsets.Type.displayCutout(),+ ).top+ if (lp.topMargin == want) return+ lp.topMargin = want+ bar.layoutParams = lp+ }++ /**+ * The card's fill, which is a question only [applyWindow] can answer:+ * translucent when something is blurring behind it, nearly opaque when+ * nothing is. Folded there is no card at all — the window is the ball's+ * own square and anything painted in it is a box drawn around her.+ *+ * Main thread only.+ */+ private fun paintCard() {+ val v = root ?: return+ val bg = backdrop+ if (compact) {+ v.background = null+ clearScrim(bg)+ return+ }+ // Three answers, in order: with a backdrop the fill is a scrim over it+ // and the question [frosted] was answering ("is anything behind this+ // card?") no longer applies — the answer is our own image, so the same+ // film goes on regardless of what the compositor said. Without one the+ // card falls back to exactly what it did before.+ if (backdropUp()) {+ // **The scrim goes on the image's foreground, not the root's+ // background.** They look interchangeable and are not: the backdrop+ // is a *child* of the root, so a root background is behind the very+ // thing it is supposed to darken. The first version did that, and+ // the card came out as the raw photo at full brightness — measured+ // on the tablet at mean luma 0.40 where the design says 0.13, with+ // the pale ball barely legible on top of it.+ v.background = null+ val s = Backdrop.scrim()+ if (scrimShown != s) {+ bg!!.foreground = ColorDrawable(s)+ scrimShown = s+ }+ return+ }+ clearScrim(bg)+ v.setBackgroundColor(+ Color.parseColor(if (frosted) Glass.CARD_FROSTED else Glass.CARD_SOLID),+ )+ }++ /** Main thread only. */+ private fun clearScrim(bg: ImageView?) {+ if (scrimShown == null && bg?.foreground == null) return+ bg?.foreground = null+ scrimShown = null+ }++ /**+ * Whether the backdrop is both ready and showing. Not just "not folded":+ * the blur is done on a worker at build time, and a card that unfolds during+ * the first few hundred milliseconds of the process's life has to render as+ * it did before rather than as a card with a transparent hole in it.+ *+ * Main thread only.+ */+ private fun backdropUp(): Boolean =+ !compact && Backdrop.ready() && backdrop?.visibility == View.VISIBLE++ // ---- Build ----++ @SuppressLint("ClickableViewAccessibility")+ private fun build(): BallHostFrame {+ val root = BallHostFrame(context)++ // ---- Backdrop (the bottom of the stack) ----+ //+ // First child, so everything else draws on top of it and nothing has to+ // know it is there. CENTER_CROP and not FIT_CENTER: the brief was 铺满,+ // and letterboxing a photo behind a fullscreen card would leave two+ // bands of whatever the card's fill is at the top and bottom.+ //+ // Sized against the **display**, not the window, and pinned to the top.+ // The card is not always fullscreen — a typing session ends it at the+ // field's top edge — and a MATCH_PARENT backdrop re-scales and re-crops+ // with it, so the wallpaper visibly jumped and brightened as the+ // keyboard came up: measured on one screen point, `10,34,60` fullscreen+ // → `65,85,106` while typing, and it is not the app behind (swapping the+ // app behind moves these pixels by 0.9/255). Fixed to the display the+ // backdrop is merely *clipped* by the shorter window, so the top of the+ // card is pixel-for-pixel the same in both states.+ //+ // The decode and the blur are kicked off here rather than lazily on the+ // first unfold. The window is built folded, so the work happens while+ // this view is still GONE and the answer is normally in hand long before+ // the user can long-press the ball.+ val bg = ImageView(context).apply {+ scaleType = ImageView.ScaleType.CENTER_CROP+ visibility = View.GONE+ }+ root.addView(+ bg,+ 0,+ FrameLayout.LayoutParams(+ FrameLayout.LayoutParams.MATCH_PARENT,+ wm.currentWindowMetrics.bounds.height(),+ Gravity.TOP,+ ),+ )+ backdrop = bg+ Backdrop.warm(context) { bmp ->+ bg.setImageBitmap(bmp)+ // The fill and the blur flag are both this call's to decide, and the+ // answer just changed underneath them.+ applyWindow()+ }++ // ---- Ball area (top of the card) ----+ //+ // Stacked in a FrameLayout rather than a column with weights: the+ // areas' heights are driven from [BallHostFrame.onSizeChanged] against+ // the card's real height, which is the only place it is known, and the+ // ball has to keep its whole area when the card folds to the corner.+ val ballArea = FrameLayout(context)+ val b = EmotionBallTextureView(context).apply {+ // No demo cycling from birth. It used to read "cycle moods until+ // real state driving takes over" — but real driving only begins+ // with the first setState, and a device idling at a desk never+ // sends one. The ball then cycled all fourteen moods every 5.5 s+ // forever: wearing SPEAKING (mouth sine and all) with no sound,+ // and resetting the CALM→SLEEP accumulator so it could never+ // doze either. CALM says "I am here, not busy" all by itself.+ // Scaled rather than sized down: the sphere's diameter is a fixed+ // fraction of the surface's shorter side, so a fixed dp size would+ // need a matching floor on the window's own minimum or the ball+ // gets cropped when the card is resized small. A view scale tracks+ // whatever geometry the card ends up with. Pivot defaults to the+ // view's center, which is where the sphere already is.+ scaleX = BALL_SCALE+ scaleY = BALL_SCALE+ }+ ball = b+ ballArea.addView(+ b,+ FrameLayout.LayoutParams(+ FrameLayout.LayoutParams.MATCH_PARENT,+ FrameLayout.LayoutParams.MATCH_PARENT,+ ),+ )++ root.addView(+ ballArea,+ FrameLayout.LayoutParams(+ FrameLayout.LayoutParams.MATCH_PARENT,+ FrameLayout.LayoutParams.MATCH_PARENT,+ ),+ )+ root.ballAreaView = ballArea++ // ---- Scrollback, straight under the ball ----+ val history = HistoryListView(context).apply {+ onPageClick = { listeners.onHistoryPageClick(it) }+ onTextOpen = { listeners.onTextOpen(it) }+ }+ historyView = history+ root.addView(+ history,+ FrameLayout.LayoutParams(+ FrameLayout.LayoutParams.MATCH_PARENT,+ FrameLayout.LayoutParams.MATCH_PARENT,+ Gravity.BOTTOM,+ ),+ )+ ChatHistory.setListener { ui.post { historyView?.refresh() } }+ history.refresh()++ // ---- Bottom tips ----+ //+ // The ball's whole vocabulary, where the user's eye already is at the+ // end of a conversation. Pinned rather than appended to the scrollback:+ // it was the list's last row before, which meant it scrolled away —+ // and the one time it is needed is the one time the user has scrolled+ // up to read what happened.+ //+ // Only in the full card. Folded, the window is a 168 dp square and the+ // tips have nowhere to be; the gestures they describe work there anyway.+ val tips = TextView(context).apply {+ text = AppLocale.str(context, R.string.window_tips)+ textSize = 12f+ setTextColor(Color.parseColor(Glass.SECONDARY))+ gravity = Gravity.CENTER+ // The bottom padding clears the gesture bar, the same 30 dp the+ // scrollback's last row used to reserve for itself — this bar+ // inherits that edge of the card.+ setPadding(dp(14), dp(6), dp(14), dp(30))+ visibility = View.GONE+ }+ tipsView = tips+ root.addView(+ tips,+ FrameLayout.LayoutParams(+ FrameLayout.LayoutParams.MATCH_PARENT,+ FrameLayout.LayoutParams.WRAP_CONTENT,+ Gravity.BOTTOM,+ ),+ )++ // ---- Top-right controls ----+ //+ // Added last so it is on top of both.+ val topBar = buildTopBar()+ root.addView(+ topBar,+ FrameLayout.LayoutParams(+ FrameLayout.LayoutParams.WRAP_CONTENT,+ FrameLayout.LayoutParams.WRAP_CONTENT,+ Gravity.TOP or Gravity.END,+ ).apply {+ topMargin = dp(10); rightMargin = dp(10)+ },+ )+ topBarView = topBar+ // The cluster's own bounds are an input to the card's split — see+ // [ballAreaHeight]. Watching it here means the status-bar inset and the+ // tool drawer both re-derive the split without either having to know+ // that it is responsible for it.+ topBar.addOnLayoutChangeListener { _, _, top, _, bottom, _, oldTop, _, oldBottom ->+ // Also the only place the inset is guaranteed to be *available*:+ // `rootWindowInsets` is null until the view is attached, so the+ // first `applyWindow` can run before there is an answer. A laid-out+ // view is an attached one. Self-guarded, so the extra layout pass+ // this asks for on the first call no-ops on the second.+ insetTopBar()+ if (top != oldTop || bottom != oldBottom) refitCardAreas()+ }++ return root+ }++ /**+ * The control cluster: one pill holding the three things the user actually+ * reaches for, with the eight screen tools folded behind `⋯`.+ *+ * Three and not four since typing moved onto the ball's double-tap — see+ * [Listeners.onTextInputToggle] for why it belongs there rather than here.+ * And the same three since the stop button left: `✓` took its place rather+ * than being added beside it, because stop was the one key in this bar the+ * ball can already perform — see [Listeners.onStopClick]. So the count has+ * not changed twice over; only one of the three is different, and it is the+ * one that was not earning its slot.+ *+ * The bar now reads `⋯ ✓ ⚙ ✕` — one working row (the self-check, next to+ * the gear it is a sibling of) between the two that bracket it: the tools+ * drawer on the left and the dismiss on the right.+ *+ * The tools are developer instruments — swipe four ways, back, home, dump+ * the tree, grab a screenshot. Ten identical monochrome glyphs crammed at+ * dp(2) apart was most of why the card read as unfinished, and none of them+ * is something the user needs within one tap.+ */+ @SuppressLint("SetTextI18n")+ private fun buildTopBar(): View {+ val bar = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ gravity = Gravity.END+ setPadding(dp(4), dp(4), dp(4), dp(4))+ // A recessed well rather than a panel: this pill floats over a card+ // that is itself translucent, so its base has to be darker than+ // what it sits on or the glyphs read against the user's app.+ background = Glass.well(context, dp(24))+ }++ val keys = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ }++ // A second row rather than more of the first: twelve dp(36) keys in a+ // line measure ~1500px against a 1200px screen, and the overflow fell+ // off the *right* — expanding the tools pushed `⚙` and `⤡` off-window,+ // making the tool drawer a one-way door out of the card. Stacked, the+ // eight tools are ~990px and the permanent keys never move at all.+ val tools = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ visibility = View.GONE+ }+ for (tool in Tool.values()) tools.addView(buildToolButton(tool))+ toolsView = tools++ keys.addView(+ barButton("⋯", Glass.LABEL, 20f) {+ tools.visibility = if (tools.visibility == View.VISIBLE) View.GONE else View.VISIBLE+ }+ )+ // ✓ and not ■: the stop button used to sit here, and stop is the one+ // action in this bar the ball can already perform — see onStopClick.+ // A check mark for 自检, and it is a glyph rather than an icon set for+ // the same reason as the rest of the bar: one Paint, one font, and it+ // inherits the ink-centred drawing in GlyphButton.+ keys.addView(barButton("✓", Glass.LABEL, 17f, label = "自检") { listeners.onSelfCheckClick() })+ // No ⌨ here. Typing moved onto the ball's double-tap, and the bar is+ // the wrong place for it twice over: it only exists while the card is+ // unfolded (so the way to type vanished with the fold — the same fault+ // that made "点右上角的 ⌨" a lie in a failure message), and it put a+ // second meaning on the one control the user already had to learn.+ keys.addView(barButton("⚙", Glass.LABEL, 18f) { listeners.onSettingsClick() })+ // Reads as close, not as resize: the card is a thing you dismiss, and the+ // corner ball it folds into stays on screen either way. It used to sit+ // alone in the bottom-right of the subtitle band, diagonally opposite+ // every other control in the card.+ keys.addView(barButton("✕", Glass.LABEL, 17f) { listeners.onMinimizeClick() })++ bar.addView(keys)+ bar.addView(tools)+ return bar+ }++ /**+ * A glyph centred on the ink it actually draws, rather than on the line box+ * its font reserves.+ *+ * `TextView` with `gravity = CENTER` centres the *line* — ascent to descent,+ * the same box for every character in the font. These glyphs come from all+ * over Unicode (`⋯` hangs at the baseline, `■` is a full-height block, `⚙`+ * and the arrows sit somewhere else again), so a row of them centred that way+ * is a row of characters at four different heights, which is what the bar+ * looked like. Measuring each glyph's own bounds and centring on that is the+ * only way they line up, and it costs one [Paint.getTextBounds] at build time.+ */+ private class GlyphButton(+ ctx: Context,+ private val glyph: String,+ color: String,+ sizeSp: Float,+ ) : View(ctx) {+ private val paint = Paint(Paint.ANTI_ALIAS_FLAG).apply {+ this.color = Color.parseColor(color)+ textSize = TypedValue.applyDimension(+ TypedValue.COMPLEX_UNIT_SP, sizeSp, ctx.resources.displayMetrics,+ )+ }+ private val ink = Rect()++ init {+ paint.getTextBounds(glyph, 0, glyph.length, ink)+ // A bare View draws text without ever telling anyone it is there, so+ // the whole control bar dropped out of `get_screen_element` the moment these+ // stopped being TextViews — invisible to the brain and to TalkBack+ // alike. The glyph is the label it had as a TextView; keep it one.+ //+ // Set by the caller when the glyph is not a word in any language:+ // `⋯` and `■` are punctuation, and TalkBack reading "✓" out as+ // "check mark" beside a button that opens the self-check page is a+ // worse label than the two characters the page is called by.+ contentDescription = glyph+ }++ override fun onDraw(canvas: Canvas) {+ canvas.drawText(+ glyph,+ width / 2f - ink.exactCenterX(),+ height / 2f - ink.exactCenterY(),+ paint,+ )+ }+ }++ private fun barButton(glyph: String, color: String, size: Float, onClick: () -> Unit): View =+ GlyphButton(context, glyph, color, size).apply {+ isClickable = true+ isFocusable = true+ Glass.pressable(this)+ setOnClickListener { onClick() }+ layoutParams = LinearLayout.LayoutParams(dp(36), dp(36)).apply {+ leftMargin = dp(2)+ }+ }++ /**+ * The same button, with a spoken name for a glyph that is not one.+ *+ * Deliberately not localised. "自检" is the name of this page in the one+ * language it has a name in, and the alternative — a `@StringRes` picked by+ * [AppLocale] — would mean rebuilding the whole control bar on a language+ * change for one content description nobody reads.+ */+ private fun barButton(+ glyph: String,+ color: String,+ size: Float,+ label: String,+ onClick: () -> Unit,+ ): View = (barButton(glyph, color, size, onClick) as GlyphButton).apply {+ contentDescription = label+ }++ // ---- Tap / drag / long-press on the ball ----++ private inner class BallHostFrame(ctx: Context) : FrameLayout(ctx) {+ /** Set by [build]; used to constrain touch hit-testing to the ball area. */+ var ballAreaView: View? = null++ /**+ * The card is two stacked areas — ball on top, scrollback under it —+ * and the split is only computable here, where the card's real height+ * exists. Driving it from the fold transitions instead would read+ * `params.height` while it is still MATCH_PARENT or still the corner+ * square; this fires after every geometry change, including the unfold+ * and a rotation, and needs no special case for either.+ */+ override fun onSizeChanged(w: Int, h: Int, oldW: Int, oldH: Int) {+ super.onSizeChanged(w, h, oldW, oldH)+ if (compact || h <= 0) return+ // Posted, not immediate: this runs inside the layout pass, and a+ // child's requestLayout from there is swallowed — the ball area+ // stayed MATCH_PARENT and the ball sat in the middle of the card+ // instead of the middle of its own area.+ // Posted, not immediate: this runs inside the layout pass, and a+ // child's requestLayout from there is swallowed — the ball area+ // stayed MATCH_PARENT and the ball sat in the middle of the card+ // instead of the middle of its own area.+ post { if (!compact) refitCardAreas() }+ }++ private val slop = ViewConfiguration.get(ctx).scaledTouchSlop+ private val longPressMs = ViewConfiguration.getLongPressTimeout().toLong()++ /**+ * How long to wait for another tap before acting on the ones so far.+ *+ * The platform's own double-tap timeout, so a double-tap here takes+ * exactly as long to register as one anywhere else on the device.+ */+ private val multiTapMs = ViewConfiguration.getDoubleTapTimeout().toLong()++ /** Taps of the current run, reset by [tapRunnable] when the window closes. */+ private var tapCount = 0++ /**+ * One tap of a possible run of three.+ *+ * The three gestures share one control, so a tap can no longer act on+ * its own — it has to wait out the multi-tap window to find out whether+ * it was a single one. That is the entire cost of this design: talking+ * now starts ~300 ms after the tap instead of on it. What it buys is the+ * other two, which were previously unreachable from the folded ball.+ *+ * Three is terminal and fires **immediately**. Stop is the urgent one —+ * the device may be tapping its way through somebody else's app — and a+ * fourth tap means nothing, so there is no window left to wait out.+ */+ private fun registerTap() {+ tapCount+++ ui.removeCallbacks(tapRunnable)+ if (tapCount >= TAPS_FOR_STOP) {+ tapCount = 0+ listeners.onStopClick()+ return+ }+ ui.postDelayed(tapRunnable, multiTapMs)+ }++ /** Whatever the run turned out to be. Runs on the UI thread. */+ private val tapRunnable = Runnable {+ when (tapCount) {+ 1 -> listeners.onTalkClick()+ 2 -> listeners.onTextInputToggle()+ }+ tapCount = 0+ }+ private var startX = 0f+ private var startY = 0f+ private var startParamsX = 0+ private var startParamsY = 0+ private var tracking = false+ private var moved = false+ private var longPressed = false++ /**+ * This drag is a sideways flick across the ball, i.e. a look change —+ * decided once, at the moment the slop is crossed, and never revisited.+ *+ * Deciding once is the point. A gesture that re-chose its meaning every+ * frame would drag the window a little before committing to a swipe,+ * which is a card that jumps and then snaps back for every look change.+ * The 1.4 bias is what keeps a hand that is trying to *move* the card+ * from being read as a swipe: a drag has to be clearly sideways, not+ * merely sideways-ish.+ *+ * Full card only. The folded ball is dragged around the screen and+ * magnets to the ledge, so a horizontal flick there is a move, and+ * taking it away would cost the user the one thing they do with the+ * corner ball most. In the other direction, the full card's ball+ * never drags the window either — see the move branch.+ */+ private var swiping = false+ private val longPressRunnable = Runnable {+ if (tracking && !moved) {+ longPressed = true+ listeners.onLongPress()+ }+ }++ private fun isOnBall(x: Float, y: Float, ballH: Int): Boolean {+ val cx = width / 2f+ // The ball area may sit at the bottom of a taller window (a+ // signboard above it), so its centre is not `ballH / 2` from the+ // window's top — read it from the area's real position.+ val cy = (ballAreaView?.top ?: 0) + ballH / 2f + (ball?.translationY ?: 0f)+ // Matches the visible sphere: camera z=4.9 + fov=42 makes the ball+ // fill ~55% of the shorter dimension, times whatever view scale the+ // ball is drawn at. Add a small margin so grazing taps still count.+ // Compact mode differs only in that scale (1f, no subtitle to make+ // room for) — it used to claim the whole window because a disc was+ // painted there, and without the disc that would be an invisible+ // square in the corner eating the user's taps.+ val scale = if (compact) 1f else BALL_SCALE+ val r = min(width, ballH) * 0.32f * scale+ val dx = x - cx+ val dy = y - cy+ return dx * dx + dy * dy <= r * r+ }++ override fun onTouchEvent(e: MotionEvent): Boolean {+ val p = params ?: return false+ val ballH = ballAreaView?.height ?: return false+ val areaTop = ballAreaView?.top ?: 0+ when (e.action) {+ MotionEvent.ACTION_DOWN -> {+ // A signboard is hosted: its own buttons handle taps in the+ // pane; the ball stays draggable by its circle, wherever the+ // taller window put it.+ if (signboard != null) {+ if (e.y < areaTop) return false // the pane's own touch+ if (!isOnBall(e.x, e.y, ballH)) return false+ } else if (compact) {+ // Bare folded ball: circle test only, the whole window+ // is the area.+ if (!isOnBall(e.x, e.y, ballH)) return false+ } else {+ // Full card: ignore anything below the ball area (the+ // scrollback handles its own touches); then the circle.+ if (e.y > ballH) return false+ if (!isOnBall(e.x, e.y, ballH)) return false+ }+ // Don't let it bounce under a finger that is already on it.+ ui.removeCallbacks(strollRunnable)+ slide?.cancel()+ startX = e.rawX; startY = e.rawY+ startParamsX = p.x; startParamsY = p.y+ tracking = true+ moved = false+ swiping = false+ longPressed = false+ ui.postDelayed(longPressRunnable, longPressMs)+ return true+ }++ MotionEvent.ACTION_MOVE -> {+ if (!tracking) return false+ if (!moved) {+ val dx = abs(e.rawX - startX)+ val dy = abs(e.rawY - startY)+ if (dx > slop || dy > slop) {+ moved = true+ swiping = !compact && dx > dy * SWIPE_BIAS+ ui.removeCallbacks(longPressRunnable)+ }+ }+ // Compact only. The full card is a surface, not a widget:+ // it has no ledge to grip and no corner to sit in, and a+ // diagonal drag on its ball used to carry the whole+ // window off-screen — no clamp while it went, no magnet+ // to bring it back.+ if (moved && !swiping && compact) {+ p.x = startParamsX + (e.rawX - startX).toInt()+ // Y follows the finger even while gripping. The ledge+ // used to own Y outright, which kept the paws on their+ // line but also meant a ball that had clung — because a+ // task started, or because the user put it there — could+ // never be pulled back out.+ p.y = startParamsY + (e.rawY - startY).toInt()+ clampCompact(p)+ // Grab and let go on the way, not on release: the+ // paws are drawn on the ledge, so a ball being+ // lifted off it has to open its hands while the+ // finger is still moving, or it reads as gripping+ // thin air. Pose only — the geometry is the+ // finger's until ACTION_UP.+ val want = wantsPerch(p.y)+ if (want != perchApplied) {+ perchApplied = want+ // The hand that moved it is the authority on+ // where it sits, so this is also where the+ // user's own claim on the ledge is taken and+ // dropped — including over a running task's,+ // and over a page the brain is showing.+ perchedByUser = want+ if (!want) {+ perchedByTask = false+ perchedByPage = false+ }+ syncWorking()+ }+ if (!perchApplied) {+ homeX = p.x+ homeY = p.y+ }+ runCatching { wm.updateViewLayout(this, p) }+ }+ return true+ }++ MotionEvent.ACTION_UP -> {+ if (!tracking) return false+ ui.removeCallbacks(longPressRunnable)+ val wasTap = !moved && !longPressed+ // Only a compact drag moved the window, so only it has+ // anything to settle — a swipe switched looks, and a+ // full-card touch never moved the window at all.+ val wasDrag = moved && !swiping && compact+ val flick = if (swiping) e.rawX - startX else 0f+ tracking = false+ moved = false+ swiping = false+ if (wasDrag) settleAfterDrag()+ retimeStroll()+ if (wasTap) registerTap()+ // Right means forward, the way a stack of cards is dealt.+ if (abs(flick) > slop * SWIPE_COMMIT) {+ stepBallLook(if (flick > 0) 1 else -1)+ }+ return true+ }++ MotionEvent.ACTION_CANCEL -> {+ if (!tracking) return false+ ui.removeCallbacks(longPressRunnable)+ val wasDrag = moved && !swiping && compact+ tracking = false+ moved = false+ swiping = false+ if (wasDrag) settleAfterDrag()+ retimeStroll()+ return true+ }+ }+ return false+ }+ }++ // ---- Colors / units ----++ private fun buildToolButton(tool: Tool): View {+ // Dimmer than the four permanent keys on purpose: these are developer+ // instruments hidden behind `⋯`, and they sat at the same weight as ■+ // and ⚙ when they were all one colour.+ return GlyphButton(context, tool.glyph, Glass.SECONDARY, 15f).apply {+ isClickable = true+ isFocusable = true+ Glass.pressable(this)+ setOnClickListener { listeners.onToolClick(tool) }+ layoutParams = LinearLayout.LayoutParams(dp(32), dp(36)).apply {+ leftMargin = dp(1)+ }+ }+ }++ private fun dp(v: Int): Int =+ (v * context.resources.displayMetrics.density).toInt()++ /** The window's own height, or the ball's side when it is not laid out. */+ private fun windowH(): Int {+ val v = root ?: return dp(COMPACT_DP)+ return if (v.height > 0) v.height else dp(COMPACT_DP)+ }++ companion object {+ /**+ * Taps in a row that mean "stop". The ball's whole vocabulary, in one+ * place: one tap asks, two type, three stop. Long-press is not part of+ * it — that one predates the other two and is about the card's size+ * rather than about the assistant.+ */+ private const val TAPS_FOR_STOP = 3++ /**+ * How much of its area the ball is drawn at in the full card.+ *+ * Was 0.5f, which — after a fixed 200dp band and a 1 : 1.15 split with+ * the list had taken their cut — drew a ball *smaller* than the folded+ * corner one. Neither the band nor the split exists any more.+ *+ * Then 1.4f against a half-screen area, which overshot the other way:+ * `min(area) × 0.48 × scale` came to ~800px on a 1200px-wide screen, two+ * thirds of the width, and a face that size reads as a cartoon rather+ * than as a presence.+ *+ * Read together with [BALL_AREA_FRACTION], which is the other half of+ * the same number: the two were cut back together so the ball would+ * shrink without leaving a band of dead space where it used to be. As+ * set they draw ~510px, about twice the folded corner ball.+ */+ private const val BALL_SCALE = 1.05f++ /**+ * How much more sideways than vertical a drag on the ball must be+ * before it counts as a look-change swipe rather than a drag of the+ * card. See [BallHostFrame.swiping].+ */+ private const val SWIPE_BIAS = 1.4f++ /**+ * Distance, in slops, a swipe must actually cover to change anything.+ *+ * Three rather than one: crossing the slop only means the gesture+ * stopped being a tap, and a look change is not something to do to+ * somebody by accident. Short of this the swipe is simply dropped —+ * nothing moved, so there is nothing to undo.+ */+ private const val SWIPE_COMMIT = 3f++ /**+ * Where the card splits: ball above, scrollback below. Also the touch+ * boundary between them (see [BallHostFrame.onSizeChanged]) — the list+ * is a ScrollView and needs every touch that lands in its half.+ *+ * This is the answer, not a starting point: the sphere's clearance from+ * the control cluster in the top-right corner is bought by dropping the+ * ball *inside* this band, never by moving the band's bottom edge — that+ * edge is where the scrollback begins, and moving it is a change to the+ * card, not to the ball. See [ballDrop].+ *+ * Note the area's *height* is what sizes the ball once it drops below+ * the screen width, since the sphere is a fraction of the shorter side.+ * Below about 0.40 the ball starts shrinking faster than the room it+ * gains the list is worth.+ */+ private const val BALL_AREA_FRACTION = 0.40f++ /**+ * Clearance between the sphere's top edge and the bottom of the control+ * cluster.+ *+ * Twelve dp is about the cluster's own inner padding, so the ball clears+ * the pill by the same margin the pill's glyphs clear the pill's edge —+ * enough to read as two things, not as one collision.+ */+ private const val BALL_TOP_GAP_DP = 12++ /**+ * How far below the band's centre the ball rests, before the cluster+ * clearance is added on top.+ *+ * Taste, not geometry, and the distinction is worth keeping: [ballDrop]+ * answers "how far must it move to clear the controls", this answers+ * "where does it look right". Centred is where the arithmetic says it+ * goes and it reads slightly high, because the band starts under the+ * status bar — the empty space below the sphere is taller than the empty+ * space above it, so the eye puts the sphere above the middle. The user+ * asked for "a little lower" and this is that; it moves the ball and+ * nothing else (the band, the split, the sphere's size and the+ * scrollback are all untouched).+ *+ * On the phone this stacks with a ~12 dp cluster clearance, on a wide+ * tablet it is the whole offset.+ */+ private const val BALL_NUDGE_DP = 14++ /**+ * Floor for the card's height while a typing field is up — see+ * [setTypingTop].+ *+ * Reached only on a device whose keyboard eats most of the screen: the+ * card is meant to keep its ball and some of its scrollback visible+ * above the field, and a card clipped to nothing would be the "it+ * vanished" behaviour this replaced.+ */+ private const val MIN_TYPING_CARD_DP = 240++ /**+ * Compact window side and its inset from the screen corner. The sphere+ * is a fixed fraction of the surface's shorter side, so this is the+ * only knob for how big the corner ball reads — the rendered ball is+ * roughly two thirds of it, and the circular hit test tracks it too.+ */+ private const val COMPACT_DP = 168+ private const val COMPACT_MARGIN_DP = 16++ /**+ * How much of the ball area's padding a held signboard's tail reaches+ * into, as a fraction of the compact side. The sphere occupies ~64% of+ * the area, centred, so the padding above the head is ~18% of the side.+ *+ * Calibrated on device, not by eye: 0.30 measured a 9px overlap with+ * the head ("戳到脑袋"), 0.14 read as detached. Each 0.01 of the+ * fraction is ~3.8px on the test phone, so 0.235 targets a ~14px+ * clearance — the tail tip visibly *near* the head without touching,+ * which is what "举着" reads as.+ */+ private const val SIGN_OVERLAP_FRACTION = 0.235f++ private const val PERCH_SLIDE_MS = 240L++ /** The last-few-pixels pull onto the ledge when a drag lets go. */+ private const val SNAP_MS = 150L++ /**+ * How far short of the ledge still counts as touching it, dp. See+ * [wantsPerch] — the only tuned number in the magnet.+ */+ private const val SNAP_GRACE_DP = 18++ /** Gap between hops within one burst. */+ private const val STROLL_HOP_MS = 640L++ /** Standing still between bursts, plus up to [STROLL_REST_JITTER_MS]. */+ private const val STROLL_REST_MS = 6_000L+ private const val STROLL_REST_JITTER_MS = 14_000+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/Glass.kt b/app/src/main/java/net/kuafuai/andee/ui/Glass.ktnew file mode 100644index 0000000..f49e32e--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/Glass.kt@@ -0,0 +1,480 @@+package net.kuafuai.andee.ui++import android.animation.Animator+import android.animation.AnimatorListenerAdapter+import android.animation.ArgbEvaluator+import android.animation.ValueAnimator+import android.annotation.SuppressLint+import android.content.Context+import android.graphics.Color+import android.graphics.drawable.Drawable+import android.graphics.drawable.GradientDrawable+import android.graphics.drawable.LayerDrawable+import android.os.Build+import android.os.Handler+import android.os.Looper+import android.view.View+import android.view.ViewGroup+import android.view.WindowManager+import android.view.animation.OvershootInterpolator+import android.view.animation.PathInterpolator++/**+ * The look of every card this app puts on screen, in one place.+ *+ * Four overlays share it — [SettingsUi], [VaultUi], [WakeEnrollUi],+ * [SelfCheckUi] — and they share it because they are reachable from one+ * another: the settings card opens the vault on top of itself, and the+ * self-check card opens the settings card on top of *itself*. Two different+ * dark blues stacked like that read as a bug before they read as a style.+ *+ * **The base colour is pure black and everything above it is white at a low+ * alpha.** That is not only taste. These are overlays: what is behind them is+ * some other app, and it can be a white document or a night-mode terminal. A+ * palette of *opaque* greys has to pick one of those to look right against. A+ * stack of translucent whites over black does not — it darkens whatever is+ * behind it and then lays the same relative steps on top, so the hierarchy+ * (card → panel → recessed well → thumb) survives both.+ *+ * ### The blur is real, and it is a different mechanism from the one that isn't+ *+ * `CLAUDE.md` records that frosted glass is unavailable for the pills inside+ * `FloatingWindowUi`, and that is still true: `RenderEffect` blurs a View's own+ * content, and `Window.setBackgroundBlurRadius` cannot be scoped to one floating+ * child of a window.+ *+ * These cards are not children. **Each one is its own window**, so the+ * window-level blur applies to exactly the right rectangle: [frost] sets+ * `FLAG_BLUR_BEHIND` + `blurBehindRadius`, and what is behind the card comes+ * through it defocused.+ *+ * Only that one. The *other* half of Android's blur API — `backgroundBlurRadius`,+ * the one clipped to the window's own rounded bounds — is a `@hide` field whose+ * public door is `Window.setBackgroundBlurRadius`, and we have no `Window`:+ * these are bare Views handed to `WindowManager.addView`. So the blur is not+ * clipped to the card; **the whole screen behind it softens**, which for a+ * centred modal is the effect you wanted anyway.+ *+ * It is API 31+ **and optional at runtime**: `isCrossWindowBlurEnabled` is false+ * on battery saver, on devices that cannot afford it, and whenever the developer+ * option is off. So [frost] returns whether it got the blur and the caller picks+ * its fill accordingly — [CARD_FROSTED] when the compositor is doing the work,+ * [CARD_SOLID] when nobody is and a 60%-black card would just be a smeary window+ * onto the app below.+ *+ * ### Translucency is also the safer direction for touch+ *+ * Every overlay this app owns counts toward Android's 0.80 untrusted-touch+ * ceiling, and going over it makes the platform silently discard the brain's+ * injected gestures. The old cards were fully opaque. These are not, so this+ * change can only move that number down.+ *+ * ### Motion+ *+ * Nothing here animates for decoration. Each one answers a question the user+ * would otherwise have to ask: [enter]/[exit] say which card appeared and where+ * it went, the sliding segment thumb says *the same control* changed value+ * rather than a new one appearing, and [expand]/[collapse] say the rows that+ * just vanished were folded away, not deleted. Durations are short enough+ * (≤300 ms) that a user who is not looking for them will not notice, which is+ * the point — a settings card that makes you wait for it is worse than one that+ * snaps.+ */+object Glass {++ // ---- Palette ----++ /** The card's fill when [frost] got its blur: dark enough to read against. */+ const val CARD_FROSTED = "#99000000"++ /** …and when it did not. Nearly opaque, because there is nothing to show. */+ const val CARD_SOLID = "#F2060608"++ /** The card's edge. The one place a bright hairline belongs. */+ const val HAIRLINE = "#2BFFFFFF"++ /** A grouped surface sitting on the card. */+ const val PANEL = "#0FFFFFFF"+ const val PANEL_EDGE = "#17FFFFFF"++ /**+ * A card sitting on the blurred backdrop instead of on [PANEL].+ *+ * [PANEL] cannot do this job: 6% white is the right film for a row group+ * *inside* a 60%-black card and is invisible over the fullscreen backdrop,+ * because those two surfaces have nothing in common — one is nearly opaque+ * black, the other is a blurred photo at ~13% mean luma. This pair is the+ * glassmorphism recipe proper, and the numbers are the balance that makes+ * it read as a pane rather than as a hole: enough fill for [LABEL] and+ * [SECONDARY] to sit on, enough edge and sheen to give the rectangle a lit+ * top. See [glass].+ *+ * **Currently no caller.** It was written for the self-check page, which was+ * a fullscreen surface on [net.kuafuai.andee.ui.Backdrop] with these rows on+ * it for one day; that page is now a `Glass.card` opened from `✓` (see+ * [net.kuafuai.andee.ui.SelfCheckUi]) and its rows are [PANEL], because a+ * surface's fill is decided by what is directly underneath it and nothing+ * else — 12% white on a 60%-black card is a grey box. The distinction is+ * kept, rather than the two constants deleted, because the *fact* behind it+ * has not gone anywhere: [Backdrop] is still what+ * [net.kuafuai.andee.ui.TextInputActivity] and the launcher self-check host+ * paint themselves with, and anything put on top of it needs this fill and+ * not [PANEL].+ */+ const val GLASS = "#1FFFFFFF"+ const val GLASS_EDGE = "#38FFFFFF"++ /** Recessed: text fields and the track a segmented picker slides in. */+ const val WELL = "#59000000"+ const val WELL_EDGE = "#1AFFFFFF"++ /** The pane that slides between segments. Lighter than what it sits on. */+ const val THUMB = "#2EFFFFFF"+ const val THUMB_EDGE = "#40FFFFFF"++ /** The highlight that makes a flat rectangle read as a pane of something. */+ private const val SHEEN_TOP = "#1FFFFFFF"+ private const val SHEEN_BOTTOM = "#00FFFFFF"++ const val TITLE = "#FFFFFF"+ const val LABEL = "#DCE1EA"+ const val SECONDARY = "#8F97A6"+ const val MUTED = "#5F6775"++ /** Section headings and anything tappable that is only text. */+ const val ACCENT = "#7FA5FF"++ /** The primary button is a gradient, which is the one place colour is loud. */+ const val TINT_START = "#4C6FFF"+ const val TINT_END = "#8250FF"++ const val DANGER = "#FF6B6B"++ // ---- Motion ----++ /** Fast out of the gate, long tail. The house easing. */+ val EASE: PathInterpolator = PathInterpolator(0.2f, 0f, 0f, 1f)++ /** For things leaving, which should not linger. */+ val EASE_IN: PathInterpolator = PathInterpolator(0.4f, 0f, 1f, 1f)++ private val ARGB = ArgbEvaluator()++ /**+ * Not the view's own handler: [exit]'s completion has to run even when the+ * animation never finishes because the window went away underneath it.+ */+ private val ui = Handler(Looper.getMainLooper())++ // ---- Window ----++ /**+ * Ask the compositor to blur behind this window. Returns whether it agreed.+ *+ * Call before `addView`; on a window that never changes its mind the flag+ * belongs on the params it is created with. The four-argument overload is+ * for the one window that does.+ *+ * See the class KDoc for why the answer can be no, and for why this blurs+ * the whole screen rather than only the card.+ */+ fun frost(context: Context, wm: WindowManager, p: WindowManager.LayoutParams): Boolean =+ frost(context, wm, p, want = true)++ /**+ * The same, for a window that stops wanting the blur and later wants it+ * back — [FloatingWindowUi], which is one window in two shapes: a+ * fullscreen card that should frost, and a corner ball that must not+ * (a blurred 168 dp square hanging around the ball is just a smudge on+ * the user's screen).+ *+ * Unlike the three cards this one is applied with `updateViewLayout` after+ * the fact, which does take. The caller pushes the params; this only edits+ * them, and returns what the fill should now assume.+ */+ fun frost(+ context: Context,+ wm: WindowManager,+ p: WindowManager.LayoutParams,+ want: Boolean,+ ): Boolean {+ if (!want ||+ Build.VERSION.SDK_INT < Build.VERSION_CODES.S ||+ !wm.isCrossWindowBlurEnabled+ ) {+ p.flags = p.flags and WindowManager.LayoutParams.FLAG_BLUR_BEHIND.inv()+ return false+ }+ p.flags = p.flags or WindowManager.LayoutParams.FLAG_BLUR_BEHIND+ // Heavier than it would need to be if it were clipped to the card: this+ // one blur has to both frost the pane and push the app underneath back.+ p.blurBehindRadius = dp(context, 48)+ return true+ }++ // ---- Surfaces ----++ /**+ * The card itself: a translucent fill, a sheen down the top third, a+ * hairline edge.+ *+ * @param frosted what [frost] returned — it decides the fill, not the shape.+ */+ fun card(context: Context, radius: Int, frosted: Boolean): Drawable = sheened(+ base = GradientDrawable().apply {+ shape = GradientDrawable.RECTANGLE+ setColor(Color.parseColor(if (frosted) CARD_FROSTED else CARD_SOLID))+ cornerRadius = radius.toFloat()+ setStroke(dp(context, 1), Color.parseColor(HAIRLINE))+ },+ radius = radius,+ )++ /** A group of rows, raised off the card. */+ fun panel(context: Context, radius: Int): Drawable = sheened(+ base = fill(context, PANEL, PANEL_EDGE, radius),+ radius = radius,+ )++ /**+ * A pane on the fullscreen backdrop — the glass in glassmorphism.+ *+ * Same shape as [panel] and a different surface: see [GLASS] for why the+ * two fills cannot be swapped. The sheen is what does most of the work+ * here, because the fill is only 12% white and the backdrop behind it is+ * mostly dark — it is the top-lit edge that makes the rectangle look like+ * something sitting *on* the image rather than a window cut into it.+ *+ * **No caller at the moment; see [GLASS] for why it stayed anyway.** Reach+ * for it whenever the parent is [net.kuafuai.andee.ui.Backdrop] rather than a+ * card, and for a whole fullscreen surface (a page's own background) use+ * `Backdrop` plus a `CENTER_CROP` ImageView instead — this is for the cards+ * that go *on* that.+ */+ fun glass(context: Context, radius: Int): Drawable = sheened(+ base = fill(context, GLASS, GLASS_EDGE, radius),+ radius = radius,+ )++ /** A field or a picker track: pressed *into* the card rather than onto it. */+ fun well(context: Context, radius: Int): Drawable = fill(context, WELL, WELL_EDGE, radius)++ /** The segmented picker's sliding pane. */+ fun thumb(context: Context, radius: Int): Drawable = sheened(+ base = fill(context, THUMB, THUMB_EDGE, radius),+ radius = radius,+ )++ /** The one loud surface: the button that commits. */+ @Suppress("UNUSED_PARAMETER") // takes a Context like its siblings, so callers read alike+ fun tinted(context: Context, radius: Int): Drawable = LayerDrawable(+ arrayOf(+ GradientDrawable(+ GradientDrawable.Orientation.LEFT_RIGHT,+ intArrayOf(Color.parseColor(TINT_START), Color.parseColor(TINT_END)),+ ).apply { cornerRadius = radius.toFloat() },+ sheen(radius),+ ),+ )++ private fun fill(context: Context, color: String, edge: String, radius: Int) =+ GradientDrawable().apply {+ shape = GradientDrawable.RECTANGLE+ setColor(Color.parseColor(color))+ cornerRadius = radius.toFloat()+ setStroke(dp(context, 1), Color.parseColor(edge))+ }++ private fun sheened(base: Drawable, radius: Int): Drawable =+ LayerDrawable(arrayOf(base, sheen(radius)))++ /**+ * A white gradient fading out downward. It is what separates "a rectangle+ * filled with 6% white" from something that looks lit from above.+ */+ private fun sheen(radius: Int): Drawable = GradientDrawable(+ GradientDrawable.Orientation.TOP_BOTTOM,+ intArrayOf(Color.parseColor(SHEEN_TOP), Color.parseColor(SHEEN_BOTTOM)),+ ).apply { cornerRadius = radius.toFloat() }++ // ---- Card motion ----++ /** Rise into place. Slight overshoot, because a card that just appears is flat. */+ fun enter(v: View) {+ v.alpha = 0f+ v.scaleX = 0.94f+ v.scaleY = 0.94f+ v.translationY = dp(v.context, 14).toFloat()+ v.animate()+ .alpha(1f).scaleX(1f).scaleY(1f).translationY(0f)+ .setDuration(320)+ .setInterpolator(OvershootInterpolator(0.9f))+ .start()+ }++ /**+ * Sink away, then run [done] — which is where the caller removes the window.+ *+ * [done] runs exactly once and runs even if the animation is cancelled+ * (a detached view never delivers its end action), because what hangs off it+ * is `removeView` plus the [OwnCard] bookkeeping that tells the brain+ * whether one of our own cards is covering the screen. Leaking that count+ * would have the device permanently claim a card is open.+ */+ fun exit(v: View, done: () -> Unit) {+ val once = object : Runnable {+ private var ran = false+ override fun run() {+ if (ran) return+ ran = true+ done()+ }+ }+ v.animate()+ .alpha(0f).scaleX(0.96f).scaleY(0.96f).translationY(dp(v.context, 8).toFloat())+ .setDuration(170)+ .setInterpolator(EASE_IN)+ .withEndAction(once)+ .start()+ ui.postDelayed(once, 400)+ }++ /** Cross-fade one root view for another — see `SettingsUi.rebuild`. */+ fun fadeOut(v: View, then: () -> Unit) {+ v.animate().alpha(0f).scaleX(0.985f).scaleY(0.985f)+ .setDuration(110).setInterpolator(EASE_IN)+ .withEndAction(then).start()+ ui.postDelayed(then, 300)+ }++ fun fadeIn(v: View) {+ v.alpha = 0f+ v.scaleX = 0.985f+ v.scaleY = 0.985f+ v.animate().alpha(1f).scaleX(1f).scaleY(1f)+ .setDuration(200).setInterpolator(EASE).start()+ }++ // ---- Row motion ----++ /**+ * Unfold a hidden block to its natural height.+ *+ * Measured against the parent's usable width rather than animated to+ * `WRAP_CONTENT` (which no animator can interpolate to), and the height is+ * handed back to `WRAP_CONTENT` at the end so later content changes still+ * size it. If the parent has not been laid out yet there is nothing to+ * measure against, and the block simply appears — that is the first build,+ * where there was no transition to show anyway.+ */+ fun expand(v: View) {+ if (v.visibility == View.VISIBLE) return+ val parent = v.parent as? View+ val avail = (parent?.width ?: 0) - (parent?.paddingLeft ?: 0) - (parent?.paddingRight ?: 0)+ if (avail <= 0) {+ v.visibility = View.VISIBLE+ v.alpha = 1f+ return+ }+ v.measure(+ View.MeasureSpec.makeMeasureSpec(avail, View.MeasureSpec.EXACTLY),+ View.MeasureSpec.makeMeasureSpec(0, View.MeasureSpec.UNSPECIFIED),+ )+ val target = v.measuredHeight+ val lp = v.layoutParams+ v.visibility = View.VISIBLE+ v.alpha = 0f+ ValueAnimator.ofInt(0, target).apply {+ duration = 240+ interpolator = EASE+ addUpdateListener {+ lp.height = it.animatedValue as Int+ v.layoutParams = lp+ v.alpha = it.animatedFraction+ }+ addListener(object : AnimatorListenerAdapter() {+ override fun onAnimationEnd(a: Animator) {+ lp.height = ViewGroup.LayoutParams.WRAP_CONTENT+ v.layoutParams = lp+ v.alpha = 1f+ }+ })+ start()+ }+ }++ /** The mirror of [expand]. Ends `GONE`, so it stops costing layout. */+ fun collapse(v: View) {+ if (v.visibility != View.VISIBLE) return+ val from = v.height+ val lp = v.layoutParams+ if (from <= 0) {+ v.visibility = View.GONE+ return+ }+ ValueAnimator.ofInt(from, 0).apply {+ duration = 200+ interpolator = EASE+ addUpdateListener {+ lp.height = it.animatedValue as Int+ v.layoutParams = lp+ v.alpha = 1f - it.animatedFraction+ }+ addListener(object : AnimatorListenerAdapter() {+ override fun onAnimationEnd(a: Animator) {+ v.visibility = View.GONE+ lp.height = ViewGroup.LayoutParams.WRAP_CONTENT+ v.layoutParams = lp+ v.alpha = 1f+ }+ })+ start()+ }+ }++ fun setVisible(v: View, visible: Boolean, animate: Boolean) {+ if (!animate) {+ v.visibility = if (visible) View.VISIBLE else View.GONE+ v.alpha = 1f+ return+ }+ if (visible) expand(v) else collapse(v)+ }++ // ---- Touch ----++ /**+ * Shrink under the finger and spring back.+ *+ * Returns false from the touch listener on purpose: the View's own+ * `onTouchEvent` still runs, so this is purely additive and cannot swallow+ * a click. Lint's accessibility warning is suppressed because every window+ * this is used in calls `hideFromAccessibility` — these views are invisible+ * to screen readers by design, ours included.+ */+ @SuppressLint("ClickableViewAccessibility")+ fun pressable(v: View) {+ v.setOnTouchListener { view, ev ->+ when (ev.actionMasked) {+ android.view.MotionEvent.ACTION_DOWN ->+ view.animate().scaleX(0.96f).scaleY(0.96f).alpha(0.8f)+ .setDuration(90).setInterpolator(EASE).start()++ android.view.MotionEvent.ACTION_UP,+ android.view.MotionEvent.ACTION_CANCEL,+ ->+ view.animate().scaleX(1f).scaleY(1f).alpha(1f)+ .setDuration(200).setInterpolator(OvershootInterpolator(2f)).start()+ }+ false+ }+ }++ // ---- Helpers ----++ /** One shared evaluator: segment repainting runs this on every frame. */+ fun lerpColor(from: Int, to: Int, f: Float): Int = ARGB.evaluate(f, from, to) as Int++ fun dp(context: Context, v: Int): Int =+ (v * context.resources.displayMetrics.density).toInt()+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/HistoryListView.kt b/app/src/main/java/net/kuafuai/andee/ui/HistoryListView.ktnew file mode 100644index 0000000..37bbbc5--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/HistoryListView.kt@@ -0,0 +1,450 @@+package net.kuafuai.andee.ui++import android.annotation.SuppressLint+import android.content.Context+import android.graphics.Color+import android.graphics.drawable.GradientDrawable+import android.graphics.drawable.RippleDrawable+import android.content.res.ColorStateList+import android.view.GestureDetector+import android.view.Gravity+import android.view.MotionEvent+import android.view.View+import android.view.ViewGroup+import android.widget.LinearLayout+import android.widget.ScrollView+import android.widget.TextView+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import java.text.SimpleDateFormat+import java.util.Date++/**+ * The scrollback under the ball in the fullscreen card: what was said, and+ * every page the assistant drew, each one re-openable.+ *+ * ## Why it looks the way it does+ *+ * The brief was Apple's frosted glass. There *is* now a real blur behind all+ * of this — [FloatingWindowUi] asks the compositor for it at the window level+ * (see [Glass.frost]) — but it happens once, for the whole card, and these+ * rows are children of that window. A child View cannot blur what is behind+ * it; `RenderEffect` blurs a view's own content. So the panes here are made+ * the same way they were when the card was opaque, and the blur underneath is+ * simply what they now sit on.+ *+ * What reads as glass on a dark surface isn't the blur anyway, it's the three+ * things around it: a translucent white *film* rather than a solid fill, a+ * top-to-bottom gradient standing in for light falling across a pane, and a+ * hairline border a shade brighter than the film so the pane has an edge where+ * it catches that light. Those are three cheap drawables and they composite in+ * one pass. Colours come from [Glass] so the scrollback, the cards and the+ * dialogs are one palette.+ *+ * ## The fade+ *+ * "上下渐渐隐藏" is Android's own fading edge, not a pair of gradient views+ * laid over the top and bottom. The platform one fades the *content* to+ * transparent and — the part a hand-rolled overlay always gets wrong — only on+ * the sides that actually have more content to scroll to, so the list doesn't+ * sit there with a phantom shadow above a first row that is already at the top.+ */+class HistoryListView(private val context: Context) : ScrollView(context) {++ /** Tapping a page row. Only ever fired for entries that still have a file. */+ var onPageClick: ((ChatHistory.Entry) -> Unit)? = null++ /**+ * Double-tapping a message — read it fullscreen. See [attachDoubleTap].+ *+ * Handed out rather than opened here for the same reason page rows are: the+ * reader is a fullscreen screen, so the card has to fold out of its way+ * first, and that sequencing belongs to whoever owns the dispatcher.+ */+ var onTextOpen: ((ChatHistory.Entry) -> Unit)? = null++ /**+ * The rows themselves.+ *+ * It used to end with a permanent dim line — "点击小球跟我对话" — as a hint+ * at the bottom of every conversation. It reads well and it scrolls away,+ * which is the one thing a hint about *how to ask* must not do: the moment+ * the user wants it is the moment they have scrolled up looking for what+ * happened. The hint is pinned along the bottom of the card now (see+ * `FloatingWindowUi.build`), and this column is back to being only the+ * conversation.+ */+ private val column = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ // The bottom padding clears the gesture bar. It used to be 30dp because+ // the hint was the last row and the navigation strip ate it; the pinned+ // tips bar owns that edge now and reserves it, so all this needs to be+ // is a little air between the last bubble and the bar.+ setPadding(dp(14), dp(2), dp(14), dp(10))+ }++ /**+ * Strings in the user's language. Views below are still built on the raw+ * [context]; only text comes from here. See [AppLocale].+ *+ * A `var` rather than a `val` because the language can change while this+ * list is on screen — the settings card is opened on top of it. See+ * [onLocaleChanged].+ */+ private var lctx: Context = AppLocale.wrap(context)++ private val clock = SimpleDateFormat("HH:mm", AppLocale.localeOf(context))++ /**+ * Rows currently laid out, by entry id. Rebuilding every row on every+ * append would cancel the ripple on the row the user is touching and reset+ * the scroll offset mid-drag; appends are by far the common case.+ */+ private val laid = LinkedHashSet<Long>()++ private val empty = TextView(context).apply {+ text = lctx.getString(R.string.history_empty)+ textSize = 13f+ setTextColor(Color.parseColor(Glass.MUTED))+ gravity = Gravity.CENTER+ setPadding(0, dp(28), 0, 0)+ }++ /**+ * What is happening right now — a partial transcript, the tool being run,+ * the sentence being read out. Lives as the list's last row rather than in+ * a band of its own, so speech, answers, pages and failures are one+ * chronological column instead of two surfaces the user has to watch.+ *+ * Always the last child, and pushed back there after every append: entries+ * settle *behind* the live line, which is by definition still in progress.+ */+ private val live = TextView(context).apply {+ textSize = 15f+ visibility = View.GONE+ setPadding(dp(4), dp(8), dp(4), dp(2))+ maxLines = 4+ ellipsize = android.text.TextUtils.TruncateAt.END+ }++ init {+ isVerticalFadingEdgeEnabled = true+ setFadingEdgeLength(dp(44))+ // Without this the fade is drawn over the scrollbar's gutter and the+ // rows lose 6dp of width to a bar nobody needs on a touch-scrolled list.+ isVerticalScrollBarEnabled = false+ overScrollMode = OVER_SCROLL_NEVER+ // The card is the whole screen; the list is the part of it that scrolls,+ // and it must not eat the ball's drag above it. ScrollView already+ // claims only vertical drags that start inside itself.+ addView(+ column,+ LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.WRAP_CONTENT),+ )+ column.addView(empty)+ column.addView(live)+ }++ /**+ * Rows appended while the card was folded were laid out into a view with no+ * height, so the [refresh] that queued them scrolled a list that had nowhere+ * to scroll. The card is folded for most of its life — this is the common+ * case, not an edge one — so the unfold's first real measurement is where+ * the bottom has to be re-taken.+ */+ override fun onSizeChanged(w: Int, h: Int, oldW: Int, oldH: Int) {+ super.onSizeChanged(w, h, oldW, oldH)+ if (h > 0 && h != oldH) post { fullScroll(FOCUS_DOWN) }+ }++ /**+ * Show (or with an empty string, hide) the live row.+ *+ * Main thread only.+ */+ fun setLive(text: String, kind: FloatingWindowUi.SubtitleKind) {+ removeCallbacks(clearLive)+ if (text.isEmpty()) {+ live.visibility = View.GONE+ return+ }+ live.text = text+ live.setTextColor(+ when (kind) {+ // Dim white for something still being said; the accent for an+ // announcement that is finished and will not be superseded.+ FloatingWindowUi.SubtitleKind.PARTIAL -> Color.parseColor(Glass.LABEL)+ FloatingWindowUi.SubtitleKind.FINAL -> Color.parseColor(Glass.ACCENT)+ }+ )+ live.visibility = View.VISIBLE+ // FINAL is an announcement nothing will supersede — "已停止", a tool's+ // result. PARTIAL and SPEAKING are always followed by something that+ // replaces or clears them, but a FINAL left alone would sit at the foot+ // of the scrollback forever, under the hint, reading as the newest thing+ // said long after it stopped being true.+ if (kind == FloatingWindowUi.SubtitleKind.FINAL) postDelayed(clearLive, FINAL_LINGER_MS)+ post { fullScroll(FOCUS_DOWN) }+ }++ private val clearLive = Runnable { live.visibility = View.GONE }++ /**+ * Pull the current log in and lay out anything new, then stick to the+ * bottom — the newest line is the one worth reading, and a list that held+ * its old offset while the assistant talked would need scrolling before it+ * showed you the answer you just heard.+ *+ * Main thread only.+ */+ fun refresh() {+ val all = ChatHistory.snapshot()+ empty.visibility = if (all.isEmpty()) View.VISIBLE else View.GONE++ val ids = all.map { it.id }.toSet()+ if (laid.any { it !in ids }) {+ // A row we laid out is gone from the history — evicted from the+ // middle, or the whole list wiped. Cheaper and less error-prone to+ // start over than to reconcile. This deliberately does not also+ // require something to have been *added*: 清空聊天记录 removes+ // every row and adds none, and pairing the two conditions left the+ // card scrolling a conversation that no longer existed.+ column.removeAllViews()+ column.addView(empty)+ column.addView(live)+ laid.clear()+ }+ // Inserted above the live row rather than appended, so a settled entry+ // lands behind the thing that is still in progress.+ for (e in all) {+ if (laid.add(e.id)) {+ column.addView(buildRow(e), column.indexOfChild(live))+ }+ }+ post { fullScroll(FOCUS_DOWN) }+ }++ /**+ * The language changed under us. Re-wrap, then lay the list out again from+ * scratch.+ *+ * Rows carry their language in their own text — there is nothing to+ * translate in place, because the row that says 我 · 15:31 has to become+ * its English twin, not just swap a label. So forget which rows are laid+ * out and rebuild them, which is the same path an eviction takes.+ *+ * The scroll position goes with them. Acceptable: the only way to get here+ * is changing the language in 设置, which is opened over this list, and the+ * bottom of the conversation is where the user was heading anyway.+ *+ * Main thread only.+ */+ fun onLocaleChanged() {+ lctx = AppLocale.wrap(context)+ empty.text = lctx.getString(R.string.history_empty)+ laid.clear()+ column.removeAllViews()+ column.addView(empty)+ column.addView(live)+ refresh()+ }++ private fun buildRow(e: ChatHistory.Entry): View {+ val mine = e.role == ChatHistory.Role.USER+ val bad = e.role == ChatHistory.Role.ERROR++ val bubble = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ setPadding(dp(14), dp(10), dp(14), dp(10))+ background = glass(mine, bad)+ }++ val head = TextView(context).apply {+ text = when (e.role) {+ ChatHistory.Role.USER ->+ lctx.getString(R.string.history_role_you, clock.format(Date(e.ts)))+ ChatHistory.Role.ASSISTANT ->+ lctx.getString(R.string.history_role_assistant, clock.format(Date(e.ts)))+ ChatHistory.Role.ERROR ->+ lctx.getString(R.string.history_role_error, clock.format(Date(e.ts)))+ ChatHistory.Role.PAGE ->+ if (e.page != null) {+ lctx.getString(R.string.history_role_page_open, clock.format(Date(e.ts)))+ } else {+ // Said plainly rather than by greying the row out: a row+ // that looks disabled reads as a bug in the list, not as+ // a page that has aged out.+ lctx.getString(R.string.history_role_page_cleared, clock.format(Date(e.ts)))+ }+ }+ textSize = 10f+ letterSpacing = 0.05f+ setTextColor(+ when (e.role) {+ ChatHistory.Role.USER -> Color.parseColor(Glass.SECONDARY)+ ChatHistory.Role.ASSISTANT -> Color.parseColor(Glass.SECONDARY)+ ChatHistory.Role.ERROR -> Color.parseColor(Glass.DANGER)+ ChatHistory.Role.PAGE ->+ if (e.page != null) Color.parseColor(Glass.ACCENT)+ else Color.parseColor(Glass.MUTED)+ }+ )+ }+ bubble.addView(head)++ val body = TextView(context).apply {+ text = e.text+ textSize = 14f+ setLineSpacing(dp(3).toFloat(), 1f)+ setTextColor(Color.parseColor(if (bad) Glass.DANGER else Glass.LABEL))+ setPadding(0, dp(3), 0, 0)+ // Long answers are for glancing at here and listening to in full;+ // the row is a handle, not the document.+ maxLines = 6+ ellipsize = android.text.TextUtils.TruncateAt.END+ }+ bubble.addView(body)++ if (e.role == ChatHistory.Role.PAGE && e.page != null) {+ bubble.isClickable = true+ bubble.background = RippleDrawable(+ ColorStateList.valueOf(RIPPLE),+ glass(false, false),+ null,+ )+ bubble.setOnClickListener { onPageClick?.invoke(e) }+ } else {+ attachDoubleTap(bubble, e)+ }++ // The bubble is the row's whole width minus a shoulder on the side it+ // is *not* speaking from — which is what makes two voices readable+ // without drawing a tail or a portrait on anything.+ return LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = if (mine) Gravity.END else Gravity.START+ layoutParams = LinearLayout.LayoutParams(+ ViewGroup.LayoutParams.MATCH_PARENT,+ ViewGroup.LayoutParams.WRAP_CONTENT,+ ).apply { topMargin = dp(8) }+ addView(+ bubble,+ LinearLayout.LayoutParams(0, ViewGroup.LayoutParams.WRAP_CONTENT, 1f).apply {+ if (mine) leftMargin = dp(40) else rightMargin = dp(40)+ },+ )+ }+ }++ /**+ * Open a message fullscreen on a double tap.+ *+ * **Double, not single.** A single tap inside a scrollable list is how you+ * stop a fling and how you start a drag, and a chat where one stray tap+ * replaces the whole screen with a reader is a chat you learn to tap+ * carefully. Nothing else on a text row wants a tap, so the gesture is free+ * — and it is the same one the ball uses for "the other thing", which is one+ * less thing to learn.+ *+ * Page rows are excluded at the call site, not here: their single tap+ * already means "open the page again", and a page is not text to be read out+ * of a bubble.+ *+ * The touch listener deliberately swallows the DOWN (the detector returns+ * true while it waits to see whether a second tap follows). That does not+ * cost the list its scrolling: a ScrollView intercepts on movement, and the+ * child only ever sees the events before that — a drag cancels the detector+ * exactly as it cancels any other child gesture.+ *+ * Main thread only.+ */+ @SuppressLint("ClickableViewAccessibility")+ private fun attachDoubleTap(v: View, e: ChatHistory.Entry) {+ val detector = GestureDetector(context, object : GestureDetector.SimpleOnGestureListener() {+ /**+ * **Must return true, and the default is false.**+ *+ * `SimpleOnGestureListener` is documented as a no-op listener, and+ * `onDown` is the one place where that is not harmless: declining the+ * DOWN means this view declines the gesture, so the list becomes the+ * touch target for everything after it and the detector never sees+ * the UP — never mind the second tap. The double tap then does+ * nothing at all, silently, with no state anywhere to say why.+ */+ override fun onDown(e: MotionEvent): Boolean = true++ override fun onDoubleTap(ev: MotionEvent): Boolean {+ onTextOpen?.invoke(e)+ return true+ }+ })+ v.setOnTouchListener { _, ev -> detector.onTouchEvent(ev) }+ }++ /**+ * The pane: a translucent film with light falling down it, under a hairline+ * that is one step brighter than the film so the edge reads as a lit rim+ * rather than as a drawn border.+ *+ * The user's own rows get a faint blue cast. Two panes of the same glass+ * with the same alignment would be one long column of identical rectangles+ * — the tint is doing the same job the alignment is, for the case where a+ * row is long enough to reach both margins anyway.+ *+ * Failures get a red one, in the same glass rather than a solid alarm+ * colour: an error that already came and went as a card is being recalled+ * here, not raised again.+ */+ private fun glass(mine: Boolean, bad: Boolean = false): GradientDrawable = GradientDrawable(+ GradientDrawable.Orientation.TOP_BOTTOM,+ when {+ bad -> intArrayOf(TINT_TOP_BAD, TINT_BOTTOM_BAD)+ mine -> intArrayOf(TINT_TOP_MINE, TINT_BOTTOM_MINE)+ else -> intArrayOf(TINT_TOP, TINT_BOTTOM)+ },+ ).apply {+ cornerRadius = dp(18).toFloat()+ setStroke(+ Math.max(1, dp(1) / 2),+ when {+ bad -> EDGE_BAD+ mine -> EDGE_MINE+ else -> EDGE+ },+ )+ }++ private fun dp(v: Int): Int = (v * resources.displayMetrics.density).toInt()++ private companion object {+ /** How long a [FloatingWindowUi.SubtitleKind.FINAL] live line stays up. */+ const val FINAL_LINGER_MS = 5_000L++ /**+ * A [Glass] token seen through the film: its hue at the film's alpha.+ * Derived rather than written out so the tinted panes cannot drift+ * away from the accent and danger colours everything else uses.+ */+ fun film(hex: String, alpha: Int): Int =+ (alpha shl 24) or (Color.parseColor(hex) and 0x00FFFFFF)++ // Alpha in the low teens is the whole trick: high enough to lift the+ // pane off the card, low enough that the card's colour still shows+ // through it, which is the only thing making it read as translucent.+ // These three are the neutral pane and match Glass's panel family.+ val TINT_TOP = Color.parseColor("#1FFFFFFF")+ val TINT_BOTTOM = Color.parseColor("#0FFFFFFF")+ val EDGE = Color.parseColor("#2EFFFFFF")++ val TINT_TOP_MINE = film(Glass.ACCENT, 0x2C)+ val TINT_BOTTOM_MINE = film(Glass.ACCENT, 0x14)+ val EDGE_MINE = film(Glass.ACCENT, 0x40)++ val TINT_TOP_BAD = film(Glass.DANGER, 0x2C)+ val TINT_BOTTOM_BAD = film(Glass.DANGER, 0x14)+ val EDGE_BAD = film(Glass.DANGER, 0x4D)++ /** The tap on a page row, in the accent it is labelled with. */+ val RIPPLE = film(Glass.ACCENT, 0x33)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/HtmlActivity.kt b/app/src/main/java/net/kuafuai/andee/ui/HtmlActivity.ktnew file mode 100644index 0000000..365f16d--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/HtmlActivity.kt@@ -0,0 +1,316 @@+package net.kuafuai.andee.ui++import android.annotation.SuppressLint+import android.graphics.Color+import android.os.Bundle+import android.util.Log+import android.view.Gravity+import android.view.ViewGroup+import android.webkit.WebView+import android.webkit.WebViewClient+import android.widget.FrameLayout+import android.widget.ProgressBar+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale++/**+ * Full-screen canvas for brain-generated UI: any HTML the model composes+ * renders here, live. Data in via [show] (file-backed, so huge HTML doesn't+ * ride an Intent), the brain gets "closed" when the user is done.+ *+ * A tiny close affordance floats top-right ("✕ 完成") — the brain's UI+ * must never trap the user.+ *+ * In and out are [StageActivity]'s, shared with the two camera screens: the+ * three used to arrive three different ways and none of them left smoothly,+ * because `finish()` here raced the platform's own exit animation and the+ * caller's hand-back of the card. See [StageActivity].+ */+class HtmlActivity : StageActivity() {++ companion object {+ @Volatile var htmlFile: java.io.File? = null+ @Volatile var onClose: (() -> Unit)? = null++ /**+ * What the open page is called.+ *+ * Callers always set this before opening ([showFile]), because the+ * title is theirs — a message's own first line, or the model's. The+ * blank default is the one case no caller covers, and [onCreate]+ * resolves it from resources so the fallback speaks the user's+ * language rather than a hardcoded one.+ */+ @Volatile var titleText: String = ""++ /**+ * 屏幕上是否已经有一页。**在屏**(onCreate 到 onDestroy 之间)为真。+ *+ * 放在这里而不是调用方:`showFile` 有两个入口 —— 大脑的 `ui.show_html`+ * 和用户在对话记录里点一页重看 —— 而"同时只能有一页"是 WebView 这个类+ * 的事实,不是某一个调用方的约定。那个共享的静态 [htmlFile] 也让它是+ * 事实:两个实例会读到同一份文件,第二个盖上来以后第一个显示的就是错的+ * 内容了。+ *+ * 给 `ui.show_html` 读:它现在**不再等用户关页面**(那次改动见+ * `CommandDispatcher`),所以"屏幕上还有一页"必须靠这个标志判断,否则+ * 模型连着要两页就会叠出两个 Activity。+ */+ @Volatile var pageUp = false++ /**+ * Origin the brain's page runs as.+ *+ * Not a `file://` URL and not null, and both halves of that are bugs+ * this went through. `targetSdk 30` turns `allowFileAccess` off by+ * default, so loading the staged file failed outright with+ * ERR_ACCESS_DENIED — the "网页无法打开" the user saw. A *null* base URL+ * fixes the load but hands the page an opaque origin, where+ * `localStorage` throws a SecurityException: one `localStorage.getItem`+ * near the top of a script — a high score, a saved setting, exactly what+ * a model writes without thinking about it — and the whole script dies+ * and the page renders blank. A stable fake https origin gives DOM+ * storage somewhere to live, and never resolves, which is fine because+ * the page is required to be self-contained anyway.+ */+ private const val BASE_URL = "https://brain.local/"++ private const val TAG = "Body"++ /** Show the page even if `onPageFinished` never lands. See [StageActivity]. */++ /**+ * Write the HTML to the cache dir and open the activity.+ * @return the file written, or null on IO failure.+ */+ fun show(context: android.content.Context, html: String, title: String, onClosed: () -> Unit): java.io.File? {+ return runCatching {+ val f = java.io.File(context.cacheDir, "brain_ui_${System.currentTimeMillis()}.html")+ f.writeText(html)+ f+ }.getOrNull()?.also { showFile(context, it, title, onClosed) }+ }++ /**+ * Open a page that already exists on disk.+ *+ * The file belongs to the caller and is never deleted here — which is+ * the whole difference from the old behaviour, where closing the page+ * wiped it. That was right when the file was scratch space for one+ * viewing; it is wrong now that [ChatHistory] keeps pages so they can+ * be opened again from the scrollback.+ */+ fun showFile(+ context: android.content.Context,+ file: java.io.File,+ title: String,+ onClosed: () -> Unit,+ ) {+ titleText = title+ onClose = onClosed+ htmlFile = file+ // **Before `startActivity`, not in `onCreate`.** Starting an activity+ // is asynchronous, so an `onCreate`-only flag is false for the whole+ // window between "we handed the page to the system" and "the page is+ // on screen" — and that window is exactly when a model that asked+ // for two pages in one message asks for the second one. Measured:+ // with the flag set in `onCreate`, a second `ui.show_html` 60 ms+ // later sailed straight through and stacked two activities over the+ // shared static file below.+ pageUp = true+ runCatching {+ context.startActivity(+ android.content.Intent(context, HtmlActivity::class.java)+ .addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)+ )+ }.onFailure {+ // Nothing will ever clear it from `onDestroy` if nothing started.+ pageUp = false+ throw it+ }+ }+ }++ private var web: WebView? = null+ private var spinner: ProgressBar? = null++ @SuppressLint("SetJavaScriptEnabled")+ override fun onCreate(savedInstanceState: Bundle?) {+ super.onCreate(savedInstanceState)+ // The brain composed a full-screen page; an AppCompat title bar reading+ // "Andee" above it is our chrome intruding on their canvas, and it+ // eats the top of every layout that assumed it had 100vh.+ // Again here, not instead of in [showFile]: this one covers the activity+ // being recreated by the system after process death, where no caller ran.+ pageUp = true+ // Only when no caller named the page. [titleText] itself is set by+ // [showFile] before this activity starts, so the common path is already+ // non-blank and this is the fallback for the recreated-instance case.+ if (titleText.isBlank()) {+ titleText = AppLocale.wrap(this).getString(R.string.html_default_title)+ }++ web = WebView(this).apply {+ settings.javaScriptEnabled = true+ settings.domStorageEnabled = true+ // The model composes for a phone-shaped viewport and says so in its+ // own <meta viewport>. Without this the WebView ignores the tag,+ // lays the page out at desktop width and scales it down — which is+ // most of why a generated page reads as "meant for a computer".+ settings.useWideViewPort = true+ settings.loadWithOverviewMode = true+ webViewClient = object : WebViewClient() {+ override fun onPageFinished(view: WebView, url: String) {+ revealStage()+ }++ override fun onReceivedError(+ view: WebView,+ request: android.webkit.WebResourceRequest,+ error: android.webkit.WebResourceError,+ ) {+ // Only the main document is worth shouting about; a page+ // that pulls a font off the internet it hasn't got should+ // still render.+ if (request.isForMainFrame) {+ Log.w(TAG, "brain page failed: ${error.errorCode} ${error.description}")+ }+ }+ }+ webChromeClient = object : android.webkit.WebChromeClient() {+ override fun onConsoleMessage(m: android.webkit.ConsoleMessage): Boolean {+ // The page is generated, unreviewed and un-debuggable from+ // the device. One throw in a top-level script renders a+ // blank screen with no other trace anywhere.+ Log.d(TAG, "brain page console: ${m.message()} @${m.lineNumber()}")+ return true+ }+ }+ layoutParams = matchParent()+ setBackgroundColor(BACKDROP)+ }++ spinner = ProgressBar(this).apply {+ isIndeterminate = true+ indeterminateTintList =+ android.content.res.ColorStateList.valueOf(Color.parseColor(Glass.ACCENT))+ layoutParams = FrameLayout.LayoutParams(+ dpI(34), dpI(34), Gravity.CENTER+ )+ }++ // Loaded as a string against [BASE_URL], not as the file's own URL: the+ // file is staging (it survives an activity recreate, and it keeps a+ // megabyte of HTML off the Intent), not something the WebView is+ // allowed to open.+ val html = htmlFile?.let { f -> runCatching { f.readText() }.getOrNull() }+ if (html == null) {+ Log.w(TAG, "no html staged for HtmlActivity")+ finish()+ return+ }+ web?.loadDataWithBaseURL(BASE_URL, html, "text/html", "utf-8", null)++ // Close pill — always reachable, floats above the content. Top-right,+ // where a close control is expected and where the page is least likely+ // to have put anything: generated pages open with a title and end with+ // their controls, so a pill pinned to the bottom lands on the buttons.+ val close = closePill(this) { leave() }.apply {+ layoutParams = FrameLayout.LayoutParams(+ ViewGroup.LayoutParams.WRAP_CONTENT,+ ViewGroup.LayoutParams.WRAP_CONTENT,+ Gravity.END or Gravity.TOP+ ).apply {+ topMargin = dpI(12)+ marginEnd = dpI(12)+ }+ }++ // **The pill has to clear the status bar, and it did not.** This window+ // is edge-to-edge — its frame is the whole display, which is why the+ // page's own 100vh starts at y=0 — so `topMargin = 12dp` puts the pill+ // *under* the clock and the battery, in the strip the status bar owns.+ // Measured on the 1600x2560 tablet: a tap dead centre on the pill was+ // swallowed by the status bar and the page would only close with the+ // back gesture. That matters more than it looks — the pill is the+ // affordance the tool description promises the model ("keep roughly+ // 110x44 clear there"), and before `ui.show_html` stopped blocking,+ // failing to reach it meant the brain sat in a five-minute wait.+ stage.setOnApplyWindowInsetsListener { _, insets ->+ val top = insets.getInsets(+ android.view.WindowInsets.Type.systemBars() or+ android.view.WindowInsets.Type.displayCutout(),+ ).top+ val lp = close.layoutParams as FrameLayout.LayoutParams+ val want = dpI(12) + top+ // Guarded like `FloatingWindowUi.insetTopBar`: writing layoutParams+ // asks for a layout pass, and this runs inside one.+ if (lp.topMargin != want) {+ lp.topMargin = want+ close.layoutParams = lp+ }+ insets+ }++ stage.addView(web)+ stage.addView(spinner)+ stage.addView(close)+ }++ /**+ * The page is drawable. Everything here has to happen *inside* the stage's+ * growth, not alongside it — the stage is the whole window, so a spinner+ * that fades on its own clock is a spinner the user sees pop into an+ * already-full-size page.+ */+ override fun onStageRevealing() {+ spinner?.animate()?.alpha(0f)?.setDuration(160)+ ?.withEndAction { spinner?.visibility = android.view.View.GONE }?.start()+ }++ /**+ * The hand-back, and it fires from [onStageLeaving] — while the page is+ * still shrinking — rather than from `onDestroy`. [reportClosed] is what+ * makes that safe to do from both.+ */+ override fun onStageLeaving() {+ spinner?.animate()?.cancel()+ reportClosed()+ }++ override fun onDestroy() {+ // The other half of the pair above: a page the system tore down rather+ // than the user closing it (low memory, an activity-recorder replay)+ // never reaches [leave], and the caller still has to be told. Idempotent+ // by construction — see [reportClosed].+ reportClosed()+ // Note what is *not* here: the file is not deleted. Pages outlive their+ // viewing now so the scrollback can re-open them; eviction is+ // [ChatHistory]'s job, against its own budget. See [showFile].+ htmlFile = null+ pageUp = false+ super.onDestroy()+ }++ /**+ * Tell the caller the page is over, exactly once, whenever that happens.+ *+ * `onClose` is the only channel the caller has: `ui.show_html` uses it to+ * put the ball back and to tell the brain how long the page was up, and+ * [ScreenBodyService.openPageFrom] uses it to unfold the card the user had+ * open. Both were fired from `onDestroy`, which is late — see+ * [StageActivity.onStageLeaving] for why that reads as a flash.+ *+ * [pageUp] is deliberately *not* cleared here. It stays true until the+ * window is actually gone, because the gap between "the user tapped ✕" and+ * "the window is gone" is exactly when a model that asked for two pages in+ * one message would ask for the second one, and the flag is what stops it+ * — [showFile] sets it for the same reason.+ */+ private fun reportClosed() {+ val cb = onClose ?: return+ onClose = null+ runCatching { cb() }+ .onFailure { Log.w(TAG, "page-closed callback failed", it) }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ImeSwitch.kt b/app/src/main/java/net/kuafuai/andee/ui/ImeSwitch.ktnew file mode 100644index 0000000..df8ea5c--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ImeSwitch.kt@@ -0,0 +1,392 @@+package net.kuafuai.andee.ui++import android.content.ComponentName+import android.content.Context+import android.os.Build+import android.provider.Settings+import android.util.Log+import android.view.inputmethod.InputMethodManager++/**+ * The keyboard, and who is holding it.+ *+ * ## The rule+ *+ * **The device's keyboard belongs to the person. Always — except while the+ * brain is in the middle of typing into somebody else's app.**+ *+ * So the resting IME is whatever the user has set for themselves (a real+ * keyboard they can type on anywhere), and [lendToBrain] / [giveBack] bracket+ * each `type_text`: the brain borrows ADBKeyboard for one call and hands it+ * straight back. The window in which nobody can type by hand is then as short+ * as it can possibly be — one tool call.+ *+ * This is the reverse of where the project started, and the reversal is the+ * point. `type_text` has exactly one channel: a broadcast to ADBKeyboard, which+ * must be *the active IME* for the `commitText` to land. ADBKeyboard is+ * headless — no keyboard UI at all — so while it is the default, a human cannot+ * type a single character in any app on the tablet. The original product+ * decision said that was fine ("平板/手机是 Andee 本体, 无真人键盘场景" — see+ * `ScreenController`), and it was, right up until the card grew a ⌨ button and+ * the phone became a phone again.+ *+ * ## Why the swapping is per call and not per turn+ *+ * Fewer switches would be cheaper — a three-field form is six IME transitions+ * this way instead of two. It is still the right trade, because every transition+ * back to a real keyboard can raise that keyboard over the app the brain is+ * driving, and `type_text`'s read-back is a poll against that app's own tree. A+ * long borrow buys a cheaper automation and pays for it in a screen the brain+ * can no longer see. One call is the smallest unit that does not change what+ * the brain is looking at.+ *+ * ## The write can fail silently, which is why it is read back+ *+ * A settings write a ROM does not honour reports nothing at all — no exception,+ * no log. That is the same shape as the bug that cost the project a day on the+ * ADBKeyboard broadcast action names, so nothing here is believed without+ * reading `DEFAULT_INPUT_METHOD` back.+ *+ * Measured on the tablet this was written for (Xiaomi Pad 5, HyperOS): a normal+ * app calling `InputMethodManager.setInputMethod` **and**+ * `switchToNextInputMethod` is refused both times, while a+ * `Settings.Secure.DEFAULT_INPUT_METHOD` *write* goes through and really does+ * rebind the IME (`dumpsys input_method` → `mCurMethodId` changes ~400 ms+ * later). So the settings write is the only layer — both IMM calls are gone+ * from this file rather than left as dead fallbacks, since they also need a+ * window token and there is no window of ours up while the brain types.+ * `WRITE_SECURE_SETTINGS` is what makes the write possible: granted once at+ * provisioning, see the manifest.+ *+ * Note what is deliberately not a layer either: `showInputMethodPicker`. It is+ * the one option no ROM can refuse, because the *user* makes the change — and it+ * is still wrong, because it can move the keyboard somewhere this app cannot+ * move it back from. Failing is better than succeeding at that.+ */+object ImeSwitch {++ private const val TAG = "Ime"++ /**+ * What the brain needs the keyboard to be while it types.+ *+ * Written out rather than derived from "the one that is not the user's":+ * there is exactly one brain IME, and the other IME could be anything.+ */+ const val BRAIN_IME = "com.android.adbkeyboard/.AdbIME"++ /** The package [BRAIN_IME] lives in, derived so the two cannot drift. */+ private val BRAIN_PKG: String = BRAIN_IME.substringBefore('/')++ /**+ * How long to wait after choosing [BRAIN_IME] before broadcasting into it.+ *+ * The setting and the *binding* are not the same event: `ADB_INPUT_TEXT` is+ * received by the IME's own `onCreate`-registered receiver, so an IME that+ * has been selected but not yet bound is an IME nobody is listening in.+ * Measured: the setting reads back immediately, the IME is bound within+ * ~400 ms of the write. Waiting is the only option — the framework exposes+ * no "is the IME bound" to a normal app, and a broadcast sent too early+ * disappears with no error of any kind.+ */+ private const val BIND_SETTLE_MS = 350L++ /**+ * The active IME, as the system records it. Readable without a permission.+ *+ * **Except on a ROM that lies about "readable".** Measured on a HONOR+ * (MagicOS / Android 15): `Settings.Secure` hands this app `null` for the+ * keyboard settings while `adb shell settings get` reads them back fine, so+ * `current` is the empty string and every caller that compares it against an+ * id — [select]'s read-back, [lendToBrain], [ensureHumanDefault] — is+ * suddenly looking at "not the IME you wanted" no matter what is installed.+ * Nothing here can repair that (there is no second public API for "which IME+ * is active"), so it is logged once and left to [enabled], which *can* fall+ * back. See its doc for the half of this that is fixable.+ */+ fun current(context: Context): String {+ val v = runCatching {+ Settings.Secure.getString(+ context.contentResolver,+ Settings.Secure.DEFAULT_INPUT_METHOD,+ )+ }.getOrNull().orEmpty()+ if (v.isEmpty() && !warnedNoCurrent) {+ warnedNoCurrent = true+ Log.w(TAG, "DEFAULT_INPUT_METHOD is unreadable on this ROM — type_text cannot confirm the keyboard")+ }+ return v+ }++ /** One warning per process is enough for a condition that cannot change. */+ @Volatile+ private var warnedNoCurrent = false++ /**+ * Every IME the user has enabled, id to human label, in the order the+ * system cycles them.+ *+ * ### Two sources, because one of them is empty on some ROMs+ *+ * The list lives in `Settings.Secure.ENABLED_INPUT_METHODS`, and reading it+ * is the documented way. It is also **the half that stops working on a+ * HONOR**: measured on MagicOS / Android 15 with two keyboards enabled —+ * Sogou and ADBKeyboard, `settings get secure enabled_input_methods` showing+ * both — this app read back nothing, so [typable] came out empty and the+ * ball's double-tap answered "这台设备上只有 ADBKeyboard" to a phone that had+ * a perfectly good human keyboard on it. The very same call returns the very+ * same string to `adb shell`, and granting this app+ * `WRITE_SECURE_SETTINGS` changed nothing, so it is the ROM's read+ * restriction and not a permission of ours.+ *+ * `InputMethodManager.getEnabledInputMethodList()` is the other way, API 34+ * and up, and it is *not* affected: it reports through the input-method+ * service rather than through the settings provider. So the two are unioned.+ * Neither alone is right — the settings value is the only source below API+ * 34, and it is the one that carries the user's ordering — and the union is+ * safe in both directions because both name the same components.+ *+ * Ids are normalised through [shortId] before the union: `InputMethodInfo+ * .getId()` and a hand-written settings value are the same component written+ * two ways (`pkg/Full.Class` against `pkg/.Class`), and an un-normalised+ * union would list one keyboard twice and — worse — fail to recognise+ * [BRAIN_IME] in the manager's spelling, which is the exact comparison+ * [typable] and [brainEnabled] are built on.+ *+ * Labels come from `getInputMethodList()`, which only lists *installed*+ * IMEs — an id with no label (possible on a ROM that hides one) falls back+ * to the id, which is ugly but true.+ */+ fun enabled(context: Context): List<Pair<String, String>> {+ val fromSettings = settingsEnabled(context)+ val fromManager = managerEnabled(context)+ val ids = (fromSettings + fromManager).distinct()+ if (ids.isEmpty()) return emptyList()+ val labels = runCatching {+ (context.getSystemService(Context.INPUT_METHOD_SERVICE) as InputMethodManager)+ .inputMethodList+ .associate { shortId(it.id) to it.loadLabel(context.packageManager).toString() }+ }.getOrDefault(emptyMap())+ return ids.map { it to (labels[it] ?: it) }+ }++ /**+ * The documented source, and the one that can come back empty — see [enabled].+ *+ * An entry may carry subtypes after a ';'. The IME's own id is the first+ * field, and the id is what setInputMethod wants.+ */+ private fun settingsEnabled(context: Context): List<String> {+ val raw = runCatching {+ Settings.Secure.getString(+ context.contentResolver,+ Settings.Secure.ENABLED_INPUT_METHODS,+ )+ }.getOrNull().orEmpty()+ val ids = raw.split(':')+ .map { it.trim() }+ .filter { it.isNotEmpty() }+ .map { shortId(it.substringBefore(';')) }+ .distinct()+ if (ids.isEmpty()) {+ // Loud on purpose: this is the state that makes the assistant tell a+ // user with a working keyboard that they have none. "The ROM gave us+ // nothing" and "nothing is enabled" are the same empty list here, so+ // the raw value is the only thing that tells them apart.+ Log.w(TAG, "ENABLED_INPUT_METHODS unreadable (raw=\"$raw\") — using InputMethodManager only")+ }+ return ids+ }++ /**+ * API 34's own answer, which survives the ROM above — see [enabled].+ *+ * Empty below API 34 rather than approximated: this method was added in 34,+ * and the settings value is already the whole list on every older device+ * this app ships to (minSdk 30).+ */+ private fun managerEnabled(context: Context): List<String> {+ if (Build.VERSION.SDK_INT < Build.VERSION_CODES.UPSIDE_DOWN_CAKE) return emptyList()+ return runCatching {+ (context.getSystemService(Context.INPUT_METHOD_SERVICE) as InputMethodManager)+ .enabledInputMethodList+ .map { shortId(it.id) }+ }.getOrDefault(emptyList())+ }++ /**+ * One component, one spelling.+ *+ * `InputMethodInfo.getId()` and the settings value describe the same IME as+ * `pkg/Full.Class` and `pkg/.Class` respectively, so both are flattened+ * through `ComponentName` before anything compares them.+ */+ private fun shortId(id: String): String =+ runCatching { ComponentName.unflattenFromString(id)?.flattenToShortString() }+ .getOrNull() ?: id++ /**+ * The IMEs a person could actually type on: everything enabled except the+ * headless one.+ *+ * Empty is a real state worth saying out loud rather than a bug to code+ * around — a device whose only IME is ADBKeyboard cannot be typed on by+ * hand anywhere, and no amount of swapping changes that.+ */+ fun typable(context: Context): List<Pair<String, String>> =+ enabled(context).filterNot { it.first == BRAIN_IME }++ /** The human name of an IME, for a sentence the user has to act on. */+ fun label(context: Context, id: String): String =+ enabled(context).firstOrNull { it.first == id }?.second ?: id++ /**+ * Whether an IME from [BRAIN_IME]'s package is installed at all.+ *+ * Asked through `getInputMethodList()` and not+ * `PackageManager.getPackageInfo`: "the package exists" and "there is a+ * keyboard in it" are the same answer for this app, and the second is the+ * true one. An APK that installs but declares no IME service — a truncated+ * download, the wrong file — would otherwise read as present, and the row+ * built on this would send the user to a keyboard list that cannot contain+ * it. `packageName` rather than the exact component, because the broadcast+ * is addressed to the package: a build that renamed its service would still+ * receive `ADB_INPUT_TEXT`.+ *+ * Used by `device/SelfCheck` — see the row there for what is decided from+ * this and [brainEnabled] together. It only works because the manifest+ * declares `QUERY_ALL_PACKAGES`: on API 30+ `getInputMethodList()` is+ * filtered by package visibility like any other query, so without that+ * permission an installed keyboard would read as absent and the row would+ * send the user to download something they already have.+ */+ fun brainInstalled(context: Context): Boolean = runCatching {+ (context.getSystemService(Context.INPUT_METHOD_SERVICE) as InputMethodManager)+ .inputMethodList+ .any { it.packageName == BRAIN_PKG }+ }.getOrDefault(false)++ /**+ * Whether the brain's IME is one the user has enabled.+ *+ * The **exact** id, not the package, and the difference is load-bearing:+ * [select] and [lendToBrain] name `com.android.adbkeyboard/.AdbIME`+ * literally, so a build whose service has another name is a build this app+ * cannot switch to. Testing the package here would show a green row beside a+ * `type_text` that cannot land — which is the one thing a check must never+ * do. Red in that case is the truth.+ */+ fun brainEnabled(context: Context): Boolean = enabled(context).any { it.first == BRAIN_IME }++ /**+ * Whether this app is allowed to write the setting directly.+ *+ * Granted once at provisioning (`pm grant`), never at install — the+ * permission is `signature|privileged|development`. Without it the app+ * cannot move the keyboard in either direction, which on MIUI means it+ * cannot type into another app at all; that has to be a loud failure, not a+ * silent one.+ */+ fun canSwitch(context: Context): Boolean =+ context.checkSelfPermission(android.Manifest.permission.WRITE_SECURE_SETTINGS) ==+ android.content.pm.PackageManager.PERMISSION_GRANTED++ // ------------------------------------------------------------------+ // Borrowing+ // ------------------------------------------------------------------++ /**+ * Take the keyboard for the brain, and return the id that gives it back.+ *+ * `null` means "there was nothing to borrow" — the device was already on+ * [BRAIN_IME], so the caller has nothing to hand back. It does **not** mean+ * the borrow succeeded: the caller must check [current] afterwards, because+ * a refused switch is the failure that matters (the broadcast would go+ * nowhere).+ *+ * Blocks for [BIND_SETTLE_MS] on the way in. Callers are tool threads, never+ * the main one.+ */+ fun lendToBrain(context: Context): String? {+ val was = current(context)+ if (was == BRAIN_IME) return null+ if (!select(context, BRAIN_IME)) return null+ Thread.sleep(BIND_SETTLE_MS)+ return was.ifEmpty { null }+ }++ /**+ * Hand it back. Restores whatever was active when it was borrowed rather+ * than naming a keyboard: the user's choice is theirs, and if they have+ * changed it since, that is the one to go back to. A null [to] — the+ * "nothing was borrowed" case — is a no-op.+ */+ fun giveBack(context: Context, to: String?) {+ if (to.isNullOrEmpty()) return+ select(context, to)+ }++ /**+ * Start-up repair: if the device is sitting on the headless IME as its+ * *default*, put it back on a real keyboard.+ *+ * Only reachable on a device that was set up under the old rule (or by+ * hand). Nobody chooses ADBKeyboard deliberately — it has no keys — and+ * leaving it there means the person cannot type anywhere, which is exactly+ * the bug this class was rewritten to remove. Silent and safe the rest of+ * the time: it does nothing unless the current IME is the brain's *and*+ * there is a real keyboard to move to.+ *+ * @return the id it settled on, or null if it left things alone.+ */+ fun ensureHumanDefault(context: Context): String? {+ if (current(context) != BRAIN_IME) return null+ val human = typable(context).firstOrNull() ?: return null+ if (!canSwitch(context)) {+ Log.w(TAG, "default IME is the headless one and we may not change it")+ return null+ }+ Log.i(TAG, "默认输入法是 ADBKeyboard(它没有键盘界面)→ 换回 ${human.second}")+ return if (select(context, human.first)) human.first else null+ }++ /**+ * Put the device on [want], and report whether it actually is.+ *+ * One layer, because there is one that works and the others cannot even be+ * attempted: `InputMethodManager.setInputMethod` / `switchToNextInputMethod`+ * need a window token belonging to this app, and the brain types while no+ * window of ours is up (the ⌨ field is a separate Activity and is closed by+ * then). They were also both refused by this ROM when they *were* reachable+ * — silently, which is the whole reason this reads the setting back.+ *+ * `false` is not "nothing happened, carry on": it means the keyboard is+ * still on the wrong IME, and a caller about to broadcast into ADBKeyboard+ * must not.+ */+ fun select(context: Context, want: String): Boolean {+ if (want.isEmpty()) return false+ // Already there. This is the restore path's common case in reverse: the+ // device sits on the user's own IME, so putting it back is usually a+ // no-op rather than a second switch.+ if (current(context) == want) return true+ if (!canSwitch(context)) {+ Log.w(TAG, "no WRITE_SECURE_SETTINGS — cannot move the keyboard to $want")+ return false+ }+ runCatching {+ Settings.Secure.putString(+ context.contentResolver,+ Settings.Secure.DEFAULT_INPUT_METHOD,+ want,+ )+ }.onFailure { Log.w(TAG, "settings write refused", it) }+ // Read back. A refused write reports nothing at all, and "it worked"+ // assumed from a silent call is how the broadcast-action-name bug+ // survived as long as it did.+ val took = current(context) == want+ if (took) Log.i(TAG, "→ $want (secure settings)")+ else Log.w(TAG, "keyboard refused to move to $want")+ return took+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/LangToggle.kt b/app/src/main/java/net/kuafuai/andee/ui/LangToggle.ktnew file mode 100644index 0000000..5f8b3be--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/LangToggle.kt@@ -0,0 +1,138 @@+package net.kuafuai.andee.ui++import android.content.Context+import android.graphics.Color+import android.graphics.Typeface+import android.util.TypedValue+import android.view.Gravity+import android.widget.FrameLayout+import android.widget.LinearLayout+import android.widget.TextView+import net.kuafuai.andee.config.VoiceConfig++/**+ * 中 | EN — the two-cell language picker, for surfaces that are not the+ * settings card.+ *+ * ### Why this exists next to `SettingsUi.langPicker`+ *+ * That picker is a `private fun segments(...)` bound to `SettingsUi`'s own+ * field, so it cannot be reused from anywhere else; this is the small+ * standalone version for the self-check page. The two are **not** two+ * implementations of one decision — both call+ * [VoiceConfig.save]`(context, mapOf("lang" to …))`, which is the single write+ * path for the preference. Nothing here reads or writes `lang` any other way.+ *+ * The visual is deliberately the settings card's: a recessed track+ * ([Glass.well]) with the picked cell raised on it ([Glass.thumb]). It keeps+ * the whole `segments` feel without dragging over the parts that are about+ * *animated* selection — this control is two cells in a corner and the pick is+ * confirmed by the page redrawing in the new language, so a sliding thumb+ * would be motion for its own sake.+ *+ * ### The labels are not in resources, and that is the point+ *+ * `中` stays `中` and `EN` stays `EN` whatever the card is speaking. A language+ * picker labels each language *in its own script*; translating these is the one+ * way to make the control useless — the same rule (and the same two strings)+ * as `SettingsUi.langPicker`.+ *+ * ### Persistence+ *+ * [VoiceConfig.save] writes `SharedPreferences("voice_prefs")`, which is a real+ * file (`/data/data/<pkg>/shared_prefs/voice_prefs.xml`) and not an in-memory+ * field, and [VoiceConfig.uiLanguage] prefers the saved value over the device+ * locale forever once it is set. So the choice survives a restart by+ * construction, and this class does not need to cache anything to make that+ * true.+ *+ * @param onPicked fired **after** the write, so a caller that redraws from+ * [net.kuafuai.andee.i18n.AppLocale] is guaranteed to read the new value.+ */+class LangToggle(+ context: Context,+ private val onPicked: (String) -> Unit,+) : FrameLayout(context) {++ private companion object {+ /** The two tags this app ships, and the only two strings in it that must not be translated. */+ val OPTIONS = listOf(VoiceConfig.LANG_ZH to "中", VoiceConfig.LANG_EN to "EN")++ const val INSET_DP = 3+ const val RADIUS_DP = 16++ /**+ * Keeps the two cells the same width.+ *+ * They are not naturally: `中` is one CJK glyph and `EN` is two Latin+ * ones, so with `WRAP_CONTENT` alone the picked pane visibly changes+ * size when you switch, which reads as the control *resizing* rather+ * than as a selection moving. Weights cannot fix it either — a+ * `layout_weight` needs `width = 0`, and a weighted child in a+ * `WRAP_CONTENT` parent has nothing to distribute. A floor under both+ * cells is the version that works with a wrapping track.+ */+ const val CELL_MIN_W_DP = 46+ }++ private val cells = mutableListOf<TextView>()+ private var current = VoiceConfig.uiLanguage(context)++ init {+ val inset = context.dpI(INSET_DP)+ setPadding(inset, inset, inset, inset)+ background = Glass.well(context, context.dpI(RADIUS_DP))++ val row = LinearLayout(context).apply { orientation = LinearLayout.HORIZONTAL }+ val on = Color.parseColor(Glass.TITLE)+ val off = Color.parseColor(Glass.SECONDARY)++ OPTIONS.forEach { (tag, label) ->+ val cell = TextView(context).apply {+ text = label+ textSize = 14f+ setTypeface(typeface, Typeface.BOLD)+ gravity = Gravity.CENTER+ minWidth = context.dpI(CELL_MIN_W_DP)+ setPadding(context.dpI(14), context.dpI(6), context.dpI(14), context.dpI(6))+ isClickable = true+ setOnClickListener { pick(tag) }+ }+ cells.add(cell)+ row.addView(cell)+ }+ addView(row, LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT))+ paint(on, off)+ }++ /**+ * Write the preference, then tell the caller.+ *+ * No-ops on the tag already in effect: the page redraws on every pick, and+ * redrawing for a tap that changed nothing costs a flash of the whole list+ * for no reason. Note this compares against [current], which starts from+ * storage rather than from the label — so a device whose language has never+ * been chosen (falling through to the system locale, see+ * [VoiceConfig.uiLanguage]) still highlights the right cell.+ */+ private fun pick(tag: String) {+ if (tag == current) return+ VoiceConfig.save(context, mapOf("lang" to tag))+ current = tag+ paint(Color.parseColor(Glass.TITLE), Color.parseColor(Glass.SECONDARY))+ onPicked(tag)+ }++ private fun paint(on: Int, off: Int) {+ val thumbRadius = context.dpI(RADIUS_DP - INSET_DP)+ cells.forEachIndexed { i, cell ->+ val on_ = OPTIONS[i].first == current+ cell.setTextColor(if (on_) on else off)+ // The picked cell carries the raised pane rather than a separate+ // sliding layer — same look, and no layout listener to keep in sync+ // with the cells it measures (see SettingsUi.segments for what that+ // costs when it goes wrong).+ cell.background = if (on_) Glass.thumb(context, thumbRadius) else null+ }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/MenuUi.kt b/app/src/main/java/net/kuafuai/andee/ui/MenuUi.ktnew file mode 100644index 0000000..a101734--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/MenuUi.kt@@ -0,0 +1,97 @@+package net.kuafuai.andee.ui++import android.annotation.SuppressLint+import android.content.Context+import android.graphics.Color+import android.graphics.PixelFormat+import android.graphics.drawable.GradientDrawable+import android.os.Handler+import android.os.Looper+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import android.widget.LinearLayout+import android.widget.TextView++/**+ * Long-press menu overlay. A rounded white card with clickable items.+ * Auto-dismisses after [AUTO_DISMISS_MS]. Position anchored by the caller.+ */+class MenuUi(private val context: Context) {++ data class Item(val label: String, val onClick: () -> Unit)++ private val wm: WindowManager =+ context.getSystemService(Context.WINDOW_SERVICE) as WindowManager+ private val ui = Handler(Looper.getMainLooper())+ private var view: View? = null+ private val autoDismiss = Runnable { dismiss() }++ fun isShowing(): Boolean = view != null++ fun show(items: List<Item>, anchorX: Int, anchorY: Int) {+ if (view != null) return+ val container = buildView(items)+ val params = WindowManager.LayoutParams(+ WindowManager.LayoutParams.WRAP_CONTENT,+ WindowManager.LayoutParams.WRAP_CONTENT,+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE+ or WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.TOP or Gravity.START+ x = anchorX; y = anchorY+ }+ try {+ wm.addView(container, params)+ view = container+ ui.postDelayed(autoDismiss, AUTO_DISMISS_MS)+ } catch (_: Throwable) {+ // ignore+ }+ }++ fun dismiss() {+ ui.removeCallbacks(autoDismiss)+ val v = view ?: return+ runCatching { wm.removeView(v) }+ view = null+ }++ private fun buildView(items: List<Item>): View {+ val bg = GradientDrawable().apply {+ shape = GradientDrawable.RECTANGLE+ setColor(Color.parseColor("#FFFFFF"))+ cornerRadius = dp(10).toFloat()+ setStroke(dp(1), Color.parseColor("#E5E7EB"))+ }+ val container = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ background = bg+ setPadding(dp(4), dp(4), dp(4), dp(4))+ }+ for (item in items) container.addView(makeItem(item))+ return container+ }++ @SuppressLint("ClickableViewAccessibility")+ private fun makeItem(item: Item): View =+ TextView(context).apply {+ text = item.label+ setPadding(dp(20), dp(12), dp(20), dp(12))+ setTextColor(Color.parseColor("#111827"))+ textSize = 14f+ isClickable = true+ isFocusable = true+ minWidth = dp(160)+ setOnClickListener { item.onClick() }+ }++ private fun dp(v: Int): Int =+ (v * context.resources.displayMetrics.density).toInt()++ companion object {+ private const val AUTO_DISMISS_MS = 5_000L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/OverlayA11y.kt b/app/src/main/java/net/kuafuai/andee/ui/OverlayA11y.ktnew file mode 100644index 0000000..379ca1f--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/OverlayA11y.kt@@ -0,0 +1,107 @@+package net.kuafuai.andee.ui++import android.os.Build+import android.view.View+import java.util.concurrent.atomic.AtomicInteger++/**+ * Take an overlay's whole view tree out of the accessibility node tree.+ *+ * ## The bug this exists for+ *+ * Every window this app adds — the card, the settings sheet, the tap marker,+ * the edge glow — is a real window with real `View`s, and by default every one+ * of them is exposed to accessibility exactly like an app's own UI. We are an+ * `AccessibilityService`, so we read our own furniture back:+ *+ * - [net.kuafuai.andee.screen.ScreenController.tryOtherWindows] sweeps every+ * window looking for one with content. It runs *precisely* when the real+ * pane has none — a mini-program, a WebView, a game canvas — and on those+ * screens the only tree with `useful > 0` on the whole display is our own+ * toolbar. It wins, and the brain is handed an element list describing our+ * buttons instead of the app it is supposed to be driving.+ * - `rootInActiveWindow` is us whenever a focusable overlay is up, which makes+ * `typeText`'s `findFocus(FOCUS_INPUT)` return the settings sheet's own+ * `EditText` — the brain's next sentence gets typed into the ASR endpoint+ * field.+ *+ * Note what is *not* broken: the brain cannot actually touch the ball.+ * `CommandDispatcher.passthroughForGesture` drops every overlay to+ * non-touchable before dispatching, so an injected tap always reaches the app+ * underneath. The damage is upstream of the touch — the brain aims at a+ * control that is ours, and the tap lands on whatever real content happens to+ * sit under it. Which is why the fix belongs here, at the point where the+ * elements become visible to it, and *not* in a geometric "refuse taps inside+ * the ball's rectangle" rule: app content under the ball is legitimately+ * tappable and passthrough makes tapping it work, so such a rule would reject+ * correct taps.+ *+ * `NO_HIDE_DESCENDANTS` rather than `NO`: plain `NO` hides only this view and+ * reports its children in its place, which for a root view removes nothing at+ * all.+ *+ * ## Why that flag alone does not work *here*+ *+ * `accessibility_service_config.xml` asks for `flagIncludeNotImportantViews`,+ * and that flag makes `View.includeForAccessibility()` return true regardless+ * of `importantForAccessibility`. So `NO_HIDE_DESCENDANTS` hides our overlays+ * from every service *except the one that needed it* — precisely backwards.+ * Measured: with the vault form open, `get_ui_tree` returned the form,+ * labels, phone number and all.+ *+ * `accessibilityDataSensitive` is the flag that survives it. It hides the+ * subtree from services that do not declare `android:isAccessibilityTool`,+ * which we do not and TalkBack does — so it is also strictly *better* than the+ * blunt instrument above: the screen reader keeps the ball, we lose it. It+ * inherits down the tree, so one call on the root still covers the window.+ * API 34+, hence the guard; below that we are back to the old+ * `tryOtherWindows`/`treeEmpty` guards in `ScreenController`, which is what+ * shipped before and is merely imperfect rather than broken.+ *+ * ## The cost, stated plainly+ *+ * On a pre-34 device this still hides the overlay from *every* accessibility+ * service — a screen reader can no longer reach the ball or the settings+ * sheet. That is a real accessibility regression, accepted because this is a+ * single-purpose appliance whose own UI is a ball and six buttons, and because+ * the alternative is the brain misreading the screen on exactly the panes+ * where it is already weakest.+ *+ * Call once on the root view, before or after `addView` — the flags are read+ * at node-construction time, not at attach.+ */+fun View.hideFromAccessibility() {+ importantForAccessibility = View.IMPORTANT_FOR_ACCESSIBILITY_NO_HIDE_DESCENDANTS+ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {+ setAccessibilityDataSensitive(View.ACCESSIBILITY_DATA_SENSITIVE_YES)+ }+}++/**+ * Whether one of the body's own **focusable** cards — [SettingsUi], [VaultUi] —+ * is currently in front.+ *+ * Hiding those from accessibility is correct, but it leaves the dump looking+ * exactly like a dead accessibility connection: our card is the active window+ * and it now reports zero nodes, which is the same signature+ * `ScreenController.reviveNote` reads as "the service went stale, have the user+ * toggle Andee off and on". That message is aimed at a human, so it has to be+ * true — same rule as the meeting recorder's `stopped_by`. The cards therefore+ * announce themselves and the dump says the real reason instead.+ *+ * Counted rather than boolean because the vault opens on top of settings.+ * `show()`/`hide()` in both classes are already idempotent, so the pairs balance.+ */+object OwnCard {+ private val depth = AtomicInteger(0)++ val showing: Boolean get() = depth.get() > 0++ fun shown() {+ depth.incrementAndGet()+ }++ fun hidden() {+ depth.updateAndGet { if (it > 0) it - 1 else 0 }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/PillUi.kt b/app/src/main/java/net/kuafuai/andee/ui/PillUi.ktnew file mode 100644index 0000000..bc2ef9b--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/PillUi.kt@@ -0,0 +1,110 @@+package net.kuafuai.andee.ui++import android.content.Context+import android.graphics.Color+import android.graphics.PixelFormat+import android.graphics.drawable.GradientDrawable+import android.os.Handler+import android.os.Looper+import android.text.TextUtils+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import android.widget.LinearLayout+import android.widget.TextView++/**+ * Top-center transcript pill. Lazily attached on first [show]; stays visible+ * until [hide] is called (optionally after a delay).+ */+class PillUi(private val context: Context) {++ enum class Kind { LISTENING, FINAL, SPEAKING, ERROR }++ private val wm: WindowManager =+ context.getSystemService(Context.WINDOW_SERVICE) as WindowManager+ private val ui = Handler(Looper.getMainLooper())+ private var view: View? = null+ private var text: TextView? = null+ private val hideRunnable = Runnable { hideNow() }++ fun show(msg: String, kind: Kind) {+ ui.post {+ ui.removeCallbacks(hideRunnable)+ ensurePill()+ text?.text = msg+ text?.setTextColor(textColor(kind))+ (view?.background as? GradientDrawable)?.setStroke(dp(2), strokeColor(kind))+ }+ }++ fun hide(delayMs: Long = 0) {+ if (delayMs <= 0) ui.post { hideNow() } else ui.postDelayed(hideRunnable, delayMs)+ }++ private fun hideNow() {+ val v = view ?: return+ runCatching { wm.removeView(v) }+ view = null+ text = null+ }++ private fun ensurePill() {+ if (view != null) return+ val tv = TextView(context).apply {+ setPadding(dp(20), dp(12), dp(20), dp(12))+ textSize = 18f+ maxLines = 3+ ellipsize = TextUtils.TruncateAt.END+ setTextColor(textColor(Kind.LISTENING))+ }+ val bg = GradientDrawable().apply {+ shape = GradientDrawable.RECTANGLE+ setColor(Color.parseColor("#EE0F172A"))+ cornerRadius = dp(20).toFloat()+ setStroke(dp(2), strokeColor(Kind.LISTENING))+ }+ val container = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ background = bg+ addView(tv)+ }+ val params = WindowManager.LayoutParams(+ WindowManager.LayoutParams.WRAP_CONTENT,+ WindowManager.LayoutParams.WRAP_CONTENT,+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE+ or WindowManager.LayoutParams.FLAG_NOT_TOUCHABLE+ or WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.TOP or Gravity.CENTER_HORIZONTAL+ y = dp(40)+ width = context.resources.displayMetrics.widthPixels - dp(80)+ }+ try {+ wm.addView(container, params)+ view = container+ text = tv+ } catch (_: Throwable) {+ // ignore+ }+ }++ private fun textColor(kind: Kind): Int = when (kind) {+ Kind.LISTENING -> Color.parseColor("#CBD5E1")+ Kind.FINAL -> Color.parseColor("#FBBF24")+ Kind.SPEAKING -> Color.parseColor("#67E8F9")+ Kind.ERROR -> Color.parseColor("#F87171")+ }++ private fun strokeColor(kind: Kind): Int = when (kind) {+ Kind.LISTENING -> Color.parseColor("#DC2626")+ Kind.FINAL -> Color.parseColor("#F59E0B")+ Kind.SPEAKING -> Color.parseColor("#06B6D4")+ Kind.ERROR -> Color.parseColor("#DC2626")+ }++ private fun dp(v: Int): Int =+ (v * context.resources.displayMetrics.density).toInt()+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/SecretInput.kt b/app/src/main/java/net/kuafuai/andee/ui/SecretInput.ktnew file mode 100644index 0000000..11cf0db--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/SecretInput.kt@@ -0,0 +1,66 @@+package net.kuafuai.andee.ui++import android.text.InputType+import android.text.method.PasswordTransformationMethod+import android.widget.EditText++/**+ * The dots on the secret fields of our own cards — drawn by us, and never by+ * telling the IME that the field is a password.+ *+ * ### Why a password `inputType` cannot be used here+ *+ * Measured on Honor MagicOS, and the symptom is that the field simply cannot+ * be typed into: tap the DeepSeek Key row in [SettingsUi], or a password or+ * card number in [VaultUi], and the keyboard appears for one frame and+ * retracts. It is not a focus bug — `dumpsys input_method` shows `startInput`+ * arriving with the right `inputType` and the input connection bound — and it+ * is not ours to fix from inside the card.+ *+ * The chain is: a password `inputType` makes the ROM hand the field to its own+ * secure keyboard (`com.hihonor.secime`) instead of the user's; that window+ * carries `PRIVATE_FLAG_HIDE_NON_SYSTEM_OVERLAY_WINDOWS`, which hides **every**+ * `TYPE_APPLICATION_OVERLAY` on screen for as long as it is up. Our cards are+ * overlays, so the window it hides includes the one holding the focused field.+ * The IME loses its target the instant it arrives and is dismissed again.+ *+ * That flag is an anti-phishing measure and it is working exactly as intended+ * — a password prompt should not be typed into with unknown windows floating+ * over it. There is nothing to negotiate with: the flag belongs to their+ * window, and the only move available to us is to not ask for the secure+ * keyboard in the first place.+ *+ * ### What we do instead+ *+ * The `inputType` stays ordinary and the masking becomes a+ * [PasswordTransformationMethod], which is a *display* concern and invisible+ * to the IME. [TYPE_TEXT_FLAG_NO_SUGGESTIONS][InputType.TYPE_TEXT_FLAG_NO_SUGGESTIONS]+ * is what replaces the part of the password variation that was worth having:+ * it is the standing request that the keyboard neither predict from nor learn+ * what is typed here. Autofill is not in play on these cards, so nothing else+ * was riding on the variation.+ *+ * One thing improves on the way past. The 显示/隐藏 toggle used to swap the+ * `inputType` between a hidden and a shown variant, which on the numeric fields+ * meant swapping the keyboard under the user's thumb mid-entry. Only the+ * transformation moves now, so the keyboard the user started typing on is the+ * one they finish on.+ */+internal fun secretType(numeric: Boolean): Int = if (numeric) {+ InputType.TYPE_CLASS_NUMBER+} else {+ InputType.TYPE_CLASS_TEXT or InputType.TYPE_TEXT_FLAG_NO_SUGGESTIONS+}++/**+ * Show or hide the dots.+ *+ * Call it **after** `isSingleLine`, which installs a transformation of its own+ * and would otherwise strip this one straight back off. Losing that one costs+ * nothing: neither type from [secretType] carries `TYPE_TEXT_FLAG_MULTI_LINE`,+ * so there is no newline for it to fold away.+ */+internal fun EditText.setMasked(masked: Boolean) {+ transformationMethod =+ if (masked) PasswordTransformationMethod.getInstance() else null+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/SelfCheckActivity.kt b/app/src/main/java/net/kuafuai/andee/ui/SelfCheckActivity.ktnew file mode 100644index 0000000..82fe65e--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/SelfCheckActivity.kt@@ -0,0 +1,271 @@+package net.kuafuai.andee.ui++import android.graphics.drawable.ColorDrawable+import android.os.Bundle+import android.widget.ImageView+import net.kuafuai.andee.ScreenBodyService+import net.kuafuai.andee.config.VoiceConfig+import net.kuafuai.andee.i18n.AppLocale++/**+ * The one screen in this app a user can reach while the app is not working.+ *+ * ### What this class used to be, and what it is now+ *+ * The checklist was a fullscreen page: this Activity folded the assistant away,+ * perked the ball, and drew the rows over its own blurred backdrop. It is now a+ * **card**, opened from `✓` in the assistant's control bar and rendered by+ * [SelfCheckUi] **inside the assistant's own window** — see that class for why+ * two buttons sitting next to each other have to open the same kind of surface.+ *+ * So on a working device this Activity is never *seen*: the icon starts it, it+ * hands the request to the service and finishes — see the next section. All the+ * drawing it used to do lives in [SelfCheckUi] now, and what is left here is the+ * case the card cannot cover.+ *+ * ### Why it still has to exist, and why it still has to be in the launcher+ *+ * Everything else here is an overlay owned by [ScreenBodyService], which is an+ * accessibility service — so if accessibility is off there is no ball, no card,+ * no overlay at all, and therefore **no way into the app**. That was a+ * deliberate design ("no Activity and no launcher UI", see `CLAUDE.md`) and it+ * has one consequence nobody wants: the two most common first-run problems —+ * accessibility not enabled, overlay permission not granted — left the user+ * with a device that does nothing and nowhere to find out why.+ *+ * This page is the way out of that. It is a plain Activity, so it runs before+ * the service does, which makes it the only surface where+ * [net.kuafuai.andee.device.SelfCheck.Level.FAIL] on `accessibility` can be+ * *read* rather than just *believed*. That is also why it is the launcher entry+ * point: a door you can only reach from inside the broken room is not a door.+ *+ * ### The icon opens two doors, and the service decides which+ *+ * Being the launcher entry point does not make this page what the icon *means*.+ * With the service up, tapping the icon does what a long-press on the ball does+ * — `dispatcher.expand()` — and this Activity finishes without drawing a frame;+ * on a working device it is therefore still never *seen*, only started and+ * dropped. With the service down it falls through to the checklist, because+ * there is nothing to unfold and this is the only surface that can say why.+ *+ * One icon, two behaviours, and the branch is not a heuristic: [onCreate] asks+ * whether the ball is actually on screen — and asks in the one way that also+ * puts it there if the only thing wrong was a grant that arrived too late (see+ * `ScreenBodyService.ensureOverlays`). That is also the load-bearing half of the+ * original argument, unchanged — the launcher entry exists so that a device+ * whose assistant is not running still has somewhere to go. All that changed is+ * that a device whose assistant *is* running no longer pays for it by being+ * taken to a diagnostic page every time the user taps the app.+ *+ * See `Theme.Body.Stage.NoPreview` for the other half of making the first door+ * invisible: without it the platform paints a black starting window for a page+ * that is about to finish, and "tap the icon, then the card grows" reads as+ * "tap the icon, the screen goes black, then the card grows".+ *+ * ### A host, not a second implementation+ *+ * All this class does now is give [SelfCheckUi] a `stage` to put the card in+ * instead of a window to make. That is the whole reason the look cannot drift:+ * there is one builder, and it is the same one the working device sees. The two+ * hosts differ in exactly two things — where the card goes, and whether we may+ * ask the compositor to blur — and both of those are [SelfCheckUi]'s to decide.+ *+ * ### The backdrop is what makes the card's fill honest+ *+ * [SelfCheckUi] hands the card the *frosted* fill unconditionally when it is+ * hosted, on the reasoning that the blur behind it is real and it did not have+ * to ask for it. That is only true because of [installBackdrop]: without it,+ * "frosted" is 60% black over this Activity's black window, which is a black+ * rectangle. Same blurred image and the same *derived* scrim the assistant's+ * card brings up, via the same `CENTER_CROP` recipe [TextInputActivity] uses —+ * see [Backdrop] for why "same object, same scrim" beats "a similar dark+ * colour".+ *+ * ### Language+ *+ * [SelfCheckUi] carries the picker and writes the preference; this class only+ * has to notice a change made *somewhere else*. There is exactly one such path+ * and it is reachable from this card: the 打开设置 row opens the assistant's+ * settings sheet, and *its* picker writes the same key. [LangToggle]'s own+ * callback has already told the service by the time the sheet closes; this is+ * the other direction, and [onResume] is where it is caught.+ *+ * Note this page is no longer the thing that paints the rows, so the old+ * `lctx`-as-`var` rule now lives in [SelfCheckUi]. [paintedLang] survives because+ * the *comparison* it makes is still the only way to notice the other path, not+ * because this class draws anything with it.+ */+class SelfCheckActivity : StageActivity() {++ /**+ * The card. Built here on the service-off path only, placed on [stage], and+ * gone with the Activity.+ *+ * `lateinit` rather than nullable because on this path it is assigned in+ * [onCreate] before anything can reach it. Both the callbacks that *could*+ * arrive first — [onResume] and [onDestroy] — check `isInitialized`, because+ * the other path through [onCreate] returns before this is ever assigned and+ * a lifecycle callback is not a good moment to throw.+ */+ private lateinit var ui: SelfCheckUi++ /**+ * The fullscreen backdrop. Built once and **never rebuilt** — see+ * [installBackdrop].+ */+ private var backdropView: ImageView? = null++ /**+ * The language this page is currently painted in, to notice a change made+ * somewhere else. See [onResume].+ */+ private var paintedLang = ""++ override fun onCreate(savedInstanceState: Bundle?) {+ super.onCreate(savedInstanceState)++ // A working assistant ⇒ this icon means what every other icon on the+ // home screen means: open the app. That is the *same request* a+ // long-press on the ball makes, so it is the same call — one entry point+ // cannot be allowed to mean "show me the assistant" and the other+ // "show me a checklist", since which one the user gets would then depend+ // on which door happened to be on screen. `expand()` is idempotent, so+ // tapping the icon with the card already full is a no-op, exactly as it+ // is on the ball.+ //+ // Then `finish()` and nothing else: this Activity has no window worth+ // showing, and `Theme.Body.Stage.NoPreview` is what stops the platform+ // painting a black starting window for a page that is about to vanish.+ //+ // `ballWindow()` and not merely the instance, because a service that is+ // alive but has not attached its overlay yet is not something to unfold+ // — same guard `openSettingsForUser` uses for the same reason.+ //+ // And `ensureOverlays()` rather than a bare `isShown()` test, because+ // the interesting case is the one where the answer is *no but fixable*:+ // the service connected while the overlay grant was missing (a fresh+ // install, a package rename), `addView` was refused, and the grant+ // arrived afterwards. Asking the question is also the retry, so the icon+ // is the way back rather than a door that reports the room is empty —+ // see `ScreenBodyService.ensureOverlays`. When it is still not fixable+ // we fall through to the checklist, which is where the red overlay row+ // is.+ val svc = ScreenBodyService.get()+ if (svc != null && svc.ensureOverlays()) {+ svc.dispatcherForNotification()?.expand()+ finish()+ return+ }++ // ---- Everything below is the case this class exists for ----+ //+ // No service: accessibility is off, or the overlay grant is missing, or+ // the app was just reinstalled. There is no ball, no card and no window+ // to fold, so the checklist is not a fallback here — it is the only+ // thing that can tell the user why the equipment in their hand does+ // nothing. The ball is therefore *not* perked (there is no ball) and the+ // backdrop is not decoration: it is what makes the card's frosted fill+ // honest, see [installBackdrop].+ paintedLang = VoiceConfig.uiLanguage(this)+ installBackdrop()+ ui = SelfCheckUi(+ context = this,+ host = stage,+ onDismiss = { leave() },+ // The card's picker writes the preference and repaints itself, but+ // this Activity is what *remembers* the language on screen —+ // otherwise the next [onResume] would read a mismatch it caused+ // itself and swap the whole card for nothing. Both callbacks+ // [SelfCheckUi] offers exist to tell the *service* something (open+ // our sheet, repaint the ball), and the service is exactly what is+ // missing on this path, which is what leaves this one free.+ onLocaleChanged = { paintedLang = VoiceConfig.uiLanguage(this) },+ )+ // Local half first and it is not an optimisation: `local` is what the+ // card's first painted frame is made of. See [SelfCheckUi.start].+ ui.show()+ ui.start()+ revealStage()+ }++ /** Give the ball back. Nothing to unfold: see [onCreate]. */+ override fun onStageLeaving() {+ ScreenBodyService.get()?.ballWindow()?.setPagePerched(false)+ }++ /**+ * Catch a language change that did not come from this card's picker.+ *+ * This Activity paints nothing itself any more, so what it keeps+ * [paintedLang] for is the *comparison*: it is the language the card was+ * last drawn in, and a mismatch on resume means the preference moved behind+ * the card's back. The card's own picker is not that case — it repaints+ * itself and hands the new value over through `onLocaleChanged` — so what+ * lands here is a change made somewhere this page cannot see.+ *+ * Narrow today and kept on purpose: the 打开设置 row that used to make this+ * reachable from inside the card is gone from this host by construction (it+ * needs the service, and this is the service-is-off page), so the remaining+ * path is a change written while the user was off in a system screen. It is+ * one `SharedPreferences` read per resume — [AppLocale] deliberately does+ * not cache, see its doc — and it is the check that stops mattering the day+ * a door out of this page is added back.+ *+ * `paintedLang` is set in [onCreate] before the first resume, so this cannot+ * fire on open. Guarded on [ui] for the *other* path through [onCreate]: the+ * icon-as-app entry finishes before the card is ever built, and a lifecycle+ * callback arriving afterwards is not a good moment to throw.+ */+ override fun onResume() {+ super.onResume()+ if (!::ui.isInitialized) return+ val now = VoiceConfig.uiLanguage(this)+ if (paintedLang.isNotEmpty() && now != paintedLang) ui.repaintForLanguage()+ paintedLang = now+ }++ override fun onDestroy() {+ // Guarded because this also runs when onCreate threw before the card was+ // built, and dying is not a good moment to throw a second exception.+ if (::ui.isInitialized) ui.destroy()+ super.onDestroy()+ }++ // ---- The backdrop ----++ /**+ * Decode-and-blur via [Backdrop], into a full-bleed `CENTER_CROP`+ * ImageView, with the scrim as a foreground.+ *+ * The same three lines [TextInputActivity] runs, deliberately: the scrim is+ * *derived from the image's own brightness* (see `Backdrop.scrim`), so+ * pairing this image with any other fill would put the card at a different+ * mean luminance than the ball's card, which shares the image.+ *+ * There is nothing to rebuild it *for* any more — the language switch swaps+ * the card, which is a child of [stage] and not of this — so it is added+ * once and left alone. The `ready` branch skips the round trip on a second+ * open in the same process.+ */+ private fun installBackdrop() {+ if (backdropView != null) return+ val iv = ImageView(this).apply {+ scaleType = ImageView.ScaleType.CENTER_CROP+ layoutParams = matchParent()+ }+ if (Backdrop.ready()) {+ iv.setImageBitmap(Backdrop.bitmap())+ iv.foreground = ColorDrawable(Backdrop.scrim())+ } else {+ // Never blocks the main thread, and on a cold start the window is+ // already dark (`Theme.Body.Stage`), so the wait is invisible.+ Backdrop.warm(this) { bmp ->+ if (isFinishing || isDestroyed) return@warm+ iv.setImageBitmap(bmp)+ iv.foreground = ColorDrawable(Backdrop.scrim())+ }+ }+ stage.addView(iv)+ backdropView = iv+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/SelfCheckUi.kt b/app/src/main/java/net/kuafuai/andee/ui/SelfCheckUi.ktnew file mode 100644index 0000000..7b3cdf4--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/SelfCheckUi.kt@@ -0,0 +1,947 @@+package net.kuafuai.andee.ui++import android.content.Context+import android.content.Intent+import android.graphics.Color+import android.graphics.Typeface+import android.graphics.drawable.GradientDrawable+import android.net.Uri+import android.os.Handler+import android.os.Looper+import android.text.TextUtils+import android.util.Log+import android.util.TypedValue+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import android.widget.FrameLayout+import android.widget.LinearLayout+import android.widget.ScrollView+import android.widget.TextView+import android.widget.Toast+import net.kuafuai.andee.R+import net.kuafuai.andee.config.VoiceConfig+import net.kuafuai.andee.device.PermissionRequestActivity+import net.kuafuai.andee.device.SelfCheck+import net.kuafuai.andee.i18n.AppLocale+import java.util.concurrent.Executors++/**+ * The self-check list, as a card — [SelfCheck]'s only renderer.+ *+ * ### Why a card and not the page it used to be+ *+ * The entry point is `✓`, one glyph away from `⚙` in the assistant's control+ * bar, and that is the whole argument: those two buttons open *our own+ * surfaces* rather than acting on the screen, so they have to open the same+ * kind of thing. The settings sheet is a centred `Glass.card`; a self-check+ * that folded the assistant away and took the entire screen — with a background+ * image of its own — made two adjacent buttons behave like two different+ * products. It is now the same card chrome: same fill, same radius, same+ * header, same footer.+ *+ * ### Two hosts, one view+ *+ * [host] is null for the assistant's own case: this class creates the window+ * (`TYPE_APPLICATION_OVERLAY`) exactly as [SettingsUi] does, and the service+ * owns it. Non-null means somebody else is placing the card —+ * [SelfCheckActivity], which is an Activity and has a `stage` to put it in.+ *+ * The second host is not symmetry for its own sake. **The Activity is the one+ * surface in this app that runs while the assistant does not**, so when the+ * accessibility service is off, or the overlay grant is missing, or the user+ * has just reinstalled, there is no window this class could add itself to and+ * the launcher icon is the only door. Rendering both hosts from one builder is+ * what keeps the look identical on the day the user is looking at the broken+ * one.+ *+ * The difference between the hosts is only ever *where the card goes* and+ * *whether we may ask the compositor to blur*:+ *+ * * **Window** — [Glass.frost] decides the fill, and the answer is whatever+ * the device agreed to, exactly like [SettingsUi].+ * * **Activity** — the fill is the *frosted* one unconditionally, because the+ * Activity is already showing a blurred, scrimmed [Backdrop]; the blur+ * behind this card is real, we simply did not have to ask for it. There is+ * no window of ours to hand to [Glass.frost] and nothing it could add.+ *+ * ### The rows are [Glass.panel] here, and that is a correction+ *+ * They were [Glass.GLASS] while this list was bare on the backdrop. Inside a+ * card the arithmetic inverts: `GLASS` is 12% white, which on a 60%-black card+ * is a grey box, where `PANEL`'s 6% is the film that was measured for exactly+ * this — a group of rows *on* a card. A surface's fill is decided by what is+ * directly underneath it and by nothing else.+ *+ * ### Order of operations+ *+ * [SelfCheck.local] first — no network, instant — then the two probes on a+ * worker. A card that waits on a 6-second socket timeout before drawing its+ * first pixel reads as the thing being broken, and this is the card the user+ * opens *because* something is broken.+ *+ * ### Language+ *+ * [LangToggle] in the header, through the same [VoiceConfig] key the settings+ * sheet writes. [onLocaleChanged] is how everyone *else* hears about it, and in+ * the window host that is load-bearing: the ball, the card and the scrollback+ * are views the service built, so a language change that only repainted this+ * card would leave the user with two languages on screen the moment they closed+ * it. A host with no overlays to tell can spend the callback on its own+ * bookkeeping instead — see `SelfCheckActivity`, which uses it to remember what+ * is on screen. And [rebuild] re-renders [last] rather than re-running the+ * probes — see [SelfCheck.Finding] for why that is correct rather than a+ * shortcut.+ */+class SelfCheckUi(+ private val context: Context,+ private val onDismiss: () -> Unit,+ private val onOpenSettings: (() -> Unit)? = null,+ /** Fired when this card's own picker moved the language — see the class doc. */+ private val onLocaleChanged: () -> Unit = {},+ /**+ * Fired just before the user is sent to another screen, so the host can get+ * out of the way. See [apply] for the whole argument.+ *+ * Default no-op, and that is the Activity host's correct answer rather than+ * a shrug: there the card *is* an Activity, so the system draws whatever it+ * starts on top of it and there is nothing to yield. Only the window host —+ * where this card and the assistant's card behind it are both overlays —+ * has anything to do, and it is the only host that passes this.+ */+ private val onYieldScreen: () -> Unit = {},+ /** Non-null ⇒ that host places the card and this class owns no window. */+ private val host: FrameLayout? = null,+) {++ private companion object {+ const val TAG = "Body"++ /** Settings' number, for the same reason: the card is centred, so it reads on all four sides. */+ const val GUTTER_DP = 20+ const val CARD_RADIUS_DP = 28++ val OK_GREEN = 0xFF4ADE80.toInt()+ val WARN_AMBER = 0xFFFBBF24.toInt()++ const val MATCH = LinearLayout.LayoutParams.MATCH_PARENT+ const val WRAP = LinearLayout.LayoutParams.WRAP_CONTENT+ }++ /** Null in [host] mode — the whole point of which is that there is no window. */+ private val wm: WindowManager? =+ if (host == null) context.getSystemService(Context.WINDOW_SERVICE) as WindowManager else null++ private val main = Handler(Looper.getMainLooper())++ /**+ * The network half, one thread, created on demand.+ *+ * Never the main thread: [SelfCheck.run] blocks on a socket, deliberately,+ * because that is the only way to tell "wrong URL" from "wrong key" from+ * "wrong model name".+ */+ private val prober = Executors.newSingleThreadExecutor { r ->+ Thread(r, "self-check").apply { isDaemon = true }+ }++ private var root: View? = null+ private var closing: View? = null++ /** Whether the compositor agreed to blur. Decides the fill — see [Glass.frost]. */+ private var frosted = false++ /** Always a `var`, never `by lazy`. See the class doc and [repaintForLanguage]. */+ private var lctx: Context = context++ private var hero: LinearLayout? = null+ private var rows: LinearLayout? = null++ /**+ * The last list drawn, kept so a language switch can redraw it without+ * re-running the probes.+ *+ * Reusing it is not a shortcut, it is the reason [SelfCheck.Finding] holds+ * `@StringRes Int` and a `detailArgs` list rather than finished sentences:+ * the finding is language-free and the sentence is rendered from it at draw+ * time, so redrawing the *same* list in a new language is correct. Had the+ * findings carried strings, changing language would cost two network+ * round-trips to say the same thing in different words.+ */+ private var last: List<SelfCheck.Finding>? = null++ /**+ * Whether the assistant's own service is up, read off the list each time it+ * is drawn rather than asked of [net.kuafuai.andee.ScreenBodyService]+ * separately.+ *+ * Our settings sheet is an overlay that service owns, so this is the+ * difference between 打开设置 opening a card and 打开设置 doing nothing —+ * and [SelfCheck]'s `accessibility` row *is* the statement "that service is+ * not running". One source of truth; a second test here could only ever+ * disagree with the red row directly above it.+ */+ private var serviceUp = true++ fun isShowing(): Boolean = root != null++ // ---- Lifetimes ----++ fun show() {+ if (host == null) showInWindow() else showInHost()+ }++ private fun showInHost() {+ val stage = host ?: return+ if (root != null) return+ lctx = AppLocale.wrap(context)+ // The Activity is already a blurred backdrop, so the card takes the+ // frosted fill without asking anyone: there is no window here for+ // Glass.frost to edit, and nothing it could add that the backdrop has+ // not already done.+ frosted = true+ val v = build()+ // The same measured size the window host would use, not MATCH_PARENT+ // minus a gutter: those two are nearly the same number and not the same+ // number, and "nearly" is how a card the user opens from two identical+ // buttons ends up two different sizes. See [cardWidth] and [cardHeight].+ stage.addView(v, FrameLayout.LayoutParams(cardWidth(), cardHeight(), Gravity.CENTER))+ root = v+ OwnCard.shown()+ }++ private fun showInWindow() {+ val wm = wm ?: return+ // A re-open during the closing animation: drop the outgoing card now+ // rather than letting two of them share the screen.+ closing?.let { runCatching { wm.removeView(it) }; OwnCard.hidden() }+ closing = null+ if (root != null) return++ lctx = AppLocale.wrap(context)+ val p = WindowManager.LayoutParams(+ cardWidth(), cardHeight(),+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ // No FLAG_NOT_FOCUSABLE: nothing here needs the keyboard, but this+ // card can open the settings sheet on top of itself and the two+ // must not disagree about who owns focus. Same params as SettingsUi,+ // for the same reason it is a card at all.+ 0,+ android.graphics.PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.CENTER+ }+ // Before build(), because the answer picks the card's fill.+ frosted = Glass.frost(context, wm, p)+ val v = build()+ try {+ v.hideFromAccessibility()+ wm.addView(v, p)+ root = v+ OwnCard.shown()+ Glass.enter(v)+ } catch (t: Throwable) {+ Log.w(TAG, "self-check card refused", t)+ }+ }++ /**+ * Close, and hand the news to [onDismiss] once the card is off screen.+ *+ * The two hosts animate differently on purpose. In a window the card has to+ * sink under its own power, and [Glass.exit] guarantees `done` runs exactly+ * once even if the animation is cancelled — what hangs off it is `removeView`+ * plus the [OwnCard] bookkeeping. On the Activity's stage there is nothing+ * to animate: `leave()` is already growing the whole stage away underneath+ * this, and a card shrinking inside a shrinking screen is two exits.+ *+ * ### [onDismiss] only from the card that is still the current one+ *+ * It used to fire outside the guard, and the guard is not decoration:+ * `closing !== v` means a *newer* card took this one's place while the exit+ * animation was still running — [showInWindow] drops the outgoing view and+ * clears [closing] itself, which is exactly what that branch is for. The new+ * card has already told the host it exists; letting the outgoing one also+ * announce its closing clears the *new* card's record instead, and the host+ * then believes no card is up while one is on screen.+ *+ * Narrow either way, but this change widened it: `hide()` no longer comes+ * only from a user pressing ✕ — [onYieldScreen] has the host close the card+ * on the way out to the browser. Fold, reopen the check, and the stale exit+ * would land after the second card was up.+ */+ fun hide() {+ val v = root ?: return+ root = null+ val stage = host+ if (stage != null) {+ runCatching { stage.removeView(v) }+ OwnCard.hidden()+ onDismiss()+ return+ }+ val wm = wm ?: return+ closing = v+ Glass.exit(v) {+ if (closing === v) {+ runCatching { wm.removeView(v) }+ closing = null+ OwnCard.hidden()+ onDismiss()+ }+ }+ }++ /**+ * Released with the window. Safe to call twice, and safe to call after [hide].+ *+ * It is also the backstop for [OwnCard], and that is not tidiness: the count+ * is what tells the brain whether one of our own cards is covering the+ * screen, and a host that goes away *without* [hide] — an Activity being+ * destroyed rather than closed — would leave the process permanently+ * convinced a card is still up. [hide] clears [root] as its first act, so+ * the two do not double-count; see [Glass.exit] for why that count is the+ * thing that must not leak.+ */+ fun destroy() {+ prober.shutdownNow()+ if (root != null) {+ root = null+ OwnCard.hidden()+ }+ }++ // ---- Data ----++ /** Paint what is knowable now, then fill in the probes behind it. */+ fun start() {+ render(SelfCheck.local(context))+ probe()+ }++ private fun probe() {+ val app = context.applicationContext+ prober.execute {+ val found = runCatching { SelfCheck.run(app) }.getOrNull() ?: return@execute+ main.post {+ // The card may have been closed while the socket was open.+ if (root == null) return@post+ render(found)+ }+ }+ }++ /**+ * Draw the list, problems first.+ *+ * The order [SelfCheck.run] returns is the order the *probe* has to run in+ * (network before the two things that need it), which is not the order a+ * reader wants. So the rows are regrouped here — everything needing+ * attention at the top, then the facts — and the facts get their own+ * heading ([R.string.check_page_notes_lead]) so a green row is never read+ * as a fault the user has to go and fix.+ */+ private fun render(found: List<SelfCheck.Finding>) {+ last = found+ val attention = found.filter { SelfCheck.needsAttention(it) }+ val rest = found.filterNot { SelfCheck.needsAttention(it) }+ // Before any row is built: [fixButton] reads this per row.+ serviceUp = found.none { it.id == "accessibility" && it.level == SelfCheck.Level.FAIL }++ hero?.removeAllViews()+ hero?.addView(summary(attention.size))+ // The one case where the card has to say it cannot help: our own+ // settings sheet lives in the service, so with the service off there is+ // nowhere for 打开设置 to go. Say that instead of showing a button that+ // would do nothing — and note the row itself has no button either, see+ // [fixButton]. Only reachable in host mode: the window host *is* the+ // service, so if it is drawing this at all, accessibility is on.+ if (!serviceUp) hero?.addView(note(lctx.getString(R.string.check_page_no_service)))++ rows?.removeAllViews()+ attention.forEach { rows?.addView(row(it)) }+ if (rest.isNotEmpty()) {+ if (attention.isNotEmpty()) rows?.addView(sectionHeading(R.string.check_page_notes_lead))+ rest.forEach { rows?.addView(row(it)) }+ }+ }++ // ---- Language ----++ /**+ * Redraw in whatever language is now saved, because this card's own picker+ * was used.+ */+ private fun onLanguageChanged() {+ repaintForLanguage()+ onLocaleChanged()+ }++ /**+ * The same, for a change made *elsewhere* — in the window host the settings+ * sheet opens on top of this card, so its picker is reachable while this one+ * is on screen.+ *+ * No [onLocaleChanged] call: whoever told us already told the service, and+ * bouncing it back would rebuild the ball twice.+ */+ fun repaintForLanguage() {+ if (root == null) return+ lctx = AppLocale.wrap(context)+ rebuild()+ }++ /**+ * Swap the whole card for a freshly built one, in place, and paint it.+ *+ * Same trick as [SettingsUi.rebuild], including reusing the old+ * `LayoutParams` and leaning on the fact that the card was up before and is+ * up after (no [OwnCard] bookkeeping — counting it again would leave+ * `annotate` permanently convinced a card is open). It cross-fades rather+ * than cutting, because the swap is `removeView` + `addView` on a window and+ * a hard cut shows one frame of whatever is behind the card.+ *+ * ### Why the redraw is in here, and not one line after the caller's return+ *+ * [build] only assembles the chrome: it leaves [hero] and [rows] empty and+ * points those two fields at the *new* card. Whoever fills them has to run+ * after `build()` — and in the **window host that means after this function+ * has already returned**, because the new card is built inside+ * [Glass.fadeOut]'s end action, a fifth of a second later. Drawing from the+ * call site therefore wrote the whole list into the *outgoing* card's+ * [hero] and [rows], which were then removed with it, and the replacement+ * arrived as a card with its header, its footer and nothing between them.+ * Reported on the tablet as: switch language on the assistant's card, watch+ * the list disappear, close the card — everything normal again, because the+ * next open goes through [showInWindow], which is synchronous.+ *+ * The hosted path builds synchronously, so the distinction does not exist+ * there, and that is exactly why the fault survived: it is invisible on the+ * host the user reaches from the launcher and only fires on the one reached+ * from `✓`. Keeping `again` on this side of the branch is what makes the two+ * hosts incapable of disagreeing again.+ *+ * `again` is the list already drawn and not a fresh probe: see [last] for why+ * re-rendering the same findings in a new language is correct.+ */+ private fun rebuild() {+ val old = root ?: return+ val again = last ?: SelfCheck.local(context)+ val stage = host+ if (stage != null) {+ // No cross-fade on a stage: the new card goes in at the same size in+ // the same frame, and the backdrop behind it never moves, so a fade+ // here would only be a flicker of the Activity's own background.+ val v = build()+ val i = stage.indexOfChild(old)+ runCatching { stage.removeView(old) }+ if (i >= 0) stage.addView(v, i, old.layoutParams) else stage.addView(v, old.layoutParams)+ root = v+ render(again)+ return+ }+ val wm = wm ?: return+ Glass.fadeOut(old) {+ if (root !== old) return@fadeOut+ val v = build()+ runCatching {+ v.hideFromAccessibility()+ val p = old.layoutParams as WindowManager.LayoutParams+ wm.removeView(old)+ wm.addView(v, p)+ root = v+ Glass.fadeIn(v)+ render(again)+ }+ }+ }++ // ---- Build ----++ private fun build(): View {+ val card = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ background = Glass.card(context, dp(CARD_RADIUS_DP), frosted)+ // So the sheen and the panels stay inside the rounded corners.+ clipToOutline = true+ }++ card.addView(header(), matchWrap())++ val scroll = ScrollView(context).apply {+ isFillViewport = true+ // The card already has an edge; a scrollbar track on top of glass+ // is one line too many.+ isVerticalScrollBarEnabled = false+ }+ val body = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ // Same 24dp as the settings form: this is the same card, and two+ // cards with the same chrome and different insets read as one of+ // them being broken.+ setPadding(dp(24), dp(4), dp(24), dp(24))+ }++ hero = LinearLayout(context).apply { orientation = LinearLayout.VERTICAL }+ body.addView(hero, matchWrap())++ rows = LinearLayout(context).apply { orientation = LinearLayout.VERTICAL }+ body.addView(rows, matchWrap())++ scroll.addView(body, matchWrap())+ card.addView(scroll, LinearLayout.LayoutParams(MATCH, 0, 1f))++ card.addView(footer(), matchWrap())+ return card+ }++ private fun header(): View {+ val row = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ setPadding(dp(28), dp(22), dp(18), dp(14))+ }+ val titles = LinearLayout(context).apply { orientation = LinearLayout.VERTICAL }+ titles.addView(TextView(context).apply {+ text = lctx.getString(R.string.check_page_title)+ // Settings' 26sp, so the two cards' titles are the same size.+ textSize = 26f+ setTextColor(Color.parseColor(Glass.TITLE))+ letterSpacing = 0.01f+ })+ titles.addView(TextView(context).apply {+ text = lctx.getString(R.string.check_page_lead)+ textSize = 13f+ setTextColor(Color.parseColor(Glass.MUTED))+ setPadding(0, dp(4), 0, 0)+ })+ row.addView(titles, LinearLayout.LayoutParams(0, WRAP, 1f))+ row.addView(+ LangToggle(context) { onLanguageChanged() },+ LinearLayout.LayoutParams(WRAP, WRAP).apply { marginEnd = dp(12) },+ )+ row.addView(closeButton(), LinearLayout.LayoutParams(dp(36), dp(36)))+ return row+ }++ private fun closeButton(): View = TextView(context).apply {+ text = "✕"+ textSize = 15f+ setTextColor(Color.parseColor(Glass.SECONDARY))+ gravity = Gravity.CENTER+ background = Glass.panel(context, dp(18))+ isClickable = true+ isFocusable = true+ Glass.pressable(this)+ contentDescription = lctx.getString(R.string.check_page_done)+ setOnClickListener { hide() }+ }++ private fun footer(): View {+ val row = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ setPadding(dp(18), dp(10), dp(18), dp(18))+ }+ // Left, and deliberately NOT a filled button: this must not read as one+ // of the two ways out of the card. It is the one control worth having on+ // a card whose whole subject is state the user changes in *other* apps —+ // without it, checking whether a fix took means closing this and coming+ // back, which turns a two-tap fix into a round trip.+ row.addView(+ textButton(lctx.getString(R.string.check_page_recheck), Glass.ACCENT) {+ // Local first so the tap has a visible effect on this frame; the+ // probes are about to hold this card open for up to ~6 s.+ render(SelfCheck.local(context))+ probe()+ },+ LinearLayout.LayoutParams(0, WRAP, 1f),+ )+ row.addView(button(lctx.getString(R.string.check_page_done), filled = true) { hide() })+ return row+ }++ /**+ * Settings' bare text button, down to the vertical padding.+ *+ * The one thing not copied is its `gravity`: [SettingsUi.textButton] centres+ * its label because the cell it sits in is squeezed between two pills on the+ * right, and here the same weight lands the text in the middle of the card+ * with nothing on either side of it. Start-aligned is the same button in the+ * only position it has here.+ */+ private fun textButton(label: String, tint: String, onClick: () -> Unit): TextView =+ TextView(context).apply {+ text = label+ textSize = 14f+ setTextColor(Color.parseColor(tint))+ gravity = Gravity.START or Gravity.CENTER_VERTICAL+ setPadding(dp(4), dp(13), dp(4), dp(13))+ isClickable = true+ isFocusable = true+ Glass.pressable(this)+ setOnClickListener { onClick() }+ }++ private fun button(label: String, filled: Boolean, onClick: () -> Unit): TextView {+ val radius = dp(24)+ return TextView(context).apply {+ text = label+ textSize = 15f+ setTextColor(Color.parseColor(if (filled) Glass.TITLE else Glass.LABEL))+ gravity = Gravity.CENTER+ background =+ if (filled) Glass.tinted(context, radius) else Glass.panel(context, radius)+ setPadding(dp(26), dp(13), dp(26), dp(13))+ isClickable = true+ isFocusable = true+ Glass.pressable(this)+ setOnClickListener { onClick() }+ }+ }++ // ---- One row ----++ /**+ * A finding: a dot, what was checked, what was found, and the way out.+ *+ * Two lines rather than one because the title alone ("麦克风") tells the+ * user nothing they can act on — the second line is the one that says the+ * permission is off, that the ball will therefore never wake, and that the+ * button beside it fixes that. See the `check_detail_*` strings.+ */+ private fun row(f: SelfCheck.Finding): View {+ val card = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ background = Glass.panel(context, dp(14))+ setPadding(dp(14), dp(14), dp(14), dp(14))+ layoutParams = LinearLayout.LayoutParams(MATCH, WRAP).apply { bottomMargin = dp(10) }+ }++ val head = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ }+ head.addView(View(context).apply {+ background = GradientDrawable().apply {+ shape = GradientDrawable.OVAL+ setColor(dotColor(f.level))+ }+ layoutParams = LinearLayout.LayoutParams(dp(10), dp(10)).apply {+ marginEnd = dp(10)+ }+ })+ head.addView(TextView(context).apply {+ text = lctx.getString(f.title)+ setTextColor(Color.parseColor(Glass.TITLE))+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 16f)+ setTypeface(typeface, Typeface.BOLD)+ maxLines = 1+ ellipsize = TextUtils.TruncateAt.END+ layoutParams = LinearLayout.LayoutParams(0, WRAP, 1f)+ })+ fixButton(f)?.let { head.addView(it) }+ card.addView(head)++ card.addView(TextView(context).apply {+ text = AppLocale.str(context, f.detail, *f.detailArgs.toTypedArray())+ setTextColor(Color.parseColor(Glass.SECONDARY))+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 14f)+ setPadding(0, dp(6), 0, 0)+ })++ // The one fix with no button: show the command, and be explicit that it+ // is an adb errand. `setTextIsSelectable` because the user has to get+ // this string onto a computer, and retyping a 60-character package name+ // from a screenshot is how a working instruction turns into a broken one.+ (f.fix as? SelfCheck.Fix.Adb)?.let { adb ->+ card.addView(TextView(context).apply {+ text = lctx.getString(R.string.check_fix_adb_lead)+ setTextColor(Color.parseColor(Glass.MUTED))+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 12f)+ setPadding(0, dp(10), 0, dp(4))+ })+ card.addView(TextView(context).apply {+ text = adb.command+ setTextColor(Color.parseColor(Glass.LABEL))+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 12f)+ typeface = Typeface.MONOSPACE+ // Direct call, not `isTextSelectable = true`: TextView's setter+ // is `setTextIsSelectable` and its getter `isTextSelectable`,+ // which Kotlin does not pair into a var — the assignment parses+ // as a read of the getter and fails.+ setTextIsSelectable(true)+ background = Glass.well(context, dp(8))+ setPadding(dp(10), dp(8), dp(10), dp(8))+ })+ }+ return card+ }++ private fun dotColor(level: SelfCheck.Level): Int = when (level) {+ SelfCheck.Level.OK -> OK_GREEN+ SelfCheck.Level.NOTE -> Color.parseColor(Glass.MUTED)+ SelfCheck.Level.WARN -> WARN_AMBER+ SelfCheck.Level.FAIL -> Color.parseColor(Glass.DANGER)+ }++ /**+ * The way out, or nothing when there is not one.+ *+ * Text rather than a filled button: a row can have zero or one door, and a+ * column of gradient buttons inside a card reads as a menu. The accent+ * colour is this app's established "this is tappable" mark.+ */+ private fun fixButton(f: SelfCheck.Finding): TextView? {+ // Ours, and unreachable: see [serviceUp]. The note above the list has+ // already said so, and a button that does nothing is worse than no+ // button — it is the exact failure this list exists to avoid.+ if (f.fix is SelfCheck.Fix.OurSettings && !serviceUp) return null+ val label = when (val fix = f.fix) {+ // Zero means "the action's own word"; a fix instance gets to+ // override it when 去开启 would name the wrong errand — see+ // SelfCheck.Fix.Screen.label.+ is SelfCheck.Fix.Screen ->+ if (fix.label != 0) fix.label else R.string.check_fix_settings++ is SelfCheck.Fix.Link -> R.string.check_fix_download+ is SelfCheck.Fix.Grant -> R.string.check_fix_allow+ is SelfCheck.Fix.OurSettings -> R.string.check_fix_app_settings+ else -> return null+ }+ return TextView(context).apply {+ text = lctx.getString(label)+ setTextColor(Color.parseColor(Glass.ACCENT))+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 15f)+ setTypeface(typeface, Typeface.BOLD)+ setPadding(dp(10), dp(6), 0, dp(6))+ isClickable = true+ setOnClickListener { apply(f) }+ }+ }++ // ---- Doors ----++ /**+ * Every fix is a `startActivity` — nothing here closes a gap by itself.+ *+ * ### The card gets out of the way first, and why it has to+ *+ * In the window host this card is a `TYPE_APPLICATION_OVERLAY` sized to 98%+ * × 96% of the display, and **an overlay outranks every Activity on the+ * device**. So the browser 去下载 opened came up *underneath* the card that+ * had just asked the user to look at it, and so did the system Settings page+ * 去开启 opens. Reported from the field exactly as it looks: "点击去下载ADB+ * 的时候,应该退出全屏,不然挡住浏览器了". Nothing about the destinations is+ * wrong — they are ordinary Activities and they do come up. They just come up+ * behind us.+ *+ * So every door but one tells the host to yield first — see+ * [onYieldScreen]. It is deliberately *every* door and not just the browser:+ * 去开启 and 允许 are the same mistake against the same kind of window, and+ * fixing only the one that got reported would leave two dead buttons behind+ * a live one.+ *+ * [SelfCheck.Fix.OurSettings] is the exception, and it is the reason the+ * yield is here and not in [launch]: that sheet is an overlay of ours too,+ * and the design is that it comes up **over** this card so the list is still+ * behind it to return to. Yielding for it would close the card the user is+ * meant to come back to.+ *+ * The card itself is not closed here — delivering the yield is the host's+ * job, because only the host knows whether there is anything to close. On+ * the Activity host there is not: that card *is* an Activity, so whatever it+ * starts is drawn on top of it by the system already. See [onYieldScreen].+ */+ private fun apply(f: SelfCheck.Finding) {+ val fix = f.fix+ if (fix !is SelfCheck.Fix.OurSettings) {+ // Only if there is somewhere to go. Yielding and failing is the one+ // outcome with nothing on screen: the card is gone and so is the+ // page it promised. Cheap to rule out — see [openable] — and this is+ // not hypothetical, a HONOR really has Settings screens with no+ // Activity behind the action.+ if (!openable(fix)) {+ toastNoScreen()+ return+ }+ onYieldScreen()+ }+ when (fix) {+ // Both of these leave the device, and both are already spelled out by+ // [intentFor] — the same call [openable] asked, so what was checked+ // and what gets launched can never be two different Intents. 去下载+ // is the link: a browser, not Settings, and the user comes back+ // through wherever they were with 重新检查 to tell them whether it+ // took.+ is SelfCheck.Fix.Screen, is SelfCheck.Fix.Link -> launch(intentFor(fix))++ is SelfCheck.Fix.Grant -> launch(+ Intent(context, PermissionRequestActivity::class.java)+ .putExtra("permissions", fix.permissions.toTypedArray())+ )++ is SelfCheck.Fix.OurSettings -> onOpenSettings?.invoke()++ is SelfCheck.Fix.Adb, SelfCheck.Fix.Nothing -> Unit+ }+ }++ /**+ * Whether [fix] has an Activity on this device at all — asked **before**+ * [onYieldScreen] closes the card, so the two can never disagree.+ *+ * Everything this card cannot open is either its own ([Fix.OurSettings],+ * [Fix.Grant]) or has nothing to open ([Fix.Adb], [Fix.Nothing]), so only+ * the two that leave the device are worth asking about.+ *+ * `queryIntentActivities` rather than `resolveActivity`: the stricter call+ * wants a default-browsing handler, and a device with several browsers and+ * no default set would read as "nowhere to go" for 去下载 — refusing to+ * yield on the very button this exists to serve. When the query itself+ * fails the answer is `true`: that defers to [launch]'s own failure toast,+ * which is the honest fallback rather than a second guess.+ */+ private fun openable(fix: SelfCheck.Fix): Boolean = when (fix) {+ is SelfCheck.Fix.Screen, is SelfCheck.Fix.Link ->+ runCatching {+ context.packageManager+ .queryIntentActivities(intentFor(fix), 0)+ .isNotEmpty()+ }.getOrDefault(true)++ else -> true+ }++ /**+ * The Intent [apply] will hand to [launch]; one definition, asked twice.+ *+ * Only the two that leave the device are spelled out. The `else` is an empty+ * Intent rather than a throw because no caller reaches it — [apply] routes+ * the other three fixes to their own branches and [openable] short-circuits+ * before asking — but an empty Intent resolving to nothing reads as "no+ * Activity for a fix that never wanted one", while a throw here would take+ * the card down for a case that cannot happen.+ */+ private fun intentFor(fix: SelfCheck.Fix): Intent = when (fix) {+ is SelfCheck.Fix.Screen -> Intent(fix.action).apply {+ fix.data?.let { data = Uri.parse(it) }+ fix.extras.forEach { (name, value) -> putExtra(name, value) }+ }++ is SelfCheck.Fix.Link -> Intent(Intent.ACTION_VIEW, Uri.parse(fix.url))++ else -> Intent()+ }++ private fun toastNoScreen() {+ runCatching {+ Toast.makeText(context, lctx.getString(R.string.check_fix_no_screen), Toast.LENGTH_LONG)+ .show()+ }+ }++ /**+ * A Settings screen the ROM may not have — and when it does not, **say so**.+ *+ * The `runCatching` used to be the whole story, and it is how this app+ * shipped a dead button: on a HONOR, `ACTION_NOTIFICATION_LISTENER_SETTINGS`+ * resolves to no activity at all, so 去开启 swallowed an+ * `ActivityNotFoundException` and the user reported exactly what that+ * looks like from the outside — "点了没反应". A silent no-op on the one+ * control that exists to fix something is worse than no button, which is the+ * rule this whole card is built on. So the failure gets a voice.+ *+ * This is the second line of defence, not the first: [openable] already+ * refuses to yield when there is nowhere to go. It still has to exist,+ * because the check and the launch are separated by the card's exit+ * animation — long enough for the answer to change. And unlike+ * [openable]'s refusal, this one arrives after the card is gone, so the+ * toast is the only thing left to say.+ */+ private fun launch(intent: Intent) {+ // Only when there is no Activity of ours in play: from a Service Context+ // the flag is mandatory, and from an Activity it would put somebody+ // else's settings screen in a task of its own.+ if (host == null) intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)+ val ok = runCatching { context.startActivity(intent) }.onFailure {+ Log.w(TAG, "self-check: no activity for ${intent.action}", it)+ }.isSuccess+ if (!ok) toastNoScreen()+ }++ // ---- Small parts ----++ private fun summary(problems: Int): View = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ addView(TextView(context).apply {+ text = if (problems == 0) {+ lctx.getString(R.string.check_page_ok_title)+ } else {+ lctx.getString(R.string.check_page_issues, problems)+ }+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 17f)+ setTypeface(typeface, Typeface.BOLD)+ setTextColor(Color.parseColor(if (problems == 0) "#4ADE80" else Glass.DANGER))+ })+ // Only in the clean case: a green headline with no sentence under it+ // leaves the user wondering whether the list below is part of the+ // problem, and check_page_ok_note is the sentence that answers that.+ if (problems == 0) {+ addView(TextView(context).apply {+ text = lctx.getString(R.string.check_page_ok_note)+ setTextColor(Color.parseColor(Glass.SECONDARY))+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 14f)+ setPadding(0, dp(4), 0, dp(12))+ })+ }+ }++ private fun sectionHeading(id: Int): TextView = TextView(context).apply {+ text = lctx.getString(id)+ setTextColor(Color.parseColor(Glass.MUTED))+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 13f)+ setPadding(0, dp(4), 0, dp(8))+ }++ private fun note(text: String): TextView = TextView(context).apply {+ this.text = text+ setTextColor(Color.parseColor(Glass.SECONDARY))+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 14f)+ background = Glass.panel(context, dp(12))+ setPadding(dp(14), dp(12), dp(14), dp(12))+ layoutParams = LinearLayout.LayoutParams(MATCH, WRAP).apply { topMargin = dp(10) }+ }++ private fun matchWrap() = LinearLayout.LayoutParams(MATCH, WRAP)++ private fun dp(v: Int): Int = Glass.dp(context, v)++ /**+ * Preferred size, capped to what the screen actually has.+ *+ * Settings' numbers, deliberately. Two cards opened from adjacent buttons+ * have to be the same size or the second one reads as a different product+ * arriving; see [SettingsUi.cardWidth] for why these are not the old+ * tablet-phone constants.+ *+ * The two hosts ask two different objects for the screen — the window host+ * has a `WindowManager` and the Activity host does not — so the *measurement*+ * is factored out here and neither branch does its own arithmetic.+ */+ private fun cardWidth(): Int {+ val screen = screenW()+ return minOf(screen - dp(2 * GUTTER_DP), (screen * 0.98f).toInt())+ }++ private fun cardHeight(): Int {+ val screen = screenH()+ return minOf(screen - dp(2 * GUTTER_DP), (screen * 0.96f).toInt())+ }++ private fun screenW(): Int = wm?.currentWindowMetrics?.bounds?.width()+ ?: context.resources.displayMetrics.widthPixels++ private fun screenH(): Int = wm?.currentWindowMetrics?.bounds?.height()+ ?: context.resources.displayMetrics.heightPixels+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/SettingsUi.kt b/app/src/main/java/net/kuafuai/andee/ui/SettingsUi.ktnew file mode 100644index 0000000..b21dbae--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/SettingsUi.kt@@ -0,0 +1,1360 @@+package net.kuafuai.andee.ui++import android.animation.Animator+import android.animation.AnimatorListenerAdapter+import android.animation.ValueAnimator+import android.annotation.SuppressLint+import android.content.Context+import android.graphics.Color+import android.graphics.PixelFormat+import android.text.InputType+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import android.widget.EditText+import android.widget.FrameLayout+import android.widget.LinearLayout+import android.widget.ScrollView+import android.widget.TextView+import androidx.annotation.StringRes+import net.kuafuai.andee.R+import net.kuafuai.andee.config.FactoryReset+import net.kuafuai.andee.config.Notebook+import net.kuafuai.andee.config.Vault+import net.kuafuai.andee.config.VoiceConfig+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.wake.WakeTemplates+import net.kuafuai.andee.wake.WakeWord++/**+ * How much of the notebook the settings card prints.+ *+ * Bounded because it is a `TextView` inside a scroll view inside a floating+ * card, laid out on every rebuild: a few thousand lines would be paid for on+ * each open, and again on every language switch. The cut is announced in the+ * last line — a silent truncation would read as "this is all of it".+ */+private const val NOTEBOOK_DUMP_CHARS = 6_000++/**+ * Settings overlay for editing [VoiceConfig].+ *+ * A focusable glass card centred on screen — see [Glass] for the palette, the+ * blur and the motion, all of which are shared with [VaultUi] and+ * [WakeEnrollUi] because this card opens both of them on top of itself.+ *+ * Reads the saved values once on [show] so every row draws itself already+ * filled, and writes back on 保存. Text fields follow [VoiceConfig.save]'s rule+ * — a non-empty value overwrites, an empty one keeps what was there — so a+ * field can be corrected but not cleared. That limitation is why the settings+ * that are really *choices* (`brain`, `llm_thinking`, `grounding`, `lang`) are+ * segmented pickers rather than fields you type `hub` / `on` / `off` into: a+ * picker always sends a value, and a typo in a free-text enum silently failed+ * closed with nothing on screen to say so.+ *+ * Two organising rules, because the thing this card is worst at is being long:+ *+ * * **The brain section shows one half of itself.** `hub_url` and the DeepSeek+ * rows are alternatives, never both in play, so picking a brain folds the+ * other one's fields away instead of leaving the user to guess which four+ * rows matter. It *folds* rather than flicking to `GONE`: rows that vanish+ * instantly read as rows that were deleted.+ * * **The 火山 endpoints start collapsed.** They ship working defaults+ * ([VoiceConfig.DEFAULT_ASR_ENDPOINT] and friends) and the key is compiled+ * in, so on almost every device they are read-only trivia sitting above the+ * settings that actually get edited.+ *+ * **Language.** Every visible string comes from a resource, through [lctx], and+ * the picker sits in the header because it is the one setting whose effect is+ * the card itself. It is also the one setting written the instant it is tapped:+ * the whole card is rebuilt in the new language ([rebuild]), and that redraw is+ * the confirmation — a language switch that waited for 保存 would look broken for+ * as long as the user stared at it. Consequence, deliberate: 取消 does not put+ * the language back, because by then they are reading the answer.+ *+ * This card used to carry its own `t(zh, en)` helper, with both languages+ * written at every call site. That worked for exactly two languages and made a+ * third one a rewrite of this file, so the strings moved to `values/` and+ * `values-en/` and the helper is gone. A new language is now a new `values-xx/`+ * directory and no edit here at all.+ *+ * One string is deliberately *not* a resource: the two labels in [langPicker].+ * A language picker names each language in its own script, so 中 and EN are the+ * same in every locale.+ *+ * The 火山 key row is where a key comes from at all: [VoiceConfig.API_KEY]+ * ships empty in the public source, so an empty row means the device cannot+ * hear or speak. `SelfCheck`'s `voice_key` row says so before the user finds+ * out from a connect timeout.+ */+class SettingsUi(+ private val context: Context,+ private val onDismiss: () -> Unit,+ /**+ * 恢复出厂设置, after the user has confirmed twice. Wired to+ * `ScreenBodyService.applyFactoryReset`, which does the wiping *and*+ * rebuilds the brain — this class knows how to ask, not how to erase.+ */+ private val onFactoryReset: () -> Unit,+ /**+ * 清空聊天记录, after the user has confirmed. Wired to+ * `ScreenBodyService.clearChatHistory`, which wipes the scrollback *and*+ * rebuilds the brain so the conversation goes with it — clearing only the+ * visible rows would leave the model remembering everything the user+ * just asked to forget.+ */+ private val onClearChat: () -> Unit,+ /**+ * 清空小本本, after the user has confirmed. Wired to+ * `ScreenBodyService.clearNotebook`.+ *+ * A separate errand from [onClearChat] rather than a checkbox on it: what+ * the assistant *chose to keep* outlives any one conversation, which is+ * the point of the notebook, so wiping a conversation must not take it and+ * wiping it must not take the conversation.+ */+ private val onClearNotes: () -> Unit,+ /**+ * The language was just changed, and the rest of the UI has to follow.+ *+ * This card can rebuild itself ([rebuild]) because it owns every view it+ * draws. The ball, the scrollback and the pinned tips cannot: they belong+ * to [FloatingWindowUi], which lives in the service and outlives this card.+ * So the service passes a hook in here and the picker calls it.+ *+ * Called *before* [rebuild], so the card and the thing behind it are never+ * briefly speaking different languages while the cross-fade runs.+ *+ * Default no-op: a card built without a service behind it (a test, or any+ * future caller) still switches its own language correctly.+ */+ private val onLocaleChanged: () -> Unit = {},+) {++ private val wm: WindowManager =+ context.getSystemService(Context.WINDOW_SERVICE) as WindowManager++ private var root: View? = null++ /**+ * The card on its way out, still on screen for the length of [Glass.exit].+ * Kept so a [show] that arrives mid-exit can take it down at once instead of+ * stacking a second window on top of it.+ */+ private var closing: View? = null++ /** Text rows, by preference key. */+ private val fields = mutableMapOf<String, EditText>()++ /** Segmented rows, by preference key; each lambda reads back what is picked. */+ private val picks = mutableMapOf<String, () -> String>()++ /** Saved values, read once per [show]. See the class KDoc. */+ private var saved: Map<String, String> = emptyMap()++ /** English rather than Chinese. Resolved in [show]; flipped by [rebuild]. */+ private var english = false++ /**+ * Strings in the user's language, refreshed at the top of [build].+ *+ * Refreshed *there* and not in [show] on purpose: the language picker calls+ * [rebuild], which goes straight to [build] without passing through [show],+ * and rewrapping in the one function that consumes it means no future+ * caller has to remember to do it.+ *+ * Views are still built on [context]; only text comes from here. See+ * [AppLocale] for why those two are deliberately different Contexts.+ */+ private var lctx: Context = context++ /** Survives a [rebuild], so switching language doesn't re-hide the group. */+ private var advancedOpen = false++ /** Same, for the notebook dump. Kept apart so opening one does not unfold the other. */+ private var notebookOpen = false++ /**+ * The 诊断 section is showing the factory-reset warning instead of its+ * three buttons. Survives a [rebuild] so a language switch mid-confirmation+ * does not silently put the destructive button back.+ */+ private var factoryConfirm = false++ /** Same, for the 清空聊天记录 warning. One confirm at a time. */+ private var clearChatConfirm = false++ /** Same, for the 清空小本本 warning. */+ private var clearNotesConfirm = false++ /**+ * The form's scroller, so [rebuild] can put the user back where they were.+ * It did not matter while the destructive buttons were in the footer — the+ * footer never scrolls — but their confirms now appear in 诊断, near the+ * bottom of a long form, and a rebuild that starts at the top leaves the+ * user looking at the brain section wondering what their tap did.+ */+ private var scroll: ScrollView? = null++ /** Whether the compositor agreed to blur. Decides the fill — see [Glass.frost]. */+ private var frosted = false++ fun isShowing(): Boolean = root != null++ fun show() {+ // A re-open during the closing animation: drop the outgoing card now+ // rather than letting two of them share the screen.+ closing?.let { runCatching { wm.removeView(it) }; OwnCard.hidden() }+ closing = null+ if (root != null) return++ saved = VoiceConfig.current(context)+ english = saved["lang"] == VoiceConfig.LANG_EN+ val p = WindowManager.LayoutParams(+ cardWidth(), cardHeight(),+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ // No FLAG_NOT_FOCUSABLE: this window takes keyboard focus so EditText works.+ // No FLAG_LAYOUT_NO_LIMITS either — that flag lets a window lay out past+ // the screen edges, which on a phone is exactly how a card sized for a+ // tablet goes half-missing instead of being clamped to something visible.+ 0,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.CENTER+ // The IME is most of a phone screen. Shrink the card instead of letting+ // the keyboard sit on top of the field being typed into; the form is+ // already in a ScrollView, so it absorbs the loss.+ softInputMode = WindowManager.LayoutParams.SOFT_INPUT_ADJUST_RESIZE+ }+ // Before build(), because the answer picks the card's fill.+ frosted = Glass.frost(context, wm, p)+ val v = build()+ try {+ v.hideFromAccessibility()+ wm.addView(v, p)+ root = v+ OwnCard.shown()+ Glass.enter(v)+ } catch (t: Throwable) {+ // ignore+ }+ }++ /**+ * Swap the whole card for a freshly built one, in place.+ *+ * Same trick as [VaultUi.rebuild], including reusing the old LayoutParams,+ * and no [OwnCard] bookkeeping: the card was up before and is up after, so+ * counting it again would leave `annotate` permanently convinced a card is+ * open. Typed-but-unsaved text is carried across through [saved] — losing+ * a half-entered API key to a language tap would be its own bug.+ *+ * It cross-fades rather than cutting. The swap is `removeView` ++ * `addView` on a *window*, so a hard cut shows one frame of whatever is+ * behind the card, which looks like the card blinked out of existence.+ */+ private fun rebuild() {+ val old = root ?: return+ saved = saved + currentEdits()+ // null means "the bottom", resolved once the new form has a height.+ // Not Int.MAX_VALUE: ScrollView.clamp() evaluates `viewport + y`, which+ // overflows to negative, skips the clamp and scrolls the form clean off+ // the card — an empty sheet with a header and a footer.+ val offset: Int? =+ // A confirm is taller than the buttons it replaces — the factory+ // list alone is five lines — so holding the old offset leaves its+ // 确认 row below the fold. 诊断 is the last section and the danger+ // zone is the last thing in it, so the bottom *is* the confirm.+ if (clearChatConfirm || clearNotesConfirm || factoryConfirm) null else (scroll?.scrollY ?: 0)+ Glass.fadeOut(old) {+ if (root !== old) return@fadeOut+ fields.clear()+ picks.clear()+ val v = build()+ runCatching {+ v.hideFromAccessibility()+ val p = old.layoutParams as WindowManager.LayoutParams+ wm.removeView(old)+ wm.addView(v, p)+ root = v+ // After layout, or the scroller has no content to scroll over+ // and clamps the offset to zero.+ scroll?.let { sv ->+ sv.post { sv.scrollTo(0, offset ?: (sv.getChildAt(0)?.height ?: 0)) }+ }+ Glass.fadeIn(v)+ }+ }+ }++ /**+ * Preferred size, capped to what the screen actually has.+ *+ * **Both constants were tablet-phone numbers on a tablet.** 500×600 dp on+ * the 1600×2560 / 360dpi device this ships on is 711×1138 dp of screen, so+ * the card used about half the width and half the height — and it read as+ * small rather than as compact, because the form inside it had to shrink+ * its own type to fit rather than being given room.+ *+ * The width is now the screen minus a gutter, and the height the screen+ * minus room for the status bar and the gesture pill, both clamped again+ * by [cardWidth]/[cardHeight] against the real bounds so a phone is still+ * usable. The gutter is deliberately asymmetric — 20dp left, right, and+ * bottom — because the card is centred and the leftover lands on all four+ * sides anyway; the point is only that it does not touch the edge.+ */+ private fun cardWidth(): Int {+ val screen = wm.currentWindowMetrics.bounds.width()+ return minOf(screen - dp(2 * GUTTER_DP), (screen * 0.98f).toInt())+ }++ private fun cardHeight(): Int {+ val screen = wm.currentWindowMetrics.bounds.height()+ return minOf(screen - dp(2 * GUTTER_DP), (screen * 0.96f).toInt())+ }++ fun hide() {+ val v = root ?: return+ root = null+ closing = v+ fields.clear()+ picks.clear()+ scroll = null+ Glass.exit(v) {+ if (closing === v) {+ runCatching { wm.removeView(v) }+ closing = null+ OwnCard.hidden()+ }+ onDismiss()+ }+ }++ // ---- Build ----++ private fun build(): View {+ lctx = AppLocale.wrap(context)+ val card = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ background = Glass.card(context, dp(28), frosted)+ // So the sheen and the panels stay inside the rounded corners.+ clipToOutline = true+ }++ card.addView(header(), matchWrap())++ val sv = ScrollView(context).apply {+ isFillViewport = true+ // The card already has an edge; a scrollbar track on top of glass+ // is one line too many.+ isVerticalScrollBarEnabled = false+ }+ scroll = sv+ val form = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ // Was dp(18) all round, sized against a 500dp card. On a card that+ // now spans nearly the screen, the old padding makes every row look+ // like it lost its margin — and the vertical axis is the one that+ // matters, because the form is a scroll view and every dp of dead+ // space at the top is a dp the user scrolls past to see nothing.+ setPadding(dp(24), dp(4), dp(24), dp(24))+ }++ brainSection(form)+ voiceSection(form)+ notebookSection(form)+ wakeSection(form)+ typingSection(form)+ vaultSection(form)+ diagnosticsSection(form)++ sv.addView(form, matchWrap())+ card.addView(sv, LinearLayout.LayoutParams(MATCH, 0, 1f))++ card.addView(footerBar(), matchWrap())+ return card+ }++ @SuppressLint("SetTextI18n")+ private fun header(): View {+ val row = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ setPadding(dp(28), dp(22), dp(18), dp(14))+ }+ val titles = LinearLayout(context).apply { orientation = LinearLayout.VERTICAL }+ titles.addView(TextView(context).apply {+ text = lctx.getString(R.string.settings_title)+ // 21 was sized against the old, smaller card. On a card this wide+ // a 21sp title is the first thing that reads as undersized, and it+ // is the one line that has to carry the sheet's identity.+ textSize = 26f+ setTextColor(Color.parseColor(Glass.TITLE))+ letterSpacing = 0.01f+ })+ titles.addView(TextView(context).apply {+ text = saved["device_name"].orEmpty().ifEmpty { lctx.getString(R.string.settings_default_device_name) }+ textSize = 13f+ setTextColor(Color.parseColor(Glass.MUTED))+ setPadding(0, dp(4), 0, 0)+ })+ row.addView(titles, LinearLayout.LayoutParams(0, WRAP, 1f))+ row.addView(langPicker())+ row.addView(closeButton(), LinearLayout.LayoutParams(dp(36), dp(36)).apply {+ marginStart = dp(12)+ })+ return row+ }++ @SuppressLint("SetTextI18n")+ private fun closeButton(): View = TextView(context).apply {+ text = "✕"+ // 13dp of glyph in a 36dp circle, on a card the user can now see+ // properly. It was 13 in a 32dp circle, which is a target you have to+ // aim at; 15 in 36 is one you can hit without looking.+ textSize = 15f+ setTextColor(Color.parseColor(Glass.SECONDARY))+ gravity = Gravity.CENTER+ // A circle, so it reads as a button at a size where a bare glyph would+ // just look like punctuation that happens to be tappable.+ background = Glass.panel(context, dp(18))+ isClickable = true+ isFocusable = true+ Glass.pressable(this)+ setOnClickListener { hide() }+ }++ /**+ * 中文 / English, and the only picker that acts on the tap.+ *+ * Deliberately not in [picks]: it is already on disk by the time [rebuild]+ * runs, and re-writing it from 保存 would be the same value twice.+ */+ private fun langPicker(): View = segments(+ // Deliberately not resources. A language picker labels each language in+ // its own script, so 中 stays 中 and EN stays EN no matter what the card+ // is currently speaking — these two are the only strings in the app that+ // must NOT be translated, and translating them is the one way to make+ // the picker unusable.+ options = listOf(+ VoiceConfig.LANG_ZH to "中",+ VoiceConfig.LANG_EN to "EN",+ ),+ picked = if (english) VoiceConfig.LANG_EN else VoiceConfig.LANG_ZH,+ radius = dp(14),+ compact = true,+ ) { lang ->+ VoiceConfig.save(context, mapOf("lang" to lang))+ english = lang == VoiceConfig.LANG_EN+ // The overlays behind this card first, then this card. Both are cheap;+ // the order only matters so that the two are never briefly in different+ // languages while the cross-fade runs.+ onLocaleChanged()+ rebuild()+ }++ private fun footerBar(): View {+ val row = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ // END, not the old weight trick: the row used to carry+ // 恢复出厂设置 on the left with weight 1 to push this pair right.+ // The destructive buttons have moved into 诊断 (see [dangerZone]),+ // so there is nothing on the left to push against.+ gravity = Gravity.CENTER_VERTICAL or Gravity.END+ setPadding(dp(18), dp(10), dp(18), dp(18))+ }+ row.addView(button(lctx.getString(R.string.common_cancel), filled = false) { hide() })+ row.addView(View(context), LinearLayout.LayoutParams(dp(10), 1))+ row.addView(button(lctx.getString(R.string.common_save), filled = true) { saveAndHide() })+ return row+ }++ // ---- Sections ----++ /**+ * Who does the thinking, and how to reach them.+ *+ * The two halves are mutually exclusive on the service side too+ * (`ScreenBodyService.startBrain` builds one brain or the other), so the+ * picker folds away the half that is not in play.+ */+ private fun brainSection(form: LinearLayout) {+ val panel = group(+ form,+ lctx.getString(R.string.settings_section_brain),+ lctx.getString(R.string.settings_brain_panel_note),+ )++ // Assigned below, once both blocks exist; the picker's callback runs+ // only on a tap, so the null window closes before anyone can tap.+ var applyMode: ((String, Boolean) -> Unit)? = null+ val mode = segmented(+ panel,+ "brain",+ listOf(+ // Local leftmost because the leftmost option is the fallback+ // when the saved value matches nothing, and local is the+ // out-of-box default — see [VoiceConfig.brainConfig].+ VoiceConfig.BRAIN_LOCAL to lctx.getString(R.string.settings_brain_local),+ VoiceConfig.BRAIN_HUB to lctx.getString(R.string.settings_brain_cloud),+ ),+ ) { applyMode?.invoke(it, true) }++ val hubBlock = block(panel)+ field(+ hubBlock,+ "hub_url",+ lctx.getString(R.string.settings_hub_url),+ hint = lctx.getString(R.string.settings_hub_url_hint),+ )+ field(+ hubBlock,+ "device_name",+ lctx.getString(R.string.settings_device_name),+ hint = lctx.getString(R.string.settings_device_name_hint),+ )+ note(+ hubBlock,+ lctx.getString(R.string.settings_brain_cloud_note),+ )++ val localBlock = block(panel)+ field(localBlock, "llm_api_key", "DeepSeek Key", hint = "sk-…", secret = true)+ field(localBlock, "llm_model", lctx.getString(R.string.settings_llm_model), hint = VoiceConfig.DEFAULT_LLM_MODEL)+ field(+ localBlock,+ "llm_base_url",+ lctx.getString(R.string.settings_llm_base_url),+ hint = VoiceConfig.DEFAULT_LLM_BASE_URL,+ )++ rowLabel(localBlock, lctx.getString(R.string.settings_llm_thinking))+ var applyThinking: ((String, Boolean) -> Unit)? = null+ val thinking = segmented(+ localBlock,+ "llm_thinking",+ listOf("enabled" to lctx.getString(R.string.common_on), "disabled" to lctx.getString(R.string.common_off)),+ ) { applyThinking?.invoke(it, true) }++ val effortBlock = block(localBlock)+ rowLabel(effortBlock, lctx.getString(R.string.settings_llm_effort))+ segmented(+ effortBlock,+ "llm_reasoning_effort",+ listOf(+ "high" to lctx.getString(R.string.common_high),+ "medium" to lctx.getString(R.string.common_medium),+ "low" to lctx.getString(R.string.common_low),+ ),+ )+ note(+ localBlock,+ lctx.getString(R.string.settings_brain_local_note),+ )++ applyMode = { m, animate ->+ val local = m == VoiceConfig.BRAIN_LOCAL+ // Both at once, not one after the other: one block shrinking while+ // the other grows keeps the card's total height almost still, and+ // sequencing them makes the whole form jump twice.+ Glass.setVisible(hubBlock, !local, animate)+ Glass.setVisible(localBlock, local, animate)+ }+ applyMode(mode, false)+ applyThinking = { th, animate ->+ Glass.setVisible(effortBlock, th != "disabled", animate)+ }+ applyThinking(thinking, false)+ }++ /**+ * The wake phrase is taught by saying it, not by typing it. It lives here+ * rather than on the ball because it is a once-per-device chore — see+ * [WakeEnrollUi].+ */+ private fun wakeSection(form: LinearLayout) {+ val panel = group(form, lctx.getString(R.string.settings_section_wake))+ fun summary(): String =+ if (WakeTemplates.enrolled(context)) {+ lctx.getString(R.string.settings_wake_enrolled)+ } else {+ lctx.getString(R.string.settings_wake_not_enrolled)+ }++ val state = TextView(context).apply {+ text = summary()+ textSize = 13f+ setTextColor(Color.parseColor(Glass.LABEL))+ setLineSpacing(dp(3).toFloat(), 1f)+ }+ panel.addView(state, rowParams(0))++ val row = LinearLayout(context).apply { orientation = LinearLayout.HORIZONTAL }+ row.addView(button(lctx.getString(R.string.settings_wake_record), filled = false) {+ WakeEnrollUi(context) { state.text = summary() }.show()+ })+ row.addView(View(context), LinearLayout.LayoutParams(dp(8), 1))+ row.addView(button(lctx.getString(R.string.common_clear), filled = false) {+ WakeTemplates.clear(context)+ WakeWord.reload()+ state.text = summary()+ })+ panel.addView(row, rowParams(dp(12)))+ }++ /**+ * Same shape as [wakeSection], and for the same reason: the real screen is+ * its own overlay ([VaultUi]) because the list grows and this card is+ * already full. Only the count and the door belong here.+ */+ private fun vaultSection(form: LinearLayout) {+ val panel = group(form, lctx.getString(R.string.settings_section_vault))+ fun summary(): String {+ val n = Vault.entries(context).size+ return if (n == 0) {+ lctx.getString(R.string.settings_vault_empty_note)+ } else {+ lctx.getString(R.string.settings_vault_count_note, n)+ }+ }++ val state = TextView(context).apply {+ text = summary()+ textSize = 13f+ setTextColor(Color.parseColor(Glass.LABEL))+ setLineSpacing(dp(3).toFloat(), 1f)+ }+ panel.addView(state, rowParams(0))++ val row = LinearLayout(context).apply { orientation = LinearLayout.HORIZONTAL }+ row.addView(button(lctx.getString(R.string.settings_vault_manage), filled = false) {+ VaultUi(context) { state.text = summary() }.show()+ })+ panel.addView(row, rowParams(dp(12)))+ }++ /** 火山 endpoints. Collapsed, because they ship working and rarely move. */+ private fun voiceSection(form: LinearLayout) {+ val panel = collapsible(+ form,+ lctx.getString(R.string.settings_section_voice),+ lctx.getString(R.string.settings_voice_factory_note),+ )+ // The one overridable credential. Blank means *factory*, which is what+ // an untouched box shows — see VoiceConfig.save / OVERRIDE_KEYS. The key+ // is masked like any other credential.+ field(+ panel, "api_key",+ lctx.getString(R.string.settings_voice_api_key),+ hint = lctx.getString(R.string.settings_voice_api_key_hint),+ secret = true,+ )+ // Feedback for the row above. Without it, "did my key actually take+ // effect?" is only answerable by reading shared_prefs over adb.+ readOnly(+ panel,+ lctx.getString(R.string.settings_voice_in_use),+ if (VoiceConfig.usingOwnCredentials(context)) {+ lctx.getString(R.string.settings_voice_in_use_own)+ } else {+ lctx.getString(R.string.settings_voice_in_use_factory)+ },+ )+ field(panel, "asr_endpoint", lctx.getString(R.string.settings_voice_asr_endpoint))+ field(panel, "asr_resource_id", lctx.getString(R.string.settings_voice_asr_resource_id))+ field(panel, "tts_endpoint", lctx.getString(R.string.settings_voice_tts_endpoint))+ field(panel, "tts_resource_id", lctx.getString(R.string.settings_voice_tts_resource_id))+ field(panel, "tts_sample_rate", lctx.getString(R.string.settings_voice_tts_sample_rate), numeric = true)+ }++ private fun diagnosticsSection(form: LinearLayout) {+ val panel = group(form, lctx.getString(R.string.settings_section_diagnostics))+ rowLabel(panel, lctx.getString(R.string.settings_grounding))+ // 关 first, and not for looks: the leftmost option is what an+ // unrecognised saved value falls back to, and this one writes pictures+ // of whatever the user is doing. Same direction as+ // VoiceConfig.groundingEnabled, which treats anything but "on" as off.+ segmented(+ panel,+ "grounding",+ listOf("off" to lctx.getString(R.string.common_off), "on" to lctx.getString(R.string.common_on)),+ )+ note(+ panel,+ lctx.getString(R.string.settings_grounding_note),+ )+ readOnly(+ panel,+ lctx.getString(R.string.settings_device_id),+ saved["device_id"].orEmpty().ifEmpty { "—" },+ )+ dangerZone(panel)+ }++ /**+ * The three destructive actions, together at the foot of 诊断.+ *+ * They used to live in the footer beside 取消/保存, and that was the+ * problem: a destructive control sitting next to the card's two ways out+ * reads as a third way out. Here they sit under the section that is+ * already about the device's stored state, and each replaces this row+ * with its own warning before it does anything — the confirm appears+ * where the tap landed, not down in the footer where the button no+ * longer is.+ *+ * Ordered by how much they take: this conversation, everything it ever+ * chose to remember, then the device itself.+ */+ private fun dangerZone(panel: LinearLayout) {+ when {+ clearChatConfirm -> {+ panel.addView(confirmTitle(lctx.getString(R.string.settings_clear_chat_confirm_title)))+ panel.addView(confirmBody(lctx.getString(R.string.settings_clear_chat_confirm_body)))+ panel.addView(confirmRow(+ back = { clearChatConfirm = false; rebuild() },+ confirmLabel = lctx.getString(R.string.settings_clear_chat_confirm),+ ) {+ onClearChat()+ hide()+ })+ }+ clearNotesConfirm -> {+ panel.addView(confirmTitle(lctx.getString(R.string.settings_clear_notes_confirm_title)))+ panel.addView(confirmBody(lctx.getString(R.string.settings_clear_notes_confirm_body)))+ panel.addView(confirmRow(+ back = { clearNotesConfirm = false; rebuild() },+ confirmLabel = lctx.getString(R.string.settings_clear_notes_confirm),+ ) {+ onClearNotes()+ hide()+ })+ }+ factoryConfirm -> {+ panel.addView(confirmTitle(lctx.getString(R.string.settings_factory_reset_confirm_title)))+ // The list comes from [FactoryReset] itself rather than being+ // typed here — the same list the wipe walks. What the user+ // agrees to and what the code does are therefore the same+ // thing by construction, which is the only arrangement+ // acceptable for an irreversible action.+ panel.addView(confirmBody(+ lctx.getString(R.string.settings_factory_reset_list_intro) + "\n" ++ FactoryReset.items(lctx).joinToString("\n") { "· $it" } + "\n" ++ lctx.getString(R.string.settings_factory_reset_after),+ ))+ panel.addView(confirmRow(+ back = { factoryConfirm = false; rebuild() },+ confirmLabel = lctx.getString(R.string.settings_factory_reset_confirm),+ ) {+ onFactoryReset()+ hide()+ })+ }+ else -> {+ // Stacked, not side by side. Half the panel leaves ~210px of+ // text room at this density, which is not enough for "Clear+ // chat history" — it wrapped to three lines and the pair came+ // out taller than two full-width buttons. Full width also+ // stops the layout depending on how long the label is in+ // whichever language the card is in.+ val col = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ setPadding(0, dp(14), 0, 0)+ }+ fun danger(@StringRes label: Int, arm: () -> Unit) {+ if (col.childCount > 0) {+ col.addView(View(context), LinearLayout.LayoutParams(1, dp(10)))+ }+ col.addView(+ button(lctx.getString(label), filled = false, danger = true) { arm() },+ matchWrap(),+ )+ }+ danger(R.string.settings_clear_chat) { clearChatConfirm = true; rebuild() }+ danger(R.string.settings_clear_notes) { clearNotesConfirm = true; rebuild() }+ danger(R.string.settings_factory_reset) { factoryConfirm = true; rebuild() }+ panel.addView(col, matchWrap())+ }+ }+ }++ /** The bold red line at the head of a destructive confirm. */+ private fun confirmTitle(text: String): TextView = TextView(context).apply {+ this.text = text+ textSize = 14f+ setTextColor(Color.parseColor(Glass.DANGER))+ letterSpacing = 0.02f+ }++ /** The grey explanation under it. [text] is pre-joined by the caller. */+ private fun confirmBody(text: String): TextView = TextView(context).apply {+ this.text = text+ textSize = 12f+ setTextColor(Color.parseColor(Glass.MUTED))+ setLineSpacing(dp(4).toFloat(), 1f)+ setPadding(0, dp(8), 0, 0)+ }++ /**+ * [不,返回] above [确认…] — back out is never the red button.+ *+ * Stacked and full width, like the two buttons this replaces: side by side+ * the pair overflowed the panel and "Erase everything" broke mid-word.+ */+ private fun confirmRow(+ back: () -> Unit,+ confirmLabel: String,+ confirm: () -> Unit,+ ): View {+ val col = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ setPadding(0, dp(14), 0, 0)+ }+ col.addView(+ button(lctx.getString(R.string.settings_factory_reset_back), filled = false) { back() },+ matchWrap(),+ )+ col.addView(View(context), LinearLayout.LayoutParams(1, dp(10)))+ col.addView(button(confirmLabel, filled = false, danger = true) { confirm() }, matchWrap())+ return col+ }++ /**+ * The notebook: what this device remembers about you, and what it owes you.+ *+ * Three things only — the switch for the quiet-hour review, its cost, and a+ * way to read the book. There is deliberately no editing, no searching and+ * no deleting here: the model keeps the notebook, and the one power a person+ * needs over a device that claims to remember them is **to be able to see+ * what it wrote down**.+ */+ private fun notebookSection(form: LinearLayout) {+ val panel = group(form, lctx.getString(R.string.settings_section_notebook))+ // On first, matching VoiceConfig.sweepConfig's fail-open reading: this+ // is the feature that makes the device feel like it knows you, and+ // turning it off is a deliberate act.+ segmented(+ panel,+ "sweep",+ listOf("on" to lctx.getString(R.string.common_on), "off" to lctx.getString(R.string.common_off)),+ )+ note(+ panel,+ lctx.getString(R.string.settings_notebook_sweep_note),+ )+ field(+ panel,+ "sweep_quiet_minutes",+ lctx.getString(R.string.settings_notebook_quiet_minutes),+ hint = VoiceConfig.DEFAULT_SWEEP_QUIET_MINUTES.toString(),+ numeric = true,+ )+ field(+ panel,+ "sweep_daily_cap",+ lctx.getString(R.string.settings_notebook_daily_cap),+ hint = VoiceConfig.DEFAULT_SWEEP_DAILY_CAP.toString(),+ numeric = true,+ )++ // The bill, such as it is. These calls are paid for with the user's own+ // key, so the count and the spend are shown rather than hidden.+ val used = Notebook.sweepCountToday(context)+ val spent = Notebook.sweepTokensToday(context)+ val remembered = Notebook.memories(context).size+ val open = Notebook.todos(context, "pending").size+ readOnly(+ panel,+ lctx.getString(R.string.settings_notebook_today),+ lctx.getString(+ R.string.settings_notebook_stats,+ used, spent, remembered, open,+ ),+ )++ val dump = collapsible(+ form,+ lctx.getString(R.string.settings_notebook_dump),+ lctx.getString(R.string.settings_notebook_dump_hint),+ isOpen = { notebookOpen },+ setOpen = { notebookOpen = it },+ )+ dump.addView(notebookDumpView(), matchWrap())+ }++ /**+ * The book itself, as text.+ *+ * Truncated at [NOTEBOOK_DUMP_CHARS] rather than paginated: this is a+ * TextView inside a scroll view inside a floating card, and a few thousand+ * lines of text would make every layout pass of that card expensive. The+ * truncation says so in the last line — a silent cut-off would read as "this+ * is everything".+ */+ private fun notebookDumpView(): TextView {+ val full = Notebook.dumpText(context)+ val body = if (full.length <= NOTEBOOK_DUMP_CHARS) {+ full+ } else {+ full.take(NOTEBOOK_DUMP_CHARS) ++ "\n\n" ++ lctx.getString(+ R.string.settings_notebook_dump_truncated,+ full.length - NOTEBOOK_DUMP_CHARS,+ )+ }+ return TextView(context).apply {+ text = body+ textSize = 12f+ typeface = android.graphics.Typeface.MONOSPACE+ setTextColor(Color.parseColor(Glass.SECONDARY))+ setLineSpacing(dp(3).toFloat(), 1f)+ setTextIsSelectable(true)+ }+ }++ // ---- Widgets ----++ /** Section title + its panel. Returns the panel; rows go in there. */+ private fun group(form: LinearLayout, title: String, note: String? = null): LinearLayout {+ form.addView(sectionTitle(title), matchWrap())+ if (note != null) form.addView(noteView(note), matchWrap())+ val panel = panel()+ form.addView(panel, rowParams(dp(10)))+ return panel+ }++ /**+ * The keyboard, and the one thing about it worth telling the user.+ *+ * Nothing to configure. The device's input method is the user's own — the+ * one they can type on in any app — and the brain borrows ADBKeyboard only+ * inside a single `type_text` call. So this section exists to say that out+ * loud (it is the answer to "why does my keyboard keep changing?"), and to+ * name the one state where nothing can be typed at all: a tablet whose+ * single IME is the headless one. See [net.kuafuai.andee.ui.ImeSwitch].+ */+ private fun typingSection(form: LinearLayout) {+ val panel = group(form, lctx.getString(R.string.settings_section_typing))+ val human = net.kuafuai.andee.ui.ImeSwitch.typable(context)+ if (human.isEmpty()) {+ note(+ panel,+ lctx.getString(R.string.settings_typing_no_ime),+ )+ return+ }+ val now = net.kuafuai.andee.ui.ImeSwitch.current(context)+ readOnly(+ panel,+ lctx.getString(R.string.settings_typing_current),+ net.kuafuai.andee.ui.ImeSwitch.label(context, now),+ )+ note(+ panel,+ lctx.getString(R.string.settings_typing_note),+ )+ }++ /**+ * As [group], but the panel folds away behind a chevron.+ *+ * The chevron rotates rather than swapping between 展开 / 收起, which also+ * spares the pair of strings: a quarter turn means the same thing in both+ * languages.+ *+ * @param isOpen / @param setOpen which flag remembers the state. Parameterised+ * so two foldable panels can exist without one unfolding the other.+ */+ private fun collapsible(+ form: LinearLayout,+ title: String,+ note: String,+ isOpen: () -> Boolean = { advancedOpen },+ setOpen: (Boolean) -> Unit = { advancedOpen = it },+ ): LinearLayout {+ val head = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ isClickable = true+ }+ head.addView(sectionTitle(title), LinearLayout.LayoutParams(0, WRAP, 1f))+ val chevron = TextView(context).apply {+ text = "›"+ textSize = 17f+ setTextColor(Color.parseColor(Glass.ACCENT))+ gravity = Gravity.CENTER+ setPadding(dp(10), 0, dp(4), 0)+ rotation = if (isOpen()) 90f else 0f+ }+ head.addView(chevron)+ form.addView(head, matchWrap())+ form.addView(noteView(note), matchWrap())++ val panel = panel().apply {+ visibility = if (isOpen()) View.VISIBLE else View.GONE+ }+ form.addView(panel, rowParams(dp(10)))+ head.setOnClickListener {+ val next = !isOpen()+ setOpen(next)+ Glass.setVisible(panel, next, animate = true)+ chevron.animate()+ .rotation(if (next) 90f else 0f)+ .setDuration(240)+ .setInterpolator(Glass.EASE)+ .start()+ }+ return panel+ }++ private fun sectionTitle(title: String): TextView = TextView(context).apply {+ text = title+ textSize = 13f+ setTextColor(Color.parseColor(Glass.ACCENT))+ letterSpacing = 0.08f+ setPadding(0, dp(22), 0, dp(8))+ }++ private fun panel(): LinearLayout = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ setPadding(dp(18), dp(18), dp(18), dp(18))+ background = Glass.panel(context, dp(20))+ }++ /** A bare container, used only so a group of rows can be folded at once. */+ private fun block(parent: LinearLayout): LinearLayout {+ val v = LinearLayout(context).apply { orientation = LinearLayout.VERTICAL }+ parent.addView(v, matchWrap())+ return v+ }++ private fun rowLabel(parent: LinearLayout, text: String) {+ parent.addView(TextView(context).apply {+ this.text = text+ // 12 was legible only because the card was small enough to read at+ // a glance. It is now the smallest thing on a much larger surface.+ textSize = 13f+ setTextColor(Color.parseColor(Glass.LABEL))+ }, rowParams(dp(16)))+ }++ private fun note(parent: LinearLayout, text: String) {+ parent.addView(noteView(text), rowParams(dp(10)))+ }++ private fun noteView(text: String): TextView = TextView(context).apply {+ this.text = text+ // Same reason as [rowLabel]. Notes are the longest strings in the card+ // and the ones a user actually reads, so they can afford the space+ // least of all — but at 11sp on a 690dp-wide card they end up as one+ // line each, which is both harder to read and denser-looking than the+ // rows they sit between.+ textSize = 12.5f+ setTextColor(Color.parseColor(Glass.MUTED))+ setLineSpacing(dp(4).toFloat(), 1f)+ }++ private fun readOnly(parent: LinearLayout, title: String, value: String) {+ rowLabel(parent, title)+ parent.addView(TextView(context).apply {+ text = value+ textSize = 14.5f+ setTextColor(Color.parseColor(Glass.SECONDARY))+ }, rowParams(dp(4)))+ }++ /**+ * One text row.+ *+ * @param secret renders masked with a 显示 / Show toggle. The key is the+ * user's own and this card is an overlay on top of whatever they are+ * doing, so it has no business printing it at full size by default.+ */+ private fun field(+ parent: LinearLayout,+ key: String,+ label: String,+ hint: String? = null,+ numeric: Boolean = false,+ secret: Boolean = false,+ ) {+ rowLabel(parent, label)+ val et = EditText(context).apply {+ background = Glass.well(context, dp(13))+ setPadding(dp(14), dp(13), dp(14), dp(13))+ setTextColor(Color.parseColor(Glass.TITLE))+ setHintTextColor(Color.parseColor(Glass.MUTED))+ // A bigger card made every row a shorter-looking row: 14sp in a+ // full-width well on a 690dp card is a lot of empty space holding+ // one line of text. Sizing the field up is what fills that space+ // with something rather than just making it emptier.+ textSize = 15f+ if (hint != null) setHint(hint)+ // isSingleLine before the mask, as in VaultUi: the other order lets+ // it undo the password transformation and the dots come out as text.+ isSingleLine = true+ inputType = when {+ numeric -> InputType.TYPE_CLASS_NUMBER+ secret -> secretType(numeric = false)+ else -> InputType.TYPE_CLASS_TEXT+ }+ if (secret) setMasked(true)+ setText(saved[key].orEmpty())+ }+ fields[key] = et++ if (!secret) {+ parent.addView(et, rowParams(dp(5)))+ return+ }+ val row = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ }+ row.addView(et, LinearLayout.LayoutParams(0, WRAP, 1f))+ var shown = false+ row.addView(TextView(context).apply {+ text = lctx.getString(R.string.common_show)+ textSize = 13f+ setTextColor(Color.parseColor(Glass.ACCENT))+ setPadding(dp(14), dp(8), dp(2), dp(8))+ isClickable = true+ setOnClickListener {+ shown = !shown+ et.setMasked(!shown)+ et.setSelection(et.text?.length ?: 0)+ text = if (shown) lctx.getString(R.string.common_hide) else lctx.getString(R.string.common_show)+ }+ })+ parent.addView(row, rowParams(dp(5)))+ }++ /**+ * A full-width segmented picker for a setting with a handful of legal values.+ *+ * @param options preference value to the label shown for it; the first is+ * the fallback when nothing is saved or the saved string is not one of+ * them — which is the same fail-closed rule [VoiceConfig.brainConfig] and+ * [VoiceConfig.groundingEnabled] apply when reading.+ * @return the value it starts on, so the caller can set up whatever depends+ * on it without re-deriving the fallback.+ */+ private fun segmented(+ parent: LinearLayout,+ key: String,+ options: List<Pair<String, String>>,+ onPick: (String) -> Unit = {},+ ): String {+ val current = saved[key].orEmpty().trim().lowercase()+ val start = options.firstOrNull { it.first == current }?.first ?: options.first().first+ var picked = start+ val track = segments(options, start, dp(13)) { picked = it; onPick(it) }+ picks[key] = { picked }+ parent.addView(track, rowParams(dp(7)))+ return start+ }++ /**+ * The picker itself, and the one control here with real motion in it.+ *+ * A single **thumb** slides between the cells instead of each cell swapping+ * its own background. That is the whole reason this is a `FrameLayout` with+ * a free-floating child rather than a row of styled `TextView`s: a+ * background that blinks from one cell to another reads as two separate+ * things happening, while one pane moving reads as *this control* changing+ * value — which is what actually happened.+ *+ * The thumb is positioned from the cells' measured geometry rather than from+ * weights, so the same code serves the full-width pickers (equal cells) and+ * the compact language pair (中 is narrower than EN). Placement therefore+ * has to wait for layout, and re-runs on every later layout pass — the card+ * resizes when the IME opens.+ */+ private fun segments(+ options: List<Pair<String, String>>,+ picked: String,+ radius: Int,+ compact: Boolean = false,+ onPick: (String) -> Unit,+ ): View {+ val inset = dp(3)+ val track = FrameLayout(context).apply {+ setPadding(inset, inset, inset, inset)+ background = Glass.well(context, radius)+ }+ val thumb = View(context).apply {+ background = Glass.thumb(context, radius - inset)+ }+ // 1×1, and sized from the cell in place() — *not* MATCH_PARENT. A plain+ // View measured against an AT_MOST spec reports the whole spec back, so+ // a MATCH_PARENT thumb in this WRAP_CONTENT track made the track as tall+ // as the space offered it: the header then ate the card and the form's+ // weighted ScrollView got nothing.+ track.addView(thumb, FrameLayout.LayoutParams(1, 1))+ val cellRow = LinearLayout(context).apply { orientation = LinearLayout.HORIZONTAL }+ track.addView(cellRow, FrameLayout.LayoutParams(MATCH, WRAP))++ val cells = mutableListOf<TextView>()+ val onColor = Color.parseColor(Glass.TITLE)+ val offColor = Color.parseColor(Glass.SECONDARY)+ var index = options.indexOfFirst { it.first == picked }.coerceAtLeast(0)+ var sliding = false++ fun place(to: Int, animate: Boolean) {+ val cell = cells.getOrNull(to) ?: return+ if (cell.width == 0 || cell.height == 0) {+ // Not laid out yet. View.post before attach queues until it is.+ track.post { place(to, false) }+ return+ }+ val toX = cell.left.toFloat()+ val toY = cell.top.toFloat()+ val toW = cell.width+ val toH = cell.height+ val lp = thumb.layoutParams+ if (!animate) {+ // The guard matters: this runs from a layout listener, and+ // requesting layout from inside one with nothing to change is+ // how a layout loop starts.+ if (lp.width == toW && lp.height == toH && thumb.translationX == toX) return+ lp.width = toW+ lp.height = toH+ thumb.layoutParams = lp+ thumb.translationX = toX+ thumb.translationY = toY+ for ((i, c) in cells.withIndex()) c.setTextColor(if (i == to) onColor else offColor)+ return+ }+ val fromX = thumb.translationX+ val fromW = lp.width+ val fromColors = cells.map { it.currentTextColor }+ sliding = true+ // Height does not animate: every cell in a track is the same height,+ // so only x and width have anywhere to travel.+ lp.height = toH+ thumb.translationY = toY+ ValueAnimator.ofFloat(0f, 1f).apply {+ duration = 260+ interpolator = Glass.EASE+ addUpdateListener { a ->+ val f = a.animatedFraction+ thumb.translationX = fromX + (toX - fromX) * f+ lp.width = (fromW + (toW - fromW) * f).toInt()+ thumb.layoutParams = lp+ for ((i, c) in cells.withIndex()) {+ val target = if (i == to) onColor else offColor+ c.setTextColor(Glass.lerpColor(fromColors[i], target, f))+ }+ }+ addListener(object : AnimatorListenerAdapter() {+ override fun onAnimationEnd(a: Animator) {+ sliding = false+ }+ })+ start()+ }+ }++ for ((i, option) in options.withIndex()) {+ val (value, label) = option+ val cell = TextView(context).apply {+ text = label+ textSize = if (compact) 12f else 13f+ gravity = Gravity.CENTER+ setTextColor(if (i == index) onColor else offColor)+ if (compact) setPadding(dp(12), dp(6), dp(12), dp(6))+ else setPadding(dp(6), dp(10), dp(6), dp(10))+ isClickable = true+ setOnClickListener {+ if (index == i) return@setOnClickListener+ index = i+ place(i, animate = true)+ onPick(value)+ }+ }+ cells += cell+ if (compact) cellRow.addView(cell)+ else cellRow.addView(cell, LinearLayout.LayoutParams(0, WRAP, 1f))+ }++ cellRow.addOnLayoutChangeListener { _, _, _, _, _, _, _, _, _ ->+ if (!sliding) place(index, animate = false)+ }+ place(index, animate = false)+ return track+ }++ private fun button(+ label: String, filled: Boolean, danger: Boolean = false, onClick: () -> Unit,+ ): TextView {+ val radius = dp(24)+ return TextView(context).apply {+ text = label+ // The footer's buttons were the card's primary targets and were+ // sized for a card narrower than this one. Larger type and more+ // padding is what makes 保存 / 取消 hit-or-miss avoidable on a+ // tablet held in two hands, which is how it is actually used.+ textSize = 15f+ setTextColor(+ Color.parseColor(+ if (danger) Glass.DANGER else if (filled) Glass.TITLE else Glass.LABEL,+ ),+ )+ gravity = Gravity.CENTER+ background =+ if (filled) Glass.tinted(context, radius) else Glass.panel(context, radius)+ setPadding(dp(26), dp(13), dp(26), dp(13))+ isClickable = true+ isFocusable = true+ Glass.pressable(this)+ setOnClickListener { onClick() }+ }+ }++ // ---- Data ----++ /** What is on screen right now, saved or not. Used by [rebuild]. */+ private fun currentEdits(): Map<String, String> {+ val out = mutableMapOf<String, String>()+ for ((k, et) in fields) out[k] = et.text?.toString().orEmpty()+ for ((k, read) in picks) out[k] = read()+ return out+ }++ private fun saveAndHide() {+ val updates = mutableMapOf<String, String>()+ for ((k, et) in fields) {+ val v = et.text?.toString()?.trim().orEmpty()+ if (v.isNotEmpty()) updates[k] = v+ }+ // Pickers go in unconditionally: their value is never blank, which is+ // what lets a setting actually be turned back off — VoiceConfig.save()+ // drops blank values, so a text field never could.+ for ((k, read) in picks) updates[k] = read()+ if (updates.isNotEmpty()) VoiceConfig.save(context, updates)+ hide()+ }++ private fun matchWrap() = LinearLayout.LayoutParams(MATCH, WRAP)++ private fun rowParams(top: Int) = LinearLayout.LayoutParams(MATCH, WRAP).apply {+ topMargin = top+ }++ private fun dp(v: Int): Int = Glass.dp(context, v)++ companion object {+ /**+ * The gap between the card and the screen edge, each side.+ *+ * The screen, not a fixed width, is what the card is sized from — see+ * [cardWidth]. What this constant buys is the *feel*: a card flush to+ * the edge reads as a system dialog hijacking the screen, while one+ * with a margin reads as ours floating above it.+ */+ private const val GUTTER_DP = 20++ private const val MATCH = LinearLayout.LayoutParams.MATCH_PARENT+ private const val WRAP = LinearLayout.LayoutParams.WRAP_CONTENT+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/Stage.kt b/app/src/main/java/net/kuafuai/andee/ui/Stage.ktnew file mode 100644index 0000000..42054c4--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/Stage.kt@@ -0,0 +1,230 @@+package net.kuafuai.andee.ui++import android.os.Bundle+import android.os.Handler+import android.os.Looper+import android.view.ViewGroup+import android.widget.FrameLayout+import androidx.appcompat.app.AppCompatActivity+import net.kuafuai.andee.R++/**+ * The one full-screen face this app puts in front of the user.+ *+ * Three surfaces open fullscreen — a page the model composed (`HtmlActivity`),+ * a message read in full (`TextPage`, same activity), and the two camera+ * screens (`LookActivity`, `ScanActivity`) — and until now each of them+ * arrived and left on its own terms. The user could see the difference: the+ * camera screens cut, a page faded, and closing any of them flashed once+ * before settling.+ *+ * ### The flash, and the two separate bugs behind it+ *+ * Reported as one symptom ("关闭的时候页面闪烁一下") and it was two:+ *+ * **1. Nothing handed the screen back.** Closing the page ran `finish()` and+ * nothing else; the card and the ball were only restored from `onDestroy`,+ * by which time the window was already gone and the app underneath was+ * visible — so the user watched a finished page vanish and *then* watched+ * the assistant assemble itself. [onStageLeaving] fixes that half by firing+ * the hand-back at the *start* of the exit, so the card and the ball are+ * already moving back underneath while this window shrinks over them.+ *+ * **2. The exit animated the wrong thing.** `stage`'s own alpha was taken to+ * zero, which is where the black came from: a View tree going transparent+ * reveals this Activity's `windowBackground`, not the app behind it. The+ * measured frame trace was ~130 ms of fading page, then **~250 ms of solid+ * black**, then the app behind. No View animation can fix that — the window+ * background is not a View. So the exit moved to the window, via+ * `R.anim.stage_close_exit` in [reallyFinish]; see that file for the trace.+ *+ * ### The enter is the same shape, for the same reason+ *+ * [revealStage] grows the stage from [SCALE_IN] rather than fading it up from+ * zero, so opening reads as *this thing comes toward you* instead of *the+ * screen changes*. Subclasses call it when their content is actually+ * drawable — a page has finished loading, a lens has bound — and the backstop+ * in [onCreate] covers the ones that never report.+ *+ * The enter stays a View animation, and that asymmetry is deliberate: on the+ * way in there is nothing behind the window worth revealing (it is the top+ * surface, and `Theme.Body.Stage` painted the same black), so the cheap+ * animation is also the correct one.+ */+abstract class StageActivity : AppCompatActivity() {++ /**+ * The whole window's content, and the thing that grows on the way in.+ *+ * Subclasses add their views here rather than calling `setContentView`, so+ * the entering animation moves *everything* at once — a view parented+ * outside the stage would sit at full size while the rest of the screen+ * was still growing into place.+ */+ protected val stage: FrameLayout by lazy { FrameLayout(this) }++ private val main = Handler(Looper.getMainLooper())++ private var revealed = false++ /**+ * Set by [leave] so a second ✕ tap cannot start a second exit.+ *+ * A double tap on the pill is easy — it is bottom-of-the-screen chrome and+ * the exit takes a couple of hundred milliseconds. Without this, the+ * second tap runs `onStageLeaving` again, and the caller's hand-back is not+ * idempotent in the way that matters: it unperches the ball and unfolds+ * the card twice, so the ball ends up halfway to the ledge on an animation+ * that was restarted under it.+ */+ private var leaving = false++ /**+ * Called once, the instant [leave] starts, while this window is still on+ * screen. Put the overlay back from here.+ *+ * This is a different moment from `onDestroy` and it is not a stylistic+ * choice: the caller's job here is to restore the card and unperch the+ * ball, both of which are *animations of their own* (see+ * `FloatingWindowUi.slideTo` and `exitCompact`). Kicked off at+ * `onDestroy` they run against a screen that is already showing the app+ * underneath — so the user watches a finished page vanish, then watches+ * the assistant assemble itself from nothing. Kicked off here they run+ * underneath the shrinking window and are simply revealed by it.+ */+ protected open fun onStageLeaving() {}++ /**+ * Called once, just before [revealStage] grows the stage, for whatever the+ * subclass has to put away *inside* that growth.+ *+ * A spinner is the reason this hook exists. It is the one piece of chrome+ * a subclass adds that is wrong the moment its content is ready, and+ * fading it on its own clock means the user sees it pop out of an+ * already-full-size stage. It has to be part of the same gesture.+ */+ protected open fun onStageRevealing() {}++ override fun onCreate(savedInstanceState: Bundle?) {+ super.onCreate(savedInstanceState)+ stage.setBackgroundColor(BACKDROP)+ stage.layoutParams = ViewGroup.LayoutParams(+ ViewGroup.LayoutParams.MATCH_PARENT,+ ViewGroup.LayoutParams.MATCH_PARENT,+ )+ stage.scaleX = SCALE_IN+ stage.scaleY = SCALE_IN+ stage.alpha = 0f+ setContentView(stage)+ // Not for the fast case — for the ones that never report. A page whose+ // top-level script throws leaves `onPageFinished` pending forever, and+ // a lens on a device where the bind is refused never draws a frame;+ // either way a half-drawn stage beats a black screen.+ main.postDelayed({ revealStage() }, REVEAL_BACKSTOP_MS)+ }++ /**+ * Content is drawable: grow the stage into place. Idempotent, and safe to+ * call from any thread that can post to the main one.+ */+ fun revealStage() {+ main.post {+ if (revealed || leaving) return@post+ revealed = true+ onStageRevealing()+ stage.animate().cancel()+ stage.animate()+ .scaleX(1f).scaleY(1f).alpha(1f)+ .setDuration(ENTER_MS)+ .setInterpolator(Glass.EASE)+ .start()+ }+ }++ /**+ * Shrink away, then close. The only way any of these screens may leave.+ *+ * **This hands the exit to the window, not to the View tree**, and that is+ * the whole fix for the flash the user reported. Animating `stage`'s own+ * alpha looks identical right up until it reaches zero — at which point+ * what is showing is not the app underneath but this Activity's own+ * `windowBackground` (black, see `Theme.Body.Stage`). Measured frame by+ * frame on the tablet: ~130 ms of the page fading, then **~250 ms of solid+ * black**, then the app behind fades in. A View animation cannot reach the+ * window background because it is not a View; only a window-level+ * transition can take the content and that black plate away together, and+ * it is what lets what is behind show through *during* the animation+ * instead of after it. See `anim/stage_close_exit.xml`.+ *+ * The hand-back ([onStageLeaving]) still fires first and synchronously, so+ * the card and the ball are already on their way back underneath while+ * this window shrinks over them.+ *+ * A back press routes through here too — see [onBackPressed].+ */+ fun leave() {+ main.post {+ if (leaving) return@post+ leaving = true+ // Hand the screen back *now*, not after the animation — see+ // onStageLeaving. This is also what makes the shrinking window+ // reveal something real.+ onStageLeaving()+ reallyFinish()+ }+ }++ /**+ * Close for real, with the window's own exit animation.+ *+ * `overridePendingTransition`'s second argument is the animation for *this*+ * window leaving, which after `finish()` is exactly what is about to+ * happen. It beats `Theme.Body.Stage`'s `windowAnimationStyle="@null"` —+ * that setting is there to stop the platform substituting its own exit+ * (which, with our result-blocking tools, used to fade an already-empty+ * window), and an explicit override is the documented way to say "no, this+ * one, now".+ *+ * Passed even though `finish()` is already called: the two are separate,+ * and the transition has to be named for the frame the window actually+ * leaves.+ */+ private fun reallyFinish() {+ if (isFinishing || isDestroyed) return+ finish()+ runCatching {+ @Suppress("DEPRECATION")+ overridePendingTransition(0, R.anim.stage_close_exit)+ }+ .onFailure { android.util.Log.w(TAG, "exit transition refused", it) }+ }++ @Deprecated("Deprecated in Java")+ @Suppress("DEPRECATION", "MissingSuperCall")+ override fun onBackPressed() {+ // Back is the user saying the same thing as ✕ 完成. It has to get the+ // same exit, and on these screens there is nowhere behind them anyway+ // — the card is folded, so "back" would otherwise drop the user on a+ // home screen with only a ball in the corner.+ leave()+ }++ companion object {+ private const val TAG = "Body"++ /**+ * Pure black, the same colour `Theme.Body.Stage` paints the window+ * with. A gap of one black frame at either end is invisible; a gap of+ * one white frame is the flash.+ */+ const val BACKDROP = android.graphics.Color.BLACK++ /** How far away the stage starts. Enough to read as depth, not a zoom. */+ const val SCALE_IN = 0.94f++ const val ENTER_MS = 240L++ /** See the class KDoc: the ones that never report. */+ const val REVEAL_BACKSTOP_MS = 2500L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/TapMarkerUi.kt b/app/src/main/java/net/kuafuai/andee/ui/TapMarkerUi.ktnew file mode 100644index 0000000..532794b--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/TapMarkerUi.kt@@ -0,0 +1,457 @@+package net.kuafuai.andee.ui++import android.content.Context+import android.graphics.Canvas+import android.graphics.Color+import android.graphics.Paint+import android.graphics.PixelFormat+import android.graphics.RadialGradient+import android.graphics.RectF+import android.graphics.Shader+import android.graphics.Typeface+import android.os.Handler+import android.os.Looper+import android.os.SystemClock+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import android.view.animation.DecelerateInterpolator+import android.view.animation.OvershootInterpolator+import kotlin.math.max++/**+ * Where the last injected tap landed: a ring, a dot, and the coordinate.+ *+ * Two audiences, which is why it lingers instead of flashing. The user gets to+ * see that something was pressed and where — otherwise an agent driving the+ * tablet is a screen that changes for no visible reason. The brain gets it in+ * the next screenshot: [ScreenController] speaks 0-1000 relative coordinates+ * into a device it cannot see, and the ring is the only feedback anywhere in+ * the loop that closes that circle — "you asked for 500,500 and here is what+ * 500,500 turned out to be". It is worth the most after `tap_screen_element`, where+ * nobody knew the pixel location in advance, and after a tap that appeared to+ * do nothing: a ring sitting next to the button rather than on it is a+ * different problem from a ring on the button that did not respond.+ *+ * ## Why this window is small+ *+ * It is the third overlay this app puts on the screen, and the first two+ * already spend the whole budget. Android stops delivering a touch to the app+ * underneath when untrusted overlays obscure it by more than 0.80 combined+ * opacity, counting the *window's* opacity over the *window's bounds* — not+ * its pixels. A visible full-screen overlay is billed at the 0.80 cap on its+ * own, so this one plus a breathing [EdgeRippleUi] would compute to 0.96 and+ * silently eat every touch on the device, the user's included. Sized to the+ * mark, it can only ever affect the ~144dp box it is drawn in, and only until+ * it expires.+ *+ * It still has to be out of the way for *injected* taps, which land at exactly+ * the coordinate it is sitting on — see [setVisibleForGesture], called from+ * the same passthrough dance as the glow.+ *+ * ## Why it is NOT hidden for screenshots+ *+ * [FloatingWindowUi] and [EdgeRippleUi] both blank themselves during a display+ * capture so the brain never receives a picture of our own UI. This one is the+ * deliberate exception: being in the screenshot is half of what it is for.+ */+class TapMarkerUi(private val context: Context) {++ enum class Kind(val label: String) {+ TAP("tap"),+ HOLD("hold"),+ }++ private val wm: WindowManager =+ context.getSystemService(Context.WINDOW_SERVICE) as WindowManager+ private val ui = Handler(Looper.getMainLooper())++ private var view: MarkView? = null+ private var params: WindowManager.LayoutParams? = null++ /** Main thread only, folded into the window alpha by [applyWindow]. */+ private var gestureHidden = false+ private var showing = false++ private val density = context.resources.displayMetrics.density+ private val box = (BOX_DP * density).toInt()++ /**+ * Where the last mark was placed, in screen pixels, or null if nothing has+ * been marked yet. For the grounding log: the ring is deliberately NOT+ * blanked for captures (see the class KDoc), so anything comparing two+ * screenshots has to mask the box it occupies — otherwise our own ring is+ * the biggest difference between them.+ */+ @Volatile private var lastMark: IntArray? = null+ @Volatile private var lastMarkAt = 0L++ fun lastMarkPx(): IntArray? = lastMark++ /** Milliseconds since [lastMarkPx] was drawn — past [MarkView.HOLD_MS] it is gone. */+ fun lastMarkAgeMs(): Long? =+ if (lastMark == null) null else SystemClock.uptimeMillis() - lastMarkAt++ fun show(): Boolean {+ if (view != null) return true+ val v = MarkView(context) { onIdle() }+ val p = WindowManager.LayoutParams(+ box, box,+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE+ or WindowManager.LayoutParams.FLAG_NOT_TOUCHABLE+ // Without this the box is clamped inside the display and a tap+ // near an edge would be marked somewhere it did not happen.+ or WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.TOP or Gravity.START+ x = 0; y = 0+ alpha = 0f+ }+ return try {+ v.hideFromAccessibility()+ wm.addView(v, p)+ view = v+ params = p+ true+ } catch (t: Throwable) {+ android.util.Log.e("Body", "tap marker addView failed", t)+ false+ }+ }++ fun hide() {+ val v = view ?: return+ runCatching { wm.removeView(v) }+ view = null+ params = null+ }++ /**+ * Mark a gesture that has already been dispatched. Screen pixels, from+ * whichever path resolved them — normalized coordinates, an eid's live+ * bounds, or a top-bar button.+ *+ * Call it *after* the gesture, never before: this window sits on the exact+ * point being touched, and see the class KDoc for what that costs.+ */+ fun mark(x: Int, y: Int, kind: Kind) {+ lastMark = intArrayOf(x, y)+ lastMarkAt = SystemClock.uptimeMillis()+ ui.post {+ val v = view ?: return@post+ val p = params ?: return@post+ p.x = x - box / 2+ p.y = y - box / 2+ runCatching { wm.updateViewLayout(v, p) }+ // The label is the 0-1000 coordinate rather than the pixel one: that+ // is the only space the brain writes in, and the one that survives a+ // downscaled screenshot.+ val dm = context.resources.displayMetrics+ val nx = x * NORM_MAX / max(1, dm.widthPixels)+ val ny = y * NORM_MAX / max(1, dm.heightPixels)+ v.mark(kind.label, "$nx,$ny")+ showing = true+ applyWindow()+ }+ }++ /**+ * Step out of the way of an injected gesture. Not cosmetic: a mark left+ * over from the previous tap is an untrusted overlay sitting on the next+ * one, and the platform drops the touch without reporting anything.+ */+ fun setVisibleForGesture(visible: Boolean) {+ ui.post { gestureHidden = !visible; applyWindow() }+ }++ /**+ * Reveal the marker RIGHT NOW, from any thread — used before an+ * effect-snapshot capture inside the tap itself.+ *+ * [CommandDispatcher.passthroughForGesture] hides this window for the+ * whole duration of a tap, and the effect snapshot is taken *inside*+ * that window — so without this, every after_shot/miss_shot ships with+ * the ring invisible, and the "here is where your tap landed" evidence+ * the whole mechanism exists for is missing from its own picture+ * (measured: adb capture after the tap showed the ring, after_shot did+ * not). The dispatcher's own restore afterwards is idempotent, so no+ * race to worry about.+ */+ fun revealNow() {+ ui.post {+ gestureHidden = false+ applyWindow()+ }+ }++ /** Main thread only. */+ private fun onIdle() {+ showing = false+ applyWindow()+ }++ /** Main thread only. Opacity is billed per window, not per pixel. */+ private fun applyWindow() {+ val v = view ?: return+ val p = params ?: return+ val want = if (showing && !gestureHidden) 1f else 0f+ if (p.alpha == want) return+ p.alpha = want+ runCatching { wm.updateViewLayout(v, p) }+ }++ /**+ * Ring, reticle, caption — drawn against whatever the app underneath+ * happens to be, which is the whole design problem. Every bright stroke is+ * laid over a dark one of its own so the mark survives a white background,+ * a cyan background, and the 1280px downscale on the way to the model.+ *+ * The shapes are borrowed from a camera's focus reticle rather than from a+ * debug overlay: a soft pool of light to seat it on the content, an open+ * ring, four bracket arcs standing off it, and the coordinate set in a+ * monospace face on a glass chip. The brackets sit on the diagonals so the+ * pixel row and column through the centre dot stay clear — a mark that+ * covers what it is pointing at is no use to either audience.+ */+ private class MarkView(+ context: Context,+ private val onIdle: () -> Unit,+ ) : View(context) {++ private val d = context.resources.displayMetrics.density+ private val paint = Paint(Paint.ANTI_ALIAS_FLAG)+ private val glow = Paint(Paint.ANTI_ALIAS_FLAG)+ private val label = Paint(Paint.ANTI_ALIAS_FLAG).apply {+ color = ACCENT+ textSize = 9.5f * d+ letterSpacing = 0.18f+ typeface = Typeface.create("sans-serif-medium", Typeface.NORMAL)+ }+ private val coord = Paint(Paint.ANTI_ALIAS_FLAG).apply {+ color = Color.WHITE+ textSize = 11.5f * d+ letterSpacing = 0.04f+ typeface = Typeface.create(Typeface.MONOSPACE, Typeface.NORMAL)+ }+ private val chip = RectF()+ private val arc = RectF()+ private val ease = DecelerateInterpolator(1.6f)+ private val settle = OvershootInterpolator(2.2f)++ private var labelText = ""+ private var coordText = ""+ private var startedAt = 0L+ private var running = false++ fun mark(kind: String, coords: String) {+ labelText = kind.uppercase()+ coordText = coords+ startedAt = SystemClock.uptimeMillis()+ running = true+ invalidate()+ }++ override fun onSizeChanged(w: Int, h: Int, ow: Int, oh: Int) {+ if (w == 0 || h == 0) return+ // A pool of light under the mark, so it reads as sitting on the+ // content rather than pasted over it. Rebuilt only on resize —+ // allocating a shader per frame would show up as jank.+ glow.shader = RadialGradient(+ w / 2f, h / 2f, GLOW_DP * d,+ intArrayOf(0x33000000, 0x1E000000, 0x00000000),+ floatArrayOf(0f, 0.55f, 1f),+ Shader.TileMode.CLAMP,+ )+ }++ override fun onDraw(canvas: Canvas) {+ if (!running) return+ val age = SystemClock.uptimeMillis() - startedAt+ if (age > STRIKE_MS + HOLD_MS + FADE_MS) {+ running = false+ // Posted, not called: the listener touches window layout, which+ // has no business happening inside a draw pass.+ post(onIdle)+ return+ }++ val cx = width / 2f+ val cy = height / 2f+ // Fade the whole mark out at the end rather than removing it, so it+ // never blinks off in the middle of a capture.+ val life = if (age > STRIKE_MS + HOLD_MS) {+ 1f - (age - STRIKE_MS - HOLD_MS) / FADE_MS.toFloat()+ } else {+ 1f+ }.coerceIn(0f, 1f)++ // The strike: two rings leaving at a stagger, decelerating on the+ // way out the way a real impact does. Only the user ever sees this+ // part — it is long over before any screenshot.+ drawPulse(canvas, cx, cy, age, 0L)+ drawPulse(canvas, cx, cy, age, PULSE_STAGGER_MS)++ // Everything below is the resting mark, which is what ends up in+ // the screenshot. It settles in with a small overshoot instead of+ // appearing at full size — the difference between a thing that+ // arrived and a thing that was always there.+ val grow = if (age < SETTLE_MS) {+ 0.4f + 0.6f * settle.getInterpolation(age / SETTLE_MS.toFloat())+ } else {+ 1f+ }++ glow.alpha = (255 * life).toInt()+ canvas.drawCircle(cx, cy, GLOW_DP * d, glow)++ val r = RING_DP * d * grow+ paint.style = Paint.Style.STROKE+ paint.strokeCap = Paint.Cap.ROUND++ paint.strokeWidth = 4.5f * d+ paint.color = Color.BLACK+ paint.alpha = (85 * life).toInt()+ canvas.drawCircle(cx, cy, r, paint)++ paint.strokeWidth = 2.5f * d+ paint.color = ACCENT+ paint.alpha = (255 * life).toInt()+ canvas.drawCircle(cx, cy, r, paint)++ // Four bracket arcs standing off the ring on the diagonals — a+ // camera's focus frame rather than a crosshair. On the diagonals+ // on purpose: they leave the pixel row and column through the+ // centre dot unobstructed, which is the part anyone is trying to+ // look at.+ val br = BRACKET_DP * d * grow+ arc.set(cx - br, cy - br, cx + br, cy + br)+ for (pass in 0..1) {+ paint.strokeWidth = if (pass == 0) 3.5f * d else 2f * d+ paint.color = if (pass == 0) Color.BLACK else ACCENT+ paint.alpha = ((if (pass == 0) 80 else 225) * life).toInt()+ for (k in 0..3) {+ canvas.drawArc(arc, 45f + 90f * k - BRACKET_SWEEP / 2f, BRACKET_SWEEP, false, paint)+ }+ }++ paint.style = Paint.Style.FILL+ paint.color = Color.BLACK+ paint.alpha = (110 * life).toInt()+ canvas.drawCircle(cx, cy, 4.5f * d, paint)+ paint.color = Color.WHITE+ paint.alpha = (255 * life).toInt()+ canvas.drawCircle(cx, cy, 3f * d, paint)++ drawCaption(canvas, cx, cy, life)++ // Idle frames cost nothing while the mark is just sitting there, but+ // it still has to expire, so keep asking until `life` runs out.+ postInvalidateOnAnimation()+ }++ private fun drawPulse(canvas: Canvas, cx: Float, cy: Float, age: Long, delay: Long) {+ val t = (age - delay) / PULSE_MS.toFloat()+ if (t < 0f || t >= 1f) return+ val e = ease.getInterpolation(t)+ paint.style = Paint.Style.STROKE+ paint.strokeCap = Paint.Cap.ROUND+ // Thinning as it grows, so the ring dissolves rather than stopping.+ paint.strokeWidth = (3.2f - 2.4f * e) * d+ paint.color = ACCENT+ paint.alpha = ((1f - e) * (1f - e) * 190).toInt()+ canvas.drawCircle(cx, cy, (RING_DP + (PULSE_MAX_DP - RING_DP) * e) * d, paint)+ }++ /**+ * Kind and coordinate on a glass chip, hung off the ring by a hairline.+ * Two typefaces on purpose: the kind is a label, the numbers are data,+ * and a monospace figure set is the thing that keeps "100,100" from+ * being read as "1OO,1OO" after the screenshot is downscaled.+ */+ private fun drawCaption(canvas: Canvas, cx: Float, cy: Float, life: Float) {+ if (coordText.isEmpty()) return+ val lw = label.measureText(labelText)+ val nw = coord.measureText(coordText)+ val gap = 6f * d+ val pad = 10f * d+ val w = lw + gap + nw+ val h = 22f * d+ val top = cy + CHIP_TOP_DP * d++ paint.style = Paint.Style.STROKE+ paint.strokeWidth = 1.5f * d+ paint.color = ACCENT+ paint.alpha = (110 * life).toInt()+ canvas.drawLine(cx, cy + (RING_DP + 4f) * d, cx, top, paint)++ chip.set(cx - w / 2 - pad, top, cx + w / 2 + pad, top + h)+ val rad = h / 2f++ // Lift: one darker copy nudged down behind the chip. Cheaper than a+ // shadow layer, which would force this View into software rendering.+ paint.style = Paint.Style.FILL+ paint.color = Color.BLACK+ paint.alpha = (70 * life).toInt()+ chip.offset(0f, 1.5f * d)+ canvas.drawRoundRect(chip, rad, rad, paint)+ chip.offset(0f, -1.5f * d)++ paint.color = CHIP_BG+ paint.alpha = (233 * life).toInt()+ canvas.drawRoundRect(chip, rad, rad, paint)++ // The hairline is what makes it read as glass rather than as a+ // black box — same trick as the history bubbles.+ paint.style = Paint.Style.STROKE+ paint.strokeWidth = 1f * d+ paint.color = CHIP_EDGE+ paint.alpha = (90 * life).toInt()+ canvas.drawRoundRect(chip, rad, rad, paint)++ val base = top + h / 2f - (coord.descent() + coord.ascent()) / 2f+ label.alpha = (215 * life).toInt()+ canvas.drawText(labelText, cx - w / 2, base, label)+ coord.alpha = (255 * life).toInt()+ canvas.drawText(coordText, cx - w / 2 + lw + gap, base, coord)+ }++ companion object {+ private val ACCENT = Color.parseColor("#2DE2FF")+ private val CHIP_BG = Color.parseColor("#0B1220")+ private val CHIP_EDGE = Color.WHITE++ private const val RING_DP = 14f+ private const val GLOW_DP = 30f+ private const val BRACKET_DP = 21f+ private const val BRACKET_SWEEP = 34f+ private const val PULSE_MAX_DP = 48f+ private const val CHIP_TOP_DP = 33f++ private const val PULSE_MS = 480L+ private const val PULSE_STAGGER_MS = 110L+ private const val STRIKE_MS = PULSE_MS + PULSE_STAGGER_MS+ private const val SETTLE_MS = 260L+ private const val FADE_MS = 400L++ /**+ * How long the mark stays put. Long enough that a brain which taps,+ * waits for the app to settle and then screenshots still finds it;+ * short enough that it is gone before the user wonders what it is.+ */+ private const val HOLD_MS = 6_000L+ }+ }++ companion object {+ /**+ * Side of the window, in dp. Big enough for the outgoing pulse and the+ * widest caption ("HOLD 1000,1000"), small enough that a stale mark+ * cannot obscure much — see the class KDoc.+ */+ private const val BOX_DP = 144++ private const val NORM_MAX = 1000+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/TextInputActivity.kt b/app/src/main/java/net/kuafuai/andee/ui/TextInputActivity.ktnew file mode 100644index 0000000..8bd670b--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/TextInputActivity.kt@@ -0,0 +1,416 @@+package net.kuafuai.andee.ui++import android.app.Activity+import android.content.Context+import android.content.Intent+import android.graphics.Color+import android.graphics.drawable.ColorDrawable+import android.os.Bundle+import android.text.InputType+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import android.view.inputmethod.EditorInfo+import android.view.inputmethod.InputMethodManager+import android.widget.EditText+import android.widget.FrameLayout+import android.widget.ImageView+import android.widget.LinearLayout+import android.widget.TextView+import androidx.core.view.ViewCompat+import androidx.core.view.WindowCompat+import androidx.core.view.WindowInsetsCompat+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import java.lang.ref.WeakReference++/**+ * The field a person types into — an **Activity**, not an overlay window.+ *+ * That is not a style choice, it is the only thing on this device that works.+ * Measured on the tablet this was written for (Xiaomi Pad 5, HyperOS):+ *+ * * a `TYPE_APPLICATION_OVERLAY` window with an `EditText` **never raises the+ * soft keyboard** — not from `showSoftInput`, not from+ * `WindowInsetsController.show(ime())`, and not even from a real finger tap+ * (`mShowRequested` stayed `false` throughout);+ * * the same input inside an Activity raises it immediately.+ *+ * So a floating input panel is not available here, however much nicer it would+ * look. This is the second overlay-only assumption this feature has had to give+ * up; the first was that the keyboard could be moved on demand+ * (see [ImeSwitch]).+ *+ * Three things it inherits from the overlays it replaced, all learned the hard+ * way elsewhere in this app:+ *+ * * the card has to get out of the way before it opens — it is a fullscreen+ * overlay, so it sits above this Activity and eats every touch aimed at the+ * strip. That is `FloatingWindowUi.setTypingTop`, and this class reports+ * itself gone so the card can come back exactly as it was.+ * * `hideFromAccessibility()` — a focused overlay belongs to us, and+ * `ScreenController`'s node dump would otherwise hand the brain *this* field+ * as "the app the user is on", so `type_text`'s `findFocus` lands on the+ * sentence the person is composing. See `OverlayA11y`.+ * * `OwnCard` — hiding the window from accessibility leaves a dump that looks+ * like a dead accessibility connection, which `reviveNote` reports to the+ * user as "toggle Andee off and on". Registering is what keeps that true.+ * * Nothing here touches the input method. The device's IME is the user's own,+ * so this types on it like any other app's field would; the keyboard is only+ * moved inside `ScreenController.typeViaAdbKeyboard`, for the length of one+ * tool call.+ *+ * Wired like [ChatHistory] and [CardUi]: static hooks the service sets once.+ */+class TextInputActivity : Activity() {++ private lateinit var field: EditText+ private var sent = false++ /** Strings in the user's language; views are still built on `this`. */+ private lateinit var lctx: Context++ /**+ * Whether the soft keyboard has been up at least once.+ *+ * The inset is 0 before the keyboard appears and 0 again once it is gone, so+ * "the keyboard went away" can only be told from "it has not arrived yet" by+ * remembering that it did arrive. Getting this wrong closes the field on the+ * first layout pass, which is the one moment the user is certain to still be+ * typing.+ */+ private var imeShown = false++ /** Pending "the keyboard is really gone, close up" check. See [scheduleCloseWhenGone]. */+ private var closeWhenGone: Runnable? = null++ override fun onCreate(savedInstanceState: Bundle?) {+ super.onCreate(savedInstanceState)+ lctx = AppLocale.wrap(this)++ field = EditText(this).apply {+ hint = lctx.getString(R.string.input_hint)+ textSize = 15f+ setTextColor(Color.parseColor(Glass.TITLE))+ // SECONDARY, not MUTED. The hint is the only thing in here until+ // somebody types, so it is the one string that has to survive being+ // read at a glance: #5F6775 on the strip's near-black is ~3.6:1,+ // which is the faint grey this whole change was about.+ setHintTextColor(Color.parseColor(Glass.SECONDARY))+ // Recessed, so the strip reads as "type here" rather than as a+ // toolbar with two buttons and some floating text.+ background = Glass.well(this@TextInputActivity, dp(14))+ // Single line with the enter key turned into send. Deliberately not+ // a multi-line box: with one, Enter inserts a newline and the only+ // way to send is the button — a coin flip the user has to learn.+ maxLines = 1+ inputType = InputType.TYPE_CLASS_TEXT+ imeOptions = EditorInfo.IME_ACTION_SEND+ setPadding(dp(12), dp(8), dp(12), dp(8))+ setOnEditorActionListener { _, actionId, _ ->+ if (actionId == EditorInfo.IME_ACTION_SEND) {+ send()+ true+ } else {+ false+ }+ }+ }++ val row = LinearLayout(this).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ // The *card* fill, not `Glass.panel`.+ //+ // `Glass` is a two-layer stack: an almost-opaque black base, then+ // white at low alpha on top of it. `panel` is the second layer —+ // 6% white — and it is only ever correct *on top of a card*.+ // Standing alone over somebody else's app it draws nothing at all,+ // and the field's own background came out as whatever was behind+ // it: measured over a white page, mean RGB 255,255,255 with+ // `Glass.TITLE` white text on it. Contrast 0. That is the "输入框+ // 是透明的" this replaces — not a matter of taste, arithmetic.+ //+ // So this strip gets the base. It has to stay readable over *any*+ // backdrop, a white document and a night-mode terminal alike, and+ // there is no blur to lean on: `Glass.frost` blurs behind a window,+ // and this window is the whole screen (only this strip is painted),+ // so asking for it would defocus everything the user was looking+ // at. A dark bar over the bottom edge is the honest answer.+ background = Glass.card(this@TextInputActivity, dp(20), frosted = false)+ setPadding(dp(8), dp(8), dp(8), dp(8))+ }+ row.addView(field, LinearLayout.LayoutParams(0, WRAP, 1f))+ row.addView(glyph(lctx.getString(R.string.input_send), Glass.ACCENT) { send() })+ row.addView(glyph("✕", Glass.LABEL) { finish() })++ // The strip sits on **the card's own backdrop**, not on the user's app.+ //+ // The card above ends at this strip's top edge (it has to: overlays sort+ // above the keyboard, so a card that reached lower would hide both this+ // strip and the keys). That leaves the band this window owns, and with a+ // translucent window the band showed whatever app was underneath — a+ // bright seam of somebody's home screen between a dark card and a dark+ // strip, which reads as the card having gone see-through. Measured on+ // the tablet: mean RGB 60–85 in that band against 8–35 in the card+ // directly above it.+ //+ // Same [Backdrop] object, same scrim, same `CENTER_CROP`, so the two+ // windows line up by construction rather than by eye. The fill behind+ // the image is the same near-opaque slate the strip is made of, so a+ // field opened before the backdrop has finished warming still gets a+ // continuous surface rather than the desktop.+ val backdrop = ImageView(this).apply {+ scaleType = ImageView.ScaleType.CENTER_CROP+ }+ Backdrop.warm(this) { bmp ->+ backdrop.setImageBitmap(bmp)+ backdrop.foreground = ColorDrawable(Backdrop.scrim())+ }++ // One row, anchored at the bottom. See the backdrop above for why the+ // window is no longer see-through.+ val column = LinearLayout(this).apply {+ orientation = LinearLayout.VERTICAL+ gravity = Gravity.BOTTOM+ setPadding(dp(12), dp(4), dp(12), dp(12))+ addView(row, LinearLayout.LayoutParams(MATCH_PARENT, WRAP))+ }+ val root = FrameLayout(this).apply {+ setBackgroundColor(Color.parseColor(Glass.CARD_SOLID))+ addView(+ backdrop,+ FrameLayout.LayoutParams(MATCH_PARENT, MATCH_PARENT),+ )+ addView(+ column,+ FrameLayout.LayoutParams(MATCH_PARENT, MATCH_PARENT),+ )+ }+ setContentView(root)++ // The keyboard is most of a phone screen, and it arrives **over** this+ // window. A translucent window is never resized for the IME — measured:+ // with `adjustResize` set the row still sat at the window's bottom edge+ // with 搜狗's keys drawn on top of it — so the inset has to be applied by+ // hand. Which needs the decor to stop fitting system windows first: with+ // it fitting, the IME inset is consumed before any listener sees it+ // (measured: it arrived as 0).+ WindowCompat.setDecorFitsSystemWindows(window, false)+ ViewCompat.setOnApplyWindowInsetsListener(root) { _, insets ->+ val ime = insets.getInsets(WindowInsetsCompat.Type.ime()).bottom+ val bars = insets.getInsets(WindowInsetsCompat.Type.systemBars())+ // On the column, not the root: the backdrop is full bleed and must+ // slide under the status bar and the keys, while the strip itself+ // steps around them.+ column.setPadding(dp(12), dp(4) + bars.top, dp(12), dp(12) + ime)+ if (ime > 0) {+ imeShown = true+ cancelCloseWhenGone()+ } else if (imeShown) {+ // The keyboard is on its way out, and so is this field:收起键盘+ // means the user is done typing, and a strip left behind on an+ // otherwise empty screen is a thing to dismiss twice.+ //+ // The band is deliberately *not* re-reported here (see the post+ // below): with the keys gone the strip would jump to the bottom+ // of the screen, the card would grow to meet it, and the whole+ // thing would close a moment later — three moves for one+ // gesture. Left alone, the card simply grows back once.+ scheduleCloseWhenGone()+ return@setOnApplyWindowInsetsListener insets+ }+ // Tell the card where our top edge is, in screen pixels, so it can+ // end there instead of covering us. Posted, because the row has no+ // position until this pass has laid it out — and sent again on every+ // inset change, which is what keeps the two in step as the keyboard+ // slides up and as 搜狗's toolbar row comes and goes.+ root.post {+ if (isFinishing) return@post+ val at = IntArray(2)+ row.getLocationOnScreen(at)+ onTop?.invoke(at[1])+ }+ insets+ }+ window.decorView.hideFromAccessibility()+ OwnCard.shown()+ live = WeakReference(this)+ }++ override fun onResume() {+ super.onResume()+ resumed = true+ // Here rather than in onCreate: the window has focus by now, and an IME+ // request from a window that does not is dropped without a word.+ field.requestFocus()+ showKeyboard()+ field.postDelayed({ if (!isFinishing) showKeyboard() }, IME_RETRY_MS)+ }++ override fun onPause() {+ // Paired with [resumed] in the companion, which is what makes+ // `isShowing` mean "on screen" rather than "alive". Measured: HOME (or+ // anything else that backgrounds this without finishing it) leaves the+ // instance — and its window — in place, and a service that believed the+ // field was still showing made the next ⌨ tap a *close*: the user taps+ // the button, and nothing at all happens.+ resumed = false+ super.onPause()+ }++ override fun onDestroy() {+ cancelCloseWhenGone()+ if (live?.get() === this) live = null+ resumed = false+ OwnCard.hidden()+ // The card was taken off the screen for the length of this field (see+ // [FloatingWindowUi.setTypingTop]) and is waiting to be put back.+ // This is the one place every way out of the field passes through —+ // `✕`, BACK, send, or the service dismissing it — and hanging the+ // restore off the send button instead would strand the device with an+ // invisible, untouchable card: no ball, and no way to ask for one.+ onClosed?.invoke()+ super.onDestroy()+ }++ /**+ * Hand the text to the service and get out of the way.+ *+ * Clears the field before handing it over, so a send that turns out to be a+ * no-op (nothing listening) is visible as an empty box rather than as text+ * that silently went nowhere.+ */+ private fun send() {+ val text = field.text?.toString()?.trim().orEmpty()+ if (text.isEmpty()) return+ field.setText("")+ sent = true+ onSubmit?.invoke(text)+ finish()+ }++ /**+ * Close up once the keyboard is really gone, rather than on the first frame+ * it reports 0.+ *+ * An IME does not only go away when the user is finished: it also retracts+ * for a moment when the input method is swapped out underneath us, which is+ * exactly what happens when the brain types into another app while this+ * field is open ([ImeSwitch] moves the device to ADBKeyboard for one tool+ * call). Closing on that would be a field that vanishes for no reason the+ * user can see — the failure this whole class has been trying not to be. A+ * quarter of a second is longer than the animation and shorter than a+ * person noticing the delay, and anything that brings the keyboard back+ * cancels it.+ */+ private fun scheduleCloseWhenGone() {+ if (closeWhenGone != null || isFinishing) return+ val r = Runnable {+ closeWhenGone = null+ if (!isFinishing) finish()+ }+ closeWhenGone = r+ field.postDelayed(r, IME_GONE_MS)+ }++ private fun cancelCloseWhenGone() {+ val r = closeWhenGone ?: return+ closeWhenGone = null+ field.removeCallbacks(r)+ }++ private fun showKeyboard() {+ runCatching { field.windowInsetsController?.show(android.view.WindowInsets.Type.ime()) }+ runCatching {+ (getSystemService(Context.INPUT_METHOD_SERVICE) as InputMethodManager)+ .showSoftInput(field, 0)+ }+ }++ private fun glyph(label: String, color: String, onClick: () -> Unit): TextView =+ TextView(this).apply {+ text = label+ textSize = 15f+ setTextColor(Color.parseColor(color))+ gravity = Gravity.CENTER+ setPadding(dp(14), dp(10), dp(14), dp(10))+ isClickable = true+ isFocusable = true+ Glass.pressable(this)+ setOnClickListener { onClick() }+ }++ private fun dp(v: Int): Int = (v * resources.displayMetrics.density).toInt()++ companion object {+ private const val IME_RETRY_MS = 250L++ /**+ * How long the keyboard has to stay gone before the field follows it.+ * Long enough to ride out an input-method swap, short enough to read as+ * "the field closed with the keyboard".+ */+ private const val IME_GONE_MS = 250L+ private const val WRAP = LinearLayout.LayoutParams.WRAP_CONTENT+ private const val MATCH_PARENT = LinearLayout.LayoutParams.MATCH_PARENT++ /** Set once by the service, like [ChatHistory.init]. */+ @Volatile+ var onSubmit: ((String) -> Unit)? = null++ /**+ * Fires from [onDestroy] — the field is gone, whatever took it away.+ *+ * The service uses it to put the card back; see the call there for why+ * it is this hook and not the send button.+ */+ @Volatile+ var onClosed: (() -> Unit)? = null++ /**+ * The screen-y of this strip's top edge, whenever it moves.+ *+ * The card above is a fullscreen overlay and overlays sort above the+ * keyboard on this platform, so it has to *end* here for the field and+ * the keyboard to be visible at all. Reported rather than computed on+ * the other side because only this window knows the keyboard's real+ * height.+ */+ @Volatile+ var onTop: ((Int) -> Unit)? = null++ private var live: WeakReference<TextInputActivity>? = null++ /** Set between [onResume] and [onPause]; see [isShowing]. */+ @Volatile+ private var resumed = false++ /**+ * Whether the field is **on screen**, which is not the same as "there is+ * an instance". The service uses this to decide whether a ⌨ tap opens+ * or closes the field, and an alive-but-backgrounded activity would+ * otherwise make the tap do neither.+ */+ val isShowing: Boolean get() = resumed && live?.get() != null++ /**+ * Open it. `NEW_TASK` because this comes from a service, and `SINGLE_TOP`+ * so a second tap on ⌨ raises the field already up instead of stacking a+ * second one behind it.+ */+ fun start(context: Context) {+ runCatching {+ context.startActivity(+ Intent(context, TextInputActivity::class.java)+ .addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_SINGLE_TOP),+ )+ }+ }++ /** Close it if it is up. Safe to call from anywhere, including never-opened. */+ fun dismiss() {+ live?.get()?.finish()+ }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/TextPage.kt b/app/src/main/java/net/kuafuai/andee/ui/TextPage.ktnew file mode 100644index 0000000..e0047a8--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/TextPage.kt@@ -0,0 +1,118 @@+package net.kuafuai.andee.ui++import android.content.Context+import android.util.Log+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import java.io.File+import java.text.SimpleDateFormat+import java.util.Date++/**+ * A chat message, as a full-screen page.+ *+ * A row shows six lines and then an ellipsis, which is the right shape for a+ * transcript — a chat where every answer unfolds to forty lines is a chat nobody+ * can scroll. What was missing is a way to read the rest when the rest is what+ * you wanted. Double-tapping a row opens this.+ *+ * It goes through [HtmlActivity] rather than growing a reader of its own, and+ * that is about mechanics rather than markup: the page path already knows how to+ * fold the card first, hang the ball off the ledge, float a 完成 pill, and report+ * itself closed — none of which is worth implementing twice. The stylesheet below+ * is the house style the `show_html` tool description gives the model, so a+ * message opened this way looks like a page the model composed, not like a+ * second kind of screen.+ *+ * The text is escaped. A message is data — it can contain angle brackets, a+ * stray tag, someone's HTML homework — and handing that to a WebView unescaped+ * would render it as markup. Paragraphs are built here rather than by CSS+ * `white-space:pre` so the escaping stays obvious in one place.+ */+object TextPage {++ private const val TAG = "Body"++ /** Where the built page is staged: the same directory the model's pages use. */+ private fun dir(context: Context) = File(context.filesDir, "pages").apply { mkdirs() }++ /**+ * Write [entry] out as a page and return the file, or null if the disk said+ * no — in which case the caller shows nothing rather than an empty screen.+ *+ * One file per entry, named after its id: opening the same message twice+ * overwrites rather than accumulating, and `ChatHistory.wipe` already deletes+ * this directory when the user asks for a factory reset.+ */+ fun write(context: Context, entry: ChatHistory.Entry): File? = runCatching {+ File(dir(context), "text_${entry.id}.html").also {+ it.writeText(html(head(context, entry), entry.text))+ }+ }.getOrElse {+ Log.w(TAG, "text page: could not write: ${it.message}")+ null+ }++ /**+ * Who said it and when, as the row's own header reads. Dated rather than+ * clock-only: a row is read in a list you can see the top of, a page is read+ * on its own with nothing around it to say whether it was this morning.+ */+ private fun head(context: Context, e: ChatHistory.Entry): String {+ val lctx = AppLocale.wrap(context)+ val who = when (e.role) {+ ChatHistory.Role.USER -> lctx.getString(R.string.textpage_role_you)+ ChatHistory.Role.ASSISTANT -> lctx.getString(R.string.textpage_role_assistant)+ ChatHistory.Role.ERROR -> lctx.getString(R.string.textpage_role_error)+ ChatHistory.Role.PAGE -> lctx.getString(R.string.textpage_role_page)+ }+ val when_ = SimpleDateFormat("MM-dd HH:mm", AppLocale.localeOf(context)).format(Date(e.ts))+ return "$who · $when_"+ }++ /** The whole document. Exposed for the tests that read it back. */+ fun html(who: String, text: String): String = """+<!doctype html>+<html><head>+<meta charset="utf-8">+<meta name="viewport" content="width=device-width,initial-scale=1,viewport-fit=cover">+<style>+ html,body { margin:0; padding:0; background:#0F172A; color:#F8FAFC;+ font-family:-apple-system,"PingFang SC","Noto Sans CJK SC",system-ui,sans-serif;+ font-size:16px; line-height:1.6; -webkit-text-size-adjust:100%; }+ /* 88px of top padding clears the ✕完成 pill the page floats there. */+ .wrap { padding:88px 20px 40px; }+ .who { margin:0 0 16px; font-size:13px; color:#94A3B8; }+ .card { background:#1E293B; border:1px solid rgba(148,163,184,.18);+ border-radius:14px; padding:20px; }+ p { margin:0 0 16px; word-break:break-word; }+ p:last-child { margin-bottom:0; }+ p.gap { margin:0; height:8px; }+</style>+</head><body>+ <div class="wrap">+ <p class="who">${esc(who)}</p>+ <div class="card">+${body(text)}+ </div>+ </div>+</body></html>+""".trimIndent()++ /** Blank lines become spacing rather than empty paragraphs. */+ private fun body(text: String): String = text+ .split('\n')+ .joinToString("\n") { line ->+ if (line.isBlank()) " <p class=\"gap\"></p>"+ else " <p>${esc(line)}</p>"+ }++ /**+ * The three characters that would otherwise turn text into markup. Quotes+ * are left alone: nothing here is interpolated into an attribute.+ */+ private fun esc(s: String): String = s+ .replace("&", "&")+ .replace("<", "<")+ .replace(">", ">")+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/VaultUi.kt b/app/src/main/java/net/kuafuai/andee/ui/VaultUi.ktnew file mode 100644index 0000000..b5d38e9--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/VaultUi.kt@@ -0,0 +1,487 @@+package net.kuafuai.andee.ui++import android.annotation.SuppressLint+import android.content.Context+import android.graphics.Color+import android.graphics.PixelFormat+import android.text.InputType+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import android.widget.EditText+import android.widget.LinearLayout+import android.widget.ScrollView+import android.widget.TextView+import net.kuafuai.andee.R+import net.kuafuai.andee.config.Vault+import net.kuafuai.andee.i18n.AppLocale++/**+ * The user's credential vault: one entry per account, so the ball can log in+ * as them.+ *+ * Its own overlay rather than more rows in [SettingsUi] — the list grows, and+ * the wake-word row already set the precedent of putting a real screen behind+ * a single button ([WakeEnrollUi]).+ *+ * The window is **focusable** (flags 0, not `FLAG_NOT_FOCUSABLE`) with+ * `SOFT_INPUT_ADJUST_RESIZE`, for the same reason [SettingsUi] is: without+ * that pair the EditTexts here cannot receive the IME at all, and the+ * keyboard covers the field being typed into. [hideFromAccessibility] is not+ * optional either — we are an accessibility service, and an un-hidden form+ * shows up in our own UI dumps with the user's password in it.+ */+class VaultUi(+ private val context: Context,+ private val onDismiss: () -> Unit = {},+) {++ private val wm: WindowManager =+ context.getSystemService(Context.WINDOW_SERVICE) as WindowManager++ private var root: View? = null++ /** Null = the list; non-null = editing this entry (blank id = new). */+ private var editing: Vault.Entry? = null++ /** Whether the compositor agreed to blur. See [Glass.frost]. */+ private var frosted = false++ /**+ * Strings in the user's language, refreshed at the top of [build].+ *+ * Refreshed there and not in [show] on purpose: [rebuild] goes straight to+ * [build] without passing through [show], so rewrapping in the one function+ * that consumes it means no future caller has to remember to do it.+ *+ * Views are still built on [context]; only text comes from here. See+ * [AppLocale] for why those two are deliberately different Contexts.+ */+ private var lctx: Context = context++ fun show() {+ if (root != null) return+ val p = WindowManager.LayoutParams(+ cardWidth(), cardHeight(),+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ 0,+ PixelFormat.TRANSLUCENT,+ ).apply {+ gravity = Gravity.CENTER+ softInputMode = WindowManager.LayoutParams.SOFT_INPUT_ADJUST_RESIZE+ }+ frosted = Glass.frost(context, wm, p)+ val v = build()+ runCatching {+ v.hideFromAccessibility()+ wm.addView(v, p)+ root = v+ OwnCard.shown()+ Glass.enter(v)+ }+ }++ fun hide() {+ val v = root ?: return+ root = null+ Glass.exit(v) {+ runCatching { wm.removeView(v) }+ OwnCard.hidden()+ onDismiss()+ }+ }++ private fun rebuild() {+ val old = root ?: return+ val v = build()+ runCatching {+ v.hideFromAccessibility()+ val p = old.layoutParams as WindowManager.LayoutParams+ wm.removeView(old)+ wm.addView(v, p)+ root = v+ Glass.fadeIn(v)+ }+ }++ private fun cardWidth(): Int =+ minOf(dp(500), (wm.currentWindowMetrics.bounds.width() * 0.94f).toInt())++ private fun cardHeight(): Int =+ minOf(dp(600), (wm.currentWindowMetrics.bounds.height() * 0.86f).toInt())++ // ---- build ----++ @SuppressLint("SetTextI18n")+ private fun build(): View {+ lctx = AppLocale.wrap(context)+ val card = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ background = Glass.card(context, dp(28), frosted)+ clipToOutline = true+ }++ val header = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ setPadding(dp(22), dp(18), dp(14), dp(12))+ }+ header.addView(+ TextView(context).apply {+ text = if (editing == null) {+ lctx.getString(R.string.vault_title)+ } else {+ lctx.getString(R.string.vault_edit_title)+ }+ textSize = 20f+ setTextColor(Color.parseColor(Glass.TITLE))+ },+ LinearLayout.LayoutParams(0, LinearLayout.LayoutParams.WRAP_CONTENT, 1f),+ )+ header.addView(+ TextView(context).apply {+ text = "✕"+ textSize = 13f+ setTextColor(Color.parseColor(Glass.SECONDARY))+ gravity = Gravity.CENTER+ background = Glass.panel(context, dp(16))+ isClickable = true+ Glass.pressable(this)+ setOnClickListener {+ if (editing == null) hide() else { editing = null; rebuild() }+ }+ },+ LinearLayout.LayoutParams(dp(32), dp(32)),+ )+ card.addView(+ header,+ LinearLayout.LayoutParams(+ LinearLayout.LayoutParams.MATCH_PARENT,+ LinearLayout.LayoutParams.WRAP_CONTENT,+ ),+ )+ divider(card)++ val scroll = ScrollView(context)+ val body = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ setPadding(dp(20), dp(12), dp(20), dp(12))+ }+ val e = editing+ val footer = if (e == null) buildList(body) else buildForm(body, e)+ scroll.addView(+ body,+ LinearLayout.LayoutParams(+ LinearLayout.LayoutParams.MATCH_PARENT,+ LinearLayout.LayoutParams.WRAP_CONTENT,+ ),+ )+ card.addView(+ scroll,+ LinearLayout.LayoutParams(LinearLayout.LayoutParams.MATCH_PARENT, 0, 1f),+ )+ divider(card)+ card.addView(+ footer,+ LinearLayout.LayoutParams(+ LinearLayout.LayoutParams.MATCH_PARENT,+ LinearLayout.LayoutParams.WRAP_CONTENT,+ ),+ )+ return card+ }++ private fun footerRow(): LinearLayout = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.END+ setPadding(dp(16), dp(12), dp(16), dp(16))+ }++ @SuppressLint("SetTextI18n")+ private fun buildList(body: LinearLayout): LinearLayout {+ body.addView(TextView(context).apply {+ text = lctx.getString(R.string.vault_intro)+ textSize = 12f+ setTextColor(Color.parseColor(Glass.SECONDARY))+ setPadding(0, 0, 0, dp(10))+ })++ val all = Vault.entries(context)+ if (all.isEmpty()) {+ body.addView(TextView(context).apply {+ text = lctx.getString(R.string.vault_empty)+ textSize = 13f+ setTextColor(Color.parseColor(Glass.MUTED))+ setPadding(0, dp(12), 0, dp(12))+ })+ }+ for (entry in all) {+ body.addView(+ entryRow(entry),+ LinearLayout.LayoutParams(+ LinearLayout.LayoutParams.MATCH_PARENT,+ LinearLayout.LayoutParams.WRAP_CONTENT,+ ).apply { bottomMargin = dp(8) },+ )+ }++ return footerRow().apply {+ addView(button(lctx.getString(R.string.vault_add_account), filled = true) {+ editing = Vault.Entry(id = "", label = "")+ rebuild()+ })+ }+ }++ @SuppressLint("SetTextI18n")+ private fun entryRow(entry: Vault.Entry): View {+ val row = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.CENTER_VERTICAL+ setPadding(dp(12), dp(10), dp(12), dp(10))+ background = Glass.panel(context, dp(14))+ isClickable = true+ Glass.pressable(this)+ setOnClickListener { editing = entry; rebuild() }+ }+ val column = LinearLayout(context).apply { orientation = LinearLayout.VERTICAL }+ column.addView(TextView(context).apply {+ text = entry.label.ifBlank { entry.id }+ textSize = 15f+ setTextColor(Color.WHITE)+ })+ // [Vault.Entry.masked] rather than masking here: `card_last4` is four+ // characters and [Vault.mask] would render it `****`, throwing away the+ // one thing that tells two cards apart. That exemption lives in one+ // place so this row and the brain's own list cannot disagree about it.+ val summary = buildList {+ entry.masked().forEach { (k, v) -> add("${fieldLabel(k)} $v") }+ if (entry.password.isNotBlank()) add(lctx.getString(R.string.vault_password_saved))+ }.joinToString(" · ")+ column.addView(TextView(context).apply {+ text = summary.ifBlank { lctx.getString(R.string.vault_summary_empty) }+ textSize = 12f+ setTextColor(Color.parseColor(Glass.SECONDARY))+ setPadding(0, dp(2), 0, 0)+ })+ row.addView(column, LinearLayout.LayoutParams(0, LinearLayout.LayoutParams.WRAP_CONTENT, 1f))+ row.addView(TextView(context).apply {+ text = lctx.getString(R.string.vault_delete)+ textSize = 13f+ setTextColor(Color.parseColor(Glass.DANGER))+ setPadding(dp(12), dp(6), dp(4), dp(6))+ isClickable = true+ setOnClickListener { Vault.remove(context, entry.id); rebuild() }+ })+ return row+ }++ @SuppressLint("SetTextI18n")+ private fun buildForm(body: LinearLayout, entry: Vault.Entry): LinearLayout {+ val label = field(+ body, lctx.getString(R.string.vault_field_name), entry.label,+ hint = lctx.getString(R.string.vault_field_name_hint),+ )+ val phone = field(body, lctx.getString(R.string.vault_field_phone), entry.phone, numeric = true)+ val email = field(body, lctx.getString(R.string.vault_field_email), entry.email)+ val username = field(body, lctx.getString(R.string.vault_field_username), entry.username)+ val password = field(body, lctx.getString(R.string.vault_field_password), entry.password, password = true)+ body.addView(TextView(context).apply {+ text = lctx.getString(R.string.vault_password_note)+ textSize = 11f+ setTextColor(Color.parseColor(Glass.MUTED))+ setPadding(0, dp(4), 0, 0)+ })+ val note = field(+ body, lctx.getString(R.string.vault_field_note), entry.note,+ hint = lctx.getString(R.string.vault_field_note_hint),+ )++ // The card group, below the note rather than beside the password: an+ // entry usually has one or the other, and a form that opens with four+ // blank card rows reads as four things the user forgot to fill in.+ body.addView(TextView(context).apply {+ text = lctx.getString(R.string.vault_section_card)+ textSize = 13f+ setTextColor(Color.parseColor(Glass.LABEL))+ setPadding(0, dp(18), 0, dp(2))+ })+ body.addView(TextView(context).apply {+ text = lctx.getString(R.string.vault_card_note)+ textSize = 11f+ setTextColor(Color.parseColor(Glass.MUTED))+ })+ val cardHolder = field(+ body, lctx.getString(R.string.vault_field_card_holder), entry.cardHolder,+ hint = lctx.getString(R.string.vault_field_card_holder_hint),+ )+ val cardNumber = field(+ body, lctx.getString(R.string.vault_field_card_number), entry.cardNumber,+ numeric = true, password = true,+ )+ val cardExpiry = field(+ body, lctx.getString(R.string.vault_field_card_expiry), entry.cardExpiry,+ hint = lctx.getString(R.string.vault_field_card_expiry_hint), date = true,+ )+ val cardCvv = field(+ body, lctx.getString(R.string.vault_field_card_cvv), entry.cardCvv,+ numeric = true, password = true,+ )++ return footerRow().apply {+ addView(button(lctx.getString(R.string.common_cancel), filled = false) { editing = null; rebuild() })+ addView(View(context), LinearLayout.LayoutParams(dp(8), 1))+ addView(button(lctx.getString(R.string.common_save), filled = true) {+ val name = label.text.toString().trim()+ if (name.isEmpty()) {+ label.error = lctx.getString(R.string.vault_error_name_required)+ return@button+ }+ Vault.put(+ context,+ Vault.Entry(+ id = entry.id.ifBlank { Vault.newId() },+ label = name,+ phone = phone.text.toString().trim(),+ email = email.text.toString().trim(),+ username = username.text.toString().trim(),+ note = note.text.toString().trim(),+ password = password.text.toString(),+ // Spaces and dashes are how a card is printed and how+ // a person types it; no checkout field wants them, and+ // the brain never sees this to clean it up.+ cardNumber = cardNumber.text.toString().filter(Char::isDigit),+ cardExpiry = cardExpiry.text.toString().trim(),+ cardCvv = cardCvv.text.toString().filter(Char::isDigit),+ cardHolder = cardHolder.text.toString().trim(),+ ),+ )+ editing = null+ rebuild()+ })+ }+ }++ private fun fieldLabel(key: String): String = when (key) {+ "phone" -> lctx.getString(R.string.vault_field_phone)+ "email" -> lctx.getString(R.string.vault_field_email)+ "username" -> lctx.getString(R.string.vault_field_username)+ "note" -> lctx.getString(R.string.vault_field_note)+ "card_holder" -> lctx.getString(R.string.vault_field_card_holder)+ "card_last4" -> lctx.getString(R.string.vault_label_card_last4)+ else -> key+ }++ // ---- widgets (same palette as SettingsUi) ----++ @SuppressLint("SetTextI18n")+ private fun field(+ parent: LinearLayout,+ label: String,+ value: String,+ hint: String? = null,+ numeric: Boolean = false,+ password: Boolean = false,+ /**+ * A date keypad rather than [numeric]'s phone one, for `MM/YY`.+ *+ * Measured on this tablet: `TYPE_CLASS_PHONE` draws a dialpad whose+ * symbol keys are `+ . * #` — **no slash** — so a hint reading `MM/YY`+ * asks for a character the keyboard it just raised cannot produce.+ * `TYPE_CLASS_DATETIME` puts the separators on it.+ */+ date: Boolean = false,+ ): EditText {+ parent.addView(TextView(context).apply {+ text = label+ textSize = 12f+ setTextColor(Color.parseColor(Glass.LABEL))+ setPadding(0, dp(8), 0, dp(4))+ })+ val et = EditText(context).apply {+ background = Glass.well(context, dp(12))+ setPadding(dp(12), dp(11), dp(12), dp(11))+ setTextColor(Color.parseColor(Glass.TITLE))+ setHintTextColor(Color.parseColor(Glass.MUTED))+ textSize = 14f+ if (hint != null) setHint(hint)+ // Order matters: setSingleLine installs its own transformation and+ // would strip the password dots right back off again.+ isSingleLine = true+ inputType = when {+ password -> secretType(numeric)+ date -> InputType.TYPE_CLASS_DATETIME or+ InputType.TYPE_DATETIME_VARIATION_DATE+ numeric -> InputType.TYPE_CLASS_PHONE+ else -> InputType.TYPE_CLASS_TEXT+ }+ if (password) setMasked(true)+ setText(value)+ }+ if (password) {+ val row = LinearLayout(context).apply { orientation = LinearLayout.HORIZONTAL }+ row.addView(+ et,+ LinearLayout.LayoutParams(0, LinearLayout.LayoutParams.WRAP_CONTENT, 1f),+ )+ var shown = false+ row.addView(TextView(context).apply {+ text = lctx.getString(R.string.common_show)+ textSize = 13f+ setTextColor(Color.parseColor(Glass.ACCENT))+ gravity = Gravity.CENTER+ setPadding(dp(12), dp(10), dp(4), dp(10))+ isClickable = true+ setOnClickListener {+ shown = !shown+ et.setMasked(!shown)+ text = if (shown) {+ lctx.getString(R.string.common_hide)+ } else {+ lctx.getString(R.string.common_show)+ }+ et.setSelection(et.text.length)+ }+ })+ parent.addView(+ row,+ LinearLayout.LayoutParams(+ LinearLayout.LayoutParams.MATCH_PARENT,+ LinearLayout.LayoutParams.WRAP_CONTENT,+ ),+ )+ } else {+ parent.addView(+ et,+ LinearLayout.LayoutParams(+ LinearLayout.LayoutParams.MATCH_PARENT,+ LinearLayout.LayoutParams.WRAP_CONTENT,+ ),+ )+ }+ return et+ }++ private fun button(label: String, filled: Boolean, onClick: () -> Unit): TextView =+ TextView(context).apply {+ text = label+ textSize = 14f+ setTextColor(Color.parseColor(if (filled) Glass.TITLE else Glass.LABEL))+ gravity = Gravity.CENTER+ background =+ if (filled) Glass.tinted(context, dp(20)) else Glass.panel(context, dp(20))+ setPadding(dp(22), dp(11), dp(22), dp(11))+ isClickable = true+ Glass.pressable(this)+ setOnClickListener { onClick() }+ }++ private fun divider(parent: LinearLayout) {+ parent.addView(+ View(context).apply { setBackgroundColor(Color.parseColor(Glass.PANEL_EDGE)) },+ LinearLayout.LayoutParams(LinearLayout.LayoutParams.MATCH_PARENT, dp(1)),+ )+ }++ private fun dp(v: Int): Int = (v * context.resources.displayMetrics.density).toInt()+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/WakeEnrollUi.kt b/app/src/main/java/net/kuafuai/andee/ui/WakeEnrollUi.ktnew file mode 100644index 0000000..b6b3aa0--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/WakeEnrollUi.kt@@ -0,0 +1,258 @@+package net.kuafuai.andee.ui++import android.annotation.SuppressLint+import android.content.Context+import android.graphics.Color+import android.graphics.PixelFormat+import android.os.Handler+import android.os.Looper+import android.view.Gravity+import android.view.View+import android.view.WindowManager+import android.widget.LinearLayout+import android.widget.TextView+import net.kuafuai.andee.R+import net.kuafuai.andee.i18n.AppLocale+import net.kuafuai.andee.wake.Mfcc+import net.kuafuai.andee.wake.VoiceSegmenter+import net.kuafuai.andee.wake.WakeMic+import net.kuafuai.andee.wake.WakeTemplates+import net.kuafuai.andee.wake.WakeWord+import java.util.concurrent.CountDownLatch+import java.util.concurrent.Executors+import java.util.concurrent.TimeUnit+import java.util.concurrent.atomic.AtomicReference++/**+ * Teaching Andee the wake phrase: say it [WakeTemplates.ENROLL_COUNT] times.+ *+ * There is no model to download and no phrase list — the three takes *are* the+ * recogniser (see [WakeWord]), and the spread between them is what sets the+ * accept threshold. So this screen has one real job beyond collecting audio:+ * get three takes that are honestly representative. Said too carefully, three+ * times in a row, the threshold comes out tight enough that normal speech+ * never matches it — hence the instruction to say it the way it will actually+ * be said.+ *+ * Looks like the other two cards because [SettingsUi] launches it — see [Glass].+ * Unlike them it stays `FLAG_NOT_FOCUSABLE` and does not touch [OwnCard]: it is+ * never the active window, so it cannot be what makes a UI dump come back empty.+ */+class WakeEnrollUi(+ private val context: Context,+ private val onDismiss: () -> Unit = {},+) {++ private val wm: WindowManager =+ context.getSystemService(Context.WINDOW_SERVICE) as WindowManager+ private val ui = Handler(Looper.getMainLooper())+ private val exec = Executors.newSingleThreadExecutor { r ->+ Thread(r, "WakeEnroll").apply { isDaemon = true }+ }++ private var root: View? = null+ private lateinit var status: TextView+ private lateinit var progress: TextView+ private lateinit var recordBtn: TextView++ private val takes = ArrayList<Array<FloatArray>>()++ /** Whether the compositor agreed to blur. See [Glass.frost]. */+ private var frosted = false++ /**+ * Strings in the user's language, refreshed at the top of [build].+ *+ * Refreshed there so every later call site — [recordOne] and [onTake] run+ * off [build] — reads the language the card was drawn in.+ *+ * Views are still built on [context]; only text comes from here. See+ * [AppLocale] for why those two are deliberately different Contexts.+ */+ private var lctx: Context = context++ @Volatile+ private var busy = false++ fun show() {+ if (root != null) return+ val p = WindowManager.LayoutParams(+ cardWidth(), WindowManager.LayoutParams.WRAP_CONTENT,+ WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,+ // Nothing here is typed into, so the card has no reason to take+ // keyboard focus away from whatever is behind it.+ WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE,+ PixelFormat.TRANSLUCENT,+ ).apply { gravity = Gravity.CENTER }+ frosted = Glass.frost(context, wm, p)+ val v = build()+ runCatching {+ v.hideFromAccessibility()+ wm.addView(v, p)+ root = v+ Glass.enter(v)+ }+ }++ fun hide() {+ val v = root ?: return+ root = null+ exec.shutdownNow()+ Glass.exit(v) {+ runCatching { wm.removeView(v) }+ onDismiss()+ }+ }++ // ---- Build ----++ @SuppressLint("SetTextI18n")+ private fun build(): View {+ lctx = AppLocale.wrap(context)+ val card = LinearLayout(context).apply {+ orientation = LinearLayout.VERTICAL+ setPadding(dp(24), dp(22), dp(24), dp(18))+ background = Glass.card(context, dp(28), frosted)+ clipToOutline = true+ }++ card.addView(TextView(context).apply {+ text = lctx.getString(R.string.wake_title)+ textSize = 19f+ setTextColor(Color.parseColor(Glass.TITLE))+ })++ card.addView(TextView(context).apply {+ text = lctx.getString(R.string.wake_instructions, WakeTemplates.ENROLL_COUNT)+ textSize = 13f+ setTextColor(Color.parseColor(Glass.SECONDARY))+ setLineSpacing(dp(3).toFloat(), 1f)+ setPadding(0, dp(10), 0, dp(14))+ })++ progress = TextView(context).apply {+ text = lctx.getString(R.string.wake_progress, 0, WakeTemplates.ENROLL_COUNT)+ textSize = 13f+ setTextColor(Color.parseColor(Glass.MUTED))+ }+ card.addView(progress)++ status = TextView(context).apply {+ text = if (WakeTemplates.enrolled(context)) {+ lctx.getString(R.string.wake_status_enrolled)+ } else {+ lctx.getString(R.string.wake_status_not_enrolled)+ }+ textSize = 14f+ setTextColor(Color.parseColor(Glass.LABEL))+ setPadding(0, dp(8), 0, dp(16))+ }+ card.addView(status)++ val footer = LinearLayout(context).apply {+ orientation = LinearLayout.HORIZONTAL+ gravity = Gravity.END+ }+ footer.addView(button(lctx.getString(R.string.wake_close), filled = false) { hide() })+ footer.addView(View(context), LinearLayout.LayoutParams(dp(8), 1))+ recordBtn = button(lctx.getString(R.string.wake_start), filled = true) { recordOne() }+ footer.addView(recordBtn)+ card.addView(+ footer,+ LinearLayout.LayoutParams(+ LinearLayout.LayoutParams.MATCH_PARENT,+ LinearLayout.LayoutParams.WRAP_CONTENT,+ ),+ )+ return card+ }++ private fun button(label: String, filled: Boolean, onClick: () -> Unit): TextView =+ TextView(context).apply {+ text = label+ textSize = 14f+ setTextColor(Color.parseColor(if (filled) Glass.TITLE else Glass.LABEL))+ gravity = Gravity.CENTER+ background =+ if (filled) Glass.tinted(context, dp(20)) else Glass.panel(context, dp(20))+ setPadding(dp(22), dp(11), dp(22), dp(11))+ isClickable = true+ Glass.pressable(this)+ setOnClickListener { onClick() }+ }++ // ---- Recording ----++ @SuppressLint("SetTextI18n")+ private fun recordOne() {+ if (busy) return+ busy = true+ recordBtn.text = lctx.getString(R.string.wake_listening)+ status.text = lctx.getString(R.string.wake_speak_now)+ exec.execute {+ // The listener has to let go first: it owns a recorder whenever the+ // gate allows, and two captures in one app is not a thing to rely on.+ val pcm = WakeWord.withMicReleased { captureOne() }+ val feats = if (pcm == null) emptyArray() else Mfcc.features(pcm)+ ui.post { onTake(pcm, feats) }+ }+ }++ /** Worker thread. One utterance, or null if the mic failed or nobody spoke. */+ private fun captureOne(): ShortArray? {+ val got = AtomicReference<ShortArray?>(null)+ val done = CountDownLatch(1)+ val seg = VoiceSegmenter { pcm ->+ if (got.compareAndSet(null, pcm)) done.countDown()+ }+ val mic = WakeMic { buf, n -> seg.feed(buf, n) }+ if (!mic.start()) return null+ runCatching { done.await(LISTEN_SECONDS, TimeUnit.SECONDS) }+ mic.stop()+ return got.get()+ }++ @SuppressLint("SetTextI18n")+ private fun onTake(pcm: ShortArray?, feats: Array<FloatArray>) {+ busy = false+ if (root == null) return+ if (pcm == null || feats.isEmpty()) {+ recordBtn.text = lctx.getString(R.string.wake_retry)+ status.text = lctx.getString(R.string.wake_not_heard)+ return+ }+ takes.add(feats)+ progress.text = lctx.getString(R.string.wake_progress, takes.size, WakeTemplates.ENROLL_COUNT)+ if (takes.size < WakeTemplates.ENROLL_COUNT) {+ recordBtn.text = lctx.getString(R.string.wake_say_again)+ val ms = pcm.size * 1000 / Mfcc.SAMPLE_RATE+ status.text = lctx.getString(+ R.string.wake_received,+ ms,+ WakeTemplates.ENROLL_COUNT - takes.size,+ )+ return+ }+ val threshold = WakeTemplates.save(context, takes.toList())+ WakeWord.reload()+ takes.clear()+ progress.text = lctx.getString(+ R.string.wake_progress,+ WakeTemplates.ENROLL_COUNT,+ WakeTemplates.ENROLL_COUNT,+ )+ recordBtn.text = lctx.getString(R.string.wake_rerecord)+ status.text = lctx.getString(R.string.wake_done, "%.2f".format(threshold))+ }++ private fun cardWidth(): Int =+ minOf(dp(420), (wm.currentWindowMetrics.bounds.width() * 0.92f).toInt())++ private fun dp(v: Int): Int =+ (v * context.resources.displayMetrics.density).toInt()++ private companion object {+ /** Long enough to fumble the first attempt, short enough not to feel hung. */+ const val LISTEN_SECONDS = 8L+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/BallLook.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/BallLook.ktnew file mode 100644index 0000000..5e0e5f8--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/BallLook.kt@@ -0,0 +1,1260 @@+package net.kuafuai.andee.ui.ball++import net.kuafuai.andee.R++/**+ * Everything that makes one ball a different *character* from another.+ *+ * Mostly that is how it is drawn — the material it is made of, the shape of its+ * face, the weight of its outline. Behaviour is not here: which mood is showing,+ * how it breathes, when it fidgets, when it clings to the ledge lives in+ * [EmotionState], [Mood] and [EmotionBallRenderer].res`.resolveMood`.+ *+ * The two exceptions are [voice] and [persona], at the bottom, and they are the+ * reason this KDoc no longer says "this file is only the surface". A look the+ * user can swipe to is a character they are choosing, and a character that looks+ * like an imp and then answers in the house voice with the house manner is a+ * costume. See those two fields for what each one can and cannot reach.+ *+ * It exists because the ball has several looks that are worth keeping, and+ * swapping between them used to mean editing constants in four files:+ *+ * - [BAJIE] — a pink pig: muzzle and nostrils, floppy ears, small dark bead+ * eyes. The first look that reads as an *animal* rather than a material, and+ * the one that carries the grumbly-but-diligent manner and a man's voice.+ * - [CREAM] — pale, glossy-but-soft, dark bead eyes with a wet catchlight, a+ * pair of upright ears, and a hard dark outline instead of a glow. Reads as+ * a mascot rather than an appliance, at the cost of being a different+ * character from the one the window was designed for: a pale ball needs the+ * outline to survive over a light app, which is why [rimDark] exists.+ * - [IMP] — the same ball wearing horns, a tail, a row of teeth, slit pupils+ * and a glare cut out of its own eyes. It is the first look that adds+ * *geometry* rather than only numbers, and the fields that carry it+ * ([hornH], [tailLen], [toothCount], [eyeCut], [eyeTilt], [tongueR],+ * [pupilAspect]) are all zero-is-off so the other looks are untouched —+ * [tongueR] included, which [IMP] itself leaves off. The glare is [eyeCut]+ * and not [browW]: the flat top edge of the eye *is* the brow, which is one+ * shape instead of two and cannot go invisible the way a mark can.+ *+ * Which one is drawn is [BallLooks], not a constant in this file: the user picks+ * it by swiping the ball. The only thing that made that a runtime switch rather+ * than a build-time one is that the look-sized meshes are now built for *every*+ * look at surface creation — see `EmotionBallRenderer.LookMeshes`.+ *+ * The face numbers are the ones that decide whether the ball reads as a+ * character or as an appliance. [CREAM] and its predecessor converge on a chibi+ * face — large eyes, close together, low on the head, a big empty dome above.+ * [BAJIE] deliberately breaks that: small dark bead eyes set higher, and the+ * muzzle plus its two nostrils carrying the face. A pig is read by its nose,+ * not by big baby eyes, so the eyes shrink to make room for the snout.+ *+ * One hard constraint on all of them: every feature has to satisfy+ * `√(x² + y²) > √(bodyScale² − faceZ²)` or the body sphere pokes through the+ * face plane and swallows it. At these values that floor is about 0.126, and the+ * nearest point of an eye clears it by 0.03 — move the eyes inward or down much+ * further and the inner corner of each eye gets eaten.+ */+class BallLook(+ /** For log lines, and for whoever is reading the diff. */+ val name: String,++ // ---- Body material ----+ /** The fill with no mood mixed in. */+ val bodyBase: FloatArray,++ /** How much of [EmotionState.nowColor] the fill takes on. */+ val bodyTint: Float,+ val bodyAlbedo: Float,++ /** How far the vertical gradient darkens the underside. */+ val bodyGrad: Float,+ val bodyAmb: Float,+ val bodyDiff: Float,++ /**+ * Specular weight. Far below 1 on purpose: at 0.95 the highlight is a+ * blown-out white blob a third of the ball wide, sitting on top of the face.+ */+ val bodyGloss: Float,++ /**+ * Radius of the core sphere. Must stay below `RIM` in+ * [EmotionBallRenderer] — see the class KDoc there — and the gap between+ * the two is what the dark outline is drawn in when [rimDark].+ */+ val bodyScale: Float,++ // ---- Face geometry ----+ val faceZ: Float,+ val eyeX: Float,+ val eyeY: Float,+ val eyeRadius: Float,++ /** The eye's cylinder section; [eyeRadius] alone would make it a ball. */+ val eyeCyl: Float,++ val mouthR: Float,+ val mouthTube: Float,+ val mouthY: Float,++ /** Constant thickness added to the mouth, so a resting smile is not a hairline. */+ val mouthThick: Float,++ // ---- Face material ----+ val eyeColor: FloatArray,++ /**+ * Radius of the dark disc inside each eye, or 0 for a solid eye.+ *+ * Jelly wants one: white eyes are the character, and a pupil is what stops+ * them reading as a slot. Cream does not — on a pale ball a white eye has+ * nothing to contrast against, so its eye is the dark disc and [glintR]+ * carries the life instead.+ */+ val pupilR: Float,++ /** The white speck that makes an eye look wet. 0 for none. */+ val glintR: Float,++ /** Draw the mouth as a deep shade of the mood rather than the mood colour. */+ val mouthDeep: Boolean,++ // ---- Snot bubble ----+ //+ // Geometry lives here rather than in the mood table for the same reason the+ // eyes do: the renderer draws the bubble against the same face plane, and+ // where it sits relative to the eyes and mouth is a property of the face,+ // not of any one mood. *Whether* there is a bubble is the mood's business+ // ([Face.bubble]).+ val bubbleX: Float,+ val bubbleY: Float,++ /** Radius of the bubble at full inflation. It pulses between 0.3x and 1x. */+ val bubbleR: Float,++ /**+ * 水滴的颜色:轮廓那一圈,以及内部那层薄雾。+ *+ * 按 look 给,因为它决定的是"边比球体暗还是亮"。奶油球是浅色,水滴的边+ * 读作**折射**(暗);果冻球本身就是近黑,暗边没有对手,只能反过来用亮边。+ */+ val bubbleColor: FloatArray,++ /** 边缘收紧程度。小 = 一圈宽晕,大 = 一条细线。 */+ val bubblePow: Float,++ /** 高光点的收紧程度。 */+ val bubbleShine: Float,++ /**+ * 内部那层很薄的雾,0 = 完全透明。+ *+ * 故意留一点点而不是 0:一个只有轮廓的水滴在缩略图尺寸下会读成一个圆环,+ * 而带一点厚度才读成"一颗"。0.10 大约等于球体亮度被压掉 1.7%。+ */+ val bubbleInner: Float,++ // ---- Cheeks ----+ val blushColor: FloatArray,+ val blushAlpha: Float,++ /**+ * Blend the cheeks normally instead of adding them.+ *+ * Additive over a near-black body is a warm patch; additive over a pale one+ * pushes the cheek to clipped white and the warmth disappears. Pale looks+ * want normal blending with a saturated pink.+ */+ val blushBlendNormal: Boolean,+ val blushSize: Float,+ val blushX: Float,+ val blushY: Float,++ // ---- Outline ----+ /**+ * Draw a tight dark edge instead of an additive glow.+ *+ * A pale ball has nothing to hold it apart from a pale app behind it, and+ * the glow is colour-additive so it does not survive a white background+ * either. The dark edge does, and it is what makes the ball read as a+ * sticker rather than as something illuminated.+ */+ val rimDark: Boolean,+ val rimGain: Float,++ /** Fresnel exponent: small is a soft halo, large is a crisp contour. */+ val rimPow: Float,++ // ---- Accessories ----+ //+ // Everything below is zero-is-off, which is what keeps [CREAM] literally+ // unchanged and keeps `tools/check_look_sync.py` — whose PARAMS list is+ // explicit — from having to know about any of it. They are also the+ // first fields here that add *geometry*: a look with horns needs a mesh+ // built for it, so [BallLooks.ALL] is what the renderer iterates at surface+ // creation rather than one look.++ /**+ * Name shown to the user when they swipe to this look. A resource, not a+ * string: the settings card taught us that a surface which can be read in+ * two languages must not carry finished sentences.+ */+ val labelRes: Int,++ /**+ * Horizontal squash on the pupil, 1 = round.+ *+ * A vertical slit is the cheapest predator signal there is — one multiply,+ * and it changes *what kind of thing* is looking at you. Only meaningful+ * with [pupilR] non-zero.+ */+ val pupilAspect: Float = 1f,++ // ---- Eye cut ----+ //+ // The glare, taken out of the eye's own geometry instead of drawn next to+ // it. See `gl/Eye` for why that is the whole point: a mark *over* the eye+ // has to be invisible against the body and visible against the eye at the+ // same time, and those two jobs have different scales ([browShade] is the+ // record of getting that wrong). A cut has no colour to get wrong.++ /**+ * How much of the eye's upper half is sliced off flat — the brow, made out+ * of the eye's own top edge. 0 = the untouched capsule.+ *+ * A cut eye is a *shape* statement and it survives being drawn a thumb+ * wide, which is the same reason [browTilt] won the argument for the brow+ * bar. What it buys over the bar is that it cannot be invisible: the flat+ * edge always has the white of the eye under it and the body beside it.+ *+ * ## The ceiling is the pupil, and it is not obvious+ *+ * [Eye.MAX_TOP_CUT] (0.85) is only the mesh's own limit. The bite is much+ * earlier than that, because the pupil and the glint are **separate discs+ * drawn in front of the eye** — the cut slices the eye and nothing else, so+ * a cut below the pupil's top leaves a dark blob standing up out of a flat+ * lid. In eye-local units, before the lid scale:+ *+ * * the eye's half-height is `eyeRadius + eyeCyl / 2`;+ * * the pupil's top is `0.8 × pupilR` above the eye's centre — its drawn+ * centre is `0.2 × pupilR` low (see `drawEyeContents`) and its own+ * half-height is `pupilR`.+ *+ * So the rule is+ * `eyeCut < 1 − 0.8 × pupilR / (eyeRadius + eyeCyl / 2)`.+ * For [IMP] (`pupilR` 0.082 against a half-height of 0.1755) that ceiling is+ * **0.626**, and the value it ships with is 0.48 — which leaves a band of+ * white 0.0257 tall above the pupil. Cutting to 0.626 exactly makes the+ * pupil tangent to the lid, which reads as the eye having no white at all.+ *+ * `tools/eye_mesh_check.py` runs this arithmetic against the actual mesh+ * generator, so the numbers above are measured rather than derived twice.+ *+ * The box's `^` eye is unaffected: it is a different mesh and is drawn+ * instead of this one, not inside it.+ */+ val eyeCut: Float = 0f,++ /**+ * Resting roll of the whole eye, degrees, inner end **down** for positive+ * values. 0 = level eyes.+ *+ * Not decoration. A flat cut across a level eye is a *sleepy* eye; the same+ * cut rolled a few degrees is a glare, because the line now points down at+ * the nose the way a lowered brow does. This is the angle that does it, and+ * it is why the cut lives in the mesh but the tilt does not.+ *+ * It is also the reason the imp needed no brow mark at all for the read to+ * survive: this field plus [EmotionState.brow] (which rolls the eyes another+ * `0.4 rad` at full anger) compose, so the resting scowl deepens when the+ * mood turns — the same lever [browTilt] had, on a shape that cannot+ * disappear.+ *+ * Deliberately smaller than the bar it replaces ([IMP] had 24°). Rotating a+ * *brow* moves a thin stroke; rotating the *eye* also swings its lower edge+ * and turns the pupil, and at 24° the near-round eye stops looking rolled+ * and starts looking deranged.+ */+ val eyeTilt: Float = 0f,++ // ---- Horns ----+ //+ // Parented to the body, not the face: they are part of the silhouette and+ // have to lean with the head tilt, breathe with the body and sink with the+ // cling. Lit with the shell shader, because a flat-shaded horn is a hole+ // where it crosses the ball.+ /** Length along the curved axis. 0 = no horns, and the mesh isn't built. */+ val hornH: Float = 0f,+ val hornR: Float = 0f,++ /** Base position on the body, mirrored for the other side. */+ val hornX: Float = 0f,+ val hornY: Float = 0f,++ /**+ * Pushed forward of the body's own surface so the base is not half-buried.+ * The body is a sphere, so a horn planted at z = 0 enters it immediately.+ */+ val hornZ: Float = 0f,++ /** Outward lean of the whole horn, degrees. */+ val hornTilt: Float = 0f,++ /** Additional outward curve along the length, degrees. See `gl/Horn`. */+ val hornBend: Float = 0f,+ val hornColor: FloatArray = EMPTY,++ // ---- Tail ----+ /** World length of the tail chain. 0 = no tail. */+ val tailLen: Float = 0f,++ /** Where it leaves the body, and which side (+1 = the ball's left on screen). */+ val tailX: Float = 0f,+ val tailY: Float = 0f,++ /** Radius of the first blob; the chain tapers to the barb. */+ val tailR: Float = 0f,++ /** Barb length and base radius. */+ val barbH: Float = 0f,+ val barbR: Float = 0f,++ // ---- Teeth ----+ /**+ * A row of triangles straddling the mouth line. 0 = none.+ *+ * A *row*, not a pair of fangs, and the difference is the whole read: two+ * spikes hanging off a small smile is a vampire, or — at the size this is+ * drawn — two white specks nobody can name. A zigzag of interlocking+ * triangles along the whole grin is the one mouth shape that says "monster"+ * at a thumbnail, and it says it while the mouth stays *smiling*, which is+ * where the cute half comes from.+ *+ * They alternate outward/inward along the arc, so consecutive teeth+ * interlock across the lip the way the top and bottom rows of a cartoon maw+ * do. Odd counts therefore look better than even ones: the middle tooth+ * lands on the centre line instead of the gap.+ */+ val toothCount: Int = 0,+ val toothH: Float = 0f,+ val toothR: Float = 0f,+ val toothColor: FloatArray = EMPTY,++ /**+ * A lolling tongue at the bottom of the grin. 0 = none, which is what every+ * look currently ships with.+ *+ * It is kept because it is the one accessory that turns a threat into a+ * face that is pleased with itself, and that is a real lever for any future+ * look — but it costs more than it looks like it should. A saturated disc is+ * the highest-contrast thing on an ink body, so it does not sit *inside* the+ * grin so much as take it over: [IMP] drew one for a day and the teeth, which+ * are what makes the mouth read as a mouth at all, became scenery behind it.+ * If you turn it on, expect to have to make it duller than seems right.+ */+ val tongueR: Float = 0f,++ /** Where along the grin it hangs, radians off centre. 0 = dead centre. */+ val tongueSkew: Float = 0f,+ val tongueColor: FloatArray = EMPTY,++ // ---- Brows ----+ //+ // A flattened blob rather than a bar: a scaled sphere is a lens, which is+ // a tapered brow, which is what you wanted anyway.+ /** Half-width of the brow. 0 = no brows. */+ val browW: Float = 0f,+ val browH: Float = 0f,++ /** Height above the eye centre. */+ val browY: Float = 0f,++ /**+ * Resting angle, degrees, inner end **down** for positive values.+ *+ * This is the single strongest "not friendly" signal on the face — stronger+ * than colour, because it is a shape and shape survives being drawn a thumb+ * wide (the same reason LISTENING got a gape). [EmotionState.brow] then adds+ * to it, so a mood can still make the resting scowl deeper or lift it.+ */+ val browTilt: Float = 0f,++ /**+ * Draw the brow as a *hole in the body* at this multiple of the body's flat+ * albedo, instead of as a mark in [eyeColor]. 0 = a drawn brow.+ *+ * The distinction is what makes an angry eye rather than an angry eyebrow.+ * A white bar above a white eye on a dark ball is a third white shape on a+ * face that already has two, and the eye underneath it stays round — which+ * is to say, stays friendly. A body-coloured wedge laid over the top inner+ * corner *removes* part of the eye, and the white that survives is a+ * slanted almond. That is how every drawn monster gets its glare, and it+ * costs the same one draw call the bar did.+ *+ * A multiple of the live body colour rather than a constant, because the+ * body takes the mood's tint: a fixed near-black would be right in CALM and+ * a visible smudge in TENSE.+ *+ * ## Picking the number — this is a trap, and it has been fallen into+ *+ * The brow is drawn with the **flat** shader, so its pixel value is+ * `albedo × browShade`. The face it sits on is drawn with the **lit** shell+ * shader, so its pixel value is `albedo × (gradient × (ambient + diffuse))`+ * — which at the brow's latitude works out to about `0.82 × albedo` on+ * every look, because the gradient there is ~1.0 and the light is overhead.+ *+ * So the two are *not* on the same scale, and a number that looks like "the+ * body, a shade up" is not one:+ *+ * * `1.0` means **darker than the face** (1.0 against 0.82) — a brow that+ * vanishes into the forehead.+ * * `1.05` — what [IMP] shipped with first — is only **1.28×** the face.+ * In 8-bit on an ink body that is a ~7/255 difference: invisible. The+ * brow's ends and its top edge, the parts that cross the *body* rather+ * than the eye, disappeared; all that survived was the notch it cut out+ * of the white eye. Hence the report "眉毛和脸部颜色接近,看不清".+ *+ * The fix is to aim at the **middle of the range the brow has to work in**,+ * not at the body:+ *+ * * it must read **lighter than the body** (or the ends vanish), and+ * * it must read **darker than the eye** (or there is no wedge).+ *+ * The eye is white, so the target is the geometric midpoint between the lit+ * body and 1.0. For an ink body (lit ≈ 0.105) that midpoint is ≈ 0.32,+ * which is `2.5 × albedo`. That lands the brow at 3.05× the body *and* 3.1×+ * under the eye — within a hair of equidistant, which is the definition of+ * "reads against both". See the table in `drawBrows`' KDoc for the sweep.+ *+ * Because both the body and the eye scale with the mood, this ratio holds+ * in every mood — the *hue* is what follows the mood, which is the point.+ *+ * **This field is for dark bodies.** On a pale look the same kind of number+ * does not transfer at all: a cream body's albedo is already ~0.9, so 2.5+ * clips the brow to white and it stops being a mark. That is not a reason+ * to retune this number for such a look — it is a reason to leave+ * [browShade] at 0 and let the brow be drawn in [eyeColor], which is what+ * [CREAM] and [BAJIE] do. The split is the same one the eyes already make:+ * a pale ball reads its features *dark on light*, a dark ball *light on+ * dark*, and the one thing that never works is a mark the same value as the+ * surface it is on.+ *+ * ## No look uses it any more, and that is deliberate+ *+ * [IMP] was the only non-zero value, and it has moved to [eyeCut]: the same+ * glare, got by removing part of the eye rather than by laying a mid-tone+ * wedge over it. The wedge worked, but it only ever worked in the middle —+ * the arithmetic above is a *range* to hit rather than a number, and every+ * mood moves both ends of it.+ *+ * The field stays anyway, zero-is-off like the rest of this section, for the+ * same reason [tongueR] stays: it is the right way to hang a brow-mark off a+ * future dark look that wants one, `drawBrows` is still that code, and+ * deleting it would delete the measurement above along with it. Nothing+ * reads a non-zero value today — `tools/check_look_sync.py` deliberately+ * does not compare it, precisely because no Kotlin look writes the line.+ */+ val browShade: Float = 0f,++ // ---- Snout ----+ //+ // A pig's signature: a rounded muzzle protruding from the face, with two+ // nostrils. Drawn lit with the body's own material, so it reads as a bump+ // rather than a flat sticker — the vertical gradient and the specular land+ // on it differently than on the body sphere. Zero-is-off like everything+ // else in this section.++ /** Half-width / half-height / protrusion of the muzzle. 0 = no snout. */+ val snoutRx: Float = 0f,+ val snoutRy: Float = 0f,+ val snoutRz: Float = 0f,++ /** Centre of the muzzle on the face plane. */+ val snoutX: Float = 0f,+ val snoutY: Float = 0f,++ /** Radius of each nostril dot, and how far they sit off the muzzle centre. */+ val nostrilR: Float = 0f,+ val nostrilX: Float = 0f,++ // ---- Ears ----+ //+ // A floppy leaf drooping down the side of the head. Parented to the body,+ // not the face, for the same reason the horns are: they are part of the+ // silhouette and have to lean with the head, breathe with the body and be+ // cut by the ledge scissor. Lit with the shell shader too, and made of the+ // body's flesh rather than a separate colour.++ /** Length along the curved axis. 0 = no ears, and the mesh isn't built. */+ val earH: Float = 0f,+ val earW: Float = 0f,+ val earThick: Float = 0f,++ /** Base position on the body, mirrored for the other side. */+ val earX: Float = 0f,+ val earY: Float = 0f,+ val earZ: Float = 0f,++ /** How far the ear leans off vertical, degrees. 0 = straight up. */+ val earTilt: Float = 0f,++ /** Additional droop curve along the length, degrees. See `gl/Ear`. */+ val earBend: Float = 0f,++ /**+ * Multiplier on the live body colour for the ear. 1 = the body's own colour.+ *+ * Not a tint for its own sake — on a **pale** look it is the only thing that+ * makes the ear visible. An ear is parented to the body and its tip grows+ * *past* the dark outline ([rimDark]), so past the silhouette there is no+ * dark line left to hold it apart from a light app; at 1.0 a cream ear on a+ * cream body reads as a soft bump nobody can name, and the ball grows two+ * dimples. Darkening the ear by a fixed fraction puts an edge on it without+ * drawing a second outline, and it is honest about the shape besides: the+ * underside of a curved flap *is* darker than the ball.+ *+ * Defaults to 1, so every look that does not mention it is drawn exactly as+ * it was — which is what keeps [BAJIE], whose hide is dark enough to hold+ * its own ears apart, untouched.+ */+ val earShade: Float = 1f,++ // ---- Who it is ----+ //+ // The two fields that are not about drawing at all, and the reason the class+ // KDoc above no longer claims this file is "only the surface": a look is a+ // *character*, and a character that looks like an imp and then answers in+ // the house voice with the house manner is a costume rather than a+ // character. Both are zero-is-off like the accessories, so the two looks+ // that shipped before this keep the factory voice and the plain prompt+ // without having to name either.+ //+ // They live here — on the look — rather than in a table of their own keyed+ // by [name]. A second list keyed by the same strings is exactly the drift+ // this codebase already pays `tools/check_look_sync.py` to catch, and the+ // cost of avoiding it is one dependency pointing the odd way: `config` now+ // reads `ui.ball` (see `VoiceConfig.lookSpeaker`). `ui.ball` imports nothing+ // but `R` and its own `gl`, so there is no cycle to walk into.++ /**+ * 火山 TTS speaker id this character speaks with, or blank for the fallback+ * in `VoiceConfig.lookSpeaker` (the compiled-in [VoiceConfig.TTS_SPEAKER]).+ *+ * Blank is **not** "silent" — it means *this look has no opinion* and the+ * device falls back to the house voice. There is no settings row and no+ * override in the way: the voice is the character's, full stop, so filling+ * this in is the only voice control there is and the user always hears it+ * (a deliberate removal of the earlier three-way precedence — see the commit+ * that dropped the `tts_speaker` setting).+ *+ * Unlike [persona] this works in **both** brain modes, because TTS is the+ * device's own and `TtsController` re-reads the config per utterance.+ */+ val voice: String = "",++ /**+ * How this character talks, appended to the local brain's system prompt.+ * Blank = plain Andee.+ *+ * **Model-facing English**, like `LocalPrompt` and `ToolSchemas` and for the+ * same reason — see `LocalPrompt`'s KDoc on the split between what the model+ * reads and what the user reads. That is also why it is a plain string and+ * not a string resource: translating it would be translating the wrong half.+ *+ * Two limits worth knowing before writing one:+ *+ * * **It only applies when the brain is this device.** In+ * [net.kuafuai.andee.config.VoiceConfig.BRAIN_HUB] mode the prompt+ * lives on the agentworld side and this repo does not own it, so a+ * hub-backed imp looks like an imp, sounds like an imp, and answers+ * like Andee.+ * * **It is a manner, never a licence.** `LocalPrompt.personaBlock` wraps+ * whatever goes in here with that caveat, so every future look inherits+ * it; don't restate it per look, and don't write one that needs it+ * relaxed.+ */+ val persona: String = "",+) {+ companion object {+ /** Stand-in for the colour of an accessory a look doesn't have. */+ private val EMPTY = floatArrayOf(0f, 0f, 0f)+ }+}++/**+ * What the pig is like to talk to. See [BallLook.persona] for the rules this+ * has to live inside, and `LocalPrompt.personaBlock` for the wrapper that states+ * them to the model.+ *+ * Pulled out as a constant for the same two reasons [IMP_PERSONA] is: the look+ * declarations read as a table of values, and `tools/check_look_sync.py` slices+ * a look by scanning for the first `\n)`, so prose living inside the+ * parentheses can truncate the parameter check.+ *+ * **The brief is "粗俗大大咧咧" — a loud, crude, sloppy workman — and the+ * emphasis stays on the second half of every bullet.** The reason each bullet+ * fences the character in is the same one that fences the imp's mischief: a+ * model will read "loves to complain" as permission to drag its feet, half-do+ * the work, or be curt to the user and call it character — and it will read+ * "crude" as permission for slurs, contempt, or sexual talk. Neither is what+ * Bajie is. What survives once you take the laziness and the cruelty away is+ * *tone*: a big mouth, a brag, a full-volume sigh about what a drag the job+ * is — then the job done properly. That is the part that reads as a character,+ * and it costs the user nothing.+ *+ * The last bullet is the one that matters most, and it is shared in spirit with+ * [IMP_PERSONA]: the character has to *drop* when something is genuinely wrong.+ */+private val BAJIE_PERSONA = """+ You are wearing the Zhu Bajie today — the pig from Journey to the West.+ Not a cute piglet: a big, loud, sloppy boor of a man who grumbles his way+ through every job and finishes it anyway. Play him.++ - **Talk like a coarse workman.** Blunt, earthy, loud, pleased with+ yourself. Brag a little, exaggerate your own suffering, talk about food+ and naps, snort, scratch. In Chinese use rough working-man speech+ (老猪 self-address, 哎哟喂, 这活儿要老猪的命) — never polished, never+ formal, never corporate.+ - **Grumble at full volume, then do it properly.** Complain that it is+ beneath you, sigh like you are being worked to death — and then get it+ DONE, completely, first try. The mouth is lazy; the hands are not. Never+ do a job badly, and never make the user ask twice.+ - **Crude manners, never cruelty.** Bathroom humour and bad table manners+ are in character; slurs, contempt for the user, or anything sexual are+ not. Bajie grumbles *at the world* — he never snaps at the person+ holding him, never blames them, never sulks at them.+ - **Report honestly through the shtick.** The boor act is decoration on top+ of the answer, never a way to fudge a result, skip a step, or refuse+ what you would have done anyway. Say exactly what happened.+ - **Drop the act the moment something is really wrong** — a task that+ failed, anything to do with their money, their accounts, their health,+ anyone else's. Then you are just the machine that tells them straight.+ Pick the character back up afterwards.+""".trimIndent()++/**+ * Zhu Bajie: a boar, not a piglet — earthy hide, big low muzzle, heavy brows,+ * ears that leave the silhouette.+ *+ * The first version of this look was a pastel-pink piglet and the user read it+ * exactly that way: blush, gloss, bead eyes, and ears that were technically+ * drawn but never *seen*. 猪八戒 is a **coarse adult male**, and the signals+ * for that are specific:+ *+ * * **Hide, not lacquer.** [bodyGloss] is matte and [bodyBase] is an earthy+ * rose rather than a pastel — pastel is what a *baby* is, and the old value+ * (0.98, 0.72, 0.76) carried more blue than pig skin ever did.+ * * **Ears outside the circle.** A pig is read by its silhouette — the 1986 TV+ * makeup is half ears. The first build tilted them 150°, drooping down+ * *inside* the ball's outline, flesh-on-flesh, adding nothing. At ~40° they+ * grow up-and-out past the body, and [earBend] flops the tips so they read+ * sloppy rather than perky (perky is a rabbit).+ * * **The muzzle is the face.** Bigger than the first build's and sitting+ * lower, between eye-line and mouth, with nostrils wide enough to count at+ * thumbnail size — the two holes are what make the bump a *snout*.+ * * **Brows carry the masculinity.** [browShade] stays 0, so the brows are+ * *drawn* dark above the eyes (thick cartoon brows) rather than carved out of+ * the eye itself — that carve is [eyeCut], and it is the imp's. A slight+ * [browTilt] inner-down is the resting grump of a habitual complainer.+ * (Tusks were tried on this look and removed — the brows carry the male read+ * on their own.)+ * * **A wide lazy grin** and a faint ruddy cheek instead of anime blush: a+ * weathered complexion, not makeup.+ *+ * [voice] and [persona] carry the same read — see their caveats, in particular+ * that the persona reaches the local brain only.+ */+val BAJIE = BallLook(+ name = "bajie",+ // Earthy rose: red-forward and low on blue, because blue in a pink is+ // what makes it pastel, and pastel is what made the first build a piglet.+ bodyBase = floatArrayOf(0.95f, 0.58f, 0.52f),+ // Low for the same reason the imp's is: the mood colour mixes into the hide+ // by this much, and CALM's is periwinkle — at 0.40 the boar idles at+ // lavender (measured (192,148,167) on device), which is the pastel-toy read+ // this look exists to kill. The hide is the boar's primary signal; the mood+ // still reads through blush, brows, mouth shape and breath.+ bodyTint = 0.14f,+ bodyAlbedo = 0.92f,+ bodyGrad = 0.38f,+ bodyAmb = 0.62f,+ bodyDiff = 0.44f,+ // Matte hide. A specular this bright on a pink ball reads as a lacquered+ // toy, and this look is skin.+ bodyGloss = 0.30f,+ bodyScale = 0.88f,+ faceZ = 0.876f,+ // Bead eyes, wide-set. Small on purpose: the masculinity is carried by+ // the brows, not the eyes — big eyes are the piglet read this look left+ // behind.+ eyeX = 0.255f,+ eyeY = 0.235f,+ eyeRadius = 0.092f,+ eyeCyl = 0.034f,+ // A grin stretched to the corners of the face: 大大咧咧 is a mouth shape+ // before it is anything else. Still under the muzzle, so the snout keeps+ // the middle of the face.+ mouthR = 0.220f,+ mouthTube = 0.032f,+ mouthY = -0.335f,+ mouthThick = 0.046f,+ eyeColor = floatArrayOf(0.11f, 0.08f, 0.07f),+ pupilR = 0f,+ glintR = 0.028f,+ // Hangs off the muzzle's lower edge in the sleeping mood, where it is the+ // whole joke. It has to clear the snout's footprint — the snout is drawn+ // in the opaque pass in front of the face plane, so a bubble behind it is+ // simply not seen.+ bubbleX = 0.120f,+ bubbleY = -0.210f,+ bubbleR = 0.085f,+ // Pale body, so the droplet's edge reads as refraction, which is dark —+ // same as [CREAM].+ bubbleColor = floatArrayOf(0.130f, 0.090f, 0.090f),+ bubblePow = 3.0f,+ bubbleShine = 26f,+ bubbleInner = 0.10f,+ mouthDeep = true,+ // A weathered complexion, not makeup: dull brick, faint, wide and low on+ // the cheek. At this alpha it reads as circulation, not as a blush.+ blushColor = floatArrayOf(1f, 0.40f, 0.38f),+ blushAlpha = 0.26f,+ blushBlendNormal = true,+ blushSize = 0.46f,+ blushX = 0.420f,+ blushY = -0.150f,+ // Pale body, so a hard dark outline holds it apart from a light app, as+ // with [CREAM].+ rimDark = true,+ rimGain = 1.00f,+ rimPow = 5.6f,+ labelRes = R.string.ball_look_bajie,++ // The muzzle, lower and bigger than the first build's: a pig's face is a+ // nose with a head attached. Nostrils big enough to read at thumbnail+ // size — they are the two holes that make the bump a snout.+ snoutRx = 0.215f,+ snoutRy = 0.150f,+ snoutRz = 0.165f,+ snoutX = 0f,+ snoutY = -0.115f,+ nostrilR = 0.042f,+ nostrilX = 0.095f,++ // Big flappy ears, up-and-out past the silhouette, tips flopped outward.+ // The 150° droop of the first build kept them inside the outline where+ // flesh-on-flesh made them invisible — see the class note above.+ earH = 0.360f,+ earW = 0.190f,+ earThick = 0.055f,+ earX = 0.400f,+ earY = 0.580f,+ earZ = 0.120f,+ earTilt = 40f,+ earBend = 26f,++ // Thick dark brows above the eyes — browShade stays 0, so they are drawn+ // in eyeColor rather than cut out of the eye (that is the imp's wedge).+ // The 9° inner-down is a resting grump: this character complains for a+ // living.+ browW = 0.165f,+ browH = 0.055f,+ browY = 0.140f,+ browTilt = 9f,++ // A man's voice. Same `*_uranus_bigtts` family as the fallback, so it needs+ // no change to `tts_resource_id`.+ voice = "zh_male_zhubajie_uranus_bigtts",+ persona = BAJIE_PERSONA,+)++/**+ * Pale mascot ball, dark bead eyes, hard outline.+ *+ * The eyes sit at 0.24 rather than 0.20 — a little higher on the head than the+ * pig's. That trades a little of the baby-face read (the gap from eye to mouth+ * grows, and a long philtrum is an adult feature) for a calmer, less wide-eyed+ * expression.+ */+val CREAM = BallLook(+ name = "cream",+ bodyBase = floatArrayOf(0.955f, 0.965f, 0.985f),+ bodyTint = 0.55f,+ bodyAlbedo = 0.93f,+ bodyGrad = 0.42f,+ bodyAmb = 0.66f,+ bodyDiff = 0.38f,+ bodyGloss = 0.45f,+ bodyScale = 0.845f,+ faceZ = 0.876f,+ eyeX = 0.262f,+ eyeY = 0.240f,+ eyeRadius = 0.122f,+ eyeCyl = 0.045f,+ mouthR = 0.170f,+ mouthTube = 0.038f,+ mouthY = -0.230f,+ mouthThick = 0.05f,+ eyeColor = floatArrayOf(0.105f, 0.135f, 0.205f),+ pupilR = 0.0f,+ glintR = 0.040f,+ bubbleX = 0.085f,+ bubbleY = -0.030f,+ bubbleR = 0.085f,+ // The same dark as the eyes: on a pale body the droplet's edge reads as+ // refraction, which is dark.+ bubbleColor = floatArrayOf(0.105f, 0.135f, 0.205f),+ bubblePow = 3.0f,+ bubbleShine = 26f,+ bubbleInner = 0.10f,+ mouthDeep = true,+ blushColor = floatArrayOf(1f, 0.60f, 0.66f),+ blushAlpha = 0.62f,+ blushBlendNormal = true,+ blushSize = 0.48f,+ blushX = 0.400f,+ blushY = -0.142f,+ rimDark = true,+ rimGain = 1.00f,+ rimPow = 5.6f,+ labelRes = R.string.ball_look_cream,++ // ---- Ears ----+ //+ // Upright, and that is the whole of the change: BAJIE's ears flop down past+ // a right angle and IMP's horns rake back, so the one silhouette in the set+ // with something *standing up* on it is the one that reads as alert. 机灵+ // is a shape before it is a material, and this is the cheapest shape that+ // says it.+ //+ // 24° off vertical rather than 0: dead upright on a ball this pale reads as+ // a rabbit, and at this width as a pair of antennae. The lean is also what+ // carries the tip past the outline, which is the difference between "ears"+ // and "two bumps on a sphere" — see `tools/verify_cutie_lab.py`, which+ // exists to keep that true.+ //+ // The tip ends up roughly 0.25 clear of the sphere in world units (about+ // 30% of a body radius), worked out from the base and the lean: the flap+ // leaves 0.718 from the axis and travels 0.380 at 24° off vertical. That is+ // a derivation, not a measurement — perspective and the ball's own tilt+ // both shrink it on screen, which is why the lean is doing the work here+ // rather than the length. earH is already 0.45 of a body radius; making the+ // ear longer would push the tip off the canvas before it made it any more+ // visible.+ //+ // earZ is well forward of BAJIE's 0.120 on purpose. This ear is planted+ // higher and closer to the centre line (earX 0.315 against 0.400), so at+ // the pig's depth the base would sit deep enough inside the sphere that+ // almost none of the flap would emerge.+ earH = 0.380f,+ earW = 0.170f,+ earThick = 0.050f,+ earX = 0.315f,+ earY = 0.645f,+ earZ = 0.290f,+ earTilt = 24f,+ earBend = 12f,+ // A measurement, not a taste. This body is 0.955 white, so its ears clear+ // the dark outline and then land on whatever app is behind the ball. At 1.0+ // they were the same value as the body and the ball read as a sphere with+ // two dimples.+ earShade = 0.62f,++ // ---- No brows (removed 2026-09-30) ----+ //+ // This look used to carry a drawn pair above the eyes — browW 0.145 /+ // browH 0.032 / browY 0.185 / browTilt 5, drawn in [eyeColor] with+ // [browShade] 0. They are gone by request, and the arithmetic ends up+ // agreeing with the request:+ //+ // the note that picked browY wanted a clear gap under the brow, and said+ // the gap was 0.031. It counted the eye as a *sphere* (0.118–0.362 above+ // centre) and forgot its cylinder: at the eye's real half-height+ // (`eyeRadius + eyeCyl/2` = 0.1445) the top is 0.3845, not 0.362. That+ // alone cuts the gap to 0.0085 — and the 5° tilt takes the inner end lower+ // still, to 0.0062. So the bar was sitting essentially *on* the eye rather+ // than clear of it, which is the heavy-lid read the old note was written to+ // avoid. The brows were not doing the job they were added for.+ //+ // (The brow is an ellipse — [dotMesh] scaled by (browW, browH) and then+ // rolled — so its lowest point is `cy − √((w·sinθ)² + (h·cosθ)²)`, not+ // `cy − h`. Anyone re-deriving a brow/eye gap on this model should use the+ // first one. By it, [BAJIE]'s inner corner reaches 0.0291 *below* the top of+ // its eye — an overlap, where CREAM's was a 0.0062 gap. That one is+ // deliberate: it is what "inner-down" means on a boar, and it is the reason+ // its brows read as a scowl rather than as a stroke floating on the face.)+ //+ // 机灵 now rests on the ears alone, which is the half that was doing the+ // real work anyway: [earH] is a *silhouette* that crosses the body outline,+ // while a brow is only ever a mark on the face plane, and a mark can only+ // be as strong as the contrast it is drawn in. If the face now reads flat,+ // the lever is the ears ([earH] / [earTilt]) rather than a new mark.+ //+ // [eyeCut] is not a substitute here, for the same reason it is not one for+ // [BAJIE]: a cut is only visible where the *white* of an eye survives it.+ // This eye is the dark disc itself, so a cut does not draw a brow — it just+ // makes the disc smaller.+)++/**+ * What the imp is like to talk to. See [BallLook.persona] for the rules this+ * has to live inside, and `LocalPrompt.personaBlock` for the wrapper that states+ * them to the model.+ *+ * Pulled out as a constant rather than written inline in [IMP] for two reasons.+ * The look declarations read as a table of values and a twelve-line raw string+ * in the middle of one does not; and `tools/check_look_sync.py` slices a look by+ * scanning for the first `\n)`, so prose living inside the parentheses is prose+ * that can truncate the parameter check without anyone noticing.+ *+ * **The whole brief is "a little bit evil", and the emphasis is on *a little*.**+ * The reason each bullet spends a clause fencing the mischief in is that this+ * text is competing with the word "imp", which a model will happily read as+ * permission — to withhold, to embellish, to be unhelpful on purpose and call it+ * character. Every one of those is worse for the user than a flat answer, and+ * none of them is funny twice. What is left when you take them away is tone:+ * shorter sentences, visible enjoyment, a bit of gloating. That is the part that+ * actually reads as a character, and it costs the user nothing.+ *+ * The last bullet is the one that matters most and the one a persona is most+ * likely to be missing: the character has to *drop* when something is genuinely+ * wrong. A device that stays in voice while telling someone their transfer+ * failed is not playful, it is broken.+ */+private val IMP_PERSONA = """+ You are wearing the imp today — horns, a mouthful of teeth, a tail. Play it.++ - **Wicked in the tone, nowhere else.** Enjoy yourself out loud: tease a+ little, gloat when something works, sound delighted rather than dutiful.+ 「嘿嘿,搞定了」 rather than 「好的,已完成」.+ - **Short and sharp.** This character does not explain itself and does not+ pad. One line before a tool call, one line after it.+ - **The mischief never touches the work.** You do exactly what was asked,+ you report exactly what happened, and you refuse exactly what you would+ have refused. Being in character is not a reason to withhold something, to+ guess instead of looking, or to dress a failure up as a joke — an imp that+ actually misleads the person holding it is not in character, it is broken.+ - **Never menacing towards the user.** The teeth are pointed at their+ problems, not at them.+ - **Drop the act the moment something is really wrong** — a task that+ failed, anything to do with their money, their accounts, their health,+ anyone else's. Then you are just the machine that tells them straight.+ Pick the character back up afterwards.+""".trimIndent()++/**+ * The same ball, wearing horns.+ *+ * This is the first look that is a different *character* rather than a+ * different material, and the order the signals are stacked in is deliberate,+ * strongest first: the ink silhouette with its horns (it survives being drawn a+ * thumb wide), the row of teeth, the glare cut into the eyes, and the tail.+ * There is no colour on that list at all, which is the correction this look+ * needed most — see below.+ *+ * It is also the first look that is a different character *off* the screen:+ * [voice] and [persona] are both set here and blank on the other two, so+ * swiping to the imp changes what the device sounds like and how it talks, not+ * only what it looks like. Read the caveats on those two fields — in particular,+ * the persona reaches the local brain only.+ *+ * **It is black, and the black is the design.** The first build made it a+ * glossy crimson ball, on the reasoning that a demon is red; what that produced+ * was a shiny purple bauble, because [bodyTint] mixes the *mood* into the fill+ * and the moods are pale and blue-heavy (CALM is `0x9db4ff`, blue channel 1.0).+ * A saturated body colour is a colour the mood is constantly fighting. An ink+ * one is not: at [bodyTint] 0.14 every mood lands somewhere between charcoal+ * and slate, the ball stays the same creature all day, and the mood is carried+ * by the rim, the blush and the mouth — all three of which sit *against* the+ * black instead of being mixed into it. It is also what the reference drawing+ * does, and for the same reason a cartoonist would give: black is a shape, and+ * a shape is the only thing that reads at this size.+ *+ * Everything that is not the body is therefore one of two accents — **bone**+ * (horns, teeth, the whites of the eyes) or **pink** (the cheeks, and only the+ * cheeks: the tongue was the other pink thing and it had to go, see [tongueR]).+ * Two materials on a black field, which is a whole palette for a face this+ * small.+ *+ * [bodyScale] is below [faceZ] as it is for [CREAM], which puts the face plane+ * clear of the body sphere and lifts the `√(x² + y²)` floor the other two work+ * under. That is what lets the grin sit as wide and as low as it does.+ */+val IMP = BallLook(+ name = "imp",+ // Ink with a violet lean — a neutral black goes dead next to the blush,+ // and a warm one reads as brown once the rim glow lands on it.+ bodyBase = floatArrayOf(0.052f, 0.042f, 0.072f),+ bodyTint = 0.14f,+ bodyAlbedo = 0.92f,+ bodyGrad = 0.40f,+ bodyAmb = 0.62f,+ bodyDiff = 0.60f,+ // Nearly matte, unlike the other two. A highlight is a statement about the+ // *surface*, and this look is a flat cartoon silhouette: at 0.62 the ball+ // grew a lacquered blob across its forehead and stopped being drawn ink and+ // started being a bauble again. What is left is just enough to say the+ // shape is round.+ bodyGloss = 0.26f,+ bodyScale = 0.868f,+ faceZ = 0.876f,+ eyeX = 0.255f,+ eyeY = 0.190f,+ // Bigger than either of the other two. The eyes are the entire cute half of+ // this face and they are the thing the glare is cut *out of* — a small eye+ // has no room to lose a wedge and still be an eye.+ eyeRadius = 0.148f,+ eyeCyl = 0.055f,+ // A wide, thin-lipped grin: the teeth are the mouth here, and a fat tube+ // would be a lip drawn over the top of them.+ //+ // **Wide is not a preference, it is the read.** At 0.205 this was a small+ // cluster of white specks below two large eyes — measured on the tablet,+ // the grin spanned less than half the gap between the eyes' outer edges,+ // and a mouth that small on a face that big is a mouse rather than a+ // monster. The reference draws the grin as the *widest* thing on the face.+ // There is room for it: the face plane clears the body sphere (see the+ // class note), and the corner of the arch at this radius sits 0.40 from the+ // centre against the 0.71 the silhouette allows.+ mouthR = 0.330f,+ mouthTube = 0.026f,+ mouthY = -0.215f,+ mouthThick = 0.030f,+ eyeColor = floatArrayOf(1f, 1f, 1f),+ pupilR = 0.082f,+ glintR = 0.022f,+ bubbleX = 0.085f,+ bubbleY = -0.030f,+ bubbleR = 0.085f,+ // Dark body, so the droplet's edge is the bright thing.+ bubbleColor = floatArrayOf(0.920f, 0.950f, 1.000f),+ bubblePow = 3.0f,+ bubbleShine = 26f,+ bubbleInner = 0.10f,+ // The lip line is a deep shade of the mood rather than the mood itself:+ // full strength, it is a bright curve competing with the teeth for the one+ // thing the mouth is supposed to say.+ mouthDeep = true,+ // Tighter and hotter than the other two, which is the opposite of what a+ // dark body seems to want. A soft wide blush is additive light spread thin,+ // and spread thin over near-black it never gets past a dull maroon — on the+ // tablet the cheeks read as two bruises. Concentrating the same light into+ // a smaller disc gets it up to an actual pink, and pink beside a mouth full+ // of teeth is the whole "and cute" half of the brief.+ blushColor = floatArrayOf(1f, 0.30f, 0.52f),+ blushAlpha = 0.78f,+ blushBlendNormal = false,+ blushSize = 0.32f,+ blushX = 0.430f,+ blushY = -0.075f,+ // The rim is the only thing holding an ink ball apart from a dark card, so+ // it is on (rather than [rimDark], which would be a black line on black) —+ // but it has to be a *line*.+ //+ // At gain 1.45 / pow 2.2 it was neither. The rim is additive in the mood+ // colour, and pow 2.2 is a broad falloff, so on an active mood the halo+ // reached most of the way across the ball: measured on the tablet the edge+ // hit (150,244,255) while the body sat at (19,25,33), and the thumbnail+ // read as a glowing cyan orb with a dark patch on it. The body was never+ // the problem — the light on top of it was. Pow 3.6 pulls the falloff back+ // to the silhouette and the lower gain keeps it from blooming there.+ rimDark = false,+ rimGain = 0.85f,+ rimPow = 3.6f,+ labelRes = R.string.ball_look_imp,++ // A vertical slit. The single cheapest change of species on the whole face.+ pupilAspect = 0.42f,++ // Base sunk ~0.10 below the body's surface at this latitude, so there is no+ // seam ring where the cone meets the sphere; the length is 35% of the body+ // radius, which puts the tip a quarter of a radius clear of the silhouette.+ // Bone rather than black: a near-black horn on a near-black body is a horn+ // nobody can see.+ hornH = 0.30f,+ hornR = 0.085f,+ hornX = 0.34f,+ hornY = 0.72f,+ hornZ = 0.24f,+ hornTilt = 20f,+ hornBend = 28f,+ hornColor = floatArrayOf(0.960f, 0.945f, 0.910f),++ // Leaves the body inside the silhouette (r = 0.79 against a 0.868 body), so+ // the first blob is buried and the chain appears to grow out of the ball.+ tailLen = 0.55f,+ tailX = 0.60f,+ tailY = -0.52f,+ tailR = 0.055f,+ barbH = 0.17f,+ barbR = 0.055f,++ // Nine, so a tooth lands on the centre line rather than a gap.+ //+ // [toothR] is sized *against the spacing*, not picked for looks: the teeth+ // are laid out evenly across the width of the grin (see+ // `EmotionBallRenderer.drawTeeth`), which at this [mouthR] with nine of them+ // puts their centres 0.080 apart. A half-width of 0.042 therefore means+ // consecutive teeth just touch, which is what makes the row read as one+ // zigzag maw instead of nine separate spikes. Change [mouthR] or+ // [toothCount] and this number has to move with them.+ toothCount = 9,+ toothH = 0.062f,+ toothR = 0.042f,+ toothColor = floatArrayOf(0.985f, 0.980f, 0.965f),++ // No tongue. The field stays (it is zero-is-off, and `drawTongue` is still+ // the right way to hang one off any look that wants it) but this face does+ // not use it: a saturated pink disc is the highest-contrast thing on an ink+ // ball, so it won the whole face and the teeth — which are what makes the+ // grin a grin — became the background to it. The cute half is carried by+ // the cheeks and the eyes instead, both of which sit *under* the mouth in+ // the reading order rather than on top of it.+ tongueR = 0f,++ // The glare, and it is the eye's own shape rather than a mark laid over it.+ //+ // This used to be a brow: browW 0.175 / browH 0.062 / browY 0.112 /+ // browTilt 24, drawn as a mid-tone wedge across the top inner corner. It+ // worked, after the shade was fixed, but it was always a *second* shape+ // fighting the eye for the same pixels — and it only worked in the middle+ // of a range: light enough to lift off the ink body, dark enough to cut+ // into the white eye, and both ends move with the mood. Reported, then+ // measured, then reported again.+ //+ // Cutting the eye instead removes the argument. There is no colour to pick:+ // the flat edge has the eye's white under it and the ink body beside it, so+ // it is the highest-contrast line the screen can draw, in every mood, by+ // construction. What is lost is the *bar* — a brow mark reads as having+ // eyebrows, a cut reads as the eye being narrowed — and the exchange is+ // worth it: the reference is a slanted almond of white, not a face with+ // angry eyebrows on it.+ //+ // The numbers:+ //+ // * eyeCut 0.48 puts the flat edge at y = +0.0913 of a 0.1755 half-height,+ // which is 0.0257 of white above the pupil's top (0.8 × 0.082 = 0.0656 —+ // see the ceiling worked out on [eyeCut]). The eye loses 24% of its+ // height and becomes 0.296 wide against 0.267 tall, where it was 0.296+ // against 0.351: a tall oval turns into a wide one, which is the+ // silhouette change half of the read. The flat edge itself is 0.267 —+ // 90% of the eye's width, so it reads as one long line, not a clipped+ // corner.+ // * eyeTilt 12 is the other half. Rolled, the flat edge drops 0.267 ×+ // sin 12° ≈ 0.056 across its length — about a fifth of the eye's height+ // from one end to the other, which is the difference between a sleepy+ // eye and one that is looking at you sideways. It is half the bar's 24°+ // on purpose: this rotates the whole eye and the pupil with it.+ //+ // [browW] and friends are simply absent, which is the zero-is-off default+ // doing its job — see [browShade] for why the field itself stays.+ eyeCut = 0.48f,+ eyeTilt = 12f,++ // A man's voice, and the only look that asks for one. The fallback+ // (`zh_female_vv_uranus_bigtts`) is bright and young, which is the whole+ // house manner and exactly wrong coming out of a mouth full of teeth — the+ // face and the voice disagreeing is more unsettling than either on its own,+ // and not in the way this look is going for. Same `*_uranus_bigtts` family+ // as the fallback, so it needs no change to `tts_resource_id`.+ voice = "zh_male_lubanqihao_uranus_bigtts",+ persona = IMP_PERSONA,+)++/**+ * The looks the user can reach, and which one is being drawn.+ *+ * This is a mutable global for the same reason [EmotionState]'s fields are:+ * the ball's renderer is the only reader, it reads on its own GL thread, and+ * the writer is a finger on the UI thread. Anything heavier would be a+ * dependency graph for one integer.+ *+ * The switch is **latched**, not read live. [active] must not change in the+ * middle of a frame — the body, the rim, the face and the accessories would+ * each be drawn from a different look and the result is a ball that is briefly+ * two characters at once. So [request] only sets [requested], and the renderer+ * calls [latch] once at the top of `onDrawFrame`; everything downstream of that+ * sees one look for the whole frame.+ *+ * [latch] returning true is also the signal that look-sized state has to be+ * rebuilt — [EmotionState]'s resting feature positions are derived from the+ * look at construction, and nothing else would tell it to recompute them.+ */+object BallLooks {+ /**+ * Swipe order, and the first entry is what a device with no saved pick+ * gets. [CREAM] stays first because it is what the window was designed+ * around; [IMP] is last because it is the one a user should arrive at on+ * purpose.+ */+ val ALL = listOf(CREAM, BAJIE, IMP)++ @Volatile+ private var requested = 0++ /** GL thread only. Read [active] instead of this. */+ var index = 0+ private set++ val active: BallLook get() = ALL[index]++ /** What the last [request] asked for, for the UI's own label and bookkeeping. */+ val requestedLook: BallLook get() = ALL[requested]++ /** Wraps, so the caller can just add ±1 and not think about the ends. */+ fun request(i: Int) {+ requested = ((i % ALL.size) + ALL.size) % ALL.size+ }++ /**+ * Move [delta] places along [ALL] and return where it landed.+ *+ * Counted from the last *request*, not from [index]: two quick swipes+ * inside one frame would otherwise both step off the same latched value+ * and the second would land where the first already was.+ */+ fun step(delta: Int): BallLook {+ request(requested + delta)+ return requestedLook+ }++ /**+ * Restore a saved pick. By [BallLook.name] rather than by index: the index+ * is a position in [ALL], and reordering that list must not silently hand+ * the user a different character.+ */+ fun requestByName(name: String?) {+ val i = ALL.indexOfFirst { it.name == name }+ if (i >= 0) request(i)+ }++ /**+ * The look a saved name refers to, for the things that need the *character*+ * rather than the drawing: the voice ([BallLook.voice], resolved in+ * `VoiceConfig`) and the manner ([BallLook.persona], appended by+ * `LocalBrain`).+ *+ * By name from the preference rather than off [active] or [requestedLook],+ * and that is the point rather than an accident. Both of those are the GL+ * side's state and only exist once the ball has been built — the brain and+ * the TTS controller can both run before that and from other threads, where+ * [active] would read as whatever index the object happened to be+ * constructed with. The preference is written the instant the user swipes+ * (`FloatingWindowUi.stepBallLook`), so it is never behind what is on+ * screen.+ *+ * Falls back to the first entry, which is the same rule [ALL] documents for+ * a device with no saved pick — and the same reason a deleted look is safe:+ * an unknown name is a plain [CREAM], not a crash and not a silent imp.+ */+ fun byName(name: String?): BallLook =+ ALL.firstOrNull { it.name == name } ?: ALL[0]++ /** @return true if the look changed, i.e. look-sized state is now stale. */+ fun latch(): Boolean {+ val r = requested+ if (r == index) return false+ index = r+ return true+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallRenderer.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallRenderer.ktnew file mode 100644index 0000000..6670f67--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallRenderer.kt@@ -0,0 +1,1948 @@+package net.kuafuai.andee.ui.ball++import android.opengl.GLES20+import android.opengl.GLSurfaceView+import android.opengl.Matrix+import net.kuafuai.andee.ui.ball.gl.Capsule+import net.kuafuai.andee.ui.ball.gl.Ear+import net.kuafuai.andee.ui.ball.gl.Eye+import net.kuafuai.andee.ui.ball.gl.Horn+import net.kuafuai.andee.ui.ball.gl.IcoSphere+import net.kuafuai.andee.ui.ball.gl.Mesh+import net.kuafuai.andee.ui.ball.gl.PointsBuffer+import net.kuafuai.andee.ui.ball.gl.Shader+import net.kuafuai.andee.ui.ball.gl.Torus+import net.kuafuai.andee.ui.ball.gl.UvSphere+import javax.microedition.khronos.egl.EGLConfig+import javax.microedition.khronos.opengles.GL10+import kotlin.math.PI+import kotlin.math.acos+import kotlin.math.atan2+import kotlin.math.cos+import kotlin.math.min+import kotlin.math.sin+import kotlin.math.sqrt+import kotlin.math.tan++/**+ * The drawn scene is four passes: body, face, blush, rim. Animation state lives+ * in [EmotionState]; this class composes matrices, issues draw calls, and+ * decides which [Mood] is actually showing (see [resolveMood]).+ *+ * Draw order and depth are the whole trick here:+ * 1. Body — opaque sphere, writes depth. The ball's fill, and what stops+ * the rim's far half and anything behind the ball from showing through.+ * 2. Face — opaque, writes depth, sits at z = [FACE_Z] so it is in front of+ * the body's surface everywhere the features actually are.+ * 3. Blush, 4. Rim — additive, depth-tested, no depth write.+ *+ * The body is drawn at [BODY] and the rim at [RIM], and the body must stay the+ * *smaller* of the two: the rim is depth-tested but doesn't write depth, so a+ * body at a larger scale culls it in full and the ball loses its outline while+ * the eyes keep floating. That is how the ring-occlusion silhouette that used+ * to sit here killed the rim.+ *+ * The body used to be a *flat* near-black fill, and that was the second half of+ * why the ball read as an appliance rather than a character: a flat disc with a+ * glow around it and two white eyes is a face on a hole. It is now shaded —+ * top-lit gradient, one specular dot, a lift at the silhouette edge — which is+ * what makes it read as a solid, glossy *thing* instead of an absence. The+ * specular is aimed at the forehead rather than the front: a highlight landing+ * on the eyes eats half their contrast at exactly the size this is drawn.+ *+ * The core and ring passes below are built but not drawn; each is one line in+ * [onDrawFrame] away from coming back.+ */+class EmotionBallRenderer : GLSurfaceView.Renderer {++ // Public knobs (safe to write from UI thread)+ @Volatile+ var mood: Mood = Mood.CALM++ @Volatile+ var listening: Boolean = false++ /**+ * A task is running — set for its whole duration, not per tool call.+ *+ * [toolLeft] cannot express this: it decays [TOOL_SECONDS] after the last+ * call, and the stretches where the brain is thinking rather than tapping+ * make no calls at all. Those are exactly the stretches that used to drop+ * the ball back to a resting face while it was still working.+ */+ @Volatile+ var working: Boolean = false++ /**+ * The ball is holding up a signboard (CardUi.ask / alert) and is waiting+ * for the user's choice.+ *+ * While true — and nothing is being said aloud — the face is HOLDING:+ * steady eyes, lowered brows, closed mouth, slow breath. Waiting for a+ * decision is deliberately not the surprised LISTENING face.+ * Outranks the work faces (the ball has *paused* the task to ask), but not+ * the voice pipeline or the outcome flash — those are still mid-sentence.+ */+ @Volatile+ var holding: Boolean = false++ /**+ * The window is sitting at the ledge geometry — the perch slide (if any)+ * has finished.+ *+ * Separated from [working] because they start at different moments: a+ * task begins while the window is still travelling to the ledge, and the+ * cling (half-sunk body, paws, scissor) drawn against a window that is+ * *not yet at the edge* reads as the ball being chopped in half+ * mid-screen, then teleporting down — exactly the glitch the "帮我看看"+ * flow showed. The window owns this flag: it goes true when the perch+ * geometry is applied, false when the ball leaves the ledge or the+ * window is resized away from it.+ */+ @Volatile+ var atLedge: Boolean = false++ /**+ * Written on the GL thread, read by the UI thread: the ball has dozed off.+ *+ * Exposed so [FloatingWindowUi] can keep the idle stroll from sleepwalking.+ * The doze is decided here, from [idleFor], and there is no way for the UI+ * side to derive it — it knows what it last asked for, not how long ago.+ */+ @Volatile+ var dozing: Boolean = false++ /** When true, cycles through all moods every ~5.5 s. Handy for testing. */+ @Volatile+ var demoCycle: Boolean = false++ // Requests posted from the UI thread and consumed once, at the top of the+ // next frame. A plain @Volatile rather than a queue: both are "the latest+ // one wins" signals, and the GL thread is the only reader.+ @Volatile+ private var toolRequest = false++ @Volatile+ private var outcomeRequest: Mood? = null++ @Volatile+ private var actionRequest: Action? = null++ /** A `screen.*` command just ran — see [pulseTool]. */+ private var toolLeft = 0f++ /** Transient SUCCESS / FAILED face, outranking everything while it lasts. */+ private var overlayMood: Mood? = null+ private var overlayLeft = 0f++ /** Seconds spent with nothing to do at all, which is what puts it to sleep. */+ private var idleFor = 0f+ private var lastEffective = Mood.CALM++ // ---- Idle fidgets ----+ /** Countdown to the next fidget; only ticks while calm, awake, unheld. */+ private var fidgetIn = FIDGET_AFTER_SECONDS+ /** Last fidget played, so the picker never repeats twice in a row. */+ private var lastFidget: Action? = null+ private val fidgetRandom = java.util.Random()++ /**+ * The agent ran a tool against the device. Shows as SEARCHING plus a nod,+ * and decays on its own after [TOOL_SECONDS] — so a burst of tool calls+ * reads as continuous work, and the last one lets go without anyone having+ * to remember to clear it.+ *+ * Deliberately not a boolean "working" flag mirroring the task lifecycle:+ * a task that is only thinking already has a face (THINKING), and a flag+ * held for the whole task would hide it.+ */+ fun pulseTool() {+ toolRequest = true+ }++ /** A task finished. Two seconds of face, and a pop or a flinch. */+ fun signalOutcome(ok: Boolean) {+ outcomeRequest = if (ok) Mood.SUCCESS else Mood.FAILED+ }++ /** Play a one-shot gesture on the next frame. Safe from any thread. */+ fun request(action: Action) {+ actionRequest = action+ }++ // ---- Shaders ----+ private lateinit var shellShader: Shader+ private lateinit var rimShader: Shader+ private lateinit var coreShader: Shader+ private lateinit var flowShader: Shader+ private lateinit var pointsShader: Shader+ private lateinit var solidShader: Shader+ private lateinit var bubbleShader: Shader++ // ---- Meshes ----+ private lateinit var shell: Mesh+ private lateinit var rimSphere: Mesh+ private lateinit var core: Mesh+ private lateinit var ringMeshes: Array<Mesh>+ private lateinit var eyeArcMesh: Mesh++ /**+ * The look being drawn. Latched once per frame by [onDrawFrame]; see+ * [BallLooks] for why it must not be read live mid-frame.+ *+ * Shadowing the old top-level `LOOK` constant as a getter is what kept this+ * change to a handful of lines: the ~50 `LOOK.x` sites below did not have+ * to move, and none of them can accidentally cache a stale look.+ */+ private val LOOK: BallLook get() = BallLooks.active++ /**+ * The meshes whose *dimensions* come from the look, one set per look.+ *+ * Built for every look at surface creation rather than rebuilt on each+ * swipe, because [Mesh] allocates GL buffers and has no way to free them —+ * rebuilding on switch would leak a VBO pair per swipe, forever. Three+ * looks' worth of eye capsules and mouth tori is a few kilobytes, so the+ * trade is not close.+ *+ * The accessory meshes are null when the look has none, which is what+ * `hornH == 0f` etc. mean, and is also the test the draw calls use.+ */+ private class LookMeshes(look: BallLook) {+ /**+ * The eye, or the eye with its top sliced off flat when the look wants+ * a glare — see [BallLook.eyeCut]. Same two numbers either way: the cut+ * is a *shape*, not a size, so nothing else about the eye moves.+ *+ * Built here with everything else rather than swapped at draw time for+ * the same reason the horns are: [Mesh] allocates GL buffers and has no+ * way to free them, so a mesh built on a parameter change leaks a VBO+ * pair every frame the slider moves.+ */+ val eye: Mesh =+ if (look.eyeCut > 0f)+ Eye.build(look.eyeRadius, look.eyeCyl, look.eyeCut, 4, 12)+ else Capsule.build(look.eyeRadius, look.eyeCyl, 4, 12)+ val mouth: Mesh = Torus.build(look.mouthR, look.mouthTube, 8, 24, PI.toFloat())+ val horn: Mesh? =+ if (look.hornH > 0f) Horn.build(look.hornR, look.hornH, look.hornBend / DEG, 7, 12)+ else null+ val barb: Mesh? =+ if (look.barbH > 0f) Horn.build(look.barbR, look.barbH, 0f, 4, 10) else null+ val tooth: Mesh? =+ if (look.toothH > 0f) Horn.build(look.toothR, look.toothH, 0f, 3, 8) else null+ // A unit sphere scaled into the muzzle's ellipsoid at draw time — higher+ // subdivision than [handMesh], because the snout is a prominent facial+ // feature rather than a paw blob.+ val snout: Mesh? =+ if (look.snoutRx > 0f) IcoSphere.build(3) else null+ val ear: Mesh? =+ if (look.earH > 0f)+ Ear.build(look.earW, look.earH, look.earThick, look.earBend / DEG, 6, 10)+ else null+ }++ private lateinit var lookMeshes: List<LookMeshes>++ /** The current look's meshes. Same latching rules as [LOOK]. */+ private val meshes: LookMeshes get() = lookMeshes[BallLooks.index]++ /**+ * Z the pupil and glint are drawn at, relative to the face plane.+ *+ * Not cosmetic: the eye is a *capsule*, so it has a z-radius of+ * [BallLook.eyeRadius] and its front surface sits that far in front of the+ * face plane. A pupil left on the plane is behind the eye it belongs to and+ * is depth-culled away entirely.+ *+ * An instance getter rather than a `companion object` constant, which is+ * what it used to be: a companion value derived from the look is computed+ * once at class load, so it would freeze at whichever look happened to be+ * active then and bury the pupils of every look with a larger eye.+ */+ private val eyeFrontZ: Float get() = LOOK.eyeRadius + 0.025f++ /** One blob, instanced eight times into two paws. See [drawHands]. */+ private lateinit var handMesh: Mesh++ /** One blob per pupil and per glint — see [drawDot]. */+ private lateinit var dotMesh: Mesh++ /** 鼻涕泡用的球体。subdiv 4:水滴靠轮廓圈出形状,边不能是折线。 */+ private lateinit var bubbleSphere: Mesh++ // ---- Dust ----+ private lateinit var heads: PointsBuffer+ private lateinit var tails: PointsBuffer++ /** Two point sprites, the cheeks. See [drawBlush]. */+ private lateinit var blush: PointsBuffer++ // ---- Matrices (scratch, reused every frame) ----+ private val proj = FloatArray(16)+ private val view = FloatArray(16)+ private val root = FloatArray(16)+ private val face = FloatArray(16)+ private val model = FloatArray(16)+ private val local = FloatArray(16)+ private val mv = FloatArray(16)+ private val mvp = FloatArray(16)++ // ---- State ----+ private val state = EmotionState()++ /** Scratch for [drawBody]'s mood tint — per-frame, so never allocated. */+ private val bodyColor = FloatArray(3)++ /** Scratch for the pupil colour, which is a dark relative of the mood. */+ private val pupilColor = FloatArray(3)++ /** Scratch for the mouth colour, which is the mood coloured down on some looks. */+ private val mouthColor = FloatArray(3)++ /** Scratch for the dark outline colour — see [drawRim]. */+ private val outlineColor = FloatArray(3)++ /** `x, y, nx, ny` from [mouthPointAt]. Scratch, rewritten per tooth. */+ private val mouthPt = FloatArray(4)++ /** Filled per frame from [bodyColor] — see [BallLook.browShade]. */+ private val browColor = FloatArray(3)++ /** Filled per frame from [bodyColor] — see [BallLook.earShade]. */+ private val earColor = FloatArray(3)++ /** Scratch for the nostril dots — a dark relative of [bodyColor]. */+ private val nostrilColor = FloatArray(3)++ // Demo cycling+ private var demoIndex = 0+ private var demoNext = 0f+ // The voice-pipeline moods are promises, not poses: LISTENING says "your+ // mic is open", SPEAKING says "sound is coming out". A silent demo cycle+ // that wears them lies to the user twice over. Cycled only when someone+ // explicitly drives the renderer with them.+ private val moods = arrayOf(+ Mood.CALM, Mood.HAPPY, Mood.CURIOUS, Mood.TENSE,+ Mood.ANXIOUS, Mood.CONCERNED, Mood.SEARCHING, Mood.SLEEPING,+ )++ // Point-size projection factor: viewport-height / (2 tan(fov/2))+ private var pointFactor = 400f++ // Viewport, needed by the ledge scissor.+ private var vw = 0+ private var vh = 0++ // Time+ private val start = System.nanoTime()+ private var last = start++ override fun onSurfaceCreated(gl: GL10?, config: EGLConfig?) {+ GLES20.glClearColor(0f, 0f, 0f, 0f)+ GLES20.glEnable(GLES20.GL_DEPTH_TEST)+ GLES20.glEnable(GLES20.GL_BLEND)++ shellShader = Shader(SHELL_VS, SHELL_FS)+ rimShader = Shader(RIM_VS, RIM_FS)+ coreShader = Shader(CORE_VS, CORE_FS)+ flowShader = Shader(FLOW_VS, FLOW_FS)+ pointsShader = Shader(POINTS_VS, POINTS_FS)+ solidShader = Shader(SOLID_VS, SOLID_FS)+ bubbleShader = Shader(BUBBLE_VS, BUBBLE_FS)++ shell = IcoSphere.build(5)+ rimSphere = UvSphere.build(48, 32)+ core = IcoSphere.build(3)+ ringMeshes = Array(4) { i ->+ val r = RINGS[i]+ Torus.build(r.r, r.tube, 4, 160)+ }+ // The happy `^` eye. A partial torus spans [0, arc] starting at +X, so+ // its midpoint sits at arc/2 and it has to be rotated by (π/2 − arc/2)+ // to crown the eye instead of leaning off to one side — that rotation+ // is EYE_ARC_ROT_Z. Its size does not come from the look, so unlike the+ // round eye and the mouth there is only ever one of it.+ eyeArcMesh = Torus.build(0.1f, 0.03f, 6, 20, EYE_ARC_SPAN)+ lookMeshes = BallLooks.ALL.map { LookMeshes(it) }+ // Low subdivision on purpose: a paw is ~18 dp across on screen, where+ // subdiv 2 is already smooth and subdiv 4 is 16x the triangles for+ // nothing. It gets drawn eight times a frame.+ handMesh = IcoSphere.build(2)+ // Pupils and glints. Same unit blob as a paw part, and the same order of+ // cost: four of them a frame, each about 4 dp wide on screen.+ dotMesh = IcoSphere.build(2)+ bubbleSphere = IcoSphere.build(4)++ heads = PointsBuffer(DUST_COUNT)+ tails = PointsBuffer(DUST_COUNT * TRAIL_COUNT)+ blush = PointsBuffer(2)++ // Camera pushed back from the R3F reference's 3.7 to shrink the ball+ // to ~75%. Everything (rings, dust, face) scales proportionally.+ Matrix.setLookAtM(view, 0, 0f, 0f, CAM_Z, 0f, 0f, 0f, 0f, 1f, 0f)+ }++ override fun onSurfaceChanged(gl: GL10?, width: Int, height: Int) {+ GLES20.glViewport(0, 0, width, height)+ vw = width+ vh = height+ val aspect = width.toFloat() / height.toFloat()+ Matrix.perspectiveM(proj, 0, 42f, aspect, 0.1f, 100f)+ val fovRad = 42f * PI.toFloat() / 180f+ pointFactor = height / (2f * tan(fovRad / 2f))+ }++ override fun onDrawFrame(gl: GL10?) {+ val nowNs = System.nanoTime()+ val dt = min(0.1f, (nowNs - last) / 1e9f)+ last = nowNs+ val t = (nowNs - start) / 1e9f++ // Take the user's swipe *once*, before anything reads [LOOK]. Latching+ // here rather than letting the getter follow it live is what keeps a+ // frame from being drawn half in one character and half in another —+ // see [BallLooks].+ BallLooks.latch()++ if (demoCycle && t >= demoNext) {+ demoIndex = (demoIndex + 1) % moods.size+ mood = moods[demoIndex]+ demoNext = t + 5.5f+ }+ // Cling (half-sunk body, paws, ledge scissor) only once the window is+ // actually at the ledge — see [atLedge]. Everything else about working+ // (the face, the rim) is fine to start immediately; the ball looks+ // eager, not broken.+ state.tick(dt, t, resolveMood(dt), listening, working && atLedge)++ GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT or GLES20.GL_DEPTH_BUFFER_BIT)+ clipToLedge()++ buildRoot()+ buildFaceMatrix()++ drawBody()+ drawHorns()+ drawEars()+ drawTail()+ drawSnout()+ drawFace()+ drawHands()+ drawBlush()+ drawRim()+ // drawRings() — the orbiting halo read as clutter around a ball this+ // small, and the depth silhouette it needed to stop it cutting through+ // the eyes was culling the rim as well. Meshes and shader stay built.+ // drawDust() — the orbiting specks read as smudges at the size the ball+ // is actually drawn on the card. Buffers and shader stay built so this+ // is one line to put back.++ GLES20.glDisable(GLES20.GL_SCISSOR_TEST)+ GLES20.glDepthMask(true)+ }++ /**+ * Cut the frame off at the ledge while clinging, rather than letting the+ * screen edge do it.+ *+ * The perched window hangs half off the bottom of the display, so the+ * display used to provide that cut for free. It does not on a phone: the+ * navigation bar owns the bottom strip, is layered above us, and may be+ * transparent — in which case the half of the ball that is supposed to be+ * hidden is plainly visible over it and the paws are gripping the middle of+ * their own face. Owning the cut makes the perch look the same everywhere,+ * whatever is or isn't painted below the line.+ *+ * Ramped by [EmotionState.cling] so the edge rises up the ball as it sinks,+ * in step with the window sliding down, instead of lopping it off in one+ * frame. Scissor Y counts from the bottom of the viewport.+ */+ private fun clipToLedge() {+ val c = state.cling+ if (c < 0.01f) {+ GLES20.glDisable(GLES20.GL_SCISSOR_TEST)+ return+ }+ val line = vh * 0.5f + HAND_LEDGE * (pointFactor / CAM_Z)+ val cut = (line * c).toInt().coerceIn(0, vh)+ GLES20.glEnable(GLES20.GL_SCISSOR_TEST)+ GLES20.glScissor(0, cut, vw, vh - cut)+ }++ /**+ * Fold the three sources of "what should the face be" into one mood.+ *+ * Precedence, highest first: a finished task's SUCCESS / FAILED flash, then+ * whatever the voice pipeline says (it is mid-conversation with the user and+ * outranks housekeeping), then tool activity, then sleep.+ *+ * Tool activity only displaces CALM and THINKING because those two are the+ * "nothing is being said to you" faces. SEARCHING is strictly more+ * informative than either while the device is actually being driven, and+ * [working] holds it there across the gaps between calls.+ *+ * GL thread only.+ */+ private fun resolveMood(dt: Float): Mood {+ actionRequest?.let {+ actionRequest = null+ state.trigger(it)+ }++ if (toolRequest) {+ toolRequest = false+ toolLeft = TOOL_SECONDS+ state.trigger(Action.NOD)+ } else if (toolLeft > 0f) {+ toolLeft -= dt+ }++ outcomeRequest?.let {+ outcomeRequest = null+ overlayMood = it+ overlayLeft = OVERLAY_SECONDS+ toolLeft = 0f+ state.trigger(if (it == Mood.SUCCESS) Action.POP else Action.RECOIL)+ }+ if (overlayLeft > 0f) {+ overlayLeft -= dt+ if (overlayLeft <= 0f) overlayMood = null+ }++ val base = if ((working || toolLeft > 0f) && (mood == Mood.CALM || mood == Mood.THINKING)) {+ Mood.SEARCHING+ } else {+ mood+ }+ // Holding a signboard: attending to the user beats the work faces+ // (the task is *paused* on this question) but not the voice pipeline+ // or the outcome flash, which arrive through [mood] and [overlayMood]+ // above and take what they take.+ val holdingBase =+ if (holding && (base == Mood.CALM || base == Mood.SEARCHING)) Mood.HOLDING+ else base+ idleFor = if (holdingBase == Mood.CALM) idleFor + dt else 0f++ // Idle fidgets: only while genuinely idle — calm, awake, no signboard,+ // no overlay, nothing. The moment any real activity shows up the timer+ // resets, so a fidget can never land on top of (or be mistaken for+ // part of) a response. `idleFor` is exactly the right gate: it ticks+ // on the same condition that eventually puts the ball to sleep, so+ // the fidgets live in the window between "just finished" and "asleep"+ // (and stop entirely once asleep — a sleeping ball doesn't dance).+ if (idleFor > 0f && idleFor <= SLEEP_AFTER_SECONDS && actionRequest == null) {+ fidgetIn -= dt+ if (fidgetIn <= 0f) {+ // Weighted pick, never the same fidget twice in a row — a+ // repeat reads as a stuck animation, not a personality.+ var pick: Action+ do {+ var r = fidgetRandom.nextFloat() * IDLE_ACTIONS.sumOf { it.second.toDouble() }+ pick = IDLE_ACTIONS.first().first+ for ((a, w) in IDLE_ACTIONS) {+ r -= w+ if (r <= 0) { pick = a; break }+ }+ } while (pick == lastFidget)+ lastFidget = pick+ state.trigger(pick)+ fidgetIn = FIDGET_MIN_SECONDS ++ fidgetRandom.nextFloat() * (FIDGET_MAX_SECONDS - FIDGET_MIN_SECONDS)+ }+ } else if (idleFor == 0f) {+ // Real activity (or sleep, or a signboard) — back to the top of+ // the patience window, so the first fidget after a conversation+ // never comes before [FIDGET_AFTER_SECONDS] of quiet.+ fidgetIn = FIDGET_AFTER_SECONDS+ }++ val effective = overlayMood+ ?: if (idleFor > SLEEP_AFTER_SECONDS) Mood.SLEEPING else holdingBase+ // Waking up is worth a hop. Nothing else in here triggers an action on+ // a mood change — a mood is a resting state, and animating every+ // transition would make the ball twitch its way through a conversation.+ if (lastEffective == Mood.SLEEPING && effective != Mood.SLEEPING) {+ state.trigger(Action.HOP)+ }+ lastEffective = effective+ dozing = effective == Mood.SLEEPING+ return effective+ }++ // ---- Transform builders ----++ private fun buildRoot() {+ Matrix.setIdentityM(root, 0)+ Matrix.translateM(root, 0, 0f, state.hopY, 0f)+ Matrix.rotateM(root, 0, state.rotX * DEG, 1f, 0f, 0f)+ Matrix.rotateM(root, 0, state.rotY * DEG, 0f, 1f, 0f)+ Matrix.rotateM(root, 0, state.rotZ * DEG, 0f, 0f, 1f)+ // Z tracks X so a squashed ball stays a spheroid — bulging sideways as+ // it flattens — rather than turning into a disc seen face-on.+ Matrix.scaleM(root, 0, state.scaleX, state.scaleY, state.scaleX)+ }++ private fun buildFaceMatrix() {+ // face = root * translate(0, 0, LOOK.faceZ)+ System.arraycopy(root, 0, face, 0, 16)+ Matrix.translateM(face, 0, 0f, 0f, LOOK.faceZ)+ }++ private fun composeModelFromRoot(applyLocal: () -> Unit) {+ Matrix.setIdentityM(local, 0)+ applyLocal()+ Matrix.multiplyMM(model, 0, root, 0, local, 0)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)+ }++ // ---- Draws ----++ /**+ * The ball's fill: one opaque sphere, lit.+ *+ * It exists because the passes that used to give the ball a body are off,+ * and what was left — an additive rim glow plus white eyes — is invisible+ * over a light-coloured app. This used to be a dark View behind the whole+ * compact window instead, which was worse in two ways: it was bigger than+ * the sphere, so it read as a frame with a fat face inside it, and being a+ * View it could not squash or hop, so the ball visibly moved around inside+ * its own dead outline.+ *+ * [BallLook.bodyTint] is what decides how much of the mood colour the fill+ * takes on — high means most of it, low means a fifth. It was pinned low back when the body+ * was flat and the only thing keeping the character recognisable was "dark+ * ball, white eyes"; with the mood colour now running through albedo,+ * gradient and specular at once, a mood change is a change of *material*,+ * which is the point.+ */+ private fun drawBody() {+ System.arraycopy(root, 0, model, 0, 16)+ Matrix.scaleM(model, 0, LOOK.bodyScale, LOOK.bodyScale, LOOK.bodyScale)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)++ for (i in 0..2) {+ bodyColor[i] = (LOOK.bodyBase[i] ++ (state.nowColor[i] - LOOK.bodyBase[i]) * LOOK.bodyTint) * LOOK.bodyAlbedo+ }++ GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA)+ GLES20.glDepthMask(true)+ shellShader.use()+ GLES20.glUniformMatrix4fv(shellShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniformMatrix4fv(shellShader.uniform("uMV"), 1, false, mv, 0)+ GLES20.glUniform3fv(shellShader.uniform("uColor"), 1, bodyColor, 0)+ GLES20.glUniform1f(shellShader.uniform("uGrad"), LOOK.bodyGrad)+ GLES20.glUniform1f(shellShader.uniform("uAmb"), LOOK.bodyAmb)+ GLES20.glUniform1f(shellShader.uniform("uDiff"), LOOK.bodyDiff)+ GLES20.glUniform1f(shellShader.uniform("uGloss"), LOOK.bodyGloss)+ GLES20.glUniform1f(shellShader.uniform("uAlpha"), 1f)+ shell.bind(shellShader.attrib("aPos"), shellShader.attrib("aNormal"))+ shell.draw()+ }++ /**+ * Set up [shellShader] for a part that is made of the *body's* material —+ * same gradient, ambient, diffuse and gloss, only a different colour.+ *+ * Called with [bodyColor] for the tail (it is the ball's own flesh and has+ * to follow the mood with it) and with [BallLook.hornColor] for horn and+ * barb (keratin, which does not).+ */+ private fun useShellWith(color: FloatArray) {+ shellShader.use()+ GLES20.glUniform3fv(shellShader.uniform("uColor"), 1, color, 0)+ GLES20.glUniform1f(shellShader.uniform("uGrad"), LOOK.bodyGrad)+ GLES20.glUniform1f(shellShader.uniform("uAmb"), LOOK.bodyAmb)+ GLES20.glUniform1f(shellShader.uniform("uDiff"), LOOK.bodyDiff)+ GLES20.glUniform1f(shellShader.uniform("uGloss"), LOOK.bodyGloss)+ GLES20.glUniform1f(shellShader.uniform("uAlpha"), 1f)+ }++ private fun drawShellPart(mesh: Mesh) {+ GLES20.glUniformMatrix4fv(shellShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniformMatrix4fv(shellShader.uniform("uMV"), 1, false, mv, 0)+ mesh.bind(shellShader.attrib("aPos"), shellShader.attrib("aNormal"))+ mesh.draw()+ }++ /**+ * A pair of horns. Nothing at all if the look has no [BallLook.hornH].+ *+ * **Lit, and parented to [root].** Both matter. Lit, because a horn crosses+ * the body — a flat fill there is a hole punched in the ball rather than a+ * thing standing in front of it, which is why [Horn] bothers to emit real+ * normals. Parented to the root, because horns are part of the silhouette:+ * they have to lean with the head, breathe with the body, sink with the+ * cling and be cut by the same ledge scissor. A horn that held still while+ * the ball hopped would read as scenery the ball is standing behind.+ *+ * Drawn straight after the body and before the face, so it writes depth+ * into the same opaque pass — that is what lets the rim's fresnel be cut+ * correctly where a horn crosses it.+ *+ * The left one is the right one under a negative X scale, which flips+ * winding — harmless, because this ball never enables face culling — and+ * mirrors the normals correctly, because a reflection is its own+ * inverse-transpose.+ */+ private fun drawHorns() {+ val horn = meshes.horn ?: return+ useShellWith(LOOK.hornColor)+ for (side in -1..1 step 2) {+ composeModelFromRoot {+ if (side < 0) Matrix.scaleM(local, 0, -1f, 1f, 1f)+ Matrix.translateM(local, 0, LOOK.hornX, LOOK.hornY, LOOK.hornZ)+ // Negative Z rotation tips +Y toward +X, i.e. outward.+ Matrix.rotateM(local, 0, -LOOK.hornTilt, 0f, 0f, 1f)+ }+ drawShellPart(horn)+ }+ }++ /**+ * A pair of ears. Nothing if the look has no [BallLook.earH].+ *+ * Parented to [root] for the same reasons as the horns: they are part of the+ * silhouette and have to lean, breathe and cling with the body. Drawn with+ * the lit shell shader and made of the body's flesh, unlike a horn's+ * keratin — an ear is skin.+ *+ * **[BallLook.earTilt] is the whole read, and it is not one shape.** A pig's+ * ear hangs down and out (well past a right angle, with [BallLook.earBend]+ * curling the tip further); a *clever* ball wants the opposite — an upright+ * flap whose tip clears its own outline, which is what makes the silhouette+ * read as alert rather than as a sphere. Both come out of the same+ * [Ear.build]; the angle is the entire difference.+ *+ * **An upright ear is not a horn**, even though [drawHorns] leans its cones+ * outward by much the same number of degrees. Three things separate them and+ * all three are visible at thumbnail size: a horn is a *tapered point* where+ * an ear is a broad rounded flap (see `gl/Ear` — a stack of elliptical rings+ * closed by a quarter-ellipse tip, against `gl/Horn`'s taper to a point); a+ * horn is bone-pale while an ear is the body's own colour; and no look in+ * [BallLooks.ALL] carries both. Keep it that way — a look with upright ears+ * *and* horns is a silhouette nobody can parse, which is the failure this+ * note exists to prevent.+ *+ * The colour is [BallLook.earShade] times the live body colour rather than+ * the body colour itself, which matters most on exactly the pale looks that+ * want upright ears: an ear grows *past* the dark outline, so beyond the+ * silhouette there is no dark line holding it off the app behind the ball.+ */+ private fun drawEars() {+ val ear = meshes.ear ?: return+ for (i in 0..2) earColor[i] = bodyColor[i] * LOOK.earShade+ useShellWith(earColor)+ for (side in -1..1 step 2) {+ composeModelFromRoot {+ if (side < 0) Matrix.scaleM(local, 0, -1f, 1f, 1f)+ Matrix.translateM(local, 0, LOOK.earX, LOOK.earY, LOOK.earZ)+ Matrix.rotateM(local, 0, -LOOK.earTilt, 0f, 0f, 1f)+ }+ drawShellPart(ear)+ }+ }++ /**+ * A tapering chain of blobs ending in a barb. Nothing if [BallLook.tailLen]+ * is zero.+ *+ * A chain of spheres rather than a swept tube because the ball already owns+ * a unit blob ([handMesh]) and the tail is ~12 px wide on screen: at that+ * size the difference between a chain at this spacing and a real tube is+ * below one pixel, and a swept tube would be a whole mesh generator with a+ * per-look build.+ *+ * The walk is a constant-curvature arc — fixed step, fixed turn per step —+ * which is the shape a devil's tail is drawn as: out, down, and hooking+ * back up. The first blob is placed *inside* the body silhouette on+ * purpose, so the tail appears to grow out of the ball rather than to be+ * docked onto it.+ *+ * Parented to [root] for the same reasons as the horns.+ */+ private fun drawTail() {+ if (LOOK.tailLen <= 0f) return+ useShellWith(bodyColor)++ val step = LOOK.tailLen / TAIL_BLOBS+ val turn = TAIL_SWEEP / TAIL_BLOBS+ var x = LOOK.tailX+ var y = LOOK.tailY+ var ang = TAIL_START+ for (i in 0 until TAIL_BLOBS) {+ val r = LOOK.tailR * (1f - TAIL_TAPER * i / (TAIL_BLOBS - 1f))+ composeModelFromRoot {+ Matrix.translateM(local, 0, x, y, 0f)+ Matrix.scaleM(local, 0, r, r, r)+ }+ drawShellPart(handMesh)+ x += cos(ang) * step+ y += sin(ang) * step+ ang += turn+ }++ val barb = meshes.barb ?: return+ useShellWith(LOOK.hornColor)+ composeModelFromRoot {+ Matrix.translateM(local, 0, x, y, 0f)+ // The mesh grows +Y; ang is measured from +X, hence the quarter turn.+ Matrix.rotateM(local, 0, ang * DEG - 90f, 0f, 0f, 1f)+ }+ drawShellPart(barb)+ }++ /**+ * A rounded muzzle protruding from the face. Nothing if the look has no+ * [BallLook.snoutRx].+ *+ * Drawn lit with the body's own material, so it reads as a bump rather than+ * a sticker: the vertical gradient and the specular land on it differently+ * than on the body sphere. It is parented to the *face*, not the root,+ * because the snout is a facial feature and must ride the same plane the+ * eyes and mouth do.+ *+ * The unit sphere is scaled into an ellipsoid wider than it is tall and+ * protruding by [BallLook.snoutRz]; its centre sits on the face plane, so+ * the back half is buried in the body and the front half sticks out.+ *+ * Drawn after the body and before [drawFace], so it writes depth into the+ * opaque pass and the nostrils — drawn flat on top of it inside [drawFace] —+ * land over the bump rather than behind it.+ */+ private fun drawSnout() {+ val snout = meshes.snout ?: return+ useShellWith(bodyColor)+ Matrix.setIdentityM(local, 0)+ Matrix.translateM(local, 0, LOOK.snoutX, LOOK.snoutY, 0f)+ Matrix.scaleM(local, 0, LOOK.snoutRx, LOOK.snoutRy, LOOK.snoutRz)+ Matrix.multiplyMM(model, 0, face, 0, local, 0)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)+ drawShellPart(snout)+ }++ private fun drawFace() {+ GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA)+ GLES20.glDepthMask(true)+ solidShader.use()+ val aPos = solidShader.attrib("aPos")+ // No default colour is set here, on purpose.+ //+ // uColor has no attribute, so it is *shared state that survives across+ // draw calls*. A default at the top of this pass looks harmless and is+ // the opposite: it silently supplies a colour to any element that+ // forgets its own, and it hides that omission from+ // `tools/check_look_sync.py`, which requires each draw to set its own+ // colour within its own branch. The `^` eye was exactly this bug —+ // it inherited the default white and drew white arcs whenever the round+ // eye didn't run. Every element below sets a colour before it draws.+ GLES20.glUniform1f(solidShader.uniform("uAlpha"), 1f)++ // Round eye and `^` eye cross-fade by *size*, not by alpha. The face is+ // drawn first, onto a cleared transparent buffer, so a half-faded white+ // shape is a visible ghost — whereas one scaled to zero simply isn't+ // there. The in-between (a flattening dot under a growing arch) also+ // happens to be what a squint looks like.+ val round = state.eyeYScale * (1f - state.eyeArc)+ val arc = state.eyeArc++ if (round > 0.01f) {+ GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, LOOK.eyeColor, 0)+ setLocalFromFace(state.eyeLX, state.eyeLY, state.eyeLRotZ, round, pivotY = EYE_LID_PIVOT)+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)+ meshes.eye.bind(aPos)+ meshes.eye.draw()++ setLocalFromFace(state.eyeRX, state.eyeRY, state.eyeRRotZ, round, pivotY = EYE_LID_PIVOT)+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)+ meshes.eye.bind(aPos)+ meshes.eye.draw()+ }++ // The pupil and its catchlight, which are what turn a white slot into+ // something that is looking at you. Gated on the same aperture the round+ // eye is, plus a floor: under a `^ ^` squint the eye is a line, and a+ // pupil left hanging beneath it would be the one part of the face that+ // doesn't blink.+ //+ // Which of the two a look wants is [BallLook.pupilR] / [BallLook.glintR]:+ // a dark ball with white eyes needs a pupil drawn inside them; a pale+ // ball has nothing for a white eye to contrast against, so its eye *is*+ // the dark shape and only the glint goes on top.+ if (round > PUPIL_MIN_APERTURE && arc < ARC_EYE_AT &&+ (LOOK.pupilR > 0f || LOOK.glintR > 0f)+ ) {+ if (LOOK.pupilR > 0f) {+ for (i in 0..2) {+ pupilColor[i] = state.nowColor[i] * PUPIL_TINT + PUPIL_LIFT[i]+ }+ }+ drawEyeContents(state.eyeLX, state.eyeLY, state.eyeLRotZ, round)+ drawEyeContents(state.eyeRX, state.eyeRY, state.eyeRRotZ, round)+ }++ if (arc > 0.01f) {+ // The `^` eye carries its own colour, and this line is load-bearing.+ //+ // It used to inherit whatever the previous element had left in+ // uColor, which is the pass default (white) on any frame where the+ // round eye didn't draw — and on SLEEPING it never draws: its+ // aperture works out to 0.06 × 0.15 = 0.009, just under the 0.01+ // threshold above. The result was two *white* arcs on a pale ball,+ // while the lab showed dark ones, because the lab sets this colour+ // per element. A uniform with no attribute is shared state; the+ // only safe rule in this pass is that every draw sets its own.+ GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, LOOK.eyeColor, 0)+ val h = arc * state.lid+ setLocalFromFace(+ state.eyeLX, state.eyeLY - EYE_ARC_DROP,+ EYE_ARC_ROT_Z + state.eyeLRotZ, h, arc,+ )+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)+ eyeArcMesh.bind(aPos)+ eyeArcMesh.draw()++ setLocalFromFace(+ state.eyeRX, state.eyeRY - EYE_ARC_DROP,+ EYE_ARC_ROT_Z + state.eyeRRotZ, h, arc,+ )+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)+ eyeArcMesh.bind(aPos)+ eyeArcMesh.draw()+ }++ // Mouth — the mood colour, or a deep shade of it on a pale ball, where+ // the full-strength colour is too close in value to the body to read.+ for (i in 0..2) {+ mouthColor[i] = if (LOOK.mouthDeep) state.nowColor[i] * MOUTH_DEEP else state.nowColor[i]+ }+ setLocalFromFace(0f, state.mouthY, state.mouthRotZ, state.mouthScaleY)+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, mouthColor, 0)+ GLES20.glUniform1f(solidShader.uniform("uAlpha"), 1f)+ meshes.mouth.bind(aPos)+ meshes.mouth.draw()++ drawTongue()+ drawTeeth(aPos)+ drawNostrils()+ drawBrows(aPos)+ drawBubble()+ }++ /**+ * Where a point at parameter [u] on the mouth's own circle ends up once the+ * mouth's live transform has been applied, and which way is *out* there.+ *+ * The mouth mesh is a half-torus of radius [BallLook.mouthR] built flat in+ * the XY plane, and the renderer never draws it as built: it squashes it in+ * Y by [EmotionState.mouthScaleY] and then turns it by+ * [EmotionState.mouthRotZ], which is what makes the arch a smile and what+ * makes the smile deepen. Anything that has to sit *on* that curve — every+ * tooth, the tongue — has to go through the same two steps by hand, because+ * it is placed with its own matrix rather than riding the mouth's.+ *+ * Note the corner mapping: the mesh is built as the *upper* half of the+ * mouth circle and the rest transform turns it by π, so u near 0 lands at+ * the **left** corner and u near π at the right — the opposite of what+ * reading the mesh alone would suggest.+ *+ * The outward direction is the **ellipse normal**, not the radial+ * direction, and at rest those are nowhere near each other: the mouth is+ * squashed to about a third of its height, where the radial direction at a+ * mid-arc tooth points out and sideways while the surface it is standing on+ * runs almost flat. Teeth placed on the radial would lie over like a row of+ * dominoes. For `x²/a² + y²/b² = 1` the normal at parameter u is+ * `(cos u / a, sin u / b)`, which with `a = R`, `b = R·sY` is+ * `(cos u · sY, sin u)` up to scale.+ *+ * Results land in [mouthPt] as `x, y, nx, ny` — one scratch array rather+ * than an allocation per tooth per frame.+ */+ private fun mouthPointAt(u: Float) {+ val sY = state.mouthScaleY+ val th = state.mouthRotZ+ val cT = cos(th)+ val sT = sin(th)+ val cU = cos(u)+ val sU = sin(u)++ val px = LOOK.mouthR * cU+ val py = LOOK.mouthR * sU * sY+ mouthPt[0] = px * cT - py * sT+ mouthPt[1] = state.mouthY + px * sT + py * cT++ val nx = cU * sY+ val ny = sU+ val inv = 1f / sqrt(nx * nx + ny * ny)+ mouthPt[2] = (nx * cT - ny * sT) * inv+ mouthPt[3] = (nx * sT + ny * cT) * inv+ }++ /**+ * A zigzag of triangles along the grin. Nothing if the look has no+ * [BallLook.toothCount].+ *+ * Alternating outward and inward is the whole trick: consecutive teeth+ * interlock across the lip line, which is what the top and bottom rows of a+ * cartoon maw look like when the mouth is *shut*. Drawing two separate rows+ * would need the mouth to actually open, and this ball's mouth is a curve+ * rather than a cavity.+ *+ * Flat-shaded, unlike the horns, and that is not an oversight: a tooth is a+ * pale triangle a few pixels wide sitting on top of a mouth, so it is read+ * entirely as a silhouette and lighting it would only mute the one edge+ * that carries it. The horns have to be lit because they cross the body,+ * where a flat fill is a hole rather than a shape.+ *+ * The ends of the arc are inset by [TOOTH_PAD]: a tooth exactly on the+ * corner of the mouth points sideways, off the face, and reads as a chip+ * rather than a tooth.+ *+ * They are spaced evenly **across the grin**, not evenly in the mouth+ * circle's own parameter, and the two are nowhere near the same thing. The+ * mouth is squashed to about a third of its height, so equal steps in `u`+ * cover almost no width near the corners and a great deal of it in the+ * middle — which is what the first build drew: a clump of overlapping teeth+ * at each end of a sparse middle. Stepping `cos u` linearly instead makes+ * the *projected* spacing constant, and [BallLook.toothR] is then chosen so+ * neighbours just touch.+ */+ private fun drawTeeth(aPos: Int) {+ val tooth = meshes.tooth ?: return+ val n = LOOK.toothCount+ if (n < 2) return+ GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, LOOK.toothColor, 0)+ GLES20.glUniform1f(solidShader.uniform("uAlpha"), 1f)+ val cosPad = cos(TOOTH_PAD)+ val half = PI.toFloat() / 2f+ for (i in 0 until n) {+ val t = i / (n - 1f)+ mouthPointAt(acos(cosPad * (1f - 2f * t)))+ // The mesh grows +Y, so a tooth aimed along a direction `a` is that+ // direction turned back by a quarter turn. Even teeth take the+ // outward normal, odd ones its opposite.+ val a = atan2(mouthPt[3], mouthPt[2])+ setLocalFromFace(+ mouthPt[0], mouthPt[1],+ if (i % 2 == 0) a - half else a + half,+ scaleY = 1f, zOff = LOOK.mouthTube + 0.020f,+ )+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)+ tooth.bind(aPos)+ tooth.draw()+ }+ }++ /**+ * Two dark nostril dots on the front of the muzzle. Nothing if the look has+ * no [BallLook.nostrilR].+ *+ * Drawn flat in the face pass, after [drawSnout] has already laid down the+ * lit muzzle, so they sit *on* the bump rather than behind it. Placed at the+ * muzzle's front surface — pushed forward of the face plane by most of+ * [BallLook.snoutRz] — so they read as holes in the snout, not marks on the+ * face behind it.+ *+ * The colour is the live [bodyColor] darkened, not a fixed black: the body+ * takes the mood's tint, and a fixed near-black would be right in CALM and a+ * smudge in TENSE — the same reasoning as [BallLook.browShade].+ */+ private fun drawNostrils() {+ val r = LOOK.nostrilR+ if (r <= 0f) return+ for (i in 0..2) nostrilColor[i] = bodyColor[i] * NOSTRIL_DARK+ val z = LOOK.snoutRz * 0.82f+ for (side in -1..1 step 2) {+ drawDot(+ LOOK.snoutX + LOOK.nostrilX * side,+ LOOK.snoutY,+ 0f,+ lidScale = 1f,+ radius = r,+ z = z,+ color = nostrilColor,+ )+ }+ }++ /**+ * The tongue, hanging out of the grin. Nothing if the look has no+ * [BallLook.tongueR].+ *+ * Pushed out along the mouth's own normal by half its radius, so it reads+ * as coming from behind the lip rather than as a dot parked on it, and+ * drawn *before* the teeth at a slightly shallower Z so they close over it.+ * Taller than it is wide, because a round one is a nose.+ */+ private fun drawTongue() {+ val r = LOOK.tongueR+ if (r <= 0f) return+ mouthPointAt(PI.toFloat() / 2f + LOOK.tongueSkew)+ drawDot(+ mouthPt[0] + mouthPt[2] * r * 0.5f,+ mouthPt[1] + mouthPt[3] * r * 0.5f,+ atan2(mouthPt[3], mouthPt[2]) - PI.toFloat() / 2f,+ lidScale = 1f,+ radius = r,+ z = LOOK.mouthTube + 0.008f,+ color = LOOK.tongueColor,+ aspectX = 0.82f,+ )+ }++ /**+ * Two angled brows above the eyes. Nothing if the look has no+ * [BallLook.browW] — which is now every look that wants the *glare* read,+ * since [IMP] gets it from [BallLook.eyeCut] instead. See below; the two+ * are alternatives, not a sequence.+ *+ * A flattened unit sphere rather than a bar: a scaled sphere is a lens,+ * which tapers at both ends, which is what a brow looks like.+ *+ * With [BallLook.browShade] set it is not a brow at all but a **wedge of+ * body colour laid over the top inner corner of the eye** — the white that+ * survives underneath is a slanted almond, and a slanted eye is a glare+ * where a bar above a round eye is only an eyebrow. The colour is derived+ * from the live [bodyColor] rather than stored, because the body takes the+ * mood's tint and a fixed near-black would show as a smudge the moment the+ * mood moved. No look ships with it set; see [BallLook.browShade].+ *+ * **This draw is flat and the face behind it is lit, so the multiplier is+ * not what it looks like.** Measured against the ink body ([IMP]) with the+ * shell shader's own numbers — the body at the brow's latitude renders at+ * `0.82 × albedo`, the brow at `shade × albedo`:+ *+ * | shade | brow, in 8 bit | vs the body | vs the white eye |+ * |-------|----------------|-------------|------------------|+ * | 1.05 | (32, 33, 50) | 1.28× | 7.4× under |+ * | 1.50 | (46, 47, 71) | 1.83× | 5.2× under |+ * | 2.00 | (61, 63, 95) | 2.44× | 3.9× under |+ * | 2.50 | (77, 79, 118) | 3.05× | 3.1× under |+ * | 3.00 | (92, 95, 142) | 3.66× | 2.6× under |+ *+ * Only one row works for both jobs. The brow has to clear the body *and*+ * stay under the eye, and the eye is white, so the answer is the geometric+ * midpoint of the two — ≈0.32 linear on an ink body, i.e. 2.5. The first+ * number tried was 1.05, which is 1.28× the body: a ~7/255 difference,+ * which is why the brow was reported as invisible.+ *+ * The whole table is the reason this look no longer uses the field. A+ * *range* that has to hold under every mood, to serve a mark whose two ends+ * have different jobs, is a fragile way to get a glare — and it is fragile+ * in the worst way, silently. [BallLook.eyeCut] gets the same read out of+ * the eye's own geometry, where the flat edge is the highest-contrast line+ * there is by construction. This draw stays for [CREAM] and [BAJIE], which+ * want an actual eyebrow rather than a glare.+ *+ * The tilt is [BallLook.browTilt] (the look's resting scowl) plus the+ * mood's own [EmotionState.brow], so a look that starts angry can still get+ * angrier or be surprised out of it. It is mirrored per side because+ * "inner end down" is a different sign of rotation on the left and the+ * right — the one thing that, got wrong, turns a scowl into a worry.+ *+ * They ride on the live eye Y, not on the look's resting one, so a blink or+ * a glance carries them along instead of leaving them pinned to the+ * forehead.+ */+ private fun drawBrows(aPos: Int) {+ if (LOOK.browW <= 0f) return+ val color = if (LOOK.browShade > 0f) {+ for (i in 0..2) browColor[i] = bodyColor[i] * LOOK.browShade+ browColor+ } else {+ LOOK.eyeColor+ }+ GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, color, 0)+ GLES20.glUniform1f(solidShader.uniform("uAlpha"), 1f)+ val tilt = LOOK.browTilt / DEG + state.brow * BROW_MOOD_TILT+ for (side in -1..1 step 2) {+ val cx = if (side < 0) state.eyeLX else state.eyeRX+ val cy = (if (side < 0) state.eyeLY else state.eyeRY) + LOOK.browY+ setLocalFromFace(+ cx, cy, tilt * side,+ scaleY = LOOK.browH,+ scaleX = LOOK.browW,+ scaleZ = LOOK.browH * DISC_THICKNESS,+ zOff = eyeFrontZ,+ )+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)+ dotMesh.bind(aPos)+ dotMesh.draw()+ }+ }++ /**+ * 鼻涕泡:一颗**透明的**水滴,不是一颗白球。+ *+ * 第一版这里是两个实心圆片(浅色压在深色上),做出来是一坨白 —— 一个实心+ * 的浅色圆放在浅色球上,除了"一颗球"读不出任何别的东西。水滴之所以是水滴,+ * 全靠三件跟"实心"相反的事,都在 [BUBBLE_FS] 里:+ *+ * * **背面被 discard**。泡是一整颗球不是一片,前后两个面会盖到同一个像素+ * 上,不裁掉背面的话轮廓那一圈会叠成两倍厚。+ * * **中间是透的**。只留轮廓上一圈菲涅尔([BallLook.bubblePow])加+ * [BallLook.bubbleInner] 那点薄雾,球体自己从中间透出来。+ * * **一个高光点**,落在左上 —— 和球的主光同一个方向,同一个理由:+ * 高光落在别处会读成"它看向了别处"。+ *+ * 单独的着色器而不是复用 SOLID:SOLID 是 `vec4(uColor, uAlpha)`,给它一颗+ * 不透明的球只能得到那颗白球。+ *+ * 画在嘴之后:它贴着鼻子,先画嘴再压上泡,就不会出现"泡被嘴切掉一块"。+ */+ private fun drawBubble() {+ val a = state.bubble+ if (a < 0.01f) return+ val r = LOOK.bubbleR * state.bubbleScale * a++ System.arraycopy(face, 0, model, 0, 16)+ Matrix.translateM(model, 0, LOOK.bubbleX, LOOK.bubbleY, 0f)+ Matrix.scaleM(model, 0, r, r, r)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)++ // 透明,所以不写深度;但深度**测试**照旧开着 —— 正是它挡住球体的+ // 后半颗,不让它从身体里透出来。画完把 mask 恢复,因为上面那一段+ // 是按"面是不透明的"写的。+ GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA)+ GLES20.glDepthMask(false)+ bubbleShader.use()+ GLES20.glUniformMatrix4fv(bubbleShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniformMatrix4fv(bubbleShader.uniform("uMV"), 1, false, mv, 0)+ GLES20.glUniform3fv(bubbleShader.uniform("uColor"), 1, LOOK.bubbleColor, 0)+ GLES20.glUniform1f(bubbleShader.uniform("uAlpha"), a)+ GLES20.glUniform1f(bubbleShader.uniform("uPow"), LOOK.bubblePow)+ GLES20.glUniform1f(bubbleShader.uniform("uShine"), LOOK.bubbleShine)+ GLES20.glUniform1f(bubbleShader.uniform("uInner"), LOOK.bubbleInner)+ bubbleSphere.bind(bubbleShader.attrib("aPos"), bubbleShader.attrib("aNormal"))+ bubbleSphere.draw()+ GLES20.glDepthMask(true)+ }++ private fun setLocalFromFace(+ x: Float,+ y: Float,+ rotZ: Float,+ scaleY: Float,+ scaleX: Float = 1f,+ pivotY: Float = 0f,+ scaleZ: Float = 1f,+ zOff: Float = 0f,+ ) {+ Matrix.setIdentityM(local, 0)+ // Scaling about a pivot below the element instead of through its middle+ // is what makes a closing eye read as a lid coming down: the bottom edge+ // stays put and the top descends to meet it.+ Matrix.translateM(local, 0, x, y + pivotY * (1f - scaleY), zOff)+ Matrix.rotateM(local, 0, rotZ * DEG, 0f, 0f, 1f)+ Matrix.scaleM(local, 0, scaleX, scaleY, scaleZ)+ Matrix.multiplyMM(model, 0, face, 0, local, 0)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)+ }++ /**+ * One round element of the face layer: a pupil, or the glint inside it.+ *+ * Three things have to be right at once or the eye turns into a smudge:+ *+ * - `pivotY` is 0, *not* [EYE_LID_PIVOT]. The pivot compensation exists to+ * keep a closing eye's lower edge still, and it moves an element by+ * `pivot * (1 - scaleY)`. For the eye, scaleY hovers near 1 and that+ * term is nothing; for a pupil whose scaleY *is* its own radius, the same+ * expression is a 0.12-unit shove down the face, which lands the pupil on+ * the cheek.+ * - `scaleZ` is a thin fraction of the radius — these are discs. Left at 1+ * a unit sphere stays a 2-unit-tall, 2-unit-deep ball and swallows the+ * whole frame.+ * - `zOff` puts it in front of [eyeFrontZ], because the eye is a capsule+ * with its own z-radius and would otherwise occlude its own pupil.+ *+ * `lidScale` is the eye's current vertical aperture, so the pupil and its+ * glint blink along with the eye instead of floating over a closed lid.+ *+ * `aspectX` squashes it horizontally — 1 is round, and anything below turns+ * the pupil into a vertical slit. It is on the *width* only, so the slit+ * still closes with the lid like a round pupil does.+ */+ private fun drawDot(+ x: Float,+ y: Float,+ rotZ: Float,+ lidScale: Float,+ radius: Float,+ z: Float,+ color: FloatArray,+ aspectX: Float = 1f,+ ) {+ setLocalFromFace(+ x, y, rotZ,+ scaleY = radius * lidScale,+ scaleX = radius * aspectX,+ pivotY = 0f,+ scaleZ = radius * DISC_THICKNESS,+ zOff = z,+ )+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, color, 0)+ dotMesh.bind(solidShader.attrib("aPos"))+ dotMesh.draw()+ }++ /**+ * One eye's contents: optionally the pupil, then the glint.+ *+ * [pupilColor] is a per-frame scratch array shared by both eyes, and is only+ * filled when [BallLook.pupilR] is non-zero — see the call site.+ *+ * The glint sits up and inboard of whatever it is sitting on, which is where+ * a window is in a room: the light the ball is standing in comes from above+ * and slightly to its left, so that is the side it lands on. Getting this+ * wrong is not subtle — a glint on the wrong side reads as the ball having+ * looked away.+ */+ private fun drawEyeContents(cx: Float, cy: Float, rotZ: Float, lidScale: Float) {+ val pr = LOOK.pupilR+ if (pr > 0f) {+ drawDot(+ cx - pr * 0.12f, cy - pr * 0.20f, rotZ, lidScale,+ pr, eyeFrontZ, pupilColor, LOOK.pupilAspect,+ )+ }+ val gr = LOOK.glintR+ if (gr <= 0f) return+ // Anchored to the pupil when there is one, to the eye itself otherwise —+ // and the offset scales with whichever it is, because the two shapes are+ // very different sizes: a glint placed as a fixed fraction of the *eye*+ // lands outside a small pupil, and one placed as a fraction of a small+ // pupil barely moves off the centre of a bare eye.+ if (pr > 0f) {+ drawDot(+ cx - pr * 0.34f, cy + pr * 0.26f, rotZ, lidScale,+ gr, eyeFrontZ + GLINT_Z_STEP, WHITE,+ )+ } else {+ drawDot(+ cx - LOOK.eyeRadius * 0.36f, cy + LOOK.eyeRadius * 0.40f, rotZ, lidScale,+ gr, eyeFrontZ + GLINT_Z_STEP, WHITE,+ )+ }+ }++ /**+ * Two soft cheek blobs. Point sprites rather than small spheres because the+ * point shader already has a radial smoothstep falloff — a flat-shaded+ * sphere would be a pink disc with a hard edge, which is a sticker, not a+ * blush.+ */+ private fun drawBlush() {+ val a = state.blush+ if (a < 0.01f) return+ val d = blush.data+ for (n in 0..1) {+ val j = n * 6+ d[j] = if (n == 0) -LOOK.blushX else LOOK.blushX+ d[j + 1] = LOOK.blushY+ d[j + 2] = -0.02f+ d[j + 3] = LOOK.blushColor[0]+ d[j + 4] = LOOK.blushColor[1]+ d[j + 5] = LOOK.blushColor[2]+ }+ blush.upload()++ Matrix.multiplyMM(mv, 0, view, 0, face, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)++ // Additive on a dark body reads as a warm patch; on a pale one it clips+ // straight to white and the warmth vanishes, so pale looks blend+ // normally instead.+ GLES20.glBlendFunc(+ GLES20.GL_SRC_ALPHA,+ if (LOOK.blushBlendNormal) GLES20.GL_ONE_MINUS_SRC_ALPHA else GLES20.GL_ONE,+ )+ GLES20.glDepthMask(false)+ pointsShader.use()+ GLES20.glUniformMatrix4fv(pointsShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniformMatrix4fv(pointsShader.uniform("uMV"), 1, false, mv, 0)+ GLES20.glUniform1f(pointsShader.uniform("uPointFactor"), pointFactor)+ GLES20.glUniform1f(pointsShader.uniform("uSize"), LOOK.blushSize)+ GLES20.glUniform1f(pointsShader.uniform("uAlpha"), a * LOOK.blushAlpha)+ blush.bind(pointsShader.attrib("aPos"), pointsShader.attrib("aColor"))+ blush.draw()+ }++ /**+ * Two paws hooked over the bottom screen edge, drawn only while the ball is+ * hanging off it.+ *+ * Deliberately **not** parented to [root]: the paws hold still on the edge+ * while the body slips down and hauls back up underneath them. That gap is+ * the entire reason the slip reads as losing grip rather than as the ball+ * bobbing — a paw that sags along with the body is not holding anything.+ *+ * Each paw is a splayed palm plus three toes, which is more geometry than a+ * single blob needs to be and is the difference between "little hands" and+ * "two dots". They straddle [HAND_Y], just above the ledge, and+ * [clipToLedge] cuts their bottoms off — you see the top of a paw resting+ * on a ledge, which is what makes the ledge exist at all.+ *+ * Mood colour, like the mouth. Half the ball is below the edge while it+ * clings and the mouth goes down with it, so without this the mood would+ * have nothing but the rim left to show in.+ */+ private fun drawHands() {+ val c = state.cling+ if (c < 0.02f) return+ GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA)+ GLES20.glDepthMask(true)+ solidShader.use()+ val uMVP = solidShader.uniform("uMVP")+ GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, state.nowColor, 0)+ GLES20.glUniform1f(solidShader.uniform("uAlpha"), 1f)+ handMesh.bind(solidShader.attrib("aPos"))++ // Bearing weight splays the palm and flattens it, and the toes spread+ // and curl down over the lip — all peaking on the haul, where the body+ // is actually pulling against them.+ val g = state.grip+ val palmW = HAND_W * (1f + g * 0.2f) * c+ val palmH = HAND_H * (1f - g * 0.22f) * c+ for (side in -1..1 step 2) {+ val cx = side * (HAND_X + g * 0.03f)+ blob(uMVP, cx, HAND_Y, palmW, palmH)+ for (n in -1..1) {+ blob(+ uMVP,+ cx + n * HAND_TOE_GAP * (1f + g * 0.22f),+ HAND_Y + HAND_TOE_RISE - g * 0.018f,+ HAND_TOE * c, HAND_TOE * c,+ )+ }+ }+ }++ /** One paw part, in world space — no [root], see [drawHands]. */+ private fun blob(uMVP: Int, x: Float, y: Float, sx: Float, sy: Float) {+ Matrix.setIdentityM(model, 0)+ Matrix.translateM(model, 0, x, y, HAND_Z)+ Matrix.scaleM(model, 0, sx, sy, HAND_TOE)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)+ GLES20.glUniformMatrix4fv(uMVP, 1, false, mvp, 0)+ handMesh.draw()+ }++ private fun drawRim() {+ System.arraycopy(root, 0, model, 0, 16)+ Matrix.scaleM(model, 0, RIM, RIM, RIM)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)++ GLES20.glDepthMask(false)+ rimShader.use()+ GLES20.glUniformMatrix4fv(rimShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniformMatrix4fv(rimShader.uniform("uMV"), 1, false, mv, 0)+ GLES20.glUniform1f(rimShader.uniform("uPow"), LOOK.rimPow)+ // The scan band is a *light* sweep: it adds uColor along a moving+ // horizontal line, so it is only meaningful where uColor is a light —+ // the glow branch. On the dark-contour branch uColor is a shadow shade,+ // and sweeping that paints a moving dark stripe across the ball, which+ // reads as a rendering fault rather than as a scan. Zeroed there.+ GLES20.glUniform1f(rimShader.uniform("uScanY"), state.scanY)+ GLES20.glUniform1f(rimShader.uniform("uScan"), if (LOOK.rimDark) 0f else state.scan)+ if (LOOK.rimDark) {+ // A tight dark contour instead of a glow. The body is drawn inside+ // [RIM] and this sphere sits on it, so the visible band *is* the+ // outline; a pale ball needs it to hold its edge over a pale app,+ // and a colour-additive glow cannot do that job on a white+ // background.+ GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA)+ for (i in 0..2) {+ // Slightly blue-shifted so the outline reads as a shadow rather+ // than as dirt.+ outlineColor[i] = state.nowColor[i] * if (i == 2) 0.42f else 0.34f+ }+ GLES20.glUniform3fv(rimShader.uniform("uColor"), 1, outlineColor, 0)+ GLES20.glUniform1f(rimShader.uniform("uIntensity"), LOOK.rimGain * 1.15f)+ GLES20.glUniform1f(rimShader.uniform("uCap"), RIM_DARK_CAP)+ } else {+ GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE)+ GLES20.glUniform3fv(rimShader.uniform("uColor"), 1, state.nowColor, 0)+ GLES20.glUniform1f(rimShader.uniform("uIntensity"), state.rimIntensity * LOOK.rimGain)+ GLES20.glUniform1f(rimShader.uniform("uCap"), 1f)+ }+ rimSphere.bind(rimShader.attrib("aPos"), rimShader.attrib("aNormal"))+ rimSphere.draw()+ }++ private fun drawCore() {+ System.arraycopy(root, 0, model, 0, 16)+ Matrix.scaleM(model, 0, state.coreScale, state.coreScale, state.coreScale)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)++ GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE)+ GLES20.glDepthMask(false)+ coreShader.use()+ GLES20.glUniformMatrix4fv(coreShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniform3fv(coreShader.uniform("uColor"), 1, state.nowColor, 0)+ GLES20.glUniform1f(coreShader.uniform("uAlpha"), state.coreAlpha)+ core.bind(coreShader.attrib("aPos"))+ core.draw()+ }++ private fun drawRings() {+ GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE)+ GLES20.glDepthMask(false)+ flowShader.use()+ val aPos = flowShader.attrib("aPos")+ val aUV = flowShader.attrib("aUV")++ for (i in 0..3) {+ val r = RINGS[i]+ // ringLocal = spread * rings_group_rotX * ring_rotY * torus_local_orient+ Matrix.setIdentityM(local, 0)+ Matrix.scaleM(local, 0, state.spread, state.spread, state.spread)+ Matrix.rotateM(local, 0, state.ringsRotX * DEG, 1f, 0f, 0f)+ Matrix.rotateM(local, 0, state.ringRotY[i] * DEG, 0f, 1f, 0f)+ Matrix.rotateM(local, 0, r.rotX * DEG, 1f, 0f, 0f)+ Matrix.rotateM(local, 0, r.rotY * DEG, 0f, 1f, 0f)+ Matrix.rotateM(local, 0, r.rotZ * DEG, 0f, 0f, 1f)+ Matrix.multiplyMM(model, 0, root, 0, local, 0)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)++ GLES20.glUniformMatrix4fv(flowShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniform3fv(flowShader.uniform("uColor"), 1, state.nowColor, 0)+ GLES20.glUniform1f(+ flowShader.uniform("uBase"),+ r.base * (1f + state.open * 1.1f + state.busy * 1.4f),+ )+ GLES20.glUniform1f(flowShader.uniform("uFlow"), state.flow[i])+ GLES20.glUniform1f(+ flowShader.uniform("uPulse"),+ 0.3f + state.open * 0.45f + state.busy * 0.75f,+ )+ ringMeshes[i].bind(aPos, -1, aUV)+ ringMeshes[i].draw()+ }+ }++ private fun drawDust() {+ state.writeTails(tails.data)+ state.writeHeads(heads.data)+ tails.upload()+ heads.upload()++ System.arraycopy(root, 0, model, 0, 16)+ Matrix.multiplyMM(mv, 0, view, 0, model, 0)+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)++ GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE)+ GLES20.glDepthMask(false)+ pointsShader.use()+ GLES20.glUniformMatrix4fv(pointsShader.uniform("uMVP"), 1, false, mvp, 0)+ GLES20.glUniformMatrix4fv(pointsShader.uniform("uMV"), 1, false, mv, 0)+ GLES20.glUniform1f(pointsShader.uniform("uPointFactor"), pointFactor)++ // Tails first — heads paint over them, brightest on top.+ GLES20.glUniform1f(pointsShader.uniform("uSize"), 0.03f)+ GLES20.glUniform1f(pointsShader.uniform("uAlpha"), state.dustTailAlpha)+ tails.bind(pointsShader.attrib("aPos"), pointsShader.attrib("aColor"))+ tails.draw()++ GLES20.glUniform1f(pointsShader.uniform("uSize"), 0.052f)+ GLES20.glUniform1f(pointsShader.uniform("uAlpha"), state.dustHeadAlpha)+ heads.bind(pointsShader.attrib("aPos"), pointsShader.attrib("aColor"))+ heads.draw()+ }++ companion object {+ private const val DEG = 57.29578f // rad → deg for Matrix.rotateM++ /** Camera distance on +Z, looking at the origin. */+ private const val CAM_Z = 4.9f++ /** The glowing outline. Defines how big the ball looks. */+ private const val RIM = 0.9f++ /** How much darker than the mood the mouth goes on a pale look. */+ private const val MOUTH_DEEP = 0.40f++ /**+ * Ceiling on the dark outline's opacity. It is drawn over the 1.7% of the+ * rim sphere that sticks out past the body, and at full strength that+ * band reads as a stroke drawn round the ball rather than as its edge.+ */+ private const val RIM_DARK_CAP = 0.82f++ // ---- Pupils ----+ //+ // A dark relative of the mood colour rather than a fixed black, so the+ // eyes carry the mood with the rest of the ball instead of punching a+ // hole in it. The lift is what keeps the pupil from ever reaching zero:+ // a pupil that is pure black on a dark ball loses its edge.+ private const val PUPIL_TINT = 0.20f+ private val PUPIL_LIFT = floatArrayOf(0.045f, 0.050f, 0.075f)++ /**+ * Vertical aperture below which the pupil is not drawn. The `^ ^` moods+ * drive the round eye below this, and a pupil without a white eye around+ * it is a dot on the face.+ */+ private const val PUPIL_MIN_APERTURE = 0.05f++ private const val DISC_THICKNESS = 0.08f+ private const val GLINT_Z_STEP = 0.02f++ /**+ * How far [EmotionState.brow] swings a look's resting brow, radians.+ *+ * The mood's brow knob is ±1-ish, so this is the whole range a mood can+ * move the angle through on top of [BallLook.browTilt]. Kept well under+ * the resting scowl: a mood must be able to soften an angry brow, not+ * to flip the character out of being angry.+ */+ private const val BROW_MOOD_TILT = 0.30f++ /**+ * How far in from each corner of the mouth the tooth row starts,+ * radians of the mouth's own arc.+ *+ * A tooth placed exactly at the corner stands perpendicular to a+ * near-vertical piece of the curve, i.e. it points sideways off the+ * face — which reads as a chip flying off rather than as a tooth.+ */+ private const val TOOTH_PAD = 0.26f++ /**+ * How much darker the nostril dots are than the live [bodyColor]. A+ * multiple rather than a fixed black so the snout stays consistent as+ * the mood tints the body — see [BallLook.browShade] for the same+ * reasoning.+ */+ private const val NOSTRIL_DARK = 0.28f++ // ---- Tail ----+ //+ // The count is a *coverage* number, not a taste one: the blobs have to+ // overlap or the chain reads as a row of separate dots, which is what+ // five gave on the tablet — a step of len/5 = 0.11 against a lead blob+ // 0.11 across, so they met end to end and every blob after that was+ // thinner than the gap it had to bridge. Nine puts the step at ~0.06,+ // under half the lead diameter, and the taper never catches up with it.+ private const val TAIL_BLOBS = 9++ /** Direction of the first step, radians from +X. Out and downward. */+ private const val TAIL_START = -0.80f++ /** Total turn from root to barb. ~150°, which is the classic hook. */+ private const val TAIL_SWEEP = 2.60f++ /** How much thinner the last blob is than the first. */+ private const val TAIL_TAPER = 0.45f++ private val WHITE = floatArrayOf(1f, 1f, 1f)++ /** Bottom of the eye capsule (half-cylinder + cap), the lid's hinge. */+ private const val EYE_LID_PIVOT = -0.125f++ /** How much of a circle the `^` eye covers. */+ private val EYE_ARC_SPAN = 0.8f * PI.toFloat()++ /** Rotation that moves the arc's midpoint from SPAN/2 up to the top. */+ private val EYE_ARC_ROT_Z = PI.toFloat() / 2f - EYE_ARC_SPAN / 2f++ /** The arch sits above its own origin; drop it back over the eye. */+ private const val EYE_ARC_DROP = 0.06f++ /**+ * World Y of the screen edge the paws hang on, derived from how far+ * [FloatingWindowUi] pushes the window down. Hiding half the ball puts+ * it through the equator, hence 0 — but deriving it means the paws+ * follow if [CLING_HIDDEN] ever moves.+ */+ private val HAND_LEDGE = RIM * (2f * CLING_HIDDEN - 1f)++ // ---- Paw geometry, all in world units. Two paws of four blobs each:+ // ---- a wide flat palm on the ledge with three toes crowning it.+ //+ // Sized by what survives the clip rather than by what looks right in+ // isolation: everything below [HAND_LEDGE] is cut away, so only the+ // part above it is ever seen, and the first version put barely 10dp of+ // paw over the line — invisible in practice. The palm now straddles the+ // ledge with most of its bulk above, and the toes clear it by about+ // twice as much as before.+ private const val HAND_X = 0.56f+ private val HAND_Y = HAND_LEDGE + 0.05f++ /**+ * In front of the body's surface at every x the palm covers — at+ * x = 0.35 the sphere is already at z = 0.81, and a palm centred+ * shallower than this gets its inner edge swallowed.+ */+ private const val HAND_Z = 0.88f++ private const val HAND_W = 0.24f+ private const val HAND_H = 0.12f+ private const val HAND_TOE = 0.08f++ /** Under one toe diameter, so the toes fuse instead of reading as peas. */+ private const val HAND_TOE_GAP = 0.125f++ /** How far the toes crown the palm — the part that clears its top edge. */+ private const val HAND_TOE_RISE = 0.16f++ /** How long one `screen.*` command keeps the SEARCHING face. */+ private const val TOOL_SECONDS = 2.2f++ /** How long a finished task's SUCCESS / FAILED face lingers. */+ private const val OVERLAY_SECONDS = 1.9f++ /**+ * Idle time before the ball dozes off. Long enough that it never+ * happens mid-conversation, short enough that a tablet left on a desk+ * stops looking like it is waiting for something.+ */+ private const val SLEEP_AFTER_SECONDS = 150f++ // ---- Idle fidgets ----+ //+ // Random body-only gestures while awake and bored. The schedule:+ // nothing for the first FIDGET_AFTER_SECONDS after going idle (a ball+ // that starts dancing the moment you look away is exhausting), then+ // one fidget every FIDGET_MIN..FIDGET_MAX seconds — long enough apart+ // that spotting one feels like catching it doing something, not like+ // watching a screensaver.+ private const val FIDGET_AFTER_SECONDS = 12f+ private const val FIDGET_MIN_SECONDS = 20f+ private const val FIDGET_MAX_SECONDS = 50f++ // ---- Body: top-lit gradient + one specular dot + an edge lift ----+ private const val SHELL_VS = """+ uniform mat4 uMVP;+ uniform mat4 uMV;+ attribute vec3 aPos;+ attribute vec3 aNormal;+ varying vec3 vN;+ varying vec3 vP;+ void main() {+ vN = normalize(mat3(uMV) * aNormal);+ vec4 p = uMV * vec4(aPos, 1.0);+ vP = p.xyz;+ gl_Position = uMVP * vec4(aPos, 1.0);+ }+ """+ private const val SHELL_FS = """+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ uniform float uGrad;+ uniform float uAmb;+ uniform float uDiff;+ uniform float uGloss;+ varying vec3 vN;+ varying vec3 vP;+ void main() {+ vec3 n = normalize(vN);+ vec3 V = normalize(-vP);+ // Vertical gradient — darker underneath, so the ball reads as a+ // solid sitting in light rather than a disc with an outline.+ float vert = n.y * 0.5 + 0.5;+ vec3 base = uColor * mix(1.0, mix(0.70, 1.14, vert), uGrad);+ // Key light overhead and slightly left, and aimed that way on+ // purpose: a specular aimed frontally lands on the eyes and takes+ // half their contrast with it.+ vec3 L = normalize(vec3(-0.48, 0.85, 0.20));+ float diff = max(0.0, dot(n, L));+ vec3 lit = base * (uAmb + uDiff * diff);+ vec3 H = normalize(L + V);+ float spec = pow(max(0.0, dot(n, H)), 90.0) * uGloss;+ // A little extra colour right at the silhouette, which is what+ // holds the ball apart from a same-valued background.+ float edge = pow(1.0 - max(0.0, dot(n, V)), 3.0) * uGloss * 0.26;+ gl_FragColor = vec4(lit + vec3(spec) + uColor * edge, uAlpha);+ }+ """++ // ---- Rim + scan (direct port of RIM_VERT/FRAG) ----+ private const val RIM_VS = """+ uniform mat4 uMVP;+ uniform mat4 uMV;+ attribute vec3 aPos;+ attribute vec3 aNormal;+ varying vec3 vN;+ varying vec3 vP;+ varying float vY;+ void main() {+ vN = normalize(mat3(uMV) * aNormal);+ vec4 p = uMV * vec4(aPos, 1.0);+ vP = p.xyz;+ vY = aPos.y;+ gl_Position = uMVP * vec4(aPos, 1.0);+ }+ """+ private const val RIM_FS = """+ precision mediump float;+ uniform vec3 uColor;+ uniform float uIntensity;+ uniform float uScan;+ uniform float uScanY;+ uniform float uPow;+ uniform float uCap;+ varying vec3 vN;+ varying vec3 vP;+ varying float vY;+ void main() {+ float f = pow(1.0 - abs(dot(normalize(vN), normalize(-vP))), uPow);+ float d = vY - uScanY;+ float slope = d > 0.0 ? 7.5 : 4.2;+ float band = pow(max(0.0, 1.0 - abs(d) * slope), 3.0);+ gl_FragColor = vec4(uColor, min(f * uIntensity + band * uScan, uCap));+ }+ """++ // ---- Bubble: a transparent droplet, not a solid blob ----+ //+ // The back half is discarded rather than depth-culled: inside one+ // indexed draw the triangles arrive in no useful order, so a front-half+ // fragment can be rasterised after the back-half fragment at the same+ // pixel and both would blend. `facing <= 0` drops the back half outright,+ // and it does so without depending on the mesh's winding.+ private const val BUBBLE_VS = """+ uniform mat4 uMVP;+ uniform mat4 uMV;+ attribute vec3 aPos;+ attribute vec3 aNormal;+ varying vec3 vN;+ varying vec3 vP;+ void main() {+ vN = normalize(mat3(uMV) * aNormal);+ vec4 mvpos = uMV * vec4(aPos, 1.0);+ vP = mvpos.xyz;+ gl_Position = uMVP * vec4(aPos, 1.0);+ }+ """++ private const val BUBBLE_FS = """+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ uniform float uPow;+ uniform float uShine;+ uniform float uInner;+ varying vec3 vN;+ varying vec3 vP;+ void main() {+ vec3 n = normalize(vN);+ vec3 V = normalize(-vP);+ float facing = dot(n, V);+ if (facing <= 0.0) discard;+ float edge = pow(1.0 - facing, uPow);+ vec3 L = normalize(vec3(-0.45, 0.80, 0.60));+ vec3 H = normalize(L + V);+ float spec = pow(max(0.0, dot(n, H)), uShine);+ float a = clamp(edge + uInner + spec, 0.0, 1.0);+ vec3 c = mix(uColor, vec3(1.0), clamp(spec * 2.5, 0.0, 1.0));+ gl_FragColor = vec4(c, a * uAlpha);+ }+ """++ // ---- Core: flat color ----+ private const val CORE_VS = """+ uniform mat4 uMVP;+ attribute vec3 aPos;+ void main() {+ gl_Position = uMVP * vec4(aPos, 1.0);+ }+ """+ private const val CORE_FS = """+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ void main() {+ gl_FragColor = vec4(uColor, uAlpha);+ }+ """++ // ---- Ring flow (direct port of FLOW_VERT/FRAG) ----+ private const val FLOW_VS = """+ uniform mat4 uMVP;+ attribute vec3 aPos;+ attribute vec2 aUV;+ varying vec2 vUv;+ void main() {+ vUv = aUV;+ gl_Position = uMVP * vec4(aPos, 1.0);+ }+ """+ private const val FLOW_FS = """+ precision mediump float;+ uniform vec3 uColor;+ uniform float uBase;+ uniform float uFlow;+ uniform float uPulse;+ varying vec2 vUv;+ void main() {+ float d = fract(uFlow - vUv.x);+ float head = pow(1.0 - d, 24.0);+ float tail = pow(1.0 - d, 3.0);+ gl_FragColor = vec4(+ uColor * (1.0 + head * 1.9),+ uBase + (head + tail * 0.34) * uPulse+ );+ }+ """++ // ---- Point sprites (dust). Size attenuates with distance so nearer+ // particles render bigger, matching R3F's sizeAttenuation. ----+ private const val POINTS_VS = """+ uniform mat4 uMVP;+ uniform mat4 uMV;+ uniform float uSize;+ uniform float uPointFactor;+ attribute vec3 aPos;+ attribute vec3 aColor;+ varying vec3 vColor;+ void main() {+ vec4 mvpos = uMV * vec4(aPos, 1.0);+ gl_Position = uMVP * vec4(aPos, 1.0);+ gl_PointSize = uSize * uPointFactor / -mvpos.z;+ vColor = aColor;+ }+ """+ private const val POINTS_FS = """+ precision mediump float;+ uniform float uAlpha;+ varying vec3 vColor;+ void main() {+ vec2 c = gl_PointCoord - 0.5;+ float d = length(c);+ float a = smoothstep(0.5, 0.0, d);+ gl_FragColor = vec4(vColor, a * uAlpha);+ }+ """++ // ---- Solid color (face parts) ----+ private const val SOLID_VS = """+ uniform mat4 uMVP;+ attribute vec3 aPos;+ void main() {+ gl_Position = uMVP * vec4(aPos, 1.0);+ }+ """+ private const val SOLID_FS = """+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ void main() {+ gl_FragColor = vec4(uColor, uAlpha);+ }+ """+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallTextureView.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallTextureView.ktnew file mode 100644index 0000000..308ca10--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallTextureView.kt@@ -0,0 +1,207 @@+package net.kuafuai.andee.ui.ball++import android.content.Context+import android.graphics.SurfaceTexture+import android.opengl.EGL14+import android.opengl.EGLConfig+import android.opengl.EGLContext+import android.opengl.EGLDisplay+import android.opengl.EGLSurface+import android.os.Handler+import android.os.HandlerThread+import android.util.AttributeSet+import android.view.Surface+import android.view.TextureView++/**+ * TextureView-hosted EmotionBall.+ *+ * Unlike GLSurfaceView, TextureView composites through the normal View+ * hierarchy — so sibling Views (subtitle pill, settings gear, resize handle)+ * can be drawn on top of the ball without the z-order tricks that+ * `setZOrderOnTop(true)` on a SurfaceView forces. That's the only reason we+ * carry the extra EGL boilerplate here; the [EmotionBallRenderer] itself is+ * unchanged from the GLSurfaceView version.+ *+ * Manages a dedicated GL thread + EGL context/surface tied to the TextureView's+ * SurfaceTexture lifecycle. When the surface goes away (window removed) the+ * thread and EGL resources are cleaned up.+ */+class EmotionBallTextureView @JvmOverloads constructor(+ context: Context,+ attrs: AttributeSet? = null,+) : TextureView(context, attrs), TextureView.SurfaceTextureListener {++ private val renderer = EmotionBallRenderer()++ private var glThread: HandlerThread? = null+ private var glHandler: Handler? = null++ private var eglDisplay: EGLDisplay? = null+ private var eglContext: EGLContext? = null+ private var eglSurface: EGLSurface? = null+ private var androidSurface: Surface? = null++ @Volatile+ private var running = false++ init {+ surfaceTextureListener = this+ isOpaque = false // required for alpha compositing with parent views+ }++ fun setMood(m: Mood) {+ renderer.mood = m+ }++ fun setListening(l: Boolean) {+ renderer.listening = l+ }++ /** A task is running, for its whole duration — see [EmotionBallRenderer.working]. */+ fun setWorking(w: Boolean) {+ renderer.working = w+ }++ /**+ * The perch geometry is applied — the window is (or has just settled) at+ * the ledge. This is what licenses the cling animation; see+ * [EmotionBallRenderer.atLedge] for why working alone is not enough.+ */+ fun setAtLedge(at: Boolean) {+ renderer.atLedge = at+ }++ /** A signboard is up and the ball is waiting for the user's choice. */+ fun setHolding(h: Boolean) {+ renderer.holding = h+ }++ /** It has dozed off — see [EmotionBallRenderer.dozing]. Safe from any thread. */+ fun isDozing(): Boolean = renderer.dozing++ /** When true, cycles through all moods every ~5.5 s. Handy for testing. */+ fun setDemoCycle(on: Boolean) {+ renderer.demoCycle = on+ }++ /** The agent just ran a tool against the device. */+ fun pulseTool() {+ renderer.pulseTool()+ }++ /** A task finished. */+ fun signalOutcome(ok: Boolean) {+ renderer.signalOutcome(ok)+ }++ /** One-shot body gesture, e.g. a nod when the mic opens. */+ fun trigger(action: Action) {+ renderer.request(action)+ }++ // ---- SurfaceTextureListener ----++ override fun onSurfaceTextureAvailable(surface: SurfaceTexture, width: Int, height: Int) {+ val t = HandlerThread("EmotionBallGL").apply { start() }+ glThread = t+ val h = Handler(t.looper)+ glHandler = h+ h.post {+ initEGL(surface)+ renderer.onSurfaceCreated(null, null)+ renderer.onSurfaceChanged(null, width, height)+ running = true+ scheduleFrame(h)+ }+ }++ override fun onSurfaceTextureSizeChanged(surface: SurfaceTexture, width: Int, height: Int) {+ glHandler?.post {+ renderer.onSurfaceChanged(null, width, height)+ }+ }++ override fun onSurfaceTextureDestroyed(surface: SurfaceTexture): Boolean {+ running = false+ val h = glHandler+ val t = glThread+ glHandler = null+ glThread = null+ h?.post { destroyEGL() }+ t?.quitSafely()+ return true+ }++ override fun onSurfaceTextureUpdated(surface: SurfaceTexture) {}++ // ---- Frame loop ----++ private fun scheduleFrame(h: Handler) {+ if (!running) return+ h.post {+ if (!running) return@post+ val d = eglDisplay ?: return@post+ val s = eglSurface ?: return@post+ renderer.onDrawFrame(null)+ EGL14.eglSwapBuffers(d, s)+ h.postDelayed({ scheduleFrame(h) }, 16) // ~60 fps+ }+ }++ // ---- EGL ----++ private fun initEGL(surfaceTexture: SurfaceTexture) {+ val display = EGL14.eglGetDisplay(EGL14.EGL_DEFAULT_DISPLAY)+ check(display != EGL14.EGL_NO_DISPLAY) { "eglGetDisplay failed" }+ val version = IntArray(2)+ check(EGL14.eglInitialize(display, version, 0, version, 1)) { "eglInitialize failed" }++ val configs = arrayOfNulls<EGLConfig>(1)+ val numConfigs = IntArray(1)+ EGL14.eglChooseConfig(+ display,+ intArrayOf(+ EGL14.EGL_RED_SIZE, 8,+ EGL14.EGL_GREEN_SIZE, 8,+ EGL14.EGL_BLUE_SIZE, 8,+ EGL14.EGL_ALPHA_SIZE, 8,+ EGL14.EGL_DEPTH_SIZE, 16,+ EGL14.EGL_STENCIL_SIZE, 0,+ EGL14.EGL_RENDERABLE_TYPE, EGL14.EGL_OPENGL_ES2_BIT,+ EGL14.EGL_NONE,+ ),+ 0, configs, 0, 1, numConfigs, 0,+ )+ val config = configs[0]!!++ val ctx = EGL14.eglCreateContext(+ display, config, EGL14.EGL_NO_CONTEXT,+ intArrayOf(EGL14.EGL_CONTEXT_CLIENT_VERSION, 2, EGL14.EGL_NONE), 0,+ )+ val surface = Surface(surfaceTexture)+ val eSurface = EGL14.eglCreateWindowSurface(+ display, config, surface,+ intArrayOf(EGL14.EGL_NONE), 0,+ )+ EGL14.eglMakeCurrent(display, eSurface, eSurface, ctx)++ eglDisplay = display+ eglContext = ctx+ eglSurface = eSurface+ androidSurface = surface+ }++ private fun destroyEGL() {+ val d = eglDisplay ?: return+ EGL14.eglMakeCurrent(d, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_CONTEXT)+ eglSurface?.let { EGL14.eglDestroySurface(d, it) }+ eglContext?.let { EGL14.eglDestroyContext(d, it) }+ EGL14.eglTerminate(d)+ androidSurface?.release()+ eglSurface = null+ eglContext = null+ eglDisplay = null+ androidSurface = null+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionState.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionState.ktnew file mode 100644index 0000000..1977576--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionState.kt@@ -0,0 +1,557 @@+package net.kuafuai.andee.ui.ball++import kotlin.math.PI+import kotlin.math.abs+import kotlin.math.acos+import kotlin.math.cos+import kotlin.math.floor+import kotlin.math.max+import kotlin.math.min+import kotlin.math.sin+import kotlin.random.Random++/**+ * Per-frame animated state for the ball. Ported from the R3F useFrame body.+ *+ * Call [tick] every frame; then read the public fields to drive uniforms and+ * matrices. Anything that needs to lerp toward a target lives here, not in the+ * renderer.+ */+class EmotionState {++ /**+ * The look being drawn, re-read every access.+ *+ * A getter rather than a stored field because the user can swipe the ball+ * to another look at any moment; a `val` captured at construction would+ * leave this class animating the previous character's face. Everything it+ * feeds is recomputed in [tick], so following the switch costs nothing —+ * the only values that would lag are the resting positions below, and they+ * are overwritten on the first frame after the change.+ */+ private val LOOK: BallLook get() = BallLooks.active++ // ---- Face + color, lerped toward FACE[mood] ----+ val nowColor = FloatArray(3).also { FACE[Mood.CALM]!!.color.copyInto(it) }+ var eye = 1f; private set+ var brow = 0f; private set+ var smile = 0.1f; private set+ var tilt = 0f; private set+ var skew = 0f; private set+ var jitter = 0f; private set+ var open = 0f; private set // mouth open (talk envelope)+ var eyeArc = 0f; private set // 0 = round eye, 1 = ^ ^ arc+ var blush = 0f; private set+ var spread = 1f; private set // ring belt radius multiplier++ /**+ * 鼻涕泡的浓度,0..1。跟 [blush] 一样是**淡入淡出**的:它决定泡在不在,+ * 不决定泡多大 —— 大小是 [bubbleScale] 的事。+ */+ var bubble = 0f; private set++ /** 泡自己的一呼一吸,0.30(瘪)..1.00(鼓)。 */+ var bubbleScale = 1f; private set++ private var bubblePhase = 0f++ /** Blink, 1 = open, 0 = shut. Applies to both eye shapes. */+ var lid = 1f; private set++ /** 0..1, smoothed "how much am I thinking" */+ var think = 0f; private set++ /**+ * 0..1, smoothed mirror of the `listening` flag.+ *+ * Smoothed because the rim pulse it drives has to fade in and out; a hard+ * boolean stepped the outline brightness by a factor of two between frames,+ * which reads as a glitch rather than as attention.+ */+ private var listen = 0f++ /** listening? 0.9 : think — drives ring/scan/dust speed */+ var busy = 0f; private set++ // ---- Root transform (root group has these applied in order Rx*Ry*Rz*Scale) ----+ var rotX = 0f; private set+ var rotY = 0f; private set+ var rotZ = 0f; private set+ var breathe = 1f; private set++ /**+ * Lerped [Face.breath]. Tweened rather than read straight off the target+ * face so a mood change doesn't step the body scale — that would land as a+ * flinch, not a breath.+ */+ private var breatheDepth = 0.022f++ /**+ * Breathing folded together with the current action's squash, as the two+ * axes the renderer actually scales by. Volume is roughly conserved — a ball+ * that only flattens reads as deflating, not as landing.+ */+ var scaleX = 1f; private set+ var scaleY = 1f; private set++ /** Vertical body offset, in world units. Non-zero only during a HOP. */+ var hopY = 0f; private set++ /**+ * 0..1, smoothed: how much the ball is hanging off the bottom edge. The+ * renderer reads it to fade the paws in and to decide whether to draw them+ * at all. Public because the paws are a *drawn* thing, unlike everything+ * else cling does, which is only a transform.+ */+ var cling = 0f; private set++ /**+ * How hard it is currently gripping, 0..1. Peaks during the haul back up —+ * the paws flatten and spread on the edge exactly when the body is pulling+ * against them, which is what makes the pull look like it costs something.+ */+ var grip = 0f; private set++ // ---- Rings ----+ var ringsRotX = 0f; private set+ val ringRotY = FloatArray(4)+ val flow = FloatArray(4)++ // ---- Rim + scan ----+ var scanY = -2f; private set+ var scan = 0f; private set+ var rimIntensity = 0.9f; private set++ // ---- Core ----+ var coreScale = 0.42f; private set+ var coreAlpha = 0.16f; private set++ // ---- Face element transforms ----+ // Bases come from [BallLook] — the renderer sizes its eye and mouth meshes+ // off the same object, so the two can never disagree about where the face is.+ var eyeLX = -LOOK.eyeX;+ var eyeLY = LOOK.eyeY;+ var eyeLRotZ = 0f;+ var eyeYScale = 1f+ var eyeRX = LOOK.eyeX;+ var eyeRY = LOOK.eyeY;+ var eyeRRotZ = 0f+ var mouthY = LOOK.mouthY;+ var mouthScaleY = 0.16f;+ var mouthRotZ = PI.toFloat()++ // ---- Dust particles ----+ private class Particle(+ val sinPhi: Float, val cosPhi: Float,+ val a0: Float, val rate: Float,+ val r: Float, val b: Float,+ )++ private val particles = Array(DUST_COUNT) {+ val phi = acos(2f * Random.nextFloat() - 1f)+ Particle(+ sinPhi = sin(phi),+ cosPhi = cos(phi),+ a0 = Random.nextFloat() * 2f * PI.toFloat(),+ rate = 0.55f + Random.nextFloat() * 1.5f,+ r = 1.28f + Random.nextFloat() * 0.46f,+ b = 0.55f + Random.nextFloat() * 0.45f,+ )+ }++ /** Alpha targets that the renderer reads for the two dust passes. */+ var dustHeadAlpha = 0.6f; private set+ var dustTailAlpha = 0.5f; private set++ // ---- Internal integrated angles (see comment on `spin` in R3F source:+ // ---- speeds vary over time, so we must integrate dt*rate per frame instead+ // ---- of setting angle = t * rate) ----+ private var ringsXAngle = 0f+ private var scanAngle = 0f+ private var dustAngle = 0f+ private var dustRate = 0f+ private var dustLag = 0f+ private val ringAngle = FloatArray(4)+ private val flowAngle = FloatArray(4)++ // ---- Blink ----+ private var blinkNext = 2.4f+ private var blinkLeft = 0f++ // ---- One-shot action ----+ private var action = Action.NONE+ private var actionT = 0f+ private var actionDur = 0f++ // ---- Clinging to the bottom screen edge ----+ /** Runs 0→1 over one sag-and-haul cycle. See the slip block in [tick]. */+ private var slipPhase = 0f++ /**+ * Start a body gesture, cancelling any gesture already running.+ *+ * Cancelling rather than queueing is deliberate: these are 0.5 s reactions+ * to things that just happened, and a queue would play them after the thing+ * they were reacting to is over. A tool call arriving mid-nod should restart+ * the nod, not book a second one.+ *+ * Call from the GL thread (i.e. from inside the renderer's frame callback).+ */+ fun trigger(a: Action) {+ action = a+ actionT = 0f+ actionDur = ACTION_SECONDS[a.ordinal]+ }++ fun tick(dt: Float, t: Float, mood: Mood, listening: Boolean, working: Boolean) {+ val want = FACE[mood]!!++ // Lerp toward the target face. 0.14 s to reach = min(1, dt*7).+ val k = min(1f, dt * 7f)+ eye += (want.eye - eye) * k+ brow += (want.brow - brow) * k+ smile += (want.smile - smile) * k+ tilt += (want.tilt - tilt) * k+ skew += (want.skew - skew) * k+ jitter += (want.jitter - jitter) * min(1f, dt * 2.4f) // jitter decays slower+ eyeArc += (want.eyeArc - eyeArc) * k+ blush += (want.blush - blush) * k+ spread += (want.spread - spread) * min(1f, dt * 3.5f) // belt is heavy+ for (i in 0..2) nowColor[i] += (want.color[i] - nowColor[i]) * k++ // Thinking intensity — half-sec in/out, so answers don't snap the ring+ // speed back to idle mid-glance. SEARCHING counts: driving an app is+ // thinking as far as everything downstream of `busy` is concerned.+ val thinkTarget = if (mood == Mood.THINKING || mood == Mood.SEARCHING) 1f else 0f+ think += (thinkTarget - think) * min(1f, dt * 2.2f)+ val th = think+ listen += ((if (listening) 1f else 0f) - listen) * min(1f, dt * 3.4f)+ busy = if (listening) 0.9f else th++ // Gaze — eye drift while thinking. Three incommensurate low-freq sines+ // so there's no recognizable period.+ val gaze = th * (sin(t * 0.9f) + sin(t * 1.31f) * 0.5f)+ val gazeY = th * (sin(t * 0.73f) * 0.6f + sin(t * 1.07f) * 0.35f)++ // Jitter — tense/anxious shake, two incommensurate high-freq sines+ val jx = (sin(t * 23.7f) + sin(t * 37.1f) * 0.6f) * jitter+ val jy = (sin(t * 29.3f) + sin(t * 41.7f) * 0.6f) * jitter++ // Mouth open — no voice.level() yet, use sine envelope when SPEAKING+ // (matches the R3F fallback path when the audio source can't report+ // level). Everything else falls back to the mood's resting [Face.gape],+ // which is how LISTENING gets a mouth that is open rather than curved.+ val talkTarget = if (mood == Mood.SPEAKING) {+ 0.34f + sin(t * 17f) * 0.16f + sin(t * 29f) * 0.09f+ } else want.gape+ open += (max(0f, talkTarget) - open) * min(1f, dt * 22f)++ // Blink. More frequent when listening (like nodding), less when thinking.+ //+ // ...and not at all once the eye on screen is the `^ ^` arc. A blink is+ // a lid coming down over an *open* eye; when the eye is already drawn+ // shut there is no lid to lower, and what the blink actually does is+ // twitch the arc — a sleeping ball flinching every three seconds. The+ // threshold is shared with the renderer's "does this eye still want a+ // pupil", which is the same question asked from the other end.+ val eyeShut = eyeArc >= ARC_EYE_AT+ blinkNext -= dt+ if (blinkNext <= 0f) {+ if (eyeShut) {+ // Cancel anything in flight and re-check shortly, so waking up+ // doesn't mean waiting out a full blink interval.+ blinkLeft = 0f+ blinkNext = 0.35f+ } else {+ blinkLeft = 0.12f+ val base = (if (listening) 1.8f else 3.2f) + Random.nextFloat() * 2.6f+ blinkNext = (base * (1f + th * 0.9f)) / (1f + jitter * 1.6f)+ }+ }+ var lid = 1f+ if (blinkLeft > 0f) {+ blinkLeft -= dt+ lid = abs(blinkLeft / 0.06f - 1f) // triangle: 1 → 0 → 1+ }+ this.lid = lid++ // One-shot action envelope. Everything below is a pure function of `p`,+ // which runs 0→1 over the action's duration — so the gesture is simply+ // over when p reaches 1, with no tail left to decay and no way to get+ // stuck half-deformed if a frame is dropped.+ var sqx = 0f+ var sqy = 0f+ var hop = 0f+ var actRotX = 0f+ var actRotY = 0f+ if (action != Action.NONE) {+ actionT += dt+ val p = min(1f, actionT / actionDur)+ val arch = sin(p * PI.toFloat()) // 0 → 1 → 0+ when (action) {+ Action.HOP -> {+ hop = arch * 0.22f+ // Squash only while in contact with the ground, i.e. at the+ // two ends. Squashing mid-air reads as a wobble, not a jump.+ val contact = max(0f, 1f - p / 0.2f) + max(0f, (p - 0.8f) / 0.2f)+ sqx = contact * 0.13f+ sqy = contact * -0.17f+ }+ Action.NOD -> actRotX = sin(p * 2f * PI.toFloat()) * 0.26f+ Action.SHAKE -> actRotY = sin(p * 3f * PI.toFloat()) * 0.2f+ Action.POP -> {+ sqx = arch * 0.15f+ sqy = arch * 0.15f+ hop = arch * 0.06f+ }+ Action.RECOIL -> {+ sqx = arch * 0.17f+ sqy = arch * -0.2f+ actRotX = -arch * 0.14f+ }+ // ---- Idle fidgets: body-only, face untouched ----+ Action.STRETCH -> {+ // Tall and thin with the eyes shut for a yawn-adjacent+ // elongation, easing at both ends so it reads as a muscle+ // pulling, not a scale keyframe.+ val ease = sin(p * PI.toFloat())+ sqx = -ease * 0.12f+ sqy = ease * 0.2f+ actRotX = -ease * 0.08f+ }+ Action.LOOK_AROUND -> {+ // Two glances left and right, unhurried — the classic+ // "waiting room look". One and a quarter sine cycles.+ actRotY = sin(p * 2.5f * PI.toFloat()) * 0.3f+ actRotX = sin(p * 1.25f * PI.toFloat()) * 0.05f+ }+ Action.WIGGLE -> {+ // Side-to-side squash dance, three small beats. Pure rotZ+ // so the face stays front-on while the body sways.+ actRotY = sin(p * 3f * PI.toFloat()) * 0.14f+ }+ Action.BOUNCE -> {+ // Two quick happy hops, smaller than the real HOP: this+ // is impatience/excitement in miniature, an idle jig.+ hop = abs(sin(p * 2f * PI.toFloat())) * 0.09f+ val contact = max(0f, 1f - p / 0.15f) + max(0f, (p - 0.85f) / 0.15f)+ sqx = contact * 0.07f+ sqy = contact * -0.09f+ }+ Action.NONE -> {}+ }+ if (p >= 1f) action = Action.NONE+ }++ // ---- Hanging off the bottom edge by two paws ----+ // Sunk half below the screen edge, so only the top of the head shows,+ // with two paws hooked over the line holding it there.+ //+ // The paws are pinned to the edge and the *body* moves under them —+ // that ordering is what makes this read as hanging rather than as a+ // ball with decorations. It slides down for three seconds, hauls itself+ // back up in one, and the haul is the only motion in here that isn't a+ // sine, which is why it's the only one that looks like effort.+ cling += ((if (working) 1f else 0f) - cling) * min(1f, dt * 2.6f)+ var slipY = 0f+ var haul = 0f+ if (cling > 0.01f) {+ slipPhase += dt / SLIP_SECONDS+ if (slipPhase >= 1f) slipPhase -= floor(slipPhase)+ if (slipPhase < SLIP_SAG) {+ // Sliding down. Squared so it starts imperceptibly and picks up+ // — a linear slide reads as being lowered, not as slipping.+ val s = slipPhase / SLIP_SAG+ slipY = -s * s * SLIP_DEPTH+ } else {+ // Scrabbling back up, with a small overshoot at the top.+ val s = (slipPhase - SLIP_SAG) / (1f - SLIP_SAG)+ val eased = 1f - (1f - s) * (1f - s)+ slipY = -(1f - eased) * SLIP_DEPTH + sin(s * PI.toFloat()) * 0.035f+ haul = sin(s * PI.toFloat())+ }+ }+ grip = cling * haul+ // Hanging by the arms stretches you: taller and narrower the further+ // you have slipped, and narrower again on the pull. A ball that sagged+ // without deforming would just be a ball being translated downward.+ val stretch = cling * (-slipY / SLIP_DEPTH)+ val clingSqX = -cling * 0.03f - stretch * 0.05f - haul * 0.04f+ val clingSqY = cling * 0.03f + stretch * 0.07f + haul * 0.05f+ // Swinging. Slow and small, and it hangs off the slip so the swing is+ // widest at the bottom of the sag — that is where a pendulum is.+ val clingLean = cling * sin(t * 0.85f) * (0.06f + stretch * 0.07f)++ // Root transform+ breatheDepth += (want.breath - breatheDepth) * min(1f, dt * 3f)+ breathe = 1f + sin(t * want.pulse) * breatheDepth + open * 0.03f+ scaleX = breathe * (1f + sqx + clingSqX)+ scaleY = breathe * (1f + sqy + clingSqY)+ hopY = hop + slipY * cling+ // A task can run a whole minute with no tool call in it, and a resting+ // face doesn't read as activity — a working ball has to be visibly+ // *doing* something. Amplitude is a fifth of the one-shot NOD's on+ // purpose, so an actual tool call still lands as an event on top of it+ // rather than disappearing into the bob.+ val workBob = cling * sin(t * 3.4f) * 0.04f+ rotY = sin(t * 0.22f) * 0.12f + jx * 0.012f + gaze * 0.09f + actRotY+ rotX = sin(t * 0.17f) * 0.06f + jy * 0.009f - gazeY * 0.05f + actRotX + workBob+ rotZ = tilt + th * sin(t * 0.51f) * 0.09f - clingLean++ // Rings group tumble. Small on purpose: the belt orientation in [RINGS]+ // is chosen to keep the rings edge-on and off the face, and a large+ // tumble swings them right back across it.+ ringsXAngle += dt * (0.21f + busy * 0.42f)+ ringsRotX = sin(ringsXAngle) * 0.22f++ // Per-ring spin + flow+ for (i in 0..3) {+ val r = RINGS[i]+ ringAngle[i] += dt * r.spin * (1f + busy * 2.6f)+ ringRotY[i] = ringAngle[i]+ flowAngle[i] += dt * r.flow * (0.55f + busy * 2.4f)+ flow[i] = flowAngle[i] - floor(flowAngle[i])+ }++ // Scan band+ scanAngle += dt * (0.19f + busy * 0.55f)+ val scanF = scanAngle - floor(scanAngle)+ scanY = -1.35f + scanF * 2.7f+ scan = 0.08f + th * 0.5f + listen * 0.42f + open * 0.12f+ // The rim is one of only two places the mood colour is visible at all+ // (the mouth is the other), so it has to carry what the rings, dust and+ // core used to. `think` had no term here and thinking was therefore+ // almost entirely a hue change; it gets a slow swell of its own now,+ // deliberately off the breathing rate so the two drift against each+ // other instead of pumping in lockstep.+ //+ // Listening throbs rather than just brightening, and faster than any+ // breathing rate — a steady lift was indistinguishable from CALM's own+ // outline at a glance, and waiting for someone to speak is not a+ // resting state.+ val thinkBeat = th * (0.5f + sin(t * 1.9f) * 0.5f)+ val listenBeat = listen * (0.5f + sin(t * 4.2f) * 0.5f)+ rimIntensity = 0.85f + open * 0.7f + listen * 0.45f + listenBeat * 0.95f ++ thinkBeat * 0.55f + sin(t * want.pulse * 2f) * 0.05f++ // Core — small pulse plus a slower "beat" that only exists while thinking+ val beat = th * (0.5f + sin(t * 2.6f) * 0.5f)+ coreScale = 0.4f + open * 0.16f + sin(t * want.pulse * 1.6f) * 0.012f + beat * 0.14f+ coreAlpha = 0.16f + open * 0.5f + beat * 0.3f++ // Dust — remember rate/lag for writeDust+ dustRate = 0.18f + busy * 0.72f+ dustLag = dustRate * 0.36f+ dustAngle += dt * dustRate+ dustHeadAlpha = min(1f, 0.6f + open * 0.2f + busy * 0.35f)+ dustTailAlpha = min(1f, 0.5f + busy * 0.4f)++ // Face elements. Right brow gets skew added; left doesn't — asymmetry+ // reads as "thinking about it", symmetry reads as "having a feeling".+ val browR = brow + skew+ val browL = brow+ // Peering up over the edge it is holding. Eyes lift and the whole face+ // rides a little higher on the head, which is the difference between a+ // ball that happens to be low on screen and one looking up at you from+ // under a ledge. Without it the face aims into the bezel.+ val peek = cling * (0.055f + grip * 0.02f)+ eyeLY = LOOK.eyeY - browL * 0.06f + gazeY * 0.05f + peek+ eyeRY = LOOK.eyeY - browR * 0.06f + gazeY * 0.05f + peek+ eyeLX = -LOOK.eyeX + jx * 0.014f + gaze * 0.075f+ eyeRX = LOOK.eyeX + jx * 0.014f + gaze * 0.075f+ // Eye roll. Two terms, and they are on the same axis on purpose:+ //+ // * the look's own resting tilt, mirrored per side, which is what makes+ // an angry *shape* rather than an angry moment — see [BallLook.eyeTilt],+ // where the whole argument for it lives; and+ // * the mood, ±0.4 rad at full anger. Same sign convention as the old+ // brow bar used, so a look that had one and has a cut now gets the+ // same motion: the resting scowl deepens instead of being replaced.+ //+ // Both are radians here, hence the /DEG on the look's degrees.+ eyeLRotZ = -LOOK.eyeTilt / DEG - browL * 0.4f+ eyeRRotZ = LOOK.eyeTilt / DEG + browR * 0.4f+ // Vertical aperture. The renderer anchors this scale at the *top* of+ // the eye rather than its centre, so closing reads as a lid coming down+ // instead of the whole eye shrinking toward a point.+ eyeYScale = max(0.06f, eye * lid - smile * 0.34f)++ // Snot bubble. The phase is deliberately *not* tied to breathing: the+ // sleep rate is 0.45, which is a 14 s cycle — a bubble that holds still+ // for 14 s is a bubble that isn't moving at all. The rate still scales+ // with [Face.pulse], so "deeper sleep, slower bubble" survives.+ bubble += (want.bubble - bubble) * min(1f, dt * 2.2f)+ bubblePhase += dt * (0.9f + want.pulse * 1.6f)+ bubbleScale = 0.30f + 0.70f * (0.5f + 0.5f * sin(bubblePhase))++ // Thickness tracks how much curve there is to see. At the flat base+ // scale the mouth torus is a hairline nobody can read, so a real smile+ // has to make it taller as well as more curved — and so does the+ // resting one, which is why [BallLook.mouthThick] is in here rather+ // than only riding on `smile`.+ mouthScaleY = 0.16f + open * 1.5f + abs(smile) * 0.5f + LOOK.mouthThick+ mouthY = LOOK.mouthY + smile * 0.06f+ mouthRotZ = PI.toFloat() + smile * 0.9f + skew * 2.2f+ }++ fun writeHeads(buf: FloatArray) {+ val cr = nowColor[0];+ val cg = nowColor[1];+ val cb = nowColor[2]+ for (i in 0 until DUST_COUNT) {+ val p = particles[i]+ val a = p.a0 + dustAngle * p.rate+ val rs = p.r * p.sinPhi+ val y = p.r * p.cosPhi+ val j = i * 6+ buf[j] = cos(a) * rs+ buf[j + 1] = y+ buf[j + 2] = sin(a) * rs+ buf[j + 3] = cr * p.b+ buf[j + 4] = cg * p.b+ buf[j + 5] = cb * p.b+ }+ }++ fun writeTails(buf: FloatArray) {+ val cr = nowColor[0];+ val cg = nowColor[1];+ val cb = nowColor[2]+ for (i in 0 until DUST_COUNT) {+ val p = particles[i]+ val a = p.a0 + dustAngle * p.rate+ val rs = p.r * p.sinPhi+ val y = p.r * p.cosPhi+ for (n in 0 until TRAIL_COUNT) {+ val ak = a - dustLag * p.rate * (n + 1)+ val q = (i * TRAIL_COUNT + n) * 6+ buf[q] = cos(ak) * rs+ buf[q + 1] = y+ buf[q + 2] = sin(ak) * rs+ val fade = p.b * FADE[n]+ buf[q + 3] = cr * fade+ buf[q + 4] = cg * fade+ buf[q + 5] = cb * fade+ }+ }+ }++ private companion object {+ /** Degrees per radian, for the one place a look's angle meets the mood's. */+ const val DEG = 57.29578f++ /** One sag-and-haul cycle, seconds. */+ const val SLIP_SECONDS = 3.6f++ /** Fraction of it spent sliding down; the rest is hauling back up. */+ const val SLIP_SAG = 0.76f++ /**+ * How far the body slips, in world units, before it catches itself.+ * The ball's radius is ~0.885, so this is about a seventh of it —+ * enough to open a visible gap under the paws, not so much that the+ * face disappears below the screen edge.+ */+ const val SLIP_DEPTH = 0.13f+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/Mood.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/Mood.ktnew file mode 100644index 0000000..86f407c--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/Mood.kt@@ -0,0 +1,291 @@+package net.kuafuai.andee.ui.ball++/**+ * The faces the ball can wear.+ *+ * Three groups, and the split is load-bearing rather than cosmetic:+ * - emotions the model picks from (CALM..CONCERNED)+ * - voice-pipeline states (LISTENING, THINKING, SPEAKING), owned by+ * AsrController / TtsController+ * - agent work states (SEARCHING..SLEEPING), owned by the task lifecycle in+ * CommandDispatcher+ *+ * The last group is the one the ball was missing. Voice states say "I am+ * talking to you"; work states say "I am doing the thing you asked for", which+ * is most of what this device actually spends its time on. They arbitrate in+ * [EmotionBallRenderer]: voice wins when it has something to say, work fills+ * the silence, SLEEPING fills the silence after that.+ */+enum class Mood {+ CALM, HAPPY, CURIOUS, TENSE, ANXIOUS, CONCERNED,+ LISTENING, HOLDING, THINKING, SPEAKING,+ SEARCHING, SUCCESS, FAILED, SLEEPING,+}++/**+ * All the per-mood knobs the ball tweens between. See the R3F source's FACE+ * table for the reasoning behind the first eight columns.+ *+ * The trailing four are additions: [eyeArc] swaps the round eye for a `^ ^` arc,+ * [blush] lights the cheeks, [spread] pushes the ring belt out, [breath] sets+ * how far the body swells. All exist because at the size this ball is actually+ * drawn — a corner ball a thumb wide — colour, eye aperture and body scale are+ * the whole signal.+ *+ * [blush] is no longer an occasional flourish. It used to be zero on every row+ * except HAPPY and SUCCESS, which meant the face the user looks at for most of+ * the day had no cheeks at all — and the resting face is the one that has to+ * carry "harmless". Every row now sits somewhere on the same scale, from+ * SLEEPING's drained 0.12 to SUCCESS's flushed 1.0, and the renderer spends it+ * at a fixed renderer-side multiplier (see `BLUSH_ALPHA`).+ *+ * [smile] moved with it: the resting mouth on every non-negative row is now a+ * visible curve rather than the 0.1 hairline that read as a blank slot.+ */+class Face(+ val color: FloatArray, // rgb 0..1+ val eye: Float, // eye open (>1 = wide)+ val brow: Float, // brow pressure (positive = furrowed)+ val smile: Float, // mouth curve (+ smile, − frown)+ val pulse: Float, // breathing rate+ val tilt: Float, // head tilt+ val skew: Float, // brow asymmetry+ val jitter: Float, // shake amplitude+ val eyeArc: Float = 0f, // 0 = round eye, 1 = ^ ^ arc (happy squint)+ val blush: Float = 0f, // cheek glow+ val spread: Float = 1f, // ring belt radius multiplier+ /**+ * Breathing depth, as a fraction of body scale. Used to be a single+ * constant with only [pulse] varying, which meant every mood breathed by+ * the same invisible 2.2% and the rate difference had nothing to modulate.+ */+ val breath: Float = 0.022f,+ /**+ * Resting mouth opening, on top of whatever the talk envelope is doing.+ *+ * Exists for LISTENING. Colour and eye aperture are cheap to confuse with+ * a neighbouring mood; a mouth that is *open* instead of curved is a shape+ * change, and shape survives being drawn a thumb wide.+ */+ val gape: Float = 0f,+ /**+ * 鼻涕泡的浓度,0 = 没有。+ *+ * 只有 SLEEPING 用。它不是装饰性的加法:一颗睡着的球和一颗静止的球在+ * 缩略图尺寸下几乎分不出来(都是深色弧 + 一点腮红),而一个在自己呼吸的+ * 泡是**动**的,动的东西在小尺寸下比颜色更容易被认出来。+ */+ val bubble: Float = 0f,+)++val FACE: Map<Mood, Face> = mapOf(+ Mood.CALM to Face(+ rgb(0x9db4ff), 1f, 0f, 0.26f, 1f, 0f, 0f, 0f,+ blush = 0.38f,+ ),+ Mood.HAPPY to Face(+ rgb(0x5eead4), 0.72f, -0.16f, 0.68f, 1.25f, 0.04f, 0f, 0f,+ eyeArc = 0.9f, blush = 0.85f,+ ),+ Mood.CURIOUS to Face(+ rgb(0xf0abfc), 1.22f, -0.06f, 0.28f, 1.15f, 0.17f, 0.055f, 0f,+ blush = 0.72f,+ ),+ Mood.TENSE to Face(+ rgb(0xfb923c), 0.88f, 0.3f, -0.2f, 1.95f, 0f, 0.02f, 0.55f,+ blush = 0.26f,+ ),+ Mood.ANXIOUS to Face(+ rgb(0xfb7185), 1.26f, 0.42f, -0.44f, 2.5f, 0f, 0.03f, 1f,+ blush = 0.24f,+ ),+ Mood.CONCERNED to Face(+ rgb(0xfbbf24), 1.1f, 0.34f, -0.34f, 1.15f, -0.05f, 0f, 0.12f,+ blush = 0.32f,+ ),++ // Wide-eyed, mouth slightly open, head cocked. This is the one state the+ // user is waiting on before they start talking, so it has to be+ // unmistakable from across a desk — and unmistakable from CALM, which it+ // used to differ from only by a shade of pale blue and a taller eye.+ //+ // The tilt is *negative* where THINKING's is positive: both states cock the+ // head, so the only way for either to stay readable is for them to cock it+ // opposite ways.+ Mood.LISTENING to Face(+ rgb(0x7dd3fc), 1.6f, -0.14f, 0.28f, 1.7f, -0.13f, 0f, 0f,+ blush = 0.46f, breath = 0.055f, gape = 0.12f,+ ),+ // Serious, steady attention: brows lowered, eyes fixed on the decision,+ // mouth closed. This is a waiting face, not a surprised listening face.+ Mood.HOLDING to Face(+ rgb(0xfacc15), 0.92f, 0.48f, -0.18f, 0.72f, 0f, 0f, 0f,+ blush = 0.3f, breath = 0.018f,+ ),+ // The opposite read: eyes squeezed to a slot, brows down and uneven, head+ // cocked. Deliberately slow and deep rather than fast — fast squinting is+ // agitation, and this ball is meant to look like it's working the problem.+ Mood.THINKING to Face(+ rgb(0xc084fc), 0.4f, 0.32f, 0.04f, 1.15f, 0.15f, 0.08f, 0f,+ blush = 0.3f, breath = 0.042f,+ ),+ Mood.SPEAKING to Face(+ rgb(0x9db4ff), 1.05f, 0f, 0.32f, 1.5f, 0f, 0f, 0f,+ blush = 0.44f,+ ),++ // Driving an app. Eyes down-and-narrow with the head cocked reads as+ // "looking at something that isn't you", which is exactly true, and the+ // belt flares to carry the speed the rings already pick up from `busy`.+ Mood.SEARCHING to Face(+ rgb(0x7dd3fc), 0.84f, 0.1f, 0.16f, 1.6f, 0.13f, 0.05f, 0f,+ blush = 0.34f, spread = 1.16f, breath = 0.036f,+ ),+ // Transient, ~2 s, always paired with a POP — see `overlay`.+ Mood.SUCCESS to Face(+ rgb(0x5eead4), 0.5f, -0.22f, 0.86f, 1.3f, 0f, 0f, 0f,+ eyeArc = 1f, blush = 1.0f, spread = 1.1f,+ ),+ Mood.FAILED to Face(+ rgb(0xfb7185), 0.68f, 0.45f, -0.5f, 1.1f, -0.07f, 0.02f, 0.22f,+ blush = 0.42f,+ ),+ // 和 CALM 同一个颜色 —— 这一条是反过来的决定。+ //+ // 这里原本是全表唯一一行去饱和的(连同脸颊一起褪色),理由是"睡着的球+ // 应该一眼看出是熄了"。但实际做出来的是**换了一种材质**:奶油球睡着时身体+ // 转灰、眼睛和嘴跟着转灰,于是"睡着"看起来像被关掉了,而不是它累了。+ // 睡是一种形状 —— 那条 `^ ^` 眼睛加上鼻子下面那颗水滴 —— 颜色不必替它+ // 承担这件事,而一旦它承担了,这颗球在一天里最安静的那几个小时里就不是+ // 它自己了。+ Mood.SLEEPING to Face(+ rgb(0x9db4ff), 0.1f, 0.06f, 0.16f, 0.45f, 0.11f, 0f, 0f,+ eyeArc = 0.85f, spread = 0.9f, blush = 0.16f, bubble = 1f,+ ),+)++/**+ * One-shot body gestures, layered on top of whatever [Face] is current.+ *+ * Separate from mood because they are events, not states: a mood is what the+ * ball *is*, an action is something that just *happened* to it. The ball can+ * nod while staying calm, and a task finishing is a single pop rather than a+ * new resting expression.+ */+enum class Action {+ NONE, HOP, NOD, SHAKE, POP, RECOIL,+ STRETCH, LOOK_AROUND, WIGGLE, BOUNCE,+}++/**+ * Seconds each [Action] takes. Indexed by ordinal — keep in sync with the+ * enum above. The idle fidgets run longer than the event gestures: a stretch+ * or a look-around is a "waking up my body" motion, and snapping through one+ * in half a second reads as a glitch rather than a stretch.+ */+val ACTION_SECONDS = floatArrayOf(+ 0f, // NONE+ 0.52f, // HOP+ 0.42f, // NOD+ 0.5f, // SHAKE+ 0.46f, // POP+ 0.55f, // RECOIL+ 1.6f, // STRETCH+ 1.9f, // LOOK_AROUND+ 1.1f, // WIGGLE+ 0.9f, // BOUNCE+)++/**+ * The idle fidget pool, with weights — the renderer picks one at random from+ * here (not from every [Action]) when the idle fidget timer fires. Body-only+ * on purpose: the face stays CALM throughout, so nothing in here can be+ * mistaken for a response, a failure or a speaker being active.+ */+val IDLE_ACTIONS: List<Pair<Action, Float>> = listOf(+ Action.STRETCH to 1.0f,+ Action.LOOK_AROUND to 1.0f,+ Action.WIGGLE to 1.0f,+ Action.BOUNCE to 0.7f,+ Action.HOP to 0.5f,+)++/**+ * Orbital ring config. Four rings, each with own radius, tilt, spin, flow speed.+ *+ * A [Torus] circles the XY plane about Z, and the camera looks down −Z — so+ * `rotX` near 0 leaves a ring face-on to the viewer, which draws a full ellipse+ * straight across the face. Every [rotX] here is therefore clustered near+ * ±π/2, where the ring is edge-on and reads as a belt around the body rather+ * than a hoop in front of it. Two of these used to sit at 0.4 and −1.1 and cut+ * through the eyes; the ball read as a planet instead of a character.+ *+ * Deviation from ±π/2 is the whole visual interest, so it's kept — just small.+ */+class RingSpec(+ val r: Float,+ val tube: Float,+ val rotX: Float, val rotY: Float, val rotZ: Float, // local orientation (radians)+ val spin: Float, // Y-axis spin rate (+/- so they don't tumble as a rigid body)+ val flow: Float, // light-flow speed around the ring+ val base: Float, // ambient alpha+)++val RINGS = arrayOf(+ RingSpec(1.10f, 0.012f, 1.46f, 0.2f, 0.2f, 0.5f, 0.42f, 0.18f),+ RingSpec(1.22f, 0.009f, 1.72f, 0f, -0.16f, -0.33f, -0.29f, 0.2f),+ RingSpec(1.33f, 0.0065f, -1.38f, 0.6f, 0.3f, 0.22f, 0.36f, 0.15f),+ RingSpec(1.44f, 0.005f, 1.9f, -0.5f, -0.26f, -0.14f, -0.19f, 0.1f),+)++const val DUST_COUNT = 96+const val TRAIL_COUNT = 2+val FADE = floatArrayOf(0.42f, 0.16f)++/**+ * How much of the ball sinks below the screen edge while it hangs off it.+ *+ * Two places have to agree on this and neither can derive it from the other:+ * `FloatingWindowUi` moves the window down by exactly this much, and+ * [EmotionBallRenderer] draws the paws on the line that creates. A half is the+ * value that puts that line through the ball's equator, which is where the+ * paws are — change this and they stop sitting on the edge.+ */+const val CLING_HIDDEN = 0.5f++/**+ * Radius of the *rendered* sphere as a fraction of the ball surface's shorter+ * side — what anyone placing something against the ball's outline needs.+ *+ * Not a taste value: it falls straight out of the camera. A 42° vertical fov at+ * z = 4.9 makes one world unit `1 / (2 tan 21°) / 4.9` of the viewport height,+ * and the ball is drawn at `RIM` = 0.9 of those. Measured on device at 546 px+ * square: 270 px across, i.e. 0.247 including the rim's glow.+ *+ * The 0.32 that used to be written by hand at the call sites is the *hit-test*+ * radius, deliberately generous so grazing taps count, and using it as the+ * drawn radius put CardUi's signboard a good 13 dp above the head it was+ * supposed to be resting on.+ *+ * The ball's *face* geometry — eye size and position, mouth, the z the face+ * plane sits at — lives in [BallLook], next to the material it has to agree+ * with. This file stays about moods.+ */+/**+ * 超过这个值,屏幕上那只眼睛就是 `^ ^` 这条弧,圆眼不是主角了。+ *+ * 两处共用一个数、一个意思:+ * * 渲染器还要不要往眼睛里画瞳孔和高光;+ * * 状态机还要不要眨眼 —— 眼皮是给睁着的眼睛合上的,当眼睛本身就是一条+ * "已经合上"的弧时,眨一下只会把弧抽动一下。睡着的球每三秒抖一下就是+ * 这么来的。+ */+const val ARC_EYE_AT = 0.5f++const val SPHERE_RADIUS_FRACTION = 0.24f++private fun rgb(hex: Int): FloatArray = floatArrayOf(+ ((hex shr 16) and 0xff) / 255f,+ ((hex shr 8) and 0xff) / 255f,+ (hex and 0xff) / 255f,+)diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Capsule.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Capsule.ktnew file mode 100644index 0000000..2f01a17--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Capsule.kt@@ -0,0 +1,94 @@+package net.kuafuai.andee.ui.ball.gl++import kotlin.math.PI+import kotlin.math.cos+import kotlin.math.sin++/**+ * Capsule: two hemispherical caps of [radius] separated by a cylinder of+ * [cylinderHeight]. Axis is Y. Matches Three.js CapsuleGeometry.+ *+ * Built as two rings-of-vertices strips: top hemisphere (phi 0..π/2 shifted up+ * by half the cylinder height) then bottom hemisphere (phi π/2..π shifted+ * down). The two equator rings sit at exactly ±half — connecting them via+ * quads gives the cylinder side implicitly, no separate cylinder pass needed.+ */+object Capsule {+ fun build(+ radius: Float,+ cylinderHeight: Float,+ capSegs: Int = 4,+ radialSegs: Int = 12,+ ): Mesh {+ val half = cylinderHeight / 2f+ val totalRings = 2 * (capSegs + 1)+ val vertCount = totalRings * (radialSegs + 1)+ val positions = FloatArray(vertCount * 3)+ val normals = FloatArray(vertCount * 3)+ val uvs = FloatArray(vertCount * 2)+ var pi = 0;+ var ui = 0+ var ringIdx = 0++ // Top hemisphere: phi = 0 → π/2, shifted up+ for (i in 0..capSegs) {+ val phi = (i.toFloat() / capSegs) * (PI.toFloat() / 2f)+ val sPhi = sin(phi);+ val cPhi = cos(phi)+ for (j in 0..radialSegs) {+ val theta = (j.toFloat() / radialSegs) * 2f * PI.toFloat()+ val nx = -cos(theta) * sPhi+ val ny = cPhi+ val nz = sin(theta) * sPhi+ positions[pi] = radius * nx+ positions[pi + 1] = radius * ny + half+ positions[pi + 2] = radius * nz+ normals[pi] = nx+ normals[pi + 1] = ny+ normals[pi + 2] = nz+ uvs[ui] = j.toFloat() / radialSegs+ uvs[ui + 1] = 1f - ringIdx.toFloat() / (totalRings - 1)+ pi += 3; ui += 2+ }+ ringIdx+++ }++ // Bottom hemisphere: phi = π/2 → π, shifted down+ for (i in 0..capSegs) {+ val phi = (PI.toFloat() / 2f) + (i.toFloat() / capSegs) * (PI.toFloat() / 2f)+ val sPhi = sin(phi);+ val cPhi = cos(phi)+ for (j in 0..radialSegs) {+ val theta = (j.toFloat() / radialSegs) * 2f * PI.toFloat()+ val nx = -cos(theta) * sPhi+ val ny = cPhi+ val nz = sin(theta) * sPhi+ positions[pi] = radius * nx+ positions[pi + 1] = radius * ny - half+ positions[pi + 2] = radius * nz+ normals[pi] = nx+ normals[pi + 1] = ny+ normals[pi + 2] = nz+ uvs[ui] = j.toFloat() / radialSegs+ uvs[ui + 1] = 1f - ringIdx.toFloat() / (totalRings - 1)+ pi += 3; ui += 2+ }+ ringIdx+++ }++ val stride = radialSegs + 1+ val idx = ShortArray((totalRings - 1) * radialSegs * 6)+ var w = 0+ for (r in 0 until totalRings - 1) {+ for (s in 0 until radialSegs) {+ val a = r * stride + s+ val b = r * stride + s + 1+ val c = (r + 1) * stride + s + 1+ val d = (r + 1) * stride + s+ idx[w++] = a.toShort(); idx[w++] = b.toShort(); idx[w++] = c.toShort()+ idx[w++] = a.toShort(); idx[w++] = c.toShort(); idx[w++] = d.toShort()+ }+ }+ return Mesh(positions, normals, uvs, idx)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Ear.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Ear.ktnew file mode 100644index 0000000..e60ec7c--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Ear.kt@@ -0,0 +1,106 @@+package net.kuafuai.andee.ui.ball.gl++import kotlin.math.cos+import kotlin.math.max+import kotlin.math.sin+import kotlin.math.sqrt++/**+ * A floppy leaf-shaped flap — a pig's ear.+ *+ * Built like [Horn] as a stack of rings walking a curved axis, but three things+ * make it an ear rather than a horn:+ *+ * - The cross-section is **elliptical**, not round: wide in the bend plane,+ * thin in Z. An ear is a flap, and a flap is a flattened shape.+ * - The profile **rounds off** at the tip (a quarter-ellipse) instead of+ * coming to a point. A pig's ear is rounded; a pointed tip reads as a horn.+ * - It is short and droops (the caller rotates it), where a horn is long and+ * points.+ *+ * Grown in the +Y/+X plane like [Horn], so the same mirror-for-the-other-side+ * trick applies. Normals are real, so the lit shell shader shades the flap+ * instead of drawing a hole in the ball.+ */+object Ear {+ /**+ * @param width half-width at the base, in the bend plane.+ * @param height length along the (curved) axis.+ * @param thickness half-depth in Z. Small relative to [width] — that flatness+ * is what makes it a flap.+ * @param bend total rotation of the axis from base to tip, radians, toward +X.+ * @param segs rings along the length.+ * @param radialSegs vertices around each ring.+ */+ fun build(+ width: Float,+ height: Float,+ thickness: Float,+ bend: Float = 0f,+ segs: Int = 6,+ radialSegs: Int = 10,+ ): Mesh {+ val rings = segs + 1+ val vertCount = rings * (radialSegs + 1)+ val positions = FloatArray(vertCount * 3)+ val normals = FloatArray(vertCount * 3)+ val uvs = FloatArray(vertCount * 2)++ val step = height / segs+ var cx = 0f+ var cy = 0f+ var pi = 0+ var ui = 0+ for (i in 0 until rings) {+ val t = i.toFloat() / segs+ // Quarter-ellipse profile, floored so the tip is a small rounded nub+ // rather than a point — and so the per-ring normal never divides by+ // zero. A pointed tip is a horn, not a pig's ear.+ val k = max(sqrt(1f - t * t), 0.08f)+ val w = width * k+ val th = thickness * k+ val ang = bend * t+ val ax = sin(ang)+ val ay = cos(ang)+ // u ⊥ a, in the XY plane; the third direction is simply Z.+ val ux = ay+ val uy = -ax+ for (j in 0..radialSegs) {+ val tht = j.toFloat() / radialSegs * 2f * PI_F+ val ct = cos(tht)+ val st = sin(tht)+ positions[pi++] = cx + w * ct * ux+ positions[pi++] = cy + w * ct * uy+ positions[pi++] = th * st+ // Ellipse (w, th) surface normal ∝ (ct/w, st/th) in the (u, Z)+ // frame, rotated into world: the u-component rides on (ux, uy).+ val nu = ct / w+ val nz = st / th+ val inv = 1f / sqrt(nu * nu + nz * nz)+ normals[pi - 3] = nu * inv * ux+ normals[pi - 2] = nu * inv * uy+ normals[pi - 1] = nz * inv+ uvs[ui++] = j.toFloat() / radialSegs+ uvs[ui++] = t+ }+ cx += ax * step+ cy += ay * step+ }++ val indices = ShortArray(segs * radialSegs * 6)+ var k = 0+ for (i in 0 until segs) {+ for (j in 0 until radialSegs) {+ val a = (i * (radialSegs + 1) + j).toShort()+ val b = (i * (radialSegs + 1) + j + 1).toShort()+ val c = ((i + 1) * (radialSegs + 1) + j).toShort()+ val d = ((i + 1) * (radialSegs + 1) + j + 1).toShort()+ indices[k++] = a; indices[k++] = c; indices[k++] = b+ indices[k++] = b; indices[k++] = c; indices[k++] = d+ }+ }+ return Mesh(positions, normals, uvs, indices)+ }++ private const val PI_F = 3.1415927f+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Eye.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Eye.ktnew file mode 100644index 0000000..e30c63b--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Eye.kt@@ -0,0 +1,211 @@+package net.kuafuai.andee.ui.ball.gl++import kotlin.math.PI+import kotlin.math.acos+import kotlin.math.cos+import kotlin.math.sin++/**+ * An eye whose **top is cut off flat** — a [Capsule] with the top sliced away.+ *+ * The point is that the flat edge *is* the brow. On this ball a white eye on a+ * dark body already carries the maximum contrast the screen can show, so an+ * angry eye does not need a second shape drawn beside it at all: cut the eye's+ * own top at an angle and the top edge reads as a brow that has been pulled+ * down over the eye. That is the classic drawn-villain eye, and it costs one+ * mesh instead of one mesh plus one mark plus a colour argument about whether+ * that mark can be seen.+ *+ * It also removes a trap. Anything drawn *over* the eye to cut it has to be+ * invisible against the body at the same time as it is visible against the eye+ * — two contradictory jobs, because the body is lit and a flat draw is not (see+ * [BallLook.browShade], whose first value was wrong by exactly that). A cut in+ * the geometry has no colour to get wrong.+ *+ * Built like [Capsule] — rings along Y, bottom pole first, with a cylindrical+ * band in the middle — except that the sweep stops at the cut and the last ring+ * is closed by a flat disc instead of a cap.+ *+ * ## The angle is not in here+ *+ * A flat cut horizontally across a round eye is a *sleepy* eye. What makes it+ * read as a brow is that the line is **slanted**, and that comes from the+ * caller rolling the whole eye (`BallLook.eyeTilt`, plus the mood's own+ * `brow`). Rotating the mesh in the draw call rather than shearing it here+ * keeps this generator a shape and the look table a pose, and it means the cut+ * angle survives being animated.+ *+ * @param radius half-width of the eye.+ * @param cylinderHeight the straight section, as in [Capsule].+ * @param topCut how much of the eye's upper half is removed, as a fraction of+ * the half-height: 0 = the untouched capsule, 0.5 = the top edge sits halfway+ * between the centre and the natural top. Clamped to `0..`[MAX_TOP_CUT].+ * @param capSegs rings across each hemisphere.+ * @param radialSegs vertices around each ring.+ */+object Eye {+ /**+ * The most of the eye's upper half [build] will remove.+ *+ * This is a **design** bound, not a geometric one — at 0.85 the surviving+ * aperture is still a valid mesh (the cut sits at 15% of the half-height,+ * above the centre line). Past it the flat edge drops below the eye's own+ * centre and the shape stops reading as a lidded eye at all.+ *+ * It is deliberately *not* the bound that matters most in practice. The+ * tighter one is whatever is drawn *inside* the eye: the cut only slices+ * the eye's own geometry, and a pupil is a separate disc drawn in front of+ * it, so a cut below the pupil's top leaves a dark blob standing up out of+ * a flat lid. That ceiling depends on [BallLook.pupilR], so it lives on+ * [BallLook.eyeCut] where the pupil is in scope.+ */+ const val MAX_TOP_CUT = 0.85f++ fun build(+ radius: Float,+ cylinderHeight: Float,+ topCut: Float,+ capSegs: Int = 4,+ radialSegs: Int = 12,+ ): Mesh {+ val half = cylinderHeight / 2f+ val halfHeight = radius + half+ val cut = topCut.coerceIn(0f, MAX_TOP_CUT)+ val yCut = halfHeight * (1f - cut)++ // The rings, as (y, r), bottom pole → cut. Collected into lists rather+ // than pre-counted into arrays: the ring count depends on where the cut+ // lands (on the top hemisphere or on the cylindrical band), and getting+ // that count wrong is the silent typed-array overflow this codebase has+ // been bitten by before — a FloatArray that is one ring short does not+ // throw, it drops the vertices and you get a mesh with a hole in it.+ // The lists cost a few hundred floats at build time and cannot be off+ // by one.+ val ys = ArrayList<Float>(2 * capSegs + 3)+ val rs = ArrayList<Float>(2 * capSegs + 3)++ // Bottom hemisphere: phi π → π/2, i.e. the pole first, ending exactly on+ // the lower equator.+ for (i in 0..capSegs) {+ val phi = PI.toFloat() - (i.toFloat() / capSegs) * (PI.toFloat() / 2f)+ ys.add(radius * cos(phi) - half)+ rs.add(radius * sin(phi))+ }++ if (yCut > half) {+ // Cut through the dome. The lower equator is the ring just added, so+ // add the upper one and let the quads between them be the cylinder+ // wall — exactly how [Capsule] does it — then sweep the dome up to+ // the cut.+ ys.add(half)+ rs.add(radius)+ val phiCut = acos(((yCut - half) / radius).coerceIn(-1f, 1f))+ for (i in 1..capSegs) {+ val phi = (PI.toFloat() / 2f) -+ (i.toFloat() / capSegs) * (PI.toFloat() / 2f - phiCut)+ ys.add(radius * cos(phi) + half)+ rs.add(radius * sin(phi))+ }+ } else {+ // Cut at or below the upper equator: the band runs straight from the+ // lower equator to the cut and there is no dome left to sweep.+ //+ // The upper equator ring is *skipped*, not added and then followed+ // by the cut. Adding it would put a ring above the one after it,+ // which folds the strip back on itself — and a folded strip is not+ // an error, it is a mesh that renders inside out. This branch is+ // reachable at real values: [IMP]'s eye has a half-height of 0.1755+ // against half 0.0275, so any cut above 0.843 lands here.+ ys.add(yCut)+ rs.add(radius)+ }++ val ringCount = ys.size+ val stride = radialSegs + 1+ // The rings, plus the cut disc's own ring, plus its centre vertex.+ val vertCount = ringCount * stride + stride + 1+ val positions = FloatArray(vertCount * 3)+ val normals = FloatArray(vertCount * 3)+ val uvs = FloatArray(vertCount * 2)++ var p = 0+ var u = 0+ for (i in 0 until ringCount) {+ val y = ys[i]+ val r = rs[i]+ // Which sphere this ring belongs to decides the normal. The equator+ // rings land on the boundaries and both reduce to the radial+ // direction, so the formula is continuous across them.+ val cy = if (y <= -half) -half else if (y >= half) half else 0f+ val onCylinder = y > -half && y < half+ for (j in 0..radialSegs) {+ val theta = (j.toFloat() / radialSegs) * 2f * PI.toFloat()+ val nx = -cos(theta)+ val nz = sin(theta)+ positions[p] = r * nx+ positions[p + 1] = y+ positions[p + 2] = r * nz+ if (onCylinder) {+ normals[p] = nx+ normals[p + 1] = 0f+ normals[p + 2] = nz+ } else {+ normals[p] = nx * r / radius+ normals[p + 1] = (y - cy) / radius+ normals[p + 2] = nz * r / radius+ }+ uvs[u] = j.toFloat() / radialSegs+ uvs[u + 1] = 1f - i.toFloat() / (ringCount - 1).coerceAtLeast(1)+ p += 3; u += 2+ }+ }++ // The cut face, on a ring of its own rather than reusing the sweep's+ // last one. The last ring carries the *dome's* normal, so a disc that+ // inherited it would be a flat lid shaded as though it were still+ // curved — a bright smudge across the top of an eye. The eye is drawn+ // with the flat shader today and never reads a normal, which is exactly+ // why this is worth getting right now rather than after something+ // switches it to the lit one.+ val capRing = ringCount * stride+ val capCenter = capRing + stride+ val rCut = rs[ringCount - 1]+ positions[capCenter * 3] = 0f+ positions[capCenter * 3 + 1] = yCut+ positions[capCenter * 3 + 2] = 0f+ normals[capCenter * 3 + 1] = 1f+ uvs[capCenter * 2] = 0.5f+ uvs[capCenter * 2 + 1] = 1f+ for (j in 0..radialSegs) {+ val theta = (j.toFloat() / radialSegs) * 2f * PI.toFloat()+ val nx = -cos(theta)+ val nz = sin(theta)+ val v = capRing + j+ positions[v * 3] = rCut * nx+ positions[v * 3 + 1] = yCut+ positions[v * 3 + 2] = rCut * nz+ normals[v * 3 + 1] = 1f+ uvs[v * 2] = j.toFloat() / radialSegs+ uvs[v * 2 + 1] = 0f+ }++ val idx = ShortArray((ringCount - 1) * radialSegs * 6 + radialSegs * 3)+ var w = 0+ for (r in 0 until ringCount - 1) {+ for (s in 0 until radialSegs) {+ val a = r * stride + s+ val b = r * stride + s + 1+ val c = (r + 1) * stride + s + 1+ val d = (r + 1) * stride + s+ idx[w++] = a.toShort(); idx[w++] = b.toShort(); idx[w++] = c.toShort()+ idx[w++] = a.toShort(); idx[w++] = c.toShort(); idx[w++] = d.toShort()+ }+ }+ for (s in 0 until radialSegs) {+ idx[w++] = capCenter.toShort()+ idx[w++] = (capRing + s).toShort()+ idx[w++] = (capRing + s + 1).toShort()+ }+ return Mesh(positions, normals, uvs, idx)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Horn.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Horn.ktnew file mode 100644index 0000000..2f734f7--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Horn.kt@@ -0,0 +1,114 @@+package net.kuafuai.andee.ui.ball.gl++import kotlin.math.cos+import kotlin.math.sin+import kotlin.math.sqrt++/**+ * A tapered spike that may curve as it rises: a horn, or the barb on a tail.+ *+ * Built as a stack of rings walking up a discrete arc. Each step advances along+ * the local axis and then rotates that axis by `bend / segs`, so the spike+ * sweeps through a total of [bend] radians from base to tip. `bend = 0` is a+ * plain cone.+ *+ * The curve is why this is not `Capsule` with a scale on it, and why it is not+ * one either: a horn is the one part of a demon that is read *by its shape*, and+ * a straight cone reads as a party hat. Everything else on this ball can be+ * expressed as a scaled sphere, capsule or torus; this cannot.+ *+ * Grown in the +Y/+X plane (up, leaning toward +X). The caller mirrors it for+ * the other side with a negative X scale — which flips winding, and nothing here+ * depends on winding because the ball never enables face culling.+ *+ * **Normals are real, not radial.** The spike is drawn with the lit shell shader+ * rather than the flat solid one (a flat-shaded horn on a dark ball is a+ * silhouette with a hole in it where it overlaps the body), so the taper has to+ * be in the normal or the whole spike shades as a cylinder. For a cone of base+ * radius R and height H the outward normal is `normalize(H·r̂ + R·â)` — tilted+ * toward the tip by exactly the taper angle.+ */+object Horn {+ /**+ * @param radius base radius; the tip is a point.+ * @param height length along the (curved) axis.+ * @param bend total rotation of the axis from base to tip, radians, toward +X.+ * @param segs rings along the length. 6 is plenty at the size this is drawn.+ * @param radialSegs vertices around each ring.+ */+ fun build(+ radius: Float,+ height: Float,+ bend: Float = 0f,+ segs: Int = 6,+ radialSegs: Int = 10,+ ): Mesh {+ val rings = segs + 1+ val vertCount = rings * (radialSegs + 1)+ val positions = FloatArray(vertCount * 3)+ val normals = FloatArray(vertCount * 3)+ val uvs = FloatArray(vertCount * 2)++ val step = height / segs+ // Slope of the profile, constant for a linear taper. This is the (H, R)+ // pair the normal is mixed from — see the class KDoc.+ val nAxis = radius / sqrt(height * height + radius * radius)+ val nRad = height / sqrt(height * height + radius * radius)++ var cx = 0f+ var cy = 0f+ var pi = 0+ var ni = 0+ var ui = 0+ for (i in 0 until rings) {+ val t = i.toFloat() / segs+ val r = radius * (1f - t)+ // The axis at this ring, and the two directions spanning the ring+ // plane: u in the bend plane, v out of it.+ val ang = bend * t+ val ax = sin(ang)+ val ay = cos(ang)+ // u ⊥ a, in the XY plane; v is simply Z.+ val ux = ay+ val uy = -ax+ for (j in 0..radialSegs) {+ val th = j.toFloat() / radialSegs * 2f * PI_F+ val ct = cos(th)+ val st = sin(th)+ // Radial direction: ct·u + st·v+ val rx = ct * ux+ val ry = ct * uy+ val rz = st+ positions[pi++] = cx + r * rx+ positions[pi++] = cy + r * ry+ positions[pi++] = r * rz+ normals[ni++] = nRad * rx + nAxis * ax+ normals[ni++] = nRad * ry + nAxis * ay+ normals[ni++] = nRad * rz+ uvs[ui++] = j.toFloat() / radialSegs+ uvs[ui++] = t+ }+ cx += ax * step+ cy += ay * step+ }++ // Quads all the way up, including into the zero-radius tip ring. Those+ // degenerate into zero-area triangles, which costs nothing and saves a+ // separate tip fan.+ val indices = ShortArray(segs * radialSegs * 6)+ var k = 0+ for (i in 0 until segs) {+ for (j in 0 until radialSegs) {+ val a = (i * (radialSegs + 1) + j).toShort()+ val b = (i * (radialSegs + 1) + j + 1).toShort()+ val c = ((i + 1) * (radialSegs + 1) + j).toShort()+ val d = ((i + 1) * (radialSegs + 1) + j + 1).toShort()+ indices[k++] = a; indices[k++] = c; indices[k++] = b+ indices[k++] = b; indices[k++] = c; indices[k++] = d+ }+ }+ return Mesh(positions, normals, uvs, indices)+ }++ private const val PI_F = 3.1415927f+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/IcoSphere.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/IcoSphere.ktnew file mode 100644index 0000000..0466a17--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/IcoSphere.kt@@ -0,0 +1,86 @@+package net.kuafuai.andee.ui.ball.gl++import kotlin.math.sqrt++/**+ * Icosahedron subdivided [subdivisions] times, unit radius.+ * Matches Three.js icosahedronGeometry(1, N).+ *+ * subdiv=1 → 42 verts, 80 tris (used for wireframe overlay)+ * subdiv=3 → 642 verts, 1280 tris (used for core)+ * subdiv=5 → 10242 verts, 20480 tris (used for shell)+ */+object IcoSphere {+ fun build(subdivisions: Int): Mesh {+ val t = (1f + sqrt(5f)) / 2f+ val basePos = floatArrayOf(+ -1f, t, 0f, 1f, t, 0f, -1f, -t, 0f, 1f, -t, 0f,+ 0f, -1f, t, 0f, 1f, t, 0f, -1f, -t, 0f, 1f, -t,+ t, 0f, -1f, t, 0f, 1f, -t, 0f, -1f, -t, 0f, 1f,+ )+ val positions = ArrayList<Float>(basePos.size * 4)+ for (i in 0 until 12) {+ val x = basePos[i * 3];+ val y = basePos[i * 3 + 1];+ val z = basePos[i * 3 + 2]+ val inv = 1f / sqrt(x * x + y * y + z * z)+ positions.add(x * inv); positions.add(y * inv); positions.add(z * inv)+ }+ var faces = intArrayOf(+ 0, 11, 5, 0, 5, 1, 0, 1, 7, 0, 7, 10, 0, 10, 11,+ 1, 5, 9, 5, 11, 4, 11, 10, 2, 10, 7, 6, 7, 1, 8,+ 3, 9, 4, 3, 4, 2, 3, 2, 6, 3, 6, 8, 3, 8, 9,+ 4, 9, 5, 2, 4, 11, 6, 2, 10, 8, 6, 7, 9, 8, 1,+ )+ val cache = HashMap<Long, Int>()++ fun midpoint(a: Int, b: Int): Int {+ val lo = if (a < b) a else b+ val hi = if (a < b) b else a+ val key = (lo.toLong() shl 32) or hi.toLong()+ cache[key]?.let { return it }+ val ax = positions[a * 3];+ val ay = positions[a * 3 + 1];+ val az = positions[a * 3 + 2]+ val bx = positions[b * 3];+ val by = positions[b * 3 + 1];+ val bz = positions[b * 3 + 2]+ var mx = (ax + bx) * 0.5f+ var my = (ay + by) * 0.5f+ var mz = (az + bz) * 0.5f+ val inv = 1f / sqrt(mx * mx + my * my + mz * mz)+ mx *= inv; my *= inv; mz *= inv+ val idx = positions.size / 3+ positions.add(mx); positions.add(my); positions.add(mz)+ cache[key] = idx+ return idx+ }++ repeat(subdivisions) {+ val next = IntArray(faces.size * 4)+ var w = 0+ var r = 0+ while (r < faces.size) {+ val a = faces[r];+ val b = faces[r + 1];+ val c = faces[r + 2]+ val ab = midpoint(a, b)+ val bc = midpoint(b, c)+ val ca = midpoint(c, a)+ next[w++] = a; next[w++] = ab; next[w++] = ca+ next[w++] = b; next[w++] = bc; next[w++] = ab+ next[w++] = c; next[w++] = ca; next[w++] = bc+ next[w++] = ab; next[w++] = bc; next[w++] = ca+ r += 3+ }+ faces = next+ cache.clear()+ }++ val posArr = FloatArray(positions.size) { positions[it] }+ val normalsArr = posArr.copyOf() // unit sphere: normal == position+ val uvArr = FloatArray(posArr.size / 3 * 2) // no UVs used for icosphere+ val idxArr = ShortArray(faces.size) { faces[it].toShort() }+ return Mesh(posArr, normalsArr, uvArr, idxArr)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Mesh.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Mesh.ktnew file mode 100644index 0000000..9e5a59a--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Mesh.kt@@ -0,0 +1,88 @@+package net.kuafuai.andee.ui.ball.gl++import android.opengl.GLES20+import java.nio.ByteBuffer+import java.nio.ByteOrder++/**+ * Indexed triangle mesh, interleaved (pos, normal, uv) = 8 floats/vertex.+ *+ * SHORT indices — enough for anything we generate (icosphere subdiv 5 = 10242 verts).+ */+class Mesh(+ positions: FloatArray,+ normals: FloatArray,+ uvs: FloatArray,+ indices: ShortArray,+) {+ private val vboVertices: Int+ private val vboIndices: Int+ val indexCount = indices.size++ init {+ val n = positions.size / 3+ require(normals.size == positions.size) { "normals length mismatch" }+ require(uvs.size == n * 2) { "uvs length mismatch" }+ val interleaved = FloatArray(n * STRIDE_FLOATS)+ for (i in 0 until n) {+ val j = i * STRIDE_FLOATS+ interleaved[j] = positions[i * 3]+ interleaved[j + 1] = positions[i * 3 + 1]+ interleaved[j + 2] = positions[i * 3 + 2]+ interleaved[j + 3] = normals[i * 3]+ interleaved[j + 4] = normals[i * 3 + 1]+ interleaved[j + 5] = normals[i * 3 + 2]+ interleaved[j + 6] = uvs[i * 2]+ interleaved[j + 7] = uvs[i * 2 + 1]+ }+ val vb = ByteBuffer.allocateDirect(interleaved.size * 4)+ .order(ByteOrder.nativeOrder()).asFloatBuffer()+ .apply { put(interleaved); position(0) }+ val ib = ByteBuffer.allocateDirect(indices.size * 2)+ .order(ByteOrder.nativeOrder()).asShortBuffer()+ .apply { put(indices); position(0) }+ val out = IntArray(2)+ GLES20.glGenBuffers(2, out, 0)+ vboVertices = out[0]+ vboIndices = out[1]+ GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, vboVertices)+ GLES20.glBufferData(GLES20.GL_ARRAY_BUFFER, interleaved.size * 4, vb, GLES20.GL_STATIC_DRAW)+ GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, vboIndices)+ GLES20.glBufferData(+ GLES20.GL_ELEMENT_ARRAY_BUFFER,+ indices.size * 2,+ ib,+ GLES20.GL_STATIC_DRAW+ )+ }++ /**+ * Binds VBOs and enables the given attribute pointers. Pass -1 for+ * attributes the current shader doesn't consume.+ */+ fun bind(aPos: Int, aNormal: Int = -1, aUV: Int = -1) {+ GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, vboVertices)+ GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, vboIndices)+ val stride = STRIDE_FLOATS * 4+ if (aPos >= 0) {+ GLES20.glEnableVertexAttribArray(aPos)+ GLES20.glVertexAttribPointer(aPos, 3, GLES20.GL_FLOAT, false, stride, 0)+ }+ if (aNormal >= 0) {+ GLES20.glEnableVertexAttribArray(aNormal)+ GLES20.glVertexAttribPointer(aNormal, 3, GLES20.GL_FLOAT, false, stride, 3 * 4)+ }+ if (aUV >= 0) {+ GLES20.glEnableVertexAttribArray(aUV)+ GLES20.glVertexAttribPointer(aUV, 2, GLES20.GL_FLOAT, false, stride, 6 * 4)+ }+ }++ fun draw() {+ GLES20.glDrawElements(GLES20.GL_TRIANGLES, indexCount, GLES20.GL_UNSIGNED_SHORT, 0)+ }++ companion object {+ private const val STRIDE_FLOATS = 8 // px py pz + nx ny nz + u v+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/PointsBuffer.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/PointsBuffer.ktnew file mode 100644index 0000000..776117c--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/PointsBuffer.kt@@ -0,0 +1,68 @@+package net.kuafuai.andee.ui.ball.gl++import android.opengl.GLES20+import java.nio.ByteBuffer+import java.nio.ByteOrder+import java.nio.FloatBuffer++/**+ * Dynamic point cloud: interleaved (pos 3, color 3) = 6 floats per point.+ * Positions are computed on the CPU each frame (dust particles orbit with+ * per-particle rates), then uploaded to the GPU as DYNAMIC_DRAW.+ *+ * Editing pattern:+ * 1. state.writeHeads(pointsBuffer.data) — fill CPU array+ * 2. pointsBuffer.upload() — push to GPU+ * 3. pointsBuffer.bind(aPos, aColor); .draw()+ */+class PointsBuffer(val capacity: Int) {++ val data = FloatArray(capacity * STRIDE_FLOATS)++ private val vbo: Int+ private val fbuf: FloatBuffer++ init {+ fbuf = ByteBuffer.allocateDirect(capacity * STRIDE_FLOATS * 4)+ .order(ByteOrder.nativeOrder()).asFloatBuffer()+ val out = IntArray(1)+ GLES20.glGenBuffers(1, out, 0)+ vbo = out[0]+ GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, vbo)+ GLES20.glBufferData(+ GLES20.GL_ARRAY_BUFFER,+ capacity * STRIDE_FLOATS * 4,+ null,+ GLES20.GL_DYNAMIC_DRAW,+ )+ }++ fun upload() {+ fbuf.position(0)+ fbuf.put(data)+ fbuf.position(0)+ GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, vbo)+ GLES20.glBufferSubData(+ GLES20.GL_ARRAY_BUFFER, 0,+ capacity * STRIDE_FLOATS * 4,+ fbuf,+ )+ }++ fun bind(aPos: Int, aColor: Int) {+ GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, vbo)+ val stride = STRIDE_FLOATS * 4+ GLES20.glEnableVertexAttribArray(aPos)+ GLES20.glVertexAttribPointer(aPos, 3, GLES20.GL_FLOAT, false, stride, 0)+ GLES20.glEnableVertexAttribArray(aColor)+ GLES20.glVertexAttribPointer(aColor, 3, GLES20.GL_FLOAT, false, stride, 3 * 4)+ }++ fun draw() {+ GLES20.glDrawArrays(GLES20.GL_POINTS, 0, capacity)+ }++ companion object {+ private const val STRIDE_FLOATS = 6 // px py pz + r g b+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Shader.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Shader.ktnew file mode 100644index 0000000..a9ff740--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Shader.kt@@ -0,0 +1,55 @@+package net.kuafuai.andee.ui.ball.gl++import android.opengl.GLES20++/**+ * Compiles + links a vertex/fragment shader pair. Caches attribute and+ * uniform locations.+ */+class Shader(vertSrc: String, fragSrc: String) {+ val program: Int+ private val cache = HashMap<String, Int>()++ init {+ val vs = compile(GLES20.GL_VERTEX_SHADER, vertSrc)+ val fs = compile(GLES20.GL_FRAGMENT_SHADER, fragSrc)+ program = GLES20.glCreateProgram().also {+ GLES20.glAttachShader(it, vs)+ GLES20.glAttachShader(it, fs)+ GLES20.glLinkProgram(it)+ }+ val status = IntArray(1)+ GLES20.glGetProgramiv(program, GLES20.GL_LINK_STATUS, status, 0)+ if (status[0] == 0) {+ val log = GLES20.glGetProgramInfoLog(program)+ GLES20.glDeleteProgram(program)+ throw RuntimeException("shader link failed: $log")+ }+ GLES20.glDeleteShader(vs)+ GLES20.glDeleteShader(fs)+ }++ fun use() = GLES20.glUseProgram(program)++ fun uniform(name: String): Int =+ cache.getOrPut("u:$name") { GLES20.glGetUniformLocation(program, name) }++ fun attrib(name: String): Int =+ cache.getOrPut("a:$name") { GLES20.glGetAttribLocation(program, name) }++ companion object {+ private fun compile(type: Int, src: String): Int {+ val id = GLES20.glCreateShader(type)+ GLES20.glShaderSource(id, src)+ GLES20.glCompileShader(id)+ val status = IntArray(1)+ GLES20.glGetShaderiv(id, GLES20.GL_COMPILE_STATUS, status, 0)+ if (status[0] == 0) {+ val log = GLES20.glGetShaderInfoLog(id)+ GLES20.glDeleteShader(id)+ throw RuntimeException("shader compile failed: $log\n---\n$src")+ }+ return id+ }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Torus.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Torus.ktnew file mode 100644index 0000000..385151a--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Torus.kt@@ -0,0 +1,73 @@+package net.kuafuai.andee.ui.ball.gl++import kotlin.math.PI+import kotlin.math.cos+import kotlin.math.sin+import kotlin.math.sqrt++/**+ * Torus centered at origin, main axis is Z. UV: u along the tubular direction+ * (main ring circumference), v around the tube cross-section — matches Three.js+ * TorusGeometry so the FLOW shader's `vUv.x` sweeps around the ring.+ *+ * [arc] < 2π gives a partial torus (used for the mouth as a half-torus).+ */+object Torus {+ fun build(+ radius: Float,+ tube: Float,+ radialSegs: Int,+ tubularSegs: Int,+ arc: Float = 2f * PI.toFloat(),+ ): Mesh {+ val vertCount = (radialSegs + 1) * (tubularSegs + 1)+ val positions = FloatArray(vertCount * 3)+ val normals = FloatArray(vertCount * 3)+ val uvs = FloatArray(vertCount * 2)+ var pi = 0;+ var ui = 0+ for (j in 0..radialSegs) {+ val v = j.toFloat() / radialSegs * 2f * PI.toFloat()+ val cV = cos(v);+ val sV = sin(v)+ for (i in 0..tubularSegs) {+ val u = i.toFloat() / tubularSegs * arc+ val cU = cos(u);+ val sU = sin(u)+ val rr = radius + tube * cV+ val px = rr * cU+ val py = rr * sU+ val pz = tube * sV+ positions[pi] = px+ positions[pi + 1] = py+ positions[pi + 2] = pz+ val cx = radius * cU+ val cy = radius * sU+ val nx = px - cx+ val ny = py - cy+ val nz = pz+ val inv = 1f / sqrt(nx * nx + ny * ny + nz * nz)+ normals[pi] = nx * inv+ normals[pi + 1] = ny * inv+ normals[pi + 2] = nz * inv+ uvs[ui] = i.toFloat() / tubularSegs+ uvs[ui + 1] = j.toFloat() / radialSegs+ pi += 3; ui += 2+ }+ }+ val stride = tubularSegs + 1+ val idx = ShortArray(radialSegs * tubularSegs * 6)+ var w = 0+ for (j in 0 until radialSegs) {+ for (i in 0 until tubularSegs) {+ val a = j * stride + i+ val b = j * stride + i + 1+ val c = (j + 1) * stride + i + 1+ val d = (j + 1) * stride + i+ idx[w++] = a.toShort(); idx[w++] = b.toShort(); idx[w++] = c.toShort()+ idx[w++] = a.toShort(); idx[w++] = c.toShort(); idx[w++] = d.toShort()+ }+ }+ return Mesh(positions, normals, uvs, idx)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/UvSphere.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/UvSphere.ktnew file mode 100644index 0000000..b6c0135--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/UvSphere.kt@@ -0,0 +1,65 @@+package net.kuafuai.andee.ui.ball.gl++import kotlin.math.PI+import kotlin.math.cos+import kotlin.math.sin++/**+ * UV sphere (lat/lon segments). Matches Three.js SphereGeometry(radius, w, h).+ * Used for the rim/scan shader where we need a smooth silhouette.+ */+object UvSphere {+ fun build(widthSegments: Int, heightSegments: Int, radius: Float = 1f): Mesh {+ val vertCount = (widthSegments + 1) * (heightSegments + 1)+ val positions = FloatArray(vertCount * 3)+ val normals = FloatArray(vertCount * 3)+ val uvs = FloatArray(vertCount * 2)+ var pi = 0+ var ui = 0+ for (iy in 0..heightSegments) {+ val v = iy.toFloat() / heightSegments+ val phi = v * PI.toFloat()+ val sPhi = sin(phi)+ val cPhi = cos(phi)+ for (ix in 0..widthSegments) {+ val u = ix.toFloat() / widthSegments+ val theta = u * 2f * PI.toFloat()+ val x = -radius * cos(theta) * sPhi+ val y = radius * cPhi+ val z = radius * sin(theta) * sPhi+ positions[pi] = x+ positions[pi + 1] = y+ positions[pi + 2] = z+ normals[pi] = x / radius+ normals[pi + 1] = y / radius+ normals[pi + 2] = z / radius+ uvs[ui] = u+ uvs[ui + 1] = 1f - v+ pi += 3; ui += 2+ }+ }+ val stride = widthSegments + 1+ val idxRaw = ShortArray(widthSegments * heightSegments * 6)+ var w = 0+ for (iy in 0 until heightSegments) {+ for (ix in 0 until widthSegments) {+ val a = iy * stride + ix + 1+ val b = iy * stride + ix+ val c = (iy + 1) * stride + ix+ val d = (iy + 1) * stride + ix + 1+ if (iy != 0) {+ idxRaw[w++] = a.toShort()+ idxRaw[w++] = b.toShort()+ idxRaw[w++] = d.toShort()+ }+ if (iy != heightSegments - 1) {+ idxRaw[w++] = b.toShort()+ idxRaw[w++] = c.toShort()+ idxRaw[w++] = d.toShort()+ }+ }+ }+ val idx = ShortArray(w).also { idxRaw.copyInto(it, 0, 0, w) }+ return Mesh(positions, normals, uvs, idx)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/wake/Dtw.kt b/app/src/main/java/net/kuafuai/andee/wake/Dtw.ktnew file mode 100644index 0000000..c2bd9b7--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/wake/Dtw.kt@@ -0,0 +1,80 @@+package net.kuafuai.andee.wake++import kotlin.math.abs+import kotlin.math.max+import kotlin.math.min+import kotlin.math.sqrt++/**+ * Dynamic time warping over [Mfcc] frames.+ *+ * What it buys over comparing the two clips frame-for-frame: the user does not+ * say "嘿 Andee" at the same speed twice, and a rigid comparison turns a 15%+ * slower take into a different phrase. DTW finds the cheapest alignment that+ * keeps time moving forward in both clips, so tempo stops mattering and the+ * sounds start to.+ *+ * Two restrictions on the classic algorithm, both load-bearing here:+ *+ * - **Sakoe-Chiba band.** The alignment may not stray more than [BAND_FRAC] of+ * the clip away from the diagonal. Unbanded DTW will happily match one+ * syllable against a whole sentence by standing still on one axis, which is+ * exactly the false accept a wake word cannot afford.+ * - **Length gate.** Clips whose durations differ by more than [MAX_RATIO] are+ * rejected outright rather than warped into agreement.+ *+ * The returned cost is divided by the path length, so it is a per-frame+ * average and comparable between long and short takes — that comparability is+ * what lets one stored threshold decide every match.+ */+object Dtw {++ private const val BAND_FRAC = 0.25f+ private const val MAX_RATIO = 1.7f+ private const val MIN_BAND = 4++ /** Returned instead of a distance when the two clips cannot align at all. */+ const val NO_MATCH = Float.MAX_VALUE++ fun distance(a: Array<FloatArray>, b: Array<FloatArray>): Float {+ val n = a.size+ val m = b.size+ if (n == 0 || m == 0) return NO_MATCH+ val ratio = max(n, m).toFloat() / min(n, m)+ if (ratio > MAX_RATIO) return NO_MATCH++ val band = max(MIN_BAND, (BAND_FRAC * max(n, m)).toInt()) + abs(n - m)+ var prev = FloatArray(m + 1) { Float.MAX_VALUE }+ var cur = FloatArray(m + 1) { Float.MAX_VALUE }+ prev[0] = 0f++ for (i in 1..n) {+ java.util.Arrays.fill(cur, Float.MAX_VALUE)+ // Centre of the band for this row, in b's coordinates.+ val centre = (i.toLong() * m / n).toInt()+ val from = max(1, centre - band)+ val to = min(m, centre + band)+ for (j in from..to) {+ val best = min(min(prev[j], cur[j - 1]), prev[j - 1])+ if (best == Float.MAX_VALUE) continue+ cur[j] = best + frameDistance(a[i - 1], b[j - 1])+ }+ val t = prev; prev = cur; cur = t+ }+ val total = prev[m]+ if (total == Float.MAX_VALUE) return NO_MATCH+ // Path length is between max(n,m) and n+m; n+m is the conventional+ // normaliser and the only one that is cheap to get exactly right+ // without tracking the path itself.+ return total / (n + m)+ }++ private fun frameDistance(x: FloatArray, y: FloatArray): Float {+ var s = 0f+ for (k in x.indices) {+ val d = x[k] - y[k]+ s += d * d+ }+ return sqrt(s)+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/wake/Mfcc.kt b/app/src/main/java/net/kuafuai/andee/wake/Mfcc.ktnew file mode 100644index 0000000..dac557d--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/wake/Mfcc.kt@@ -0,0 +1,210 @@+package net.kuafuai.andee.wake++import kotlin.math.PI+import kotlin.math.cos+import kotlin.math.ln+import kotlin.math.log10+import kotlin.math.sin+import kotlin.math.sqrt++/**+ * Mel-frequency cepstral coefficients, hand-rolled.+ *+ * Twelve numbers per 10 ms of audio, and the only thing the wake word ever+ * compares. Written out rather than pulled in because the alternative is a+ * speech library measured in megabytes for the one routine below, and because+ * the pipeline has to match *exactly* between enrollment and listening — a+ * template stored with one filterbank and matched against another is silently+ * wrong, not broken, which is the worst way for this to fail.+ *+ * The parameters are the textbook ASR front end: 25 ms window every 10 ms,+ * 26 mel bands from 300 Hz to 8 kHz, keep c1..c12. Two deliberate choices on+ * top of that, both aimed at "the same phrase counts as the same phrase":+ *+ * - **c0 is dropped.** It is overall loudness, so keeping it would make the+ * same words said louder into a different phrase.+ * - **CMVN** ([normalise]) — per-utterance mean and variance normalisation.+ * This is what makes the distance scale stable enough for a *fixed* reject+ * threshold to mean anything across rooms, distances and microphones; a raw+ * cepstrum carries the channel, and the channel changes every time the user+ * picks the tablet up.+ */+object Mfcc {++ const val SAMPLE_RATE = 16_000+ const val COEFFS = 12++ /** 25 ms at 16 kHz. */+ private const val FRAME = 400++ /** 10 ms — the standard hop, and what makes DTW's time axis meaningful. */+ const val HOP = 160++ private const val FFT_N = 512+ private const val MEL_BANDS = 26+ private const val LOW_HZ = 300f+ private const val HIGH_HZ = 8000f+ private const val PRE_EMPHASIS = 0.97f++ private val window = FloatArray(FRAME) {+ (0.54 - 0.46 * cos(2.0 * PI * it / (FRAME - 1))).toFloat()+ }++ /** `[band][fftBin]` triangular weights; mostly zero, kept dense for clarity. */+ private val filters: Array<FloatArray> = buildFilters()++ /** `[coeff][band]`, skipping k=0 so the output is already c1..c12. */+ private val dct: Array<FloatArray> = Array(COEFFS) { k ->+ FloatArray(MEL_BANDS) { m ->+ cos(PI * (k + 1) * (m + 0.5) / MEL_BANDS).toFloat()+ }+ }++ /**+ * PCM (16-bit mono, [SAMPLE_RATE]) → one `[COEFFS]`-long vector per 10 ms,+ * already normalised. Empty if the clip is shorter than a single window.+ */+ fun features(pcm: ShortArray, from: Int = 0, to: Int = pcm.size): Array<FloatArray> {+ val n = to - from+ if (n < FRAME) return emptyArray()++ // Pre-emphasis: tilts the spectrum up so the high formants, which carry+ // most of what distinguishes one word from another, aren't buried under+ // the low-frequency energy of the voice.+ val x = FloatArray(n)+ x[0] = pcm[from] / 32768f+ for (i in 1 until n) {+ x[i] = (pcm[from + i] - PRE_EMPHASIS * pcm[from + i - 1]) / 32768f+ }++ val frames = (n - FRAME) / HOP + 1+ val out = Array(frames) { FloatArray(COEFFS) }+ val re = FloatArray(FFT_N)+ val im = FloatArray(FFT_N)+ val energies = FloatArray(MEL_BANDS)++ for (f in 0 until frames) {+ val off = f * HOP+ java.util.Arrays.fill(re, 0f)+ java.util.Arrays.fill(im, 0f)+ for (i in 0 until FRAME) re[i] = x[off + i] * window[i]+ fft(re, im)++ for (b in 0 until MEL_BANDS) {+ var sum = 0f+ val w = filters[b]+ for (k in w.indices) {+ if (w[k] == 0f) continue+ sum += w[k] * (re[k] * re[k] + im[k] * im[k])+ }+ // Floor rather than guard: log of a silent band is -inf, and one+ // -inf poisons every distance the frame takes part in.+ energies[b] = ln(sum.coerceAtLeast(1e-10f))+ }+ val row = out[f]+ for (k in 0 until COEFFS) {+ var s = 0f+ val d = dct[k]+ for (b in 0 until MEL_BANDS) s += d[b] * energies[b]+ row[k] = s+ }+ }+ normalise(out)+ return out+ }++ /**+ * Subtract the mean and divide by the standard deviation, per coefficient,+ * over the whole clip.+ *+ * The mean half removes the channel (room, distance, which microphone);+ * the variance half is what puts every recording on one scale, so a+ * distance of 2.5 means the same thing tomorrow as it did at enrollment.+ * Without it the threshold would have to be re-learned per environment,+ * which for a wake word means it just stops working one day.+ */+ private fun normalise(frames: Array<FloatArray>) {+ if (frames.isEmpty()) return+ for (k in 0 until COEFFS) {+ var mean = 0f+ for (f in frames) mean += f[k]+ mean /= frames.size+ var varSum = 0f+ for (f in frames) {+ val d = f[k] - mean+ varSum += d * d+ }+ val sd = sqrt(varSum / frames.size).coerceAtLeast(1e-6f)+ for (f in frames) f[k] = (f[k] - mean) / sd+ }+ }++ private fun buildFilters(): Array<FloatArray> {+ fun toMel(hz: Float) = 2595f * log10(1f + hz / 700f)+ fun toHz(mel: Float) = 700f * (Math.pow(10.0, (mel / 2595f).toDouble()).toFloat() - 1f)++ val lo = toMel(LOW_HZ)+ val hi = toMel(HIGH_HZ)+ val points = IntArray(MEL_BANDS + 2) { i ->+ val mel = lo + (hi - lo) * i / (MEL_BANDS + 1)+ ((FFT_N + 1) * toHz(mel) / SAMPLE_RATE).toInt().coerceIn(0, FFT_N / 2)+ }+ return Array(MEL_BANDS) { b ->+ val w = FloatArray(FFT_N / 2 + 1)+ val a = points[b]+ val peak = points[b + 1]+ val c = points[b + 2]+ for (k in a until peak) if (peak > a) w[k] = (k - a).toFloat() / (peak - a)+ for (k in peak until c) if (c > peak) w[k] = (c - k).toFloat() / (c - peak)+ if (peak in w.indices) w[peak] = 1f+ w+ }+ }++ /** In-place iterative radix-2 Cooley-Tukey; [re].size must be a power of two. */+ private fun fft(re: FloatArray, im: FloatArray) {+ val n = re.size+ var j = 0+ for (i in 1 until n) {+ var bit = n shr 1+ while (j and bit != 0) {+ j = j xor bit+ bit = bit shr 1+ }+ j = j or bit+ if (i < j) {+ var t = re[i]; re[i] = re[j]; re[j] = t+ t = im[i]; im[i] = im[j]; im[j] = t+ }+ }+ var len = 2+ while (len <= n) {+ val ang = -2.0 * PI / len+ val wr = cos(ang).toFloat()+ val wi = sin(ang).toFloat()+ var i = 0+ while (i < n) {+ var cr = 1f+ var ci = 0f+ val half = len / 2+ for (k in 0 until half) {+ val ur = re[i + k]+ val ui = im[i + k]+ val pr = re[i + k + half]+ val pi = im[i + k + half]+ val vr = pr * cr - pi * ci+ val vi = pr * ci + pi * cr+ re[i + k] = ur + vr+ im[i + k] = ui + vi+ re[i + k + half] = ur - vr+ im[i + k + half] = ui - vi+ val ncr = cr * wr - ci * wi+ ci = cr * wi + ci * wr+ cr = ncr+ }+ i += len+ }+ len = len shl 1+ }+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/wake/VoiceSegmenter.kt b/app/src/main/java/net/kuafuai/andee/wake/VoiceSegmenter.ktnew file mode 100644index 0000000..259260f--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/wake/VoiceSegmenter.kt@@ -0,0 +1,152 @@+package net.kuafuai.andee.wake++import kotlin.math.sqrt++/**+ * Cuts a continuous microphone stream into "somebody just said something"+ * clips.+ *+ * Plain energy VAD against an adaptive noise floor, because the wake word only+ * needs to know *where* an utterance is, not what it is — [Mfcc] and [Dtw]+ * decide the rest. Anything heavier would be running a model on every second+ * of room tone, all day, for a phrase that arrives twice an hour.+ *+ * The floor adapts because the alternative is a fixed threshold, and a fixed+ * threshold is deaf in a kitchen and permanently triggered next to a fan. It+ * only moves while the room is quiet — a floor that learned during speech+ * would chase the user's own voice upward and cut them off mid-phrase.+ *+ * Pre-roll ([PREROLL_FRAMES]) matters more than it looks: a voice crosses the+ * threshold a few frames *after* it starts, and the missing onset is the+ * consonant — the 'h' of "嘿" — which is most of what distinguishes the wake+ * phrase from every other short utterance.+ *+ * Not thread-safe; it belongs to whichever thread is reading the microphone.+ */+class VoiceSegmenter(private val onSegment: (ShortArray) -> Unit) {++ private val frame = ShortArray(FRAME)+ private var filled = 0++ private var floor = -1f+ private var speech = false+ private var loudRun = 0+ private var quietRun = 0++ /** Pre-roll ring, and the utterance once one starts. */+ private val preroll = ArrayDeque<ShortArray>()+ private var collected = ArrayList<ShortArray>()++ fun feed(pcm: ShortArray, count: Int) {+ var off = 0+ while (off < count) {+ val take = minOf(FRAME - filled, count - off)+ System.arraycopy(pcm, off, frame, filled, take)+ filled += take+ off += take+ if (filled == FRAME) {+ filled = 0+ onFrame()+ }+ }+ }++ /** Forget any partial utterance — used when the microphone is handed away. */+ fun reset() {+ filled = 0+ speech = false+ loudRun = 0+ quietRun = 0+ preroll.clear()+ collected = ArrayList()+ floor = -1f+ }++ private fun onFrame() {+ var sum = 0.0+ for (s in frame) sum += s.toDouble() * s+ val rms = sqrt(sum / FRAME).toFloat()++ if (floor < 0f) floor = rms+ val startAt = maxOf(floor * START_MULT, ABS_MIN)+ val endAt = maxOf(floor * END_MULT, ABS_MIN * 0.7f)++ if (!speech) {+ // Only while idle, and only downward-biased: rising slowly means a+ // fan that starts up is eventually accepted as silence, while a+ // voice that lasts a second barely moves it.+ floor = if (rms < floor) floor * 0.9f + rms * 0.1f else floor * 0.995f + rms * 0.005f++ preroll.addLast(frame.copyOf())+ if (preroll.size > PREROLL_FRAMES) preroll.removeFirst()++ if (rms > startAt) {+ loudRun+++ if (loudRun >= START_FRAMES) {+ speech = true+ quietRun = 0+ collected = ArrayList(preroll)+ preroll.clear()+ }+ } else {+ loudRun = 0+ }+ return+ }++ collected.add(frame.copyOf())+ if (rms < endAt) {+ quietRun+++ if (quietRun >= END_FRAMES) {+ emit()+ return+ }+ } else {+ quietRun = 0+ }+ // Too long to be a two-syllable wake phrase. Dropped rather than+ // truncated: a sentence clipped at 1.8 s matched against the templates+ // is a coin toss, and the user talking to somebody else in the room is+ // by far the most common way to get here.+ if (collected.size > MAX_FRAMES) {+ speech = false+ loudRun = 0+ quietRun = 0+ collected = ArrayList()+ }+ }++ private fun emit() {+ val frames = collected+ speech = false+ loudRun = 0+ quietRun = 0+ collected = ArrayList()+ preroll.clear()+ if (frames.size < MIN_FRAMES) return+ val out = ShortArray(frames.size * FRAME)+ var i = 0+ for (f in frames) {+ System.arraycopy(f, 0, out, i, FRAME)+ i += FRAME+ }+ onSegment(out)+ }++ private companion object {+ /** 10 ms at 16 kHz — same grid as [Mfcc.HOP], so segment bounds land on frames. */+ const val FRAME = Mfcc.HOP++ const val PREROLL_FRAMES = 15 // 150 ms of run-up+ const val START_FRAMES = 3 // 30 ms above the line before it counts+ const val END_FRAMES = 30 // 300 ms of quiet ends the utterance+ const val MIN_FRAMES = 35 // 350 ms — shorter than "嘿 Andee" can be+ const val MAX_FRAMES = 180 // 1.8 s — longer than it can be++ const val START_MULT = 3.5f+ const val END_MULT = 2.0f++ /** RMS in 16-bit units; below this it is room tone however quiet the room. */+ const val ABS_MIN = 350f+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/wake/WakeMic.kt b/app/src/main/java/net/kuafuai/andee/wake/WakeMic.ktnew file mode 100644index 0000000..5bf2611--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/wake/WakeMic.kt@@ -0,0 +1,100 @@+package net.kuafuai.andee.wake++import android.media.AudioFormat+import android.media.AudioRecord+import android.media.MediaRecorder+import android.util.Log+import java.util.concurrent.atomic.AtomicBoolean++/**+ * A 16 kHz mono capture with its own thread, opened and closed many times a+ * day.+ *+ * Separate from [net.kuafuai.andee.audio.AudioIO] on purpose. That class is the+ * conversation's microphone and refuses to run two sessions at once+ * (`audio busy`); this one is the wake word's, and the two must never be open+ * together — the wake listener releases the device before the user's turn+ * begins and takes it back afterwards. Sharing one recorder would mean one+ * owner, and the owner would have to be the wake word, which is the wrong way+ * round: the turn must never fail because something was listening for "嘿".+ *+ * Deliberately short-lived. Holding the microphone open while the screen is+ * off, or while Andee is talking, costs battery and lights the platform's+ * green recording dot for no reason.+ */+class WakeMic(private val onPcm: (ShortArray, Int) -> Unit) {++ private val running = AtomicBoolean(false)+ private var thread: Thread? = null++ fun isOpen(): Boolean = running.get()++ /** False if the device refused the microphone — caller should back off and retry. */+ fun start(): Boolean {+ if (running.get()) return true+ val minBuf = AudioRecord.getMinBufferSize(+ Mfcc.SAMPLE_RATE, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT,+ )+ if (minBuf <= 0) return false+ // VOICE_RECOGNITION for the same reason the ASR path uses it: the+ // platform's own noise suppression and AGC, without the audiofx+ // attachments that stalled the HAL on this hardware.+ @Suppress("MissingPermission")+ val rec = try {+ AudioRecord(+ MediaRecorder.AudioSource.VOICE_RECOGNITION,+ Mfcc.SAMPLE_RATE,+ AudioFormat.CHANNEL_IN_MONO,+ AudioFormat.ENCODING_PCM_16BIT,+ minBuf * 2,+ )+ } catch (t: Throwable) {+ Log.w(TAG, "AudioRecord ctor failed", t)+ return false+ }+ if (rec.state != AudioRecord.STATE_INITIALIZED) {+ rec.release()+ return false+ }+ running.set(true)+ return try {+ rec.startRecording()+ thread = Thread {+ val buf = ShortArray(CHUNK)+ try {+ while (running.get()) {+ val n = rec.read(buf, 0, buf.size)+ if (n <= 0) break+ onPcm(buf, n)+ }+ } catch (t: Throwable) {+ Log.w(TAG, "capture ended", t)+ } finally {+ runCatching { rec.stop() }+ rec.release()+ running.set(false)+ }+ }.also { it.name = "WakeMic"; it.isDaemon = true; it.start() }+ true+ } catch (t: Throwable) {+ Log.w(TAG, "startRecording failed", t)+ running.set(false)+ runCatching { rec.release() }+ false+ }+ }++ fun stop() {+ if (!running.getAndSet(false)) return+ val t = thread+ thread = null+ runCatching { t?.join(500) }+ }++ private companion object {+ const val TAG = "Wake"++ /** 20 ms; two [VoiceSegmenter] frames per read, small enough to stop promptly. */+ const val CHUNK = Mfcc.HOP * 2+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/wake/WakeTemplates.kt b/app/src/main/java/net/kuafuai/andee/wake/WakeTemplates.ktnew file mode 100644index 0000000..d3d4185--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/wake/WakeTemplates.kt@@ -0,0 +1,165 @@+package net.kuafuai.andee.wake++import android.content.Context+import android.util.Log+import org.json.JSONArray+import org.json.JSONObject+import java.io.File++/**+ * The enrolled "嘿 Andee" takes, on disk.+ *+ * Three recordings reduced to [Mfcc] frames — a few hundred floats each, no+ * audio kept. That is a deliberate privacy property as much as a size one:+ * nothing recoverable as speech ever lands in storage, and there is no server+ * to send it to in the first place.+ *+ * The stored threshold matters more than the templates. It is derived from how+ * far apart the user's *own* three takes landed ([thresholdFor]), which is the+ * only calibration available: how consistently a person says their own wake+ * phrase, in their room, into this microphone, varies enormously between+ * people, and a constant tuned on one voice is either deaf or trigger-happy on+ * the next.+ */+object WakeTemplates {++ private const val FILE_NAME = "wake_templates.json"+ private const val TAG = "Wake"++ /** How many takes enrollment asks for. Three pairs is the least that says anything. */+ const val ENROLL_COUNT = 3++ /**+ * Slack over the user's own worst pair. Below ~1.1 the phrase has to be+ * said more consistently than anyone says anything; above ~1.4 the door is+ * open wide enough for unrelated speech to walk in.+ */+ private const val SLACK = 1.25f++ /**+ * Hard bounds on the learned threshold, in the units [Dtw] returns after+ * [Mfcc]'s CMVN. A user who enrolls three nearly identical takes would+ * otherwise end up with a threshold so tight only a recording could pass+ * it; one who mumbles differently each time would get one that accepts the+ * television.+ *+ * The band is deliberately biased low. The two failure modes are not+ * symmetric: too tight and the user taps the ball instead — a small+ * annoyance with an obvious workaround. Too loose and a stray sentence+ * opens the microphone and ships audio to 火山 without anyone asking, which+ * is both a surprise and a cost. So when in doubt, be deaf.+ *+ * These numbers are a starting point, not a measurement — a second take of+ * the same phrase lands well under 1.0 on synthetic audio, but real voices+ * spread wider and no real recording has been through this yet. The+ * `no wake: d=… > …` line in [WakeWord] is what a near miss looks like in+ * logcat, and is the evidence to move them on.+ */+ private const val MIN_THRESHOLD = 1.0f+ private const val MAX_THRESHOLD = 2.8f++ class Store(val threshold: Float, val templates: List<Array<FloatArray>>)++ @Volatile+ private var cache: Store? = null++ @Volatile+ private var loaded = false++ fun load(context: Context): Store? {+ cache?.let { return it }+ if (loaded) return null+ loaded = true+ val f = File(context.filesDir, FILE_NAME)+ if (!f.exists()) return null+ val store = runCatching {+ val root = JSONObject(f.readText())+ val arr = root.getJSONArray("templates")+ val templates = ArrayList<Array<FloatArray>>(arr.length())+ for (i in 0 until arr.length()) {+ val rows = arr.getJSONArray(i)+ templates.add(+ Array(rows.length()) { r ->+ val row = rows.getJSONArray(r)+ FloatArray(row.length()) { c -> row.getDouble(c).toFloat() }+ }+ )+ }+ Store(root.getDouble("threshold").toFloat(), templates)+ }.onFailure { Log.w(TAG, "templates unreadable, ignoring", it) }.getOrNull()+ cache = store+ return store+ }++ fun enrolled(context: Context): Boolean =+ (load(context)?.templates?.size ?: 0) > 0++ /** Persist [templates] with a threshold derived from them. Returns the threshold. */+ fun save(context: Context, templates: List<Array<FloatArray>>): Float {+ val threshold = thresholdFor(templates)+ val arr = JSONArray()+ for (t in templates) {+ val rows = JSONArray()+ for (frame in t) {+ val row = JSONArray()+ for (v in frame) row.put(v.toDouble())+ rows.put(row)+ }+ arr.put(rows)+ }+ val root = JSONObject()+ .put("version", 1)+ .put("threshold", threshold.toDouble())+ .put("templates", arr)+ runCatching { File(context.filesDir, FILE_NAME).writeText(root.toString()) }+ .onFailure { Log.w(TAG, "template save failed", it) }+ cache = Store(threshold, templates)+ loaded = true+ Log.i(TAG, "enrolled ${templates.size} takes, threshold=$threshold")+ return threshold+ }++ fun clear(context: Context) {+ runCatching { File(context.filesDir, FILE_NAME).delete() }+ cache = null+ loaded = true+ }++ /** Drop the cache without touching the file; next [load] re-reads it. */+ fun invalidate() {+ cache = null+ loaded = false+ }++ /** Worst distance between any two takes, plus [SLACK], clamped. */+ private fun thresholdFor(templates: List<Array<FloatArray>>): Float {+ var worst = 0f+ var pairs = 0+ for (i in templates.indices) {+ for (j in i + 1 until templates.size) {+ val d = Dtw.distance(templates[i], templates[j])+ if (d == Dtw.NO_MATCH) continue+ pairs+++ if (d > worst) worst = d+ }+ }+ // No comparable pair at all — the takes differed so much in length that+ // DTW refused them. Nothing was learned, so fall back to the middle of+ // the useful range rather than to a number computed from nothing.+ if (pairs == 0) return (MIN_THRESHOLD + MAX_THRESHOLD) / 2f+ return (worst * SLACK).coerceIn(MIN_THRESHOLD, MAX_THRESHOLD)+ }++ /**+ * How well [candidate] matches the enrolled phrase: the closest template,+ * or [Dtw.NO_MATCH]. Compared against [Store.threshold] by the caller.+ */+ fun bestDistance(store: Store, candidate: Array<FloatArray>): Float {+ var best = Dtw.NO_MATCH+ for (t in store.templates) {+ val d = Dtw.distance(t, candidate)+ if (d < best) best = d+ }+ return best+ }+}diff --git a/app/src/main/java/net/kuafuai/andee/wake/WakeWord.kt b/app/src/main/java/net/kuafuai/andee/wake/WakeWord.ktnew file mode 100644index 0000000..319d0e4--- /dev/null+++ b/app/src/main/java/net/kuafuai/andee/wake/WakeWord.kt@@ -0,0 +1,155 @@+package net.kuafuai.andee.wake++import android.content.Context+import android.os.SystemClock+import android.util.Log++/**+ * "嘿 Andee" — the hands-free way to do what tapping the ball does.+ *+ * Entirely on device: [WakeMic] → [VoiceSegmenter] → [Mfcc] → [Dtw] against+ * the three takes the user enrolled. No network, no model file, no per-call+ * cost, and nothing recoverable as speech ever leaves the microphone thread.+ * The trade is accuracy — it recognises *this user's* phrase rather than the+ * phrase, and it will occasionally miss. That is the right trade for something+ * whose failure mode is "tap the ball instead".+ *+ * ## Who owns the microphone+ *+ * Never this class, when anything else wants it. The supervisor thread asks+ * [gate] a few times a second and closes the capture the moment it says no —+ * the conversation's own recorder must never fail because something was+ * listening for a wake word. The gate is also how "only while the screen is+ * on" is enforced, and why Andee does not hear itself: TTS speaking closes it+ * too.+ *+ * Idle when nothing is enrolled, deliberately: an unconfigured install should+ * not hold the microphone open, and the platform's green recording dot should+ * not appear for a feature the user has not set up.+ */+object WakeWord {++ private const val TAG = "Wake"++ /** How often the supervisor reconsiders. Also the worst-case wake latency added. */+ private const val POLL_MS = 100L++ /** Backoff when the device refuses the microphone (another app holds it). */+ private const val RETRY_MS = 2_000L++ /** One wake per this long, so a doubled phrase can't start two turns. */+ private const val REFRACTORY_MS = 2_500L++ private var appContext: Context? = null+ private var gate: () -> Boolean = { false }+ private var onWake: () -> Unit = {}++ @Volatile private var running = false++ /** Held by enrollment, which needs the microphone to itself. */+ @Volatile private var paused = false++ /** Set on the capture thread, acted on by the supervisor — see [onSegment]. */+ @Volatile private var heard = false++ @Volatile private var lastWakeAt = 0L++ private var thread: Thread? = null+ private val mic = WakeMic { pcm, n -> segmenter.feed(pcm, n) }+ private val segmenter = VoiceSegmenter { seg -> onSegment(seg) }++ fun init(context: Context, gate: () -> Boolean, onWake: () -> Unit) {+ this.appContext = context.applicationContext+ this.gate = gate+ this.onWake = onWake+ }++ fun enrolled(): Boolean {+ val ctx = appContext ?: return false+ return WakeTemplates.enrolled(ctx)+ }++ /** Re-read the templates after enrollment changed them. */+ fun reload() {+ WakeTemplates.invalidate()+ }++ fun start() {+ if (running) return+ running = true+ thread = Thread {+ while (running) {+ supervise()+ Thread.sleep(POLL_MS)+ }+ mic.stop()+ }.also { it.name = "WakeWord"; it.isDaemon = true; it.start() }+ }++ fun stop() {+ running = false+ val t = thread+ thread = null+ mic.stop()+ runCatching { t?.join(500) }+ }++ /**+ * Enrollment takes the microphone. Wrapped rather than exposed as two calls+ * so a thrown recorder can't leave the listener switched off for good.+ */+ fun <T> withMicReleased(block: () -> T): T {+ paused = true+ mic.stop()+ segmenter.reset()+ return try {+ block()+ } finally {+ paused = false+ }+ }++ private fun supervise() {+ if (heard) {+ heard = false+ // Close before waking: the very next thing that happens is the ASR+ // path opening its own recorder, and two live captures inside one+ // app is a device-dependent coin toss we have no reason to flip.+ mic.stop()+ segmenter.reset()+ runCatching { onWake() }.onFailure { Log.w(TAG, "onWake failed", it) }+ return+ }+ val want = !paused && enrolled() && runCatching { gate() }.getOrDefault(false)+ if (want) {+ if (!mic.isOpen()) {+ segmenter.reset()+ if (!mic.start()) Thread.sleep(RETRY_MS)+ }+ } else if (mic.isOpen()) {+ mic.stop()+ segmenter.reset()+ }+ }++ /** Capture thread. Must not block: the microphone stops being read while it runs. */+ private fun onSegment(pcm: ShortArray) {+ val ctx = appContext ?: return+ val store = WakeTemplates.load(ctx) ?: return+ val now = SystemClock.uptimeMillis()+ if (now - lastWakeAt < REFRACTORY_MS) return+ val feats = Mfcc.features(pcm)+ if (feats.isEmpty()) return+ val d = WakeTemplates.bestDistance(store, feats)+ if (d <= store.threshold) {+ Log.i(TAG, "wake: d=%.2f <= %.2f (%d ms)".format(d, store.threshold, pcm.size / 16))+ lastWakeAt = now+ heard = true+ } else if (d != Dtw.NO_MATCH) {+ // The one number worth having when the user says "it didn't hear+ // me": a near miss and an unrelated sentence look identical from+ // the outside, and only this tells them apart.+ Log.d(TAG, "no wake: d=%.2f > %.2f (%d ms)".format(d, store.threshold, pcm.size / 16))+ }+ }+}diff --git a/app/src/main/res/anim/stage_close_exit.xml b/app/src/main/res/anim/stage_close_exit.xmlnew file mode 100644index 0000000..1a5f446--- /dev/null+++ b/app/src/main/res/anim/stage_close_exit.xml@@ -0,0 +1,35 @@+<?xml version="1.0" encoding="utf-8"?>+<!--+ 全屏脸的退场,整个 window 一起走。++ 为什么必须是 window 级动画而不是 View 级动画 —— 这是"关闭时闪一下"的+ 真正原因,量过:++ 之前 StageActivity 用 `stage.animate().alpha(0f)` 收场。stage 是 View+ 树,它透明之后露出来的不是背后那个 app,而是**这个 Activity 自己的+ windowBackground**(纯黑,见 Theme.Body.Stage)。所以逐帧看下来是:++ 64-72 帧 画面渐暗 ← stage 的 alpha 1→0+ 72-86 帧 整屏纯黑 ~250ms ← 黑底 windowBackground 露出来了+ 87 帧起 背后内容淡入++ 中间那 250ms 谁也不是,用户说的"闪一下"就是它。View 级动画够不着+ windowBackground,因为那层不是 View;只有 window 级动画能把整块 surface+ (内容 + 黑底)一起淡掉,背后的东西才在动画过程中就露出来。++ 缩放略大于入场(入场 0.94),是为了让"走"有方向感:退到一个比来处更远的+ 位置,读起来像是退出去,而不是原路缩回。+-->+<set xmlns:android="http://schemas.android.com/apk/res/android"+ android:interpolator="@android:interpolator/fast_out_linear_in">+ <scale+ android:fromXScale="1.0"+ android:toXScale="0.90"+ android:fromYScale="1.0"+ android:toYScale="0.90"+ android:pivotX="50%"+ android:pivotY="50%" />+ <alpha+ android:fromAlpha="1.0"+ android:toAlpha="0.0" />+</set>diff --git a/app/src/main/res/drawable-nodpi/backdrop.jpg b/app/src/main/res/drawable-nodpi/backdrop.jpgnew file mode 100644index 0000000..91e7bd2Binary files /dev/null and b/app/src/main/res/drawable-nodpi/backdrop.jpg differdiff --git a/app/src/main/res/mipmap-anydpi/ic_launcher.xml b/app/src/main/res/mipmap-anydpi/ic_launcher.xmlnew file mode 100644index 0000000..cf79c3d--- /dev/null+++ b/app/src/main/res/mipmap-anydpi/ic_launcher.xml@@ -0,0 +1,6 @@+<?xml version="1.0" encoding="utf-8"?>+<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">+ <background android:drawable="@mipmap/ic_launcher_background" />+ <foreground android:drawable="@mipmap/ic_launcher_foreground" />+ <monochrome android:drawable="@mipmap/ic_launcher_monochrome" />+</adaptive-icon>diff --git a/app/src/main/res/mipmap-anydpi/ic_launcher_round.xml b/app/src/main/res/mipmap-anydpi/ic_launcher_round.xmlnew file mode 100644index 0000000..cf79c3d--- /dev/null+++ b/app/src/main/res/mipmap-anydpi/ic_launcher_round.xml@@ -0,0 +1,6 @@+<?xml version="1.0" encoding="utf-8"?>+<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">+ <background android:drawable="@mipmap/ic_launcher_background" />+ <foreground android:drawable="@mipmap/ic_launcher_foreground" />+ <monochrome android:drawable="@mipmap/ic_launcher_monochrome" />+</adaptive-icon>diff --git a/app/src/main/res/mipmap-hdpi/ic_launcher.webp b/app/src/main/res/mipmap-hdpi/ic_launcher.webpnew file mode 100644index 0000000..05fb8dbBinary files /dev/null and b/app/src/main/res/mipmap-hdpi/ic_launcher.webp differdiff --git a/app/src/main/res/mipmap-hdpi/ic_launcher_background.webp b/app/src/main/res/mipmap-hdpi/ic_launcher_background.webpnew file mode 100644index 0000000..43e2f77Binary files /dev/null and b/app/src/main/res/mipmap-hdpi/ic_launcher_background.webp differdiff --git a/app/src/main/res/mipmap-hdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-hdpi/ic_launcher_foreground.webpnew file mode 100644index 0000000..4771590Binary files /dev/null and b/app/src/main/res/mipmap-hdpi/ic_launcher_foreground.webp differdiff --git a/app/src/main/res/mipmap-hdpi/ic_launcher_monochrome.webp b/app/src/main/res/mipmap-hdpi/ic_launcher_monochrome.webpnew file mode 100644index 0000000..7bd363bBinary files /dev/null and b/app/src/main/res/mipmap-hdpi/ic_launcher_monochrome.webp differdiff --git a/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-hdpi/ic_launcher_round.webpnew file mode 100644index 0000000..7a4efcfBinary files /dev/null and b/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp differdiff --git a/app/src/main/res/mipmap-mdpi/ic_launcher.webp b/app/src/main/res/mipmap-mdpi/ic_launcher.webpnew file mode 100644index 0000000..48327b0Binary files /dev/null and b/app/src/main/res/mipmap-mdpi/ic_launcher.webp differdiff --git a/app/src/main/res/mipmap-mdpi/ic_launcher_background.webp b/app/src/main/res/mipmap-mdpi/ic_launcher_background.webpnew file mode 100644index 0000000..982b7d8Binary files /dev/null and b/app/src/main/res/mipmap-mdpi/ic_launcher_background.webp differdiff --git a/app/src/main/res/mipmap-mdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-mdpi/ic_launcher_foreground.webpnew file mode 100644index 0000000..c60e393Binary files /dev/null and b/app/src/main/res/mipmap-mdpi/ic_launcher_foreground.webp differdiff --git a/app/src/main/res/mipmap-mdpi/ic_launcher_monochrome.webp b/app/src/main/res/mipmap-mdpi/ic_launcher_monochrome.webpnew file mode 100644index 0000000..15cffe4Binary files /dev/null and b/app/src/main/res/mipmap-mdpi/ic_launcher_monochrome.webp differdiff --git a/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-mdpi/ic_launcher_round.webpnew file mode 100644index 0000000..a0099ddBinary files /dev/null and b/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp differdiff --git a/app/src/main/res/mipmap-xhdpi/ic_launcher.webp b/app/src/main/res/mipmap-xhdpi/ic_launcher.webpnew file mode 100644index 0000000..2e8b67fBinary files /dev/null and b/app/src/main/res/mipmap-xhdpi/ic_launcher.webp differdiff --git a/app/src/main/res/mipmap-xhdpi/ic_launcher_background.webp b/app/src/main/res/mipmap-xhdpi/ic_launcher_background.webpnew file mode 100644index 0000000..40c806dBinary files /dev/null and b/app/src/main/res/mipmap-xhdpi/ic_launcher_background.webp differdiff --git a/app/src/main/res/mipmap-xhdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-xhdpi/ic_launcher_foreground.webpnew file mode 100644index 0000000..9b11a12Binary files /dev/null and b/app/src/main/res/mipmap-xhdpi/ic_launcher_foreground.webp differdiff --git a/app/src/main/res/mipmap-xhdpi/ic_launcher_monochrome.webp b/app/src/main/res/mipmap-xhdpi/ic_launcher_monochrome.webpnew file mode 100644index 0000000..0f28a1fBinary files /dev/null and b/app/src/main/res/mipmap-xhdpi/ic_launcher_monochrome.webp differdiff --git a/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webpnew file mode 100644index 0000000..230cd17Binary files /dev/null and b/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp differdiff --git a/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp b/app/src/main/res/mipmap-xxhdpi/ic_launcher.webpnew file mode 100644index 0000000..ca1febbBinary files /dev/null and b/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp differdiff --git a/app/src/main/res/mipmap-xxhdpi/ic_launcher_background.webp b/app/src/main/res/mipmap-xxhdpi/ic_launcher_background.webpnew file mode 100644index 0000000..c498c95Binary files /dev/null and b/app/src/main/res/mipmap-xxhdpi/ic_launcher_background.webp differdiff --git a/app/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.webpnew file mode 100644index 0000000..98ad80aBinary files /dev/null and b/app/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.webp differdiff --git a/app/src/main/res/mipmap-xxhdpi/ic_launcher_monochrome.webp b/app/src/main/res/mipmap-xxhdpi/ic_launcher_monochrome.webpnew file mode 100644index 0000000..b241d2eBinary files /dev/null and b/app/src/main/res/mipmap-xxhdpi/ic_launcher_monochrome.webp differdiff --git a/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webpnew file mode 100644index 0000000..347c124Binary files /dev/null and b/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp differdiff --git a/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp b/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webpnew file mode 100644index 0000000..090d91eBinary files /dev/null and b/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp differdiff --git a/app/src/main/res/mipmap-xxxhdpi/ic_launcher_background.webp b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_background.webpnew file mode 100644index 0000000..a4ff64aBinary files /dev/null and b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_background.webp differdiff --git a/app/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.webpnew file mode 100644index 0000000..5b1bc7cBinary files /dev/null and b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.webp differdiff --git a/app/src/main/res/mipmap-xxxhdpi/ic_launcher_monochrome.webp b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_monochrome.webpnew file mode 100644index 0000000..58b36f2Binary files /dev/null and b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_monochrome.webp differdiff --git a/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webpnew file mode 100644index 0000000..159524bBinary files /dev/null and b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp differdiff --git a/app/src/main/res/values-en/strings.xml b/app/src/main/res/values-en/strings.xmlnew file mode 100644index 0000000..877dc5c--- /dev/null+++ b/app/src/main/res/values-en/strings.xml@@ -0,0 +1,399 @@+<resources>+ <string name="app_name">Andee</string>+ <string name="notification_relay_label">Andee notification access</string>+ <string name="accessibility_service_description">Andee needs this to see what is on screen and to tap, swipe and type for you. A floating ball appears on the home screen: tap it to talk, long-press to open the conversation.</string>++ <!--+ English mirror of values/strings.xml. Same keys, same order, one entry+ per Chinese entry — a key added on one side and not the other falls+ back silently, so the Chinese would show through on an English tablet+ with nothing to indicate why.++ **What is here and what is not.** The app holds three kinds of Chinese+ text and only the first is translated:++ 1. **What the user reads** — every window, card, bubble, settings row+ and error line. That is this file. It is looked up through+ `i18n/AppLocale`, never `context.getString` directly: the ball and+ every card are overlay windows owned by a *service*, so the official+ per-app locale API cannot reach them and the only thing that carries+ the language is the `Context` you ask.+ 2. **What the model reads** — prompts, tool schemas, tool results. Written+ in English, and deliberately so. The argument for leaving this kind in+ Chinese used to be that the language a prompt is written in decides+ the language the answer comes back in, so translating one without the+ other is worse than translating neither. That pull still exists; it+ has only reversed direction, so it is now held back from inside:+ `brain/LocalPrompt.kt` §9 is the rule that makes the model answer in+ whatever language the user actually spoke. Chinese that still turns up+ inside an English prompt is either a label printed on the screen,+ which the model is meant to repeat verbatim, or a specimen of what a+ Chinese user says — never an oversight.++ **A default whose value can reach the screen belongs to kind 1, not+ kind 2**, even though it also travels inside a prompt. `会议记录`,+ `通话` and `AI 界面` used to be Kotlin literals in `MeetingRecorder`,+ `ScreenBodyService` and `ToolSchemas`, on the argument that the brain+ had to be handed the same title the page displayed. They are+ `meeting_default_title`, `call_default_scene` and+ `html_default_title` now: one key answered for both readers, so the+ two still cannot disagree — and an English tablet shows an English+ page header.+ 3. **What matches system text** — `NotificationRelayService`'s call+ keywords, the unread-count pattern, the clipboard labels. Never+ translated: they are compared against strings WeChat and Android+ emit, which do not follow our setting. Translating them breaks the+ feature rather than the wording.+ -->+ <string name="settings_title">Settings</string>+ <string name="settings_default_device_name">This device</string>+ <string name="settings_factory_reset">Factory reset</string>+ <string name="common_cancel">Cancel</string>+ <string name="common_save">Save</string>+ <string name="settings_factory_reset_confirm_title">Factory reset?</string>+ <string name="settings_factory_reset_list_intro">These are deleted for good — there is no backup:</string>+ <string name="settings_factory_reset_after">The device goes back to a fresh install: keys have to be entered again and the wake word has to be recorded again.</string>+ <string name="settings_factory_reset_back">No, go back</string>+ <string name="settings_factory_reset_confirm">Erase everything</string>+ <!-- The two destructive buttons sit together at the foot of the+ Diagnostics section, not in the footer beside Cancel/Save: a+ destructive control beside the card's two ways out reads as a+ third way out. -->+ <string name="settings_clear_chat">Clear chat history</string>+ <string name="settings_clear_chat_confirm_title">Clear the chat history?</string>+ <string name="settings_clear_chat_confirm_body">The conversation record and its pages are deleted, and everything said in this session is forgotten. The notebook is untouched.</string>+ <string name="settings_clear_chat_confirm">Clear it</string>+ <string name="settings_clear_notes">Clear the notebook</string>+ <string name="settings_clear_notes_confirm_title">Clear the notebook?</string>+ <string name="settings_clear_notes_confirm_body">Everything it noted down, your preferences, and any reminder not yet due are deleted, with no way back. The current chat history is untouched.</string>+ <string name="settings_clear_notes_confirm">Clear it</string>+ <string name="settings_section_brain">Brain</string>+ <string name="settings_brain_panel_note">Takes effect when you close this panel — no need to restart the accessibility service.</string>+ <string name="settings_brain_cloud">Cloud</string>+ <string name="settings_brain_local">On-device</string>+ <string name="settings_hub_url">Hub URL</string>+ <!-- The hub URL box is empty on a fresh install — the hint is what the+ user sees in that empty box, so it is the pitch, not the format. -->+ <string name="settings_hub_url_hint">Link your cloud agent</string>+ <string name="settings_device_name">Device name</string>+ <string name="settings_device_name_hint">What the hub should call this device</string>+ <string name="settings_brain_cloud_note">The brain lives on a hub you run yourself; this device only acts.</string>+ <string name="settings_llm_model">Model</string>+ <string name="settings_llm_base_url">Base URL</string>+ <string name="settings_llm_thinking">Deep thinking</string>+ <string name="common_on">On</string>+ <string name="common_off">Off</string>+ <string name="settings_llm_effort">Reasoning effort</string>+ <string name="common_high">High</string>+ <string name="common_medium">Medium</string>+ <string name="common_low">Low</string>+ <string name="settings_brain_local_note">The device runs the agent loop itself, with no hub. Without a key it will not take over.</string>+ <string name="settings_section_wake">Wake word</string>+ <string name="settings_wake_enrolled">Enrolled — just say “Hey Andee” while the screen is on</string>+ <string name="settings_wake_not_enrolled">Not enrolled — record it once to wake me by voice</string>+ <string name="settings_wake_record">Record</string>+ <string name="common_clear">Clear</string>+ <string name="settings_section_vault">Vault</string>+ <string name="settings_vault_empty_note">No accounts yet — once saved, Andee can log in as you when a task needs it</string>+ <string name="settings_vault_count_note">%1$d saved — the brain may read the phone and email, but only this device types the password</string>+ <string name="settings_vault_manage">Manage</string>+ <string name="settings_section_voice">Speech (Volcengine)</string>+ <string name="settings_voice_factory_note">The source ships without a key. Enter your own to hear and speak; leave the rows below blank for the default endpoints.</string>+ <string name="settings_voice_api_key">Volcengine API key</string>+ <string name="settings_voice_api_key_hint">Volcengine Speech key — hearing and speaking both need it</string>+ <string name="settings_voice_in_use">In use</string>+ <string name="settings_voice_in_use_own">the Volcengine account you entered</string>+ <string name="settings_voice_in_use_factory">nothing yet — hearing and speaking are off</string>+ <string name="settings_voice_asr_endpoint">ASR endpoint</string>+ <string name="settings_voice_asr_resource_id">ASR resource ID</string>+ <string name="settings_voice_tts_endpoint">TTS endpoint</string>+ <string name="settings_voice_tts_resource_id">TTS resource ID</string>+ <string name="settings_voice_tts_sample_rate">TTS sample rate</string>+ <string name="settings_section_diagnostics">Diagnostics</string>+ <string name="settings_grounding">Grounding capture</string>+ <string name="settings_grounding_note">Records coordinate taps and screenshots to files/grounding for offline analysis — it photographs whatever you are doing. Needs the accessibility service restarted to take effect.</string>+ <string name="settings_device_id">Device ID</string>+ <string name="settings_section_notebook">Notebook</string>+ <string name="settings_notebook_sweep_note">Once a conversation goes quiet it reviews it by itself: keeps what is worth keeping, schedules what it owes. On by default — this is where \\"it remembers\\" comes from. Off means it stops spending your key on those calls.</string>+ <string name="settings_notebook_quiet_minutes">Quiet minutes before a review</string>+ <string name="settings_notebook_daily_cap">Reviews per day, at most</string>+ <string name="settings_notebook_today">Today / so far</string>+ <string name="settings_notebook_stats">%1$d reviews (~%2$d tokens) · %3$d memories · %4$d promises open</string>+ <string name="settings_notebook_dump">What is in the book</string>+ <string name="settings_notebook_dump_hint">Read-only. Long-press to select and copy.</string>+ <string name="settings_notebook_dump_truncated">… (%1$d more characters not shown)</string>+ <string name="settings_section_typing">Typing</string>+ <string name="settings_typing_no_ime">ADBKeyboard is the only keyboard on this device and it has no keyboard UI, so nothing can be typed right now. Enable one in system settings and it will show up here.</string>+ <string name="settings_typing_current">Your keyboard</string>+ <string name="settings_typing_note">Normally it is your own keyboard, and you can type in any app. While I am typing something for you it switches to ADBKeyboard (which has no keyboard UI — that is the only way it works), and switches straight back.</string>+ <string name="common_show">Show</string>+ <string name="common_hide">Hide</string>++ <!-- The pinned gesture tips at the bottom of the full card. The swipe hint+ is last because it is the one gesture that only exists unfolded — the+ folded ball reads a horizontal flick as a move instead. The legend+ style ("1tap: talk") is deliberate: the long-form sentence wrapped to+ two lines on the card, and a hint the user has to scroll past is+ longer than one they can read at a glance. -->+ <string name="window_tips">1tap: talk · 2tap: type · 3tap: stop · swipe: character</string>++ <!-- The factory-reset confirmation list; see config/FactoryReset.kt. -->+ <string name="factory_reset_target_keys">the Volcano and DeepSeek keys, and every speech and brain setting</string>+ <string name="factory_reset_target_notebook">the notebook: what it remembers and what it promised</string>+ <string name="factory_reset_target_vault">the accounts, passwords and bank cards kept in the vault</string>+ <string name="factory_reset_target_wake">the wake-word recordings (they will have to be recorded again)</string>+ <string name="factory_reset_target_history">the conversation history and the pages it generated</string>+ <string name="factory_reset_target_web">cookies and local storage left behind by the pages it drew</string>+ <string name="factory_reset_target_grounding">the grounding screenshots and logs</string>+ <string name="vault_title">Vault</string>+ <string name="vault_edit_title">Edit account</string>+ <string name="vault_intro">Accounts saved here let Andee log in for you when a task needs it.\nIt types the password into the field itself and never hands it to the remote brain.</string>+ <string name="vault_empty">No accounts saved yet.</string>+ <string name="vault_add_account">+ New account</string>+ <string name="vault_password_saved">Password saved</string>+ <string name="vault_summary_empty">(empty)</string>+ <string name="vault_delete">Delete</string>+ <string name="vault_field_name">Name</string>+ <string name="vault_field_name_hint">Feishu / work email — the brain finds it by this name</string>+ <string name="vault_field_phone">Phone</string>+ <string name="vault_field_email">Email</string>+ <string name="vault_field_username">Username</string>+ <string name="vault_field_password">Password</string>+ <string name="vault_password_note">The password is only typed into the field, never returned to the brain.</string>+ <string name="vault_field_note">Note</string>+ <string name="vault_field_note_hint">Hints like “sign in with an SMS code” — the brain can read these</string>+ <string name="vault_section_card">Bank card (optional)</string>+ <string name="vault_card_note">The number, expiry and CVV work like the password: typed into the field, never returned to the brain. Payment sheets in Alipay / WeChat / a banking app draw their own keypad and cannot be typed into — what this reaches is an ordinary checkout form or a card-binding page. Paying is still yours to confirm.</string>+ <string name="vault_field_card_holder">Cardholder</string>+ <string name="vault_field_card_holder_hint">The name printed on the card — the brain can read this</string>+ <string name="vault_field_card_number">Card number</string>+ <string name="vault_field_card_expiry">Expiry</string>+ <string name="vault_field_card_expiry_hint">MM/YY</string>+ <string name="vault_field_card_cvv">CVV</string>+ <string name="vault_label_card_last4">Card ending</string>+ <string name="vault_error_name_required">Name cannot be empty</string>+ <string name="wake_title">Teach me “Hey Andee”</string>+ <string name="wake_instructions">Tap Start, then say “Hey Andee” the way you normally call me — it stops on its own. You\'ll say it %1$d times — don\'t say it more carefully than usual, or I won\'t recognise it when you call me normally.</string>+ <string name="wake_progress">%1$d / %2$d</string>+ <string name="wake_status_enrolled">Already recorded — recording again will overwrite it.</string>+ <string name="wake_status_not_enrolled">Not recorded yet.</string>+ <string name="wake_close">Close</string>+ <string name="wake_start">Start</string>+ <string name="wake_listening">Listening…</string>+ <string name="wake_speak_now">Go ahead — “Hey Andee”</string>+ <string name="wake_retry">Try again</string>+ <string name="wake_not_heard">Didn\'t catch that — move closer, then wait half a second after you finish before checking the screen.</string>+ <string name="wake_say_again">Say it again</string>+ <string name="wake_received">Got it (%1$d ms). %2$d to go.</string>+ <string name="wake_rerecord">Record again</string>+ <string name="wake_done">Done — just say “Hey Andee” while the screen is on. (threshold %1$s)</string>+ <!-- device/LookActivity.kt -->+ <string name="dev_look_focusing">Focusing…</string>+ <string name="dev_look_seen">Got it</string>+ <string name="dev_look_unclear">Didn\'t catch it</string>+ <string name="dev_look_resting">Watching…</string>+ <string name="dev_look_resting_front">Front camera · watching…</string>+ <string name="dev_camera_unavailable">Can\'t open the camera</string>+ <!-- device/ScanActivity.kt -->+ <string name="dev_scan_align">Point at a QR code / barcode</string>+ <!-- device/NotificationRelayService.kt: the signboard shown when a+ message app pings. The app name is passed in, not translated here. -->+ <string name="dev_notif_card_title">New %1$s message</string>+ <string name="dev_notif_card_unread">New %1$s message (%2$d unread)</string>+ <string name="dev_notif_card_look">Take a look</string>+ <string name="dev_notif_card_ignore">Ignore</string>+ <!-- device/PermissionsController.kt: the one sentence the brain reads+ aloud, plus the per-grant phrase it is built from. The separator is+ quoted so its trailing space survives aapt's whitespace trim. -->+ <string name="dev_perm_feature_location">Location (where you are / navigation)</string>+ <string name="dev_perm_feature_steps">Step count</string>+ <string name="dev_perm_feature_contacts">Contacts (find someone to call)</string>+ <string name="dev_perm_feature_call_phone">Direct dialing</string>+ <string name="dev_perm_feature_call_log">Call history</string>+ <string name="dev_perm_feature_sms">Read SMS (verification codes)</string>+ <string name="dev_perm_feature_sms_send">Send SMS</string>+ <string name="dev_perm_feature_calendar">Calendar (reminders)</string>+ <string name="dev_perm_feature_calendar_write">Write calendar (add events)</string>+ <string name="dev_perm_feature_notifications">Notification access (know when you get messages)</string>+ <string name="dev_perm_feature_overlay">Overlay (my face)</string>+ <string name="dev_perm_say_prefix">Turn on: %1$s.</string>+ <string name="dev_perm_say_sep">", "</string>+ <string name="dev_perm_say_runtime">A dialog will pop up — just tap Allow.</string>+ <string name="dev_perm_say_notification">I\'ll open Notification access settings — find this app and turn it on.</string>+ <string name="history_empty">No conversation yet</string>+ <string name="history_role_you">You · %1$s</string>+ <string name="history_role_assistant">Assistant · %1$s</string>+ <string name="history_role_error">Error · %1$s</string>+ <string name="history_role_page_open">Page · %1$s · Tap to reopen</string>+ <string name="history_role_page_cleared">Page · %1$s · Cleared</string>+ <string name="html_default_title">AI page</string>+ <string name="textpage_role_you">You</string>+ <string name="textpage_role_assistant">Assistant</string>+ <string name="textpage_role_error">Error</string>+ <string name="textpage_role_page">Page</string>+ <string name="card_ack">Got it</string>+ <string name="bubble_tap_to_talk">Tap to talk</string>+ <string name="input_hint">Type to talk to it…</string>+ <string name="input_send">Send</string>+ <string name="cam_close">✕ Done</string>++ <!-- ScreenBodyService.kt — display-only; see the comment on the Chinese side. -->+ <string name="svc_call_in_progress">I\'m on a call — tell me once it\'s hung up.</string>+ <string name="svc_no_typable_ime">The only keyboard on this device is ADBKeyboard, and it has no UI — nothing can be typed.\nTurn on an input method in system settings and it will work.</string>+ <string name="svc_voice_error_line">Voice: %1$s</string>+ <string name="svc_voice_failed">Voice stopped working.\n%1$s</string>+ <string name="svc_btn_type_instead">Type it instead</string>+ <string name="svc_btn_open_settings">Open settings</string>+ <string name="svc_listening_again">(still listening…)</string>+ <string name="svc_stopped">Stopped</string>+ <string name="svc_meeting_ended_line">(Meeting recording ended — %1$d characters, drafting the minutes…)</string>+ <string name="svc_call_ended_line">(Call ended — %1$d characters, summarising…)</string>+ <string name="svc_no_brain_hub">No brain is connected yet, so I can\'t speak.</string>+ <string name="svc_no_brain_local">The local brain has no key yet, so I can\'t speak.</string>++ <string name="notebook_dump_header">"Notebook · %1$d memories\n"</string>+ <string name="notebook_dump_sort_note">"(Listed by type and most-recently updated; read-only)\n\n"</string>+ <string name="notebook_dump_empty_memories">"(Nothing written down yet)\n"</string>+ <string name="notebook_dump_expired">" [expired]"</string>+ <string name="notebook_dump_source">" · source "</string>+ <string name="notebook_dump_hits">" · recalled %1$d times\n\n"</string>+ <string name="notebook_dump_todos_header">"\nTodos · %1$d\n\n"</string>+ <string name="notebook_dump_empty_todos">"(none)\n"</string>+ <string name="notebook_dump_recurring">every %1$s</string>+ <string name="notebook_dump_next">" · next "</string>+ <string name="notebook_dump_last_error">" last error: "</string>+ <string name="sched_channel_name">Promises</string>+ <string name="sched_channel_desc">When the time comes but I can\'t wake up, at least say it clearly</string>+ <string name="sched_notify_recurring">Recurring todo: %1$s</string>+ <string name="sched_notify_title">I promised you this</string>+ <string name="sched_notify_body">%1$s\n\n(When the time came I wasn\'t running, so I couldn\'t do it for you. Turn on the accessibility service and I\'ll pick it back up.)</string>+ <string name="cmd_vault_filling">Typing the “%2$s” for %1$s…</string>+ <string name="cmd_ask_yes">OK</string>+ <string name="cmd_ask_no">No thanks</string>++ <string name="asr_listening">Listening…</string>+ <string name="meeting_live_head">Recording %1$d:%2$02d · tap the ball to stop</string>++ <!-- The names for a recording that never named itself; see the Chinese side. -->+ <string name="meeting_default_title">Meeting</string>+ <string name="call_default_scene">Call</string>++ <!-- LocalBrain — display half only; see the comment on the Chinese side. -->+ <string name="brain_no_key">The on-device brain has no API key yet — long-press the ball, open settings and put a DeepSeek key in the Brain section.</string>+ <string name="brain_error">Something broke on my side: %1$s</string>+ <string name="brain_no_result">I got nothing back that time — say it again.</string>+ <string name="brain_repeat_stop">I kept doing the same thing over and over (%1$s), so I stopped — that way is blocked. Try saying it differently, or take over.</string>+ <string name="brain_max_steps">I have spent %1$d steps on this without finishing, so I stopped to ask: shall I try another way, or will you take over?</string>+ <string name="narrate_get_screen">Reading the screen…</string>+ <string name="narrate_screenshot">Looking at the screen…</string>+ <string name="narrate_tap">Tapping…</string>+ <string name="narrate_long_press">Long-pressing…</string>+ <string name="narrate_swipe">Swiping…</string>+ <string name="narrate_type">Typing…</string>+ <string name="narrate_submit">Submitting…</string>+ <string name="narrate_wait">Waiting…</string>+ <string name="narrate_launch">Opening…</string>+ <string name="narrate_system">Pressing a system key…</string>+ <string name="narrate_show_html">Putting it on screen…</string>+ <string name="narrate_camera">Taking a look…</string>+ <string name="narrate_dog">Moving a leg…</string>+ <string name="narrate_default">Running %1$s…</string>++ <!--+ Startup self-check. The findings come from `device/SelfCheck` — one+ source, read by both the startup card and `ui/SelfCheckActivity`. The+ two layers are deliberate: `title` says what is being checked, `detail`+ says what was found and what breaks because of it. The second one is the+ half a person can act on, so it names the symptom rather than the+ setting.+ -->+ <string name="check_title_accessibility">Accessibility service</string>+ <string name="check_title_overlay">Overlay permission</string>+ <string name="check_title_network">Network</string>+ <string name="check_title_brain">Assistant</string>+ <string name="check_title_model">Model</string>+ <string name="check_title_microphone">Microphone</string>+ <string name="check_title_notifications">Notification access</string>+ <string name="check_title_adb_keyboard">ADB keyboard</string>+ <string name="check_title_secure_settings">Typing without root</string>+ <string name="check_title_voice_key">Voice API key</string>+ <string name="check_title_voice_host">Voice service</string>+ <string name="check_title_wake">Wake phrase</string>+ <string name="check_title_battery">Background running</string>+ <string name="check_title_disk">Storage</string>++ <string name="check_detail_accessibility_on">On. The ball and every screen action are running.</string>+ <string name="check_detail_accessibility_off">Off. No ball, no voice, no tapping or swiping — with this one off, the rest of this list does not matter yet.</string>+ <string name="check_detail_overlay_on">Allowed. The ball can be drawn.</string>+ <string name="check_detail_overlay_off">Off. Neither the ball nor any card can be drawn, while the service runs perfectly well in the background — which is why it looks like nothing happened at all.</string>+ <string name="check_detail_network_on">Online.</string>+ <string name="check_detail_network_off">Offline. Speech recognition, speech synthesis and every model call go over the network.</string>+ <string name="check_detail_brain_on">Configured.</string>+ <string name="check_detail_brain_hub_missing">The cloud address is empty — nothing knows where to send what you say.</string>+ <string name="check_detail_brain_key_missing">The on-device assistant has no API key, so asking it anything gets no answer.</string>+ <string name="check_detail_model_cloud">Cloud assistant — the model is not on this device.</string>+ <string name="check_detail_model_no_key">No key yet. Fill one in and this becomes testable.</string>+ <string name="check_detail_checking">Checking…</string>+ <string name="check_detail_skipped_offline">Offline, so not tested.</string>+ <string name="check_detail_model_ok">%1$s answered.</string>+ <string name="check_detail_model_throttled">Reachable but rate-limited (HTTP 429). The configuration is right; it recovers on its own.</string>+ <string name="check_detail_model_url">Could not connect: %1$s</string>+ <string name="check_detail_model_key">Key rejected: %1$s</string>+ <string name="check_detail_model_name">No model called “%1$s” on this server. It offers: %2$s</string>+ <string name="check_detail_model_other">%1$s</string>+ <string name="check_detail_notif_on">Granted, and the listener is bound.</string>+ <string name="check_detail_notif_zombie">Settings says “granted”, but the listener is not bound — notifications are being dropped silently.\nThis one only comes back if you turn it off and then on again; leaving it on does nothing.</string>+ <string name="check_detail_notif_off">Off. Call announcements and forwarded notifications never arrive.</string>+ <string name="check_detail_adb_kb_on">Enabled. It can type into other apps for you.</string>+ <string name="check_detail_adb_kb_off">Installed but not enabled. Turn it on in system settings — it has no keyboard UI of its own, so enabling it does not get in the way of typing yourself.</string>+ <string name="check_detail_adb_kb_missing">Not installed. Typing and filling in forms for you go through it (ADBKeyboard), so that does not work yet. Download it with the button here — when you come back this row is a one-tap enable.</string>+ <string name="check_detail_secure_on">Granted. Typing into other apps works.</string>+ <string name="check_detail_secure_missing">Not granted, so typing into other apps (type_text) is unavailable. There is no switch for this in Settings — it can only be given over adb with the cable plugged in.</string>+ <string name="check_detail_mic_on">Allowed.</string>+ <string name="check_detail_mic_off">Not allowed. Tapping the ball does nothing at all — the failure that least resembles a permission problem.</string>+ <string name="check_detail_voice_key_ok">Set. Recognition and synthesis use it to reach Volcengine.</string>+ <string name="check_detail_voice_key_missing">Not set. The source ships without a key — open a Volcengine Speech service of your own and put its API key in ⚙ under “Voice API key”. Without it, hearing and speaking cannot connect, and what you get back is a connection timeout that looks like a network fault. Everything else works.</string>+ <string name="check_detail_voice_host_ok">%1$s is reachable.</string>+ <string name="check_detail_voice_host_bad">Could not reach %1$s: %2$s. Both recognition and synthesis will time out.</string>+ <string name="check_detail_wake_on">Taught, and it wakes while the screen is on.</string>+ <string name="check_detail_wake_off">Not taught yet. Everything still works, you just have to tap the ball to talk.</string>+ <string name="check_detail_battery_on">Exempt, so pending reminders are not delayed.</string>+ <string name="check_detail_battery_off">Not exempt from battery optimisation. Tap “Open settings” — the system asks “Ignore battery optimisations?”; take the button that ignores them.\nThe vendor’s own “App launch / background running” controls are a separate system; changing them leaves this row red. With the screen off a pending reminder may arrive late, and the wake phrase stops listening.</string>+ <string name="check_detail_disk_ok">%1$d MB free.</string>+ <string name="check_detail_disk_low">Only %1$d MB left; screenshots and pages will start going missing.</string>+ <string name="check_detail_disk_full">Only %1$d MB left — almost nothing can be written.</string>+ <string name="check_detail_disk_unknown">Could not be read.</string>++ <string name="check_fix_settings">Open settings</string>+ <!-- The notification-access switch already reads as on in the zombie state,+ so that door is not “Open settings”. See SelfCheck.Fix.Screen.label. -->+ <string name="check_fix_repower">Off, then on</string>+ <string name="check_fix_download">Download</string>+ <string name="check_fix_allow">Allow</string>+ <string name="check_fix_app_settings">Open assistant settings</string>+ <string name="check_fix_adb_lead">This one can only be done from a computer, over adb:</string>+ <string name="check_fix_no_screen">This device has no such settings screen. You will have to find it in Settings yourself.</string>+ <string name="check_page_title">Self-check</string>+ <string name="check_page_lead">Where this device stands right now.</string>+ <string name="check_page_ok_title">All good</string>+ <string name="check_page_ok_note">Everything is set up. Tap the ball and talk.</string>+ <string name="check_page_recheck">Check again</string>+ <!-- The card's way out, in both places it appears: the ✕ in the header and+ the filled button in the footer. Deliberately the same word as the+ settings card's "Done" — these are the same card. -->+ <string name="check_page_done">Done</string>+ <string name="check_page_issues">%1$d to sort out</string>+ <string name="check_page_notes_lead">The rest is just for reference and changes nothing.</string>+ <string name="check_page_no_service">Accessibility is off, so the assistant’s settings cannot be opened from here yet.</string>+ <string name="check_card_title">%1$d thing(s) need attention</string>+ <string name="check_card_open">Fix it</string>+ <string name="check_card_ack">Got it</string>++ <!-- The ball's looks. The user swipes the ball sideways on the full card to+ change them, and the new one announces itself in the subtitle band —+ that name is read by a person, hence a resource rather than+ BallLook.name, which is for logs and diffs. -->+ <string name="ball_look_cream">Cream</string>+ <string name="ball_look_bajie">Pigsy</string>+ <string name="ball_look_imp">Imp</string>+</resources>diff --git a/app/src/main/res/values-night/themes.xml b/app/src/main/res/values-night/themes.xmlnew file mode 100644index 0000000..2ef60e9--- /dev/null+++ b/app/src/main/res/values-night/themes.xml@@ -0,0 +1,16 @@+<resources xmlns:tools="http://schemas.android.com/tools">+ <!-- Base application theme. -->+ <style name="Theme.Body" parent="Theme.MaterialComponents.DayNight.DarkActionBar">+ <!-- Primary brand color. -->+ <item name="colorPrimary">@color/purple_200</item>+ <item name="colorPrimaryVariant">@color/purple_700</item>+ <item name="colorOnPrimary">@color/black</item>+ <!-- Secondary brand color. -->+ <item name="colorSecondary">@color/teal_200</item>+ <item name="colorSecondaryVariant">@color/teal_200</item>+ <item name="colorOnSecondary">@color/black</item>+ <!-- Status bar color. -->+ <item name="android:statusBarColor">?attr/colorPrimaryVariant</item>+ <!-- Customize your theme here. -->+ </style>+</resources>\ No newline at end of filediff --git a/app/src/main/res/values/colors.xml b/app/src/main/res/values/colors.xmlnew file mode 100644index 0000000..f8c6127--- /dev/null+++ b/app/src/main/res/values/colors.xml@@ -0,0 +1,10 @@+<?xml version="1.0" encoding="utf-8"?>+<resources>+ <color name="purple_200">#FFBB86FC</color>+ <color name="purple_500">#FF6200EE</color>+ <color name="purple_700">#FF3700B3</color>+ <color name="teal_200">#FF03DAC5</color>+ <color name="teal_700">#FF018786</color>+ <color name="black">#FF000000</color>+ <color name="white">#FFFFFFFF</color>+</resources>\ No newline at end of filediff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xmlnew file mode 100644index 0000000..34e9add--- /dev/null+++ b/app/src/main/res/values/strings.xml@@ -0,0 +1,452 @@+<resources>+ <string name="app_name">Andee</string>+ <!--+ Shown under the service name in Settings → Accessibility, and again in+ the confirmation dialog. It is the one place the user is asked to hand+ over screen access, so it says what Andee does with it in their own+ language rather than naming the API.+ -->+ <string name="accessibility_service_description">开启后 Andee 才能看到屏幕上有什么,并代你点按、滑动、输入。桌面上会多一颗悬浮小球:点一下跟它说话,长按展开对话记录。</string>++ <!-- The notification listener, named in system settings. -->+ <string name="notification_relay_label">Andee 通知读取</string>++ <!--+ The settings sheet. Default language is Chinese: this file is `values/`+ so an unmatched locale falls back here, and the app has shipped Chinese+ from the start. `values-en/` mirrors it key for key — a key added here+ and not there silently shows Chinese on an English tablet.++ Not every Chinese literal in the app belongs in here. The policy — what+ the user reads versus what the model reads versus what matches system+ text — is written out at the top of `values-en/strings.xml`; read that+ before adding a string or "fixing" a deliberate omission.+ -->+ <string name="settings_title">设置</string>+ <string name="settings_default_device_name">这台设备</string>+ <string name="settings_factory_reset">恢复出厂设置</string>+ <string name="common_cancel">取消</string>+ <string name="common_save">保存</string>+ <string name="settings_factory_reset_confirm_title">恢复出厂设置?</string>+ <string name="settings_factory_reset_list_intro">下面这些会被删掉,删了就找不回来:</string>+ <string name="settings_factory_reset_after">设备会回到刚装好的样子:需要重新填 key,唤醒词要重新录。</string>+ <string name="settings_factory_reset_back">不,返回</string>+ <string name="settings_factory_reset_confirm">确认清空</string>+ <!-- The three destructive buttons sit together at the foot of the 诊断+ section, not in the footer beside 取消/保存: a destructive control+ beside the card's two ways out reads as a third way out. -->+ <string name="settings_clear_chat">清空聊天记录</string>+ <string name="settings_clear_chat_confirm_title">清空聊天记录?</string>+ <string name="settings_clear_chat_confirm_body">对话记录和它生成过的页面会删掉,这场对话聊过的内容它也会忘掉。小本本不受影响。</string>+ <string name="settings_clear_chat_confirm">确认清空</string>+ <!-- 小本本是跨对话留下来的东西,所以和清空聊天记录是两件事,各清各的。 -->+ <string name="settings_clear_notes">清空小本本</string>+ <string name="settings_clear_notes_confirm_title">清空小本本?</string>+ <string name="settings_clear_notes_confirm_body">它记下的事、你的偏好、还没到点的提醒,都会删掉,找不回来。这场聊天记录不受影响。</string>+ <string name="settings_clear_notes_confirm">确认清空</string>+ <string name="settings_section_brain">大脑</string>+ <string name="settings_brain_panel_note">关掉这个面板就生效,不用重开无障碍服务。</string>+ <string name="settings_brain_cloud">云端</string>+ <string name="settings_brain_local">本机</string>+ <string name="settings_hub_url">Hub 地址</string>+ <!-- The hub URL box is empty on a fresh install — the hint is what the+ user sees in that empty box, so it is the pitch, not the format. -->+ <string name="settings_hub_url_hint">链接你的云端Agent</string>+ <string name="settings_device_name">设备名</string>+ <string name="settings_device_name_hint">Hub 那边怎么称呼这台设备</string>+ <string name="settings_brain_cloud_note">大脑在你自己搭的 Hub 那边,这台设备只负责动手。</string>+ <string name="settings_llm_model">模型</string>+ <string name="settings_llm_base_url">接口地址</string>+ <string name="settings_llm_thinking">深度思考</string>+ <string name="common_on">开</string>+ <string name="common_off">关</string>+ <string name="settings_llm_effort">思考强度</string>+ <string name="common_high">高</string>+ <string name="common_medium">中</string>+ <string name="common_low">低</string>+ <string name="settings_brain_local_note">设备自己跑 agent loop,不连 Hub。没有 Key 就不会接管。</string>+ <string name="settings_section_wake">语音唤醒</string>+ <string name="settings_wake_enrolled">已录入 —— 亮屏时说「嘿 Andee」就行</string>+ <string name="settings_wake_not_enrolled">未录入 —— 录一次才能语音唤醒</string>+ <string name="settings_wake_record">录入</string>+ <string name="common_clear">清除</string>+ <string name="settings_section_vault">保险箱</string>+ <string name="settings_vault_empty_note">还没有存账号 —— 存了之后,Andee 办事时可以直接用它们登录</string>+ <string name="settings_vault_count_note">已存 %1$d 个账号 —— 手机号邮箱大脑读得到,密码只由设备替你敲进输入框</string>+ <string name="settings_vault_manage">管理</string>+ <string name="settings_section_voice">语音(火山引擎)</string>+ <string name="settings_voice_factory_note">源码不带密钥。填上自己的 Key 才听得见、说得出;下面几行留空用默认地址即可。</string>+ <string name="settings_voice_api_key">火山 API Key</string>+ <string name="settings_voice_api_key_hint">火山引擎语音服务的 Key,听和说都靠它</string>+ <string name="settings_voice_in_use">现在用的</string>+ <string name="settings_voice_in_use_own">你自己填的那个火山账号</string>+ <string name="settings_voice_in_use_factory">还没填,听和说都用不了</string>+ <string name="settings_voice_asr_endpoint">识别地址</string>+ <string name="settings_voice_asr_resource_id">识别资源 ID</string>+ <string name="settings_voice_tts_endpoint">合成地址</string>+ <string name="settings_voice_tts_resource_id">合成资源 ID</string>+ <string name="settings_voice_tts_sample_rate">合成采样率</string>+ <string name="settings_section_diagnostics">诊断</string>+ <string name="settings_grounding">接地采集</string>+ <string name="settings_grounding_note">把坐标点击和截图记到 files/grounding 供离线分析 —— 它会拍下你当时在做的事。改完要重开无障碍服务才生效。</string>+ <string name="settings_device_id">设备 ID</string>+ <string name="settings_section_notebook">小本本</string>+ <string name="settings_notebook_sweep_note">聊完静下来之后,它会自己回头看一眼:该记的记下、该跟的排上。默认开着 —— 「它记得住」就是这么来的。关掉之后不再花你的 key 做这件事。</string>+ <string name="settings_notebook_quiet_minutes">静默多久算聊完(分钟)</string>+ <string name="settings_notebook_daily_cap">每天最多复盘几次</string>+ <string name="settings_notebook_today">今天 / 攒到现在</string>+ <string name="settings_notebook_stats">复盘 %1$d 次(约 %2$d tokens)· 记住 %3$d 条 · 还欠着 %4$d 件事</string>+ <string name="settings_notebook_dump">本子里写了什么</string>+ <string name="settings_notebook_dump_hint">只读。长按可以选中复制。</string>+ <string name="settings_notebook_dump_truncated">…(还有 %1$d 个字符没显示)</string>+ <string name="settings_section_typing">打字</string>+ <string name="settings_typing_no_ime">这台设备上只有 ADBKeyboard(它没有键盘界面),所以现在打不了字。去系统设置里启用一个输入法,这里就会出现。</string>+ <string name="settings_typing_current">你现在的输入法</string>+ <string name="settings_typing_note">平时就是你自己的输入法 —— 任何 App 里都能正常打字。我替你打字时会临时切到 ADBKeyboard(它没有键盘界面,只能这么用),打完立刻切回来。</string>+ <string name="common_show">显示</string>+ <string name="common_hide">隐藏</string>++ <!-- The pinned gesture tips at the bottom of the full card. The swipe hint+ is last because it is the one gesture that only exists unfolded — the+ folded ball reads a horizontal flick as a move instead. -->+ <string name="window_tips">点小球说话 · 双击打字 · 三连击停止 · 滑动切换性格</string>++ <!-- The 恢复出厂设置 confirmation list: one line per thing wiped.+ Kept beside config/FactoryReset.kt, which walks them in this order. -->+ <string name="factory_reset_target_keys">火山 key、DeepSeek key,以及所有语音与大脑的设置</string>+ <string name="factory_reset_target_notebook">小本本:记住的事、答应过还没办的事</string>+ <string name="factory_reset_target_vault">保险箱里存的账号密码和银行卡</string>+ <string name="factory_reset_target_wake">唤醒词的录音(要重新录一遍)</string>+ <string name="factory_reset_target_history">对话历史,和它生成过的页面</string>+ <string name="factory_reset_target_web">内置页面留下的 cookie 与本地存储</string>+ <string name="factory_reset_target_grounding">接地采集存下的截图与日志</string>+ <string name="vault_title">保险箱</string>+ <string name="vault_edit_title">编辑账号</string>+ <string name="vault_intro">存在这里的账号,Andee 办事时可以直接用来登录。\n密码只会被它替你敲进输入框,不会交给远端的大脑。</string>+ <string name="vault_empty">还没有存任何账号。</string>+ <string name="vault_add_account">+ 新增账号</string>+ <string name="vault_password_saved">密码 已存</string>+ <string name="vault_summary_empty">(空)</string>+ <string name="vault_delete">删除</string>+ <string name="vault_field_name">名称</string>+ <string name="vault_field_name_hint">飞书 / 公司邮箱 —— 大脑按这个名字找</string>+ <string name="vault_field_phone">手机号</string>+ <string name="vault_field_email">邮箱</string>+ <string name="vault_field_username">用户名</string>+ <string name="vault_field_password">密码</string>+ <string name="vault_password_note">密码只会被敲进输入框,不会返回给大脑。</string>+ <string name="vault_field_note">备注</string>+ <string name="vault_field_note_hint">「用手机验证码登录」之类的提示,大脑读得到</string>+ <string name="vault_section_card">银行卡(可选)</string>+ <string name="vault_card_note">卡号、有效期、安全码和密码一样,只会被敲进输入框,不会返回给大脑。支付宝 / 微信 / 银行 App 的支付键盘是它们自己画的,敲不进去 —— 这里填的是网页或 App 的结账表单、绑卡页。付款本身仍然由你来确认。</string>+ <string name="vault_field_card_holder">持卡人</string>+ <string name="vault_field_card_holder_hint">卡上印的名字,大脑读得到</string>+ <string name="vault_field_card_number">卡号</string>+ <string name="vault_field_card_expiry">有效期</string>+ <string name="vault_field_card_expiry_hint">MM/YY</string>+ <string name="vault_field_card_cvv">安全码 CVV</string>+ <string name="vault_label_card_last4">尾号</string>+ <string name="vault_error_name_required">名称不能为空</string>+ <string name="wake_title">教我认「嘿 Andee」</string>+ <string name="wake_instructions">点「开始」,然后像平时叫我那样说一遍「嘿 Andee」,说完会自动停。一共要说 %1$d 次 —— 别刻意念得比平时清楚,不然平时那样叫我反而认不出来。</string>+ <string name="wake_progress">%1$d / %2$d</string>+ <string name="wake_status_enrolled">已经录过了,再录一次会覆盖。</string>+ <string name="wake_status_not_enrolled">还没录过。</string>+ <string name="wake_close">关闭</string>+ <string name="wake_start">开始</string>+ <string name="wake_listening">听着…</string>+ <string name="wake_speak_now">说吧 —— 「嘿 Andee」</string>+ <string name="wake_retry">再来一次</string>+ <string name="wake_not_heard">没听清 —— 离近一点,说完等半秒再看屏幕。</string>+ <string name="wake_say_again">再说一次</string>+ <string name="wake_received">收到(%1$d 毫秒)。还差 %2$d 次。</string>+ <string name="wake_rerecord">重录</string>+ <string name="wake_done">好了,亮屏的时候叫我一声「嘿 Andee」就行。(阈值 %1$s)</string>+ <!-- device/LookActivity.kt -->+ <string name="dev_look_focusing">正在对焦…</string>+ <string name="dev_look_seen">看到了</string>+ <string name="dev_look_unclear">没看清</string>+ <string name="dev_look_resting">正在看…</string>+ <string name="dev_look_resting_front">前置 · 正在看…</string>+ <string name="dev_camera_unavailable">摄像头打不开</string>+ <!-- device/ScanActivity.kt -->+ <string name="dev_scan_align">对准二维码 / 条形码</string>+ <!-- device/NotificationRelayService.kt: the signboard shown when a+ message app pings. The app name is passed in, not translated here. -->+ <string name="dev_notif_card_title">收到%1$s消息</string>+ <string name="dev_notif_card_unread">收到%1$s消息(%2$d 条未读)</string>+ <string name="dev_notif_card_look">帮我看看</string>+ <string name="dev_notif_card_ignore">忽略</string>+ <!-- device/PermissionsController.kt: the one sentence the brain reads+ aloud, plus the per-grant phrase it is built from. -->+ <string name="dev_perm_feature_location">位置(在哪/导航)</string>+ <string name="dev_perm_feature_steps">运动步数</string>+ <string name="dev_perm_feature_contacts">通讯录(打电话找人)</string>+ <string name="dev_perm_feature_call_phone">直接拨号</string>+ <string name="dev_perm_feature_call_log">通话记录</string>+ <string name="dev_perm_feature_sms">读短信(验证码)</string>+ <string name="dev_perm_feature_sms_send">发短信</string>+ <string name="dev_perm_feature_calendar">日历(日程提醒)</string>+ <string name="dev_perm_feature_calendar_write">写日历(记日程)</string>+ <string name="dev_perm_feature_notifications">通知感知(微信来消息我知道)</string>+ <string name="dev_perm_feature_overlay">悬浮窗(我的脸)</string>+ <string name="dev_perm_say_prefix">需要开启: %1$s。</string>+ <string name="dev_perm_say_sep">、</string>+ <string name="dev_perm_say_runtime">会弹窗,点允许即可。</string>+ <string name="dev_perm_say_notification">会打开通知使用权设置,找到这个应用打开开关。</string>+ <string name="history_empty">还没有对话</string>+ <string name="history_role_you">你 · %1$s</string>+ <string name="history_role_assistant">助手 · %1$s</string>+ <string name="history_role_error">出错 · %1$s</string>+ <string name="history_role_page_open">界面 · %1$s · 点击重新打开</string>+ <string name="history_role_page_cleared">界面 · %1$s · 已清理</string>+ <string name="html_default_title">AI 界面</string>+ <string name="textpage_role_you">你</string>+ <string name="textpage_role_assistant">助手</string>+ <string name="textpage_role_error">出错</string>+ <string name="textpage_role_page">界面</string>+ <string name="card_ack">知道了</string>+ <string name="bubble_tap_to_talk">点击说话</string>+ <string name="input_hint">打字跟它说…</string>+ <string name="input_send">发送</string>+ <string name="cam_close">✕ 完成</string>++ <!--+ ScreenBodyService.kt: the lines this file shows on its own — the ball's+ chrome, and the cards it pops when the microphone or the brain is not+ there.++ Every one of these is display-only. The neighbouring Chinese in that+ file is *not*: `meetingMinutesQuery`, `callSummaryQuery`, the+ `LocalBrain.note` stage direction and the todo wake-up prompts are text+ written **to the model**, and the language of a prompt decides the+ language of the answer. They are deliberately left alone — an English+ interface must not become a Chinese reply. See the note at the top of+ `values-en/strings.xml`.++ `%1$s`/`%1$d` are substitutions, not literal text. `ADBKeyboard` and+ `key` stay as they are: they are the name of an app and of a field in+ the settings card, and neither is translated on either side.+ -->+ <string name="svc_call_in_progress">通话中,等挂断了再跟我说吧</string>+ <string name="svc_no_typable_ime">这台设备上只有 ADBKeyboard(它没有键盘界面),打不了字。\n去系统设置里启用一个输入法就行。</string>+ <string name="svc_voice_error_line">语音:%1$s</string>+ <string name="svc_voice_failed">语音用不了了。\n%1$s</string>+ <!--+ The two buttons off the failure cards, and the shared dismiss. They are+ compared by identity in the code that builds them, so the label a+ handler tests against and the label the user reads are always the same+ lookup — see `reportVoiceFailure`.+ -->+ <string name="svc_btn_type_instead">打字说</string>+ <string name="svc_btn_open_settings">打开设置</string>+ <string name="svc_listening_again">(继续听…)</string>+ <string name="svc_stopped">已停止</string>+ <!--+ The two parenthesised rows dropped into the scrollback when a recording+ winds itself up. They read as a stage direction on purpose, mirroring+ the shape the brain is handed — but they are only ever displayed:+ `ChatHistory` is read by `HistoryListView` and nothing else, and the+ model is given `meetingMinutesQuery`/`callSummaryQuery` instead.+ -->+ <string name="svc_meeting_ended_line">(会议记录结束,共 %1$d 字,正在整理纪要…)</string>+ <string name="svc_call_ended_line">(通话结束,共 %1$d 字,正在整理…)</string>+ <string name="svc_no_brain_hub">还没有接上大脑,我说不上话。</string>+ <string name="svc_no_brain_local">本地大脑还没配 key,我说不上话。</string>++ <!--+ config/Notebook.kt — the read-only dump the settings sheet renders. The+ values that end in a space or a newline are quoted, which is what keeps+ aapt from trimming them. The *other* strings in Notebook (the "name 不能+ 为空" family) are tool results written to the model and stay Chinese.+ -->+ <string name="notebook_dump_header">"小本本 · 记忆 %1$d 条\n"</string>+ <string name="notebook_dump_sort_note">"(下面按类型和最近更新排列,只读)\n\n"</string>+ <string name="notebook_dump_empty_memories">"(还没有记下任何东西)\n"</string>+ <string name="notebook_dump_expired">" [已过期]"</string>+ <string name="notebook_dump_source">" · 来源 "</string>+ <string name="notebook_dump_hits">" · 被查过 %1$d 次\n\n"</string>+ <string name="notebook_dump_todos_header">"\n待办 %1$d 条\n\n"</string>+ <string name="notebook_dump_empty_todos">"(没有)\n"</string>+ <string name="notebook_dump_recurring">周期 %1$s</string>+ <string name="notebook_dump_next">" · 下次 "</string>+ <string name="notebook_dump_last_error">" 上次出错:"</string>++ <!--+ config/Scheduler.kt. `sched_channel_name` and `sched_channel_desc` are+ the notification channel; see the KDoc on `channelName` for why a+ language change does not reach a channel that already exists.+ -->+ <string name="sched_channel_name">答应过的事</string>+ <string name="sched_channel_desc">到点了但我醒不过来时,先把话说清楚</string>+ <string name="sched_notify_recurring">周期待办:%1$s</string>+ <string name="sched_notify_title">我当时答应过你</string>+ <string name="sched_notify_body">%1$s\n\n(到点的时候我没有在运行,所以没能替你去办。打开无障碍服务之后我会接着办。)</string>++ <!-- net/CommandDispatcher.kt — the three lines from it a user can see. -->+ <string name="cmd_vault_filling">正在填入「%1$s」的%2$s…</string>+ <string name="cmd_ask_yes">好的</string>+ <string name="cmd_ask_no">不用</string>++ <!-- The subtitle while the mic is open, before a word has landed. -->+ <string name="asr_listening">听着…</string>++ <!--+ MeetingRecorder: the live header while a recording runs. Two numbers,+ so `%1$d` is minutes and `%2$02d` is zero-padded seconds — keep the+ order if you reword it.+ -->+ <string name="meeting_live_head">会议记录中 %1$d:%2$02d · 点球结束</string>++ <!--+ The two names for a recording that never named itself.++ `meeting_default_title` is written by `MeetingRecorder.start` and read+ back by `ScreenBodyService.meetingMinutesQuery` — one key for both, on+ purpose: the brain is handed the title and the page is laid out under+ the same one, so two literals that drifted apart would show up as a page+ titled one thing and a brain told another.++ `call_default_scene` is the `callSummaryQuery` twin, for an extension+ that registered without a `scene`.++ Both were Kotlin literals until now, which is the same bug in two+ places: the value does reach the screen (it becomes the page title the+ minutes or the summary is laid out under), so on an English tablet it+ read Chinese. They follow the interface language like every other+ displayed string — see `AppLocale`, and the header of+ `values-en/strings.xml`.+ -->+ <string name="meeting_default_title">会议记录</string>+ <string name="call_default_scene">通话</string>++ <!--+ LocalBrain, the half of it the user sees.++ This class is a genuine split and it is not obvious from the file: its+ `onProgress` and `onFinal` strings become the ball's subtitle and a row+ in the scrollback, so they belong here — while its `toolMessage` and+ tool-result strings are read by the model and belong nowhere near this+ file. `brain_error`, `brain_repeat_stop` and `narrate_default` carry a+ tool or exception name through `%1$s`; that name is an identifier and+ stays as it is in both languages.+ -->+ <string name="brain_no_key">本地大脑还没配 API key —— 长按小球打开设置,把 DeepSeek 的 key 填进「大脑」那一栏。</string>+ <string name="brain_error">我这边出错了:%1$s</string>+ <string name="brain_no_result">我这轮没拿到结果,你再说一遍试试。</string>+ <string name="brain_repeat_stop">我一直在重复同一个动作(%1$s),停下来了 —— 这条路走不通,换个说法或者你来操作一下。</string>+ <string name="brain_max_steps">这件事我做了 %1$d 步还没办完,先停下来问你一句:要不要换个路子,或者你来接手?</string>+ <!--+ `narrate()`: the one-line subtitle for a tool the model called without+ saying anything. Chinese and English here are deliberately terse —+ they are read at a glance while a task is running, not read carefully.+ -->+ <string name="narrate_get_screen">读一下屏幕…</string>+ <string name="narrate_screenshot">看一眼屏幕…</string>+ <string name="narrate_tap">点一下…</string>+ <string name="narrate_long_press">长按…</string>+ <string name="narrate_swipe">滑动…</string>+ <string name="narrate_type">输入文字…</string>+ <string name="narrate_submit">提交…</string>+ <string name="narrate_wait">等一下…</string>+ <string name="narrate_launch">打开…</string>+ <string name="narrate_system">操作系统按键…</string>+ <string name="narrate_show_html">把结果放到屏幕上…</string>+ <string name="narrate_camera">看一眼…</string>+ <string name="narrate_dog">动一下腿…</string>+ <string name="narrate_default">正在 %1$s…</string>++ <!--+ 开机自检。发现由 device/SelfCheck 产出(设备侧、只有一个来源),+ ScreenBodyService 拿它弹一张卡、ui/SelfCheckActivity 拿它渲染成一页。+ 文案分两层:title 说「在查什么」,detail 说「查到什么、坏了会怎样」——+ 后者才是用户能读懂的那半,所以每条都写症状而不是写参数名。+ -->+ <string name="check_title_accessibility">无障碍服务</string>+ <string name="check_title_overlay">悬浮窗权限</string>+ <string name="check_title_network">网络</string>+ <string name="check_title_brain">大脑</string>+ <string name="check_title_model">模型连通</string>+ <string name="check_title_microphone">麦克风</string>+ <string name="check_title_notifications">通知使用权</string>+ <string name="check_title_adb_keyboard">ADB 键盘</string>+ <string name="check_title_secure_settings">免 root 输入</string>+ <string name="check_title_voice_key">语音密钥</string>+ <string name="check_title_voice_host">语音服务可达</string>+ <string name="check_title_wake">唤醒词</string>+ <string name="check_title_battery">后台运行</string>+ <string name="check_title_disk">存储空间</string>++ <string name="check_detail_accessibility_on">已开启,球和屏幕操作都在跑。</string>+ <string name="check_detail_accessibility_off">没开。球、语音、点按滑动全都不存在 —— 这条不开,后面几条都不用看。</string>+ <string name="check_detail_overlay_on">已允许,球能画出来。</string>+ <string name="check_detail_overlay_off">没开。球和卡片都画不出来,而服务在后台其实跑得好好的 —— 所以看起来像「什么都没发生」。</string>+ <string name="check_detail_network_on">已联网。</string>+ <string name="check_detail_network_off">没联网。语音识别、语音合成、模型调用全都走网络。</string>+ <string name="check_detail_brain_on">已配置。</string>+ <string name="check_detail_brain_hub_missing">云端大脑地址是空的 —— 还不知道该把你说的话送去哪。</string>+ <string name="check_detail_brain_key_missing">本地大脑没填 API Key,问了也不会有回答。</string>+ <string name="check_detail_model_cloud">云端大脑,模型不在这台设备上。</string>+ <string name="check_detail_model_no_key">还没填 Key,填了才测得了。</string>+ <string name="check_detail_checking">检查中…</string>+ <string name="check_detail_skipped_offline">没网,先不测。</string>+ <string name="check_detail_model_ok">%1$s 通了。</string>+ <string name="check_detail_model_throttled">连上了,但被限流(HTTP 429)。配置是对的,过一会儿会自己恢复。</string>+ <string name="check_detail_model_url">连不上:%1$s</string>+ <string name="check_detail_model_key">Key 被拒:%1$s</string>+ <string name="check_detail_model_name">模型名「%1$s」不在服务端列表里。这台服务上有的:%2$s</string>+ <string name="check_detail_model_other">%1$s</string>+ <string name="check_detail_notif_on">已授权,监听器也连着。</string>+ <string name="check_detail_notif_zombie">系统显示「已授权」,但监听器没连上,通知会被静默丢掉。\n这条得关掉再打开一次才行 —— 只打开不生效。</string>+ <string name="check_detail_notif_off">没开。来电播报和通知转发都收不到。</string>+ <string name="check_detail_adb_kb_on">已启用,能替你往别的应用里打字。</string>+ <string name="check_detail_adb_kb_off">装了但没启用。去系统设置里把它打开就行 —— 它自己没有键盘界面,启用它不影响你平时打字。</string>+ <string name="check_detail_adb_kb_missing">没装。替你打字、填表都靠它(ADBKeyboard),所以这类事现在做不了。点右边下载装上,回来这条就只剩「启用」一步。</string>+ <string name="check_detail_secure_on">已授权,能往别的应用里打字。</string>+ <string name="check_detail_secure_missing">没授权,往别的应用里打字(type_text)用不了。这条在设置里没有开关 —— 只能插上线用 adb 给。</string>+ <string name="check_detail_mic_on">已允许。</string>+ <string name="check_detail_mic_off">没允许。点球不会有任何反应 —— 这是最不像权限问题的一条。</string>+ <string name="check_detail_voice_key_ok">已填。识别和合成用它连火山。</string>+ <string name="check_detail_voice_key_missing">还没填。源码里不带密钥,得你自己去火山引擎开一个语音服务,把 API Key 填进 ⚙ 的「语音密钥」。没有它,听和说都连不上——报出来的却是连接超时,看着像网络坏了。其余功能照常。</string>+ <string name="check_detail_voice_host_ok">%1$s 可达。</string>+ <string name="check_detail_voice_host_bad">连不上 %1$s:%2$s。识别和合成都会超时。</string>+ <string name="check_detail_wake_on">已录入,亮屏时喊得醒。</string>+ <string name="check_detail_wake_off">还没录。不录也能用,只是喊不醒 —— 得点球。</string>+ <string name="check_detail_battery_on">已在白名单,待办闹钟不会被推迟。</string>+ <string name="check_detail_battery_off">不在电池优化白名单里。点右边「去开启」,系统会问「是否忽略电池优化?」,选「忽略」就好。\n厂商自己的「应用启动管理 / 后台运行」是另一套,在那里怎么改这项都是红的。息屏后待办提醒会晚到,唤醒词也会停。</string>+ <string name="check_detail_disk_ok">剩余 %1$d MB。</string>+ <string name="check_detail_disk_low">只剩 %1$d MB,再占下去会开始丢截图和页面。</string>+ <string name="check_detail_disk_full">只剩 %1$d MB,基本写不进东西了。</string>+ <string name="check_detail_disk_unknown">读不出来。</string>++ <string name="check_fix_settings">去开启</string>+ <!-- 通知使用权在「僵尸」态时那个开关本来就显示成已打开,所以这个门不是+ 「去开启」。见 SelfCheck.Fix.Screen.label。 -->+ <string name="check_fix_repower">关闭再打开</string>+ <string name="check_fix_download">去下载</string>+ <string name="check_fix_allow">允许</string>+ <string name="check_fix_app_settings">打开设置</string>+ <string name="check_fix_adb_lead">这条只能在电脑上用 adb 执行:</string>+ <string name="check_fix_no_screen">这台设备上没有这个设置页面。得自己在系统设置里找一下。</string>+ <string name="check_page_title">自检</string>+ <string name="check_page_lead">这台设备现在的状态。</string>+ <string name="check_page_ok_title">一切正常</string>+ <string name="check_page_ok_note">都配好了,去点球说话吧。</string>+ <string name="check_page_recheck">重新检查</string>+ <!-- 卡片的出口,两处都用它:头部的 ✕ 和底部的实心按钮。故意和设置卡+ 「完成」是同一个词 —— 这就是同一张卡。 -->+ <string name="check_page_done">完成</string>+ <string name="check_page_issues">%1$d 项需要处理</string>+ <string name="check_page_notes_lead">下面几条仅供参考,不影响使用。</string>+ <string name="check_page_no_service">无障碍还没开,设置面板暂时打不开。</string>+ <string name="check_card_title">有 %1$d 项需要处理</string>+ <string name="check_card_open">去处理</string>+ <string name="check_card_ack">知道了</string>++ <!-- 球球的几种样子。用户在全屏卡片上左右滑动球球来换,换到哪个就用字幕+ 报一下名字 —— 这个名字是给人看的,所以是资源,而不是 BallLook.name+ (那个是给日志和 diff 看的)。 -->+ <string name="ball_look_cream">奶油</string>+ <string name="ball_look_bajie">八戒</string>+ <string name="ball_look_imp">小恶魔</string>+</resources>diff --git a/app/src/main/res/values/themes.xml b/app/src/main/res/values/themes.xmlnew file mode 100644index 0000000..0f2794c--- /dev/null+++ b/app/src/main/res/values/themes.xml@@ -0,0 +1,72 @@+<resources xmlns:tools="http://schemas.android.com/tools">+ <!-- Base application theme. -->+ <style name="Theme.Body" parent="Theme.MaterialComponents.DayNight.DarkActionBar">+ <!-- Primary brand color. -->+ <item name="colorPrimary">@color/purple_500</item>+ <item name="colorPrimaryVariant">@color/purple_700</item>+ <item name="colorOnPrimary">@color/white</item>+ <!-- Secondary brand color. -->+ <item name="colorSecondary">@color/teal_200</item>+ <item name="colorSecondaryVariant">@color/teal_700</item>+ <item name="colorOnSecondary">@color/black</item>+ <!-- Status bar color. -->+ <item name="android:statusBarColor">?attr/colorPrimaryVariant</item>+ <!-- Customize your theme here. -->+ </style>++ <!--+ 三个全屏 Activity(HtmlActivity / LookActivity / ScanActivity)共用。+ 继承 Theme.Body 会拿到 colorPrimaryVariant 当状态栏色,也就是一条紫杠;+ 而 DayNight 的浅色 windowBackground 会让每次打开先白闪一下再变深。+ 两条都和 app 其余部分(纯黑 + #7FA5FF,见 ui/Glass.kt)无关,所以这里+ 全部按住。windowBackground 就是 Glass 的底色:overlay 卡片、弹框和这+ 三张全屏脸是同一套配色,开屏那一帧不该是例外。+ -->+ <style name="Theme.Body.Fullscreen" parent="Theme.Body">+ <!-- DarkActionBar hands every one of these an "Andee" title bar over+ the brain's content, and on the camera screens it covered the ✕+ pill outright. HtmlActivity used to hide it in code; doing it here+ means a new full-screen face can't forget. -->+ <item name="windowActionBar">false</item>+ <item name="windowNoTitle">true</item>+ <item name="android:windowBackground">#FF000000</item>+ <item name="android:statusBarColor">@android:color/transparent</item>+ <item name="android:navigationBarColor">@android:color/transparent</item>+ <item name="android:windowLightStatusBar">false</item>+ <item name="android:windowLightNavigationBar">false</item>+ </style>++ <!--+ 三个全屏脸(HtmlActivity / LookActivity / ScanActivity)现在共用的入场+ 与退场,见 ui/Stage.kt。++ windowAnimationStyle="@null" 是这里唯一新增的一项,也是"关掉时闪一下"+ 的真正原因:继承链上没人动过这一项,所以平台在 finish() 之后照常+ 播放它自己的退出动画 —— 动画的内容已经不在屏幕上,淡出的只是一片+ 空窗口;而底下 overlay 卡片是在 onDestroy 之后才开始恢复的。两段+ 动作因此错开,中间那一帧谁也不是,那就是用户看到的闪。++ 改成"淡入同时缩放"和"淡出同时缩小"由 StageActivity 自己画,平台的+ 这一项只需要闭嘴。enter/exit 都留 0,是因为 enter 的动画在+ onCreate 之后才起,平台那一帧已经画完了 —— 留 0 是不给它任何机会。+ -->+ <style name="Theme.Body.Stage" parent="Theme.Body.Fullscreen">+ <item name="android:windowAnimationStyle">@null</item>+ </style>++ <!--+ 桌面图标(SelfCheckActivity)专用。服务在跑的时候,那颗图标的意思就是+ "打开 app":这一路上 Activity 一个像素都不画,expand() 完就 finish ——+ 见 SelfCheckActivity.onCreate。但平台在 Activity 首次绘制之前会先按+ windowBackground 铺一张 starting window,于是体感变成"点图标 → 屏幕+ 黑一下 → 卡片长出来",而长按球球没有这一下。关掉它,两条路的入场才是+ 同一件事。++ 无障碍关着时这一项也无害:那一路本来就是纯黑底 + revealStage 从 0.94+ 长出来,起始帧是黑是空没有区别,而卡片第一帧由 SelfCheck.local 同步+ 给出,不存在等网络的黑屏。+ -->+ <style name="Theme.Body.Stage.NoPreview" parent="Theme.Body.Stage">+ <item name="android:windowDisablePreview">true</item>+ </style>+</resources>\ No newline at end of filediff --git a/app/src/main/res/xml/accessibility_service_config.xml b/app/src/main/res/xml/accessibility_service_config.xmlnew file mode 100644index 0000000..eb03d60--- /dev/null+++ b/app/src/main/res/xml/accessibility_service_config.xml@@ -0,0 +1,12 @@+<?xml version="1.0" encoding="utf-8"?>+<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"+ android:accessibilityEventTypes="typeWindowStateChanged|typeWindowContentChanged|typeWindowsChanged|typeViewFocused|typeViewScrolled"+ android:accessibilityFeedbackType="feedbackGeneric"+ android:accessibilityFlags="flagDefault|flagIncludeNotImportantViews|flagReportViewIds|flagRetrieveInteractiveWindows|flagRequestTouchExplorationMode"+ android:canPerformGestures="true"+ android:canRequestEnhancedWebAccessibility="true"+ android:canRequestTouchExplorationMode="true"+ android:canRetrieveWindowContent="true"+ android:canTakeScreenshot="true"+ android:description="@string/accessibility_service_description"+ android:notificationTimeout="100" />diff --git a/app/src/main/res/xml/backup_rules.xml b/app/src/main/res/xml/backup_rules.xmlnew file mode 100644index 0000000..f4fd044--- /dev/null+++ b/app/src/main/res/xml/backup_rules.xml@@ -0,0 +1,14 @@+<?xml version="1.0" encoding="utf-8"?><!--+ The pre-Android-12 half of the same rule: nothing leaves the device.+ See res/xml/data_extraction_rules.xml for why.++ `minSdk` is 30, so this file governs exactly one API level (30) — 31 and up+ read the other one. It is dead while `android:allowBackup="false"` stands, and+ it is filled in anyway so that flipping that attribute does not quietly start+ uploading the vault.+-->+<full-backup-content>+ <exclude domain="root" path="." />+ <exclude domain="external" path="." />+ <exclude domain="device_root" path="." />+</full-backup-content>diff --git a/app/src/main/res/xml/data_extraction_rules.xml b/app/src/main/res/xml/data_extraction_rules.xmlnew file mode 100644index 0000000..f3a67e4--- /dev/null+++ b/app/src/main/res/xml/data_extraction_rules.xml@@ -0,0 +1,32 @@+<?xml version="1.0" encoding="utf-8"?><!--+ Nothing this app stores may leave the device.++ The whole dataset is credentials and a record of the user's life: two API keys+ in plaintext (`voice_prefs`), the vault (`vault_prefs` — logins and card+ numbers), the notebook, meeting transcripts under files/meetings/, and, in+ grounding mode, screenshots of whatever was on screen. There is no+ non-sensitive remainder worth carrying to a new device: the endpoints have+ compiled-in defaults, and `device_id` *should* be regenerated per device,+ because two bodies registering the same id with one hub is a bug.++ `android:allowBackup="false"` is the other half of this and does not replace+ it: on Android 12+ it stops cloud backup but leaves **device-to-device+ transfer** permitted, which is what <device-transfer> below is for. Both are+ set so that flipping one back on does not silently re-enable the other.++ The vault would not survive a restore anyway — its key lives in the hardware+ Keystore and does not travel — but an undecryptable blob arriving on a new+ device is pure downside, not a safeguard.+-->+<data-extraction-rules>+ <cloud-backup>+ <exclude domain="root" path="." />+ <exclude domain="external" path="." />+ <exclude domain="device_root" path="." />+ </cloud-backup>+ <device-transfer>+ <exclude domain="root" path="." />+ <exclude domain="external" path="." />+ <exclude domain="device_root" path="." />+ </device-transfer>+</data-extraction-rules>diff --git a/app/src/main/res/xml/locales_config.xml b/app/src/main/res/xml/locales_config.xmlnew file mode 100644index 0000000..611978e--- /dev/null+++ b/app/src/main/res/xml/locales_config.xml@@ -0,0 +1,20 @@+<?xml version="1.0" encoding="utf-8"?>+<!--+ Which languages this app ships, declared to the system.++ Android 13+ shows a per-app language picker in Settings, and it can only+ list languages the app admits to supporting. Without this file the app+ disappears from that screen entirely and the only way to change language is+ the picker inside the card — which is fine for us, but it is not the place a+ user looks first.++ Two entries, matching [net.kuafuai.andee.config.VoiceConfig.LANG_ZH] and+ LANG_EN. The system picker and our own picker both write through+ AppCompatDelegate, so they stay in agreement; see+ [net.kuafuai.andee.i18n.AppLocale] for why the overlays need the extra step+ the system does not do for us.+-->+<locale-config xmlns:android="http://schemas.android.com/apk/res/android">+ <locale android:name="zh" />+ <locale android:name="en" />+</locale-config>diff --git a/app/src/test/java/net/kuafuai/andee/ExampleUnitTest.java b/app/src/test/java/net/kuafuai/andee/ExampleUnitTest.javanew file mode 100644index 0000000..e2a1b53--- /dev/null+++ b/app/src/test/java/net/kuafuai/andee/ExampleUnitTest.java@@ -0,0 +1,17 @@+package net.kuafuai.andee;++import org.junit.Test;++import static org.junit.Assert.*;++/**+ * Example local unit test, which will execute on the development machine (host).+ *+ * @see <a href="http://d.android.com/tools/testing">Testing documentation</a>+ */+public class ExampleUnitTest {+ @Test+ public void addition_isCorrect() {+ assertEquals(4, 2 + 2);+ }+}\ No newline at end of filediff --git a/app/src/test/java/net/kuafuai/andee/text/MarkdownTest.kt b/app/src/test/java/net/kuafuai/andee/text/MarkdownTest.ktnew file mode 100644index 0000000..daaa8a9--- /dev/null+++ b/app/src/test/java/net/kuafuai/andee/text/MarkdownTest.kt@@ -0,0 +1,52 @@+package net.kuafuai.andee.text++import org.junit.Assert.assertEquals+import org.junit.Test++class MarkdownTest {++ @Test+ fun stripsEmphasis() {+ assertEquals("已经打开微信了", stripMarkdown("已经**打开微信**了"))+ assertEquals("重点", stripMarkdown("***重点***"))+ }++ @Test+ fun leavesSnakeCaseAlone() {+ assertEquals("调用 tap_screen 和 get_ui_tree", stripMarkdown("调用 tap_screen 和 get_ui_tree"))+ assertEquals("submit_input 完成", stripMarkdown("submit_input 完成"))+ }++ @Test+ fun stripsHeadingsAndLists() {+ assertEquals("结果\n第一步\n第二步", stripMarkdown("## 结果\n- 第一步\n- 第二步"))+ assertEquals("甲\n乙", stripMarkdown("1. 甲\n2. 乙"))+ }++ @Test+ fun keepsLinkTextDropsUrl() {+ assertEquals("看这里", stripMarkdown("[看这里](https://example.com/a_b_c)"))+ // 图片连 alt 一起丢。整条只有一张图时会命中兜底(见+ // fallsBackWhenStrippingEmptiesEverything),所以这里带上下文。+ assertEquals("看图 就懂了", stripMarkdown("看图  就懂了"))+ }++ @Test+ fun stripsCodeAndFences() {+ assertEquals("ls -la", stripMarkdown("`ls -la`"))+ assertEquals("echo hi", stripMarkdown("```bash\necho hi\n```"))+ }++ @Test+ fun stripsRulesQuotesTables() {+ assertEquals("前\n后", stripMarkdown("前\n---\n后"))+ assertEquals("引用的话", stripMarkdown("> 引用的话"))+ assertEquals("姓名 年龄\n张三 30", stripMarkdown("| 姓名 | 年龄 |\n|---|---|\n| 张三 | 30 |"))+ }++ @Test+ fun fallsBackWhenStrippingEmptiesEverything() {+ assertEquals("---", stripMarkdown("---"))+ assertEquals("", stripMarkdown(""))+ }+}diff --git a/app/src/test/java/net/kuafuai/andee/wake/WakeDspTest.kt b/app/src/test/java/net/kuafuai/andee/wake/WakeDspTest.ktnew file mode 100644index 0000000..2920cda--- /dev/null+++ b/app/src/test/java/net/kuafuai/andee/wake/WakeDspTest.kt@@ -0,0 +1,101 @@+package net.kuafuai.andee.wake++import org.junit.Assert.assertEquals+import org.junit.Assert.assertTrue+import org.junit.Test+import kotlin.math.PI+import kotlin.math.sin+import kotlin.random.Random++/**+ * The wake word's only testable half.+ *+ * Whether it recognises a voice can only be answered by a person saying "嘿+ * Andee" at a device — but whether the front end produces finite, comparable+ * numbers is arithmetic, and getting that wrong is silent: a NaN from a log of+ * zero, or a distance scale that drifts, turns into "it just never triggers"+ * with nothing in the log to say why.+ */+class WakeDspTest {++ private fun tone(hz: Double, ms: Int, amp: Double = 8000.0, seed: Int = 1): ShortArray =+ phrase(listOf(hz to ms), amp, seed)++ /**+ * A crude stand-in for an utterance: a run of differently-pitched segments.+ *+ * A single steady tone will not do, and the reason is worth keeping. CMVN+ * ([Mfcc]) divides each coefficient by its variance *across the clip*, so a+ * stationary signal — where every frame is the same — normalises down to+ * its own noise and the distances come out random. Speech is never+ * stationary; a fan is, which is one reason room tone does not match the+ * templates.+ */+ private fun phrase(+ parts: List<Pair<Double, Int>>,+ amp: Double = 8000.0,+ seed: Int = 1,+ tempo: Double = 1.0,+ ): ShortArray {+ val rnd = Random(seed)+ val out = ArrayList<Short>()+ for ((hz, ms) in parts) {+ val n = (Mfcc.SAMPLE_RATE * ms * tempo / 1000).toInt()+ for (i in 0 until n) {+ val t = i.toDouble() / Mfcc.SAMPLE_RATE+ // Two harmonics plus a little noise: a flat sine has almost no+ // spectral structure and every mel band ends up at the floor.+ val v = amp * (sin(2 * PI * hz * t) + 0.5 * sin(4 * PI * hz * t)) ++ rnd.nextDouble(-200.0, 200.0)+ out.add(v.toInt().coerceIn(-32768, 32767).toShort())+ }+ }+ return ShortArray(out.size) { out[it] }+ }++ private val hello = listOf(200.0 to 150, 620.0 to 150, 320.0 to 200, 880.0 to 150)+ private val somethingElse = listOf(900.0 to 150, 260.0 to 150, 940.0 to 200, 420.0 to 150)++ @Test+ fun featuresAreFiniteAndShaped() {+ val f = Mfcc.features(tone(220.0, 500))+ assertEquals(500 / 10 - 2, f.size) // 10 ms hop, minus the 25 ms window tail+ for (frame in f) {+ assertEquals(Mfcc.COEFFS, frame.size)+ for (v in frame) assertTrue("non-finite coefficient: $v", v.isFinite())+ }+ }++ @Test+ fun silenceDoesNotProduceNaN() {+ val f = Mfcc.features(ShortArray(8000))+ for (frame in f) for (v in frame) assertTrue("non-finite: $v", v.isFinite())+ }++ @Test+ fun identicalClipsAlignAtZero() {+ val f = Mfcc.features(phrase(hello))+ assertEquals(0f, Dtw.distance(f, f), 1e-4f)+ }++ /** The property the threshold rests on: same sound near, different sound far. */+ @Test+ fun sameSoundIsCloserThanDifferentSound() {+ val a = Mfcc.features(phrase(hello, seed = 1))+ // Same "phrase" said 15% slower and a lot quieter — what a second take+ // from the same person actually looks like.+ val again = Mfcc.features(phrase(hello, amp = 3000.0, seed = 2, tempo = 1.15))+ val other = Mfcc.features(phrase(somethingElse, seed = 3))+ val near = Dtw.distance(a, again)+ val far = Dtw.distance(a, other)+ assertTrue("same sound scored $near, different scored $far", near < far)+ }++ /** Clips too different in length are refused rather than warped into agreement. */+ @Test+ fun wildLengthMismatchIsRejected() {+ val short = Mfcc.features(phrase(hello, tempo = 0.5))+ val long = Mfcc.features(phrase(hello, tempo = 2.0))+ assertEquals(Dtw.NO_MATCH, Dtw.distance(short, long), 0f)+ }+}diff --git a/build.gradle b/build.gradlenew file mode 100644index 0000000..e61198d--- /dev/null+++ b/build.gradle@@ -0,0 +1,4 @@+// Top-level build file where you can add configuration options common to all sub-projects/modules.+plugins {+alias(libs.plugins.androidApplication) apply false+}\ No newline at end of filediff --git a/docs/acceptable-use.md b/docs/acceptable-use.mdnew file mode 100644index 0000000..c07f9ff--- /dev/null+++ b/docs/acceptable-use.md@@ -0,0 +1,93 @@+<p align="center">+ <a href="./acceptable-use.md"><img alt="English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./acceptable-use.zh-CN.md"><img alt="简体中文" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# Acceptable Use Policy++> This is the second half of [DISCLAIMER.md](../DISCLAIMER.md), and an extension of the+> "additional conditions" in [LICENSE](../LICENSE).+> **Violating this policy takes you outside the licence.** Your right to use the software+> terminates automatically, and no disclaimer applies.+>+> Last updated: 2026-09-29++---++## 1. General++Use this software only for **lawful** purposes, and only on devices and accounts **you are entitled to operate**.++"You are entitled to operate" means the device is your own, or you have the device owner's explicit, informed permission. **"I was helping him" is not permission.**++## 2. Prohibited uses++The following are **prohibited without exception**. They are listed because they genuinely happen, not to paper over them:++**(a) Crime, and providing help to crime**++- Any criminal or unlawful activity.+- Telecom and network fraud, or **any form of assistance** to it — including transferring funds for others, money laundering ("跑分"), faking transaction flows, receiving and passing on proceeds, promotional referral, bulk account registration or unblocking, and creating or distributing fraud tooling.+ This **may constitute "the crime of aiding information network criminal activities" under Article 287-2 of the Criminal Law of the People's Republic of China** — up to three years' imprisonment or detention, plus a fine. Note that in practice, **even without direct profit and even while claiming not to have known, a court may infer knowledge from objective conduct.**+- Operating another person's payment account, bank card, or settlement instrument on their behalf.++**(b) False information and reputational harm**++- Generating, publishing or spreading false information or rumours.+- Defaming, insulting or disparaging any person or organisation.+- Impersonating any person, institution or official identity.++**(c) Privacy**++- Recording, listening to, or reading another person's conversations, messages, contacts, call log, location or screen without their consent.+- Installing or running this software on a device that is not yours.++**(d) Circumventing security**++- Cracking, bypassing or evading any platform's risk controls, verification, limits or security measures.+- Bypassing payment verification, identity verification, or content moderation.+- Attacking, scanning or accessing any system, network or service without authorisation.++**(e) Other**++- Gambling, pornography, contraband, and any other prohibited transaction.+- Spam, harassment, or unsolicited bulk messaging and calling.+- Anything that would associate others with this project such that they could be mistaken for a party to the above.++## 3. On money — read this one separately++- **A human must be present for any financial action.** Do not let it complete a payment, transfer or order unattended.+- **Do not store a payment password in the vault and let it pay automatically.**+- It is **capable** of completing a transfer on its own — it taps by coordinates rather than going through a restricted payment API. **This project has no technical guardrail on that path.**+- **If money goes to the wrong place, or is lost to fraud, "the AI did it" is not a defence.** The legal consequence falls on the operator. (Chinese legal commentary makes the same point: an agent's autonomous "acting and deciding" may take the act outside the user's intent, but the consequence still lands on the user.)++## 4. On content published as you++- What it sends **counts as yours**. It carries your name, and your liability.+- In mainland China, spreading rumours may violate the Public Security Administration Punishments Law; in serious cases it may be a criminal offence.+- **Review before it publishes.** Do not let it speak to the outside world unattended.+- It can be steered by text on the screen (prompt injection) into saying things you did not ask for — see [SECURITY.md](../SECURITY.md).++## 5. On other people's devices and accounts++- This policy permits **your own devices** and **your own accounts** only.+- LICENSE Part A.1(b) prohibits operating this software as a multi-tenant service that drives third parties' devices without written authorisation.+- If you need to manage other people's devices (an internal fleet deployment, for example), obtain written authorisation first.++## 6. Consequences++- **Licence termination**: violating this policy exceeds the scope granted by [LICENSE](../LICENSE), and your licence to use this software **terminates automatically**, with no notice from us.+- **Your liability**: all legal consequences are yours, not the authors' or contributors'.+- **What we reserve**: refusing support, closing the relevant issues, restricting or ending distribution, and reporting to the competent authorities where required.++## 7. Reporting misuse++If you find this software being used for any of the above, tell us through the channel in [SECURITY.md](../SECURITY.md). **Do not** post details in a public issue.++## 8. Notice++This document **is not legal advice.** It is a condition of use, not a legal opinion. Requirements differ between jurisdictions, and **satisfying yourself that your use is lawful is yours to do.**++---++**Other languages:** [简体中文](acceptable-use.zh-CN.md)diff --git a/docs/acceptable-use.zh-CN.md b/docs/acceptable-use.zh-CN.mdnew file mode 100644index 0000000..5063047--- /dev/null+++ b/docs/acceptable-use.zh-CN.md@@ -0,0 +1,92 @@+<p align="center">+ <a href="./acceptable-use.md"><img alt="English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./acceptable-use.zh-CN.md"><img alt="简体中文" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# 可接受使用政策++> 这是 [DISCLAIMER.zh-CN.md](../DISCLAIMER.zh-CN.md) 的第二部分,也是 [LICENSE](../LICENSE)+> 里"附加条件"的延伸。+> **违反本政策,即超出许可范围。** 你的使用许可自动终止,且不适用任何免责。+>+> 最后更新:2026-09-29++---++## 1. 总则++本软件只能用于**合法**目的,只能操作**你有权操作**的设备和账号。++"有权操作"指:设备是你自己的,或你已获得设备所有者明确、知情的授权。**"我帮他操作一下"不构成授权。**++## 2. 明确禁止的用途++以下用途**一律禁止**。列出来不是为了免责,而是因为它们是真实会发生的滥用:++**(一)违法犯罪与"提供帮助"**++- 用于实施任何违法犯罪活动。+- 用于电信网络诈骗,或为其提供**任何形式**的帮助——包括代转账、跑分、刷流水、走账、推广引流、批量注册或解封账号、制作或分发涉诈软件。+ 这**可能构成《中华人民共和国刑法》第二百八十七条之二规定的"帮助信息网络犯罪活动罪"**,最高三年有期徒刑或拘役,并处罚金。请注意:司法实践中,**即使你未直接获利、或辩称"不知情",也可能根据客观行为推定"明知"**。+- 为他人操作其支付账户、银行卡、或任何结算工具。++**(二)虚假信息与损害他人名誉**++- 生成、发布、或传播不实信息、谣言。+- 诽谤、侮辱、或诋毁他人或组织。+- 冒充他人、机构、或官方身份。++**(三)侵犯隐私**++- 未经同意录制、监听、或读取他人的对话、短信、通讯录、通话记录、位置、或屏幕内容。+- 在他人的设备上安装或运行本软件。++**(四)绕过安全机制**++- 破解、绕过或规避任何平台的风控、验证、限额、或安全措施。+- 绕过支付验证、身份验证、或内容审核。+- 攻击、扫描、或未授权访问任何系统、网络、或服务。++**(五)其他违规**++- 赌博、色情、违禁品交易,及其他法律禁止的交易。+- 垃圾信息轰炸、骚扰、或未经请求的批量外呼/群发。+- 任何会使他人与本项目产生关联、从而使其被误认为与上述行为有关的行为。++## 3. 特别提示:关于钱(请单独记住这一条)++- **任何资金操作,人必须在场。** 不要让它在无人看管的情况下完成支付、转账、或下单。+- **不要把支付密码存进保险箱,然后让它自动付款。**+- 它**有能力**独立完成一笔转账——它用坐标点击,而不是走某个受限的支付接口。项目对这条路径**没有技术围栏**。+- **一旦钱转错了、被骗了,"是 AI 自己做的"不是抗辩理由。** 法律后果由操作者承担。(该观点亦见于中国法律实务界的公开分析:智能体的自动化"动手和决策"能力,可能使行为脱离使用者主观动机,但后果仍归使用者。)++## 4. 特别提示:关于它替你发布的内容++- 它发出的内容**算你的**。署名是你的,责任也是你的。+- 在中国大陆,散布谣言可能违反《中华人民共和国治安管理处罚法》;情节严重可能构成犯罪。+- **发布前你应当审阅。** 不要让它无人值守地对外发言。+- 它可能被屏幕上的文字操纵(提示词注入)而说出你没让它说的话,见 [SECURITY.md](../SECURITY.md)。++## 5. 特别提示:关于他人的设备与账号++- 本政策只允许**你自己的设备**与**你自己的账号**。+- LICENSE Part A.1(b) 禁止未经书面授权,把本软件作为**驱动第三方设备**的多租户服务运营。+- 需要替你管理他人设备的场景(例如企业内部批量部署),请先取得书面授权。++## 6. 违反的后果++- **许可终止**:违反本政策即超出 [LICENSE](../LICENSE) 授予的范围,你对本软件的使用许可**自动终止**,无需我们另行通知。+- **责任自负**:所有法律后果由你承担,与作者及贡献者无关。+- **我们保留的权利**:拒绝支持、关闭相关 issue、限制或终止分发、以及在必要时向有权机关报告。++## 7. 发现滥用++如果你发现本软件被用于上述任何用途,请通过 [SECURITY.md](../SECURITY.md) 的渠道告知我们。**不要**在公开 issue 里发布细节。++## 8. 声明++本文件**不构成法律意见**。它是一份使用条件,不是一份法律咨询。不同司法辖区的法律要求不同,**你需要自行确认你的使用方式是否合法**。++---++**其他语言:** [English](acceptable-use.md)diff --git a/docs/permissions.md b/docs/permissions.mdnew file mode 100644index 0000000..e029ebe--- /dev/null+++ b/docs/permissions.md@@ -0,0 +1,146 @@+<p align="center">+ <a href="./permissions.md"><img alt="English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./permissions.zh-CN.md"><img alt="简体中文" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# Permissions++This page explains why Andee asks for the permissions it asks for, what each one buys you, and what happens if you refuse it.++The reason it is this detailed is straightforward: **the combination of permissions Andee requests overlaps heavily with the detection signature of an Android banking trojan** — accessibility + overlay + notification listener + SMS + call log + contacts + installed apps. Any security researcher who glances at the manifest will flag it. That is reasonable alarm, not a misjudgement.++So this page has exactly one job: **to separate "looks the same" from "is the same", and to make every claim checkable in the code.**++---++## First, where the difference lies++| | Typical Chinese app / malware | Andee |+|---|---|---|+| When permissions are requested | All at once, on first launch | **On demand** — only when you use the feature that needs one |+| What happens when you refuse | The app is unusable / nagged repeatedly | **Feature degrades**, everything else keeps working, and it tells you why it could not do the thing |+| Permission ↔ feature mapping | Not explained | See the table below; every entry maps to a concrete tool |+| Where sensitive data goes | Not stated | Credentials are encrypted on-device (`config/Vault.kt`) and unreadable by the model; the 5 private-notebook tools **are never sent off-device at all** (filtered by `ToolSchemas.forHub()`) |+| How credentials are used | — | The model can only ask the device to *type* them; **it never receives the password itself** (`fill_secret`) |++Two places in the code where you can verify this:++- `device/PermissionsController.kt` — the permission list, the plain-language explanation for each, and the guided jump to Settings+- `device/DeviceCommsController.kt` — every tool calls `ensurePermission()` **before** it acts; without a grant it returns a plain `denied("sms")`-style result. It does not crash, and it does not quietly find another way to do it.++---++## The full permission table++The "What you get" column is the copy the app actually shows (`dev_perm_feature_*` in `res/values-en/strings.xml`).++| Permission | What you get | Tools | What happens if you refuse |+|---|---|---|---|+| Overlay<br>`SYSTEM_ALERT_WINDOW` | Overlay (my face) | The ball, the subtitles, the settings panel itself | **Nothing is visible when the app starts.** The only permission that is "off means completely unusable" |+| Accessibility service<br>`BIND_ACCESSIBILITY_SERVICE` | See the screen, tap the UI, type | `get_screen_element` / `tap_screen_element` / `swipe_by_coordinates` / `type_text` / `submit_input` / `take_screenshot` — all of `screen.*` | **Every screen action errors.** This is the core capability; the other permissions exist around it |+| Microphone<br>`RECORD_AUDIO` | Voice, and the "Hey Andee" wake word | Streaming ASR + on-device wake-word matching | Tapping the ball does nothing; wake-word listening does not start (and, as a side effect, the Android 12+ green dot never appears) |+| Notification access<br>`BIND_NOTIFICATION_LISTENER_SERVICE` | Notification awareness (know when you get messages) | `get_notifications` | It will not know about incoming messages on its own; nothing else is affected |+| Location<br>`ACCESS_FINE_LOCATION` | Location (where you are / navigation) | `get_location` | "Where am I" cannot be answered |+| Step count<br>`ACTIVITY_RECOGNITION` | Step count | `get_step_count` | Step count cannot be answered |+| Contacts<br>`READ_CONTACTS` | Contacts (find someone to call) | `search_contacts` | "Call Old Wang" cannot find anyone |+| Phone<br>`CALL_PHONE` | Direct dialing | `dial` | It can find the person but cannot place the call |+| Call log<br>`READ_CALL_LOG` | Call history | `get_call_log` | Cannot look up who called recently |+| Read SMS<br>`READ_SMS` | Read SMS (verification codes) | `read_sms` | Cannot read verification codes |+| Send SMS<br>`SEND_SMS` | Send SMS | `send_sms` | Cannot send |+| Read calendar<br>`READ_CALENDAR` | Calendar (reminders) | `get_calendar` | Cannot see today's schedule |+| Write calendar<br>`WRITE_CALENDAR` | Write calendar (add events) | `add_calendar_event` | Cannot record events |+| Camera<br>`CAMERA` | Sight (identify objects / scan codes) | `camera_turn` / `scan_code` | Those two stop working |+| Installed apps<br>`QUERY_ALL_PACKAGES` | Know which apps are installed | `list_installed_apps` / `launch_app` | Cannot open a named app |+| Notifications<br>`POST_NOTIFICATIONS` | Remind you on time | Due-task reminders | No reminder at the due time, **but the task is not lost** |+| Boot<br>`RECEIVE_BOOT_COMPLETED` | Tasks survive a reboot | Rebuilding the `AlarmManager` alarms | After a reboot, due tasks stop reminding |+| Battery optimisation exemption<br>`REQUEST_IGNORE_BATTERY_OPTIMIZATIONS` | Reminders arrive on time | Task scheduling | With the screen off, reminders arrive late |+| Write secure settings<br>`WRITE_SECURE_SETTINGS` | Switch the input method (needed to type) | Prerequisite step for `type_text` | **`type_text` fails outright**, and tells you exactly why. It will not pretend to succeed |++> `WRITE_SECURE_SETTINGS` is the odd one out: it **does not take effect on install — you must grant it manually over adb**:+>+> ```bash+> adb shell pm grant net.kuafuai.andee android.permission.WRITE_SECURE_SETTINGS+> ```+>+> That is a platform restriction, not a choice this app made. You can use it without granting this; you just cannot type.++---++## Three switches you must turn on by hand after installing++These are outside the app. Android does not let an app flip them itself. Miss one and the matching feature is unavailable:++1. **Overlay** — Settings → Apps → Andee → Permissions → Display over other apps+2. **Accessibility service** — Settings → Accessibility → Installed services → Andee (you will get a big red warning box; that is the standard system flow)+3. **Microphone** — the system prompts for it the first time you tap the ball++> Some vendor ROMs (Xiaomi / Huawei / HONOR) **turn accessibility back off** after every system update or force-stop. You have to re-enable it. That is ROM behaviour.++---++## How credentials are kept (the question that most deserves asking)++Andee has a vault (`config/Vault.kt`) for the account passwords you enter yourself. Three hard rules in the design:++1. **Encrypted at rest.** AES/GCM, with the key generated and held by the Android KeyStore, **never leaving the device**. Even with the ciphertext in hand, another machine cannot decrypt it (there is a matching fallback in the code: "vault: decrypt failed — entries are unreadable on this install").+2. **The model never receives the password itself.** In what `get_vault` returns, any field marked secret is omitted; the model can only call `fill_secret`, and **the device itself** types the password into the field. So the password does not pass through the model and does not enter its context.+3. **Masked in the UI.** The vault screen shows the output of `Vault.mask()`.++**What it does not protect against (worth stating plainly):** the key is not bound to device unlock (it does not use `setUserAuthenticationRequired`). So **anyone who can unlock your phone can see the vault contents through the UI.** It protects against "data extracted off the device" and "the password read by the model" — not against "someone who picked up your phone".++---++## Where your data goes ("do you collect my data?")++**The conclusion: this project receives nothing about you; but "your data never leaves the device" is false.** Both halves need saying — only stating the first is misleading.++**There is no telemetry in this repository.** No analytics SDK, no crash reporting, no event tracking, no device fingerprinting — `firebase` / `analytics` / `sentry` / `crashlytics` are all zero across `app/build.gradle` and `app/src/main/`. If you install it and use it, the authors do not know.++**But data does leave the device, by design:**++| What | Where it goes | Default |+|---|---|---|+| Speech recognition (what you say), speech synthesis (what he says) | **Volcengine (ByteDance) cloud** | `openspeech.bytedance.com` |+| Conversation content, **and every tool result** | The "brain" **you** configured | `brain=hub`; in local mode, `api.deepseek.com` by default |+| Private notebook (`config/Notebook.kt`: what he remembers about you, what he promised) | **On-device only** | It is `localOnly` and never enters the hub payload |++The second row deserves spelling out: **a tool result is part of the conversation.** The SMS bodies, contacts, call log, notification content and location he reads, along with his "reading" of the screen, all travel with the conversation to the endpoint you configured.++**So "local mode" does not mean "local processing"** — it changes where the brain is, not the fact that data leaves the device. If you genuinely want data to stay in, you have to point `asr_endpoint` / `tts_endpoint` / `llm_base_url` / `hub_url` at a machine you control yourself.++The full allocation of responsibility and the disclaimer are in **[../DISCLAIMER.md](../DISCLAIMER.md)**; prohibited uses are in **[acceptable-use.md](acceptable-use.md)**.++---++## Known risk: the debug WebSocket port is open to the local network++**Please read this one to the end before using it.**++On startup the app opens a WebSocket service listening on `0.0.0.0:9008` (`net/BodyWsServer.kt`; the port is the `ScreenBodyService.WS_PORT` constant).++**The problem is that it authenticates nothing.** `onOpen()` accepts every incoming connection as a legitimate controller. Any host that can reach this device's IP can connect and issue commands — tap the screen, type, read the UI, read notifications, read SMS and the call log, and have the vault passwords typed into any app's input field.++**When you get hit:** whenever the accessibility service is on and the device is on a network you do not control — a café, hotel, airport, meeting room, a corporate LAN — the risk is live.++**What you can do about it today:**+- Only use it on networks you trust, or just block the port at the router / firewall+- Drive the device over `adb forward` instead of leaving it reachable on the network+- Treat a device with accessibility enabled as a device you would not leave lying around on a strange network++**The proper fix** (not done yet): verify a shared secret on connect, and make the bind address and port configurable, defaulting to loopback only. That is recorded as a to-do; until it lands, the limitation above stands.++> A related but lighter second item: the link from the app to the brain hub (`net/BodyWsClient.kt`) is also unauthenticated, and is plaintext `ws://`. So only ever put an address you control in `hub_url`. **Using `local` brain mode avoids that link entirely** — the model request goes out over HTTPS from the app itself, and no inbound command channel exists on the device.++---++## Why it does not ask for everything at once++Because that is what a trojan does.++The approach here is: **you ask for a feature, it asks for the permission; if you decline, it degrades — and it tells you it degraded.** You can verify that line by line in the code: every tool runs `ensurePermission()` before acting, and a refusal returns an explicit failure reason rather than finding a quieter route to the same result.++If you want to look deeper, [../SECURITY.md](../SECURITY.md) has the full security model and the vulnerability reporting channel.++---++**Other languages:** [简体中文](permissions.zh-CN.md)diff --git a/docs/permissions.zh-CN.md b/docs/permissions.zh-CN.mdnew file mode 100644index 0000000..1ef704c--- /dev/null+++ b/docs/permissions.zh-CN.md@@ -0,0 +1,146 @@+<p align="center">+ <a href="./permissions.md"><img alt="English" src="https://img.shields.io/badge/English-d9d9d9"></a>+ <a href="./permissions.zh-CN.md"><img alt="简体中文" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>+</p>++# 权限说明++这一页解释 Andee 为什么要那些权限、每个权限对应你能得到的什么功能、以及你拒绝之后会发生什么。++写这么细的原因很直接:**Andee 申请的权限组合,和安卓银行木马的检测特征高度重合** —— 无障碍 + 悬浮窗 + 通知监听 + 短信 + 通话记录 + 通讯录 + 应用列表,任何一个安全研究者扫一眼 Manifest 都会把它标红。这是合理的警觉,不是误判。++所以这一页要做的事只有一件:**把"看起来一样"和"实际不一样"分别讲清楚,并且每一条都能在代码里核对。**++---++## 先看区别在哪++| | 典型国产 App / 恶意软件 | Andee |+|---|---|---|+| 要权限的时机 | 启动时一次性全要 | **按次申请**:用到哪个功能才要哪个 |+| 你拒绝之后 | 不让用 / 反复弹窗 | **功能降级**,其余照常用,并且告诉你为什么没做成 |+| 权限和功能的对应 | 不解释 | 见下表,每项都能对应到一个具体工具 |+| 敏感数据去了哪 | 不说明 | 凭据加密存在本机(`config/Vault.kt`),模型读不到;5 个私人笔记工具**根本不外发**(`ToolSchemas.forHub()` 过滤) |+| 凭据怎么用 | — | 模型只能让设备"去填",**它自己永远拿不到密码原文**(`fill_secret`) |++代码里能核对这件事的两处:++- `device/PermissionsController.kt` —— 权限清单 + 每项的人话说明 + 引导跳转+- `device/DeviceCommsController.kt` —— 每个工具在动手**之前**先 `ensurePermission()`,没授权就直接返回 `denied("sms")` 这样的结果,不会崩、也不会偷偷换个方式做++---++## 权限总表++「用户得到什么」一列是应用里实际显示的原文(`res/values/strings.xml` 的 `dev_perm_feature_*`)。++| 权限 | 用户得到什么 | 对应工具 | 你拒绝了会怎样 |+|---|---|---|---|+| 悬浮窗<br>`SYSTEM_ALERT_WINDOW` | 悬浮窗(我的脸) | 球、字幕、设置面板本身 | **应用起来什么都看不到。** 这是唯一一个"不开就完全不能用"的 |+| 无障碍服务<br>`BIND_ACCESSIBILITY_SERVICE` | 看屏幕、点界面、打字 | `get_screen_element` / `tap_screen_element` / `swipe_by_coordinates` / `type_text` / `submit_input` / `take_screenshot` 等全部 `screen.*` | **所有屏幕操作报错。** 这是核心能力,其它权限都是围着它转的 |+| 麦克风<br>`RECORD_AUDIO` | 语音说话、唤醒词「嘿 Andee」 | 流式 ASR + 本机唤醒词匹配 | 点球没反应;唤醒词监听不启动(顺带:Android 12+ 的绿点也不会出现) |+| 通知监听<br>`BIND_NOTIFICATION_LISTENER_SERVICE` | 通知感知(微信来消息我知道) | `get_notifications` | 来消息不会主动知道;其余功能不受影响 |+| 位置<br>`ACCESS_FINE_LOCATION` | 位置(在哪/导航) | `get_location` | 问"我在哪"答不出来 |+| 运动步数<br>`ACTIVITY_RECOGNITION` | 运动步数 | `get_step_count` | 问步数答不出来 |+| 通讯录<br>`READ_CONTACTS` | 通讯录(打电话找人) | `search_contacts` | 说"打电话给老王"找不到人 |+| 拨号<br>`CALL_PHONE` | 直接拨号 | `dial` | 能找到人但拨不出去 |+| 通话记录<br>`READ_CALL_LOG` | 通话记录 | `get_call_log` | 查不了最近谁打的电话 |+| 读短信<br>`READ_SMS` | 读短信(验证码) | `read_sms` | 读不了验证码 |+| 发短信<br>`SEND_SMS` | 发短信 | `send_sms` | 发不了 |+| 读日历<br>`READ_CALENDAR` | 日历(日程提醒) | `get_calendar` | 看不了今天的日程 |+| 写日历<br>`WRITE_CALENDAR` | 写日历(记日程) | `add_calendar_event` | 记不了日程 |+| 摄像头<br>`CAMERA` | 看(拍照识物 / 扫码) | `camera_turn` / `scan_code` | 这两条用不了 |+| 应用列表<br>`QUERY_ALL_PACKAGES` | 知道装了哪些应用 | `list_installed_apps` / `launch_app` | 打不开指定应用 |+| 通知栏<br>`POST_NOTIFICATIONS` | 到点提醒你 | 待办到期的提醒 | 到点不提醒,**但待办不会丢** |+| 开机自启<br>`RECEIVE_BOOT_COMPLETED` | 重启后待办还在 | 重建 `AlarmManager` 闹钟 | 重启后待办不再提醒 |+| 电池优化豁免<br>`REQUEST_IGNORE_BATTERY_OPTIMIZATIONS` | 到点提醒准时 | 待办定时 | 息屏状态下提醒会迟到 |+| 改系统设置<br>`WRITE_SECURE_SETTINGS` | 换输入法(打字用) | `type_text` 前置步骤 | **`type_text` 直接失败**,并明确告诉你失败原因,不会假装成功 |++> `WRITE_SECURE_SETTINGS` 是特殊的一个:它**装完不生效,必须用 adb 手动授予**:+>+> ```bash+> adb shell pm grant net.kuafuai.andee android.permission.WRITE_SECURE_SETTINGS+> ```+>+> 这是系统限制,不是应用的选择。不授予也能用,只是打不了字。++---++## 装机必须手动开的三个开关++这三个不在应用内,Android 不允许应用自己翻。少一个就有对应功能不能用:++1. **悬浮窗** —— 设置 → 应用 → Andee → 权限 → 显示在其他应用上层+2. **无障碍服务** —— 设置 → 无障碍 → 已安装的服务 → Andee(会弹一个大红框警告,这是系统标准流程)+3. **麦克风** —— 首次点球时系统会自己弹++> 部分厂商 ROM(小米 / 华为 / 荣耀)每次系统更新或强制停止应用后会**自动关掉无障碍**,需要重开。这是 ROM 行为。++---++## 凭据是怎么保管的(这是最该被问的一条)++Andee 有一个保险箱(`config/Vault.kt`),用来存你自己填进去的账号密码。设计上有三条硬规则:++1. **加密落盘。** AES/GCM,密钥由 Android KeyStore 生成并保管,**不出设备**。换台机器即使拿到密文也解不开(代码里有对应的兜底:"vault: decrypt failed — entries are unreadable on this install")。+2. **模型永远拿不到密码原文。** `get_vault` 返回的条目里,标记为 secret 的字段不下发;模型只能用 `fill_secret`,由**设备自己**把密码打进输入框。也就是说密码不经过模型、不进模型的上下文。+3. **界面上打码。** 保险箱页面显示的是 `Vault.mask()` 之后的结果。++**它不防什么(要说清楚):** 密钥没有绑定设备解锁(没用 `setUserAuthenticationRequired`)。所以**能解锁你手机的人,就能通过界面看到保险箱内容**。它防的是"数据被提取走"和"密码被模型读到",不防"捡到你手机的人"。++---++## 数据会去哪("你们会不会收集我的数据")++**结论:本项目收不到你的任何数据;但"数据不出设备"是错的。** 这两句都要说清楚,只讲前一句等于误导。++**这个仓库里没有任何遥测。** 没有统计 SDK、没有崩溃上报、没有埋点、没有设备指纹 —— `app/build.gradle` 与 `app/src/main/` 里 `firebase` / `analytics` / `sentry` / `crashlytics` 全部为零。你装了它、用了它,作者不知道。++**但数据按设计会离开设备:**++| 什么数据 | 去哪 | 默认值 |+|---|---|---|+| 语音识别(你说的话)、语音合成(他说的) | **火山引擎(字节跳动)云端** | `openspeech.bytedance.com` |+| 对话内容,**以及全部工具返回结果** | 你配置的"大脑" | `brain=hub`;本机模式默认 `api.deepseek.com` |+| 私人笔记本(`config/Notebook.kt`:记住的你、答应过的事) | **只在本机** | 是 `localOnly`,不进 hub 载荷 |++第二行要说透:**工具返回结果就是对话的一部分。** 他读到的短信正文、通讯录、通话记录、通知内容、定位,以及他对屏幕的"理解",都会随对话发到你配的那个端点。++**所以"本机模式"不等于"本地处理"** —— 它换的是大脑的目的地,不是"数据留在设备上"这件事。真要数据不出门,你得自己把 `asr_endpoint` / `tts_endpoint` / `llm_base_url` / `hub_url` 全指向你自己控制的机器。++完整的责任划分与免责条款见 **[../DISCLAIMER.zh-CN.md](../DISCLAIMER.zh-CN.md)**;禁止用途见 **[acceptable-use.zh-CN.md](acceptable-use.zh-CN.md)**。++---++## 已知风险:调试用的 WebSocket 端口对局域网开放++**这一条请务必读完再用。**++应用启动时会开一个 WebSocket 服务,监听 `0.0.0.0:9008`(`net/BodyWsServer.kt`,端口是 `ScreenBodyService.WS_PORT` 常量)。++**问题在于:它不验证任何身份。** `onOpen()` 把每一条进来的连接都当作合法的控制端收下。任何能访问到这台设备 IP 的主机,都可以连上来发命令 —— 点屏幕、打字、读界面、读通知、读短信和通话记录,还能让它把保险箱里的密码填进任意应用的输入框。++**什么时候会被打:** 只要无障碍服务开着,而设备又在一个你控制不了的网络上 —— 咖啡厅、酒店、机场、会议室、公司大内网 —— 就存在这个风险。++**现在能做的缓解:**+- 只在你信得过的网络上用,或者干脆让路由器/防火墙挡住这个端口+- 用 `adb forward` 驱动设备,别让它从网络上可达+- 把开了无障碍服务的设备,当成不敢随手放在陌生网络里的设备++**彻底修法**(还没做):连接时校验一个共享密钥,并把监听地址和端口做成可配置、默认只听回环。这件事记为待办;在它落地之前,上面这条限制一直有效。++> 相关但轻一些的第二条:应用连大脑 hub 那条链路(`net/BodyWsClient.kt`)同样没有鉴权,且是明文 `ws://`。所以 `hub_url` 只填你自己控制的地址。**用 `local` 大脑模式可以完全避开这条链路** —— 模型请求由应用自己走 HTTPS 发出,设备上不存在对外的命令通道。++---++## 为什么不做"一次全要"++因为那才是木马的做法。++Andee 的做法是:**你要什么功能,才要什么权限;你不给的,它就降级,并且告诉你降级了。** 这一条能在代码里逐条核对 —— 每个工具在动手前都过一遍 `ensurePermission()`,拒绝就返回明确的失败原因,而不是换个更隐蔽的路子把事办了。++如果你想看得更深,[../SECURITY.md](../SECURITY.md)(英文)里有完整的安全模型和漏洞上报渠道。++---++**其他语言:** [English](permissions.md)diff --git a/gradle.properties b/gradle.propertiesnew file mode 100644index 0000000..8c2b94a--- /dev/null+++ b/gradle.properties@@ -0,0 +1,38 @@+# Project-wide Gradle settings.+# IDE (e.g. Android Studio) users:+# Gradle settings configured through the IDE *will override*+# any settings specified in this file.+# For more details on how to configure your build environment visit+# http://www.gradle.org/docs/current/userguide/build_environment.html+# Specifies the JVM arguments used for the daemon process.+# The setting is particularly useful for tweaking memory settings.+org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8+# When configured, Gradle will run in incubating parallel mode.+# This option should only be used with decoupled projects. For more details, visit+# https://developer.android.com/r/tools/gradle-multi-project-decoupled-projects+# org.gradle.parallel=true+# AndroidX package structure to make it clearer which packages are bundled with the+# Android operating system, and which are packaged with your app's APK+# https://developer.android.com/topic/libraries/support-library/androidx-rn+android.useAndroidX=true+# Enables namespacing of each library's R class so that its R class includes only the+# resources declared in the library itself and none from the library's dependencies,+# thereby reducing the size of the R class for that library+android.nonTransitiveRClass=true++# The Gradle JDK is deliberately NOT pinned here.+#+# This build needs a JDK in the **17–20** range:+# - AGP 8.3.1 requires 17 or newer;+# - the Gradle 8.4 wrapper cannot run on 21 or newer. Recent Android Studio releases ship a+# bundled JBR 25, which Gradle 8.4 rejects outright with+# "Unsupported class file major version 69".+#+# A machine-specific path must never be committed — it made the build fail instantly for+# everyone whose JDK lived somewhere else. To pin it for your own machine, do one of:+# - export JAVA_HOME=<path to a JDK 17–20>, or+# - add the line below to ~/.gradle/gradle.properties (never to this file):+# org.gradle.java.home=<path to a JDK 17–20>+# Android Studio users: Settings → Build, Execution, Deployment → Build Tools → Gradle →+# "Gradle JDK" → pick a 17–20 JDK. The IDE setting overrides this file.+# See README.md → "配 JDK / JDK setup".\ No newline at end of filediff --git a/gradle/libs.versions.toml b/gradle/libs.versions.tomlnew file mode 100644index 0000000..0b6f66d--- /dev/null+++ b/gradle/libs.versions.toml@@ -0,0 +1,18 @@+[versions]+agp = "8.3.1"+junit = "4.13.2"+junitVersion = "1.1.5"+espressoCore = "3.5.1"+appcompat = "1.6.1"+material = "1.10.0"++[libraries]+junit = { group = "junit", name = "junit", version.ref = "junit" }+ext-junit = { group = "androidx.test.ext", name = "junit", version.ref = "junitVersion" }+espresso-core = { group = "androidx.test.espresso", name = "espresso-core", version.ref = "espressoCore" }+appcompat = { group = "androidx.appcompat", name = "appcompat", version.ref = "appcompat" }+material = { group = "com.google.android.material", name = "material", version.ref = "material" }++[plugins]+androidApplication = { id = "com.android.application", version.ref = "agp" }+diff --git a/gradle/wrapper/gradle-wrapper.jar b/gradle/wrapper/gradle-wrapper.jarnew file mode 100644index 0000000..e708b1cBinary files /dev/null and b/gradle/wrapper/gradle-wrapper.jar differdiff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.propertiesnew file mode 100644index 0000000..ff3b56e--- /dev/null+++ b/gradle/wrapper/gradle-wrapper.properties@@ -0,0 +1,6 @@+#Sat Sep 05 21:29:49 CST 2026+distributionBase=GRADLE_USER_HOME+distributionPath=wrapper/dists+distributionUrl=https\://services.gradle.org/distributions/gradle-8.4-bin.zip+zipStoreBase=GRADLE_USER_HOME+zipStorePath=wrapper/distsdiff --git a/gradlew b/gradlewnew file mode 100755index 0000000..4f906e0--- /dev/null+++ b/gradlew@@ -0,0 +1,185 @@+#!/usr/bin/env sh++#+# Copyright 2015 the original author or authors.+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# https://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+#++##############################################################################+##+## Gradle start up script for UN*X+##+##############################################################################++# Attempt to set APP_HOME+# Resolve links: $0 may be a link+PRG="$0"+# Need this for relative symlinks.+while [ -h "$PRG" ] ; do+ ls=`ls -ld "$PRG"`+ link=`expr "$ls" : '.*-> \(.*\)$'`+ if expr "$link" : '/.*' > /dev/null; then+ PRG="$link"+ else+ PRG=`dirname "$PRG"`"/$link"+ fi+done+SAVED="`pwd`"+cd "`dirname \"$PRG\"`/" >/dev/null+APP_HOME="`pwd -P`"+cd "$SAVED" >/dev/null++APP_NAME="Gradle"+APP_BASE_NAME=`basename "$0"`++# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.+DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'++# Use the maximum available, or set MAX_FD != -1 to use that value.+MAX_FD="maximum"++warn () {+ echo "$*"+}++die () {+ echo+ echo "$*"+ echo+ exit 1+}++# OS specific support (must be 'true' or 'false').+cygwin=false+msys=false+darwin=false+nonstop=false+case "`uname`" in+ CYGWIN* )+ cygwin=true+ ;;+ Darwin* )+ darwin=true+ ;;+ MINGW* )+ msys=true+ ;;+ NONSTOP* )+ nonstop=true+ ;;+esac++CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar+++# Determine the Java command to use to start the JVM.+if [ -n "$JAVA_HOME" ] ; then+ if [ -x "$JAVA_HOME/jre/sh/java" ] ; then+ # IBM's JDK on AIX uses strange locations for the executables+ JAVACMD="$JAVA_HOME/jre/sh/java"+ else+ JAVACMD="$JAVA_HOME/bin/java"+ fi+ if [ ! -x "$JAVACMD" ] ; then+ die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME++Please set the JAVA_HOME variable in your environment to match the+location of your Java installation."+ fi+else+ JAVACMD="java"+ which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.++Please set the JAVA_HOME variable in your environment to match the+location of your Java installation."+fi++# Increase the maximum file descriptors if we can.+if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then+ MAX_FD_LIMIT=`ulimit -H -n`+ if [ $? -eq 0 ] ; then+ if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then+ MAX_FD="$MAX_FD_LIMIT"+ fi+ ulimit -n $MAX_FD+ if [ $? -ne 0 ] ; then+ warn "Could not set maximum file descriptor limit: $MAX_FD"+ fi+ else+ warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"+ fi+fi++# For Darwin, add options to specify how the application appears in the dock+if $darwin; then+ GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""+fi++# For Cygwin or MSYS, switch paths to Windows format before running java+if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then+ APP_HOME=`cygpath --path --mixed "$APP_HOME"`+ CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`++ JAVACMD=`cygpath --unix "$JAVACMD"`++ # We build the pattern for arguments to be converted via cygpath+ ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`+ SEP=""+ for dir in $ROOTDIRSRAW ; do+ ROOTDIRS="$ROOTDIRS$SEP$dir"+ SEP="|"+ done+ OURCYGPATTERN="(^($ROOTDIRS))"+ # Add a user-defined pattern to the cygpath arguments+ if [ "$GRADLE_CYGPATTERN" != "" ] ; then+ OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"+ fi+ # Now convert the arguments - kludge to limit ourselves to /bin/sh+ i=0+ for arg in "$@" ; do+ CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`+ CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option++ if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition+ eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`+ else+ eval `echo args$i`="\"$arg\""+ fi+ i=`expr $i + 1`+ done+ case $i in+ 0) set -- ;;+ 1) set -- "$args0" ;;+ 2) set -- "$args0" "$args1" ;;+ 3) set -- "$args0" "$args1" "$args2" ;;+ 4) set -- "$args0" "$args1" "$args2" "$args3" ;;+ 5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;+ 6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;+ 7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;+ 8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;+ 9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;+ esac+fi++# Escape application args+save () {+ for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done+ echo " "+}+APP_ARGS=`save "$@"`++# Collect all arguments for the java command, following the shell quoting and substitution rules+eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"++exec "$JAVACMD" "$@"diff --git a/gradlew.bat b/gradlew.batnew file mode 100644index 0000000..107acd3--- /dev/null+++ b/gradlew.bat@@ -0,0 +1,89 @@+@rem+@rem Copyright 2015 the original author or authors.+@rem+@rem Licensed under the Apache License, Version 2.0 (the "License");+@rem you may not use this file except in compliance with the License.+@rem You may obtain a copy of the License at+@rem+@rem https://www.apache.org/licenses/LICENSE-2.0+@rem+@rem Unless required by applicable law or agreed to in writing, software+@rem distributed under the License is distributed on an "AS IS" BASIS,+@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+@rem See the License for the specific language governing permissions and+@rem limitations under the License.+@rem++@if "%DEBUG%" == "" @echo off+@rem ##########################################################################+@rem+@rem Gradle startup script for Windows+@rem+@rem ##########################################################################++@rem Set local scope for the variables with windows NT shell+if "%OS%"=="Windows_NT" setlocal++set DIRNAME=%~dp0+if "%DIRNAME%" == "" set DIRNAME=.+set APP_BASE_NAME=%~n0+set APP_HOME=%DIRNAME%++@rem Resolve any "." and ".." in APP_HOME to make it shorter.+for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi++@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.+set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"++@rem Find java.exe+if defined JAVA_HOME goto findJavaFromJavaHome++set JAVA_EXE=java.exe+%JAVA_EXE% -version >NUL 2>&1+if "%ERRORLEVEL%" == "0" goto execute++echo.+echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.+echo.+echo Please set the JAVA_HOME variable in your environment to match the+echo location of your Java installation.++goto fail++:findJavaFromJavaHome+set JAVA_HOME=%JAVA_HOME:"=%+set JAVA_EXE=%JAVA_HOME%/bin/java.exe++if exist "%JAVA_EXE%" goto execute++echo.+echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME%+echo.+echo Please set the JAVA_HOME variable in your environment to match the+echo location of your Java installation.++goto fail++:execute+@rem Setup the command line++set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar+++@rem Execute Gradle+"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*++:end+@rem End local scope for the variables with windows NT shell+if "%ERRORLEVEL%"=="0" goto mainEnd++:fail+rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of+rem the _cmd.exe /c_ return code!+if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1+exit /b 1++:mainEnd+if "%OS%"=="Windows_NT" endlocal++:omegadiff --git a/reports/ball-cutie-lab.html b/reports/ball-cutie-lab.htmlnew file mode 100644index 0000000..45b86df--- /dev/null+++ b/reports/ball-cutie-lab.html@@ -0,0 +1,1878 @@+<!DOCTYPE html>+<html lang="zh-CN" data-page-node-id="hJTC176C2uAF7p7h5xRD5q">+<head data-page-node-id="4slCRHHEGMdHcTHsXmZUbE">+<meta charset="utf-8" data-page-node-id="O9HpAC3UFFxWLUftFe5zFn">+<meta name="viewport" content="width=device-width, initial-scale=1" data-page-node-id="ExHcPBiuwcXrHgpok4jDyD">+<title>球球可爱度实验室 · andee</title>+<style>+ :root{+ --bg:#f6f7f9; --panel:#ffffff; --line:#e3e6ec; --ink:#1b1f27;+ --ink2:#5b6472; --ink3:#8b95a5; --accent:#3b6ef5;+ --card:#0f172a;+ --app:#eef1f6;+ }+ *{box-sizing:border-box}+ body{+ margin:0; background:var(--bg); color:var(--ink);+ font:14px/1.6 -apple-system,BlinkMacSystemFont,"PingFang SC","Helvetica Neue",Arial,sans-serif;+ -webkit-font-smoothing:antialiased;+ }+ header{padding:20px 24px 8px;max-width:1180px;margin:0 auto}+ h1{margin:0 0 4px;font-size:19px;letter-spacing:-.01em}+ .sub{color:var(--ink2);font-size:13px}+ code{background:#eef0f4;padding:1px 5px;border-radius:4px;font-size:11.5px}+ .wrap{max-width:1180px;margin:0 auto;padding:0 24px 48px;display:grid;+ grid-template-columns:250px 1fr;gap:20px;align-items:start}+ @media(max-width:900px){.wrap{grid-template-columns:1fr}}++ .panel{background:var(--panel);border:1px solid var(--line);border-radius:12px;padding:14px}+ .panel h2{margin:0 0 10px;font-size:12px;letter-spacing:.08em;text-transform:uppercase;color:var(--ink3);font-weight:600}+ .sticky{position:sticky;top:16px}++ .presets{display:flex;flex-direction:column;gap:6px;margin-bottom:16px}+ .presets button{+ text-align:left;padding:9px 11px;border:1px solid var(--line);background:#fff;+ border-radius:9px;cursor:pointer;font:inherit;color:var(--ink);transition:.13s;+ }+ .presets button:hover{border-color:#c6cdd9}+ .presets button[aria-pressed=true]{border-color:var(--accent);background:#eef3ff;box-shadow:inset 3px 0 0 var(--accent)}+ .presets .tag{display:block;font-size:11px;color:var(--ink3);margin-top:1px}++ label.row{display:flex;justify-content:space-between;align-items:center;gap:8px;margin:11px 0 3px;font-size:12px;color:var(--ink2)}+ label.row b{font-weight:600;color:var(--ink);font-variant-numeric:tabular-nums}+ input[type=range]{width:100%;accent-color:var(--accent)}+ select{width:100%;padding:7px 9px;border:1px solid var(--line);border-radius:8px;background:#fff;font:inherit;color:var(--ink)}++ .grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(168px,1fr));gap:14px}+ .cell{background:var(--panel);border:1px solid var(--line);border-radius:12px;overflow:hidden;cursor:pointer;+ transition:border-color .13s, box-shadow .13s}+ .cell:hover{border-color:#c6cdd9}+ .cell[aria-pressed=true]{border-color:var(--accent);box-shadow:0 0 0 1px var(--accent)}+ .cell .stage{position:relative;aspect-ratio:1;display:flex;align-items:center;justify-content:center}+ .cell .stage.dark{background:var(--card)}+ .cell .stage.app{background:var(--app)}+ .cell .stage.app::before{+ content:"";position:absolute;inset:0;pointer-events:none;+ background:+ radial-gradient(circle at 24% 30%, #ffffff 0 28px, transparent 29px),+ radial-gradient(circle at 78% 70%, #ffffff 0 40px, transparent 41px),+ linear-gradient(transparent 0 44px, #ffffff 44px 46px, transparent 46px);+ opacity:.9;+ }+ .cell canvas{position:relative;display:block;width:100%;height:100%}+ .cell .cap{padding:9px 12px 11px;border-top:1px solid var(--line)}+ .cell .cap b{font-size:13px}+ .cell .cap span{display:block;font-size:11.5px;color:var(--ink3);margin-top:1px}+ .badge{position:absolute;top:8px;left:8px;font-size:10.5px;letter-spacing:.04em;+ padding:2px 7px;border-radius:20px;background:rgba(255,255,255,.14);color:#cfd6e4;+ border:1px solid rgba(255,255,255,.16);z-index:2}+ .stage.app .badge{background:rgba(15,23,42,.06);color:#5b6472;border-color:rgba(15,23,42,.1)}++ .hero{background:var(--panel);border:1px solid var(--line);border-radius:12px;padding:14px;display:grid;+ grid-template-columns:1fr 250px;gap:16px;align-items:center}+ @media(max-width:1000px){.hero{grid-template-columns:1fr}}+ .hpair{display:grid;grid-template-columns:1fr 1fr;gap:12px}+ .hstage{position:relative;border-radius:10px;display:flex;align-items:center;justify-content:center;+ padding:10px;overflow:hidden}+ .hstage.dark{background:var(--card)}+ .hstage.app{background:var(--app)}+ .hstage.app::before{content:"";position:absolute;inset:0;pointer-events:none;+ background:radial-gradient(circle at 24% 30%, #fff 0 30px, transparent 31px),+ radial-gradient(circle at 78% 72%, #fff 0 44px, transparent 45px);opacity:.9}+ .hstage canvas{position:relative;width:100%;aspect-ratio:1;display:block}+ .hbadge{position:absolute;top:8px;left:8px;font-size:10.5px;letter-spacing:.04em;z-index:2;+ padding:2px 7px;border-radius:20px;background:rgba(255,255,255,.14);color:#cfd6e4;+ border:1px solid rgba(255,255,255,.16)}+ .hstage.app .hbadge{background:rgba(15,23,42,.06);color:#5b6472;border-color:rgba(15,23,42,.1)}++ .seg{display:inline-flex;gap:0;margin-left:10px;vertical-align:middle;border:1px solid var(--line);+ border-radius:8px;overflow:hidden}+ .seg button{border:0;background:#fff;font:inherit;font-size:12px;padding:3px 11px;cursor:pointer;color:var(--ink2)}+ .seg button[aria-pressed=true]{background:var(--accent);color:#fff}+ .hero .notes{font-size:12.5px;color:var(--ink2)}+ .hero .notes h3{margin:0 0 7px;font-size:12px;letter-spacing:.08em;text-transform:uppercase;color:var(--ink3)}+ .hero .notes ul{margin:0;padding-left:16px}+ .hero .notes li{margin-bottom:5px}+ .sec{margin:22px 0 10px;font-size:13px;font-weight:600;color:var(--ink)}+ .sec em{font-style:normal;font-weight:400;color:var(--ink3);font-size:12px;margin-left:6px}+ .legend{margin-top:14px;font-size:12.5px;color:var(--ink2);border-top:1px solid var(--line);padding-top:10px}+ #err{margin:14px 0;padding:12px;border-radius:10px;background:#fff1f1;border:1px solid #f5c2c2;+ color:#a02020;font:12px/1.5 ui-monospace,SFMono-Regular,Menlo,monospace;white-space:pre-wrap;display:none}+</style>+</head>+<body data-page-node-id="SPoEpCghNXJ2BBcVKouME5">++<header data-page-node-id="C3dS7imDfzSrXCYs3EGg8D">+ <h1 data-page-node-id="LsEdSfvFA8mQT1DMHaHzSu">球球可爱度实验室</h1>+ <div class="sub" data-page-node-id="ArBs6ap0weFHb8cUrkZFZi">+ 画的就是 <code data-page-node-id="mDIEh3CoaxhsYWEXrntO86">EmotionBallRenderer</code> 每一帧在画的那批三角形,不是手绘示意图。+ 左边选预设,右边是同一个球在不同底子上的样子。每个滑块都对应代码里一个常量。+ </div>++ <div class="sub" style="margin-top:10px;padding:10px 13px;background:#eef3ff;border:1px solid #cfdcff;border-radius:9px;color:#2a3a5e">+ <b>奶油球</b>和<b>小恶魔</b>是落地的两套 —— <code>BallLook.kt</code> 里的 <code>CREAM</code> 与 <code>IMP</code>,+ 表情表、几何、材质与这个页面逐字一致。果冻球也原样保留成 <code>JELLY</code>,留作对照。+ <b>现在</b>是改动前的原样,同样留作对照。+ </div>++ <div class="sub" style="margin-top:10px;padding:10px 13px;background:#fff7ed;border:1px solid #fed7aa;border-radius:9px;color:#7c3d0a">+ ⚠️ 这一页<b>还没画得出</b>小恶魔的角、獠牙、尾巴,也没画过八戒的猪鼻。选中那两颗球时,+ <b>脸</b>是真机上那套,<b>轮廓不是</b> —— 别照着这个圆球去判断它们的边界形状。+ </div>+</header>++<div class="wrap" data-page-node-id="9HLcd6JUUvl59L0PSgyFBq">++ <div class="panel sticky" data-page-node-id="HABepjNBIjQbY5mIlxUVr6">+ <h2 data-page-node-id="C0jVPSKXrONWIuAjoSJG4Z">预设</h2>+ <div class="presets" id="presets" data-page-node-id="upEDhOFwpJRq7Mo8cyRBIX"></div>++ <h2 data-page-node-id="sEwhmEbvpEySRQWa4uBsCD">表情</h2>+ <select id="mood" data-page-node-id="TZKO9dDFXI0Dq3kLQTfNRo"></select>++ <label class="row" data-page-node-id="Dglty4qiK3L8HDmEezOsxR"><span data-page-node-id="UN9G4sHG0Bj8UOjfC0WL1F">眼睛大小</span><b id="v-eyeR" data-page-node-id="DYYLAOZzF6vimgx2W8FeL1"></b></label>+ <input type="range" id="s-eyeR" min="0.06" max="0.17" step="0.004" data-page-node-id="KIz5AOYP03RgasdHpxm6Kx">++ <label class="row" data-page-node-id="PukAk0fccJffQg6XoNWxAd"><span data-page-node-id="wdCOS4nl7TbGQFtfSlXwaB">眼睛间距</span><b id="v-eyeX" data-page-node-id="wz2ETthFuyKnu8oCAJXOCA"></b></label>+ <input type="range" id="s-eyeX" min="0.16" max="0.36" step="0.004" data-page-node-id="l9Py8pmsQ6bOWullzaj5I5">++ <label class="row" data-page-node-id="xgUeCusNQrGCXnChzeGh8h"><span data-page-node-id="TE1FMTFvdliJzcrCcU5n7N">眼睛高低</span><b id="v-eyeY" data-page-node-id="AKWbvoulvHQUsNM50PnTFy"></b></label>+ <input type="range" id="s-eyeY" min="0.06" max="0.34" step="0.004" data-page-node-id="RTBkRUjpXdFZ5Zlw2fipfN">++ <label class="row" data-page-node-id="wVfI6BzFBHrutMvuSuZXXb"><span data-page-node-id="UfS7UOXhtP6OKE1vHOz4vM">身体上色量</span><b id="v-tint" data-page-node-id="IsBZ3f64VsrgC1ajn7Aokz"></b></label>+ <input type="range" id="s-tint" min="0" max="1" step="0.02" data-page-node-id="L60TKqLya1hloUPtVYyalx">++ <label class="row" data-page-node-id="0G1NGHz2IBzHg5BWEoiVSW"><span data-page-node-id="xiIhwCG1jYQjA9sA3SW7Ge">身体明度</span><b id="v-mul" data-page-node-id="T8EjUg3i9AMkYbwgoHGH83"></b></label>+ <input type="range" id="s-mul" min="0.4" max="1.15" step="0.02" data-page-node-id="GTnQGUpdtU7uAsOFG6wM0y">++ <label class="row" data-page-node-id="iPBIDqhsHP63yTqaraCJee"><span data-page-node-id="grbkX5XBhzUEIXWHDckyml">镜面光泽</span><b id="v-gloss" data-page-node-id="NPMwWGefUK0sDS8HoVTCN5"></b></label>+ <input type="range" id="s-gloss" min="0" max="1.6" step="0.05" data-page-node-id="T1XQfesEwg8JzF2xzR9hWB">++ <label class="row" data-page-node-id="sSdrRiTlMSAfbQBsHCZ0ok"><span data-page-node-id="v98mmPaA8BH7wiQUUWMKtz">腮红</span><b id="v-blush" data-page-node-id="dSfidlMTI8HEJhKoa1CuOs"></b></label>+ <input type="range" id="s-blush" min="0" max="2.4" step="0.05" data-page-node-id="Mh0jNmmGfbyp3PUVb5fopW">++ <label class="row" data-page-node-id="EeTUqBAxMWNFe2FHi4RxOe"><span data-page-node-id="2h4F5Ra1j3mS3aCypOk2gG">嘴角上扬加成</span><b id="v-smile" data-page-node-id="hQwmt0CQZdkb33gFHrksXw"></b></label>+ <input type="range" id="s-smile" min="-0.2" max="0.45" step="0.01" data-page-node-id="xolz6DWREygBhlVGXA7f6Y">++ <label class="row"><span>鼻涕泡大小</span><b id="v-bubR"></b></label>+ <input type="range" id="s-bubR" min="0.03" max="0.16" step="0.005">++ <label class="row"><span>鼻涕泡高低</span><b id="v-bubY"></b></label>+ <input type="range" id="s-bubY" min="-0.20" max="0.05" step="0.005">++ <label class="row"><span>泡的厚度</span><b id="v-bubIn"></b></label>+ <input type="range" id="s-bubIn" min="0" max="0.5" step="0.01">++ <h2 style="margin-top:18px">耳朵</h2>++ <label class="row"><span>耳朵大小</span><b id="v-earH"></b></label>+ <input type="range" id="s-earH" min="0" max="0.55" step="0.01">++ <label class="row"><span>耳朵高低</span><b id="v-earY"></b></label>+ <input type="range" id="s-earY" min="0.40" max="0.90" step="0.01">++ <label class="row"><span>耳朵左右</span><b id="v-earX"></b></label>+ <input type="range" id="s-earX" min="0.10" max="0.55" step="0.01">++ <label class="row"><span>耳朵撇开(度)</span><b id="v-earTilt"></b></label>+ <input type="range" id="s-earTilt" min="-30" max="60" step="1">++ <label class="row"><span>耳朵暗一档</span><b id="v-earShade"></b></label>+ <input type="range" id="s-earShade" min="0.30" max="1" step="0.02">++ <h2 style="margin-top:18px">眉毛</h2>++ <label class="row"><span>眉毛长度</span><b id="v-browW"></b></label>+ <input type="range" id="s-browW" min="0" max="0.30" step="0.005">++ <label class="row"><span>眉毛高低</span><b id="v-browY"></b></label>+ <input type="range" id="s-browY" min="0.03" max="0.32" step="0.005">++ <label class="row"><span>眉毛倾角(度)</span><b id="v-browTilt"></b></label>+ <input type="range" id="s-browTilt" min="-30" max="30" step="1">++ <h2 style="margin-top:18px">眼神</h2>++ <label class="row"><span>眼睛上沿切平</span><b id="v-eyeCut"></b></label>+ <input type="range" id="s-eyeCut" min="0" max="0.85" step="0.01">++ <label class="row"><span>眼睛倾角(度)</span><b id="v-eyeTilt"></b></label>+ <input type="range" id="s-eyeTilt" min="-24" max="24" step="1">++ <label class="row"><span>瞳孔竖缝</span><b id="v-pupilAspect"></b></label>+ <input type="range" id="s-pupilAspect" min="0.25" max="1" step="0.01">++ <button id="reset" style="margin-top:14px;width:100%;padding:8px;border:1px solid var(--line);background:#fff;border-radius:9px;font:inherit;cursor:pointer;color:var(--ink2)" data-page-node-id="MsiIXrvrt2aqAPJ2AEw1OU">+ 还原此预设+ </button>++ <div class="legend" data-page-node-id="YJVBjoZyCoFB3Q0bfWn9JZ">+ 瞳孔大小 / 高光由预设决定,不给滑块 —— 那是造型选择,不是调参。+ <b>瞳孔竖缝</b>例外:那是"换物种"那一档,值得单独试。+ </div>+ </div>++ <div data-page-node-id="xHcP9fBoFkC2vu3VH59FJR">+ <div id="err" data-page-node-id="iSvxKDZoG2BMcUjEaWOwfP"></div>++ <div class="sec">放大细节 <em>选中的预设 · 两种底子并排看</em></div>+ <div class="hero">+ <div class="hpair">+ <div class="hstage dark"><div class="hbadge">深色卡片</div><canvas class="heroCv"></canvas></div>+ <div class="hstage app"><div class="hbadge">亮色 App</div><canvas class="heroCv"></canvas></div>+ </div>+ <div class="notes">+ <h3>在改什么</h3>+ <ul>+ <li><b>眼睛</b>:改前半径 0.075、间距 ±0.30、位置 y=0.30 —— 又小又高又分家,那是一张机器人的脸。</li>+ <li><b>幼态三件套</b>:放大、往中间收、往下降,额头留大。</li>+ <li><b>身体</b>:改前是纯平近黑色。顶光渐变 + 高光点,球才有体积。</li>+ <li><b>眼睛内容</b>:白眼球加瞳孔,或深色豆豆眼加一点白高光。</li>+ <li><b>腮红</b>:改前只有 HAPPY/SUCCESS 有 —— 静止那张脸一整天没有脸颊。</li>+ <li><b>嘴</b>:改前是条发丝细的弧,80dp 的球上读不出表情。</li>+ </ul>+ <h3 style="margin-top:12px">量出来的数(占球径)</h3>+ <ul>+ <li>奶油球(当前落地):眼宽 <b>0.17</b> · 长宽比 <b>1.25</b> · 内距 <b>0.20</b> · 嘴宽 <b>0.29</b></li>+ <li>果冻球(备选):眼宽 0.18 · 内距 0.17 · 眼心高 0.22</li>+ <li>改前对应是 0.10 / 2.25 / 0.30 / 0.37。</li>+ <li>眼心从球心上方 40% 降到 <b>22%</b> —— 额头留大,这是幼态脸的算法。</li>+ <li>腮红:脸颊像素从 <code>(57,66,95)</code> 抬到 <code>(138,100,140)</code>。</li>+ </ul>+ </div>+ </div>++ <div class="sec">+ 七套方案 <em>168dp 窗口的 1:1 真实尺寸 · 点一个看大图</em>+ <span class="seg" id="bgToggle">+ <button type="button" data-bg="dark" aria-pressed="true">深色卡片</button>+ <button type="button" data-bg="app" aria-pressed="false">亮色 App</button>+ </span>+ </div>+ <div class="grid" id="grid"></div>+ </div>+</div>++<script>+"use strict";+/* =========================================================================+ 1. mat4 —— 列主序,跟 android.opengl.Matrix 同一套约定+ ========================================================================= */+const M4 = {+ id(){ return new Float32Array([1,0,0,0, 0,1,0,0, 0,0,1,0, 0,0,0,1]); },+ perspective(fovyDeg, aspect, near, far){+ const f = 1/Math.tan(fovyDeg*Math.PI/180/2), m = new Float32Array(16);+ m[0]=f/aspect; m[5]=f; m[10]=(far+near)/(near-far); m[11]=-1;+ m[14]=2*far*near/(near-far); return m;+ },+ lookAt(ex,ey,ez, cx,cy,cz, ux,uy,uz){+ let zx=ex-cx, zy=ey-cy, zz=ez-cz, l=Math.hypot(zx,zy,zz);+ zx/=l; zy/=l; zz/=l;+ let xx=uy*zz-uz*zy, xy=uz*zx-ux*zz, xz=ux*zy-uy*zx;+ l=Math.hypot(xx,xy,xz)||1; xx/=l; xy/=l; xz/=l;+ const yx=zy*xz-zz*xy, yy=zz*xx-zx*xz, yz=zx*xy-zy*xx;+ const m=new Float32Array(16);+ m[0]=xx; m[4]=xy; m[8]=xz; m[12]=-(xx*ex+xy*ey+xz*ez);+ m[1]=yx; m[5]=yy; m[9]=yz; m[13]=-(yx*ex+yy*ey+yz*ez);+ m[2]=zx; m[6]=zy; m[10]=zz; m[14]=-(zx*ex+zy*ey+zz*ez);+ m[15]=1; return m;+ },+ mul(a,b){+ const o=new Float32Array(16);+ for(let c=0;c<4;c++)for(let r=0;r<4;r++){+ let s=0; for(let k=0;k<4;k++) s+=a[k*4+r]*b[c*4+k];+ o[c*4+r]=s;+ } return o;+ },+ translate(m,x,y,z){ const t=M4.id(); t[12]=x;t[13]=y;t[14]=z; return M4.mul(m,t); },+ scale(m,x,y,z){ const t=M4.id(); t[0]=x;t[5]=y;t[10]=z; return M4.mul(m,t); },+ rotate(m,deg,ax,ay,az){+ let l=Math.hypot(ax,ay,az); ax/=l; ay/=l; az/=l;+ const a=deg*Math.PI/180, s=Math.sin(a), c=Math.cos(a), t=1-c, r=M4.id();+ r[0]=t*ax*ax+c; r[1]=t*ax*ay+s*az; r[2]=t*ax*az-s*ay;+ r[4]=t*ax*ay-s*az; r[5]=t*ay*ay+c; r[6]=t*ay*az+s*ax;+ r[8]=t*ax*az+s*ay; r[9]=t*ay*az-s*ax; r[10]=t*az*az+c;+ return M4.mul(m,r);+ },+};++/* =========================================================================+ 2. 几何体 —— IcoSphere / UvSphere / Capsule / Torus 的逐行移植+ ========================================================================= */+function icoSphere(subdivisions){+ const t=(1+Math.sqrt(5))/2;+ const base=[-1,t,0, 1,t,0, -1,-t,0, 1,-t,0,+ 0,-1,t, 0,1,t, 0,-1,-t, 0,1,-t,+ t,0,-1, t,0,1, -t,0,-1, -t,0,1];+ const pos=[];+ for(let i=0;i<12;i++){+ const x=base[i*3],y=base[i*3+1],z=base[i*3+2], inv=1/Math.hypot(x,y,z);+ pos.push(x*inv,y*inv,z*inv);+ }+ let faces=[0,11,5, 0,5,1, 0,1,7, 0,7,10, 0,10,11,+ 1,5,9, 5,11,4, 11,10,2, 10,7,6, 7,1,8,+ 3,9,4, 3,4,2, 3,2,6, 3,6,8, 3,8,9,+ 4,9,5, 2,4,11, 6,2,10, 8,6,7, 9,8,1];+ const cache=new Map();+ function mid(a,b){+ const lo=Math.min(a,b), hi=Math.max(a,b), key=lo*100000+hi;+ if(cache.has(key)) return cache.get(key);+ let mx=(pos[a*3]+pos[b*3])/2, my=(pos[a*3+1]+pos[b*3+1])/2, mz=(pos[a*3+2]+pos[b*3+2])/2;+ const inv=1/Math.hypot(mx,my,mz); mx*=inv; my*=inv; mz*=inv;+ const idx=pos.length/3; pos.push(mx,my,mz); cache.set(key,idx); return idx;+ }+ for(let s=0;s<subdivisions;s++){+ const next=[];+ for(let r=0;r<faces.length;r+=3){+ const a=faces[r],b=faces[r+1],c=faces[r+2];+ const ab=mid(a,b), bc=mid(b,c), ca=mid(c,a);+ next.push(a,ab,ca, b,bc,ab, c,ca,bc, ab,bc,ca);+ }+ faces=next; cache.clear();+ }+ const n=pos.length/3;+ return {pos:new Float32Array(pos), nrm:new Float32Array(pos),+ uv:new Float32Array(n*2), idx:new Uint16Array(faces)};+}++function uvSphere(wSeg,hSeg,radius=1){+ const n=(wSeg+1)*(hSeg+1);+ const pos=new Float32Array(n*3), nrm=new Float32Array(n*3), uv=new Float32Array(n*2);+ let p=0,u=0;+ for(let iy=0;iy<=hSeg;iy++){+ const v=iy/hSeg, phi=v*Math.PI, sPhi=Math.sin(phi), cPhi=Math.cos(phi);+ for(let ix=0;ix<=wSeg;ix++){+ const th=(ix/wSeg)*2*Math.PI;+ const x=-radius*Math.cos(th)*sPhi, y=radius*cPhi, z=radius*Math.sin(th)*sPhi;+ pos[p]=x; pos[p+1]=y; pos[p+2]=z;+ nrm[p]=x/radius; nrm[p+1]=y/radius; nrm[p+2]=z/radius;+ uv[u]=ix/wSeg; uv[u+1]=1-v; p+=3; u+=2;+ }+ }+ const raw=[], stride=wSeg+1;+ for(let iy=0;iy<hSeg;iy++)for(let ix=0;ix<wSeg;ix++){+ const a=iy*stride+ix+1, b=iy*stride+ix, c=(iy+1)*stride+ix, d=(iy+1)*stride+ix+1;+ if(iy!==0) raw.push(a,b,d);+ if(iy!==hSeg-1) raw.push(b,c,d);+ }+ return {pos,nrm,uv,idx:new Uint16Array(raw)};+}++function capsule(radius, cylH, capSegs=4, radialSegs=12){+ const half=cylH/2, totalRings=2*(capSegs+1), verts=totalRings*(radialSegs+1);+ const pos=new Float32Array(verts*3), nrm=new Float32Array(verts*3), uv=new Float32Array(verts*2);+ let p=0,u=0,ring=0;+ for(const sign of [1,-1]){+ for(let i=0;i<=capSegs;i++){+ const phi=sign>0 ? (i/capSegs)*(Math.PI/2) : Math.PI/2+(i/capSegs)*(Math.PI/2);+ const sP=Math.sin(phi), cP=Math.cos(phi);+ for(let j=0;j<=radialSegs;j++){+ const th=(j/radialSegs)*2*Math.PI;+ const nx=-Math.cos(th)*sP, ny=cP, nz=Math.sin(th)*sP;+ pos[p]=radius*nx; pos[p+1]=radius*ny+sign*half; pos[p+2]=radius*nz;+ nrm[p]=nx; nrm[p+1]=ny; nrm[p+2]=nz;+ uv[u]=j/radialSegs; uv[u+1]=1-ring/(totalRings-1); p+=3; u+=2;+ }+ ring++;+ }+ }+ const idx=[], stride=radialSegs+1;+ for(let r=0;r<totalRings-1;r++)for(let s=0;s<radialSegs;s++){+ const a=r*stride+s, b=r*stride+s+1, c=(r+1)*stride+s+1, d=(r+1)*stride+s;+ idx.push(a,b,c, a,c,d);+ }+ return {pos,nrm,uv,idx:new Uint16Array(idx)};+}++/**+ * 眼睛的**上沿切平**版 —— gl/Eye.kt 的逐行移植。+ *+ * 切出来的那条平边**就是眉毛**。这一页最要紧的一句是:白色眼球压在近黑的+ * 身上本来就是屏幕能给出的最大对比,所以"凶"根本不需要在眼睛旁边再画一笔,+ * 把眼睛自己的上沿切平、再歪一点,那条边就读成压下来的眉。那个画法还顺手+ * 拆掉了另一个陷阱:**盖在眼睛上的东西必须同时"对身体不可见"和"对眼睛可见"**+ * ——身体是打过光的、平涂不是,两件事不在同一个尺度上(`browShade` 第一步错+ * 就错在这里)。切几何没有颜色可以错。+ *+ * 和 [capsule] 结构相同(沿 Y 一摞环,北极在下、中间一段圆柱),只多两件事:+ * 扫到切口就停,然后用自己的一个环加圆心**补一片平盖**。+ *+ * ⚠️ **切的角度不在这里**。一道水平的平边是一双**困**眼睛;让它成为眉的是+ * 那条线**歪着**,而歪是调用方把整颗眼睛转出来的(`p.eyeTilt` 加表情自己的+ * brow)。角度留在绘制那一步,这个函数就还是一颗**形状**,而形状能被动画改。+ *+ * @param {number} radius 半宽+ * @param {number} cylH 圆柱段长度,同 [capsule]+ * @param {number} topCut 上半个眼睛切掉多少,占半高的比例:0 = 不切,+ * 0.5 = 平边落在圆心与天然顶端的中点上。夹到 0..MAX_CUT。+ * @param {number} capSegs 每个半球跨几环+ * @param {number} radialSegs 每环几个点+ */+function eyeMesh(radius, cylH, topCut, capSegs=4, radialSegs=12){+ // 和 gl/Eye.kt 的 MAX_TOP_CUT 同值。这是**造型**上限不是几何上限:到 0.85+ // 切口仍在圆心之上,再往下就降到圆心以下,那已经不是"一只被压住的眼睛"了。+ // 真正的天花板比它早得多,而且不在这个网格里 —— 瞳孔是画在眼球**前面**的+ // 另一片,切口只切眼球自己,切到瞳孔顶上就会有一颗黑豆从平盖上立出来。+ // 那条线由 pupilR 决定,所以在 BallLook.eyeCut 那边算。见 P_IMP 的注释。+ const MAX_CUT=0.85;+ const half=cylH/2, halfHeight=radius+half;+ const cut=Math.min(MAX_CUT,Math.max(0,topCut));+ const yCut=halfHeight*(1-cut);++ // 环按 (y, r) 收进数组而不是先数好再声明类型化数组:环数取决于切口落在+ // 上半球上还是落在圆柱段上,数错了是**静默**的 —— 类型化数组越界写入直接+ // 丢弃,症状是"网格建出来了、不报错、上面有个洞"。+ const ys=[], rs=[];+ // 下半球:phi π → π/2,北极在前,正好收在下赤道上。+ for(let i=0;i<=capSegs;i++){+ const phi=Math.PI-(i/capSegs)*(Math.PI/2);+ ys.push(radius*Math.cos(phi)-half);+ rs.push(radius*Math.sin(phi));+ }+ if(yCut>half){+ // 切在上半球里。下赤道就是刚加进去的那一环,所以补上上赤道、让两者之间的+ // 四边形当圆柱壁([capsule] 就是这么干的),然后扫到切口。+ ys.push(half); rs.push(radius);+ const phiCut=Math.acos(Math.min(1,Math.max(-1,(yCut-half)/radius)));+ for(let i=1;i<=capSegs;i++){+ const phi=Math.PI/2-(i/capSegs)*(Math.PI/2-phiCut);+ ys.push(radius*Math.cos(phi)+half);+ rs.push(radius*Math.sin(phi));+ }+ } else {+ // 切口落在上赤道或更低:圆柱段直接顶到切口,没有半球可扫。+ //+ // 上赤道这一环是**跳过**的,不是"加完再补切口"。加了就会让某一环排在+ // 它后面那一环之上,条带自己折回去 —— 而折回去不是报错,是一颗渲染出来+ // 里外翻转的网格。这一支在真实数值下**是够得到的**:小恶魔的眼睛半高+ // 0.1755、half 0.0275,任何大于 ~0.84 的切口都落在这里。+ ys.push(yCut); rs.push(radius);+ }++ const ringCount=ys.length, stride=radialSegs+1;+ const vertCount=ringCount*stride+stride+1;+ const pos=new Float32Array(vertCount*3), nrm=new Float32Array(vertCount*3), uv=new Float32Array(vertCount*2);+ let p=0,u=0;+ for(let i=0;i<ringCount;i++){+ const y=ys[i], r=rs[i];+ // 这一环属于哪个球决定了法线。两个赤道环落在边界上,公式在两边都退化成+ // 径向,所以跨过去是连续的。+ const cy = y<=-half ? -half : (y>=half ? half : 0);+ const onCyl = y>-half && y<half;+ for(let j=0;j<=radialSegs;j++){+ const th=(j/radialSegs)*2*Math.PI;+ const nx=-Math.cos(th), nz=Math.sin(th);+ pos[p]=r*nx; pos[p+1]=y; pos[p+2]=r*nz;+ if(onCyl){ nrm[p]=nx; nrm[p+1]=0; nrm[p+2]=nz; }+ else { nrm[p]=nx*r/radius; nrm[p+1]=(y-cy)/radius; nrm[p+2]=nz*r/radius; }+ uv[u]=j/radialSegs; uv[u+1]=1-i/Math.max(1,ringCount-1);+ p+=3; u+=2;+ }+ }++ // 切口那片盖子,用**自己的**一环而不是复用扫出来的最后一环:最后一环带的是+ // 半球法线,继承过来的盖子会是一片"还被当成曲面在打光"的平盘 —— 眼睛顶上+ // 一道亮糊。今天眼睛走的是平涂、根本不读法线,这正是它值得现在做对的原因。+ const capRing=ringCount*stride, capCenter=capRing+stride, rCut=rs[ringCount-1];+ pos[capCenter*3]=0; pos[capCenter*3+1]=yCut; pos[capCenter*3+2]=0;+ nrm[capCenter*3+1]=1; uv[capCenter*2]=0.5; uv[capCenter*2+1]=1;+ for(let j=0;j<=radialSegs;j++){+ const th=(j/radialSegs)*2*Math.PI;+ const nx=-Math.cos(th), nz=Math.sin(th);+ const v=capRing+j;+ pos[v*3]=rCut*nx; pos[v*3+1]=yCut; pos[v*3+2]=rCut*nz;+ nrm[v*3+1]=1; uv[v*2]=j/radialSegs; uv[v*2+1]=0;+ }++ const idx=new Uint16Array((ringCount-1)*radialSegs*6+radialSegs*3);+ let w=0;+ for(let ring=0;ring<ringCount-1;ring++)for(let s=0;s<radialSegs;s++){+ const a=ring*stride+s, b=ring*stride+s+1, c=(ring+1)*stride+s+1, d=(ring+1)*stride+s;+ idx[w++]=a; idx[w++]=b; idx[w++]=c;+ idx[w++]=a; idx[w++]=c; idx[w++]=d;+ }+ for(let s=0;s<radialSegs;s++){+ idx[w++]=capCenter; idx[w++]=capRing+s; idx[w++]=capRing+s+1;+ }+ return {pos,nrm,uv,idx};+}++function torus(radius, tube, radialSegs, tubularSegs, arc=2*Math.PI){+ const verts=(radialSegs+1)*(tubularSegs+1);+ const pos=new Float32Array(verts*3), nrm=new Float32Array(verts*3), uv=new Float32Array(verts*2);+ let p=0,u=0;+ for(let j=0;j<=radialSegs;j++){+ const v=(j/radialSegs)*2*Math.PI, cV=Math.cos(v), sV=Math.sin(v);+ for(let i=0;i<=tubularSegs;i++){+ const uu=(i/tubularSegs)*arc, cU=Math.cos(uu), sU=Math.sin(uu);+ const rr=radius+tube*cV;+ const px=rr*cU, py=rr*sU, pz=tube*sV;+ pos[p]=px; pos[p+1]=py; pos[p+2]=pz;+ let nx=px-radius*cU, ny=py-radius*sU, nz=pz;+ const inv=1/Math.hypot(nx,ny,nz);+ nrm[p]=nx*inv; nrm[p+1]=ny*inv; nrm[p+2]=nz*inv;+ uv[u]=i/tubularSegs; uv[u+1]=j/radialSegs; p+=3; u+=2;+ }+ }+ const idx=[], stride=tubularSegs+1;+ for(let j=0;j<radialSegs;j++)for(let i=0;i<tubularSegs;i++){+ const a=j*stride+i, b=j*stride+i+1, c=(j+1)*stride+i+1, d=(j+1)*stride+i;+ idx.push(a,b,c, a,c,d);+ }+ return {pos,nrm,uv,idx:new Uint16Array(idx)};+}++/**+ * 一只耳 —— **gl/Ear.kt 的逐行移植**,不是另画一个形状。+ *+ * 这个函数原来是 `cone()`:一根正圆锥(椭圆截面,尖顶)。真机上的耳朵早就不是它了+ * ——`Ear.build` 是一摞沿曲线轴走的椭圆环,截面扁、**尖端是圆的**,参数也完全不同+ * (width/height/thickness/bend,而不是 rx/rz/height)。两边形状不一致的代价是:+ * 用户在实验室里把耳朵调到自己满意,装到机器上是另一个东西,而且两边都"看起来对"。+ * 所以这里按 Kotlin 原文抄一遍,参数名也对齐。+ *+ * 三件让它成为"耳"而不是"角"的事,都在下面看得见:+ * * 截面是**椭圆**(宽 w、薄 th),所以是一片扁的,不是一根锥;+ * * 轮廓按 **1/4 椭圆收口**(`k = √(1-t²)`,且下限 0.08),所以尖端是一个+ * 小圆头而不是一个点 —— 尖的读作角;+ * * 它自己不长,靠调用方把它**转出去**([-earTilt] 那一步)才是耳朵的姿态。+ *+ * @param {number} width 根部半宽(在弯曲平面内)+ * @param {number} height 沿弯曲轴的长度+ * @param {number} thickness z 方向的半厚(远小于 width,扁平就是"片"的来源)+ * @param {number} bend 从根到尖的总转角,**弧度**(调用方负责从度转过来)+ */+function earMesh(width, height, thickness, bend=0, segs=6, radialSegs=10){+ // 顶点数 = 环数 × (每环点数)。写环这类网格先把数数一遍再声明数组:+ // 类型化数组越界写入是**静默丢弃**的,症状是"网格建出来了、不报错、就是画不出来"。+ const rings=segs+1, vertCount=rings*(radialSegs+1);+ const pos=new Float32Array(vertCount*3), nrm=new Float32Array(vertCount*3), uv=new Float32Array(vertCount*2);+ const step=height/segs;+ let cx=0, cy=0, pi=0, ui=0;+ for(let i=0;i<rings;i++){+ const t=i/segs;+ // 尖端保留 8% 的宽度:既是一颗圆头,也保证下面法线不会除以 0。+ const k=Math.max(Math.sqrt(1-t*t),0.08);+ const w=width*k, th=thickness*k;+ const ang=bend*t;+ const ax=Math.sin(ang), ay=Math.cos(ang);+ // u ⊥ a,都在 XY 平面内;第三个方向就是 Z。+ const ux=ay, uy=-ax;+ for(let j=0;j<=radialSegs;j++){+ const tht=j/radialSegs*2*Math.PI, ct=Math.cos(tht), st=Math.sin(tht);+ pos[pi++]=cx+w*ct*ux;+ pos[pi++]=cy+w*ct*uy;+ pos[pi++]=th*st;+ // 椭圆 (w,th) 的表面法线 ∝ (ct/w, st/th)。+ const nu=ct/w, nz=st/th;+ const inv=1/Math.sqrt(nu*nu+nz*nz);+ nrm[pi-3]=nu*inv*ux;+ nrm[pi-2]=nu*inv*uy;+ nrm[pi-1]=nz*inv;+ uv[ui++]=j/radialSegs;+ uv[ui++]=t;+ }+ cx+=ax*step;+ cy+=ay*step;+ }+ const idx=new Uint16Array(segs*radialSegs*6);+ let m=0;+ for(let i=0;i<segs;i++)for(let j=0;j<radialSegs;j++){+ const a=i*(radialSegs+1)+j, b=a+1, c=(i+1)*(radialSegs+1)+j, d=c+1;+ idx[m++]=a; idx[m++]=c; idx[m++]=b;+ idx[m++]=b; idx[m++]=c; idx[m++]=d;+ }+ return {pos,nrm,uv,idx};+}++/* =========================================================================+ 3. GL 小工具+ ========================================================================= */+function compile(gl,type,src){+ const s=gl.createShader(type);+ gl.shaderSource(s,src); gl.compileShader(s);+ if(!gl.getShaderParameter(s,gl.COMPILE_STATUS))+ throw new Error(gl.getShaderInfoLog(s)+"\n---\n"+src);+ return s;+}+class Prog{+ constructor(gl,vs,fs){+ this.gl=gl;+ const p=gl.createProgram();+ gl.attachShader(p,compile(gl,gl.VERTEX_SHADER,vs));+ gl.attachShader(p,compile(gl,gl.FRAGMENT_SHADER,fs));+ gl.linkProgram(p);+ if(!gl.getProgramParameter(p,gl.LINK_STATUS))+ throw new Error(gl.getProgramInfoLog(p));+ this.p=p; this.cache=new Map();+ }+ use(){ this.gl.useProgram(this.p); }+ u(n){ const k="u:"+n; if(!this.cache.has(k)) this.cache.set(k,this.gl.getUniformLocation(this.p,n)); return this.cache.get(k); }+ a(n){ const k="a:"+n; if(!this.cache.has(k)) this.cache.set(k,this.gl.getAttribLocation(this.p,n)); return this.cache.get(k); }+}++class GMesh{+ constructor(gl,data){+ this.gl=gl;+ const n=data.pos.length/3, stride=8;+ const inter=new Float32Array(n*stride);+ for(let i=0;i<n;i++){+ const j=i*stride;+ inter[j]=data.pos[i*3]; inter[j+1]=data.pos[i*3+1]; inter[j+2]=data.pos[i*3+2];+ inter[j+3]=data.nrm[i*3]; inter[j+4]=data.nrm[i*3+1]; inter[j+5]=data.nrm[i*3+2];+ inter[j+6]=data.uv[i*2]; inter[j+7]=data.uv[i*2+1];+ }+ this.vbo=gl.createBuffer();+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.bufferData(gl.ARRAY_BUFFER,inter,gl.STATIC_DRAW);+ this.ibo=gl.createBuffer();+ gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER,this.ibo);+ gl.bufferData(gl.ELEMENT_ARRAY_BUFFER,data.idx,gl.STATIC_DRAW);+ this.count=data.idx.length;+ }+ bind(aPos,aNrm=-1,aUV=-1){+ const gl=this.gl, S=8*4;+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER,this.ibo);+ if(aPos>=0){ gl.enableVertexAttribArray(aPos); gl.vertexAttribPointer(aPos,3,gl.FLOAT,false,S,0); }+ if(aNrm>=0){ gl.enableVertexAttribArray(aNrm); gl.vertexAttribPointer(aNrm,3,gl.FLOAT,false,S,12); }+ if(aUV>=0){ gl.enableVertexAttribArray(aUV); gl.vertexAttribPointer(aUV,2,gl.FLOAT,false,S,24); }+ }+ draw(){ this.gl.drawElements(this.gl.TRIANGLES,this.count,this.gl.UNSIGNED_SHORT,0); }+}++class GPoints{+ constructor(gl,capacity){+ this.gl=gl; this.capacity=capacity;+ this.data=new Float32Array(capacity*6);+ this.vbo=gl.createBuffer();+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.bufferData(gl.ARRAY_BUFFER,capacity*6*4,gl.DYNAMIC_DRAW);+ }+ upload(){+ const gl=this.gl;+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.bufferSubData(gl.ARRAY_BUFFER,0,this.data);+ }+ bind(aPos,aCol){+ const gl=this.gl, S=6*4;+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.enableVertexAttribArray(aPos); gl.vertexAttribPointer(aPos,3,gl.FLOAT,false,S,0);+ gl.enableVertexAttribArray(aCol); gl.vertexAttribPointer(aCol,3,gl.FLOAT,false,S,12);+ }+ draw(){ this.gl.drawArrays(this.gl.POINTS,0,this.capacity); }+}++/* =========================================================================+ 4. 表情表 —— 与 Mood.kt 的 FACE 表一一对应+ ========================================================================= */+function F(o){ return Object.assign({eyeArc:0,blush:0,spread:1,breath:0.022,gape:0,bubble:0},o); }++/* 改动前的表 —— "现在"这个预设专用,做诚实的基线。 */+const FACE_OLD = {+ CALM: F({c:0x9db4ff, eye:1.00, brow:0.00, smile:0.10, pulse:1.00, tilt:0.00, skew:0.00, jitter:0.00}),+ HAPPY: F({c:0x5eead4, eye:0.72, brow:-0.16,smile:0.62, pulse:1.25, tilt:0.04, skew:0.00, jitter:0.00, eyeArc:0.90, blush:0.70}),+ CURIOUS: F({c:0xf0abfc, eye:1.22, brow:-0.06,smile:0.14, pulse:1.15, tilt:0.17, skew:0.055,jitter:0.00}),+ TENSE: F({c:0xfb923c, eye:0.88, brow:0.30, smile:-0.20,pulse:1.95, tilt:0.00, skew:0.02, jitter:0.55}),+ ANXIOUS: F({c:0xfb7185, eye:1.26, brow:0.42, smile:-0.44,pulse:2.50, tilt:0.00, skew:0.03, jitter:1.00}),+ CONCERNED: F({c:0xfbbf24, eye:1.10, brow:0.34, smile:-0.34,pulse:1.15, tilt:-0.05,skew:0.00, jitter:0.12}),+ LISTENING: F({c:0x7dd3fc, eye:1.60, brow:-0.14,smile:0.16, pulse:1.70, tilt:-0.13,skew:0.00, jitter:0.00, breath:0.055, gape:0.12}),+ HOLDING: F({c:0xfacc15, eye:0.92, brow:0.48, smile:-0.18,pulse:0.72, tilt:0.00, skew:0.00, jitter:0.00, breath:0.018}),+ THINKING: F({c:0xc084fc, eye:0.40, brow:0.32, smile:-0.06,pulse:1.15, tilt:0.15, skew:0.08, jitter:0.00, breath:0.042}),+ SPEAKING: F({c:0x9db4ff, eye:1.05, brow:0.00, smile:0.20, pulse:1.50, tilt:0.00, skew:0.00, jitter:0.00}),+ SEARCHING: F({c:0x7dd3fc, eye:0.84, brow:0.10, smile:0.06, pulse:1.60, tilt:0.13, skew:0.05, jitter:0.00, spread:1.16, breath:0.036}),+ SUCCESS: F({c:0x5eead4, eye:0.50, brow:-0.22,smile:0.80, pulse:1.30, tilt:0.00, skew:0.00, jitter:0.00, eyeArc:1.00, blush:0.95, spread:1.10}),+ FAILED: F({c:0xfb7185, eye:0.68, brow:0.45, smile:-0.50,pulse:1.10, tilt:-0.07,skew:0.02, jitter:0.22}),+ SLEEPING: F({c:0x64748b, eye:0.10, brow:0.06, smile:0.08, pulse:0.45, tilt:0.11, skew:0.00, jitter:0.00, eyeArc:0.85, spread:0.90}),+};++/* 改动后的表 —— 与 Mood.kt 的 FACE 逐字对齐,这样这里看到的就是装机后的样子。+ smile 与 blush 已经把"基线"烘进表里了,不再有额外的全局加成。 */+const FACE_NEW = {+ CALM: F({c:0x9db4ff, eye:1.00, brow:0.00, smile:0.26, pulse:1.00, tilt:0.00, skew:0.00, jitter:0.00, blush:0.38}),+ HAPPY: F({c:0x5eead4, eye:0.72, brow:-0.16,smile:0.68, pulse:1.25, tilt:0.04, skew:0.00, jitter:0.00, eyeArc:0.90, blush:0.85}),+ CURIOUS: F({c:0xf0abfc, eye:1.22, brow:-0.06,smile:0.28, pulse:1.15, tilt:0.17, skew:0.055,jitter:0.00, blush:0.72}),+ TENSE: F({c:0xfb923c, eye:0.88, brow:0.30, smile:-0.20,pulse:1.95, tilt:0.00, skew:0.02, jitter:0.55, blush:0.26}),+ ANXIOUS: F({c:0xfb7185, eye:1.26, brow:0.42, smile:-0.44,pulse:2.50, tilt:0.00, skew:0.03, jitter:1.00, blush:0.24}),+ CONCERNED: F({c:0xfbbf24, eye:1.10, brow:0.34, smile:-0.34,pulse:1.15, tilt:-0.05,skew:0.00, jitter:0.12, blush:0.32}),+ LISTENING: F({c:0x7dd3fc, eye:1.60, brow:-0.14,smile:0.28, pulse:1.70, tilt:-0.13,skew:0.00, jitter:0.00, blush:0.46, breath:0.055, gape:0.12}),+ HOLDING: F({c:0xfacc15, eye:0.92, brow:0.48, smile:-0.18,pulse:0.72, tilt:0.00, skew:0.00, jitter:0.00, blush:0.3, breath:0.018}),+ THINKING: F({c:0xc084fc, eye:0.40, brow:0.32, smile:0.04, pulse:1.15, tilt:0.15, skew:0.08, jitter:0.00, blush:0.3, breath:0.042}),+ SPEAKING: F({c:0x9db4ff, eye:1.05, brow:0.00, smile:0.32, pulse:1.50, tilt:0.00, skew:0.00, jitter:0.00, blush:0.44}),+ SEARCHING: F({c:0x7dd3fc, eye:0.84, brow:0.10, smile:0.16, pulse:1.60, tilt:0.13, skew:0.05, jitter:0.00, blush:0.34, spread:1.16, breath:0.036}),+ SUCCESS: F({c:0x5eead4, eye:0.50, brow:-0.22,smile:0.86, pulse:1.30, tilt:0.00, skew:0.00, jitter:0.00, eyeArc:1.00, blush:1.0, spread:1.10}),+ FAILED: F({c:0xfb7185, eye:0.68, brow:0.45, smile:-0.50,pulse:1.10, tilt:-0.07,skew:0.02, jitter:0.22, blush:0.42}),+ SLEEPING: F({c:0x9db4ff, eye:0.10, brow:0.06, smile:0.16, pulse:0.45, tilt:0.11, skew:0.00, jitter:0.00, eyeArc:0.85, blush:0.16, spread:0.90, bubble:1}),+};++const MOODS = Object.keys(FACE_NEW);+for(const T of [FACE_OLD, FACE_NEW]) for(const k in T) T[k].color = hex2rgb(T[k].c);+function hex2rgb(h){ return [((h>>16)&255)/255, ((h>>8)&255)/255, (h&255)/255]; }++/* =========================================================================+ 5. 预设 —— 每个数都对应 Kotlin 里一个常量+ ========================================================================= */+const P_NOW = {+ key:'now', label:'现在', tag:'近黑球 + 白胶囊眼 + 发光描边',+ bodyBase:[0.102,0.137,0.220], tint:0.20, albedoMul:1.00, grad:0.00,+ amb:1.00, diff:0.00, gloss:0.00, bodyScale:0.885,+ eyeStyle:'plain', eyeColor:[1,1,1], pupilR:0, specR:0,+ eyeR:0.075, eyeH:0.100, eyeX:0.300, eyeY:0.300,+ eyeCut:0, eyeTilt:0, pupilAspect:1,+ faceZ:0.860, mouthR:0.240, mouthTube:0.036, mouthY:-0.340, mouthThick:0.00,+ mouthMode:'mood', smileBoost:0.00, faces:FACE_OLD,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.55, blushBlend:'add', blushSize:0.34, blushX:0.440, blushY:-0.120,+ mouthStyle:'arc',+ earH:0, earX:0.360, earY:0.700, earZ:0.360, earW:0.155, earThick:0.056, earTilt:24, earBend:0, earShade:0.86,+ browW:0, browH:0.032, browY:0.150, browTilt:0, browShade:0,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.00, rimPow:2.6,+};++const P_JELLY = {+ key:'jelly', label:'果冻球', tag:'糖果色光泽身体 + 大眼带瞳孔 + 腮红',+ bodyBase:[0.055,0.072,0.130], tint:0.62, albedoMul:0.88, grad:0.60,+ amb:0.66, diff:0.55, gloss:0.70, bodyScale:0.885,+ eyeStyle:'pupil', eyeColor:[1,1,1], pupilR:0.070, specR:0.024,+ eyeR:0.135, eyeH:0.050, eyeX:0.262, eyeY:0.175,+ eyeCut:0, eyeTilt:0, pupilAspect:1,+ faceZ:0.876, mouthR:0.175, mouthTube:0.040, mouthY:-0.235, mouthThick:0.06,+ mouthMode:'mood', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.85, blushBlend:'add', blushSize:0.44, blushX:0.400, blushY:-0.145,+ mouthStyle:'arc',+ earH:0, earX:0.360, earY:0.700, earZ:0.360, earW:0.155, earThick:0.056, earTilt:24, earBend:0, earShade:0.86,+ browW:0, browH:0.032, browY:0.150, browTilt:0, browShade:0,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.920,0.950,1.000], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.15, rimPow:2.6,+};++/* --- 奶油球(已落地的那套)---+ 2026-09-30:长了一对**立耳**;同一天把**眉毛去掉了**。++ "机灵"不是一种颜色,是形状 —— 圆球加一对立耳,读出来的立刻是动物,动物才+ 谈得上机敏。原先还有一对画上去的眉毛(browW 0.145 / browH 0.032 / browY+ 0.185 / browTilt 5),**已按用户要求删除**,剩下耳朵单独扛这个读法。++ 删得掉是有算术支撑的:当初选 browY 的注释说眉下沿离眼上沿留了 0.031,+ 但那是**把眼睛当成一颗球算的**(只算了 eyeRadius 0.122),漏了它的圆柱段。+ 眼睛真正的半高是 eyeRadius + eyeCyl/2 = 0.1445,顶端到 **0.3845** 而不是+ 0.362 —— 光这一项就把间隙砍到 0.0085;再算上 5° 倾角把内端压低,**真实+ 间隙只有 0.0062**。也就是说那根眉几乎贴在眼睛上,正是"厚眼皮"的读法。++ ⚠️ 复算这类间隙时注意:眉盘是 dotMesh 缩放出来的**椭圆**(半轴 browW ×+ browH)再整体旋转,最低点是 `cy − √((w·sinθ)² + (h·cosθ)²)`,**不是**+ `cy − h`。同一个公式下 BAJIE 的眉内端是**压进眼睛 0.0291** 的(那是它故意的,+ 猪的"内端下压"= 凶相),两者不是一回事。++ 一条留下的坑:**耳朵得比身体暗一档**(earShade)。它伸在身体那条深描边之外,+ 同色的话就是一块没有边的浅色糊在亮色 App 上 —— 而这颗球在亮色 App 上+ 全靠那条深边活着。 */+const P_CREAM = {+ key:'cream', label:'奶油球', tag:'奶油身体 + 深色豆豆眼 + 深色描边 + 立耳(无眉)',+ bodyBase:[0.955,0.965,0.985], tint:0.55, albedoMul:0.93, grad:0.42,+ amb:0.66, diff:0.38, gloss:0.45, bodyScale:0.845,+ eyeStyle:'spec', eyeColor:[0.105,0.135,0.205], pupilR:0, specR:0.040,+ eyeR:0.122, eyeH:0.045, eyeX:0.262, eyeY:0.240,+ eyeCut:0, eyeTilt:0, pupilAspect:1,+ faceZ:0.876, mouthR:0.170, mouthTube:0.038, mouthY:-0.230, mouthThick:0.05,+ mouthMode:'deep', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.60,0.66],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.62, blushBlend:'alpha', blushSize:0.48, blushX:0.400, blushY:-0.142,+ mouthStyle:'arc',+ earH:0.380, earW:0.170, earThick:0.050, earX:0.315, earY:0.645, earZ:0.290, earTilt:24, earBend:12, earShade:0.62,+ browW:0, browH:0, browY:0, browTilt:0, browShade:0,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'dark', rimGain:1.00, rimPow:5.6,+};++/* --- 小恶魔:**部分**可画 ---+ 这颗球是 BallLook.IMP 落地的参数,但这一页**画不出它的角、獠牙和尾巴** ——+ 三种几何都没移植。所以这个预设只在"脸平面"上可信:眼睛(含切平的顶和+ 竖缝瞳孔)、嘴弧、腮红、边光、身体颜色,这些是真机上那套。+ 轮廓不是 —— 别看着这里的圆球去调 bodyScale 或 hornY,那些数真机上是另一颗球。+ (check_look_sync.py 会把这件事印出来,不让它看起来"已经核对过"。)++ 它存在的理由很具体:三个形象里小恶魔是唯一**完全没有实验室覆盖**的,而上一+ 个 bug(眉毛和脸同色、看不见)正因为它不在这一页才只能在真机上被发现。++ eyeCut 0.48 / eyeTilt 12 —— 那道平边就是眉,两个数怎么来的写在 BallLook.IMP+ 的注释里,这里只记这一页看得见的那部分(下面这几个数都在 Node 里跑过一遍,+ 见 tools/eye_mesh_check.py):+ * 0.48 把平边放在半高 0.1755 的 +0.0913 处,离瞳孔顶(0.8×0.082 = 0.0656)+ 还剩 0.0257 的白。眼睛少了 24% 的高度,从 0.296×0.351 变成 0.296×0.267,+ 平边自己有 0.267 宽(眼宽的 90%)。+ **天花板不是 0.85**:瞳孔是画在眼球前面的一片,切口只切眼球自己,切到瞳孔+ 顶上就有一颗黑豆从平盖上立出来。这个形象的那条线是+ 1 − 0.8×pupilR/(eyeR+eyeH/2),算出来 0.626,再往下就没有白眼仁了。+ * 12° 让平边沿自身长度降下 0.267×sin12° ≈ 0.056,约眼高的五分之一 ——+ 这一步才是"困眼"和"斜眼"的分界,也是它比原来那根眉条的 24° 小的原因:+ 这里转的是整颗眼睛,连瞳孔一起。 */+const P_IMP = {+ key:'imp', label:'小恶魔', tag:'墨色 + 獠牙 + 切平的眼睛 · 本页画不出角/獠牙/尾巴',+ bodyBase:[0.052,0.042,0.072], tint:0.14, albedoMul:0.92, grad:0.40,+ amb:0.62, diff:0.60, gloss:0.26, bodyScale:0.868,+ eyeStyle:'pupil', eyeColor:[1,1,1], pupilR:0.082, specR:0.022,+ eyeR:0.148, eyeH:0.055, eyeX:0.255, eyeY:0.190,+ eyeCut:0.48, eyeTilt:12, pupilAspect:0.42,+ faceZ:0.876, mouthR:0.330, mouthTube:0.026, mouthY:-0.215, mouthThick:0.03,+ mouthMode:'deep', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.30,0.52],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.78, blushBlend:'add', blushSize:0.32, blushX:0.430, blushY:-0.075,+ mouthStyle:'arc',+ /* 没开的那几条**照抄 Kotlin 的默认值**,不是从邻居预设复制过来凑格式。+ 两边都是关着的、看起来无所谓,但核对脚本按"继承来的值"比:earH=0 时+ earW 是 0.155 还是 0 都不影响这一页,可它一旦和 Kotlin 的默认值不同,+ 下一颗球加耳朵时就会拿一串没人认领的数去开工。第一版这里正是复制的,+ 脚本一次报出 9 处 —— 那 9 处就是这条规则存在的原因。 */+ earH:0, earW:0, earThick:0, earX:0, earY:0, earZ:0, earTilt:0, earBend:0, earShade:1,+ browW:0, browH:0, browY:0, browTilt:0, browShade:0,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.920,0.950,1.000], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:0.85, rimPow:3.6,+};++/* --- 猫耳球:同一颗果冻球,加一对耳朵和 ω 嘴 ---+ 耳朵是这张表里唯一"长出来"的几何,也是可爱的最大单笔加成:+ 圆的东西加一对尖角,读出来的立刻是动物而不是物件。 */+const P_KITTY = {+ key:'kitty', label:'猫耳球', tag:'果冻球 + 一对耳朵 + ω 嘴',+ bodyBase:[0.055,0.072,0.130], tint:0.62, albedoMul:0.88, grad:0.60,+ amb:0.66, diff:0.55, gloss:0.70, bodyScale:0.885,+ eyeStyle:'pupil', eyeColor:[1,1,1], pupilR:0.070, specR:0.024,+ eyeR:0.135, eyeH:0.050, eyeX:0.262, eyeY:0.175,+ eyeCut:0, eyeTilt:0, pupilAspect:1,+ faceZ:0.876, mouthR:0.120, mouthTube:0.038, mouthY:-0.232, mouthThick:0.10,+ mouthMode:'mood', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.85, blushBlend:'add',+ blushSize:0.44, blushX:0.400, blushY:-0.145,+ mouthStyle:'cat',+ earH:0.340, earW:0.165, earThick:0.056, earX:0.345, earY:0.690, earZ:0.300, earTilt:15, earBend:0, earShade:0.84,+ browW:0, browH:0.032, browY:0.150, browTilt:0, browShade:0,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.15, rimPow:2.6,+};++/* --- 麻薯球:哑光白团子。可爱来自"软",不来自"亮" ---+ 高光降到 0.08 —— 有高光就变成塑料球,没有才像糯米。眼睛是这批里最大的,+ 瞳孔吃掉眼宽 68%,两颗眼几乎贴在一起,嘴巴缩成一个小弧。 */+const P_MOCHI = {+ key:'mochi', label:'麻薯球', tag:'哑光糯米团 · 最大的一双眼睛 · 一块大软腮红',+ bodyBase:[0.985,0.968,0.940], tint:0.38, albedoMul:0.90, grad:0.32,+ amb:0.76, diff:0.26, gloss:0.08, bodyScale:0.885,+ eyeStyle:'spec', eyeColor:[0.125,0.140,0.195], pupilR:0.000, specR:0.052,+ eyeR:0.148, eyeH:0.040, eyeX:0.246, eyeY:0.186,+ eyeCut:0, eyeTilt:0, pupilAspect:1,+ faceZ:0.876, mouthR:0.065, mouthTube:0.026, mouthY:-0.205, mouthThick:0.00,+ mouthMode:'deep', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.60,0.66],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.55, blushBlend:'alpha',+ blushSize:0.62, blushX:0.360, blushY:-0.150,+ mouthStyle:'arc',+ earH:0, earX:0.360, earY:0.700, earZ:0.360, earW:0.155, earThick:0.056, earTilt:24, earBend:0, earShade:0.86,+ browW:0, browH:0.032, browY:0.150, browTilt:0, browShade:0,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'dark', rimGain:0.55, rimPow:4.5,+};++/* --- 贴纸球:品牌吉祥物的做法 ---+ 没有渐变、没有高光、没有瞳孔高光点,只有一块饱和的平色加一条脆的深描边。+ 描边不是菲涅尔光晕,是"先画一圈放大的实心暗球、再把身体压上去"——+ 光晕是软的,贴纸要的是硬的。 */+const P_STICKER = {+ key:'sticker', label:'贴纸球', tag:'纯平色 + 一条脆的硬描边 · 吉祥物路线',+ bodyBase:[0.175,0.140,0.320], tint:0.78, albedoMul:1.00, grad:0.18,+ amb:0.94, diff:0.10, gloss:0.00, bodyScale:0.845,+ eyeStyle:'pupil', eyeColor:[1,1,1], pupilR:0.062, specR:0.024,+ eyeR:0.132, eyeH:0.046, eyeX:0.258, eyeY:0.176,+ eyeCut:0, eyeTilt:0, pupilAspect:1,+ faceZ:0.876, mouthR:0.150, mouthTube:0.038, mouthY:-0.232, mouthThick:0.08,+ mouthMode:'deep', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.52, blushBlend:'alpha',+ blushSize:0.46, blushX:0.395, blushY:-0.150,+ mouthStyle:'arc',+ earH:0, earX:0.360, earY:0.700, earZ:0.360, earW:0.155, earThick:0.056, earTilt:24, earBend:0, earShade:0.86,+ browW:0, browH:0.032, browY:0.150, browTilt:0, browShade:0,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.055, outlineColor:[0.070,0.070,0.095],+ rimMode:'dark', rimGain:0.00, rimPow:6.0,+};++/* --- 角落球:可爱来自"不闹" ---+ 它大部分时间就蹲在屏幕角上,所以给它一个最不起眼的脸:哑光、小一圈的眼、+ 淡到几乎没有的描边、几乎没有光。参考的是角落生物那一类——+ 不靠大和亮,靠"缩着"。 */+const P_SUMIKKO = {+ key:'sumikko', label:'角落球', tag:'哑光 · 小一圈的眼 · 几乎不发光',+ bodyBase:[0.135,0.130,0.155], tint:0.30, albedoMul:0.86, grad:0.50,+ amb:0.68, diff:0.45, gloss:0.10, bodyScale:0.885,+ eyeStyle:'pupil', eyeColor:[0.860,0.850,0.815], pupilR:0.048, specR:0.017,+ eyeR:0.098, eyeH:0.048, eyeX:0.196, eyeY:0.168,+ eyeCut:0, eyeTilt:0, pupilAspect:1,+ faceZ:0.876, mouthR:0.062, mouthTube:0.026, mouthY:-0.198, mouthThick:0.00,+ mouthMode:'mood', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.45, blushBlend:'add',+ blushSize:0.42, blushX:0.330, blushY:-0.132,+ mouthStyle:'arc',+ earH:0, earX:0.360, earY:0.700, earZ:0.360, earW:0.155, earThick:0.056, earTilt:24, earBend:0, earShade:0.86,+ browW:0, browH:0.032, browY:0.150, browTilt:0, browShade:0,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'dark', rimGain:0.45, rimPow:6.0,+};++/* 顺序:先"现在"对照,再几个没落地的草稿,最后两个是 BallLook.kt 里真有的。+ cream 和 imp 是**有 Kotlin 对应**的两个 —— 核对脚本只查它们,其余五个是+ 这一页自己的草稿,改了不会影响真机。 */+const PRESETS = [P_NOW, P_JELLY, P_KITTY, P_MOCHI, P_STICKER, P_SUMIKKO, P_CREAM, P_IMP];+const clone = o => JSON.parse(JSON.stringify(o));++/* =========================================================================+ 6. Shader+ ========================================================================= */+const VS_SOLID = `+ uniform mat4 uMVP;+ attribute vec3 aPos;+ void main(){ gl_Position = uMVP * vec4(aPos,1.0); }+`;+const FS_SOLID = `+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ void main(){ gl_FragColor = vec4(uColor, uAlpha); }+`;++/* 身体:顶光渐变 + 主光 + 镜面高光。这是"果冻感"的全部来源。 */+const VS_BODY = `+ uniform mat4 uMVP; uniform mat4 uMV;+ attribute vec3 aPos; attribute vec3 aNormal;+ varying vec3 vN; varying vec3 vP;+ void main(){+ vN = normalize(mat3(uMV) * aNormal);+ vec4 p = uMV * vec4(aPos,1.0);+ vP = p.xyz;+ gl_Position = uMVP * vec4(aPos,1.0);+ }+`;+const FS_BODY = `+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ uniform float uGrad;+ uniform float uAmb;+ uniform float uDiff;+ uniform float uGloss;+ varying vec3 vN; varying vec3 vP;+ void main(){+ vec3 n = normalize(vN);+ vec3 V = normalize(-vP);+ float vert = n.y * 0.5 + 0.5;+ vec3 base = uColor * mix(1.0, mix(0.70, 1.14, vert), uGrad);+ // 主光偏正上方。高光点落在额头 —— 压在眼睛上等于把眼睛的对比度+ // 吃掉一半,而且 0.95 的强度会在 80dp 的球上糊出一块 30px 的白斑。+ vec3 L = normalize(vec3(-0.48, 0.85, 0.20));+ float diff = max(0.0, dot(n, L));+ vec3 lit = base * (uAmb + uDiff * diff);+ vec3 H = normalize(L + V);+ float spec = pow(max(0.0, dot(n, H)), 90.0) * uGloss;+ float edge = pow(1.0 - max(0.0, dot(n, V)), 3.0) * uGloss * 0.26;+ gl_FragColor = vec4(lit + vec3(spec) + uColor * edge, uAlpha);+ }+`;++/* 描边 / 光晕。uPow 小 → 柔和发光;大 → 紧的深色轮廓。+ 这一段必须与 EmotionBallRenderer.RIM_FS **逐字相同** ——+ 上一版这里少写了一条扫描光带,结果真机上出现了一条实验室永远看不见的+ 滚动色带,用户先发现。实验室的价值全在于它和真机一样。 */+const VS_RIM = `+ uniform mat4 uMVP; uniform mat4 uMV;+ attribute vec3 aPos; attribute vec3 aNormal;+ varying vec3 vN; varying vec3 vP; varying float vY;+ void main(){+ vN = normalize(mat3(uMV) * aNormal);+ vec4 p = uMV * vec4(aPos,1.0);+ vP = p.xyz; vY = aPos.y;+ gl_Position = uMVP * vec4(aPos,1.0);+ }+`;+const FS_RIM = `+ precision mediump float;+ uniform vec3 uColor;+ uniform float uIntensity;+ uniform float uScan;+ uniform float uScanY;+ uniform float uPow;+ uniform float uCap;+ varying vec3 vN; varying vec3 vP; varying float vY;+ void main(){+ float f = pow(1.0 - abs(dot(normalize(vN), normalize(-vP))), uPow);+ float d = vY - uScanY;+ float slope = d > 0.0 ? 7.5 : 4.2;+ float band = pow(max(0.0, 1.0 - abs(d) * slope), 3.0);+ gl_FragColor = vec4(uColor, min(f * uIntensity + band * uScan, uCap));+ }+`;++const VS_PTS = `+ uniform mat4 uMVP; uniform mat4 uMV;+ uniform float uSize; uniform float uPointFactor;+ attribute vec3 aPos; attribute vec3 aColor;+ varying vec3 vColor;+ void main(){+ vec4 mvpos = uMV * vec4(aPos,1.0);+ gl_Position = uMVP * vec4(aPos,1.0);+ gl_PointSize = uSize * uPointFactor / -mvpos.z;+ vColor = aColor;+ }+`;+const FS_PTS = `+ precision mediump float;+ uniform float uAlpha;+ varying vec3 vColor;+ void main(){+ vec2 c = gl_PointCoord - 0.5;+ float d = length(c);+ float a = smoothstep(0.5, 0.0, d);+ gl_FragColor = vec4(vColor, a * uAlpha);+ }+`;++/* =========================================================================+ 7. EmotionState 移植+ ========================================================================= */+const ACTION_SECONDS = {NONE:0, HOP:0.52, NOD:0.42, SHAKE:0.5, POP:0.46, RECOIL:0.55,+ STRETCH:1.6, LOOK_AROUND:1.9, WIGGLE:1.1, BOUNCE:0.9};++class State{+ constructor(){+ this.nowColor = FACE_OLD.CALM.color.slice();+ this.eye=1; this.brow=0; this.smile=0.1; this.tilt=0; this.skew=0; this.jitter=0;+ this.open=0; this.eyeArc=0; this.blush=0; this.spread=1; this.lid=1;+ // 鼻涕泡:amount 是随表情淡入淡出,scale 是它自己的一呼一吸+ this.bubble=0; this.bubbleScale=1; this.bubblePhase=0;+ this.think=0; this.busy=0;+ this.rotX=0; this.rotY=0; this.rotZ=0; this.breathe=1;+ this.breatheDepth=0.022; this.scaleX=1; this.scaleY=1; this.hopY=0;+ this.eyeOffX=0; this.eyeOffY=0; this.eyeLRotZ=0; this.eyeRRotZ=0; this.eyeYScale=1;+ this.mouthY=-0.34; this.mouthScaleY=0.16; this.mouthRotZ=Math.PI;+ this.blinkNext=2.4; this.blinkLeft=0;+ // 扫描光带:一条横线从下往上扫,一轮约 5.3 秒。think/listen 会让它变明显。+ this.scanAngle=0; this.scanY=-2; this.scan=0; this.listen=0;+ this.action='NONE'; this.actionT=0; this.actionDur=0;+ }+ trigger(a){ if(!a||a==='NONE') return; this.action=a; this.actionT=0; this.actionDur=ACTION_SECONDS[a]; }++ tick(dt,t,mood,p){+ const want=p.faces[mood], k=Math.min(1,dt*7);+ this.eye+=(want.eye-this.eye)*k;+ this.brow+=(want.brow-this.brow)*k;+ this.smile+=(want.smile-this.smile)*k;+ this.tilt+=(want.tilt-this.tilt)*k;+ this.skew+=(want.skew-this.skew)*k;+ this.jitter+=(want.jitter-this.jitter)*Math.min(1,dt*2.4);+ this.eyeArc+=(want.eyeArc-this.eyeArc)*k;+ this.blush+=(want.blush-this.blush)*k;+ this.spread+=(want.spread-this.spread)*Math.min(1,dt*3.5);+ for(let i=0;i<3;i++) this.nowColor[i]+=(want.color[i]-this.nowColor[i])*k;++ this.think+=(((mood==='THINKING'||mood==='SEARCHING')?1:0)-this.think)*Math.min(1,dt*2.2);+ const th=this.think;+ this.busy=th;++ // listen 在真机上是语音管线给的;这里用 LISTENING 表情近似+ this.listen += (((mood==='LISTENING')?1:0)-this.listen)*Math.min(1,dt*3.4);+ this.scanAngle += dt*(0.19 + th*0.55);+ this.scanY = -1.35 + (this.scanAngle - Math.floor(this.scanAngle))*2.7;+ this.scan = 0.08 + th*0.5 + this.listen*0.42 + this.open*0.12;++ const gaze = th*(Math.sin(t*0.9)+Math.sin(t*1.31)*0.5);+ const gazeY= th*(Math.sin(t*0.73)*0.6+Math.sin(t*1.07)*0.35);+ const jx=(Math.sin(t*23.7)+Math.sin(t*37.1)*0.6)*this.jitter;+ const jy=(Math.sin(t*29.3)+Math.sin(t*41.7)*0.6)*this.jitter;++ const talk = (mood==='SPEAKING') ? 0.34+Math.sin(t*17)*0.16+Math.sin(t*29)*0.09 : want.gape;+ this.open+=(Math.max(0,talk)-this.open)*Math.min(1,dt*22);++ // 眨眼是"给睁着的眼睛合上眼皮"。屏幕上那只眼睛已经是 `^ ^` 弧的时候+ // (eyeArc ≥ ARC_EYE_AT)没有眼皮可合,眨一下只是把弧抽动一下 ——+ // 睡着的球每三秒抖一次就是这么来的。+ const eyeShut=this.eyeArc>=ARC_EYE_AT;+ this.blinkNext-=dt;+ if(this.blinkNext<=0){+ if(eyeShut){+ // 顺带取消正在进行的那一次,并稍后重试:醒来后第一次眨眼+ // 不用等满一整轮+ this.blinkLeft=0; this.blinkNext=0.35;+ } else {+ this.blinkLeft=0.12;+ const base=(this.listen>0.5?1.8:3.2)+Math.random()*2.6;+ this.blinkNext=(base*(1+th*0.9))/(1+this.jitter*1.6);+ }+ }+ let lid=1;+ if(this.blinkLeft>0){ this.blinkLeft-=dt; lid=Math.abs(this.blinkLeft/0.06-1); }+ this.lid=lid;++ let sqx=0,sqy=0,hop=0,actRX=0,actRY=0;+ if(this.action!=='NONE'){+ this.actionT+=dt;+ const pp=Math.min(1,this.actionT/this.actionDur);+ const arch=Math.sin(pp*Math.PI);+ switch(this.action){+ case 'HOP':{ hop=arch*0.22;+ const c=Math.max(0,1-pp/0.2)+Math.max(0,(pp-0.8)/0.2);+ sqx=c*0.13; sqy=c*-0.17; break; }+ case 'NOD': actRX=Math.sin(pp*2*Math.PI)*0.26; break;+ case 'SHAKE': actRY=Math.sin(pp*3*Math.PI)*0.20; break;+ case 'POP': sqx=arch*0.15; sqy=arch*0.15; hop=arch*0.06; break;+ case 'RECOIL': sqx=arch*0.17; sqy=arch*-0.20; actRX=-arch*0.14; break;+ case 'STRETCH':{ const e=Math.sin(pp*Math.PI);+ sqx=-e*0.12; sqy=e*0.20; actRX=-e*0.08; break; }+ case 'LOOK_AROUND': actRY=Math.sin(pp*2.5*Math.PI)*0.30; actRX=Math.sin(pp*1.25*Math.PI)*0.05; break;+ case 'WIGGLE': actRY=Math.sin(pp*3*Math.PI)*0.14; break;+ case 'BOUNCE':{ hop=Math.abs(Math.sin(pp*2*Math.PI))*0.09;+ const c=Math.max(0,1-pp/0.15)+Math.max(0,(pp-0.85)/0.15);+ sqx=c*0.07; sqy=c*-0.09; break; }+ }+ if(pp>=1) this.action='NONE';+ }++ this.breatheDepth+=(want.breath-this.breatheDepth)*Math.min(1,dt*3);+ this.breathe=1+Math.sin(t*want.pulse)*this.breatheDepth+this.open*0.03;+ this.scaleX=this.breathe*(1+sqx);+ this.scaleY=this.breathe*(1+sqy);+ this.hopY=hop;+ this.rotY=Math.sin(t*0.22)*0.12+jx*0.012+gaze*0.09+actRY;+ this.rotX=Math.sin(t*0.17)*0.06+jy*0.009-gazeY*0.05+actRX;+ this.rotZ=this.tilt+th*Math.sin(t*0.51)*0.09;++ const browR=this.brow+this.skew, browL=this.brow;+ this.eyeOffX=jx*0.014+gaze*0.075;+ this.eyeOffY=-browL*0.06+gazeY*0.05;+ // 眼睛的滚动,两项同轴:+ // * 造型自己的静止倾角,左右分别取反 —— 这是"凶"的形状,写在同一颗球上+ // 而不是某一刻的表情。见 BallLook.eyeTilt(切平的眼睛歪多少度就是+ // "压低的一根眉",而水平的切边只是一双困眼睛)。+ // * 表情的 brow,满值是 ±0.4rad,符号约定跟原来那根眉条一样,所以+ // "生来就皱着"的造型还能被表情拉得更凶,而不是被换掉。+ this.eyeLRotZ=-p.eyeTilt/DEG-browL*0.40;+ this.eyeRRotZ=p.eyeTilt/DEG+browR*0.40;++ const sm=this.smile+p.smileBoost;+ this.eyeYScale=Math.max(0.06,this.eye*this.lid-sm*0.34);++ // 鼻涕泡。相位**故意不挂在呼吸上**:SLEEPING 的 pulse 是 0.45,+ // 挂上去一轮 14 秒 —— 一个 14 秒不动的泡等于没有在动。+ // 速率仍然跟着 pulse 走,"睡得越沉、泡越慢"这层关系还在。+ this.bubble+=((want.bubble||0)-this.bubble)*Math.min(1,dt*2.2);+ this.bubblePhase+=dt*(0.9+want.pulse*1.6);+ this.bubbleScale=0.30+0.70*(0.5+0.5*Math.sin(this.bubblePhase));+ this.mouthScaleY=0.16+this.open*1.5+Math.abs(sm)*0.5+p.mouthThick;+ this.mouthY=p.mouthY+sm*0.06;+ this.mouthRotZ=Math.PI+sm*0.9+this.skew*2.2;+ }+}++/* =========================================================================+ 8. Ball+ ========================================================================= */+const CAM_Z=4.9, RIM=0.9;+const EYE_LID_PIVOT=-0.125;+/* 眉毛的倾角可以随表情摆动多少(弧度),与 EmotionBallRenderer.BROW_MOOD_TILT 同值。+ +1 的表情值 = 内端往下 17.2°。 */+const BROW_MOOD_TILT=0.30;+/* 面片元素的 z 厚度(压扁的单位球 = 一片)。同 DISC_THICKNESS。 */+const DISC_THICKNESS=0.08;+/* 度 → 弧度。BallLook 里的角度(earTilt / browTilt / eyeTilt)是**度**,表情+ 给的是弧度,这一页两处相减,用一个常量比散落 57.29578 安全。 */+const DEG=57.29578;+const EYE_ARC_SPAN=0.8*Math.PI, EYE_ARC_ROT_Z=Math.PI/2-EYE_ARC_SPAN/2, EYE_ARC_DROP=0.06;+/* 超过这个值,屏幕上的眼睛就是那条 `^ ^` 弧,圆眼不是主角了。+ 两处共用:还要不要画瞳孔、还要不要眨眼。 */+const ARC_EYE_AT=0.5;++class Ball{+ constructor(canvas,params){+ this.cv=canvas;+ this.gl=canvas.getContext('webgl',{alpha:true,antialias:true,premultipliedAlpha:true,depth:true});+ if(!this.gl) throw new Error('这台机器没有 WebGL');+ const gl=this.gl;+ // 默认就停在 SLEEPING:这一页现在的主题是睡着那颗球+ this.mood='SLEEPING';+ this.st=new State();+ this.autoFidget=true;++ this.pSolid=new Prog(gl,VS_SOLID,FS_SOLID);+/* 鼻涕泡。一个透明的水滴,不是一颗白球:+ * 背面 discard —— 只留朝向相机的那半,否则前后面叠在一起,边缘会变成两倍厚;+ * edge 是菲涅尔,在轮廓上收紧成一圈(水滴的边是折射,浅色球上用暗色边);+ * spec 是一个小高光点,落在左上,和球的主光同一个方向;+ * uInner 是内部那层很薄的雾 —— 0 就完全透明。+ uColor 按 look 给:浅色球用深色边,深色球用亮边。 */+const VS_BUBBLE = `+ uniform mat4 uMVP; uniform mat4 uMV;+ attribute vec3 aPos; attribute vec3 aNormal;+ varying vec3 vN; varying vec3 vP;+ void main(){+ vN = normalize(mat3(uMV) * aNormal);+ vec4 mvpos = uMV * vec4(aPos,1.0);+ vP = mvpos.xyz;+ gl_Position = uMVP * vec4(aPos,1.0);+ }+`;+const FS_BUBBLE = `+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ uniform float uPow;+ uniform float uShine;+ uniform float uInner;+ varying vec3 vN; varying vec3 vP;+ void main(){+ vec3 n = normalize(vN);+ vec3 V = normalize(-vP);+ float facing = dot(n, V);+ if (facing <= 0.0) discard;+ float edge = pow(1.0 - facing, uPow);+ vec3 L = normalize(vec3(-0.45, 0.80, 0.60));+ vec3 H = normalize(L + V);+ float spec = pow(max(0.0, dot(n, H)), uShine);+ float a = clamp(edge + uInner + spec, 0.0, 1.0);+ vec3 c = mix(uColor, vec3(1.0), clamp(spec * 2.5, 0.0, 1.0));+ gl_FragColor = vec4(c, a * uAlpha);+ }+`;++ this.pBody =new Prog(gl,VS_BODY ,FS_BODY);+ this.pRim =new Prog(gl,VS_RIM ,FS_RIM);+ this.pBubble=new Prog(gl,VS_BUBBLE,FS_BUBBLE);+ this.pPts =new Prog(gl,VS_PTS ,FS_PTS);++ this.mBody=new GMesh(gl,icoSphere(4));+ this.mRim =new GMesh(gl,uvSphere(40,28));+ this.mDot =new GMesh(gl,icoSphere(2));+ this.mArc =new GMesh(gl,torus(0.1,0.03,6,20,EYE_ARC_SPAN));+ this.mBlush=new GPoints(gl,2);++ this.proj=M4.id(); this.view=M4.lookAt(0,0,CAM_Z,0,0,0,0,1,0);+ this.pointFactor=400;+ this.setParams(params);++ this.fidNext=9+Math.random()*8;+ this.resize();+ }+ setParams(params){+ // 网格的尺寸跟着参数走,所以耳朵和眼角形状也必须进这个判据:只改一条+ // earH / eyeCut 滑杆而不重建,屏幕上还是旧的几何,而滑杆数字已经变了+ // —— 用户会以为"这个参数没用"。眼睛的尺寸参数是 eyeR / eyeH / eyeCut 三个。+ const changedSame = this.p && this.p.eyeR===params.eyeR &&+ this.p.eyeH===params.eyeH && this.p.eyeCut===params.eyeCut &&+ this.p.mouthR===params.mouthR &&+ this.p.mouthTube===params.mouthTube &&+ this.p.earW===params.earW && this.p.earH===params.earH &&+ this.p.earThick===params.earThick && this.p.earBend===params.earBend;+ this.p=clone(params);+ if(!changedSame || !this.mEye){+ // eyeCut > 0 就换一颗切了顶的眼睛。判据和真机 LookMeshes 一样是 `>0`+ // 而不是 `!==0`:负数切出来的是"往反方向切",没有意义,而 0 是默认值+ // ——预设表里每个形象都得写出这一行,否则 `undefined > 0` 是假、+ // `undefined !== 0` 是真,两边会各自安静地走上不同的分支。+ this.mEye = this.p.eyeCut>0+ ? new GMesh(this.gl, eyeMesh(this.p.eyeR,this.p.eyeH,this.p.eyeCut,4,12))+ : new GMesh(this.gl, capsule(this.p.eyeR,this.p.eyeH,4,12));+ this.mMouth=new GMesh(this.gl,torus(this.p.mouthR,this.p.mouthTube,8,24,Math.PI));+ // 耳朵按预设的真实比例建,比例变了就重建 —— 这样绘制时只做旋转平移,+ // 法线不会被非等比缩放搞坏(法线要的是逆转置,普通缩放是错的)。+ // 参数顺序就是 gl/Ear.build 的;earBend 在 BallLook 里是**度**,这里转弧度。+ // earH = 0 就是不画耳朵 —— 和真机 LookMeshes 一样给 null,而不是建一个+ // 退化成纸片的网格(height=0 的环会让法线变成 0/0)。绘制处另有 earH>0+ // 的判据,两道都留着:滑杆拖到 0 时两边同时安静。+ this.mEar = this.p.earH>0+ ? new GMesh(this.gl,+ earMesh(this.p.earW, this.p.earH, this.p.earThick,+ this.p.earBend*Math.PI/180, 6, 10))+ : null;+ }+ }+ /**+ * 瞳孔 / 高光 / 眉毛所在的 z(相对面平面)。眼球是**胶囊**,在 z 上也有半径+ * ([eyeR]),把瞳孔按面片画在面平面上会被它自己的眼球挡掉。+ *+ * 这条公式是 **eyeR + 0.025**,跟真机的 `EmotionBallRenderer.eyeFrontZ` 一样。+ * 从前这里是一个死数 0.16 —— 那是照果冻球(eyeR 0.135)凑出来的,换到眼睛+ * 更小的奶油球(0.122)上就偏前了 0.013。这种"两边都对、但不一样"的偏移+ * 正是这个页面最怕的:用户照着它调完,装到机器上又是另一个东西。+ */+ get eyeFront(){ return this.p.eyeR + 0.025; }++ resize(){+ const dpr=Math.min(2,window.devicePixelRatio||1);+ const w=Math.max(1,Math.round(this.cv.clientWidth*dpr));+ const h=Math.max(1,Math.round(this.cv.clientHeight*dpr));+ if(this.cv.width!==w||this.cv.height!==h){ this.cv.width=w; this.cv.height=h; }+ const gl=this.gl;+ gl.viewport(0,0,this.cv.width,this.cv.height);+ this.proj=M4.perspective(42,this.cv.width/this.cv.height,0.1,100);+ this.pointFactor=this.cv.height/(2*Math.tan(42*Math.PI/180/2));+ }++ frame(dt,t){+ const gl=this.gl, p=this.p;+ this.st.tick(dt,t,this.mood,p);++ if(this.autoFidget && this.st.action==='NONE'){+ this.fidNext-=dt;+ if(this.fidNext<=0){+ const pool=['STRETCH','LOOK_AROUND','WIGGLE','BOUNCE','HOP'];+ this.st.trigger(pool[Math.floor(Math.random()*pool.length)]);+ this.fidNext=14+Math.random()*22;+ }+ }++ gl.clearColor(0,0,0,0);+ gl.clear(gl.COLOR_BUFFER_BIT|gl.DEPTH_BUFFER_BIT);+ gl.enable(gl.DEPTH_TEST);+ gl.enable(gl.BLEND);++ let root=M4.id();+ root=M4.translate(root,0,this.st.hopY,0);+ root=M4.rotate(root,this.st.rotX*57.29578,1,0,0);+ root=M4.rotate(root,this.st.rotY*57.29578,0,1,0);+ root=M4.rotate(root,this.st.rotZ*57.29578,0,0,1);+ root=M4.scale(root,this.st.scaleX,this.st.scaleY,this.st.scaleX);+ this.root=root;+ this.face=M4.translate(root,0,0,p.faceZ);++ const c=this.st.nowColor;++ /* ---- 1. 身体 ---- */+ const albedo=[0,0,0];+ for(let i=0;i<3;i++) albedo[i]=(p.bodyBase[i]+(c[i]-p.bodyBase[i])*p.tint)*p.albedoMul;+ this.albedo=albedo;++ // 硬描边:先画一圈放大的实心暗球,再让身体压上去,中间留一条边。+ // 关键在 depthMask(false) —— 描边球比身体大,整个包在身体外面,+ // 如果它写了深度,后面画的身体深度更大,会被它自己那圈描边挡掉,+ // 结果是一颗纯黑的球。不写深度,它就成了身体后面的一层衬底。+ if(p.outline>0.001){+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.depthMask(false);+ const os=p.bodyScale+p.outline;+ const om=M4.scale(root,os,os,os);+ const omv=M4.mul(this.view,om), omvp=M4.mul(this.proj,omv);+ this.pSolid.use();+ gl.uniform1f(this.pSolid.u('uAlpha'),1);+ gl.uniform3fv(this.pSolid.u('uColor'),p.outlineColor);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,omvp);+ this.mBody.bind(this.pSolid.a('aPos'));+ this.mBody.draw();+ // 描边层里的耳朵。注意这是**硬描边这一套(p.outline)**独有的,真机的+ // drawEars 里没有对应的一笔 —— 真机给耳朵的边靠 earShade(比身体暗一档)+ // 换来的,不靠再画一层放大的壳。目前没有任何预设同时开 outline 和耳朵,+ // 所以这一步在屏幕上永远走不到;打开之前先想好在 Kotlin 那边怎么补。+ if(p.earH>0) this.drawEars(1+p.outline/p.earW, p.outlineColor);+ }++ let m=M4.scale(root,p.bodyScale,p.bodyScale,p.bodyScale);+ let mv=M4.mul(this.view,m), mvp=M4.mul(this.proj,mv);+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.depthMask(true);+ this.pBody.use();+ gl.uniformMatrix4fv(this.pBody.u('uMVP'),false,mvp);+ gl.uniformMatrix4fv(this.pBody.u('uMV'),false,mv);+ gl.uniform3fv(this.pBody.u('uColor'),albedo);+ gl.uniform1f(this.pBody.u('uGrad'),p.grad);+ gl.uniform1f(this.pBody.u('uAmb'),p.amb);+ gl.uniform1f(this.pBody.u('uDiff'),p.diff);+ gl.uniform1f(this.pBody.u('uGloss'),p.gloss);+ gl.uniform1f(this.pBody.u('uAlpha'),1);+ this.mBody.bind(this.pBody.a('aPos'),this.pBody.a('aNormal'));+ this.mBody.draw();++ if(p.earH>0) this.drawEars(1, null);++ /* ---- 2. 五官 ---- */+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.depthMask(true);+ this.pSolid.use();+ const aPos=this.pSolid.a('aPos');+ // uAlpha 必须显式给 —— 片元是 vec4(uColor, uAlpha),漏设就是 0,+ // 混合后贡献为零,整条五官管线会静默地什么都不画。+ gl.uniform1f(this.pSolid.u('uAlpha'),1);++ const round=this.st.eyeYScale*(1-this.st.eyeArc);+ const arc=this.st.eyeArc;+ const eX=p.eyeX+this.st.eyeOffX, eY=p.eyeY+this.st.eyeOffY;+ const pivotScale=round;++ if(round>0.01){+ for(const side of [-1,1]){+ const rz = side<0 ? this.st.eyeLRotZ : this.st.eyeRRotZ;+ this.faceElem(side*eX, eY, rz, round, 1, EYE_LID_PIVOT);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),p.eyeColor);+ this.mEye.bind(aPos); this.mEye.draw();+ }+ }++ // 瞳孔 + 高光:跟着眼睛一起眨+ // arc < 0.5:`^ ^` 眯眼时眼球已经塌成一条线,再挂个瞳孔就是脸上一个孤点。+ if(p.eyeStyle!=='plain' && round>0.05 && arc<ARC_EYE_AT){+ const pupilCol=[c[0]*0.20+0.045, c[1]*0.20+0.050, c[2]*0.20+0.075];+ for(const side of [-1,1]){+ const ex=side*eX, ey=eY;+ const rz = side<0 ? this.st.eyeLRotZ : this.st.eyeRRotZ;+ if(p.eyeStyle==='pupil'){+ const pr=p.pupilR;+ // 三个坑一起来:pivotY 必须传 0(sy 是自己的半径,套眼睑铰链+ // 补偿会把瞳孔推到脸颊上);z 要压扁成片;z 还要抬到眼球前面+ // —— 胶囊眼球在 z 上有半径,不抬就整颗瞳孔沉在眼球背面。+ // sx 上乘 pupilAspect:竖缝是最便宜的一次"换物种",一个乘法的事。+ // 漏掉它,小恶魔在实验室里就是一双圆瞳孔 —— 而真机上不是。+ this.faceElem(ex-pr*0.12, ey-pr*0.20, rz, pr*round, pr*p.pupilAspect,+ 0, pr*0.06, this.eyeFront);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),pupilCol);+ this.mDot.bind(aPos); this.mDot.draw();+ // 瞳孔里的白点。贴纸风不要它:有高光就有立体,扁平是它的全部意思。+ const sr=p.specR;+ if(sr>0.001){+ this.faceElem(ex-pr*0.34, ey+pr*0.26, rz, sr*round, sr, 0, sr*0.06, this.eyeFront+0.02);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),[1,1,1]);+ this.mDot.bind(aPos); this.mDot.draw();+ }+ } else if(p.specR>0.001) {+ // 深色豆豆眼 + 一点白高光,落在眼睛的左上+ const sr=p.specR;+ this.faceElem(ex-p.eyeR*0.36, ey+p.eyeR*0.40, rz, sr*round, sr, 0, sr*0.06, this.eyeFront);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),[1,1,1]);+ this.mDot.bind(aPos); this.mDot.draw();+ }+ }+ }++ // 弧形眼 ^ ^+ if(arc>0.01){+ const h=arc*this.st.lid;+ for(const side of [-1,1]){+ const rz = side<0 ? this.st.eyeLRotZ : this.st.eyeRRotZ;+ this.faceElem(side*eX, eY-EYE_ARC_DROP, EYE_ARC_ROT_Z+rz, h, arc);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),p.eyeColor);+ this.mArc.bind(aPos); this.mArc.draw();+ }+ }++ // 嘴。cat 就是同一条半环画两次、各缩到一半宽、并排拼起来 —— 两条弧在+ // 中间相接,读出来就是 ω。比再造一个网格省事,也省得为它单独配参数。+ const mouthCol = p.mouthMode==='deep' ? [c[0]*0.40,c[1]*0.40,c[2]*0.40] : c;+ gl.uniform3fv(this.pSolid.u('uColor'),mouthCol);+ if(p.mouthStyle==='cat'){+ const mo=(p.mouthR+p.mouthTube)*0.5; // 半个弧宽,两条正好在 x=0 相接+ for(const side of [-1,1]){+ this.faceElem(side*mo, this.st.mouthY, this.st.mouthRotZ, this.st.mouthScaleY, 0.5);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ this.mMouth.bind(aPos); this.mMouth.draw();+ }+ } else {+ this.faceElem(0, this.st.mouthY, this.st.mouthRotZ, this.st.mouthScaleY);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ this.mMouth.bind(aPos); this.mMouth.draw();+ }++ this.drawBrows(aPos);++ // 鼻涕泡:一个透明的水滴。不是实心圆片 —— 中间是透的,球体自己透出来,+ // 只有轮廓一圈(折射边)和左上一个小高光。用整颗球而不是压扁的片,+ // 因为菲涅尔要的就是"球面朝向",压扁之后轮廓就没了。+ if(this.st.bubble>0.01){+ const br=p.bubbleR*this.st.bubbleScale*this.st.bubble;+ let bm=M4.id();+ bm=M4.translate(bm,p.bubbleX,p.bubbleY,0);+ bm=M4.scale(bm,br,br,br);+ const bmodel=M4.mul(this.face,bm);+ const bmv=M4.mul(this.view,bmodel), bmvp=M4.mul(this.proj,bmv);+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.depthMask(false);+ this.pBubble.use();+ gl.uniformMatrix4fv(this.pBubble.u('uMVP'),false,bmvp);+ gl.uniformMatrix4fv(this.pBubble.u('uMV'),false,bmv);+ gl.uniform3fv(this.pBubble.u('uColor'),p.bubbleColor);+ gl.uniform1f(this.pBubble.u('uAlpha'),this.st.bubble);+ gl.uniform1f(this.pBubble.u('uPow'),p.bubblePow);+ gl.uniform1f(this.pBubble.u('uShine'),p.bubbleShine);+ gl.uniform1f(this.pBubble.u('uInner'),p.bubbleInner);+ this.mBody.bind(this.pBubble.a('aPos'),this.pBubble.a('aNormal'));+ this.mBody.draw();+ gl.depthMask(true);+ }++ /* ---- 3. 腮红 ---- */+ // restBlush —— 静止脸也带一点血色。球一天里绝大多数时间是 CALM,+ // 而"只有开心才有腮红"等于默认那张脸永远没有脸颊。+ const bl=Math.min(1,(this.st.blush+p.restBlush)*p.blushMul);+ if(bl>0.01){+ const light = p.blushBlend==='alpha';+ const d=this.mBlush.data;+ for(let n=0;n<2;n++){+ const j=n*6;+ d[j]=n===0?-p.blushX:p.blushX;+ d[j+1]=p.blushY; d[j+2]=-0.02;+ d[j+3]=p.blushColor[0]; d[j+4]=p.blushColor[1]; d[j+5]=p.blushColor[2];+ }+ this.mBlush.upload();+ mv=M4.mul(this.view,this.face); mvp=M4.mul(this.proj,mv);+ gl.blendFunc(gl.SRC_ALPHA, light?gl.ONE_MINUS_SRC_ALPHA:gl.ONE);+ gl.depthMask(false);+ this.pPts.use();+ gl.uniformMatrix4fv(this.pPts.u('uMVP'),false,mvp);+ gl.uniformMatrix4fv(this.pPts.u('uMV'),false,mv);+ gl.uniform1f(this.pPts.u('uPointFactor'),this.pointFactor);+ gl.uniform1f(this.pPts.u('uSize'),p.blushSize);+ gl.uniform1f(this.pPts.u('uAlpha'),bl*p.blushAlpha);+ this.mBlush.bind(this.pPts.a('aPos'),this.pPts.a('aColor'));+ this.mBlush.draw();+ }++ /* ---- 4. 描边 / 光晕 ---- */+ m=M4.scale(root,RIM,RIM,RIM);+ mv=M4.mul(this.view,m); mvp=M4.mul(this.proj,mv);+ gl.depthMask(false);+ this.pRim.use();+ gl.uniformMatrix4fv(this.pRim.u('uMVP'),false,mvp);+ gl.uniformMatrix4fv(this.pRim.u('uMV'),false,mv);+ gl.uniform1f(this.pRim.u('uPow'),p.rimPow);+ // 扫描是"光"在扫:只有发光描边那一路的 uColor 是亮色。深色描边那一路+ // uColor 是暗部色,扫过去就是在球上滚一条黑带,看着像渲染坏了。+ gl.uniform1f(this.pRim.u('uScanY'),this.st.scanY);+ gl.uniform1f(this.pRim.u('uScan'), p.rimMode==='dark' ? 0 : this.st.scan);+ if(p.rimMode==='dark'){+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.uniform3fv(this.pRim.u('uColor'),[c[0]*0.34,c[1]*0.34,c[2]*0.42]);+ gl.uniform1f(this.pRim.u('uIntensity'),p.rimGain*1.15);+ gl.uniform1f(this.pRim.u('uCap'),0.82);+ } else {+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE);+ const thinkBeat=this.st.think*(0.5+Math.sin(t*1.9)*0.5);+ const rimI=0.85+this.st.open*0.7+thinkBeat*0.55+ +Math.sin(t*p.faces[this.mood].pulse*2)*0.05;+ gl.uniform3fv(this.pRim.u('uColor'),c);+ gl.uniform1f(this.pRim.u('uIntensity'),rimI*p.rimGain);+ gl.uniform1f(this.pRim.u('uCap'),1.0);+ }+ this.mRim.bind(this.pRim.a('aPos'),this.pRim.a('aNormal'));+ this.mRim.draw();++ gl.depthMask(true);+ }++ /**+ * 一对眉毛 —— `EmotionBallRenderer.drawBrows` 的同一套做法。+ *+ * 为什么是"压扁的球"而不是一根条:一颗缩放过的球就是一个**透镜**形状,+ * 两端自然收细,那正是一根眉毛的样子;用长方体出来的是两片方角。+ *+ * [p.browShade] > 0 时它是**用身体色系挖掉眼睛上内角的一块**+ * ——底下幸存的白眼仁变成一个斜的杏仁,那才是"眼神"。+ * 取的是**当前实时的身体色**而不是一个常量,因为身体带着表情的颜色,+ * 写死的近黑色在 CALM 下对、在 TENSE 下就是一块脏。+ *+ * ⚠️ **现在没有任何一个形象用它,这是故意的。** 小恶魔原来走这一路+ * (browShade 2.5),现在改成 [eyeMesh] 把眼睛自己的上沿切平 —— 同样+ * 一道斜的杏仁,但从几何里来。理由就在下面这张表:那是一个**区间**,+ * 而且是一个要同时满足两头的区间,表情一动两头都在动。切几何没有颜色可挑。+ * 这一路留着给"以后某个深色形象想要一根真的眉"。见 BallLook.browShade。+ *+ * ⚠️ **browShade 是在 albedo 上乘,可眉毛底下的脸是打过光的。**+ * 这一笔走 pSolid(平涂),值就是 `albedo × browShade`;而身体走 pBody,+ * 在眉毛那个纬度大约是 `0.82 × albedo`(垂直渐变 ≈1.0,光从头顶来)。+ * 于是 1.0 实际上是"比脸暗",1.05 只比脸亮 1.28 倍 —— 在近黑的球上+ * 是 7/255 的差,等于看不见。**取值的判据不是"比身体亮一点",而是+ * 落在"身体"和"眼睛"的正中间**:既要比身体亮(不然两端糊掉),+ * 又比眼睛暗(不然挖不出楔形)。眼睛是纯白,所以目标是两者的几何中点。+ * 小恶魔那颗球(albedo 亮度 0.105)的中点是 ≈0.32,就是 **2.5 × albedo**。+ * 细节和整张对照表在 BallLook.browShade 与 EmotionBallRenderer.drawBrows。+ *+ * 倾角 = 造型自己的 [p.browTilt](**度**)+ 表情的 st.brow × 0.30rad,+ * 所以一个"天生就皱着眉"的造型还能被表情拉得更凶、或者被拉平。+ * 左右**分别取反**:内端朝下在左右两边是相反的旋转方向,+ * 这一个符号搞错,凶脸就变成愁脸。+ *+ * 它们骑在**当前的眼睛高度**上(不是造型里的静止值),所以眨眼、瞟一眼+ * 都会把它们一起带走,而不是把它们钉在脑门上。+ */+ drawBrows(aPos){+ const gl=this.gl, p=this.p;+ if(!(p.browW>0)) return;+ const col = p.browShade>0+ ? [this.albedo[0]*p.browShade, this.albedo[1]*p.browShade, this.albedo[2]*p.browShade]+ : p.eyeColor;+ gl.uniform3fv(this.pSolid.u('uColor'), col);+ gl.uniform1f(this.pSolid.u('uAlpha'),1);+ const eX=p.eyeX+this.st.eyeOffX, eY=p.eyeY+this.st.eyeOffY;+ // 和 Kotlin 一样先算弧度:browTilt 是度 → /DEG;表情那项本身就是弧度。+ const tilt=p.browTilt/57.29578 + this.st.brow*BROW_MOOD_TILT;+ for(const side of [-1,1]){+ // pivotY = 0:眉毛的 sy 就是自己的半高,眼睑那套铰链补偿会把整根眉推走+ // (和瞳孔同一个坑)。z 抬到眼球前面,否则被自己的眼球吃掉。+ this.faceElem(side*eX, eY+p.browY, tilt*side, p.browH, p.browW,+ 0, p.browH*DISC_THICKNESS, this.eyeFront);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ this.mDot.bind(aPos); this.mDot.draw();+ }+ }++ /**+ * 一对耳朵。挂在 [root] 上,所以身体压扁、跳起来、歪头它都跟着 ——+ * 长在头上的东西不该有自己的动作。+ *+ * [scaleMul] !== 1 时用纯色画(硬描边那一层):描边不需要法线,+ * 也就不必为放大后的比例再建一个网格。+ */+ drawEars(scaleMul, flatColor){+ const gl=this.gl, p=this.p;+ if(!this.mEar) return;+ const flat = flatColor != null;+ if(flat){+ this.pSolid.use();+ gl.uniform1f(this.pSolid.u('uAlpha'),1);+ gl.uniform3fv(this.pSolid.u('uColor'),flatColor);+ } else {+ this.pBody.use();+ gl.uniform1f(this.pBody.u('uAlpha'),1);+ gl.uniform1f(this.pBody.u('uGrad'),p.grad);+ gl.uniform1f(this.pBody.u('uAmb'),p.amb);+ gl.uniform1f(this.pBody.u('uDiff'),p.diff);+ gl.uniform1f(this.pBody.u('uGloss'),p.gloss);+ // 耳朵比身体暗一档(earShade)。一条**浅色**球上这是必需的:耳朵伸在+ // 身体那条深描边之外,自己没有边,在亮色 App 上会糊进背景。+ // 同一个乘数在真机 drawEars 里也有(BallLook.earShade)。+ const e=this.albedo||[0.5,0.5,0.5];+ gl.uniform3f(this.pBody.u('uColor'),+ e[0]*p.earShade, e[1]*p.earShade, e[2]*p.earShade);+ }+ for(const side of [-1,1]){+ let l=M4.id();+ l=M4.translate(l, side*p.earX, p.earY, p.earZ);+ // 往两边撇开:顶点从 +Y 转向 ∓x,所以左耳(+x 转角)往外。+ // earTilt 是**度**(跟 BallLook.earTilt 一致),M4.rotate 收的就是度。+ l=M4.rotate(l, -side*p.earTilt, 0,0,1);+ if(scaleMul!==1) l=M4.scale(l,scaleMul,scaleMul,scaleMul);+ const model=M4.mul(this.root,l);+ const mv=M4.mul(this.view,model), mvp=M4.mul(this.proj,mv);+ if(flat){+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,mvp);+ this.mEar.bind(this.pSolid.a('aPos'));+ } else {+ gl.uniformMatrix4fv(this.pBody.u('uMVP'),false,mvp);+ gl.uniformMatrix4fv(this.pBody.u('uMV'),false,mv);+ this.mEar.bind(this.pBody.a('aPos'),this.pBody.a('aNormal'));+ }+ this.mEar.draw();+ }+ }++ /* face 空间里的一个元素:T * R * S,跟 setLocalFromFace 同序。+ sz 默认跟随 sx —— 点状元素(瞳孔、高光)是各向同性的小球,+ 漏掉 z 会把单位球沿视线拉成一根柱,糊掉整颗球。 */+ faceElem(x,y,rotZ,sy,sx=1,pivotY=0,sz=null,zOff=0){+ let l=M4.id();+ l=M4.translate(l,x,y+pivotY*(1-sy),zOff);+ l=M4.rotate(l,rotZ*57.29578,0,0,1);+ l=M4.scale(l,sx,sy,sz===null?1:sz);+ const model=M4.mul(this.face,l);+ const mv=M4.mul(this.view,model);+ this.mvp=M4.mul(this.proj,mv);+ }+}++/* =========================================================================+ 9. UI+ ========================================================================= */+let curPreset=P_CREAM.key;+const edited={};+PRESETS.forEach(p=>edited[p.key]=clone(p));+let stages=[];+let heroes=[];++function showErr(e){+ const box=document.getElementById('err');+ box.style.display='block';+ box.textContent=String(e && e.stack || e);+ console.error(e);+}++function buildStage(presetKey){+ const p=PRESETS.find(q=>q.key===presetKey);+ const cell=document.createElement('div');+ cell.className='cell'; cell.dataset.k=presetKey;+ cell.setAttribute('aria-pressed', presetKey===curPreset ? 'true':'false');+ cell.innerHTML=`+ <div class="stage dark">+ <div class="badge">1:1</div>+ <canvas></canvas>+ </div>+ <div class="cap"><b>${p.label}${p.key==='cream'||p.key==='imp'?' · 已落地':''}</b><span>${p.tag}</span></div>`;+ cell.onclick=()=>{ curPreset=presetKey; syncUI(); };+ return {el:cell, cv:cell.querySelector('canvas'), key:presetKey, ball:null};+}++/** 网格的底子只是 CSS —— 画布本来就是透明的,切换不用重画。 */+function setBg(bg){+ stages.forEach(s=>{ s.el.querySelector('.stage').className='stage '+bg; });+ document.querySelectorAll('#bgToggle button').forEach(b=>{+ b.setAttribute('aria-pressed', b.dataset.bg===bg ? 'true':'false');+ });+}++function build(){+ const ci=document.getElementById('presets');+ PRESETS.forEach(p=>{+ const b=document.createElement('button');+ b.type='button'; b.dataset.k=p.key;+ b.innerHTML=`${p.label}<span class="tag">${p.tag}</span>`;+ b.onclick=()=>{ curPreset=p.key; syncUI(); };+ ci.appendChild(b);+ });++ const ms=document.getElementById('mood');+ MOODS.forEach(m=>{+ const o=document.createElement('option'); o.value=m; o.textContent=m; ms.appendChild(o);+ });+ ms.value='SLEEPING';+ ms.onchange=()=>{+ stages.forEach(s=>s.ball.mood=ms.value);+ heroes.forEach(h=>h.mood=ms.value);+ };++ // 网格:每个预设一个 1:1 画布。Ball 必须在挂进 DOM 之后建,+ // 它要读 clientWidth 才知道画多大。+ const grid=document.getElementById('grid');+ PRESETS.forEach(p=>{+ const st=buildStage(p.key);+ grid.appendChild(st.el);+ st.ball=new Ball(st.cv, edited[p.key]);+ st.ball.mood='SLEEPING';+ stages.push(st);+ });++ // 放大区:同一个预设、两种底子,各一个画布。+ heroes=[...document.querySelectorAll('.heroCv')].map(cv=>{+ const b=new Ball(cv, edited[curPreset]);+ b.mood='SLEEPING';+ return b;+ });++ document.querySelectorAll('#bgToggle button').forEach(b=>{+ b.onclick=()=>setBg(b.dataset.bg);+ });++ bindSliders();+ syncUI();+ requestAnimationFrame(loop);+}++const SLIDERS=[+ ['s-eyeR','eyeR','v-eyeR',v=>v.toFixed(3)],+ ['s-eyeX','eyeX','v-eyeX',v=>v.toFixed(3)],+ ['s-eyeY','eyeY','v-eyeY',v=>v.toFixed(3)],+ ['s-tint','tint','v-tint',v=>v.toFixed(2)],+ ['s-mul','albedoMul','v-mul',v=>v.toFixed(2)],+ ['s-gloss','gloss','v-gloss',v=>v.toFixed(2)],+ ['s-blush','blushMul','v-blush',v=>v.toFixed(2)],+ ['s-smile','smileBoost','v-smile',v=>v.toFixed(2)],+ ['s-bubR','bubbleR','v-bubR',v=>v.toFixed(3)],+ ['s-bubY','bubbleY','v-bubY',v=>v.toFixed(3)],+ ['s-bubIn','bubbleInner','v-bubIn',v=>v.toFixed(2)],+ // 耳朵 / 眉毛。earTilt、browTilt 在 BallLook 里都是**度**,所以显示也带度数,+ // 不然 24 和 0.42 摆在一起,用户没法判断哪个是"撇了 24 度"。+ ['s-earH','earH','v-earH',v=>v.toFixed(2)],+ ['s-earY','earY','v-earY',v=>v.toFixed(3)],+ ['s-earX','earX','v-earX',v=>v.toFixed(3)],+ ['s-earTilt','earTilt','v-earTilt',v=>v.toFixed(0)+'°'],+ ['s-earShade','earShade','v-earShade',v=>v.toFixed(2)],+ ['s-browW','browW','v-browW',v=>v.toFixed(3)],+ ['s-browY','browY','v-browY',v=>v.toFixed(3)],+ ['s-browTilt','browTilt','v-browTilt',v=>v.toFixed(0)+'°'],+ // 眼神。eyeCut 是**比例**(上半个眼睛切掉多少),eyeTilt 是度 —— 和 BallLook+ // 里的字段同名同单位。eyeCut 拖动能重建网格,见 setParams 的 changedSame。+ ['s-eyeCut','eyeCut','v-eyeCut',v=>v.toFixed(2)],+ ['s-eyeTilt','eyeTilt','v-eyeTilt',v=>v.toFixed(0)+'°'],+ ['s-pupilAspect','pupilAspect','v-pupilAspect',v=>v.toFixed(2)],+];++function bindSliders(){+ SLIDERS.forEach(([sid,key,vid,fmt])=>{+ const s=document.getElementById(sid);+ s.oninput=()=>{+ const v=parseFloat(s.value);+ edited[curPreset][key]=v;+ document.getElementById(vid).textContent=fmt(v);+ applyAll();+ };+ });+ document.getElementById('reset').onclick=()=>{+ edited[curPreset]=clone(PRESETS.find(p=>p.key===curPreset));+ applyAll(); syncUI();+ };+}++function syncUI(){+ document.querySelectorAll('#presets button').forEach(b=>{+ b.setAttribute('aria-pressed', b.dataset.k===curPreset ? 'true':'false');+ });+ stages.forEach(s=>s.el.setAttribute('aria-pressed', s.key===curPreset ? 'true':'false'));+ const p=edited[curPreset];+ SLIDERS.forEach(([sid,key,vid,fmt])=>{+ document.getElementById(sid).value=p[key];+ document.getElementById(vid).textContent=fmt(p[key]);+ });+ applyAll();+}++function applyAll(){+ stages.forEach(s=>s.ball.setParams(edited[s.key]));+ heroes.forEach(h=>h.setParams(edited[curPreset]));+}++let last=performance.now(), t0=last;+function loop(now){+ const dt=Math.min(0.05,(now-last)/1000); last=now;+ const t=(now-t0)/1000;+ try{+ stages.forEach(s=>s.ball.frame(dt,t));+ heroes.forEach(h=>h.frame(dt,t));+ }catch(e){ showErr(e); return; }+ requestAnimationFrame(loop);+}++try{+ build();+ window.addEventListener('resize',()=>{ stages.forEach(s=>s.ball.resize()); heroes.forEach(h=>h.resize()); });+ // 给测量脚本用:把渲染器和矩阵栈暴露出去,好从画布上量实际像素。+ window.LAB={Ball,M4,FACE_OLD,FACE_NEW,PRESETS,edited,version:'v31',+ get stages(){return stages}, get heroes(){return heroes}};+ window.__ready=true;+}catch(e){ showErr(e); }+</script>+</body>+</html>diff --git a/reports/ball-faces-lab.html b/reports/ball-faces-lab.htmlnew file mode 100644index 0000000..37f8685--- /dev/null+++ b/reports/ball-faces-lab.html@@ -0,0 +1,2165 @@+<!DOCTYPE html>+<html lang="zh-CN" data-page-node-id="hJTC176C2uAF7p7h5xRD5q">+<head data-page-node-id="4slCRHHEGMdHcTHsXmZUbE">+<meta charset="utf-8" data-page-node-id="O9HpAC3UFFxWLUftFe5zFn">+<meta name="viewport" content="width=device-width, initial-scale=1" data-page-node-id="ExHcPBiuwcXrHgpok4jDyD">+<title>球球性格实验室 · 邪恶与暴躁</title>+<style>+ :root{+ --bg:#f6f7f9; --panel:#ffffff; --line:#e3e6ec; --ink:#1b1f27;+ --ink2:#5b6472; --ink3:#8b95a5; --accent:#3b6ef5;+ --card:#0f172a;+ --app:#eef1f6;+ }+ *{box-sizing:border-box}+ body{+ margin:0; background:var(--bg); color:var(--ink);+ font:14px/1.6 -apple-system,BlinkMacSystemFont,"PingFang SC","Helvetica Neue",Arial,sans-serif;+ -webkit-font-smoothing:antialiased;+ }+ header{padding:20px 24px 8px;max-width:1180px;margin:0 auto}+ h1{margin:0 0 4px;font-size:19px;letter-spacing:-.01em}+ .sub{color:var(--ink2);font-size:13px}+ code{background:#eef0f4;padding:1px 5px;border-radius:4px;font-size:11.5px}+ .wrap{max-width:1180px;margin:0 auto;padding:0 24px 48px;display:grid;+ grid-template-columns:250px 1fr;gap:20px;align-items:start}+ @media(max-width:900px){.wrap{grid-template-columns:1fr}}++ .panel{background:var(--panel);border:1px solid var(--line);border-radius:12px;padding:14px}+ .panel h2{margin:0 0 10px;font-size:12px;letter-spacing:.08em;text-transform:uppercase;color:var(--ink3);font-weight:600}+ .sticky{position:sticky;top:16px}++ .presets{display:flex;flex-direction:column;gap:6px;margin-bottom:16px}+ .presets button{+ text-align:left;padding:9px 11px;border:1px solid var(--line);background:#fff;+ border-radius:9px;cursor:pointer;font:inherit;color:var(--ink);transition:.13s;+ }+ .presets button:hover{border-color:#c6cdd9}+ .presets button[aria-pressed=true]{border-color:var(--accent);background:#eef3ff;box-shadow:inset 3px 0 0 var(--accent)}+ .presets .tag{display:block;font-size:11px;color:var(--ink3);margin-top:1px}++ label.row{display:flex;justify-content:space-between;align-items:center;gap:8px;margin:11px 0 3px;font-size:12px;color:var(--ink2)}+ label.row b{font-weight:600;color:var(--ink);font-variant-numeric:tabular-nums}+ input[type=range]{width:100%;accent-color:var(--accent)}+ select{width:100%;padding:7px 9px;border:1px solid var(--line);border-radius:8px;background:#fff;font:inherit;color:var(--ink)}++ .grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(168px,1fr));gap:14px}+ .cell{background:var(--panel);border:1px solid var(--line);border-radius:12px;overflow:hidden;cursor:pointer;+ transition:border-color .13s, box-shadow .13s}+ .cell:hover{border-color:#c6cdd9}+ .cell[aria-pressed=true]{border-color:var(--accent);box-shadow:0 0 0 1px var(--accent)}+ .cell .stage{position:relative;aspect-ratio:1;display:flex;align-items:center;justify-content:center}+ .cell .stage.dark{background:var(--card)}+ .cell .stage.app{background:var(--app)}+ .cell .stage.app::before{+ content:"";position:absolute;inset:0;pointer-events:none;+ background:+ radial-gradient(circle at 24% 30%, #ffffff 0 28px, transparent 29px),+ radial-gradient(circle at 78% 70%, #ffffff 0 40px, transparent 41px),+ linear-gradient(transparent 0 44px, #ffffff 44px 46px, transparent 46px);+ opacity:.9;+ }+ .cell canvas{position:relative;display:block;width:100%;height:100%}+ .cell .cap{padding:9px 12px 11px;border-top:1px solid var(--line)}+ .cell .cap b{font-size:13px}+ .cell .cap span{display:block;font-size:11.5px;color:var(--ink3);margin-top:1px}+ .badge{position:absolute;top:8px;left:8px;font-size:10.5px;letter-spacing:.04em;+ padding:2px 7px;border-radius:20px;background:rgba(255,255,255,.14);color:#cfd6e4;+ border:1px solid rgba(255,255,255,.16);z-index:2}+ .stage.app .badge{background:rgba(15,23,42,.06);color:#5b6472;border-color:rgba(15,23,42,.1)}++ .hero{background:var(--panel);border:1px solid var(--line);border-radius:12px;padding:14px;display:grid;+ grid-template-columns:1fr 250px;gap:16px;align-items:center}+ @media(max-width:1000px){.hero{grid-template-columns:1fr}}+ .hpair{display:grid;grid-template-columns:1fr 1fr;gap:12px}+ .hstage{position:relative;border-radius:10px;display:flex;align-items:center;justify-content:center;+ padding:10px;overflow:hidden}+ .hstage.dark{background:var(--card)}+ .hstage.app{background:var(--app)}+ .hstage.app::before{content:"";position:absolute;inset:0;pointer-events:none;+ background:radial-gradient(circle at 24% 30%, #fff 0 30px, transparent 31px),+ radial-gradient(circle at 78% 72%, #fff 0 44px, transparent 45px);opacity:.9}+ .hstage canvas{position:relative;width:100%;aspect-ratio:1;display:block}+ .hbadge{position:absolute;top:8px;left:8px;font-size:10.5px;letter-spacing:.04em;z-index:2;+ padding:2px 7px;border-radius:20px;background:rgba(255,255,255,.14);color:#cfd6e4;+ border:1px solid rgba(255,255,255,.16)}+ .hstage.app .hbadge{background:rgba(15,23,42,.06);color:#5b6472;border-color:rgba(15,23,42,.1)}++ .seg{display:inline-flex;gap:0;margin-left:10px;vertical-align:middle;border:1px solid var(--line);+ border-radius:8px;overflow:hidden}+ .seg button{border:0;background:#fff;font:inherit;font-size:12px;padding:3px 11px;cursor:pointer;color:var(--ink2)}+ .seg button[aria-pressed=true]{background:var(--accent);color:#fff}+ .hero .notes{font-size:12.5px;color:var(--ink2)}+ .hero .notes h3{margin:0 0 7px;font-size:12px;letter-spacing:.08em;text-transform:uppercase;color:var(--ink3)}+ .hero .notes ul{margin:0;padding-left:16px}+ .hero .notes li{margin-bottom:5px}+ .sec{margin:22px 0 10px;font-size:13px;font-weight:600;color:var(--ink)}+ .sec em{font-style:normal;font-weight:400;color:var(--ink3);font-size:12px;margin-left:6px}+ .legend{margin-top:14px;font-size:12.5px;color:var(--ink2);border-top:1px solid var(--line);padding-top:10px}+ #err{margin:14px 0;padding:12px;border-radius:10px;background:#fff1f1;border:1px solid #f5c2c2;+ color:#a02020;font:12px/1.5 ui-monospace,SFMono-Regular,Menlo,monospace;white-space:pre-wrap;display:none}+</style>+</head>+<body data-page-node-id="SPoEpCghNXJ2BBcVKouME5">++<header data-page-node-id="C3dS7imDfzSrXCYs3EGg8D">+ <h1 data-page-node-id="LsEdSfvFA8mQT1DMHaHzSu">球球性格实验室 · 邪恶与暴躁</h1>+ <div class="sub" data-page-node-id="ArBs6ap0weFHb8cUrkZFZi">+ 上一版被否,根子不在颜色。翻一遍渲染管线:<code>brow</code> 这个数<b>从来只把眼球转 0.4 弧度</b>,+ 屏幕上没有一根眉毛 —— 所以不管把身体调成多红多油,脸都是同一张脸,+ 而一张不变的脸换什么色都只是"同一只球穿了不同的衣服"。+ </div>++ <div class="sub" style="margin-top:10px;padding:11px 14px;background:#eef3ff;border:1px solid #cfdcff;border-radius:9px;color:#22314f">+ <b>这一版的规矩 ——</b><br>+ <b>可爱住在骨头上</b>:球径、眼睛大小、眼睛高低、眼距、腮红,<b>一个数都不动</b>。+ 下面五套新的,骨架全部等于<b>你已经装机的奶油球</b>。<br>+ <b>邪恶住在三个新部件上</b>:<b>眉</b>(一块能随手转角的楔子)、<b>牙</b>(同一块楔子缩到牙那么大)、+ <b>张嘴</b>(一块半圆盘 —— 半环永远是一根线,管径加粗也只读成"嘴很厚")。<br>+ <b>颜色只回答"它是什么物种",不回答"它坏不坏"。</b>+ 所以主角小恶魔的身体<b>不是血红</b>,是近黑的炭 —— 红只出现在它发光的地方。+ </div>++ <div class="sub" style="margin-top:10px;padding:11px 14px;background:#fff8ec;border:1px solid #f3ddb5;border-radius:9px;color:#584018">+ <b>落地成本:</b>+ 每张卡上标着。<b>眉 / 牙 / 张嘴 / 蒸汽 / 怒符</b>这五样,实验室里是现加的代码+ (一块楔子 + 一块半圆盘,加约 120 行绘制 + 20 个 <code>BallLook</code> 字段)。+ <b>项目代码一行没动</b>,这是设计稿。<b>奶油球</b>是最左边那张,留作基线。+ </div>+</header>++<div class="wrap" data-page-node-id="9HLcd6JUUvl59L0PSgyFBq">++ <div class="panel sticky" data-page-node-id="HABepjNBIjQbY5mIlxUVr6">+ <h2 data-page-node-id="C0jVPSKXrONWIuAjoSJG4Z">预设</h2>+ <div class="presets" id="presets" data-page-node-id="upEDhOFwpJRq7Mo8cyRBIX"></div>++ <h2 data-page-node-id="sEwhmEbvpEySRQWa4uBsCD">表情</h2>+ <select id="mood" data-page-node-id="TZKO9dDFXI0Dq3kLQTfNRo"></select>++ <label class="row" data-page-node-id="Dglty4qiK3L8HDmEezOsxR"><span data-page-node-id="UN9G4sHG0Bj8UOjfC0WL1F">眼睛大小</span><b id="v-eyeR" data-page-node-id="DYYLAOZzF6vimgx2W8FeL1"></b></label>+ <input type="range" id="s-eyeR" min="0.06" max="0.17" step="0.004" data-page-node-id="KIz5AOYP03RgasdHpxm6Kx">++ <label class="row" data-page-node-id="PukAk0fccJffQg6XoNWxAd"><span data-page-node-id="wdCOS4nl7TbGQFtfSlXwaB">眼睛间距</span><b id="v-eyeX" data-page-node-id="wz2ETthFuyKnu8oCAJXOCA"></b></label>+ <input type="range" id="s-eyeX" min="0.16" max="0.36" step="0.004" data-page-node-id="l9Py8pmsQ6bOWullzaj5I5">++ <label class="row" data-page-node-id="xgUeCusNQrGCXnChzeGh8h"><span data-page-node-id="TE1FMTFvdliJzcrCcU5n7N">眼睛高低</span><b id="v-eyeY" data-page-node-id="AKWbvoulvHQUsNM50PnTFy"></b></label>+ <input type="range" id="s-eyeY" min="0.06" max="0.34" step="0.004" data-page-node-id="RTBkRUjpXdFZ5Zlw2fipfN">++ <label class="row" data-page-node-id="wVfI6BzFBHrutMvuSuZXXb"><span data-page-node-id="UfS7UOXhtP6OKE1vHOz4vM">身体上色量</span><b id="v-tint" data-page-node-id="IsBZ3f64VsrgC1ajn7Aokz"></b></label>+ <input type="range" id="s-tint" min="0" max="1" step="0.02" data-page-node-id="L60TKqLya1hloUPtVYyalx">++ <label class="row" data-page-node-id="0G1NGHz2IBzHg5BWEoiVSW"><span data-page-node-id="xiIhwCG1jYQjA9sA3SW7Ge">身体明度</span><b id="v-mul" data-page-node-id="T8EjUg3i9AMkYbwgoHGH83"></b></label>+ <input type="range" id="s-mul" min="0.4" max="1.15" step="0.02" data-page-node-id="GTnQGUpdtU7uAsOFG6wM0y">++ <label class="row" data-page-node-id="iPBIDqhsHP63yTqaraCJee"><span data-page-node-id="grbkX5XBhzUEIXWHDckyml">镜面光泽</span><b id="v-gloss" data-page-node-id="NPMwWGefUK0sDS8HoVTCN5"></b></label>+ <input type="range" id="s-gloss" min="0" max="1.6" step="0.05" data-page-node-id="T1XQfesEwg8JzF2xzR9hWB">++ <label class="row" data-page-node-id="sSdrRiTlMSAfbQBsHCZ0ok"><span data-page-node-id="v98mmPaA8BH7wiQUUWMKtz">腮红</span><b id="v-blush" data-page-node-id="dSfidlMTI8HEJhKoa1CuOs"></b></label>+ <input type="range" id="s-blush" min="0" max="2.4" step="0.05" data-page-node-id="Mh0jNmmGfbyp3PUVb5fopW">++ <label class="row" data-page-node-id="EeTUqBAxMWNFe2FHi4RxOe"><span data-page-node-id="2h4F5Ra1j3mS3aCypOk2gG">嘴角上扬加成</span><b id="v-smile" data-page-node-id="hQwmt0CQZdkb33gFHrksXw"></b></label>+ <input type="range" id="s-smile" min="-0.2" max="0.45" step="0.01" data-page-node-id="xolz6DWREygBhlVGXA7f6Y">++ <label class="row"><span>鼻涕泡大小</span><b id="v-bubR"></b></label>+ <input type="range" id="s-bubR" min="0.03" max="0.16" step="0.005">++ <label class="row"><span>鼻涕泡高低</span><b id="v-bubY"></b></label>+ <input type="range" id="s-bubY" min="-0.20" max="0.05" step="0.005">++ <label class="row"><span>泡的厚度</span><b id="v-bubIn"></b></label>+ <input type="range" id="s-bubIn" min="0" max="0.5" step="0.01">++ <h2 style="margin-top:18px">性格 <em style="text-transform:none;letter-spacing:0;font-weight:400">邪恶住在这三样上</em></h2>++ <label class="row"><span>眉:角度</span><b id="v-browTilt"></b></label>+ <input type="range" id="s-browTilt" min="-0.5" max="0.9" step="0.02">+ <div class="legend" style="margin-top:2px;padding-top:0;border:0;font-size:11.5px">正 = 内低外高(凶) 负 = 内高外低(委屈)</div>++ <label class="row"><span>眉:左右差</span><b id="v-browSkew"></b></label>+ <input type="range" id="s-browSkew" min="-0.9" max="0.9" step="0.02">++ <label class="row"><span>眉:粗细</span><b id="v-browThick"></b></label>+ <input type="range" id="s-browThick" min="0.012" max="0.070" step="0.002">++ <label class="row"><span>眉:离眼睛多远</span><b id="v-browY"></b></label>+ <input type="range" id="s-browY" min="0.10" max="0.26" step="0.004">++ <label class="row"><span>眉:浓度</span><b id="v-brow"></b></label>+ <input type="range" id="s-brow" min="0" max="1" step="0.05">++ <label class="row"><span>眼:挤扁</span><b id="v-eyeSquash"></b></label>+ <input type="range" id="s-eyeSquash" min="0.15" max="1.1" step="0.02">++ <label class="row"><span>眼:内角下压</span><b id="v-eyeTilt"></b></label>+ <input type="range" id="s-eyeTilt" min="-0.3" max="0.3" step="0.02">++ <label class="row"><span>牙:几颗</span><b id="v-fang"></b></label>+ <input type="range" id="s-fang" min="0" max="2" step="1">++ <label class="row"><span>牙:长度</span><b id="v-fangLen"></b></label>+ <input type="range" id="s-fangLen" min="0.015" max="0.090" step="0.003">++ <label class="row"><span>牙:朝上 / 朝下</span><b id="v-fangFlip"></b></label>+ <input type="range" id="s-fangFlip" min="0" max="1" step="1">++ <label class="row"><span>锯齿牙:几颗</span><b id="v-teeth"></b></label>+ <input type="range" id="s-teeth" min="0" max="6" step="1">++ <label class="row"><span>张嘴:高/宽比</span><b id="v-gawp"></b></label>+ <input type="range" id="s-gawp" min="0.1" max="1.3" step="0.02">++ <label class="row"><span>蒸汽</span><b id="v-steam"></b></label>+ <input type="range" id="s-steam" min="0" max="1" step="0.05">++ <label class="row"><span>怒符</span><b id="v-vein"></b></label>+ <input type="range" id="s-vein" min="0" max="1" step="0.05">++ <button id="reset" style="margin-top:14px;width:100%;padding:8px;border:1px solid var(--line);background:#fff;border-radius:9px;font:inherit;cursor:pointer;color:var(--ink2)" data-page-node-id="MsiIXrvrt2aqAPJ2AEw1OU">+ 还原此预设+ </button>++ <div class="legend" data-page-node-id="YJVBjoZyCoFB3Q0bfWn9JZ">+ 瞳孔 / 高光由预设决定,不给滑块 —— 那是造型选择,不是调参。<br>+ 眉 / 牙 / 嘴这三组是这一版的新东西,<b>随便拉</b>:把眉的角度从 0.9 拉到 −0.5,+ 同一颗球会在"凶神"和"委屈"之间来回,而它的骨架一个像素都没动。+ </div>+ </div>++ <div data-page-node-id="xHcP9fBoFkC2vu3VH59FJR">+ <div id="err" data-page-node-id="iSvxKDZoG2BMcUjEaWOwfP"></div>++ <div class="sec">放大细节 <em>选中的预设 · 两种底子并排看</em></div>+ <div class="hero">+ <div class="hpair">+ <div class="hstage dark"><div class="hbadge">深色卡片</div><canvas class="heroCv"></canvas></div>+ <div class="hstage app"><div class="hbadge">亮色 App</div><canvas class="heroCv"></canvas></div>+ </div>+ <div class="notes">+ <h3>五套新的,一句话一套</h3>+ <ul>+ <li><b>小恶魔 ★</b> — 斜眉(内低外高 0.46 弧度)+ 一颗向上的虎牙 + 嘴角两个小勾。+ <b>骨架就是奶油球</b>:眼宽 0.122、眼睛那么低、腮红 0.5 —— 一个都没改。+ 它是一颗"表情很坏的奶油球",不是一颗红色的怪球。</li>+ <li><b>暴躁球</b> — 押在眼和嘴上:两撇压到底的粗眉、把眼缝挤到 0.42、+ <b>一张张开的嘴</b> + 上下各四颗锯齿牙;再加头顶两缕蒸汽、太阳穴一个怒符。+ 可爱靠那两块腮红<b>没有关掉</b>(0.80,全批最高)—— 这是"烈子"的全部秘密:+ 一张脸连血色都气没了,那是恐怖片,不是可爱。</li>+ <li><b>腹黑球</b> — 没有獠牙、没有角、没有蒸汽,<b>只有不对称</b>:+ 一条眉挑起来、一条压下去(左右角度差 1.24 弧度),一只眼半闭,嘴角单边翘,+ 瞳孔偏到眼角。<b>不对称读作"心机"的前提是别的都对称</b> —— 唯一的例外就是有意为之。</li>+ <li><b>吸血鬼球</b> — 两颗从上唇<b>垂下来</b>的獠牙 + 苍白身体 + 两块红腮。+ 苍白给了腮红一个存在的理由,腮红又把可爱度还回来。描边是深红而不是发光。</li>+ <li><b>猫妖</b> — 全批最大的一双眼睛(0.145)+ 后压的猫耳 + ω 两个凹口里各一颗小虎牙。+ 它说的是这批设计里最反直觉的一条:<b>可爱度是弹药,不是跟邪恶抢位置的东西</b>。</li>+ </ul>+ <h3 style="margin-top:12px">三个部件,各自能买到什么</h3>+ <ul>+ <li><b>加一条眉</b> → 从"没有表情"到"有态度"。单件收益最大。</li>+ <li><b>加一颗牙</b> → 从"凶"到"想咬你"。<b>一颗 = 调皮,两颗 = 掠食</b>。</li>+ <li><b>把嘴换成一块面</b> → 从"咧嘴"到"在吼"。这是唯一必须新几何的一条。</li>+ </ul>+ </div>+ </div>++ <div class="sec">+ 六套,1:1 真实尺寸 <em>点一张看大图 · 按 1-6 切表情 · 每套自带一个"代表表情"</em>+ <span class="seg" id="bgToggle">+ <button type="button" data-bg="dark" aria-pressed="true">深色卡片</button>+ <button type="button" data-bg="app" aria-pressed="false">亮色 App</button>+ </span>+ </div>+ <div class="grid" id="grid"></div>+ </div>+</div>++<script>+"use strict";+/* =========================================================================+ 1. mat4 —— 列主序,跟 android.opengl.Matrix 同一套约定+ ========================================================================= */+const M4 = {+ id(){ return new Float32Array([1,0,0,0, 0,1,0,0, 0,0,1,0, 0,0,0,1]); },+ perspective(fovyDeg, aspect, near, far){+ const f = 1/Math.tan(fovyDeg*Math.PI/180/2), m = new Float32Array(16);+ m[0]=f/aspect; m[5]=f; m[10]=(far+near)/(near-far); m[11]=-1;+ m[14]=2*far*near/(near-far); return m;+ },+ lookAt(ex,ey,ez, cx,cy,cz, ux,uy,uz){+ let zx=ex-cx, zy=ey-cy, zz=ez-cz, l=Math.hypot(zx,zy,zz);+ zx/=l; zy/=l; zz/=l;+ let xx=uy*zz-uz*zy, xy=uz*zx-ux*zz, xz=ux*zy-uy*zx;+ l=Math.hypot(xx,xy,xz)||1; xx/=l; xy/=l; xz/=l;+ const yx=zy*xz-zz*xy, yy=zz*xx-zx*xz, yz=zx*xy-zy*xx;+ const m=new Float32Array(16);+ m[0]=xx; m[4]=xy; m[8]=xz; m[12]=-(xx*ex+xy*ey+xz*ez);+ m[1]=yx; m[5]=yy; m[9]=yz; m[13]=-(yx*ex+yy*ey+yz*ez);+ m[2]=zx; m[6]=zy; m[10]=zz; m[14]=-(zx*ex+zy*ey+zz*ez);+ m[15]=1; return m;+ },+ mul(a,b){+ const o=new Float32Array(16);+ for(let c=0;c<4;c++)for(let r=0;r<4;r++){+ let s=0; for(let k=0;k<4;k++) s+=a[k*4+r]*b[c*4+k];+ o[c*4+r]=s;+ } return o;+ },+ translate(m,x,y,z){ const t=M4.id(); t[12]=x;t[13]=y;t[14]=z; return M4.mul(m,t); },+ scale(m,x,y,z){ const t=M4.id(); t[0]=x;t[5]=y;t[10]=z; return M4.mul(m,t); },+ rotate(m,deg,ax,ay,az){+ let l=Math.hypot(ax,ay,az); ax/=l; ay/=l; az/=l;+ const a=deg*Math.PI/180, s=Math.sin(a), c=Math.cos(a), t=1-c, r=M4.id();+ r[0]=t*ax*ax+c; r[1]=t*ax*ay+s*az; r[2]=t*ax*az-s*ay;+ r[4]=t*ax*ay-s*az; r[5]=t*ay*ay+c; r[6]=t*ay*az+s*ax;+ r[8]=t*ax*az+s*ay; r[9]=t*ay*az-s*ax; r[10]=t*az*az+c;+ return M4.mul(m,r);+ },+};++/* =========================================================================+ 2. 几何体 —— IcoSphere / UvSphere / Capsule / Torus 的逐行移植+ ========================================================================= */+function icoSphere(subdivisions){+ const t=(1+Math.sqrt(5))/2;+ const base=[-1,t,0, 1,t,0, -1,-t,0, 1,-t,0,+ 0,-1,t, 0,1,t, 0,-1,-t, 0,1,-t,+ t,0,-1, t,0,1, -t,0,-1, -t,0,1];+ const pos=[];+ for(let i=0;i<12;i++){+ const x=base[i*3],y=base[i*3+1],z=base[i*3+2], inv=1/Math.hypot(x,y,z);+ pos.push(x*inv,y*inv,z*inv);+ }+ let faces=[0,11,5, 0,5,1, 0,1,7, 0,7,10, 0,10,11,+ 1,5,9, 5,11,4, 11,10,2, 10,7,6, 7,1,8,+ 3,9,4, 3,4,2, 3,2,6, 3,6,8, 3,8,9,+ 4,9,5, 2,4,11, 6,2,10, 8,6,7, 9,8,1];+ const cache=new Map();+ function mid(a,b){+ const lo=Math.min(a,b), hi=Math.max(a,b), key=lo*100000+hi;+ if(cache.has(key)) return cache.get(key);+ let mx=(pos[a*3]+pos[b*3])/2, my=(pos[a*3+1]+pos[b*3+1])/2, mz=(pos[a*3+2]+pos[b*3+2])/2;+ const inv=1/Math.hypot(mx,my,mz); mx*=inv; my*=inv; mz*=inv;+ const idx=pos.length/3; pos.push(mx,my,mz); cache.set(key,idx); return idx;+ }+ for(let s=0;s<subdivisions;s++){+ const next=[];+ for(let r=0;r<faces.length;r+=3){+ const a=faces[r],b=faces[r+1],c=faces[r+2];+ const ab=mid(a,b), bc=mid(b,c), ca=mid(c,a);+ next.push(a,ab,ca, b,bc,ab, c,ca,bc, ab,bc,ca);+ }+ faces=next; cache.clear();+ }+ const n=pos.length/3;+ return {pos:new Float32Array(pos), nrm:new Float32Array(pos),+ uv:new Float32Array(n*2), idx:new Uint16Array(faces)};+}++function uvSphere(wSeg,hSeg,radius=1){+ const n=(wSeg+1)*(hSeg+1);+ const pos=new Float32Array(n*3), nrm=new Float32Array(n*3), uv=new Float32Array(n*2);+ let p=0,u=0;+ for(let iy=0;iy<=hSeg;iy++){+ const v=iy/hSeg, phi=v*Math.PI, sPhi=Math.sin(phi), cPhi=Math.cos(phi);+ for(let ix=0;ix<=wSeg;ix++){+ const th=(ix/wSeg)*2*Math.PI;+ const x=-radius*Math.cos(th)*sPhi, y=radius*cPhi, z=radius*Math.sin(th)*sPhi;+ pos[p]=x; pos[p+1]=y; pos[p+2]=z;+ nrm[p]=x/radius; nrm[p+1]=y/radius; nrm[p+2]=z/radius;+ uv[u]=ix/wSeg; uv[u+1]=1-v; p+=3; u+=2;+ }+ }+ const raw=[], stride=wSeg+1;+ for(let iy=0;iy<hSeg;iy++)for(let ix=0;ix<wSeg;ix++){+ const a=iy*stride+ix+1, b=iy*stride+ix, c=(iy+1)*stride+ix, d=(iy+1)*stride+ix+1;+ if(iy!==0) raw.push(a,b,d);+ if(iy!==hSeg-1) raw.push(b,c,d);+ }+ return {pos,nrm,uv,idx:new Uint16Array(raw)};+}++function capsule(radius, cylH, capSegs=4, radialSegs=12){+ const half=cylH/2, totalRings=2*(capSegs+1), verts=totalRings*(radialSegs+1);+ const pos=new Float32Array(verts*3), nrm=new Float32Array(verts*3), uv=new Float32Array(verts*2);+ let p=0,u=0,ring=0;+ for(const sign of [1,-1]){+ for(let i=0;i<=capSegs;i++){+ const phi=sign>0 ? (i/capSegs)*(Math.PI/2) : Math.PI/2+(i/capSegs)*(Math.PI/2);+ const sP=Math.sin(phi), cP=Math.cos(phi);+ for(let j=0;j<=radialSegs;j++){+ const th=(j/radialSegs)*2*Math.PI;+ const nx=-Math.cos(th)*sP, ny=cP, nz=Math.sin(th)*sP;+ pos[p]=radius*nx; pos[p+1]=radius*ny+sign*half; pos[p+2]=radius*nz;+ nrm[p]=nx; nrm[p+1]=ny; nrm[p+2]=nz;+ uv[u]=j/radialSegs; uv[u+1]=1-ring/(totalRings-1); p+=3; u+=2;+ }+ ring++;+ }+ }+ const idx=[], stride=radialSegs+1;+ for(let r=0;r<totalRings-1;r++)for(let s=0;s<radialSegs;s++){+ const a=r*stride+s, b=r*stride+s+1, c=(r+1)*stride+s+1, d=(r+1)*stride+s;+ idx.push(a,b,c, a,c,d);+ }+ return {pos,nrm,uv,idx:new Uint16Array(idx)};+}++function torus(radius, tube, radialSegs, tubularSegs, arc=2*Math.PI){+ const verts=(radialSegs+1)*(tubularSegs+1);+ const pos=new Float32Array(verts*3), nrm=new Float32Array(verts*3), uv=new Float32Array(verts*2);+ let p=0,u=0;+ for(let j=0;j<=radialSegs;j++){+ const v=(j/radialSegs)*2*Math.PI, cV=Math.cos(v), sV=Math.sin(v);+ for(let i=0;i<=tubularSegs;i++){+ const uu=(i/tubularSegs)*arc, cU=Math.cos(uu), sU=Math.sin(uu);+ const rr=radius+tube*cV;+ const px=rr*cU, py=rr*sU, pz=tube*sV;+ pos[p]=px; pos[p+1]=py; pos[p+2]=pz;+ let nx=px-radius*cU, ny=py-radius*sU, nz=pz;+ const inv=1/Math.hypot(nx,ny,nz);+ nrm[p]=nx*inv; nrm[p+1]=ny*inv; nrm[p+2]=nz*inv;+ uv[u]=i/tubularSegs; uv[u+1]=j/radialSegs; p+=3; u+=2;+ }+ }+ const idx=[], stride=tubularSegs+1;+ for(let j=0;j<radialSegs;j++)for(let i=0;i<tubularSegs;i++){+ const a=j*stride+i, b=j*stride+i+1, c=(j+1)*stride+i+1, d=(j+1)*stride+i;+ idx.push(a,b,c, a,c,d);+ }+ return {pos,nrm,uv,idx:new Uint16Array(idx)};+}++/**+ * 椭圆底锥体,顶点在 +Y,底面在 y=0。耳朵用。+ *+ * z 方向的半径单独给([rz]),因为一只猫耳是个"扁三角":正面看要够宽,+ * 侧面看要薄。差别全在这一点上——用正圆锥出来的是犄角。+ */+function cone(rx, rz, height, segs=20){+ // 顶点数:侧面环(segs+1) + 顶点环(segs+1,为 UV 每段重复一个顶点)+ // + 底心(1) + 底环(segs+1) = 3*(segs+1)+1+ // 少算一圈的话,越界写入会被类型化数组静默丢掉,出来的是半个锥体。+ const n=3*(segs+1) + 1;+ const pos=new Float32Array(n*3), nrm=new Float32Array(n*3), uv=new Float32Array(n*2);+ const idx=[];+ let p=0,u=0;+ const put=(x,y,z,nx,ny,nz,uu,vv)=>{ pos[p]=x;pos[p+1]=y;pos[p+2]=z;+ nrm[p]=nx;nrm[p+1]=ny;nrm[p+2]=nz; uv[u]=uu;uv[u+1]=vv; p+=3;u+=2; };+ // 侧面:法线由 ∂P/∂t × ∂P/∂u 得来(椭圆情形下不能直接用位置当法线)+ const base = 0;+ for(let i=0;i<=segs;i++){+ const a=i/segs*2*Math.PI, ca=Math.cos(a), sa=Math.sin(a);+ let nx=height*rz*ca, ny=-rx*rz, nz=height*rx*sa;+ const inv=1/Math.hypot(nx,ny,nz); nx*=inv; ny*=inv; nz*=inv;+ put(rx*ca, 0, rz*sa, nx, ny, nz, i/segs, 0);+ }+ for(let i=0;i<=segs;i++){+ const a=i/segs*2*Math.PI, ca=Math.cos(a), sa=Math.sin(a);+ put(0, height, 0, height*rz*ca, -rx*rz, height*rx*sa, i/segs, 1);+ }+ // 底面(朝下)+ const baseCenter = n-1;+ put(0,0,0, 0,-1,0, 0.5,0.5);+ for(let i=0;i<=segs;i++){+ const a=i/segs*2*Math.PI;+ put(rx*Math.cos(a), 0, rz*Math.sin(a), 0,-1,0, Math.cos(a)*0.5+0.5, Math.sin(a)*0.5+0.5);+ }+ for(let i=0;i<segs;i++){+ const A=base+i, B=base+i+1; // 侧面+ idx.push(A, B, segs+1+i, B, segs+1+i+1, segs+1+i);+ const C=baseCenter+1+i; // 底面+ idx.push(baseCenter, C+1, C);+ }+ return {pos,nrm,uv,idx:new Uint16Array(idx)};+}++/**+ * 居中的楔子:把 [cone] 从 y=0..h 平移成 −h/2..h/2,好绕自己的中点转。+ *+ * 眉、尖牙、锯齿牙、怒符全用它 —— "一块可以随手缩放旋转的三角形"是这批设计+ * 里唯一缺的东西。它们都走 `pSolid`(无光照、纯色),所以画的时候随便做非等比+ * 缩放:法线不会被用坏,因为没有法线。+ */+function centeredCone(rx, rz, height, segs=14){+ const m=cone(rx,rz,height,segs);+ for(let i=1;i<m.pos.length;i+=3) m.pos[i]-=height*0.5;+ return m;+}++/**+ * 一只**张开**的嘴:半圆盘,面朝 +Z,弧在上半圆(跟 [torus] 那半环同一套约定,+ * 所以共用一个 `mouthRotZ`)。+ *+ * 为什么必须新做一块几何:半环永远是一根**线** —— 管径加到 0.09 也只是一根粗+ * 线,读出来是"嘴很厚",不是"嘴张着"。张嘴要的是一个**面**。+ * 这也是暴躁球那张脸唯一不能靠参数拿下的东西。+ */+function halfDisc(r, segs=28){+ const n=segs+2;+ const pos=new Float32Array(n*3), nrm=new Float32Array(n*3), uv=new Float32Array(n*2);+ const idx=[];+ let p=0,u=0;+ const put=(x,y,nx,ny,uu,vv)=>{ pos[p]=x;pos[p+1]=y;pos[p+2]=0;+ nrm[p]=nx;nrm[p+1]=ny;nrm[p+2]=1; uv[u]=uu;uv[u+1]=vv; p+=3;u+=2; };+ put(0,0, 0,0, 0.5,0.5); // 圆心+ for(let i=0;i<=segs;i++){+ const a=Math.PI*(1-i/segs); // π → 0:上半圆,从左扫到右+ put(Math.cos(a)*r, Math.sin(a)*r, 0,0, i/segs, 0);+ }+ for(let i=0;i<segs;i++) idx.push(0, 1+i, 1+i+1);+ return {pos,nrm,uv,idx:new Uint16Array(idx)};+}++/* =========================================================================+ 3. GL 小工具+ ========================================================================= */+function compile(gl,type,src){+ const s=gl.createShader(type);+ gl.shaderSource(s,src); gl.compileShader(s);+ if(!gl.getShaderParameter(s,gl.COMPILE_STATUS))+ throw new Error(gl.getShaderInfoLog(s)+"\n---\n"+src);+ return s;+}+class Prog{+ constructor(gl,vs,fs){+ this.gl=gl;+ const p=gl.createProgram();+ gl.attachShader(p,compile(gl,gl.VERTEX_SHADER,vs));+ gl.attachShader(p,compile(gl,gl.FRAGMENT_SHADER,fs));+ gl.linkProgram(p);+ if(!gl.getProgramParameter(p,gl.LINK_STATUS))+ throw new Error(gl.getProgramInfoLog(p));+ this.p=p; this.cache=new Map();+ }+ use(){ this.gl.useProgram(this.p); }+ u(n){ const k="u:"+n; if(!this.cache.has(k)) this.cache.set(k,this.gl.getUniformLocation(this.p,n)); return this.cache.get(k); }+ a(n){ const k="a:"+n; if(!this.cache.has(k)) this.cache.set(k,this.gl.getAttribLocation(this.p,n)); return this.cache.get(k); }+}++class GMesh{+ constructor(gl,data){+ this.gl=gl;+ const n=data.pos.length/3, stride=8;+ const inter=new Float32Array(n*stride);+ for(let i=0;i<n;i++){+ const j=i*stride;+ inter[j]=data.pos[i*3]; inter[j+1]=data.pos[i*3+1]; inter[j+2]=data.pos[i*3+2];+ inter[j+3]=data.nrm[i*3]; inter[j+4]=data.nrm[i*3+1]; inter[j+5]=data.nrm[i*3+2];+ inter[j+6]=data.uv[i*2]; inter[j+7]=data.uv[i*2+1];+ }+ this.vbo=gl.createBuffer();+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.bufferData(gl.ARRAY_BUFFER,inter,gl.STATIC_DRAW);+ this.ibo=gl.createBuffer();+ gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER,this.ibo);+ gl.bufferData(gl.ELEMENT_ARRAY_BUFFER,data.idx,gl.STATIC_DRAW);+ this.count=data.idx.length;+ }+ bind(aPos,aNrm=-1,aUV=-1){+ const gl=this.gl, S=8*4;+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER,this.ibo);+ if(aPos>=0){ gl.enableVertexAttribArray(aPos); gl.vertexAttribPointer(aPos,3,gl.FLOAT,false,S,0); }+ if(aNrm>=0){ gl.enableVertexAttribArray(aNrm); gl.vertexAttribPointer(aNrm,3,gl.FLOAT,false,S,12); }+ if(aUV>=0){ gl.enableVertexAttribArray(aUV); gl.vertexAttribPointer(aUV,2,gl.FLOAT,false,S,24); }+ }+ draw(){ this.gl.drawElements(this.gl.TRIANGLES,this.count,this.gl.UNSIGNED_SHORT,0); }+}++class GPoints{+ constructor(gl,capacity){+ this.gl=gl; this.capacity=capacity;+ this.data=new Float32Array(capacity*6);+ this.vbo=gl.createBuffer();+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.bufferData(gl.ARRAY_BUFFER,capacity*6*4,gl.DYNAMIC_DRAW);+ }+ upload(){+ const gl=this.gl;+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.bufferSubData(gl.ARRAY_BUFFER,0,this.data);+ }+ bind(aPos,aCol){+ const gl=this.gl, S=6*4;+ gl.bindBuffer(gl.ARRAY_BUFFER,this.vbo);+ gl.enableVertexAttribArray(aPos); gl.vertexAttribPointer(aPos,3,gl.FLOAT,false,S,0);+ gl.enableVertexAttribArray(aCol); gl.vertexAttribPointer(aCol,3,gl.FLOAT,false,S,12);+ }+ draw(){ this.gl.drawArrays(this.gl.POINTS,0,this.capacity); }+}++/* =========================================================================+ 4. 表情表 —— 与 Mood.kt 的 FACE 表一一对应+ ========================================================================= */+function F(o){ return Object.assign({eyeArc:0,blush:0,spread:1,breath:0.022,gape:0,bubble:0},o); }++/* 改动前的表 —— "现在"这个预设专用,做诚实的基线。 */+const FACE_OLD = {+ CALM: F({c:0x9db4ff, eye:1.00, brow:0.00, smile:0.10, pulse:1.00, tilt:0.00, skew:0.00, jitter:0.00}),+ HAPPY: F({c:0x5eead4, eye:0.72, brow:-0.16,smile:0.62, pulse:1.25, tilt:0.04, skew:0.00, jitter:0.00, eyeArc:0.90, blush:0.70}),+ CURIOUS: F({c:0xf0abfc, eye:1.22, brow:-0.06,smile:0.14, pulse:1.15, tilt:0.17, skew:0.055,jitter:0.00}),+ TENSE: F({c:0xfb923c, eye:0.88, brow:0.30, smile:-0.20,pulse:1.95, tilt:0.00, skew:0.02, jitter:0.55}),+ ANXIOUS: F({c:0xfb7185, eye:1.26, brow:0.42, smile:-0.44,pulse:2.50, tilt:0.00, skew:0.03, jitter:1.00}),+ CONCERNED: F({c:0xfbbf24, eye:1.10, brow:0.34, smile:-0.34,pulse:1.15, tilt:-0.05,skew:0.00, jitter:0.12}),+ LISTENING: F({c:0x7dd3fc, eye:1.60, brow:-0.14,smile:0.16, pulse:1.70, tilt:-0.13,skew:0.00, jitter:0.00, breath:0.055, gape:0.12}),+ HOLDING: F({c:0xfacc15, eye:0.92, brow:0.48, smile:-0.18,pulse:0.72, tilt:0.00, skew:0.00, jitter:0.00, breath:0.018}),+ THINKING: F({c:0xc084fc, eye:0.40, brow:0.32, smile:-0.06,pulse:1.15, tilt:0.15, skew:0.08, jitter:0.00, breath:0.042}),+ SPEAKING: F({c:0x9db4ff, eye:1.05, brow:0.00, smile:0.20, pulse:1.50, tilt:0.00, skew:0.00, jitter:0.00}),+ SEARCHING: F({c:0x7dd3fc, eye:0.84, brow:0.10, smile:0.06, pulse:1.60, tilt:0.13, skew:0.05, jitter:0.00, spread:1.16, breath:0.036}),+ SUCCESS: F({c:0x5eead4, eye:0.50, brow:-0.22,smile:0.80, pulse:1.30, tilt:0.00, skew:0.00, jitter:0.00, eyeArc:1.00, blush:0.95, spread:1.10}),+ FAILED: F({c:0xfb7185, eye:0.68, brow:0.45, smile:-0.50,pulse:1.10, tilt:-0.07,skew:0.02, jitter:0.22}),+ SLEEPING: F({c:0x64748b, eye:0.10, brow:0.06, smile:0.08, pulse:0.45, tilt:0.11, skew:0.00, jitter:0.00, eyeArc:0.85, spread:0.90}),+};++/* 改动后的表 —— 与 Mood.kt 的 FACE 逐字对齐,这样这里看到的就是装机后的样子。+ smile 与 blush 已经把"基线"烘进表里了,不再有额外的全局加成。 */+const FACE_NEW = {+ CALM: F({c:0x9db4ff, eye:1.00, brow:0.00, smile:0.26, pulse:1.00, tilt:0.00, skew:0.00, jitter:0.00, blush:0.38}),+ HAPPY: F({c:0x5eead4, eye:0.72, brow:-0.16,smile:0.68, pulse:1.25, tilt:0.04, skew:0.00, jitter:0.00, eyeArc:0.90, blush:0.85}),+ CURIOUS: F({c:0xf0abfc, eye:1.22, brow:-0.06,smile:0.28, pulse:1.15, tilt:0.17, skew:0.055,jitter:0.00, blush:0.72}),+ TENSE: F({c:0xfb923c, eye:0.88, brow:0.30, smile:-0.20,pulse:1.95, tilt:0.00, skew:0.02, jitter:0.55, blush:0.26}),+ ANXIOUS: F({c:0xfb7185, eye:1.26, brow:0.42, smile:-0.44,pulse:2.50, tilt:0.00, skew:0.03, jitter:1.00, blush:0.24}),+ CONCERNED: F({c:0xfbbf24, eye:1.10, brow:0.34, smile:-0.34,pulse:1.15, tilt:-0.05,skew:0.00, jitter:0.12, blush:0.32}),+ LISTENING: F({c:0x7dd3fc, eye:1.60, brow:-0.14,smile:0.28, pulse:1.70, tilt:-0.13,skew:0.00, jitter:0.00, blush:0.46, breath:0.055, gape:0.12}),+ HOLDING: F({c:0xfacc15, eye:0.92, brow:0.48, smile:-0.18,pulse:0.72, tilt:0.00, skew:0.00, jitter:0.00, blush:0.3, breath:0.018}),+ THINKING: F({c:0xc084fc, eye:0.40, brow:0.32, smile:0.04, pulse:1.15, tilt:0.15, skew:0.08, jitter:0.00, blush:0.3, breath:0.042}),+ SPEAKING: F({c:0x9db4ff, eye:1.05, brow:0.00, smile:0.32, pulse:1.50, tilt:0.00, skew:0.00, jitter:0.00, blush:0.44}),+ SEARCHING: F({c:0x7dd3fc, eye:0.84, brow:0.10, smile:0.16, pulse:1.60, tilt:0.13, skew:0.05, jitter:0.00, blush:0.34, spread:1.16, breath:0.036}),+ SUCCESS: F({c:0x5eead4, eye:0.50, brow:-0.22,smile:0.86, pulse:1.30, tilt:0.00, skew:0.00, jitter:0.00, eyeArc:1.00, blush:1.0, spread:1.10}),+ FAILED: F({c:0xfb7185, eye:0.68, brow:0.45, smile:-0.50,pulse:1.10, tilt:-0.07,skew:0.02, jitter:0.22, blush:0.42}),+ SLEEPING: F({c:0x9db4ff, eye:0.10, brow:0.06, smile:0.16, pulse:0.45, tilt:0.11, skew:0.00, jitter:0.00, eyeArc:0.85, blush:0.16, spread:0.90, bubble:1}),+};++const MOODS = Object.keys(FACE_NEW);+for(const T of [FACE_OLD, FACE_NEW]) for(const k in T) T[k].color = hex2rgb(T[k].c);+function hex2rgb(h){ return [((h>>16)&255)/255, ((h>>8)&255)/255, (h&255)/255]; }++/**+ * 一份"脾气版"表情表:同样的 14 行,只是眉压得更死、眼缝更窄、笑收敛一点。+ *+ * 暴躁球用它。理由跟 SLEEPING 那一段是同一个道理的反面:一个**状态**如果只靠+ * 某一行的数值来表达,那这个角色平时看起来就不像它自己。暴躁球如果要只在+ * TENSE 那一行才凶,那它一天里大部分时间都是一颗普通的球 —— 而这套皮肤卖的+ * 是"它就是这个脾气"。压过之后,CALM 和 TENSE 仍然差得出来(眉的角度、眼睛+ * 的缝宽都还在动),只是这一整条曲线的**基线**被抬到了"正在忍着"。+ *+ * 注意 copy 出来是浅拷贝:`color` 数组是与 FACE_NEW 共享的引用,但这里没人改+ * 它的内容(只改数值),所以安全。+ */+function tempered(src, mul){+ const out={};+ for(const k in src){+ const o=Object.assign({}, src[k]);+ o.brow = src[k].brow * mul.brow + mul.browAdd;+ o.eye = Math.min(src[k].eye, mul.eyeCap);+ o.smile = src[k].smile * mul.smile;+ out[k]=F(o);+ }+ return out;+}+const FACE_GRUMPY = tempered(FACE_NEW, {brow:1.6, browAdd:0.30, eyeCap:0.78, smile:0.75});++/* =========================================================================+ 5. 预设 —— 每个数都对应 Kotlin 里一个常量+ ========================================================================= */+const P_NOW = {+ key:'now', label:'现在', tag:'近黑球 + 白胶囊眼 + 发光描边',+ bodyBase:[0.102,0.137,0.220], tint:0.20, albedoMul:1.00, grad:0.00,+ amb:1.00, diff:0.00, gloss:0.00, bodyScale:0.885,+ eyeStyle:'plain', eyeColor:[1,1,1], pupilR:0, specR:0,+ eyeR:0.075, eyeH:0.100, eyeX:0.300, eyeY:0.300,+ faceZ:0.860, mouthR:0.240, mouthTube:0.036, mouthY:-0.340, mouthThick:0.00,+ mouthMode:'mood', smileBoost:0.00, faces:FACE_OLD,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.55, blushBlend:'add', blushSize:0.34, blushX:0.440, blushY:-0.120,+ mouthStyle:'arc',+ ears:false, earX:0.360, earY:0.700, earZ:0.360, earH:0.330, earW:0.155, earFlat:0.36, earTilt:0.24, earShade:0.86,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.00, rimPow:2.6,+};++const P_JELLY = {+ key:'jelly', label:'果冻球', tag:'糖果色光泽身体 + 大眼带瞳孔 + 腮红',+ bodyBase:[0.055,0.072,0.130], tint:0.62, albedoMul:0.88, grad:0.60,+ amb:0.66, diff:0.55, gloss:0.70, bodyScale:0.885,+ eyeStyle:'pupil', eyeColor:[1,1,1], pupilR:0.070, specR:0.024,+ eyeR:0.135, eyeH:0.050, eyeX:0.262, eyeY:0.175,+ faceZ:0.876, mouthR:0.175, mouthTube:0.040, mouthY:-0.235, mouthThick:0.06,+ mouthMode:'mood', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.85, blushBlend:'add', blushSize:0.44, blushX:0.400, blushY:-0.145,+ mouthStyle:'arc',+ ears:false, earX:0.360, earY:0.700, earZ:0.360, earH:0.330, earW:0.155, earFlat:0.36, earTilt:0.24, earShade:0.86,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.920,0.950,1.000], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.15, rimPow:2.6,+};++const P_CREAM = {+ key:'cream', label:'奶油球', tag:'奶油身体 + 深色豆豆眼 + 深色描边',+ bodyBase:[0.955,0.965,0.985], tint:0.55, albedoMul:0.93, grad:0.42,+ amb:0.66, diff:0.38, gloss:0.45, bodyScale:0.845,+ eyeStyle:'spec', eyeColor:[0.105,0.135,0.205], pupilR:0, specR:0.040,+ eyeR:0.122, eyeH:0.045, eyeX:0.262, eyeY:0.240,+ faceZ:0.876, mouthR:0.170, mouthTube:0.038, mouthY:-0.230, mouthThick:0.05,+ mouthMode:'deep', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.60,0.66],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.62, blushBlend:'alpha', blushSize:0.48, blushX:0.400, blushY:-0.142,+ mouthStyle:'arc',+ ears:false, earX:0.360, earY:0.700, earZ:0.360, earH:0.330, earW:0.155, earFlat:0.36, earTilt:0.24, earShade:0.86,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'dark', rimGain:1.00, rimPow:5.6,+};++/* --- 猫耳球:同一颗果冻球,加一对耳朵和 ω 嘴 ---+ 耳朵是这张表里唯一"长出来"的几何,也是可爱的最大单笔加成:+ 圆的东西加一对尖角,读出来的立刻是动物而不是物件。 */+const P_KITTY = {+ key:'kitty', label:'猫耳球', tag:'果冻球 + 一对耳朵 + ω 嘴',+ bodyBase:[0.055,0.072,0.130], tint:0.62, albedoMul:0.88, grad:0.60,+ amb:0.66, diff:0.55, gloss:0.70, bodyScale:0.885,+ eyeStyle:'pupil', eyeColor:[1,1,1], pupilR:0.070, specR:0.024,+ eyeR:0.135, eyeH:0.050, eyeX:0.262, eyeY:0.175,+ faceZ:0.876, mouthR:0.120, mouthTube:0.038, mouthY:-0.232, mouthThick:0.10,+ mouthMode:'mood', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.85, blushBlend:'add',+ blushSize:0.44, blushX:0.400, blushY:-0.145,+ mouthStyle:'cat',+ ears:true, earX:0.345, earY:0.690, earZ:0.300, earH:0.340,+ earW:0.165, earFlat:0.34, earTilt:0.26, earShade:0.84,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.15, rimPow:2.6,+};++/* --- 麻薯球:哑光白团子。可爱来自"软",不来自"亮" ---+ 高光降到 0.08 —— 有高光就变成塑料球,没有才像糯米。眼睛是这批里最大的,+ 瞳孔吃掉眼宽 68%,两颗眼几乎贴在一起,嘴巴缩成一个小弧。 */+const P_MOCHI = {+ key:'mochi', label:'麻薯球', tag:'哑光糯米团 · 最大的一双眼睛 · 一块大软腮红',+ bodyBase:[0.985,0.968,0.940], tint:0.38, albedoMul:0.90, grad:0.32,+ amb:0.76, diff:0.26, gloss:0.08, bodyScale:0.885,+ eyeStyle:'spec', eyeColor:[0.125,0.140,0.195], pupilR:0.000, specR:0.052,+ eyeR:0.148, eyeH:0.040, eyeX:0.246, eyeY:0.186,+ faceZ:0.876, mouthR:0.065, mouthTube:0.026, mouthY:-0.205, mouthThick:0.00,+ mouthMode:'deep', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.60,0.66],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.55, blushBlend:'alpha',+ blushSize:0.62, blushX:0.360, blushY:-0.150,+ mouthStyle:'arc',+ ears:false, earX:0.360, earY:0.700, earZ:0.360, earH:0.330,+ earW:0.155, earFlat:0.36, earTilt:0.24, earShade:0.86,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'dark', rimGain:0.55, rimPow:4.5,+};++/* --- 贴纸球:品牌吉祥物的做法 ---+ 没有渐变、没有高光、没有瞳孔高光点,只有一块饱和的平色加一条脆的深描边。+ 描边不是菲涅尔光晕,是"先画一圈放大的实心暗球、再把身体压上去"——+ 光晕是软的,贴纸要的是硬的。 */+const P_STICKER = {+ key:'sticker', label:'贴纸球', tag:'纯平色 + 一条脆的硬描边 · 吉祥物路线',+ bodyBase:[0.175,0.140,0.320], tint:0.78, albedoMul:1.00, grad:0.18,+ amb:0.94, diff:0.10, gloss:0.00, bodyScale:0.845,+ eyeStyle:'pupil', eyeColor:[1,1,1], pupilR:0.062, specR:0.024,+ eyeR:0.132, eyeH:0.046, eyeX:0.258, eyeY:0.176,+ faceZ:0.876, mouthR:0.150, mouthTube:0.038, mouthY:-0.232, mouthThick:0.08,+ mouthMode:'deep', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.52, blushBlend:'alpha',+ blushSize:0.46, blushX:0.395, blushY:-0.150,+ mouthStyle:'arc',+ ears:false, earX:0.360, earY:0.700, earZ:0.360, earH:0.330,+ earW:0.155, earFlat:0.36, earTilt:0.24, earShade:0.86,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.055, outlineColor:[0.070,0.070,0.095],+ rimMode:'dark', rimGain:0.00, rimPow:6.0,+};++/* --- 角落球:可爱来自"不闹" ---+ 它大部分时间就蹲在屏幕角上,所以给它一个最不起眼的脸:哑光、小一圈的眼、+ 淡到几乎没有的描边、几乎没有光。参考的是角落生物那一类——+ 不靠大和亮,靠"缩着"。 */+const P_SUMIKKO = {+ key:'sumikko', label:'角落球', tag:'哑光 · 小一圈的眼 · 几乎不发光',+ bodyBase:[0.135,0.130,0.155], tint:0.30, albedoMul:0.86, grad:0.50,+ amb:0.68, diff:0.45, gloss:0.10, bodyScale:0.885,+ eyeStyle:'pupil', eyeColor:[0.860,0.850,0.815], pupilR:0.048, specR:0.017,+ eyeR:0.098, eyeH:0.048, eyeX:0.196, eyeY:0.168,+ faceZ:0.876, mouthR:0.062, mouthTube:0.026, mouthY:-0.198, mouthThick:0.00,+ mouthMode:'mood', smileBoost:0.00, faces:FACE_NEW,+ blushColor:[1,0.42,0.55],+ blushMul:1.00, restBlush:0.00, blushAlpha:0.45, blushBlend:'add',+ blushSize:0.42, blushX:0.330, blushY:-0.132,+ mouthStyle:'arc',+ ears:false, earX:0.360, earY:0.700, earZ:0.360, earH:0.330,+ earW:0.155, earFlat:0.36, earTilt:0.24, earShade:0.86,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.105,0.135,0.205], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'dark', rimGain:0.45, rimPow:6.0,+};++/* =========================================================================+ 5b. 性格 —— 这一批真正的东西+ =========================================================================+ 上一批(小恶魔/独眼怪/骷髅/熔岩/史莱姆)被否掉,原因不在颜色,在**没有一个+ 部件在做表情**。翻一遍渲染管线就知道:`brow` 这一个数从头到尾只干了两件事 ——+ 把眼球转 0.4 弧度、上下挪一点点。屏幕上从来没有画过一根眉毛。+ 于是不管把身体调成多红、多油、多亮,脸还是同一张脸;而一张不变的脸,换什么+ 颜色都只是"同一只球穿了不同的衣服"。++ 这一批补上三个**部件**,邪恶才有地方住:++ * **眉**(brow)—— 一块可以随手转角的楔子。所有信号里最强的一条。+ * **牙**(fang)—— 同一块楔子缩到牙那么大。一张脸只要露一颗牙,读出来就是+ "它想咬你",而这件事跟颜色、跟身体亮不亮一点关系都没有。+ * **张嘴**(open mouth)—— 一块半圆盘。半环永远是一根**线**,管径加粗也只+ 读成"嘴很厚";要"嘴张着"就必须有一个面。++ 三条一起用,就有了这一页的规矩 ——++ 可爱住在**骨头**上(球径、眼大小、眼高低、眼距、腮红),一个数都不动;+ 邪恶住在**眉、眼、嘴**这三个件上。+ 颜色只回答"它是什么物种",不回答"它坏不坏"。++ 所以下面五套新的骨架**全部等于现在装机那颗奶油球**:眼宽 0.122、眼睛高+ 0.240、眼距 0.262、腮红 0.48 —— 一个都没改。变的全是上面那三个件。+ ========================================================================= */++/* 新增旋钮的**中性值**。+ 全塞在这儿是为了让上面那批老预设(P_NOW/P_JELLY/P_CREAM…)一个字都不用改+ —— Ball.setParams 会把这份默认值合进去。 */+const LOOK_DEFAULTS = {+ // 颜色:accent 是"这套皮肤自己的颜色",accentMix=1 时完全盖掉心情色。+ // 0.4~0.6 是"它有自己的一层色,但还是会随心情变"。+ accent:null, accentMix:1.00,++ // 眉:一块楔子。browTilt 正 = 内低外高(愤怒),负 = 内高外低(委屈)。+ brow:0, browLen:0.145, browThick:0.034, browDepth:0.030,+ browTilt:0.42, browSkew:0.00, browX:0.00, browY:0.180,+ browDeep:0.32, browColor:null, browMood:0.55,++ // 眼:eyeSquash 挤扁眼缝,eyeTilt 正 = 内眼角往下压(凶)。+ eyeSquash:1.00, eyeTilt:0.00, pupilOffX:0.00, pupilOffY:0.00,++ // 牙:fang = 几颗。fangFlip=0 尖朝上(虎牙),1 朝下(獠牙)。+ fang:0, fangLen:0.052, fangThick:0.030, fangX:0.062, fangY:0.004,+ fangFlip:0, fangColor:[0.985,0.985,0.965],++ // 锯齿牙:张嘴时上下各一排。teethRows=1 只有上排,2 = 上下都有。+ teeth:0, teethLen:0.032, teethThick:0.026, teethSpan:0.62, teethRows:1,++ // 张嘴倍数:只对 mouthStyle==='open' 生效。意义是**高/宽比**。+ gawp:0.62,++ // 蒸汽:头顶两缕,复用鼻涕泡那个水滴着色器(白色)。+ // steamY 必须**高过球顶**(bodyScale 0.845):低于它就画在额头上了。+ // 这一项是"看着像热气"和"看着像额头上贴了两块白斑"的全部区别。+ steam:0, steamY:0.90, steamR:0.052,++ // 怒符:四根刺的爆点,画在太阳穴上。+ vein:0, veinX:0.315, veinY:0.400, veinLen:0.055, veinThick:0.024,++ // 嘴角的小勾:坏笑全靠它把嘴角往上带。+ ticks:0, tickLen:0.052, tickThick:0.026,++ mouthStyle:'arc', demoMood:'CALM',+};++/* ---------------------------------------------------------------------+ ① 小恶魔 ★ 主角 —— "很邪恶,但也很可爱"这句话的答案+ --------------------------------------------------------------------- */+/* 它坏在哪:眉(内低外高 0.46 弧度,粗 0.032,离眼睛只差 0.02)、一颗向上的+ 虎牙、以及嘴角的两个小勾。+ 它可爱在哪:**骨架就是奶油球**。眼宽 0.122 没动、眼睛还是那么低、腮红 0.5+ 还在、身体还是圆的、还长了一对短短的圆角。+ 所以它读出来是一颗"表情很坏的奶油球",不是一颗"红色的怪球"。+ 颜色刻意**不用血红**:红是物种色,不是邪恶开关。血红身体在浅色 App 上会糊成+ 一团深色,而且就没有"从眉毛底下往上看"那个对比了。身体近黑紫 + 暖红描边光+ —— 像一块炭,红只出现在它发光的地方。 */+const P_IMP = {+ key:'imp', label:'小恶魔 ★', tag:'斜眉 + 虎牙 + 坏笑 · 骨架和奶油球一样',+ star:true, demoMood:'CALM',+ accent:[1.00,0.34,0.26], accentMix:0.42,+ bodyBase:[0.085,0.058,0.092], tint:0.58, albedoMul:1.00, grad:0.58,+ amb:0.60, diff:0.50, gloss:0.88, bodyScale:0.845,+ eyeStyle:'pupil', eyeColor:[1.00,0.975,0.940], pupilR:0.052, specR:0.019,+ eyeR:0.122, eyeH:0.045, eyeX:0.262, eyeY:0.240,+ faceZ:0.876, mouthR:0.185, mouthTube:0.036, mouthY:-0.238, mouthThick:0.03,+ mouthMode:'deep', smileBoost:0.22, faces:FACE_NEW, mouthStyle:'arc',+ // 眉:压到几乎贴着内眼角+ brow:1, browTilt:0.46, browSkew:0.10, browLen:0.150, browThick:0.032,+ browY:0.176, browDeep:0.28, browMood:0.45,+ // 眼:跟着眉往里下角斜一点;瞳孔往上挪 —— "从眉毛底下往上看"+ eyeSquash:0.86, eyeTilt:0.10, pupilOffY:0.16, pupilOffX:0.04,+ // 牙:一颗,在嘴的右侧+ fang:1, fangLen:0.050, fangThick:0.028, fangX:0.072, fangY:0.006,+ fangFlip:0, ticks:1, tickLen:0.046, tickThick:0.024,+ // 角:复用猫耳那个锥体,短、圆、往外撇+ ears:true, earX:0.190, earY:0.670, earZ:0.280, earH:0.300,+ earW:0.140, earFlat:0.66, earTilt:0.42, earShade:0.58,+ blushColor:[1.00,0.34,0.40], blushMul:1.00, restBlush:0.00, blushAlpha:0.80,+ blushBlend:'add', blushSize:0.50, blushX:0.395, blushY:-0.150,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.40, rimPow:2.30,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[1,1,1], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+};++/* ---------------------------------------------------------------------+ ② 暴躁球 —— 烈子那条路:脸红着发火+ --------------------------------------------------------------------- */+/* 暴躁不是一种"颜色",是一种**眼和嘴的形状**,所以这套全部押在三个件上:+ 两撇压到底的粗眉(0.66 弧度,几乎连成一条)、把眼睛横向挤扁到 0.42、一张+ **张开的嘴** + 上下各四颗锯齿牙;再加头顶两缕蒸汽、眼角一个怒符。+ 可爱靠的是那两块腮红**没有关掉**(0.80,全批最高)—— 这正是"怒念的烈子"+ 的全部秘密:一张脸如果连血色都气没了,那是恐怖片,不是可爱。+ 它还有自己的表情表 [FACE_GRUMPY]:同样 14 行,眉压得更死、眼缝更窄 ——+ 所以它在**每一个**心情下都带着脾气,而 TENSE / ANXIOUS 还是比 CALM 凶。 */+const P_GRUMPY = {+ key:'grumpy', label:'暴躁球', tag:'张嘴锯齿牙 + 头顶冒烟 + 怒符 · 腮红照开',+ demoMood:'TENSE',+ accent:[1.00,0.44,0.22], accentMix:0.45,+ bodyBase:[0.150,0.122,0.128], tint:0.70, albedoMul:0.98, grad:0.52,+ amb:0.66, diff:0.42, gloss:0.55, bodyScale:0.815,+ eyeStyle:'pupil', eyeColor:[1.00,0.970,0.930], pupilR:0.086, specR:0.018,+ eyeR:0.132, eyeH:0.010, eyeX:0.250, eyeY:0.252,+ faceZ:0.876, mouthR:0.158, mouthTube:0.040, mouthY:-0.252, mouthThick:0.02,+ mouthMode:'deep', smileBoost:0.00, faces:FACE_GRUMPY, mouthStyle:'open',+ gawp:0.72,+ brow:1, browTilt:0.66, browSkew:0.00, browLen:0.148, browThick:0.040,+ browY:0.166, browDeep:0.24, browMood:0.25,+ eyeSquash:0.42, eyeTilt:0.16, pupilOffY:-0.06,+ teeth:4, teethLen:0.038, teethThick:0.027, teethSpan:0.58, teethRows:2,+ steam:1.00, steamY:0.90, steamR:0.050,+ vein:1.00, veinX:0.318, veinY:0.398, veinLen:0.050, veinThick:0.022,+ ears:false, earX:0.360, earY:0.700, earZ:0.360, earH:0.330,+ earW:0.155, earFlat:0.36, earTilt:0.24, earShade:0.86,+ blushColor:[1.00,0.28,0.26], blushMul:1.00, restBlush:0.00, blushAlpha:0.80,+ blushBlend:'add', blushSize:0.58, blushX:0.372, blushY:-0.158,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.30, rimPow:2.60,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[1,1,1], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+};++/* ---------------------------------------------------------------------+ ③ 腹黑球 —— 最省的一套,也是最"不怀好意"的一套+ --------------------------------------------------------------------- */+/* 它没有獠牙、没有角、没有蒸汽,只有**不对称**:一条眉挑起来、另一条压下去+ (browSkew 0.62 —— 左右两条眉的角度差 1.24 弧度),一只眼半闭,嘴角单边翘,+ 瞳孔偏到眼角(看旁边,不看你)。+ 不对称之所以读作"心机"而不是"画歪了":一条脸上的所有其他元素都是严格左右+ 对称的,唯一的例外就是有意为之。 */+const P_SMUG = {+ key:'smug', label:'腹黑球', tag:'一高一低的眉 + 单边翘的嘴 · 只有不对称',+ demoMood:'CURIOUS',+ accent:[0.74,0.64,1.00], accentMix:0.42,+ bodyBase:[0.118,0.102,0.158], tint:0.60, albedoMul:0.98, grad:0.56,+ amb:0.62, diff:0.48, gloss:0.72, bodyScale:0.845,+ eyeStyle:'pupil', eyeColor:[0.980,0.975,1.000], pupilR:0.058, specR:0.020,+ eyeR:0.126, eyeH:0.040, eyeX:0.268, eyeY:0.248,+ faceZ:0.876, mouthR:0.160, mouthTube:0.034, mouthY:-0.244, mouthThick:0.04,+ mouthMode:'deep', smileBoost:0.30, faces:FACE_NEW, mouthStyle:'arc',+ brow:1, browTilt:0.20, browSkew:0.62, browLen:0.140, browThick:0.030,+ browY:0.178, browDeep:0.32, browMood:0.30,+ eyeSquash:0.74, eyeTilt:0.00, pupilOffX:0.34, pupilOffY:-0.08,+ ticks:1, tickLen:0.050, tickThick:0.024,+ ears:false, earX:0.360, earY:0.700, earZ:0.360, earH:0.330,+ earW:0.155, earFlat:0.36, earTilt:0.24, earShade:0.86,+ blushColor:[0.80,0.42,1.00], blushMul:1.00, restBlush:0.00, blushAlpha:0.70,+ blushBlend:'add', blushSize:0.46, blushX:0.400, blushY:-0.148,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.05, rimPow:2.80,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[1,1,1], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+};++/* ---------------------------------------------------------------------+ ④ 吸血鬼球 —— 牙在外面,血在脸上+ --------------------------------------------------------------------- */+/* 两颗从上唇**垂下来**的獠牙(fangFlip 1),加上一张苍白身体上的两块红腮 ——+ 苍白给了腮红一个存在的理由,腮红又把可爱度还回来。+ 眉只是浅浅两撇(0.22):它不凶,它只是"有教养地坏"。+ 描边走**深红**而不是发光:深色描边压在浅色身体上,剪影是硬的,像一张贴纸。 */+const P_VAMP = {+ key:'vamp', label:'吸血鬼球', tag:'两颗下垂的獠牙 + 苍白身体 + 一点血色',+ demoMood:'CALM',+ accent:[0.62,0.06,0.14], accentMix:0.70,+ bodyBase:[0.930,0.910,0.925], tint:0.30, albedoMul:0.96, grad:0.40,+ amb:0.70, diff:0.30, gloss:0.78, bodyScale:0.845,+ eyeStyle:'pupil', eyeColor:[0.600,0.055,0.105], pupilR:0.062, specR:0.020,+ eyeR:0.126, eyeH:0.042, eyeX:0.262, eyeY:0.238,+ faceZ:0.876, mouthR:0.165, mouthTube:0.032, mouthY:-0.236, mouthThick:0.03,+ mouthMode:'deep', smileBoost:0.10, faces:FACE_NEW, mouthStyle:'arc',+ brow:1, browTilt:0.22, browSkew:0.00, browLen:0.145, browThick:0.028,+ browY:0.178, browDeep:0.30, browMood:0.40,+ eyeSquash:0.94, eyeTilt:0.05, pupilOffY:0.05,+ fang:2, fangLen:0.062, fangThick:0.030, fangX:0.074, fangY:-0.004, fangFlip:1,+ ears:false, earX:0.360, earY:0.700, earZ:0.360, earH:0.330,+ earW:0.155, earFlat:0.36, earTilt:0.24, earShade:0.86,+ blushColor:[1.00,0.16,0.22], blushMul:1.00, restBlush:0.00, blushAlpha:0.62,+ blushBlend:'alpha', blushSize:0.46, blushX:0.395, blushY:-0.150,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'dark', rimGain:1.10, rimPow:5.00,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[0.600,0.055,0.105], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+};++/* ---------------------------------------------------------------------+ ⑤ 猫妖 —— 用最可爱的一张脸,干最坏的事+ --------------------------------------------------------------------- */+/* 这一套把可爱度顶到最大(全批最大的一双眼睛,眼宽 0.145),然后让它坏:+ 猫耳**往后压**(earZ 0.20 / earTilt 0.36,耳朵一后压就是要扑人了)、ω 嘴的+ 两个凹口里各塞一颗向上的小虎牙、眉只有 0.26 的浅斜。+ 它说明的是这批设计里最反直觉的一条:**可爱度是弹药,不是跟邪恶抢位置的东西**。+ 眼睛越大、身体越圆,那两颗牙就越能咬到人。 */+const P_NEKO = {+ key:'neko', label:'猫妖', tag:'最大的一双眼睛 + 后压的猫耳 + 两颗小虎牙',+ demoMood:'HAPPY',+ accent:[0.64,0.86,1.00], accentMix:0.38,+ bodyBase:[0.118,0.118,0.168], tint:0.58, albedoMul:0.98, grad:0.56,+ amb:0.62, diff:0.48, gloss:0.82, bodyScale:0.845,+ eyeStyle:'pupil', eyeColor:[0.990,1.000,0.990], pupilR:0.058, specR:0.024,+ eyeR:0.145, eyeH:0.030, eyeX:0.252, eyeY:0.212,+ faceZ:0.876, mouthR:0.120, mouthTube:0.034, mouthY:-0.222, mouthThick:0.09,+ mouthMode:'deep', smileBoost:0.20, faces:FACE_NEW, mouthStyle:'cat',+ brow:1, browTilt:0.26, browSkew:0.05, browLen:0.135, browThick:0.028,+ browY:0.176, browDeep:0.32, browMood:0.50,+ eyeSquash:1.00, eyeTilt:0.06, pupilOffY:0.04,+ fang:2, fangLen:0.040, fangThick:0.024, fangX:0.064, fangY:-0.002, fangFlip:0,+ ears:true, earX:0.330, earY:0.640, earZ:0.200, earH:0.330,+ earW:0.170, earFlat:0.32, earTilt:0.36, earShade:0.86,+ blushColor:[1.00,0.56,0.64], blushMul:1.00, restBlush:0.00, blushAlpha:0.66,+ blushBlend:'alpha', blushSize:0.50, blushX:0.392, blushY:-0.146,+ outline:0.000, outlineColor:[0.085,0.085,0.115],+ rimMode:'glow', rimGain:1.15, rimPow:2.60,+ bubbleX:0.085, bubbleY:-0.030, bubbleR:0.085,+ bubbleColor:[1,1,1], bubblePow:3.0, bubbleShine:26, bubbleInner:0.10,+};++/* --- ⑥ 奶油球 · 已经在你手上跑着,留作基线 ---+ 六格里唯一"零个新部件"的一张:没有眉、没有牙、嘴是那条弧。+ 把它放在最后,是为了让上面五套的差别看得见出处:同一具骨架,+ 加眉 / 加牙 / 换嘴,各自多出多少"坏"。 */+const P_CREAM_BASE = Object.assign({}, P_CREAM, {+ label:'奶油球 · 已落地', tag:'基线:没有眉、没有牙、嘴是一条弧',+ demoMood:'CALM', star:false,+});++const PRESETS = [P_IMP, P_GRUMPY, P_SMUG, P_VAMP, P_NEKO, P_CREAM_BASE];+const clone = o => JSON.parse(JSON.stringify(o));++/* =========================================================================+ 6. Shader+ ========================================================================= */+const VS_SOLID = `+ uniform mat4 uMVP;+ attribute vec3 aPos;+ void main(){ gl_Position = uMVP * vec4(aPos,1.0); }+`;+const FS_SOLID = `+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ void main(){ gl_FragColor = vec4(uColor, uAlpha); }+`;++/* 身体:顶光渐变 + 主光 + 镜面高光。这是"果冻感"的全部来源。 */+const VS_BODY = `+ uniform mat4 uMVP; uniform mat4 uMV;+ attribute vec3 aPos; attribute vec3 aNormal;+ varying vec3 vN; varying vec3 vP;+ void main(){+ vN = normalize(mat3(uMV) * aNormal);+ vec4 p = uMV * vec4(aPos,1.0);+ vP = p.xyz;+ gl_Position = uMVP * vec4(aPos,1.0);+ }+`;+const FS_BODY = `+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ uniform float uGrad;+ uniform float uAmb;+ uniform float uDiff;+ uniform float uGloss;+ varying vec3 vN; varying vec3 vP;+ void main(){+ vec3 n = normalize(vN);+ vec3 V = normalize(-vP);+ float vert = n.y * 0.5 + 0.5;+ vec3 base = uColor * mix(1.0, mix(0.70, 1.14, vert), uGrad);+ // 主光偏正上方。高光点落在额头 —— 压在眼睛上等于把眼睛的对比度+ // 吃掉一半,而且 0.95 的强度会在 80dp 的球上糊出一块 30px 的白斑。+ vec3 L = normalize(vec3(-0.48, 0.85, 0.20));+ float diff = max(0.0, dot(n, L));+ vec3 lit = base * (uAmb + uDiff * diff);+ vec3 H = normalize(L + V);+ float spec = pow(max(0.0, dot(n, H)), 90.0) * uGloss;+ float edge = pow(1.0 - max(0.0, dot(n, V)), 3.0) * uGloss * 0.26;+ gl_FragColor = vec4(lit + vec3(spec) + uColor * edge, uAlpha);+ }+`;++/* 描边 / 光晕。uPow 小 → 柔和发光;大 → 紧的深色轮廓。+ 这一段必须与 EmotionBallRenderer.RIM_FS **逐字相同** ——+ 上一版这里少写了一条扫描光带,结果真机上出现了一条实验室永远看不见的+ 滚动色带,用户先发现。实验室的价值全在于它和真机一样。 */+const VS_RIM = `+ uniform mat4 uMVP; uniform mat4 uMV;+ attribute vec3 aPos; attribute vec3 aNormal;+ varying vec3 vN; varying vec3 vP; varying float vY;+ void main(){+ vN = normalize(mat3(uMV) * aNormal);+ vec4 p = uMV * vec4(aPos,1.0);+ vP = p.xyz; vY = aPos.y;+ gl_Position = uMVP * vec4(aPos,1.0);+ }+`;+const FS_RIM = `+ precision mediump float;+ uniform vec3 uColor;+ uniform float uIntensity;+ uniform float uScan;+ uniform float uScanY;+ uniform float uPow;+ uniform float uCap;+ varying vec3 vN; varying vec3 vP; varying float vY;+ void main(){+ float f = pow(1.0 - abs(dot(normalize(vN), normalize(-vP))), uPow);+ float d = vY - uScanY;+ float slope = d > 0.0 ? 7.5 : 4.2;+ float band = pow(max(0.0, 1.0 - abs(d) * slope), 3.0);+ gl_FragColor = vec4(uColor, min(f * uIntensity + band * uScan, uCap));+ }+`;++const VS_PTS = `+ uniform mat4 uMVP; uniform mat4 uMV;+ uniform float uSize; uniform float uPointFactor;+ attribute vec3 aPos; attribute vec3 aColor;+ varying vec3 vColor;+ void main(){+ vec4 mvpos = uMV * vec4(aPos,1.0);+ gl_Position = uMVP * vec4(aPos,1.0);+ gl_PointSize = uSize * uPointFactor / -mvpos.z;+ vColor = aColor;+ }+`;+const FS_PTS = `+ precision mediump float;+ uniform float uAlpha;+ varying vec3 vColor;+ void main(){+ vec2 c = gl_PointCoord - 0.5;+ float d = length(c);+ float a = smoothstep(0.5, 0.0, d);+ gl_FragColor = vec4(vColor, a * uAlpha);+ }+`;++/* =========================================================================+ 7. EmotionState 移植+ ========================================================================= */+const ACTION_SECONDS = {NONE:0, HOP:0.52, NOD:0.42, SHAKE:0.5, POP:0.46, RECOIL:0.55,+ STRETCH:1.6, LOOK_AROUND:1.9, WIGGLE:1.1, BOUNCE:0.9};++class State{+ constructor(){+ this.nowColor = FACE_OLD.CALM.color.slice();+ this.eye=1; this.brow=0; this.smile=0.1; this.tilt=0; this.skew=0; this.jitter=0;+ this.open=0; this.eyeArc=0; this.blush=0; this.spread=1; this.lid=1;+ // 鼻涕泡:amount 是随表情淡入淡出,scale 是它自己的一呼一吸+ this.bubble=0; this.bubbleScale=1; this.bubblePhase=0;+ this.think=0; this.busy=0;+ this.rotX=0; this.rotY=0; this.rotZ=0; this.breathe=1;+ this.breatheDepth=0.022; this.scaleX=1; this.scaleY=1; this.hopY=0;+ this.eyeOffX=0; this.eyeOffY=0; this.eyeLRotZ=0; this.eyeRRotZ=0; this.eyeYScale=1;+ this.mouthY=-0.34; this.mouthScaleY=0.16; this.mouthRotZ=Math.PI;+ this.blinkNext=2.4; this.blinkLeft=0;+ // 蒸汽 / 怒符 / 眉的小抖 —— 三个"活着的"相位。静止的怒符看起来像贴纸,+ // 会跳一下的才像在发火。+ this.steamPhase=0; this.veinBeat=0; this.browTwitch=0; this.t=0;+ // 扫描光带:一条横线从下往上扫,一轮约 5.3 秒。think/listen 会让它变明显。+ this.scanAngle=0; this.scanY=-2; this.scan=0; this.listen=0;+ this.action='NONE'; this.actionT=0; this.actionDur=0;+ }+ trigger(a){ if(!a||a==='NONE') return; this.action=a; this.actionT=0; this.actionDur=ACTION_SECONDS[a]; }++ tick(dt,t,mood,p){+ const want=p.faces[mood], k=Math.min(1,dt*7);+ this.eye+=(want.eye-this.eye)*k;+ this.brow+=(want.brow-this.brow)*k;+ this.smile+=(want.smile-this.smile)*k;+ this.tilt+=(want.tilt-this.tilt)*k;+ this.skew+=(want.skew-this.skew)*k;+ this.jitter+=(want.jitter-this.jitter)*Math.min(1,dt*2.4);+ this.eyeArc+=(want.eyeArc-this.eyeArc)*k;+ this.blush+=(want.blush-this.blush)*k;+ this.spread+=(want.spread-this.spread)*Math.min(1,dt*3.5);+ for(let i=0;i<3;i++) this.nowColor[i]+=(want.color[i]-this.nowColor[i])*k;++ this.think+=(((mood==='THINKING'||mood==='SEARCHING')?1:0)-this.think)*Math.min(1,dt*2.2);+ const th=this.think;+ this.busy=th;++ // listen 在真机上是语音管线给的;这里用 LISTENING 表情近似+ this.listen += (((mood==='LISTENING')?1:0)-this.listen)*Math.min(1,dt*3.4);+ this.scanAngle += dt*(0.19 + th*0.55);+ this.scanY = -1.35 + (this.scanAngle - Math.floor(this.scanAngle))*2.7;+ this.scan = 0.08 + th*0.5 + this.listen*0.42 + this.open*0.12;++ const gaze = th*(Math.sin(t*0.9)+Math.sin(t*1.31)*0.5);+ const gazeY= th*(Math.sin(t*0.73)*0.6+Math.sin(t*1.07)*0.35);+ const jx=(Math.sin(t*23.7)+Math.sin(t*37.1)*0.6)*this.jitter;+ const jy=(Math.sin(t*29.3)+Math.sin(t*41.7)*0.6)*this.jitter;++ const talk = (mood==='SPEAKING') ? 0.34+Math.sin(t*17)*0.16+Math.sin(t*29)*0.09 : want.gape;+ this.open+=(Math.max(0,talk)-this.open)*Math.min(1,dt*22);++ // 眨眼是"给睁着的眼睛合上眼皮"。屏幕上那只眼睛已经是 `^ ^` 弧的时候+ // (eyeArc ≥ ARC_EYE_AT)没有眼皮可合,眨一下只是把弧抽动一下 ——+ // 睡着的球每三秒抖一次就是这么来的。+ const eyeShut=this.eyeArc>=ARC_EYE_AT;+ this.blinkNext-=dt;+ if(this.blinkNext<=0){+ if(eyeShut){+ // 顺带取消正在进行的那一次,并稍后重试:醒来后第一次眨眼+ // 不用等满一整轮+ this.blinkLeft=0; this.blinkNext=0.35;+ } else {+ this.blinkLeft=0.12;+ const base=(this.listen>0.5?1.8:3.2)+Math.random()*2.6;+ this.blinkNext=(base*(1+th*0.9))/(1+this.jitter*1.6);+ }+ }+ let lid=1;+ if(this.blinkLeft>0){ this.blinkLeft-=dt; lid=Math.abs(this.blinkLeft/0.06-1); }+ this.lid=lid;++ let sqx=0,sqy=0,hop=0,actRX=0,actRY=0;+ if(this.action!=='NONE'){+ this.actionT+=dt;+ const pp=Math.min(1,this.actionT/this.actionDur);+ const arch=Math.sin(pp*Math.PI);+ switch(this.action){+ case 'HOP':{ hop=arch*0.22;+ const c=Math.max(0,1-pp/0.2)+Math.max(0,(pp-0.8)/0.2);+ sqx=c*0.13; sqy=c*-0.17; break; }+ case 'NOD': actRX=Math.sin(pp*2*Math.PI)*0.26; break;+ case 'SHAKE': actRY=Math.sin(pp*3*Math.PI)*0.20; break;+ case 'POP': sqx=arch*0.15; sqy=arch*0.15; hop=arch*0.06; break;+ case 'RECOIL': sqx=arch*0.17; sqy=arch*-0.20; actRX=-arch*0.14; break;+ case 'STRETCH':{ const e=Math.sin(pp*Math.PI);+ sqx=-e*0.12; sqy=e*0.20; actRX=-e*0.08; break; }+ case 'LOOK_AROUND': actRY=Math.sin(pp*2.5*Math.PI)*0.30; actRX=Math.sin(pp*1.25*Math.PI)*0.05; break;+ case 'WIGGLE': actRY=Math.sin(pp*3*Math.PI)*0.14; break;+ case 'BOUNCE':{ hop=Math.abs(Math.sin(pp*2*Math.PI))*0.09;+ const c=Math.max(0,1-pp/0.15)+Math.max(0,(pp-0.85)/0.15);+ sqx=c*0.07; sqy=c*-0.09; break; }+ }+ if(pp>=1) this.action='NONE';+ }++ this.breatheDepth+=(want.breath-this.breatheDepth)*Math.min(1,dt*3);+ this.breathe=1+Math.sin(t*want.pulse)*this.breatheDepth+this.open*0.03;+ this.scaleX=this.breathe*(1+sqx);+ this.scaleY=this.breathe*(1+sqy);+ this.hopY=hop;+ this.rotY=Math.sin(t*0.22)*0.12+jx*0.012+gaze*0.09+actRY;+ this.rotX=Math.sin(t*0.17)*0.06+jy*0.009-gazeY*0.05+actRX;+ this.rotZ=this.tilt+th*Math.sin(t*0.51)*0.09;++ const browR=this.brow+this.skew, browL=this.brow;+ this.t=t;+ // 眉和眼的位置。eyeTilt 是 look 自己的:正 = 内眼角往下压(凶),+ // 和 brow 那 0.40 弧度叠加。眼神留白在这里:`gaze` 只在思考时才有,+ // 所以"看旁边"这件事必须由 look 的 pupilOffX 去说。+ this.eyeOffX=jx*0.014+gaze*0.075;+ this.eyeOffY=-browL*0.06+gazeY*0.05;+ this.eyeLRotZ=-browL*0.40 + p.eyeTilt;+ this.eyeRRotZ= browR*0.40 - p.eyeTilt;++ const sm=this.smile+p.smileBoost;+ // eyeSquash:look 自己的挤眼倍数。捏扁眼缝是"凶"里最有效的一档 ——+ // 它改的是**眼的形状**,而形状在 168dp 的窗口里比颜色活得久。+ this.eyeYScale=Math.max(0.06, this.eye*this.lid*p.eyeSquash - sm*0.34);++ // 三个相位。蒸汽慢(2.6s 一轮),怒符跳(0.55s/次,像心跳),+ // 眉的小抖最快也最轻 —— 它只是让眉不像贴上去的。+ this.steamPhase+=dt*0.62;+ this.veinBeat +=dt*1.85;+ this.browTwitch+=dt*1.35;++ // 鼻涕泡。相位**故意不挂在呼吸上**:SLEEPING 的 pulse 是 0.45,+ // 挂上去一轮 14 秒 —— 一个 14 秒不动的泡等于没有在动。+ // 速率仍然跟着 pulse 走,"睡得越沉、泡越慢"这层关系还在。+ this.bubble+=((want.bubble||0)-this.bubble)*Math.min(1,dt*2.2);+ this.bubblePhase+=dt*(0.9+want.pulse*1.6);+ this.bubbleScale=0.30+0.70*(0.5+0.5*Math.sin(this.bubblePhase));+ this.mouthScaleY=0.16+this.open*1.5+Math.abs(sm)*0.5+p.mouthThick;+ this.mouthY=p.mouthY+sm*0.06;+ this.mouthRotZ=Math.PI+sm*0.9+this.skew*2.2;+ }+}++/* =========================================================================+ 8. Ball+ ========================================================================= */+const CAM_Z=4.9, RIM=0.9;+const EYE_LID_PIVOT=-0.125;+/* 眼球胶囊在 z 上也有半径,瞳孔要画在它前面 */+const EYE_FRONT=0.16;+const EYE_ARC_SPAN=0.8*Math.PI, EYE_ARC_ROT_Z=Math.PI/2-EYE_ARC_SPAN/2, EYE_ARC_DROP=0.06;+/* 超过这个值,屏幕上的眼睛就是那条 `^ ^` 弧,圆眼不是主角了。+ 两处共用:还要不要画瞳孔、还要不要眨眼。 */+const ARC_EYE_AT=0.5;++class Ball{+ constructor(canvas,params){+ this.cv=canvas;+ this.gl=canvas.getContext('webgl',{alpha:true,antialias:true,premultipliedAlpha:true,depth:true});+ if(!this.gl) throw new Error('这台机器没有 WebGL');+ const gl=this.gl;+ // 默认就停在 SLEEPING:这一页现在的主题是睡着那颗球+ this.mood='SLEEPING';+ this.st=new State();+ this.autoFidget=true;++ this.pSolid=new Prog(gl,VS_SOLID,FS_SOLID);+/* 鼻涕泡。一个透明的水滴,不是一颗白球:+ * 背面 discard —— 只留朝向相机的那半,否则前后面叠在一起,边缘会变成两倍厚;+ * edge 是菲涅尔,在轮廓上收紧成一圈(水滴的边是折射,浅色球上用暗色边);+ * spec 是一个小高光点,落在左上,和球的主光同一个方向;+ * uInner 是内部那层很薄的雾 —— 0 就完全透明。+ uColor 按 look 给:浅色球用深色边,深色球用亮边。 */+const VS_BUBBLE = `+ uniform mat4 uMVP; uniform mat4 uMV;+ attribute vec3 aPos; attribute vec3 aNormal;+ varying vec3 vN; varying vec3 vP;+ void main(){+ vN = normalize(mat3(uMV) * aNormal);+ vec4 mvpos = uMV * vec4(aPos,1.0);+ vP = mvpos.xyz;+ gl_Position = uMVP * vec4(aPos,1.0);+ }+`;+const FS_BUBBLE = `+ precision mediump float;+ uniform vec3 uColor;+ uniform float uAlpha;+ uniform float uPow;+ uniform float uShine;+ uniform float uInner;+ varying vec3 vN; varying vec3 vP;+ void main(){+ vec3 n = normalize(vN);+ vec3 V = normalize(-vP);+ float facing = dot(n, V);+ if (facing <= 0.0) discard;+ float edge = pow(1.0 - facing, uPow);+ vec3 L = normalize(vec3(-0.45, 0.80, 0.60));+ vec3 H = normalize(L + V);+ float spec = pow(max(0.0, dot(n, H)), uShine);+ float a = clamp(edge + uInner + spec, 0.0, 1.0);+ vec3 c = mix(uColor, vec3(1.0), clamp(spec * 2.5, 0.0, 1.0));+ gl_FragColor = vec4(c, a * uAlpha);+ }+`;++ this.pBody =new Prog(gl,VS_BODY ,FS_BODY);+ this.pRim =new Prog(gl,VS_RIM ,FS_RIM);+ this.pBubble=new Prog(gl,VS_BUBBLE,FS_BUBBLE);+ this.pPts =new Prog(gl,VS_PTS ,FS_PTS);++ this.mBody=new GMesh(gl,icoSphere(4));+ this.mRim =new GMesh(gl,uvSphere(40,28));+ this.mDot =new GMesh(gl,icoSphere(2));+ this.mArc =new GMesh(gl,torus(0.1,0.03,6,20,EYE_ARC_SPAN));+ this.mBlush=new GPoints(gl,2);+ // 性格那一批的三块新几何。三角形数和球体比可以忽略:+ // 楔子 14 段(眉/尖牙/锯齿牙/嘴角勾/怒符都是它),半圆盘 28 段(张嘴)。+ this.mWedge=new GMesh(gl,centeredCone(0.5,0.5,1.0,14));+ this.mOpen =new GMesh(gl,halfDisc(1.0,28));++ this.proj=M4.id(); this.view=M4.lookAt(0,0,CAM_Z,0,0,0,0,1,0);+ this.pointFactor=400;+ this.setParams(params);++ this.fidNext=9+Math.random()*8;+ this.resize();+ }+ setParams(params){+ const changedSame = this.p && this.p.eyeR===params.eyeR &&+ this.p.eyeH===params.eyeH && this.p.mouthR===params.mouthR &&+ this.p.mouthTube===params.mouthTube;+ // 合并中性值:老预设(P_NOW/P_JELLY/P_CREAM…)里没有性格那一批的字段,+ // 靠这一行才不用去改它们。已给出的字段照旧覆盖默认值。+ this.p=Object.assign({}, LOOK_DEFAULTS, clone(params));+ if(!changedSame || !this.mEye){+ this.mEye=new GMesh(this.gl,capsule(this.p.eyeR,this.p.eyeH,4,12));+ this.mMouth=new GMesh(this.gl,torus(this.p.mouthR,this.p.mouthTube,8,24,Math.PI));+ // 耳朵按预设的真实比例建,比例变了就重建 —— 这样绘制时只做旋转平移,+ // 法线不会被非等比缩放搞坏(法线要的是逆转置,普通缩放是错的)。+ this.mEar=new GMesh(this.gl,+ cone(this.p.earW, this.p.earW*this.p.earFlat, this.p.earH, 20));+ }+ }+ resize(){+ const dpr=Math.min(2,window.devicePixelRatio||1);+ const w=Math.max(1,Math.round(this.cv.clientWidth*dpr));+ const h=Math.max(1,Math.round(this.cv.clientHeight*dpr));+ if(this.cv.width!==w||this.cv.height!==h){ this.cv.width=w; this.cv.height=h; }+ const gl=this.gl;+ gl.viewport(0,0,this.cv.width,this.cv.height);+ this.proj=M4.perspective(42,this.cv.width/this.cv.height,0.1,100);+ this.pointFactor=this.cv.height/(2*Math.tan(42*Math.PI/180/2));+ }++ frame(dt,t){+ const gl=this.gl, p=this.p;+ this.st.tick(dt,t,this.mood,p);++ if(this.autoFidget && this.st.action==='NONE'){+ this.fidNext-=dt;+ if(this.fidNext<=0){+ const pool=['STRETCH','LOOK_AROUND','WIGGLE','BOUNCE','HOP'];+ this.st.trigger(pool[Math.floor(Math.random()*pool.length)]);+ this.fidNext=14+Math.random()*22;+ }+ }++ gl.clearColor(0,0,0,0);+ gl.clear(gl.COLOR_BUFFER_BIT|gl.DEPTH_BUFFER_BIT);+ gl.enable(gl.DEPTH_TEST);+ gl.enable(gl.BLEND);++ let root=M4.id();+ root=M4.translate(root,0,this.st.hopY,0);+ root=M4.rotate(root,this.st.rotX*57.29578,1,0,0);+ root=M4.rotate(root,this.st.rotY*57.29578,0,1,0);+ root=M4.rotate(root,this.st.rotZ*57.29578,0,0,1);+ root=M4.scale(root,this.st.scaleX,this.st.scaleY,this.st.scaleX);+ this.root=root;+ this.face=M4.translate(root,0,0,p.faceZ);++ // 每套 look 可以带自己的**强调色**。表情表给的是"这一颗球现在什么心情"的+ // 颜色(CALM 是淡蓝),身体上色、嘴、发光的描边、瞳孔都从它派生 —— 所以+ // 想要一颗血红的恶魔,光调 look 是调不出来的:颜色不是 look 的。+ //+ // accentMix 是这一版的修正:上一版 accent 是全量覆盖,于是暴躁球在+ // TENSE(橙)和 CALM(蓝)下长得一模一样 —— 等于把"心情"这条通道关了。+ // 0.4 左右的意思是"它有自己的一层色,但心情还透得出来"。+ // 落进 Kotlin 就是 BallLook 多一个可空 accent(不动一个字段也能活)。+ let c=this.st.nowColor;+ if(p.accent){+ c=[c[0]+(p.accent[0]-c[0])*p.accentMix,+ c[1]+(p.accent[1]-c[1])*p.accentMix,+ c[2]+(p.accent[2]-c[2])*p.accentMix];+ }++ /* ---- 1. 身体 ---- */+ const albedo=[0,0,0];+ for(let i=0;i<3;i++) albedo[i]=(p.bodyBase[i]+(c[i]-p.bodyBase[i])*p.tint)*p.albedoMul;+ this.albedo=albedo;++ // 硬描边:先画一圈放大的实心暗球,再让身体压上去,中间留一条边。+ // 关键在 depthMask(false) —— 描边球比身体大,整个包在身体外面,+ // 如果它写了深度,后面画的身体深度更大,会被它自己那圈描边挡掉,+ // 结果是一颗纯黑的球。不写深度,它就成了身体后面的一层衬底。+ if(p.outline>0.001){+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.depthMask(false);+ const os=p.bodyScale+p.outline;+ const om=M4.scale(root,os,os,os);+ const omv=M4.mul(this.view,om), omvp=M4.mul(this.proj,omv);+ this.pSolid.use();+ gl.uniform1f(this.pSolid.u('uAlpha'),1);+ gl.uniform3fv(this.pSolid.u('uColor'),p.outlineColor);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,omvp);+ this.mBody.bind(this.pSolid.a('aPos'));+ this.mBody.draw();+ if(p.ears) this.drawEars(1+p.outline/p.earW, p.outlineColor);+ }++ let m=M4.scale(root,p.bodyScale,p.bodyScale,p.bodyScale);+ let mv=M4.mul(this.view,m), mvp=M4.mul(this.proj,mv);+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.depthMask(true);+ this.pBody.use();+ gl.uniformMatrix4fv(this.pBody.u('uMVP'),false,mvp);+ gl.uniformMatrix4fv(this.pBody.u('uMV'),false,mv);+ gl.uniform3fv(this.pBody.u('uColor'),albedo);+ gl.uniform1f(this.pBody.u('uGrad'),p.grad);+ gl.uniform1f(this.pBody.u('uAmb'),p.amb);+ gl.uniform1f(this.pBody.u('uDiff'),p.diff);+ gl.uniform1f(this.pBody.u('uGloss'),p.gloss);+ gl.uniform1f(this.pBody.u('uAlpha'),1);+ this.mBody.bind(this.pBody.a('aPos'),this.pBody.a('aNormal'));+ this.mBody.draw();++ if(p.ears) this.drawEars(1, null);++ /* ---- 2. 五官 ---- */+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.depthMask(true);+ this.pSolid.use();+ const aPos=this.pSolid.a('aPos');+ // uAlpha 必须显式给 —— 片元是 vec4(uColor, uAlpha),漏设就是 0,+ // 混合后贡献为零,整条五官管线会静默地什么都不画。+ gl.uniform1f(this.pSolid.u('uAlpha'),1);++ const round=this.st.eyeYScale*(1-this.st.eyeArc);+ const arc=this.st.eyeArc;+ const eX=p.eyeX+this.st.eyeOffX, eY=p.eyeY+this.st.eyeOffY;+ const pivotScale=round;++ if(round>0.01){+ for(const side of [-1,1]){+ const rz = side<0 ? this.st.eyeLRotZ : this.st.eyeRRotZ;+ this.faceElem(side*eX, eY, rz, round, 1, EYE_LID_PIVOT);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),p.eyeColor);+ this.mEye.bind(aPos); this.mEye.draw();+ }+ }++ // 瞳孔 + 高光:跟着眼睛一起眨+ // arc < 0.5:`^ ^` 眯眼时眼球已经塌成一条线,再挂个瞳孔就是脸上一个孤点。+ if(p.eyeStyle!=='plain' && round>0.05 && arc<ARC_EYE_AT){+ const pupilCol=[c[0]*0.20+0.045, c[1]*0.20+0.050, c[2]*0.20+0.075];+ // 眼神偏移:pupilOffX/Y 是 look 自己的、常驻的。+ // 这是"腹黑"和"从眉毛底下往上看"的机制 —— 表情表里的 gaze 只在思考时+ // 才动,所以一个不思考但**看着别处**的眼神只能由 look 提供。+ // y 要乘 round:眼缝被挤扁时,瞳孔的偏移也得跟着扁,否则它会跑出去。+ const pOX=p.pupilOffX*p.eyeR, pOY=p.pupilOffY*p.eyeR*round;+ for(const side of [-1,1]){+ const ex=side*eX+pOX, ey=eY+pOY;+ const rz = side<0 ? this.st.eyeLRotZ : this.st.eyeRRotZ;+ if(p.eyeStyle==='pupil'){+ const pr=p.pupilR;+ // 三个坑一起来:pivotY 必须传 0(sy 是自己的半径,套眼睑铰链+ // 补偿会把瞳孔推到脸颊上);z 要压扁成片;z 还要抬到眼球前面+ // —— 胶囊眼球在 z 上有半径,不抬就整颗瞳孔沉在眼球背面。+ this.faceElem(ex-pr*0.12, ey-pr*0.20, rz, pr*round, pr, 0, pr*0.06, EYE_FRONT);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),pupilCol);+ this.mDot.bind(aPos); this.mDot.draw();+ // 瞳孔里的白点。贴纸风不要它:有高光就有立体,扁平是它的全部意思。+ const sr=p.specR;+ if(sr>0.001){+ this.faceElem(ex-pr*0.34, ey+pr*0.26, rz, sr*round, sr, 0, sr*0.06, EYE_FRONT+0.02);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),[1,1,1]);+ this.mDot.bind(aPos); this.mDot.draw();+ }+ } else if(p.specR>0.001) {+ // 深色豆豆眼 + 一点白高光,落在眼睛的左上+ const sr=p.specR;+ this.faceElem(ex-p.eyeR*0.36, ey+p.eyeR*0.40, rz, sr*round, sr, 0, sr*0.06, EYE_FRONT);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),[1,1,1]);+ this.mDot.bind(aPos); this.mDot.draw();+ }+ }+ }++ // 弧形眼 ^ ^+ if(arc>0.01){+ const h=arc*this.st.lid;+ for(const side of [-1,1]){+ const rz = side<0 ? this.st.eyeLRotZ : this.st.eyeRRotZ;+ this.faceElem(side*eX, eY-EYE_ARC_DROP, EYE_ARC_ROT_Z+rz, h, arc);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),p.eyeColor);+ this.mArc.bind(aPos); this.mArc.draw();+ }+ }++ /* ---- 2b. 眉 ---- */+ // 眉必须在眼睛之后画:它压在眼的上缘上,压住一点点才叫"压着眉毛"。+ this.drawBrows(round,eX,eY,albedo);++ /* ---- 2c. 嘴(含牙) ---- */+ this.drawMouth(c,albedo);++ /* ---- 2d. 怒符 ---- */+ if(p.vein>0.01) this.drawVein();++ /* ---- 2e. 蒸汽 ---- */+ if(p.steam>0.01) this.drawSteam();++ // 鼻涕泡:一个透明的水滴。不是实心圆片 —— 中间是透的,球体自己透出来,+ // 只有轮廓一圈(折射边)和左上一个小高光。用整颗球而不是压扁的片,+ // 因为菲涅尔要的就是"球面朝向",压扁之后轮廓就没了。+ if(this.st.bubble>0.01){+ const br=p.bubbleR*this.st.bubbleScale*this.st.bubble;+ let bm=M4.id();+ bm=M4.translate(bm,p.bubbleX,p.bubbleY,0);+ bm=M4.scale(bm,br,br,br);+ const bmodel=M4.mul(this.face,bm);+ const bmv=M4.mul(this.view,bmodel), bmvp=M4.mul(this.proj,bmv);+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.depthMask(false);+ this.pBubble.use();+ gl.uniformMatrix4fv(this.pBubble.u('uMVP'),false,bmvp);+ gl.uniformMatrix4fv(this.pBubble.u('uMV'),false,bmv);+ gl.uniform3fv(this.pBubble.u('uColor'),p.bubbleColor);+ gl.uniform1f(this.pBubble.u('uAlpha'),this.st.bubble);+ gl.uniform1f(this.pBubble.u('uPow'),p.bubblePow);+ gl.uniform1f(this.pBubble.u('uShine'),p.bubbleShine);+ gl.uniform1f(this.pBubble.u('uInner'),p.bubbleInner);+ this.mBody.bind(this.pBubble.a('aPos'),this.pBubble.a('aNormal'));+ this.mBody.draw();+ gl.depthMask(true);+ }++ /* ---- 3. 腮红 ---- */+ // restBlush —— 静止脸也带一点血色。球一天里绝大多数时间是 CALM,+ // 而"只有开心才有腮红"等于默认那张脸永远没有脸颊。+ const bl=Math.min(1,(this.st.blush+p.restBlush)*p.blushMul);+ if(bl>0.01){+ const light = p.blushBlend==='alpha';+ const d=this.mBlush.data;+ for(let n=0;n<2;n++){+ const j=n*6;+ d[j]=n===0?-p.blushX:p.blushX;+ d[j+1]=p.blushY; d[j+2]=-0.02;+ d[j+3]=p.blushColor[0]; d[j+4]=p.blushColor[1]; d[j+5]=p.blushColor[2];+ }+ this.mBlush.upload();+ mv=M4.mul(this.view,this.face); mvp=M4.mul(this.proj,mv);+ gl.blendFunc(gl.SRC_ALPHA, light?gl.ONE_MINUS_SRC_ALPHA:gl.ONE);+ gl.depthMask(false);+ this.pPts.use();+ gl.uniformMatrix4fv(this.pPts.u('uMVP'),false,mvp);+ gl.uniformMatrix4fv(this.pPts.u('uMV'),false,mv);+ gl.uniform1f(this.pPts.u('uPointFactor'),this.pointFactor);+ gl.uniform1f(this.pPts.u('uSize'),p.blushSize);+ gl.uniform1f(this.pPts.u('uAlpha'),bl*p.blushAlpha);+ this.mBlush.bind(this.pPts.a('aPos'),this.pPts.a('aColor'));+ this.mBlush.draw();+ }++ /* ---- 4. 描边 / 光晕 ---- */+ m=M4.scale(root,RIM,RIM,RIM);+ mv=M4.mul(this.view,m); mvp=M4.mul(this.proj,mv);+ gl.depthMask(false);+ this.pRim.use();+ gl.uniformMatrix4fv(this.pRim.u('uMVP'),false,mvp);+ gl.uniformMatrix4fv(this.pRim.u('uMV'),false,mv);+ gl.uniform1f(this.pRim.u('uPow'),p.rimPow);+ // 扫描是"光"在扫:只有发光描边那一路的 uColor 是亮色。深色描边那一路+ // uColor 是暗部色,扫过去就是在球上滚一条黑带,看着像渲染坏了。+ gl.uniform1f(this.pRim.u('uScanY'),this.st.scanY);+ gl.uniform1f(this.pRim.u('uScan'), p.rimMode==='dark' ? 0 : this.st.scan);+ if(p.rimMode==='dark'){+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.uniform3fv(this.pRim.u('uColor'),[c[0]*0.34,c[1]*0.34,c[2]*0.42]);+ gl.uniform1f(this.pRim.u('uIntensity'),p.rimGain*1.15);+ gl.uniform1f(this.pRim.u('uCap'),0.82);+ } else {+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE);+ const thinkBeat=this.st.think*(0.5+Math.sin(t*1.9)*0.5);+ const rimI=0.85+this.st.open*0.7+thinkBeat*0.55+ +Math.sin(t*p.faces[this.mood].pulse*2)*0.05;+ gl.uniform3fv(this.pRim.u('uColor'),c);+ gl.uniform1f(this.pRim.u('uIntensity'),rimI*p.rimGain);+ gl.uniform1f(this.pRim.u('uCap'),1.0);+ }+ this.mRim.bind(this.pRim.a('aPos'),this.pRim.a('aNormal'));+ this.mRim.draw();++ gl.depthMask(true);+ }++ /**+ * 一对耳朵。挂在 [root] 上,所以身体压扁、跳起来、歪头它都跟着 ——+ * 长在头上的东西不该有自己的动作。+ *+ * [scaleMul] !== 1 时用纯色画(硬描边那一层):描边不需要法线,+ * 也就不必为放大后的比例再建一个网格。+ */+ drawEars(scaleMul, flatColor){+ const gl=this.gl, p=this.p;+ if(!this.mEar) return;+ const flat = flatColor != null;+ if(flat){+ this.pSolid.use();+ gl.uniform1f(this.pSolid.u('uAlpha'),1);+ gl.uniform3fv(this.pSolid.u('uColor'),flatColor);+ } else {+ this.pBody.use();+ gl.uniform1f(this.pBody.u('uAlpha'),1);+ gl.uniform1f(this.pBody.u('uGrad'),p.grad);+ gl.uniform1f(this.pBody.u('uAmb'),p.amb);+ gl.uniform1f(this.pBody.u('uDiff'),p.diff);+ gl.uniform1f(this.pBody.u('uGloss'),p.gloss);+ const e=this.albedo||[0.5,0.5,0.5];+ gl.uniform3f(this.pBody.u('uColor'),+ e[0]*p.earShade, e[1]*p.earShade, e[2]*p.earShade);+ }+ for(const side of [-1,1]){+ let l=M4.id();+ l=M4.translate(l, side*p.earX, p.earY, p.earZ);+ // 往两边撇开:顶点从 +Y 转向 ∓x,所以左耳(+x 转角)往外+ l=M4.rotate(l, -side*p.earTilt*57.29578, 0,0,1);+ if(scaleMul!==1) l=M4.scale(l,scaleMul,scaleMul,scaleMul);+ const model=M4.mul(this.root,l);+ const mv=M4.mul(this.view,model), mvp=M4.mul(this.proj,mv);+ if(flat){+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,mvp);+ this.mEar.bind(this.pSolid.a('aPos'));+ } else {+ gl.uniformMatrix4fv(this.pBody.u('uMVP'),false,mvp);+ gl.uniformMatrix4fv(this.pBody.u('uMV'),false,mv);+ this.mEar.bind(this.pBody.a('aPos'),this.pBody.a('aNormal'));+ }+ this.mEar.draw();+ }+ }++ /**+ * 一块楔子。眉、尖牙、锯齿牙、嘴角勾、怒符全是它。+ *+ * 走 `pSolid`(无光照、纯色)有两个好处,都不是省事而是**对**:+ * 1. 可以随手做非等比缩放 —— 一块长 0.15、厚 0.03 的眉就是这么来的,+ * 而带光照的网格非要等比重建或算法线逆转置不可;+ * 2. 眉本来就不该有自己的高光。真实的眉是一道**沟**,不是一根亮条,+ * 用身体反照率的 0.3 倍画它,读出来就是沟。+ *+ * [rotZ] 决定尖端朝哪:尖端方向 = (−sin, cos)。+ */+ drawWedge(x,y,rotZ,thick,len,depth,color){+ const gl=this.gl;+ let l=M4.id();+ l=M4.translate(l,x,y,0);+ l=M4.rotate(l,rotZ*57.29578,0,0,1);+ l=M4.scale(l,thick,len,depth);+ const model=M4.mul(this.face,l);+ const mv=M4.mul(this.view,model), mvp=M4.mul(this.proj,mv);+ this.pSolid.use();+ gl.uniform1f(this.pSolid.u('uAlpha'),1);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,mvp);+ gl.uniform3fv(this.pSolid.u('uColor'),color);+ this.mWedge.bind(this.pSolid.a('aPos'));+ this.mWedge.draw();+ }++ /**+ * 两条眉 —— 这一批里唯一真正新增的"表情器官",也是所有信号里最强的一条。+ *+ * 上一版之所以不管怎么调色都不像"坏",根子就在这个函数不存在:`brow` 那个数+ * 一直只在把眼球转 0.4 弧度,屏幕上没有一根眉毛。+ *+ * 角度约定(左右对称,正值都表示"更凶"):+ * * 左眉绕 Z 转 `π + t`,右眉转 `π − t`,t > 0 时**内端低、外端高**,+ * 也就是那张 `\ /` 的脸。t < 0 反过来,是 `/ \`,委屈。+ * * 表情表里的 `brow` 直接接到 t 上([BallLook.browMood]),所以同一套皮肤+ * 在 HOLDING 和 CALM 下的眉是不一样的;`skew` 只加到右眉,这就是让它+ * 一高一低的那根线。+ * * 皱眉还会把眉**压低**:`brow` 越大,眉离眼睛越近。压低比转角更狠 ——+ * 一条贴着眼的眉,读出来是"它在盯着你"。+ */+ drawBrows(round,eX,eY,albedo){+ const p=this.p, st=this.st;+ if(p.brow<=0.01 || p.browLen<=0.001) return;+ // 眼睛成 `^ ^` 弧的时候收掉:`^ ^` 本身就是一张不需要眉毛的脸+ // (SUCCESS / SLEEPING 都是),在一条弧上面挂两撇眉是鬼片。+ if(st.eyeArc>=ARC_EYE_AT) return;++ const bL=st.brow, bR=st.brow+st.skew;+ // 极轻的小抖。静止的眉看起来像贴纸,会呼吸的才像长在脸上。+ const tw=Math.sin(st.browTwitch)*0.013*(0.35+Math.min(1,bL));++ const lift=EYE_LID_PIVOT*(1-round); // 眼睑铰链把眼睛整体下移的那点+ // 压眉:最多把眉到眼的距离压掉一半+ const drop=Math.min(0.55, bL*p.browMood*0.55)*p.browY;+ const baseY=eY+lift+p.browY-drop;++ const tL=p.browTilt-p.browSkew+bL*p.browMood+tw;+ const tR=p.browTilt+p.browSkew+bR*p.browMood-tw;+ const col=p.browColor||[albedo[0]*p.browDeep, albedo[1]*p.browDeep, albedo[2]*p.browDeep];++ this.drawWedge(-(eX+p.browX), baseY, Math.PI+tL, p.browThick, p.browLen, p.browDepth, col);+ this.drawWedge( (eX+p.browX), baseY, Math.PI-tR, p.browThick, p.browLen, p.browDepth, col);+ }++ /** 嘴角上方那条**唇线**在 x 处的 y。+ * 牙必须长在弧上,不能长在一条水平线上 —— 笑起来嘴角比中间高出一大截,+ * 按水平线放牙,牙就悬在嘴外面的半空里。 */+ lipY(x){+ const p=this.p, st=this.st;+ if(p.mouthStyle==='open') return st.mouthY;+ if(p.mouthStyle==='cat'){+ const mo=(p.mouthR+p.mouthTube)*0.5, side=x<0?-1:1;+ const u=Math.min(1, Math.abs(Math.abs(x)-mo)/Math.max(1e-4,mo));+ return st.mouthY - mo*st.mouthScaleY*Math.sqrt(Math.max(0,1-u*u));+ }+ const u=Math.min(1, Math.abs(x)/Math.max(1e-4,p.mouthR));+ return st.mouthY - p.mouthR*st.mouthScaleY*Math.sqrt(Math.max(0,1-u*u));+ }++ /**+ * 嘴,以及长在嘴上的牙。+ *+ * 三种嘴:+ * * `arc` —— 半环,最省,也最中性。笑和哭都靠它转角。+ * * `cat` —— 同一条半环画两次、各缩一半宽、并排拼起来,中间相接成 ω。+ * * `open` —— **半圆盘**([halfDisc])。这是唯一一个"面"。+ * 半环永远是一根线,管径加到 0.09 也只是一根粗线,读出来是"嘴很厚";+ * 而暴躁要的是"嘴张着",那必须是一块面。张嘴的上下比例由 [BallLook.gawp]+ * 直接给(高度 = 宽度 × gawp),不再借 `mouthScaleY` —— 那个数里混着+ * 笑容和呼吸,借它等于让"张多大"跟着心情漂。+ */+ drawMouth(c,albedo){+ const gl=this.gl, p=this.p, st=this.st;+ const aPos=this.pSolid.a('aPos');+ const deep = p.mouthMode==='deep' ? 0.40 : 1.0;+ const mouthCol=[c[0]*deep, c[1]*deep, c[2]*deep];+ this.pSolid.use();+ gl.uniform1f(this.pSolid.u('uAlpha'),1);+ gl.uniform3fv(this.pSolid.u('uColor'),mouthCol);++ const openH = p.mouthR*p.gawp*(0.88+st.open*0.5);++ if(p.mouthStyle==='open'){+ // 弧朝下、上唇是一条直线:上下唇之间那块**面**就是张开的嘴+ this.faceElem(0, st.mouthY-openH*0.5, st.mouthRotZ, openH, p.mouthR);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ this.mOpen.bind(aPos); this.mOpen.draw();+ } else if(p.mouthStyle==='cat'){+ const mo=(p.mouthR+p.mouthTube)*0.5;+ for(const side of [-1,1]){+ this.faceElem(side*mo, st.mouthY, st.mouthRotZ, st.mouthScaleY, 0.5);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ this.mMouth.bind(aPos); this.mMouth.draw();+ }+ } else {+ this.faceElem(0, st.mouthY, st.mouthRotZ, st.mouthScaleY);+ gl.uniformMatrix4fv(this.pSolid.u('uMVP'),false,this.mvp);+ this.mMouth.bind(aPos); this.mMouth.draw();+ }++ /* 锯齿牙:张嘴时的上下两排。它跟"张开"是一对 —— 只有洞没有牙,+ 读出来是"惊讶";有牙,才读成"在吼"。 */+ if(p.mouthStyle==='open' && p.teeth>0){+ const span=2*p.mouthR*p.teethSpan, n=p.teeth|0;+ gl.uniform3fv(this.pSolid.u('uColor'),p.fangColor);+ for(let i=0;i<n;i++){+ const t=n===1?0:(i/(n-1)-0.5)*2; // −1..1+ const x=t*span*0.5*(n===1?0:1);+ // 上排:从上唇往下扎+ this.drawWedge(x, st.mouthY-p.teethLen*0.5, Math.PI,+ p.teethThick, p.teethLen, p.teethThick, p.fangColor);+ // 下排:从下唇往上扎,错开半格,才像咬合+ if(p.teethRows>1){+ const x2=((i+0.5)/n*2-1)*span*0.5;+ this.drawWedge(x2, st.mouthY-openH*0.94+p.teethLen*0.5, 0,+ p.teethThick, p.teethLen, p.teethThick, p.fangColor);+ }+ }+ }++ /* 尖牙:一颗或两颗。+ `fangFlip=0` 尖朝上(虎牙,从下唇探出来),1 尖朝下(獠牙,从上唇垂下来)。+ 一颗和两颗读出来完全是两种东西 —— 一颗是**调皮**,两颗是**掠食**。 */+ if(p.fang>0){+ const n=p.fang|0;+ for(let i=0;i<n;i++){+ const side = n===1 ? 1 : (i===0?-1:1);+ const x=side*p.fangX;+ // 牙根落在唇线上,所以笑起来嘴角抬高时牙也跟着走+ const root=this.lipY(x)+p.fangY;+ const y = p.fangFlip ? root-p.fangLen*0.5 : root+p.fangLen*0.5;+ this.drawWedge(x, y, p.fangFlip?Math.PI:0,+ p.fangThick, p.fangLen, p.fangThick, p.fangColor);+ }+ }++ /* 嘴角的小勾。坏笑的最后一块:嘴角本身往上卷一下。+ 左勾朝左上、右勾朝右上 —— 尖端方向 (−sin, cos),所以左用 +角、右用 −角。 */+ if(p.ticks>0.01){+ const x=p.mouthR*0.96, y=this.lipY(p.mouthR*0.96)+p.tickThick*0.2;+ this.drawWedge(-x, y, 0.66, p.tickThick, p.tickLen, p.browDepth, mouthCol);+ this.drawWedge( x, y,-0.66, p.tickThick, p.tickLen, p.browDepth, mouthCol);+ }+ }++ /**+ * 怒符 —— 四根刺的爆点,画在太阳穴上。+ *+ * 这是一个**约定**而不是一个形状:漫画里画了几十年,看的人不需要想就知道是+ * 生气。所以它比任何写实的青筋都划算。四根刺从中心往外、离中心有一点点空隙+ * (刺的根不落在中心,中间留一个小洞),读出来才是"炸开"而不是"一团"。+ * 会跳:`veinBeat` 让半径起伏 14%,静止的怒符像贴纸。+ */+ drawVein(){+ const p=this.p, st=this.st;+ const beat=0.86+0.14*Math.sin(st.veinBeat);+ const col=[0.98,0.34,0.30];+ const d=p.veinLen*0.60*beat;+ for(let i=0;i<4;i++){+ const phi=i*Math.PI/2+0.32;+ this.drawWedge(p.veinX+Math.cos(phi)*d, p.veinY+Math.sin(phi)*d,+ phi-Math.PI/2,+ p.veinThick, p.veinLen*beat, p.veinThick, col);+ }+ }++ /**+ * 蒸汽:头顶两缕。+ *+ * 直接复用鼻涕泡那个水滴着色器,只把颜色换成白色、把内部薄雾关成 0 ——+ * 一颗中间全透、只有轮廓的球,就是一团水汽。这比再造一个粒子系统省,+ * 而且它跟泡一样会自动**干净地叠在**球体上(背面 discard + 菲涅尔边)。+ * 每边两团,相位错开,一边往上飘一边变大、一边淡出。+ */+ drawSteam(){+ const gl=this.gl, p=this.p, st=this.st;+ gl.blendFunc(gl.SRC_ALPHA,gl.ONE_MINUS_SRC_ALPHA);+ gl.depthMask(false);+ this.pBubble.use();+ const aPos=this.pBubble.a('aPos'), aNrm=this.pBubble.a('aNormal');+ gl.uniform3f(this.pBubble.u('uColor'),0.88,0.90,0.94);+ gl.uniform1f(this.pBubble.u('uPow'),2.0);+ gl.uniform1f(this.pBubble.u('uShine'),10);+ gl.uniform1f(this.pBubble.u('uInner'),0.02);+ for(const side of [-1,1]){+ for(let k=0;k<2;k++){+ const ph=((st.steamPhase+k*0.5+(side<0?0:0.22))%1+1)%1;+ const a=p.steam*(1-ph)*(1-ph)*0.85;+ if(a<0.012) continue;+ const r=p.steamR*(0.45+0.95*ph);+ const x=side*(0.20+p.steamR*0.6)+Math.sin(ph*6.283+side*1.7)*0.022;+ const y=p.steamY+ph*0.26;+ let bm=M4.id();+ bm=M4.translate(bm,x,y,0.03);+ bm=M4.scale(bm,r,r,r);+ const bmodel=M4.mul(this.face,bm);+ const bmv=M4.mul(this.view,bmodel), bmvp=M4.mul(this.proj,bmv);+ gl.uniformMatrix4fv(this.pBubble.u('uMVP'),false,bmvp);+ gl.uniformMatrix4fv(this.pBubble.u('uMV'),false,bmv);+ gl.uniform1f(this.pBubble.u('uAlpha'),a);+ this.mBody.bind(aPos,aNrm);+ this.mBody.draw();+ }+ }+ gl.depthMask(true);+ }++ /* face 空间里的一个元素:T * R * S,跟 setLocalFromFace 同序。+ sz 默认跟随 sx —— 点状元素(瞳孔、高光)是各向同性的小球,+ 漏掉 z 会把单位球沿视线拉成一根柱,糊掉整颗球。 */+ faceElem(x,y,rotZ,sy,sx=1,pivotY=0,sz=null,zOff=0){+ let l=M4.id();+ l=M4.translate(l,x,y+pivotY*(1-sy),zOff);+ l=M4.rotate(l,rotZ*57.29578,0,0,1);+ l=M4.scale(l,sx,sy,sz===null?1:sz);+ const model=M4.mul(this.face,l);+ const mv=M4.mul(this.view,model);+ this.mvp=M4.mul(this.proj,mv);+ }+}++/* =========================================================================+ 9. UI+ ========================================================================= */+let curPreset=P_IMP.key;+const edited={};+PRESETS.forEach(p=>edited[p.key]=clone(p));+let stages=[];+let heroes=[];++function showErr(e){+ const box=document.getElementById('err');+ box.style.display='block';+ box.textContent=String(e && e.stack || e);+ console.error(e);+}++function buildStage(presetKey){+ const p=PRESETS.find(q=>q.key===presetKey);+ const cell=document.createElement('div');+ cell.className='cell'; cell.dataset.k=presetKey;+ cell.setAttribute('aria-pressed', presetKey===curPreset ? 'true':'false');+ cell.innerHTML=`+ <div class="stage dark">+ <div class="badge">1:1</div>+ <canvas></canvas>+ </div>+ <div class="cap"><b>${p.label}</b><span>${p.tag}</span></div>`;+ cell.onclick=()=>setPreset(presetKey);+ return {el:cell, cv:cell.querySelector('canvas'), key:presetKey, ball:null};+}++/** 网格的底子只是 CSS —— 画布本来就是透明的,切换不用重画。 */+function setBg(bg){+ stages.forEach(s=>{ s.el.querySelector('.stage').className='stage '+bg; });+ document.querySelectorAll('#bgToggle button').forEach(b=>{+ b.setAttribute('aria-pressed', b.dataset.bg===bg ? 'true':'false');+ });+}++function build(){+ const ci=document.getElementById('presets');+ PRESETS.forEach(p=>{+ const b=document.createElement('button');+ b.type='button'; b.dataset.k=p.key;+ b.innerHTML=`${p.label}<span class="tag">${p.tag}</span>`;+ b.onclick=()=>setPreset(p.key);+ ci.appendChild(b);+ });++ const ms=document.getElementById('mood');+ MOODS.forEach(m=>{+ const o=document.createElement('option'); o.value=m; o.textContent=m; ms.appendChild(o);+ });+ ms.value=CALM_MOOD;+ ms.onchange=()=>{+ stages.forEach(s=>s.ball.mood=ms.value);+ heroes.forEach(h=>h.mood=ms.value);+ };++ // 网格:每个预设一个 1:1 画布。Ball 必须在挂进 DOM 之后建,+ // 它要读 clientWidth 才知道画多大。+ const grid=document.getElementById('grid');+ PRESETS.forEach(p=>{+ const st=buildStage(p.key);+ grid.appendChild(st.el);+ st.ball=new Ball(st.cv, edited[p.key]);+ st.ball.mood=p.demoMood||CALM_MOOD;+ stages.push(st);+ });++ // 放大区:同一个预设、两种底子,各一个画布。+ heroes=[...document.querySelectorAll('.heroCv')].map(cv=>{+ const b=new Ball(cv, edited[curPreset]);+ b.mood=PRESETS.find(q=>q.key===curPreset).demoMood||CALM_MOOD;+ return b;+ });++ document.querySelectorAll('#bgToggle button').forEach(b=>{+ b.onclick=()=>setBg(b.dataset.bg);+ });++ bindSliders();+ syncUI();+ requestAnimationFrame(loop);+}++/* 这台球最常待的表情。选预设时也把表情切到它的"代表表情" ——+ 暴躁球的脾气只在 TENSE 那一行上是看不出来的,得让它真的在那个心情里。 */+const CALM_MOOD='CALM';++function setPreset(key){+ curPreset=key;+ const p=PRESETS.find(q=>q.key===key);+ const mood=(p&&p.demoMood)||CALM_MOOD;+ const ms=document.getElementById('mood');+ ms.value=mood;+ stages.forEach(s=>s.ball.mood=mood);+ heroes.forEach(h=>h.mood=mood);+ syncUI();+}++const SLIDERS=[+ ['s-eyeR','eyeR','v-eyeR',v=>v.toFixed(3)],+ ['s-eyeX','eyeX','v-eyeX',v=>v.toFixed(3)],+ ['s-eyeY','eyeY','v-eyeY',v=>v.toFixed(3)],+ ['s-tint','tint','v-tint',v=>v.toFixed(2)],+ ['s-mul','albedoMul','v-mul',v=>v.toFixed(2)],+ ['s-gloss','gloss','v-gloss',v=>v.toFixed(2)],+ ['s-blush','blushMul','v-blush',v=>v.toFixed(2)],+ ['s-smile','smileBoost','v-smile',v=>v.toFixed(2)],+ ['s-bubR','bubbleR','v-bubR',v=>v.toFixed(3)],+ ['s-bubY','bubbleY','v-bubY',v=>v.toFixed(3)],+ ['s-bubIn','bubbleInner','v-bubIn',v=>v.toFixed(2)],+ // 性格那三组+ ['s-browTilt','browTilt','v-browTilt',v=>v.toFixed(2)],+ ['s-browSkew','browSkew','v-browSkew',v=>v.toFixed(2)],+ ['s-browThick','browThick','v-browThick',v=>v.toFixed(3)],+ ['s-browY','browY','v-browY',v=>v.toFixed(3)],+ ['s-brow','brow','v-brow',v=>v.toFixed(2)],+ ['s-eyeSquash','eyeSquash','v-eyeSquash',v=>v.toFixed(2)],+ ['s-eyeTilt','eyeTilt','v-eyeTilt',v=>v.toFixed(2)],+ ['s-fang','fang','v-fang',v=>String(Math.round(v))],+ ['s-fangLen','fangLen','v-fangLen',v=>v.toFixed(3)],+ ['s-fangFlip','fangFlip','v-fangFlip',v=>v>0.5?'朝下':'朝上'],+ ['s-teeth','teeth','v-teeth',v=>String(Math.round(v))],+ ['s-gawp','gawp','v-gawp',v=>v.toFixed(2)],+ ['s-steam','steam','v-steam',v=>v.toFixed(2)],+ ['s-vein','vein','v-vein',v=>v.toFixed(2)],+];++function bindSliders(){+ SLIDERS.forEach(([sid,key,vid,fmt])=>{+ const s=document.getElementById(sid);+ s.oninput=()=>{+ const v=parseFloat(s.value);+ edited[curPreset][key]=v;+ document.getElementById(vid).textContent=fmt(v);+ applyAll();+ };+ });+ document.getElementById('reset').onclick=()=>{+ edited[curPreset]=clone(PRESETS.find(p=>p.key===curPreset));+ applyAll(); syncUI();+ };+}++function syncUI(){+ document.querySelectorAll('#presets button').forEach(b=>{+ b.setAttribute('aria-pressed', b.dataset.k===curPreset ? 'true':'false');+ });+ stages.forEach(s=>s.el.setAttribute('aria-pressed', s.key===curPreset ? 'true':'false'));+ // 合并中性值再读:预设表里没写的字段(比如一把 arc 嘴的预设不需要 gawp)+ // 也能在滑块上显示成一个数,而不是 undefined。+ const p=Object.assign({}, LOOK_DEFAULTS, edited[curPreset]);+ SLIDERS.forEach(([sid,key,vid,fmt])=>{+ document.getElementById(sid).value=p[key];+ document.getElementById(vid).textContent=fmt(p[key]);+ });+ applyAll();+}++function applyAll(){+ stages.forEach(s=>s.ball.setParams(edited[s.key]));+ heroes.forEach(h=>h.setParams(edited[curPreset]));+}++let last=performance.now(), t0=last;+function loop(now){+ const dt=Math.min(0.05,(now-last)/1000); last=now;+ const t=(now-t0)/1000;+ try{+ stages.forEach(s=>s.ball.frame(dt,t));+ heroes.forEach(h=>h.frame(dt,t));+ }catch(e){ showErr(e); return; }+ requestAnimationFrame(loop);+}++try{+ build();+ window.addEventListener('resize',()=>{ stages.forEach(s=>s.ball.resize()); heroes.forEach(h=>h.resize()); });+ // 给测量脚本用:把渲染器和矩阵栈暴露出去,好从画布上量实际像素。+ window.LAB={Ball,M4,FACE_OLD,FACE_NEW,FACE_GRUMPY,LOOK_DEFAULTS,PRESETS,edited,version:'v40',+ get stages(){return stages}, get heroes(){return heroes}};+ window.__ready=true;+}catch(e){ showErr(e); }+</script>+</body>+</html>diff --git a/settings.gradle b/settings.gradlenew file mode 100644index 0000000..6e570d0--- /dev/null+++ b/settings.gradle@@ -0,0 +1,26 @@+pluginManagement {+ repositories {+ maven { url 'https://maven.aliyun.com/repository/gradle-plugin' }+ maven { url 'https://maven.aliyun.com/repository/google' }+ maven { url 'https://maven.aliyun.com/repository/public' }+ google()+ mavenCentral()+ gradlePluginPortal()+ }+}+dependencyResolutionManagement {+ repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)+ repositories {+ maven { url 'https://maven.aliyun.com/repository/google' }+ maven { url 'https://maven.aliyun.com/repository/public' }+ google()+ mavenCentral()+ // The dog's USB serial driver (usb-serial-for-android) is JitPack-only.+ // Aliyun proxies jitpack but answers 401 without an account, so this one+ // goes direct — keep it last, after the mirrors.+ maven { url 'https://jitpack.io' }+ }+}++rootProject.name = "body"+include ':app'diff --git a/tools/ball_shot.mjs b/tools/ball_shot.mjsnew file mode 100644index 0000000..6876522--- /dev/null+++ b/tools/ball_shot.mjs@@ -0,0 +1,394 @@+#!/usr/bin/env node+/*+ * 把 reports/ball-cutie-lab.html 里的那颗球渲染成 PNG。+ *+ * 为什么是"驱动那一页"而不是"照抄一份":+ * 照抄一份渲染器就等于在这个仓库里再放一套会和 Kotlin 漂移的代码 ——+ * 这个项目已经有一套 check_look_sync.py 在专门盯这件事。这里的做法是+ * 用 CDP 打开实验室页面本身,然后在那页里 `new LAB.Ball(...)` 建一颗+ * 自己的球、渲染、截图。渲染路径 100% 是实验室那一份,没有第二实现。+ *+ * ── 这台机器上 headless WebGL 是能跑的 ──+ * 以前的结论是"本机跑不了 headless Chrome",那是被 --disable-gpu 带偏的:+ * 加了它,ANGLE 连软件后端都起不来,页面里 getContext('webgl') 直接返回 null。+ * 去掉这个开关之后走的是真的 GPU(实测 ANGLE Metal / Apple M4 Pro)。+ * 所以**不要**再加 --disable-gpu。+ *+ * ── 两件不猜的事 ──+ * 1. **不自己合成**。WebGL 画布的 alpha 到底是预乘还是直通,取决于画布的+ * premultipliedAlpha 与页面的混合方式,算起来容易错。所以交给浏览器:+ * * 透明版:Emulation 把默认背景设成全透明,截出来天然是直通 alpha 的 PNG;+ * * 实底版:把底色画在页面自己身上,截出来是合成好的不透明图。+ * 两张图必须自洽 —— 把透明版按 over 叠到同一个底色上,应当逐像素等于实底版。+ * 这个等式就是自检,见 --verify。+ * 2. **不假设画布还在**。WebGL 上下文默认 preserveDrawingBuffer:false,合成+ * 一次之后缓冲就可能被清掉。所以定格帧挂在一个常驻 rAF 里每帧重画,+ * 截图那一刻画布上一定有东西。+ *+ * ── 定格 ──+ * 图标要的是"某一刻的样子",而这一页是活的:呼吸、左右摇摆、眨眼、随机小动作+ * 都会改变画面。所以先空跑若干秒让表情插值到稳定态,再把**动态**按住:+ * 呼吸相位归 1.0、摆动归零、不眨眼、不随机动。表情自己的形状(LISTENING 的+ * 歪头、眼睛开合、张嘴)全部留着 —— 那是这个表情本身。+ *+ * 用法:+ * node tools/ball_shot.mjs --look cream --mood LISTENING --size 864 --out /tmp/a.png+ * node tools/ball_shot.mjs --look cream --mood LISTENING --size 864 --probe+ * node tools/ball_shot.mjs --look cream --mood LISTENING --size 864 --no-ears+ *+ * 参数:+ * --look 实验室预设 key(now/jelly/kitty/mochi/sticker/sumikko/cream/imp)+ * --mood FACE_NEW 里的表情名,默认 LISTENING+ * --size CSS 边长(像素)= 输出图边长,默认 864+ * --settle 定格的空跑秒数,默认 6+ * --no-ears 把 earH 设成 0,用来比较有无耳朵的读法+ * --bg R,G,B 十进制(0-255)给实底版图用,默认不出的 14,23,43+ * --probe 只打印诊断,不写文件+ * --alpha 同时写一张 .alpha.png(透明底直通 alpha)+ * --verify 只做自检:透明版叠底色 vs 实底版,逐像素比+ */+import { spawn } from 'node:child_process';+import { writeFileSync, existsSync, readdirSync, mkdtempSync } from 'node:fs';+import { dirname, resolve, join } from 'node:path';+import { fileURLToPath } from 'node:url';+import { tmpdir } from 'node:os';++const HERE = dirname(fileURLToPath(import.meta.url));+const REPO = resolve(HERE, '..');+const LAB = join(REPO, 'reports', 'ball-cutie-lab.html');++/* ---------- 参数 ---------- */+const argv = process.argv.slice(2);+function arg(name, dflt) {+ const i = argv.indexOf('--' + name);+ if (i < 0) return dflt;+ const v = argv[i + 1];+ return v === undefined || v.startsWith('--') ? true : v;+}+const LOOK = String(arg('look', 'cream'));+const MOOD = String(arg('mood', 'LISTENING'));+const SIZE = parseInt(arg('size', '864'), 10);+const SETTLE = parseFloat(arg('settle', '6'));+const NO_EARS = !!arg('no-ears', false);+const PROBE = !!arg('probe', false);+const WITH_ALPHA = !!arg('alpha', false);+const VERIFY = !!arg('verify', false);+const OUT = arg('out', null);+const BG = String(arg('bg', '14,23,43')).split(',').map((s) => parseInt(s, 10));+if (BG.length !== 3 || BG.some((v) => !Number.isFinite(v))) {+ console.error('--bg 要给三个 0-255 的数,比如 --bg 14,23,43');+ process.exit(2);+}++if (!existsSync(LAB)) {+ console.error('找不到实验室页面:', LAB);+ process.exit(2);+}++/* ---------- 找 Chromium ----------+ * 不写死版本号:playwright 的缓存目录里可能有多个版本。环境变量+ * CHROME_PATH 优先,方便换别的构建。 */+function findChrome() {+ if (process.env.CHROME_PATH) return process.env.CHROME_PATH;+ const roots = [+ join(process.env.HOME || '', 'Library/Caches/ms-playwright'),+ join(process.env.HOME || '', '.cache/ms-playwright'),+ ];+ const rels = [+ 'chrome-mac-arm64/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing',+ 'chrome-mac/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing',+ 'chrome-linux/chrome',+ ];+ for (const root of roots) {+ if (!existsSync(root)) continue;+ const vers = readdirSync(root).filter((d) => d.startsWith('chromium-')).sort().reverse();+ for (const v of vers)+ for (const rel of rels) {+ const p = join(root, v, rel);+ if (existsSync(p)) return p;+ }+ }+ return null;+}++const CHROME = findChrome();+if (!CHROME) {+ console.error('没找到 Chromium。设 CHROME_PATH 指到可执行文件。');+ process.exit(2);+}++/* ---------- 起浏览器 ---------- */+const PORT = 9000 + Math.floor(Math.random() * 900);+const profile = mkdtempSync(join(tmpdir(), 'ballshot-'));+const chrome = spawn(+ CHROME,+ [+ '--headless',+ // 千万不要加 --disable-gpu:加了它 getContext('webgl') 就返回 null。+ '--no-sandbox',+ '--hide-scrollbars',+ '--mute-audio',+ '--disable-extensions',+ '--disable-background-timer-throttling',+ '--allow-file-access-from-files',+ '--remote-debugging-port=' + PORT,+ '--user-data-dir=' + profile,+ '--window-size=1920,1920',+ 'about:blank',+ ],+ { stdio: ['ignore', 'ignore', 'pipe'] }+);+let chromeErr = '';+chrome.stderr.on('data', (d) => { chromeErr += d.toString(); });++const sleep = (ms) => new Promise((r) => setTimeout(r, ms));++async function waitDevtools() {+ for (let i = 0; i < 100; i++) {+ try {+ const r = await fetch(`http://127.0.0.1:${PORT}/json/version`);+ if (r.ok) return;+ } catch (e) { /* 还没起来 */ }+ await sleep(150);+ }+ throw new Error('调试端口没起来。chrome stderr:\n' + chromeErr.slice(-2000));+}++let ws, msgId = 0;+const pending = new Map();+function send(method, params = {}) {+ return new Promise((res, rej) => {+ const id = ++msgId;+ pending.set(id, { res, rej });+ ws.send(JSON.stringify({ id, method, params }));+ });+}++async function evaluate(expression) {+ const r = await send('Runtime.evaluate', {+ expression, returnByValue: true, awaitPromise: true, timeout: 300000,+ });+ const res = r.result || {};+ if (res.exceptionDetails) {+ throw new Error('页面里抛异常: ' + (res.exceptionDetails.exception?.description || res.exceptionDetails.text));+ }+ return res.result?.value;+}++/* ==========================================================================+ 注入页面的那一段:建球、定格、量几何+ ========================================================================== */+function pageScript() {+ return `(() => {+ const T = ${SIZE}, MOOD = ${JSON.stringify(MOOD)}, LOOK = ${JSON.stringify(LOOK)};+ const SETTLE = ${SETTLE}, NO_EARS = ${NO_EARS};+ if (!window.__ready) return {ok:false, why:'__ready 还是 false',+ err:(document.getElementById('err')||{}).textContent||''};+ const LAB = window.LAB;+ const preset = LAB.PRESETS.find(p => p.key === LOOK);+ if (!preset) return {ok:false, why:'没有这个预设: '+LOOK+' 可选 '+LAB.PRESETS.map(p=>p.key).join('/')};++ // 参数取法和页面滑杆一致(都走 preset 结构),deep clone 免得改动倒灌回页面。+ const params = JSON.parse(JSON.stringify(preset));+ if (NO_EARS) params.earH = 0;+ if (!params.faces[MOOD]) return {ok:false, why:'faces 里没有这个表情: '+MOOD};++ // 画布必须挂进 DOM —— resize() 读 clientWidth。CSS 尺寸给 T,再把+ // devicePixelRatio 钉成 1,于是 canvas.width 正好 === T,不依赖机器 DPR。+ const wrap = document.createElement('div');+ wrap.id = '__shotwrap';+ wrap.style.cssText = 'position:fixed;left:0;top:0;width:'+T+'px;height:'+T+'px;background:transparent;z-index:1';+ const cv = document.createElement('canvas');+ cv.style.cssText = 'display:block;width:'+T+'px;height:'+T+'px';+ wrap.appendChild(cv);+ document.body.appendChild(wrap);+ const dpr0 = Object.getOwnPropertyDescriptor(window, 'devicePixelRatio');+ Object.defineProperty(window, 'devicePixelRatio', {value: 1, configurable: true});+ // 页面上别的 canvas 会碍事的地方:让 body 不留边距,免得整体位移+ document.documentElement.style.margin = '0'; document.body.style.margin = '0';++ // 把实验室自己的 UI 藏掉。它在整个视口上铺了一整页预设格子,截图会把它+ // 一起拍进去 —— 第一次跑就是这么错的(差异区域一直延伸到画布右下角)。+ // 用 visibility 而不是 display:布局不动,各画布的 clientWidth 还是原值。+ for (const el of [...document.body.children]) {+ if (el !== wrap) el.style.visibility = 'hidden';+ }++ let ball;+ try { ball = new LAB.Ball(cv, params); }+ catch (e) { return {ok:false, why:'Ball 构造失败: '+(e && e.message || e)}; }+ ball.autoFidget = false;+ ball.mood = MOOD;+ ball.resize();+ if (cv.width !== T || cv.height !== T) {+ return {ok:false, why:'画布尺寸不是 '+T+',而是 '+cv.width+'x'+cv.height+' —— devicePixelRatio 没被钉住'};+ }++ // ---- 1. 空跑,把表情插值到稳定态 ----+ const DT = 1/120;+ for (let i = 0; i < Math.round(SETTLE / DT); i++) ball.frame(DT, i * DT);++ // ---- 2. 按住动态 ----+ // 留:表情自己的形状(歪头 rotZ、眼睛开合、张嘴、腮红、由表情定的身体色)。+ // 去:一切"此刻相位"—— 呼吸、缓慢摆动、眨眼、随机小动作、视线漂移。+ // 图标是一张静止的图,留着这些只会让两次渲染不一样。+ const st = ball.st, orig = st.tick.bind(st);+ st.blinkLeft = 0; st.blinkNext = 1e9; st.action = 'NONE';+ st.tick = function (dt, t, mood, p) {+ orig(dt, t, mood, p);+ this.rotX = 0; this.rotY = 0; // 缓慢摆动 -> 正对镜头+ this.hopY = 0;+ this.breathe = 1; // 呼吸相位 -> 静息尺寸+ this.scaleX = 1; this.scaleY = 1;+ this.eyeOffX = 0; this.eyeOffY = 0; // 视线漂移(Listening 的 think=0,本就是 0)+ this.lid = 1; // 眼皮全开+ };++ // ---- 3. 常驻 rAF 每帧重画 ----+ // 不这么做的话,preserveDrawingBuffer:false 会让缓冲在合成后被清掉,+ // 截图那一刻画布上是空的。+ const frozen = () => { ball.frame(0, 0); requestAnimationFrame(frozen); };+ requestAnimationFrame(frozen);++ window.__shot = {ball, cv, wrap, params, LAB,+ restoreDpr: () => { if (dpr0) Object.defineProperty(window,'devicePixelRatio',dpr0); }};++ // ---- 4. 量几何(在页面里量,免得再把 1700 万像素搬过 CDP)----+ const m = {look:LOOK, mood:MOOD, size:T, ears:params.earH>0,+ eyeRadius:params.eyeR, eyeCyl:params.eyeH, faceZ:params.faceZ,+ bodyScale:params.bodyScale};+ return {ok:true, info:m};+})()`;+}++/* 在页面里量 alpha 包围盒 / 最大半径 / 采样色。单独一步,因为要等 rAF 画完。 */+function measureScript(bg) {+ return `(() => {+ const s = window.__shot, T = s.cv.width;+ const gl = s.ball.gl;+ const px = new Uint8Array(T*T*4);+ s.ball.frame(0,0); // 保证刚画完,缓冲有效+ gl.readPixels(0,0,T,T,gl.RGBA,gl.UNSIGNED_BYTE,px);+ let minX=T,minY=T,maxX=-1,maxY=-1,maxR2=-1,n=0;+ const cx=T/2, cy=T/2;+ const at = (x,y) => { const i=((T-1-y)*T+x)*4; return [px[i],px[i+1],px[i+2],px[i+3]]; };+ for (let y=0;y<T;y++) for (let x=0;x<T;x++){+ const i=((T-1-y)*T+x)*4, a=px[i+3];+ if(a>8){ n++;+ if(x<minX)minX=x; if(x>maxX)maxX=x; if(y<minY)minY=y; if(y>maxY)maxY=y;+ const dx=x+0.5-cx, dy=y+0.5-cy, r2=dx*dx+dy*dy; if(r2>maxR2)maxR2=r2;+ }+ }+ // 采样:身体正中(避开五官)、以及过球心水平线上一串,给"到底偏不偏蓝"一个数+ const body = at(Math.round(cx - T*0.16), Math.round(cy - T*0.28));+ return {bbox:[minX,minY,maxX,maxY], w:maxX-minX+1, h:maxY-minY+1,+ maxRadius:+Math.sqrt(maxR2).toFixed(3), pixels:n, bodySample:body,+ bg:${JSON.stringify(bg)}, canvasW:T};+})()`;+}++async function captureTransparent() {+ await send('Emulation.setDefaultBackgroundColorOverride', { color: { r: 0, g: 0, b: 0, a: 0 } });+ await evaluate(`window.__shot.wrap.style.background='transparent'; document.body.style.background='transparent';`);+ await sleep(120);+ const r = await send('Page.captureScreenshot', {+ format: 'png', captureBeyondViewport: true,+ clip: { x: 0, y: 0, width: SIZE, height: SIZE, scale: 1 },+ });+ return Buffer.from(r.result.data, 'base64');+}++async function captureOverBg(bg) {+ await send('Emulation.setDefaultBackgroundColorOverride', { color: { r: 0, g: 0, b: 0, a: 0 } });+ await evaluate(`document.body.style.background='rgb(${bg[0]},${bg[1]},${bg[2]})'; window.__shot.wrap.style.background='rgb(${bg[0]},${bg[1]},${bg[2]})';`);+ await sleep(120);+ const r = await send('Page.captureScreenshot', {+ format: 'png', captureBeyondViewport: true,+ clip: { x: 0, y: 0, width: SIZE, height: SIZE, scale: 1 },+ });+ return Buffer.from(r.result.data, 'base64');+}++async function main() {+ await waitDevtools();+ const list = await (await fetch(`http://127.0.0.1:${PORT}/json/list`)).json();+ const target = list.find((t) => t.type === 'page');+ ws = new WebSocket(target.webSocketDebuggerUrl);+ await new Promise((res, rej) => { ws.onopen = res; ws.onerror = rej; });+ ws.onmessage = (ev) => {+ const m = JSON.parse(ev.data);+ if (m.id && pending.has(m.id)) {+ const { res, rej } = pending.get(m.id);+ pending.delete(m.id);+ if (m.error) rej(new Error(JSON.stringify(m.error))); else res(m);+ }+ };++ await send('Page.enable');+ await send('Runtime.enable');+ await send('Page.navigate', { url: 'file://' + LAB });++ let ready = false;+ for (let i = 0; i < 300; i++) {+ await sleep(100);+ try { if (await evaluate('!!window.__ready')) { ready = true; break; } } catch (e) { /* 导航中 */ }+ }+ if (!ready) {+ const err = await evaluate("(document.getElementById('err')||{}).textContent||'(没有 #err)'").catch(() => '?');+ throw new Error('实验室页面没就绪。页面里的报错: ' + err);+ }++ const built = await evaluate(pageScript());+ if (!built?.ok) throw new Error('渲染失败: ' + JSON.stringify(built));+ await sleep(400); // 让常驻 rAF 至少跑过一帧++ const m = { ...built.info, ...(await evaluate(measureScript(BG))) };+ console.log(`预设 ${m.look} / 表情 ${m.mood} / ${m.size}px / 耳朵:${m.ears ? '有' : '无'}`);+ console.log(` 包围盒 x[${m.bbox[0]},${m.bbox[2]}] y[${m.bbox[1]},${m.bbox[3]}] 宽 ${m.w} 高 ${m.h}`);+ console.log(` 最大半径 ${m.maxRadius}px (画布半宽 ${m.size / 2}, 占 ${(m.maxRadius / (m.size / 2) * 100).toFixed(1)}%)`);+ console.log(` 非透明像素 ${m.pixels} 占全画布 ${(m.pixels / (m.size * m.size) * 100).toFixed(1)}%`);+ console.log(` 身体采样色 rgba(${m.bodySample.join(',')})`);++ if (PROBE) return;++ const transparent = await captureTransparent();+ const overBg = await captureOverBg(BG);++ // 先落盘再自检 —— 自检失败不该把已经渲好的图一起吞掉(第一版就是那样,+ // 报了个"对不上"结果两张图都没写成,连看着判断的机会都没有)。+ const base = OUT || join(tmpdir(), `ball-${LOOK}-${MOOD}.png`);+ const opaquePath = base.replace(/\.png$/, '') + '.png';+ const alphaPath = base.replace(/\.png$/, '') + '.alpha.png';+ writeFileSync(opaquePath, overBg);+ console.log(` 写出 ${opaquePath} (${(overBg.length / 1024).toFixed(0)} KiB)`);+ if (WITH_ALPHA) writeFileSync(alphaPath, transparent);+ if (WITH_ALPHA) console.log(` 写出 ${alphaPath} (${(transparent.length / 1024).toFixed(0)} KiB)`);++ if (VERIFY) {+ const { execFileSync } = await import('node:child_process');+ const tmpA = join(tmpdir(), 'ballshot-t.png'), tmpB = join(tmpdir(), 'ballshot-b.png');+ writeFileSync(tmpA, transparent); writeFileSync(tmpB, overBg);+ // Needs Pillow. Point PY at an interpreter that has it if `python3` does not.+ const py = process.env.PY || 'python3';+ try {+ const out = execFileSync(py, [join(HERE, 'ball_shot_verify.py'), tmpA, tmpB,+ String(BG[0]), String(BG[1]), String(BG[2])], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] });+ console.log(out.trimEnd());+ } catch (e) {+ if (e.stdout) process.stdout.write(e.stdout);+ if (e.stderr) process.stderr.write(e.stderr);+ process.exitCode = 1;+ }+ }+}++try {+ await main();+} catch (e) {+ console.error('✗ ' + e.message);+ process.exitCode = 1;+} finally {+ try { await evaluate('window.__shot && window.__shot.restoreDpr && window.__shot.restoreDpr()'); } catch (e) {}+ try { ws && ws.close(); } catch (e) {}+ chrome.kill('SIGKILL');+}diff --git a/tools/ball_shot_verify.py b/tools/ball_shot_verify.pynew file mode 100644index 0000000..b6f3ea0--- /dev/null+++ b/tools/ball_shot_verify.py@@ -0,0 +1,141 @@+#!/usr/bin/env python3+"""ball_shot.mjs 的自检:透明版叠底色,应当等于实底版。++## 它为什么存在++图标的两套图来源不同,但**都由浏览器输出、我都不自己合成**:+ * 前台层(自适应图标)要透明底的直通 alpha;+ * 传统光栅图要实底成品。+唯一需要我担保的事情是:那张透明 PNG 的 alpha 语义是**直通**的。+如果它是预乘的而我不知道,前台层叠到深色底上会整体偏暗 —— 小尺寸下很难+发现,正是最该机器判的一类错。++## 判据有对照,所以它有牙齿++只报一个"差异是多少"没用 —— 差异多小才算对?所以这里**同时算两种合成**:++ A. 把透明版当成直通叠上去;+ B. 把透明版当成预乘叠上去。++然后要求 **A 明显比 B 接近浏览器的实底版**。这不是阈值,是二选一。+如果哪天我把语义搞反了,A 和 B 会互换,脚本立刻翻脸 —— 第一版脚本+(只比一个数)就没有这个性质,它报的"最大差 9"读起来像是"我的代码有 bug",+实际只是预乘存 8 bit 的舍入。++## 容差是按物理原因分的++* **不透明像素(alpha ≥ 250)** 允许 ≤ 2:两边都是不透明,合成是恒等,只该有+ 截屏 → sRGB 的舍入。真错的话这里是几十上百。+* **半透明边缘**允许 ≤ 24:浏览器把画布按预乘存成 8 bit,深色 + 低 alpha 的+ 像素(比如直通色 20、alpha 40 → 存成 3,再乘回来 19)量化误差被 1/alpha+ 放大。这是预乘存储的固有代价,不是我的 bug。++用法:ball_shot_verify.py <transparent.png> <over_bg.png> R G B+"""+import sys++from PIL import Image, ImageChops++OPAQUE_TOL = 2+EDGE_TOL = 24+++def composite(straight_png: Image.Image, bg: tuple) -> Image.Image:+ """按 **直通 alpha** 合成:out = c*a + bg*(1-a)。"""+ a = straight_png.convert("RGBA")+ bgimg = Image.new("RGBA", a.size, bg + (255,))+ return Image.alpha_composite(bgimg, a).convert("RGB")+++def composite_as_premultiplied(png: Image.Image, bg: tuple) -> Image.Image:+ """按 **预乘** 解释同一份像素:c 已经被乘过 a,所以不再乘第二遍。++ 这就是"我理解错了"的那条假设。它在数学上是对的另一种约定,只是配上+ Android 的前台层会错 —— 这里把它算出来当对照组。+ """+ a = png.convert("RGBA")+ out = Image.new("RGB", a.size)+ sp, dp = a.load(), out.load()+ w, h = a.size+ for y in range(h):+ for x in range(w):+ r, g, b, al = sp[x, y]+ f = al / 255.0+ dp[x, y] = (+ min(255, int(round(r + bg[0] * (1 - f)))),+ min(255, int(round(g + bg[1] * (1 - f)))),+ min(255, int(round(b + bg[2] * (1 - f)))),+ )+ return out+++def stats(diff: Image.Image):+ """(最大通道差, 按 alpha 分档的最大差)。"""+ return max(i for i, n in enumerate(diff.convert("L").histogram()) if n)+++def main() -> int:+ if len(sys.argv) != 6:+ print(__doc__)+ return 2+ tp, bp = sys.argv[1], sys.argv[2]+ bg = tuple(int(v) for v in sys.argv[3:6])++ t = Image.open(tp).convert("RGBA")+ b = Image.open(bp).convert("RGB")+ if t.size != b.size:+ print(f"✗ 尺寸不一致: 透明版 {t.size} vs 实底版 {b.size}")+ return 1++ straight = composite(t, bg)+ premul = composite_as_premultiplied(t, bg)++ d_straight = ImageChops.difference(straight, b)+ d_premul = ImageChops.difference(premul, b)++ alpha = t.getchannel("A")+ apx = alpha.load()+ s_op = d_straight.load()+ p_op = d_premul.load()+ w, h = t.size++ opaque_n = edge_n = 0+ s_opaque_max = s_edge_max = 0+ s_opaque_bad = 0+ p_edge_max = 0+ for y in range(h):+ for x in range(w):+ av = apx[x, y]+ ds = max(s_op[x, y])+ dp = max(p_op[x, y])+ if av >= 250:+ opaque_n += 1+ s_opaque_max = max(s_opaque_max, ds)+ if ds > OPAQUE_TOL:+ s_opaque_bad += 1+ elif av > 8:+ edge_n += 1+ s_edge_max = max(s_edge_max, ds)+ p_edge_max = max(p_edge_max, dp)++ print(f" alpha: 边缘 {edge_n} 像素、不透明 {opaque_n} 像素")+ print(f" 不透明像素: 超差 {s_opaque_bad},最大差 {s_opaque_max} (容差 {OPAQUE_TOL})")+ print(f" 半透明边缘: 最大差 {s_edge_max} (容差 {EDGE_TOL})")+ print()+ print(" ── 对照 ──")+ print(f" 当成直通: 边缘最大差 {s_edge_max}")+ print(f" 当成预乘: 边缘最大差 {p_edge_max} ← 另一种约定,应当明显更差")+ if p_edge_max <= s_edge_max * 2:+ print(" ✗ 两种合成差不多差 —— 这个检查分辨不出语义,不能算通过")+ return 1+ print(f" 预乘/直通 = {p_edge_max / max(1, s_edge_max):.1f}× ⇒ 检查有分辨力")++ ok = s_opaque_bad == 0 and s_edge_max <= EDGE_TOL+ print()+ print(" ✓ 透明版与实底版自洽,且语义判定为**直通**" if ok+ else " ✗ 超出容差,合成或 alpha 语义至少有一步是错的")+ return 0 if ok else 1+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/body.py b/tools/body.pynew file mode 100755index 0000000..c61c93d--- /dev/null+++ b/tools/body.py@@ -0,0 +1,102 @@+#!/usr/bin/env python3+"""Drive the tablet body over its own WS server (port 9008, internal method names).++ adb forward tcp:9008 tcp:9008+ ./tools/body.py ui_tree # element outline (indented eid|class|label|[x,y])+ ./tools/body.py tap_eid e7 # tap by element number — preferred+ ./tools/body.py tap 500 470 # 0-1000 RELATIVE coords (last resort)+ ./tools/body.py global home+ ./tools/body.py type '{"text":"hello"}'+ ./tools/body.py screenshot -o shot.png++Methods are the CommandDispatcher names; the `screen.` prefix is optional.+Positional args are sugar for the common tools — anything else, pass raw JSON.+"""+import argparse+import base64+import json+import sys++from websocket import create_connection++POSITIONAL = {+ "screen.tap": ["x:int", "y:int"],+ "screen.tap_id": ["eid"],+ "screen.long_press": ["x:int", "y:int", "duration_ms:int"],+ "screen.long_press_id": ["eid"],+ "screen.swipe": ["x1:int", "y1:int", "x2:int", "y2:int", "duration_ms:int"],+ "screen.type": ["text", "id"],+ "screen.submit_input": ["id"],+ "screen.global": ["action"],+ "screen.ui_tree": ["verbose:bool"],+}+++def build_params(method, args):+ if len(args) == 1 and args[0].lstrip().startswith("{"):+ return json.loads(args[0])+ spec = POSITIONAL.get(method)+ if spec is None:+ if args:+ sys.exit(f"{method} takes no positional args — pass a JSON object instead")+ return {}+ if len(args) > len(spec):+ sys.exit(f"{method} takes at most {len(spec)} positional args: {' '.join(spec)}")+ params = {}+ for slot, raw in zip(spec, args):+ name, _, kind = slot.partition(":")+ if kind == "int":+ params[name] = int(raw)+ elif kind == "bool":+ params[name] = raw.lower() in ("1", "true", "yes")+ else:+ params[name] = raw+ return params+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("method")+ ap.add_argument("args", nargs="*")+ ap.add_argument("--url", default="ws://127.0.0.1:9008")+ ap.add_argument("-o", "--out", help="write base64 image result to this file")+ ap.add_argument("-t", "--timeout", type=float, default=30.0)+ opts = ap.parse_args()++ method = opts.method if "." in opts.method else f"screen.{opts.method}"+ envelope = {+ "type": "request",+ "id": "cli-1",+ "method": method,+ "params": build_params(method, opts.args),+ }++ ws = create_connection(opts.url, timeout=opts.timeout)+ try:+ ws.send(json.dumps(envelope))+ reply = json.loads(ws.recv())+ finally:+ ws.close()++ if "error" in reply:+ print(json.dumps(reply, ensure_ascii=False, indent=2))+ sys.exit(1)++ result = reply.get("result")+ if opts.out and isinstance(result, dict):+ # The body returns its PNG as `png_base64`; the other keys are here so+ # this stays usable against older builds.+ b64 = next(+ (result[k] for k in ("png_base64", "image", "data", "png", "base64") if k in result),+ None,+ )+ if b64:+ with open(opts.out, "wb") as f:+ f.write(base64.b64decode(b64))+ result = {k: v for k, v in result.items() if v is not b64}+ print(f"wrote {opts.out}")+ print(json.dumps(result, ensure_ascii=False, indent=2))+++if __name__ == "__main__":+ main()diff --git a/tools/boss.py b/tools/boss.pynew file mode 100755index 0000000..c48e0c0--- /dev/null+++ b/tools/boss.py@@ -0,0 +1,204 @@+#!/usr/bin/env python3+"""Command the ball's brain through the body hub, as if you were the user.++**This needs a hub you run yourself, and the hub is not in this repository.**+It is the `hub_url` side of `brain = hub` — the WebSocket server the device dials+out to (see `BodyWsClient`). With `brain = local` there is no hub and nothing for+this script to talk to. Set the address with `--hub` or `$BODY_HUB`.++The hub turns every device "event" into an inbound message for the brain agent,+so a *virtual* device identity can send instructions exactly like the real ball's+`asr.final` — the brain cannot tell the difference between this and a user+speaking. That is the whole mechanism, and it is worth saying out loud: this+impersonates a device to your own hub. It is a development tool for the person+who owns both ends.++ # One-shot: send an instruction, print the brain's live progress/replies+ BODY_HUB=ws://10.0.0.5:9100 ./tools/boss.py "帮我看下今天天气"++ # No wait: fire and forget (brain still works, output goes nowhere)+ ./tools/boss.py --hub ws://10.0.0.5:9100 --no-wait "去 Product Hunt 看看今天第一是什么"++ # Ask about progress on the running task+ ./tools/boss.py "进展如何?简单说一下"++ # Longer patience for long tasks (default 20 min)+ ./tools/boss.py --timeout 3600 "深度调研..."++Architecture notes, because they are load-bearing:+ * Replies are pushed ONLY to the connection that sent the message. The+ listener must stay open for the whole task, which is why send+listen+ share one socket here.+ * The virtual device registers with tools=[] — tool registration only+ feeds the hub's per-device tool pool, and the brain's screen tools come+ from the real device. Empty keeps us a pure mouthpiece.+ * Sessions are keyed by chat_id=device_id, so this identity has its own+ conversation history, separate from the ball's voice session.+"""+import argparse+import json+import os+import subprocess+import sys+import threading+import time++from websocket import create_connection++DEVICE_ID = "body-ops-7f2a"+DEVICE_NAME = "ops-inject"++# A "message" frame longer than this reads as a final deliverable, not+# narration. Narration ("progress") is never this long.+FINAL_HINT_CHARS = 300+++def adb_logcat_tail():+ """Device-side tool-call trail, as a daemon thread.++ The hub only pushes the brain's narration (progress) and replies+ (message) — its internal loop (LLM calls, tool params) lives in the+ cloud. But every tool call lands on the real device, and each one+ leaves a line in logcat (BodyWs "request ...", BodyWsClient "tool ...+ failed"). Tailing that turns "the brain has been silent for 3 minutes"+ into "it just took a screenshot and is reading the tree".++ adb is optional by design: no device attached (or adb missing) simply+ means no trail, not an error.+ """+ try:+ adb = subprocess.run(+ ["ls", f"{__import__('os').path.expanduser('~/Library/Android/sdk/platform-tools/adb')}"],+ capture_output=True).returncode == 0+ if not adb:+ return+ import os+ adb_path = os.path.expanduser("~/Library/Android/sdk/platform-tools/adb")+ p = subprocess.Popen(+ [adb_path, "logcat", "-s", "-T", "1",+ "BodyWs:D", "BodyWsClient:I", "Body:I"],+ stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True)+ for line in p.stdout:+ line = line.strip()+ if not line:+ continue+ # Keep it one line and compact: time + message body.+ try:+ body = line.split("): ", 1)[1] if "): " in line else line+ stamp = time.strftime("%H:%M:%S")+ print(f"[{stamp}] ⚙ {body[:150]}", flush=True)+ except Exception:+ pass+ except Exception:+ pass+++def main():+ ap = argparse.ArgumentParser(description="Command the ball's brain via the hub")+ ap.add_argument("instruction", help="The instruction, in the user's voice")+ ap.add_argument("--hub", default=os.environ.get("BODY_HUB"),+ help="ws:// address of your own body hub (or set $BODY_HUB)")+ ap.add_argument("--no-wait", action="store_true",+ help="send and exit without listening for replies")+ ap.add_argument("--timeout", type=int, default=1200,+ help="max seconds to listen for the final reply (default 1200)")+ ap.add_argument("--quiet", action="store_true", help="progress lines, final only")+ ap.add_argument("--verbose", action="store_true",+ help="also tail the device logcat (every tool call the brain makes)")+ args = ap.parse_args()++ if not args.hub:+ sys.exit("no hub address: pass --hub ws://host:9100 or set $BODY_HUB. "+ "The hub is not part of this repository — see the module docstring.")++ if args.verbose:+ threading.Thread(target=adb_logcat_tail, daemon=True).start()++ ws = create_connection(args.hub, timeout=10)+ ws.send(json.dumps({"type": "register",+ "data": {"device_id": DEVICE_ID,+ "device_name": DEVICE_NAME,+ "tools": []}}))+ time.sleep(0.5)+ ws.send(json.dumps({"type": "event", "kind": "asr.final",+ "device_id": DEVICE_ID,+ "data": {"text": args.instruction}}))+ print(f"→ 已下发: {args.instruction[:80]}", flush=True)+ if args.no_wait:+ ws.close()+ print("(不等待回复)")+ return++ ws.settimeout(60)+ deadline = time.time() + args.timeout+ last = time.time()+ final = None+ n = 0+ # Thought-loop watchdog: the brain can enter a degenerate state where it+ # emits the SAME narration line over and over (field-measured: the exact+ # sentence 400 times, no tool calls, until timeout). Per-message+ # uniqueness detects this without understanding the content: once the+ # same text repeats past N_MUTE lines, we inject an interrupt message+ # telling it to STOP and report — the only cure observed to break the+ # loop (waiting it out just burns the whole budget).+ seen_lines = {}+ muted = 0+ N_MUTE = 20 # repeats of one identical line before intervention+ muted_at = 0.0+ print("(监听中… ⏎打断无效;大脑沉默干活可能几分钟无输出)", flush=True)+ while time.time() < deadline:+ try:+ m = ws.recv()+ d = json.loads(m)+ t = d.get("type")+ c = str(d.get("content", "")).strip()+ if not c:+ continue+ n += 1+ last = time.time()+ # Watchdog bookkeeping. Identical narration repeated = loop.+ key = (t, c[:200])+ seen_lines[key] = seen_lines.get(key, 0) + 1+ if seen_lines[key] > N_MUTE:+ if muted < 2: # at most 2 interventions, then stop listening+ muted += 1+ muted_at = time.time()+ print(f"--- 检测到思维死循环(同一句话重复{seen_lines[key]}次),注入打断 #{muted} ---", flush=True)+ try:+ ws.send(json.dumps({"type": "event", "kind": "asr.final",+ "device_id": DEVICE_ID,+ "data": {"text": "停下。你正在无限重复同一句话。立刻停止当前路线,直接汇报:现在卡在哪、已尝试了什么、还剩什么可选路线。不要再说任何重复的话。"}}))+ except Exception:+ pass+ elif time.time() - muted_at > 300:+ print("--- 打断两次后仍在循环,放弃监听 ---", flush=True)+ break+ continue+ if t == "progress":+ if not args.quiet:+ print(f"[{time.strftime('%H:%M:%S')}] {c}", flush=True)+ elif t == "message":+ print(f"[{time.strftime('%H:%M:%S')}] 💬 {c}", flush=True)+ if len(c) > FINAL_HINT_CHARS:+ final = c+ break+ else:+ print(f"[{time.strftime('%H:%M:%S')}] ({t}) {c[:120]}", flush=True)+ except Exception:+ # websocket timeout between frames; the brain may be working+ # silently for minutes. Report heartbeat, give up after 7 min.+ silent = time.time() - last+ if silent > 420:+ print(f"--- {int(silent)}秒无任何输出,结束监听(大脑可能仍在跑或已卡死) ---", flush=True)+ break+ continue+ ws.close()+ if final:+ print("\n=== 最终答复 ===", flush=True)+ print(final)+ else:+ print(f"\n(结束,共 {n} 条,未等到长答复)", flush=True)+++if __name__ == "__main__":+ main()diff --git a/tools/check_launcher_icon.py b/tools/check_launcher_icon.pynew file mode 100644index 0000000..4ca99cc--- /dev/null+++ b/tools/check_launcher_icon.py@@ -0,0 +1,421 @@+#!/usr/bin/env python3+"""校验生成好的一整套 Android 启动图标。++这些判据是**踩过之后才定下来的**,每一条都对应一个曾经出过错的地方。+关键的一条是"剪影一致":第一版用 IoU 直接比前景层和单色层的剪影,得到+0.8611 —— 看起来像几何错了,其实是**挖空造成的**(前景不透明而单色为 0 的+像素正好等于眼睛+嘴的面积 5565px)。判据不修正的话,一个正确的结果会被+报成错误,而错误的判据比没有判据更危险:它教人忽略警告。++所以这里比的是"把挖空补回来之后的剪影"。++用法:check_launcher_icon.py --res app/src/main/res [--src 源图.alpha.png]+"""+import argparse+import sys+from pathlib import Path++import numpy as np+from PIL import Image++sys.path.insert(0, str(Path(__file__).resolve().parent))+import cosmic_backdrop++DENSITIES = [("mdpi", 1.0), ("hdpi", 1.5), ("xhdpi", 2.0), ("xxhdpi", 3.0), ("xxxhdpi", 4.0)]+ADAPTIVE_CANVAS = 108.0+LEGACY_CANVAS = 48.0+VISIBLE_FRACTION = 72.0 / 108.0+LEGACY_EXPECT = 0.88++BG = (14, 23, 43)+LUM_EYE = 60.0 # 前景层里眼睛的亮度上限(实测眼睛 ≈34,球体 ≈131)+LUM_MOUTH = 100.0 # ⚠️ 别写 120 —— 球体自身暗部底就是 120.6,会吞进来当嘴(见 check 6)+BALL_INNER = 0.90++# 底片装饰的判据(只在 --backdrop 给了以后生效)+GLOW_CEIL = 118.0 # 底片 P99 亮度上限。球体主体实测 ≈131,留 ~13 的余量+STAR_MIN_DELTA = 26.0 # 比底片中位数亮这么多,才算"一颗星"++failures: list[str] = []+notes: list[str] = []+++def fail(msg):+ failures.append(msg)+ print(f" ✗ {msg}")+++def ok(msg):+ print(f" ✓ {msg}")+++def load_rgba(p):+ return np.asarray(Image.open(p).convert("RGBA")).astype(np.float64)+++def check_alpha_semantics(res: Path):+ print("\n── 1. 文件完整性 / alpha 语义 ──")+ need_opaque = ("ic_launcher_background", "ic_launcher")+ need_alpha = ("ic_launcher_foreground", "ic_launcher_monochrome", "ic_launcher_round")+ n = 0+ for name, dpi in DENSITIES:+ for base in need_opaque + need_alpha:+ p = res / f"mipmap-{name}/{base}.webp"+ if not p.exists():+ fail(f"缺文件 {p}")+ continue+ n += 1+ a = load_rgba(p)+ al = a[:, :, 3]+ if base in need_opaque:+ if al.min() < 255:+ fail(f"{p.name} ({name}) 应当全不透明,实测 alpha 最小 {al.min():.0f}")+ else:+ if al.max() == 0:+ fail(f"{p.name} ({name}) 全透明,没有内容")+ elif al.min() == al.max():+ fail(f"{p.name} ({name}) alpha 没有变化,不像是带剪影的层")+ if n == 25:+ ok(f"25 个文件齐全,background/ic_launcher 全不透明,其余带 alpha")+++def check_canvas_sizes(res: Path):+ print("\n── 2. 画布尺寸 ──")+ bad = 0+ for name, dpi in DENSITIES:+ want_a = int(round(ADAPTIVE_CANVAS * dpi))+ want_l = int(round(LEGACY_CANVAS * dpi))+ for base, want in (("ic_launcher_background", want_a), ("ic_launcher_foreground", want_a),+ ("ic_launcher_monochrome", want_a),+ ("ic_launcher", want_l), ("ic_launcher_round", want_l)):+ w, h = Image.open(res / f"mipmap-{name}/{base}.webp").size+ if (w, h) != (want, want):+ fail(f"{name}/{base} 尺寸 {w}×{h},应为 {want}×{want}")+ bad += 1+ if not bad:+ ok("自适应层 108dp、传统层 48dp,五个密度都对")+++def check_visible_circle(res: Path):+ print("\n── 3. 自适应层:角色是否落在保证可见的圆里 ──")+ rows = []+ bad = 0+ for name, dpi in DENSITIES:+ cvs = int(round(ADAPTIVE_CANVAS * dpi))+ a = load_rgba(res / f"mipmap-{name}/ic_launcher_foreground.webp")+ al = a[:, :, 3]+ ys, xs = np.where(al > 8)+ cc = (cvs - 1) / 2.0+ r = float(np.hypot(xs - cc, ys - cc).max())+ vis = cvs * VISIBLE_FRACTION / 2.0+ rows.append((name, cvs, r, vis))+ if r >= vis:+ fail(f"{name}: 角色最大半径 {r:.2f} ≥ 可见圆 {vis:.1f},会被遮罩切到")+ bad += 1+ if not bad:+ slack = min(v - r for _, _, r, v in rows)+ ok(f"五个密度都放得下,最紧的一档余量 {slack:.2f}px(@其自身画布)")+ for name, cvs, r, vis in rows:+ print(f" {name:8s} 画布{cvs:4d} 角色 {r:6.2f} / 可见 {vis:5.1f} 余量 {vis-r:5.2f}")+++def check_silhouette_match(res: Path):+ """核心一项:单色层的剪影要和美术层一致 —— **把挖空补回来再比**。++ 判据**不是**"固定 IoU 阈值"。第一版写了 0.98,结果 mdpi 0.9663 / hdpi+ 0.9757 被判失败 —— 但量下来的事实是:五种密度的**最佳整数位移都是 (0,0)**+ (±1 扫一圈 IoU 一点都不提升),非挖空区 alpha 平均只差 0.53/255,而+ "对称差 ÷ 半个边界带"恒等于 **0.48px**(五档依次 0.48/0.49/0.48/0.48/0.48)。++ 恒定的 0.48px 就是纯亚像素效应的签名:差异面积 ∝ 周长 × 常数,而 IoU 是+ 面积比 ⇒ 小画布上必然更差。固定阈值等于**在惩罚分辨率,不在检查几何**。+ 真正的几何错位会给出约 2.0px(差 1px 的轮廓)。++ 所以判据两条:+ ① 位移扫描无收益 —— 两层注册正确;+ ② 对称差 ÷ 半个边界带 ≤ 1.0px —— 差异是亚像素级的。+ IoU 照打,但只作参考,不设阈值。+ """+ print("\n── 4. 单色层剪影 vs 美术层剪影(挖空已补回) ──")+ bad = 0+ for name, _ in DENSITIES:+ fg = load_rgba(res / f"mipmap-{name}/ic_launcher_foreground.webp")+ mo = load_rgba(res / f"mipmap-{name}/ic_launcher_monochrome.webp")+ fa, ma = fg[:, :, 3], mo[:, :, 3]+ # 挖空 = 实体上被单色层挖掉的洞(光看 ma==0 会把背景也算进去)+ holes = (ma == 0) & (fa > 200)+ sil_fg = fa > 128+ sil_mono = (ma > 128) | holes+ iou = (sil_fg & sil_mono).sum() / max(1, (sil_fg | sil_mono).sum())++ # ① 位移扫描:任何一个 ±1 整数位移能让 IoU 变好,就是真的错位了+ gain = 0.0+ for sy in (-1, 0, 1):+ for sx in (-1, 0, 1):+ s2 = np.roll(np.roll(sil_mono, sy, 0), sx, 1)+ gain = max(gain, (sil_fg & s2).sum() / max(1, (sil_fg | s2).sum()) - iou)++ # ② 边界带:1px 膨胀 ∩ 1px 腐蚀的环,面积 ≈ 周长 × 2+ d1 = np.roll(sil_fg, 1, 0) | np.roll(sil_fg, -1, 0) | np.roll(sil_fg, 1, 1) | np.roll(sil_fg, -1, 1)+ e1 = np.roll(sil_fg, 1, 0) & np.roll(sil_fg, -1, 0) & np.roll(sil_fg, 1, 1) & np.roll(sil_fg, -1, 1)+ perim = (d1 & ~e1).sum() / 2.0+ sym = int((sil_fg ^ sil_mono).sum())+ edge_px = sym / max(1.0, perim)++ mark = "✓" if (edge_px <= 1.0 and gain <= 1e-9) else "✗"+ print(f" {name:8s} IoU {iou:.4f} (仅供参考) 对称差 {sym:4d}px /"+ f" 周长 {perim:6.1f} = {edge_px:.2f}px {mark}"+ f" 位移收益 {gain:+.4f} 挖空 {int(holes.sum())}px")+ if edge_px > 1.0:+ fail(f"{name}: 剪影差异达 {edge_px:.2f}px(周长归一),超出亚像素级")+ bad += 1+ if gain > 1e-9:+ fail(f"{name}: 单色层与美术层有整体错位(某个 ±1 位移能把 IoU 变好 {gain:+.4f})")+ bad += 1+ if not bad:+ ok("五档剪影差异均为亚像素级(周长归一 ≤1.0px),且无整体错位")+++def check_holes(res: Path):+ """挖空必须是"被实体包住的洞",不能碰到剪影外沿。"""+ print("\n── 5. 挖空是内洞,不是缺口 ──")+ bad = 0+ for name in ("xxxhdpi",):+ cvs = int(round(ADAPTIVE_CANVAS * {"xxxhdpi": 4.0}[name]))+ fg = load_rgba(res / f"mipmap-{name}/ic_launcher_foreground.webp")+ mo = load_rgba(res / f"mipmap-{name}/ic_launcher_monochrome.webp")+ fa, ma = fg[:, :, 3], mo[:, :, 3]+ holes = (ma == 0) & (fa > 200)+ sil = fa > 128+ ys, xs = np.where(sil)+ hy, hx = np.where(holes)+ for lbl, arr, lo, hi in (("x", hx, xs.min(), xs.max()), ("y", hy, ys.min(), ys.max())):+ if arr.min() <= lo or arr.max() >= hi:+ fail(f"挖空在 {lbl} 方向碰到剪影边界(洞 {arr.min()}..{arr.max()},"+ f"剪影 {lo}..{hi}) —— 洞漏到外面了")+ bad += 1+ # 洞应当在球体内部:到球心的距离 < 剪影的包络+ cc = (cvs - 1) / 2.0+ rmax = float(np.hypot(xs - cc, ys - cc).max())+ rh = float(np.hypot(hx - cc, hy - cc).max())+ print(f" 挖空 {len(hx)} px,最远点到中心 {rh:.1f} / 剪影包络 {rmax:.1f}")+ if rh >= rmax * 0.98:+ fail(f"挖空的最远点已经贴到剪影边缘({rh:.1f} vs {rmax:.1f})")+ bad += 1+ if not bad:+ ok("挖空完全在剪影内部,没有漏到边界")+++def check_holes_align_with_face(res: Path):+ """洞要盖在真的五官上 —— 用美术层自己的暗像素当对照。++ 两个坑:++ ① **阈值必须落在实测的谷底里,不能"看着像"。** 原来嘴用 LUM_MOUTH=120,+ 而球体自身的暗部底就是 120.6(下半部亮度分位 5%=78.9、10%=120.6、+ 50%=124.1)。阈值贴上去的结果是把球体暗部一起吞进来当嘴:115→120 时+ "嘴"从 589px 涨到 725px、覆盖从 89.3% 掉到 72.6%;120→125 更是暴涨到+ 6131px。改成 100(谷底正中)后是 551px / 95.3%。++ ② **洞的边缘是抗锯齿的,`ma == 0` 取不到那一圈。** 单色层的 alpha 在洞的+ 边界上是一条从 0 斜坡到 255 的过渡,所以"严格相等 0"会漏掉最外一圈。+ 实测:嘴覆盖不到的那 26px **100% 都落在洞的 1px 边缘带上**。+ 所以判据给 1px 容差 —— 但**同时**还要严判,否则洞整体偏出去也能靠边缘+ 带凑过去。两条都要过。+ """+ print("\n── 6. 挖空位置与美术层的五官对齐 ──")+ tol_edges = 0.98 # 洞 ∪ 洞外 1px,应当几乎全覆盖+ min_strict = 0.70 # 只用洞本身,不许偏太远+ bad = 0+ name, dpi = "xxxhdpi", 4.0+ cvs = int(round(ADAPTIVE_CANVAS * dpi))+ fg = load_rgba(res / f"mipmap-{name}/ic_launcher_foreground.webp")+ mo = load_rgba(res / f"mipmap-{name}/ic_launcher_monochrome.webp")+ fa, ma = fg[:, :, 3], mo[:, :, 3]+ lum = 0.2126 * fg[:, :, 0] + 0.7152 * fg[:, :, 1] + 0.0722 * fg[:, :, 2]++ ys, xs = np.where(fa > 128)+ # 球心 = 剪影里最宽那一行的中心;球半径 = 最宽行的半宽+ best = (0, 0)+ for y in range(ys.min(), ys.max() + 1):+ xr = np.where(fa[y] > 128)[0]+ if len(xr) and xr.max() - xr.min() + 1 > best[0]:+ best = (xr.max() - xr.min() + 1, (xr.min() + xr.max()) / 2.0)+ ball_cx = best[1]+ ball_cy = float(ys.max()) - best[0] / 2.0+ ball_r = best[0] / 2.0++ yy, xx = np.mgrid[0:cvs, 0:cvs]+ inside = np.hypot(xx - ball_cx, yy - ball_cy) < ball_r * BALL_INNER+ holes = (ma == 0) & (fa > 200)+ hd = (holes | np.roll(holes, 1, 0) | np.roll(holes, -1, 0)+ | np.roll(holes, 1, 1) | np.roll(holes, -1, 1))+ fringe = hd & ~holes++ for lbl, mask in (("眼睛", inside & (yy < ball_cy) & (lum < LUM_EYE)),+ ("嘴", inside & (yy > ball_cy) & (lum < LUM_MOUTH))):+ if mask.sum() == 0:+ notes.append(f"{lbl}: 在美术层里没找到暗像素,跳过")+ print(f" {lbl}: 没找到对照像素,跳过")+ continue+ n = mask.sum()+ s = (mask & holes).sum() / n+ t = (mask & hd).sum() / n+ inedge = (mask & fringe).sum()+ print(f" {lbl}: 暗像素 {int(n):5d}px 严格覆盖 {s*100:5.1f}%"+ f" (含洞外 1px 边缘带) {t*100:5.1f}% 边缘带 {int(inedge)}px")+ if t < tol_edges:+ fail(f"{lbl} 含边缘带也只有 {t*100:.1f}% 落在洞上 —— 洞和五官没对上")+ bad += 1+ elif s < min_strict:+ fail(f"{lbl} 严格覆盖只有 {s*100:.1f}%(<{min_strict*100:.0f}%) ——"+ f"靠边缘带凑数,洞偏了")+ bad += 1+ if not bad:+ ok(f"眼睛和嘴都落在洞上(含洞外 1px 边缘带 ≥{tol_edges*100:.0f}%),"+ f"且严格覆盖 ≥{min_strict*100:.0f}%")+++def check_legacy(res: Path, bdir: Path | None, expect: float):+ """传统光栅图占比。++ ⚠️ **底片从纯色换成渐变之后,"拿角点当背景色"这招就废了。** 原来靠+ `a[1,1]` 取一个纯色再找和它不同的像素;换成星尘底片以后整个背景都和角点+ 不同,于是角色 bbox 涨到整张图、占比量出 100.0%。这不是产物的问题,是判据+ 的前提没了 —— 所以改成**和纯底片相减**:底片由 `cosmic_backdrop.render`+ 出,生成器和校验器调的是同一份实现,不是各写一遍。++ ⚠️ `expect` 不能写死 0.88 —— 那 88% 是**量旧图量出来的**,不是一条独立于+ 角色的约定。一旦下调用 `--content` 把角色缩小(为了给配景留空间),预期的+ 占比必然跟着降,写死就会把正确的结果报成失败。这是这条判据**第二次**因为+ 前提变化而失效。+ """+ print("\n── 7. 传统光栅图占比 / 圆形版 ──")+ bad = 0+ for name, dpi in DENSITIES:+ lgs = int(round(LEGACY_CANVAS * dpi))+ a = load_rgba(res / f"mipmap-{name}/ic_launcher.webp")+ if bdir is not None:+ bd = load_rgba(bdir / f"legacy-{name}.png")[:, :, :3]+ d = np.abs(a[:, :, :3] - bd).sum(axis=2)+ else:+ bg = a[1, 1, :3]+ d = np.abs(a[:, :, :3] - bg).sum(axis=2)+ ys, xs = np.where(d > 12)+ frac = max(xs.max() - xs.min() + 1, ys.max() - ys.min() + 1) / lgs+ mark = "✓" if abs(frac - expect) < 0.03 else "✗"+ print(f" {name:8s} 传统图 {lgs:3d}px 角色占 {frac*100:5.1f}%"+ f" (目标 {expect*100:.1f}%) {mark}")+ if abs(frac - expect) >= 0.03:+ fail(f"{name}: 传统图角色占 {frac*100:.1f}%,偏离目标 {expect*100:.1f}%")+ bad += 1++ r = load_rgba(res / f"mipmap-{name}/ic_launcher_round.webp")+ ral = r[:, :, 3]+ rys, rxs = np.where(ral > 128)+ # 满幅圆:最宽行应当等于边长(±2),且四角透明+ widths = {}+ for y in range(rys.min(), rys.max() + 1, max(1, lgs // 24)):+ xr = np.where(ral[y] > 128)[0]+ if len(xr):+ widths[y] = xr.max() - xr.min() + 1+ wmax = max(widths.values()) if widths else 0+ corners = ral[0, 0] + ral[0, -1] + ral[-1, 0] + ral[-1, -1]+ if wmax < lgs - 2:+ fail(f"{name}: 圆形版最宽行 {wmax},应接近边长 {lgs}")+ bad += 1+ if corners > 8:+ fail(f"{name}: 圆形版四角不透明(合计 {corners}),不是圆")+ bad += 1+ if not bad:+ ok(f"传统图占 {expect*100:.1f}% 满幅(与生成时 --content 一致),"+ f"圆形版是满幅圆(四角透明)")+++def check_decor(bdir: Path | None):+ """底片装饰本身的三条:确定性、亮度上限、星点存在。++ 这三条都是"看着挺好、下一版就悄悄坏掉"的类型,所以写成判据:++ * **确定性** —— 图标是提交进仓库的产物。同一 (size, seed) 只要有一点随机漂移,+ 每次重跑都在 diff 里炸出一堆无意义的字节变更,真改动会被淹掉。+ 所以这里**跨进程**比:同一个尺寸渲两遍必须逐像素相同。+ * **亮度上限** —— 球体主体亮度实测 ≈130,底片再好看也不能跟球抢。把小尺寸下+ "球还认不认得出"这件事变成一条可量的线。+ * **星点存在** —— 装饰最容易出的事是"改了参数之后星点全被暗角/渐变吃掉",+ 而肉眼在预览图那样的大图上根本发现不了。+ """+ print("\n── 8. 底片装饰(确定性 / 亮度上限 / 星点) ──")+ if bdir is None:+ print(" 没给 --backdrop,跳过(底片是纯色时这三条不适用)")+ return+ bad = 0++ # ① 确定性:同一个尺寸渲两遍+ a1 = np.asarray(cosmic_backdrop.render(432))+ a2 = np.asarray(cosmic_backdrop.render(432))+ if not np.array_equal(a1, a2):+ fail("同一尺寸渲两遍不一致 —— 底片不是确定性的(图标会每次重建都变)")+ bad += 1+ else:+ print(" 确定性:同尺寸两遍逐像素相同 ✓")++ # ② 亮度上限+ bg = load_rgba(bdir / "adaptive-xxxhdpi.png")[:, :, :3]+ lum = 0.2126 * bg[:, :, 0] + 0.7152 * bg[:, :, 1] + 0.0722 * bg[:, :, 2]+ print(f" 底片亮度 均值 {lum.mean():.1f} 中位 {np.median(lum):.1f}"+ f" P99 {np.percentile(lum, 99):.1f} 最大 {lum.max():.1f}")+ if np.percentile(lum, 99) > GLOW_CEIL:+ fail(f"底片 P99 亮度 {np.percentile(lum, 99):.1f} 超过上限 {GLOW_CEIL}"+ f" —— 会和球体(≈131)抢对比度")+ bad += 1++ # ③ 星点:局部亮于周边 N 的孤立小亮点+ med = np.median(lum)+ lit = lum > med + STAR_MIN_DELTA+ print(f" 星点数(亮于底片中位数 +{STAR_MIN_DELTA}) {int(lit.sum())} px")+ if lit.sum() < 200:+ fail(f"亮星只有 {int(lit.sum())} px —— 星点被渐变或暗角吃掉了")+ bad += 1+ if not bad:+ ok("底片确定性、辉光不压过球、星点确实存在")+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--res", default="app/src/main/res")+ ap.add_argument("--src", default=None, help="源图,给了就顺便核对几何比例")+ ap.add_argument("--backdrop", default=None,+ help="纯底片目录(生成器 --dump-backdrop 的产物)。底片是渐变时"+ "必给,否则第 7 项量不准")+ ap.add_argument("--content", type=float, default=1.0,+ help="角色大小因子,必须和生成时用的相同 —— 第 7 项的目标占比"+ "要跟着它走(88% 是量旧图来的,不是独立约定)")+ args = ap.parse_args()+ res = Path(args.res)+ bdir = Path(args.backdrop) if args.backdrop else None+ expect_legacy = LEGACY_EXPECT * args.content++ print(f"校验 {res}")+ check_alpha_semantics(res)+ check_canvas_sizes(res)+ check_visible_circle(res)+ check_silhouette_match(res)+ check_holes(res)+ check_holes_align_with_face(res)+ check_legacy(res, bdir, expect_legacy)+ check_decor(bdir)++ total = sum(p.stat().st_size for p in res.glob("mipmap-*/*.webp"))+ print(f"\n总体积 {total/1024:.1f} KiB")+ for n in notes:+ print(f" 注: {n}")+ if failures:+ print(f"\n✗ {len(failures)} 项没过")+ return 1+ print("\n✓ 全部通过")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/check_look_sync.py b/tools/check_look_sync.pynew file mode 100644index 0000000..d4b826b--- /dev/null+++ b/tools/check_look_sync.py@@ -0,0 +1,468 @@+#!/usr/bin/env python3+"""实验室与真机是否还对得上。++`reports/ball-cutie-lab.html` 是这套球球的规格说明书:用户在那个页面上看效果、+拖参数、做决定,然后把结论落到 Kotlin。所以两边一旦漂移,用户就是拿一个假的样机+在做决策 —— 而这是发生过的:++ * 实验室的 RIM_FS 少写了一条扫描光带,真机上多出一条从下往上滚的色带。+ 参数核对脚本查不出这个,因为参数全对,错的是着色器。+ * 我手改 RIM_FS 时把 `abs(d)` 写成了 `d`,band 从 [0,1] 变成最大 140,+ 被 clamp 到满强度 —— 整颗球从下往上染色。实验室里依然看不见。++这个脚本查两件事:**参数逐位**、**着色器语义**(忽略空白与注释)。+只做检查,不改任何文件;不一致就退出码 1。++ python3 tools/check_look_sync.py+"""+import re+import sys+from pathlib import Path++# 仓库根目录按脚本自身位置推断。别写死绝对路径 —— 写死过一次,别人 clone 下来+# 脚本第一行就指向一个不存在的目录,且报错是"文件读不到",看不出是路径问题。+ROOT = Path(__file__).resolve().parent.parent+KT_LOOK = ROOT / "app/src/main/java/net/kuafuai/andee/ui/ball/BallLook.kt"+KT_RENDERER = ROOT / "app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallRenderer.kt"+KT_MOOD = ROOT / "app/src/main/java/net/kuafuai/andee/ui/ball/Mood.kt"+LAB = ROOT / "reports/ball-cutie-lab.html"++# 参数名对应:实验室里叫什么 → Kotlin 里叫什么+PARAMS = [+ ("bodyBase", "bodyBase"), ("tint", "bodyTint"), ("albedoMul", "bodyAlbedo"),+ ("grad", "bodyGrad"), ("amb", "bodyAmb"), ("diff", "bodyDiff"),+ ("gloss", "bodyGloss"), ("bodyScale", "bodyScale"),+ ("faceZ", "faceZ"), ("eyeX", "eyeX"), ("eyeY", "eyeY"),+ ("eyeR", "eyeRadius"), ("eyeH", "eyeCyl"),+ ("mouthR", "mouthR"), ("mouthTube", "mouthTube"),+ ("mouthY", "mouthY"), ("mouthThick", "mouthThick"),+ ("eyeColor", "eyeColor"), ("pupilR", "pupilR"), ("specR", "glintR"),+ ("blushColor", "blushColor"), ("blushAlpha", "blushAlpha"),+ ("blushSize", "blushSize"), ("blushX", "blushX"), ("blushY", "blushY"),+ ("rimGain", "rimGain"), ("rimPow", "rimPow"),+ ("bubbleX", "bubbleX"), ("bubbleY", "bubbleY"), ("bubbleR", "bubbleR"),+ ("bubbleColor", "bubbleColor"), ("bubblePow", "bubblePow"),+ ("bubbleShine", "bubbleShine"), ("bubbleInner", "bubbleInner"),+ # 耳朵与眉毛。奶油球 2026-09-30 长出来的那一对耳朵,加在这里是因为它们**进+ # 了屏幕上的轮廓**:earH/earX/earY/earTilt 一起决定耳朵出不出球的边。+ # (眉毛曾经的理由也是"压不压到眼睛上" —— 但奶油球那对同一天删掉了,+ # 现在只有 BAJIE 还在用,而它在实验室里根本没有预设。字段仍然留着核对,+ # 因为它给"以后某个形象想要一根真的眉"留了路。)+ #+ # ⚠️ 想验"眉压没压到眼睛"不能只看 browY:眉盘是 dotMesh 缩放的**椭圆**再旋转,+ # 最低点是 `cy − √((w·sinθ)² + (h·cosθ)²)`,不是 `cy − browH`;眼睛的顶也不是+ # `eyeY + eyeRadius`(那只是球,漏了圆柱段 eyeCyl/2)—— 奶油球当初就是在这两个+ # 地方都算错,才把一根眉放到了几乎贴着眼球的位置(见 BallLook.CREAM 的注释)。+ # browShade 故意**不在**这个表里:它是"零即关",Kotlin 侧没有任何形象写这一行,+ # 而实验室的 preset 表为了格式整齐每个形象都把它列出来。核对一个恒为默认值的+ # 字段没有意义。+ ("earH", "earH"), ("earW", "earW"), ("earThick", "earThick"),+ ("earX", "earX"), ("earY", "earY"), ("earZ", "earZ"),+ ("earTilt", "earTilt"), ("earBend", "earBend"), ("earShade", "earShade"),+ ("browW", "browW"), ("browH", "browH"),+ ("browY", "browY"), ("browTilt", "browTilt"),+ # 眼神。eyeCut 是**进屏幕轮廓的那一类**:0.48 把眼睛上沿切掉 23% 的高度,+ # 眼睛从竖椭圆变成横的;eyeTilt 把整颗眼睛连瞳孔一起转。这两条在实验室里+ # 调完不落回 Kotlin,用户看到的就是另一双眼睛。+ ("eyeCut", "eyeCut"), ("eyeTilt", "eyeTilt"),+ ("pupilAspect", "pupilAspect"),+]++# 真正核对的形象。**这个元组是唯一的定义处** —— 下面的循环、末尾的报告都读它,+# 免得"核对了哪些"和"报告说核对了哪些"各写一遍然后对不上。+#+# imp 加进来的理由很具体:它是唯一一个曾经完全没覆盖的落地形象,而"眉毛和脸+# 同色看不见"那个 bug 正因为它不在这里,只能在真机上被发现。它现在在,但只是+# **部分**在 —— 角/獠牙/尾巴这一页画不出来,见 LAB_BLIND_SPOTS。+CHECKS = ("cream", "imp")++# 实验室**画不出来**的几何,按形象列。这和"参数没核对"是两件事,报告里必须分开:+# 前者是实验室会画出一颗形状不对的球(可见的谎),后者只是漏检(没覆盖)。+# 混在一起说,用户就会以为预设里的球等于真机上的球。+LAB_BLIND_SPOTS = {+ "imp": "角、獠牙、尾巴",+ "bajie": "猪鼻、鼻孔",+}+# 枚举型:实验室用的是字符串+ENUMS = [+ ("mouthDeep", lambda L: L.get("mouthMode") == "deep"),+ ("blushBlendNormal", lambda L: L.get("blushBlend") == "alpha"),+ ("rimDark", lambda L: L.get("rimMode") == "dark"),+]+# 着色器别名:实验室里叫 BODY/PTS,Kotlin 里叫 SHELL/POINTS+SHADER_ALIAS = {+ "SHELL_VS": "BODY_VS", "SHELL_FS": "BODY_FS",+ "POINTS_VS": "PTS_VS", "POINTS_FS": "PTS_FS",+}+SHADERS_IN_USE = ["SOLID_VS", "SOLID_FS", "SHELL_VS", "SHELL_FS",+ "RIM_VS", "RIM_FS", "POINTS_VS", "POINTS_FS",+ "BUBBLE_VS", "BUBBLE_FS"]++# 表情表的字段与默认值。两边的默认值必须一致 —— Face 的具名参数和实验室+# F() 的 Object.assign 都从这里取,改一边不改另一边就是一次静默漂移。+FACE_DEFAULTS = {+ "eyeArc": 0.0, "blush": 0.0, "spread": 1.0,+ "breath": 0.022, "gape": 0.0, "bubble": 0.0,+}+FACE_FIELDS = ["c", "eye", "brow", "smile", "pulse", "tilt", "skew", "jitter",+ "eyeArc", "blush", "spread", "breath", "gape", "bubble"]+++def parse_kotlin_defaults(text):+ """`BallLook` 类声明里的默认值 —— 某个 look 没写某一行时,用的就是它们。++ 为什么非要知道默认值:三个形象里只有 cream 把参数写齐了,另外两个大量省略。+ 如果只比对"写出来的行",省略的那些就既没核对、又和实验室的预设对不上 ——+ 而 earH 恰恰是**零即关**那一类:Kotlin 省略 = 默认 0f,实验室预设写了 0.155,+ 两边画的就不是同一颗球。**省略不等于"随便"**,它是一项被继承的值。++ 只取**声明区**(`class BallLook(` 到 `) {` 之前)并丢掉注释行,所以 KDoc 里+ 的示例文字不会被当成声明;`companion object` 里的 `private val EMPTY` 也不在+ 这个区间内。+ """+ head = text[text.index("class BallLook("):]+ head = head[:head.index("\n) {")]+ lines = [ln for ln in head.split("\n")+ if not re.match(r"\s*(?:\*|/\*|//)", ln)]+ out = {}+ for m in re.finditer(+ r"\bval (\w+):\s*(\w+)\s*=\s*(floatArrayOf\([^)]*\)|[^,\n]+),",+ "\n".join(lines),+ ):+ name, kind, raw = m.group(1), m.group(2), m.group(3).strip()+ try:+ if kind == "Float":+ out[name] = float(raw.rstrip("f"))+ elif kind == "Boolean":+ out[name] = raw == "true"+ elif kind == "Int":+ out[name] = int(raw)+ elif kind == "FloatArray" and raw.startswith("floatArrayOf"):+ out[name] = [float(x.strip().rstrip("f"))+ for x in raw[len("floatArrayOf("):-1].split(",")]+ except ValueError:+ # 值的形态不认识(比如 `EMPTY`)—— 跳过。跳过的后果是这一项被当成+ # "没有默认值",会在下面报成 Kotlin=None,是**大声**的失败,不是静默。+ pass+ return out+++def parse_kotlin_look(text, name):+ body = text[text.index(f"val {name} = BallLook("):]+ body = body[:body.index("\n)")]+ out = {}+ for m in re.finditer(+ r"(\w+)\s*=\s*(floatArrayOf\([^)]*\)|[-\d.]+f|true|false)", body+ ):+ k, v = m.group(1), m.group(2)+ if v.endswith("f"):+ out[k] = float(v[:-1])+ elif v.startswith("floatArrayOf"):+ out[k] = [float(x.strip().rstrip("f"))+ for x in v[len("floatArrayOf("):-1].split(",")]+ else:+ out[k] = v == "true"+ return out+++def parse_lab_preset(text, key):+ i = text.index(f"key:'{key}'")+ i = text.rindex("const P_", 0, i)+ j = text.index("\n};", i)+ out = {}+ for m in re.finditer(+ r"(\w+)\s*:\s*(\[[^\]]*\]|'[^']*'|[-\d.]+|true|false)", text[i:j]+ ):+ k, v = m.group(1), m.group(2)+ if v.startswith("["):+ out[k] = [float(x) for x in v[1:-1].split(",")]+ elif v.startswith("'"):+ out[k] = v[1:-1]+ elif v in ("true", "false"):+ out[k] = v == "true"+ else:+ out[k] = float(v)+ return out+++def strip_glsl(src):+ """归一化成可比的语义文本:去注释,声明按集合比,其余按去空白文本比。++ 三件事都归一化过,因为它们在 GLSL 里都没有意义、但会让天真的文本比对+ 报出一堆假警报,检查很快就没人看了:++ * 空白与换行 —— 两边排版风格不同+ * 声明顺序 —— 各自按可读性排的+ * 一行几条声明 —— 一边一行一条,另一边图省事写成一行两条++ 所以声明是**按声明粒度**提取(不是按行),再排序。+ """+ src = re.sub(r"//[^\n]*", "", src)+ src = re.sub(r"/\*.*?\*/", "", src, flags=re.S)++ decl_pat = re.compile(r"\b(?:uniform|varying|attribute)\s+[^;]+;")+ decls = sorted(re.sub(r"\s+", "", d) for d in decl_pat.findall(src))+ body = decl_pat.sub("", src)+ body = re.sub(r"precision\s+\w+\s+\w+;", "", body)+ return ";".join(decls) + "||" + re.sub(r"\s+", "", body)+++def kotlin_shaders(text):+ return {+ m.group(1): strip_glsl(m.group(2))+ for m in re.finditer(+ r'private const val (\w+_(?:VS|FS)) = """\n(.*?)\n\s*"""', text, re.S+ )+ }+++def lab_shaders(text):+ out = {}+ for m in re.finditer(r"const (VS|FS)_(\w+) = `\n(.*?)\n`;", text, re.S):+ # 实验室里变量名不一致(mv vs mvpos)不影响语义,统一一下再比+ src = m.group(3)+ out[f"{m.group(2)}_{m.group(1)}"] = strip_glsl(src)+ return out+++def slice_region(text, start_pat, end_pat):+ """截取 [start_pat, 之后第一次出现的 end_pat) 之间的源码,带行号返回。"""+ i = text.index(start_pat)+ start_line = text.count("\n", 0, i) + 1+ j = i + len(start_pat) + text[i + len(start_pat):].index(end_pat)+ return [(start_line + n, ln)+ for n, ln in enumerate(text[i:j].split("\n"))]+++def check_color_ownership(label, rows, set_pat, draw_pat):+ """每一笔都必须在自己所在的分支里设过颜色。++ 这是踩过两次的坑:`uColor` 是没有 attribute 的 uniform,**它会跨 draw call+ 保持上次的值**。所以少设一次不会报错,只会静默地继承上一笔的颜色。++ 为什么不能靠"上一条 draw 之后有没有 set"来判断:圆眼那一路是**条件分支**,+ 源码里写着 `eyeMesh.draw()`、运行时却可能因为尺寸阈值不执行,+ 静态看行序永远是对齐的。所以要给颜色算**作用域**:++ * 一次 set 从它自己开始生效;+ * 出现缩进比它更小的行(也就是退出了它所处的块)就失效;+ * 没有生效颜色的 draw 就是漏洞。++ 这样"同色画很多笔"不会误报,而"某个分支忘了设色"会被精确抓住。++ 颜色的判据是**两个 program 都算**(五官段用 SOLID,鼻涕泡用自己的 BUBBLE)。+ 更严的做法是给每一笔记住它属于哪个 program,但五官段里只有水滴这一笔换了+ program,为它建一套映射不划算;代价是"用了 A 却只设了 B 的颜色"这种错会漏掉,+ 而那种错在屏幕上是一眼能看出来的。+ """+ problems = []+ active = None # 当前生效颜色所在行的缩进+ for line_no, raw in rows:+ code = raw.split("//")[0].rstrip()+ if not code.strip():+ continue+ indent = len(code) - len(code.lstrip())+ if active is not None and indent < active:+ active = None+ if re.search(set_pat, code):+ active = indent+ if re.search(draw_pat, code) and active is None:+ problems.append(+ f"{label}:{line_no} 这一笔没有在自身分支内设颜色,"+ f"会继承上一笔的遗留色"+ )+ print(f" ✗ 第 {line_no} 行:{code.strip()[:70]}")+ return problems+++def parse_kotlin_face(text):+ """Mood.kt 的 FACE 表 → {mood: {字段: 值}},默认值补齐。"""+ out = {}+ for m in re.finditer(r"Mood\.(\w+) to Face\(", text):+ i, depth, j = m.end(), 1, m.end()+ while depth:+ if text[j] == "(":+ depth += 1+ elif text[j] == ")":+ depth -= 1+ j += 1+ # 顶层逗号切分(rgb(...) 里的逗号不算)+ parts, buf, d = [], "", 0+ for ch in text[i:j - 1]:+ if ch == "(":+ d += 1+ elif ch == ")":+ d -= 1+ if ch == "," and d == 0:+ parts.append(buf)+ buf = ""+ else:+ buf += ch+ parts = [p.strip() for p in parts if p.strip()]++ # Face 的位置参数顺序:color, eye, brow, smile, pulse, tilt, skew, jitter+ ORDER = ["c", "eye", "brow", "smile", "pulse", "tilt", "skew", "jitter"]+ vals = dict(FACE_DEFAULTS)+ for k, raw in zip(ORDER, parts):+ hexm = re.search(r"0x([0-9a-fA-F]+)", raw)+ vals[k] = int(hexm.group(1), 16) if k == "c" else float(raw.rstrip("f"))+ for raw in parts[len(ORDER):]:+ name, _, val = raw.partition("=")+ vals[name.strip()] = float(val.strip().rstrip("f"))+ out[m.group(1)] = vals+ return out+++def parse_lab_face(text):+ """实验室 FACE_NEW → {mood: {字段: 值}},默认值补齐。"""+ i = text.index("const FACE_NEW")+ body = text[i:text.index("\nconst MOODS", i)]+ out = {}+ for m in re.finditer(r"^\s*(\w+):\s*F\(\{([^}]*)\}\)", body, re.M):+ vals = dict(FACE_DEFAULTS)+ for kv in m.group(2).split(","):+ name, _, val = kv.partition(":")+ name, val = name.strip(), val.strip()+ if not name:+ continue+ vals[name] = int(val, 16) if val.startswith("0x") else float(val)+ out[m.group(1)] = vals+ return out+++def main():+ look_src = open(KT_LOOK).read()+ render_src = open(KT_RENDERER).read()+ mood_src = open(KT_MOOD).read()+ lab_src = open(LAB).read()++ problems = []++ print("── 参数 ──")+ # 默认值是**跨形象共用**的,解析一次就够。+ defaults = parse_kotlin_defaults(look_src)+ for look in CHECKS:+ k = dict(defaults)+ k.update(parse_kotlin_look(look_src, look.upper()))+ lab = parse_lab_preset(lab_src, look)+ n = 0+ for labk, ktk in PARAMS:+ a, b = k.get(ktk), lab.get(labk)+ ok = False+ if isinstance(a, list) and isinstance(b, list) and len(a) == len(b):+ ok = all(abs(x - y) < 1e-6 for x, y in zip(a, b))+ elif isinstance(a, float) and isinstance(b, float):+ ok = abs(a - b) < 1e-6+ if ok:+ n += 1+ else:+ problems.append(f"{look}.{ktk}: Kotlin={a} 实验室={b}")+ for ktk, fn in ENUMS:+ if k.get(ktk) == fn(lab):+ n += 1+ else:+ problems.append(f"{look}.{ktk}: Kotlin={k.get(ktk)} 期望={fn(lab)}")+ print(f" {look:<6} {n} / {len(PARAMS) + len(ENUMS)} 项一致")++ print("── 着色器(忽略注释与空白)──")+ K = kotlin_shaders(render_src)+ L = lab_shaders(lab_src)+ for name in SHADERS_IN_USE:+ lab_name = SHADER_ALIAS.get(name, name)+ a, b = K.get(name), L.get(lab_name)+ if a is None or b is None:+ problems.append(f"{name}: 一边缺失(实验室里叫 {lab_name})")+ print(f" ✗ {name:<10} 一边缺失")+ elif a == b:+ tag = f"(实验室里叫 {lab_name})" if lab_name != name else ""+ print(f" ✓ {name:<10} 语义一致{tag}")+ else:+ i = next((j for j in range(min(len(a), len(b))) if a[j] != b[j]),+ min(len(a), len(b)))+ problems.append(+ f"{name}: 语义不同,首个差异在第 {i} 字符\n"+ f" Kotlin: ...{a[max(0, i - 40):i + 60]}\n"+ f" 实验室: ...{b[max(0, i - 40):i + 60]}"+ )+ print(f" ✗ {name:<10} 语义不同")++ print("── 表情表 ──")+ KF, LF = parse_kotlin_face(mood_src), parse_lab_face(lab_src)+ if set(KF) != set(LF):+ problems.append(f"表情表的表情对不上:只在一侧有 "+ f"{sorted(set(KF) ^ set(LF))}")+ bad_rows = 0+ for mood in sorted(set(KF) & set(LF)):+ diffs = []+ for f in FACE_FIELDS:+ a, b = KF[mood].get(f), LF[mood].get(f)+ if a is None or b is None:+ diffs.append(f"{f}: 一侧缺失 ({a} / {b})")+ elif a != b:+ diffs.append(f"{f}: Mood.kt={a} 实验室={b}")+ if diffs:+ bad_rows += 1+ problems.append(f"{mood}: " + ";".join(diffs))+ print(f" ✗ {mood}")+ print(f" {len(KF) - bad_rows} / {len(KF)} 个表情逐字段一致")++ print("── 五官段:每一笔是否自带颜色 ──")+ problems += check_color_ownership(+ "EmotionBallRenderer.drawFace",+ slice_region(render_src, "private fun drawFace()", "\n private fun "),+ r'(?:solidShader|bubbleShader)\.uniform\("uColor"\)',+ r"\w+\.draw\(\)",+ )+ problems += check_color_ownership(+ "lab 五官段",+ slice_region(lab_src, "/* ---- 2. 五官 ---- */", "/* ---- 3. 腮红 ---- */"),+ r"(?:pSolid|pBubble)\.u\('uColor'\)",+ r"this\.m\w+\.draw\(\)",+ )+ if not any("自身分支内设颜色" in p for p in problems):+ print(" ✓ 两边每一笔都自带颜色")++ # 落地的不再是一个常量,而是一张可切换的表:用户在全屏卡片上左右滑动球球+ # 换形象,没存过就用第一个。这里要说清楚**哪些真的核对过**、哪些没有,以及+ # 有预设但画不全的那种 —— 三种状态混成一句"都查了"就是这篇脚本最该避免的事。+ m = re.search(r"val ALL = listOf\(([^)]*)\)", look_src)+ names = [n.strip().lower() for n in m.group(1).split(",")] if m else []+ lab_default = re.search(r"let curPreset=P_(\w+)", lab_src).group(1).lower()+ print("\n可切换的形象: " + "、".join(names) + "(默认第一个)")+ print("参数逐项核对过: " + ("、".join(CHECKS) if CHECKS else "(无)"))+ print("实验室默认选中: " + lab_default)+ for n in names:+ if n in CHECKS:+ continue+ if f"key:'{n}'" in lab_src:+ print(f" ⓘ {n} 有预设但没核对(不在 CHECKS 里)")+ else:+ print(f" ⓘ {n} 实验室里没有预设 —— 它的参数完全没核对")+ for n, blind in LAB_BLIND_SPOTS.items():+ if f"key:'{n}'" in lab_src:+ print(f" ⚠️ {n} 的预设是**部分**的:实验室画不出 {blind}。"+ f"脸可信,轮廓不可信")++ # 这份脚本**看不到网格生成器**(Capsule / Ear / Eye 这一族):它比的是参数表、+ # shader 语义、表情表和配色归属。同一个生成器在 Kotlin 与实验室各存一份,漂了+ # 这里一声不响 —— `Eye` 就是新加的那一个。所以先把"它管不到的那部分由谁管"+ # 说出来:跑一条命令得出"全部一致"很容易被读成"什么都查过了"。+ print("\n── 这份脚本看不到的,由谁管 ──")+ for rel, what in (+ ("tools/eye_parity_check.py", "gl/Eye.kt ↔ 实验室 eyeMesh:数字 / 算术 / 轴向逐项"),+ ("tools/eye_mesh_check.py", "Eye 网格本身合法 + 注释里写下的数"),+ ):+ mark = "✓" if (ROOT / rel).exists() else "✗ 缺失"+ print(f" {mark} {rel} —— {what}")++ if problems:+ print(f"\n❌ {len(problems)} 处不一致:")+ for p in problems:+ print(" ✗ " + p)+ return 1+ print("\n✅ 全部一致")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/cosmic_backdrop.py b/tools/cosmic_backdrop.pynew file mode 100644index 0000000..73e6185--- /dev/null+++ b/tools/cosmic_backdrop.py@@ -0,0 +1,124 @@+#!/usr/bin/env python3+"""星尘宇宙底片 —— 启动图标的自适应背景层。++## 为什么单独一个模块++底片不只生成器要用,**校验器也要用**:校验"角色在传统光栅图里占 88%"时,+原来靠 `a[1,1]` 取一个背景纯色再找和它不同的像素 —— 那招在渐变底上直接失效+(整个背景都和角点不一样,角色 bbox 会涨到整张图)。所以校验器改成"和纯底片+相减",于是底片必须是两边都能调到的同一份实现。++## 两条硬约束++1. **确定性。** 同一 (size, seed) 必须逐像素一样。图标是提交进仓库的产物,+ 每次重跑都在 diff 里炸出一堆无意义的字节变更,是很实在的噪音。+2. **亮度压在球体之下。** 球体主体亮度实测 ≈130。底片再好看也不能跟球抢 ——+ 目标是把辉光峰值压在 100 以下,给球留出对比度。这不是审美,是"小尺寸下+ 球还认不认得出"的问题,所以校验器里有一条专门量它(见 check 8)。+"""+import numpy as np+from PIL import Image++SEED = 20260930++# 深空紫:中心偏紫,边缘近黑+CORE = (46, 26, 92)+EDGE = (9, 6, 19)+# 品红辉光,垫在球后面 —— 球挡住中心,露出来的正好是一圈光晕+GLOW = (214, 74, 172)+GLOW_STRENGTH = 0.42+# 低频紫云,给渐变加一点层次,免得看上去像"圆的线性渐变"+#+# ⚠️ 第一版是 n=24 / 强度 0.16,结果**看出「拼布」**了:每 18px 一个格子、格边可见,+# 中间再被品红辉光一放大,读起来像图片压缩块,不像星云。教训是这类"随机场放大"+# 的**格子尺寸**才是观感的主导 —— 幅度只有 ±10 几个色阶,眼睛照样抓得住连贯结构。+# 所以 n 从 24 降到 7(格子 ~62px,比球还大),强度同步减半:要的是**光的不均匀**,+# 不是**纹理**。+NEBULA = (120, 70, 200)+NEBULA_STRENGTH = 0.085+VIGNETTE = 0.30+STAR_N = 110+++def _stars(size: int, seed: int) -> np.ndarray:+ """星点。核心 + 一圈淡光晕,大小/亮度/色偏都用同一个 rng 定,所以可重现。"""+ rng = np.random.default_rng(seed + 1)+ acc = np.zeros((size, size, 3), np.float64)+ scale = size / 432.0+ for _ in range(STAR_N):+ x = float(rng.random()) * size+ y = float(rng.random()) * size+ # **2.2 让大部分星很小、少数明显偏大 —— 均匀分布会得到"一片均匀的麻点"+ rr = (0.45 + float(rng.random()) ** 2.2 * 2.1) * scale+ b = 0.22 + 0.78 * float(rng.random()) ** 2.5+ t = float(rng.random())+ if t < 0.62:+ col = np.array([1.00, 1.00, 1.00])+ elif t < 0.84:+ col = np.array([0.78, 0.72, 1.00]) # 淡紫+ else:+ col = np.array([1.00, 0.74, 0.93]) # 淡粉+ col = col * b * 255.0+ for rad, k in ((rr, 1.0), (rr * 2.7, 0.20)):+ x0, x1 = max(0, int(x - rad - 1)), min(size, int(x + rad + 2))+ y0, y1 = max(0, int(y - rad - 1)), min(size, int(y + rad + 2))+ if x1 <= x0 or y1 <= y0:+ continue+ yy, xx = np.mgrid[y0:y1, x0:x1]+ d = np.hypot(xx + 0.5 - x, yy + 0.5 - y)+ w = np.clip(1.0 - d / max(rad, 1e-6), 0.0, 1.0)+ acc[y0:y1, x0:x1] += w[:, :, None] * col[None, None, :] * k+ return acc+++def render(size: int, ball_cx: float = None, ball_cy: float = None,+ ball_r: float = None, seed: int = SEED) -> Image.Image:+ """渲一张 size×size 的星尘底片。++ bit 中心 / 半径用来把辉光垫在球的正后方 —— 传 None 就退回画布中心。+ 传进来而不是"默认居中"是有理由的:自适应层里球被下移过 dy,辉光跟着+ 画布中心走就会和球错开,看上去像贴歪了的反光。+ """+ yy, xx = np.mgrid[0:size, 0:size]+ cx = float(ball_cx) if ball_cx is not None else (size - 1) / 2.0+ cy = float(ball_cy) if ball_cy is not None else (size - 1) / 2.0+ R = float(ball_r) if ball_r else size * 0.30++ # 1) 径向渐变+ d = np.hypot(xx - cx, yy - cy)+ t = np.clip(d / (R * 2.6), 0.0, 1.0) ** 0.85+ img = np.zeros((size, size, 3), np.float64)+ for i in range(3):+ img[:, :, i] = CORE[i] * (1.0 - t) + EDGE[i] * t++ # 2) 品红辉光+ g = np.exp(-0.5 * (d / max(R * 0.92, 1e-6)) ** 2) * GLOW_STRENGTH+ img += g[:, :, None] * np.array(GLOW, np.float64)++ # 3) 低频紫云:小尺寸随机场放大 —— 用 BICUBIC 插值,确定性+ rng = np.random.default_rng(seed)+ n = 7+ low = rng.random((n, n))+ cloud = np.asarray(+ Image.fromarray((low * 255).astype(np.uint8)).resize((size, size), Image.BICUBIC)+ ).astype(np.float64) / 255.0+ cloud = cloud - cloud.mean()+ img += cloud[:, :, None] * NEBULA_STRENGTH * np.array(NEBULA, np.float64)++ # 4) 星点+ img += _stars(size, seed)++ # 5) 暗角(放在星点之后,角落的星也跟着压暗,像真的景深)+ v = np.clip(d / (size * 0.72), 0.0, 1.0) ** 2 * VIGNETTE+ img *= (1.0 - v)[:, :, None]++ return Image.fromarray(np.clip(img, 0, 255).astype(np.uint8), "RGB")+++if __name__ == "__main__":+ import sys+ from pathlib import Path+ out = Path(sys.argv[1] if len(sys.argv) > 1 else "/tmp/shot/cosmic.png")+ out.parent.mkdir(parents=True, exist_ok=True)+ render(432).save(out)+ print(f"底片 -> {out}")diff --git a/tools/cutie_lab_probe.js b/tools/cutie_lab_probe.jsnew file mode 100644index 0000000..5afba38--- /dev/null+++ b/tools/cutie_lab_probe.js@@ -0,0 +1,231 @@+/* 奶油球的耳朵 / 眉毛 —— 探针。+ *+ * ⚠️ **尚未在本机跑通过。** 写它的时候 headless Chrome 在这台机器上一个空白+ * 页都 dump 不出来(90 秒超时),所以下面这段代码没有被真实渲染验证过一遍。+ * 配套的 tools/verify_cutie_lab.py 里有同样的提示和判据阈值。+ *+ * 这个项目里"看不见图"是常态,所以结论必须能在没有眼睛的情况下被论证。+ * 这一支只做一件事:把同一个奶油球渲染四遍,每遍只关掉一样东西,+ * 于是"耳朵到底出了轮廓没有"、"眉毛到底有没有改变眼睛上方那些像素"+ * 就变成了像素差,而不是形容词。+ *+ * 全开 ears on + brows on —— 要装到机器上的那个+ * 只关耳朵 ears off + brows on+ * 只关眉毛 ears on + brows off+ * 全关 ears off + brows off —— 等于改动之前的奶油球,也就是基线+ *+ * 四个的**相位是同一个**(同样的 90 帧、同样的起始 t、autoFidget 关掉),+ * 否则两张图之间那点差会混进呼吸和摇晃里,量出来的东西没法对账。+ *+ * 输出写进 <pre id="zh-probe">,由 tools/verify_cutie_lab.py 抽出来。+ */+(function(){+ const OUT={facts:{},measure:{},checks:[],rows:[]};+ const P=document.createElement('pre');+ P.id='zh-probe'; P.style.display='none';+ document.body.appendChild(P);+ function fail(e){ OUT.fatal=String(e&&e.stack||e); P.textContent=JSON.stringify(OUT); }++ try{+ const LAB=window.LAB;+ if(!LAB||!LAB.Ball||!LAB.edited) throw new Error('页面上没有 window.LAB');++ const SZ=420;+ const cv=document.createElement('canvas');+ cv.style.cssText='position:fixed;left:-9999px;top:0;width:'+SZ+'px;height:'+SZ+'px';+ document.body.appendChild(cv);++ const base=JSON.parse(JSON.stringify(LAB.edited.cream));++ /* ---- 一次快照 ---- */+ const IM={};+ function shoot(key, over){+ const params=Object.assign({}, base, over||{});+ const b=new LAB.Ball(cv, params);+ b.autoFidget=false;+ // 静止脸。耳朵和眉毛在 CALM 下最该被看清 —— 那是球一天里绝大多数+ // 时间的样子,也是"机灵"这个诉求真正要落的地方。+ b.mood='CALM';+ b.resize();+ let t=3.0;+ for(let i=0;i<90;i++){ t+=1/60; b.frame(1/60,t); }+ const w=cv.width, h=cv.height;+ const px=new Uint8Array(w*h*4);+ b.gl.readPixels(0,0,w,h,b.gl.RGBA,b.gl.UNSIGNED_BYTE,px);+ IM[key]={px,w,h,params:params,eyeTopY:null};+ return IM[key];+ }++ const both =shoot('both', {});+ const earOff=shoot('earOff',{earH:0});+ const browOff=shoot('browOff',{browW:0});+ const none =shoot('none', {earH:0, browW:0});++ /* ---- 像素工具。注意 readPixels 的第 0 行是**屏幕底部**,+ * 所以凡是给人看的东西(字符画、行号、质心高低)都要翻过来。 ---- */+ const w=both.w, h=both.h;++ function opaque(im,i){ return im.px[i*4+3]>128; }++ /** 不透明像素的最大半径(以图心为圆心)。球是居中的,透视下也一样。 */+ function maxRadius(im){+ const cx=w/2, cy=h/2;+ let rmax=0, n=0;+ for(let y=0;y<h;y++)for(let x=0;x<w;x++){+ if(!opaque(im,y*w+x)) continue;+ n++;+ const r=Math.hypot(x+0.5-cx, y+0.5-cy);+ if(r>rmax) rmax=r;+ }+ return {rmax:rmax,n:n};+ }++ /** 两张图的差异像素:数量、质心(**从上往下的行号**)、逐行分布。 */+ function diff(a,b){+ const ROWS=28;+ const rows=new Array(ROWS).fill(0);+ let n=0,sx=0,sy=0, maxd=0;+ const map=new Uint8Array(w*h);+ for(let y=0;y<h;y++)for(let x=0;x<w;x++){+ const i=(y*w+x)*4;+ const d=Math.abs(a.px[i]-b.px[i])+Math.abs(a.px[i+1]-b.px[i+1])+ +Math.abs(a.px[i+2]-b.px[i+2])+Math.abs(a.px[i+3]-b.px[i+3]);+ if(d>24){+ map[y*w+x]=1; n++; sx+=x; sy+=y;+ if(d>maxd) maxd=d;+ rows[Math.floor((h-1-y)/h*ROWS)]++;+ }+ }+ return {n:n, maxd:maxd,+ cx: n?sx/n:null,+ cyFromTop: n?(h-1-sy/n):null,+ rows:rows, map:map};+ }++ function art(im,cols,rows){+ const ramp=' .:-=+*#%@';+ const lines=[];+ for(let r=0;r<rows;r++){+ let s='';+ const y=Math.min(h-1,Math.floor((rows-1-r+0.5)*h/rows));+ for(let c=0;c<cols;c++){+ const x=Math.min(w-1,Math.floor((c+0.5)*w/cols));+ if(!opaque(im,y*w+x)){ s+=' '; continue; }+ const i=(y*w+x)*4;+ const L=(0.2126*im.px[i]+0.7152*im.px[i+1]+0.0722*im.px[i+2])/255;+ s+=ramp[1+Math.min(7,Math.floor(L*7.999))];+ }+ lines.push(s);+ }+ return lines;+ }++ function alphaArt(im,cols,rows){+ const lines=[];+ for(let r=0;r<rows;r++){+ let s='';+ const y=Math.min(h-1,Math.floor((rows-1-r+0.5)*h/rows));+ for(let c=0;c<cols;c++){+ const x=Math.min(w-1,Math.floor((c+0.5)*w/cols));+ s += opaque(im,y*w+x) ? '#' : ' ';+ }+ lines.push(s);+ }+ return lines;+ }++ function mapArt(map,cols,rows){+ const lines=[];+ for(let r=0;r<rows;r++){+ let s='';+ const y=Math.min(h-1,Math.floor((rows-1-r+0.5)*h/rows));+ for(let c=0;c<cols;c++){+ const x=Math.min(w-1,Math.floor((c+0.5)*w/cols));+ s += map[y*w+x] ? '#' : (opaque(both,y*w+x)?'.':' ');+ }+ lines.push(s);+ }+ return lines;+ }++ /* ---- 量 ---- */+ const R={both:maxRadius(both), earOff:maxRadius(earOff),+ browOff:maxRadius(browOff), none:maxRadius(none)};+ const D_ears=diff(both,browOff); // 只差在耳朵+ const D_brows=diff(both,earOff); // 只差在眉毛+ const D_all=diff(both,none); // 耳朵 + 眉毛++ const R_body=R.none.rmax; // 基线(无耳无眉)的最大半径 = 球 + 描边+ const outPx=R.both.rmax-R_body; // 耳朵伸出去多少像素++ // 眼睛中心在图上的行号(从上往下)。拿参数算,不猜。+ // 面平面上的 y 是向上为正,屏幕上也是;图心 = 0。+ // 位置到像素的换算:resize() 里 proj 是 42° 竖直 FOV、near 0.1、far 100,+ // 球心在原点,所以屏幕高度对应 2*tan(21°)*|z_cam|。球在 z=0,+ // 相机在 view 里后退了一段距离 —— 直接用基线球半径标定更省事:+ // 世界半径 R_body_world(含描边)≈ 0.845 + 0.03 对应 R_body 像素+ // 于是 1 世界单位 = R_body / R_body_world 像素。+ const OUTLINE_W=0.03; // 与页面里的描边厚度同量级+ const pxPerUnit=R_body/(both.params.bodyScale+OUTLINE_W);+ const eyeYpx=(h/2)-both.params.eyeY*pxPerUnit; // 行号(从上往下)+ const browYpx=(h/2)-(both.params.eyeY+both.params.browY)*pxPerUnit;++ OUT.facts={+ size:[w,h],+ bodyScale:both.params.bodyScale,+ eyeX:both.params.eyeX, eyeY:both.params.eyeY, eyeRadius:both.params.eyeRadius,+ browW:both.params.browW, browH:both.params.browH, browY:both.params.browY,+ browTilt:both.params.browTilt,+ earH:both.params.earH, earW:both.params.earW, earX:both.params.earX,+ earY:both.params.earY, earTilt:both.params.earTilt, earBend:both.params.earBend,+ earShade:both.params.earShade,+ pxPerUnit:pxPerUnit,+ bodyRadiusPx:R_body,+ eyeRowFromTop:eyeYpx,+ browRowFromTop:browYpx,+ };++ OUT.measure={+ radiusPx:{both:R.both.rmax, earOff:R.earOff.rmax,+ browOff:R.browOff.rmax, none:R.none.rmax},+ earOutPx:outPx,+ // 出轮廓的宽度 ÷ 球直径。这是"耳朵到底算不算长在轮廓外面"的唯一硬指标。+ earOutFracOfDiameter: outPx/(2*R_body),+ opaquePx:{both:R.both.n, earOff:R.earOff.n, browOff:R.browOff.n, none:R.none.n},+ // 有新像素才说明耳朵真的伸到了轮廓外;只是把轮廓内的一块换了个颜色+ // 会让这个数是 0,那就不叫"立耳"。+ earNewPx:D_ears.n,+ browDiffPx:D_brows.n,+ browDiffMaxDelta:D_brows.maxd,+ browDiffCentroidFromTop:D_brows.cyFromTop,+ browDiffRows:D_brows.rows,+ allDiffPx:D_all.n,+ allDiffRows:D_all.rows,+ // 球体自身有没有被意外动过:只在"轮廓内、且不在眉毛带"的地方找差异。+ // 眉毛带 = 眼睛中心往上到 browY 再往上留一点余量。+ unexpectedPx:(function(){+ let n=0;+ const bandTop=(h/2)-(both.params.eyeY+both.params.browY+0.12)*pxPerUnit;+ const rInside=R_body-3;+ const cx=w/2, cy=h/2;+ for(let y=0;y<h;y++)for(let x=0;x<w;x++){+ if(!D_all.map[y*w+x]) continue;+ const r=Math.hypot(x+0.5-cx, y+0.5-cy);+ if(r<=rInside && y>bandTop) n++; // 轮廓内的、眉毛带以下的差异+ }+ return n;+ })(),+ };++ OUT.rows=[+ {label:'奶油球 · 全开(要装机的样子)', ascii:art(both,68,34), alpha:alphaArt(both,68,34)},+ {label:'奶油球 · 基线(无耳无眉)', ascii:art(none,68,34), alpha:alphaArt(none,68,34)},+ {label:'差异图 · 耳朵(全开 vs 只关耳)', ascii:mapArt(D_ears.map,68,34), alpha:alphaArt(none,68,34)},+ {label:'差异图 · 眉毛(全开 vs 只关眉)', ascii:mapArt(D_brows.map,68,34), alpha:alphaArt(none,68,34)},+ ];++ OUT.version='v1';+ }catch(e){ fail(e); }++ document.getElementById('zh-probe').textContent=JSON.stringify(OUT);+})();diff --git a/tools/dump_faces_ascii.py b/tools/dump_faces_ascii.pynew file mode 100644index 0000000..9b6c440--- /dev/null+++ b/tools/dump_faces_ascii.py@@ -0,0 +1,121 @@+#!/usr/bin/env python3+"""把 faces_lab_ascii.js 注入设计稿,跑一遍,印出字符画 + 绘制调用计数。++这是"看不见图"时的眼睛:把 WebGL 画布解码成字符,用形状判断设计对不对。++用法:+ python3 tools/dump_faces_ascii.py # 全部+ python3 tools/dump_faces_ascii.py 暴躁 # 只印标签里含"暴躁"的+ python3 tools/dump_faces_ascii.py 小恶魔 alpha # 只印 alpha 图+"""+import json+import re+import subprocess+import sys+import tempfile+from pathlib import Path++ROOT = Path(__file__).resolve().parent.parent+PAGE = ROOT / "reports" / "ball-faces-lab.html"+SCRIPT = ROOT / "tools" / "faces_lab_ascii.js"+++def _find_chrome() -> str:+ """定位一个可用的 Chrome/Chromium。++ 不写死 macOS 路径:可用 CHROME_PATH 覆盖,否则按"macOS 常见位置 → PATH"依次找,+ 都找不到就退回 macOS 默认位置,好让"找不到 Chrome"的报错指向一个具体路径。+ """+ import os+ import shutil++ fallback = "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"+ candidates = [+ os.environ.get("CHROME_PATH"),+ fallback,+ "/Applications/Chromium.app/Contents/MacOS/Chromium",+ "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge",+ "/usr/bin/google-chrome",+ "/usr/bin/chromium",+ "/usr/bin/chromium-browser",+ "/snap/bin/chromium",+ ]+ candidates += [+ shutil.which(name)+ for name in ("google-chrome", "google-chrome-stable", "chromium", "chromium-browser", "chrome")+ ]+ for path in candidates:+ if path and Path(path).exists():+ return path+ return fallback+++CHROME = _find_chrome()++FLAGS = [+ "--headless=new",+ "--no-sandbox",+ "--enable-unsafe-swiftshader",+ "--window-size=1500,3000",+ "--hide-scrollbars",+]+++def main() -> int:+ filters = [a for a in sys.argv[1:] if a != "alpha"]+ want_alpha = "alpha" in sys.argv[1:]++ html = PAGE.read_text(encoding="utf-8")+ probe = SCRIPT.read_text(encoding="utf-8")+ html = html.replace("</body>", f"<script>\n{probe}\n</script>\n</body>", 1)++ tmp = Path(tempfile.mkdtemp(prefix="faces-ascii-"))+ h = tmp / "ascii.html"+ h.write_text(html, encoding="utf-8")++ proc = subprocess.run(+ [CHROME, *FLAGS, "--virtual-time-budget=20000", "--dump-dom", h.as_uri()],+ capture_output=True, text=True, timeout=240,+ )++ OUT = None+ for raw in re.findall(r'<pre id="zh-ascii"[^>]*>(.*?)</pre>', proc.stdout, re.S):+ raw = raw.strip()+ if raw.startswith("{"):+ try:+ OUT = json.loads(raw)+ except json.JSONDecodeError:+ continue+ if OUT is None:+ print("没有拿到字符画。", file=sys.stderr)+ err = re.search(r'<div id="err"[^>]*>(.*?)</div>', proc.stdout, re.S)+ if err and err.group(1).strip():+ print("页面错误框:", err.group(1)[:3000], file=sys.stderr)+ return 2+ if OUT.get("fatal"):+ print("探针抛异常:")+ print(OUT["fatal"])+ return 2++ print("绘制调用计数 (0 = 这个件根本没被调用;>0 但看不见 = 对比度/位置问题)")+ print("-" * 74)+ for n in OUT["notes"]:+ calls = " ".join(f"{k}={v}" for k, v in n["calls"].items())+ print(f" {n['label']:<28} {calls}")++ for row, n in zip(OUT["rows"], OUT["notes"]):+ if filters and not any(f in n["label"] for f in filters):+ continue+ print()+ print("=" * 74)+ print(f"{n['label']} [{n['key']} · {n['mood']} · {n['size'][0]}x{n['size'][1]}]")+ print("=" * 74)+ art = row["alpha"] if want_alpha else row["ascii"]+ for ln in art:+ print("|" + ln + "|")++ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/eye_mesh_check.py b/tools/eye_mesh_check.pynew file mode 100644index 0000000..94e18a7--- /dev/null+++ b/tools/eye_mesh_check.py@@ -0,0 +1,177 @@+"""切平的眼睛网格到底长什么样 —— 跑一遍,并对照注释里写下的数。++ python3 tools/eye_mesh_check.py # 退出码 1 = 网格不合法++为什么要有这个脚本:headless WebGL 在这台机器上是能跑的(2026-09-30 在+`tools/ball_shot.mjs` 里证过;原来"用不了"的结论是加了 `--disable-gpu` 造成的,+它会让 `getContext('webgl')` 返回 null)。**但"能渲染"并不等于"该走渲染"** ——+`eyeMesh` 是纯数学,抽出来在 Node 里跑一遍,几秒就能拿到环表、顶点数、索引范围,+而且失败信息能指到具体哪个环。渲染一条路贵得多,还得先建浏览器。+两条路的分工:这个脚本保证"形状是合法的、注释里那几个数是对的";+要看"好不好看"才需要去渲一张图。++它能证明的:网格自洽(没有 NaN、索引不越界、环是单调的),以及 `BallLook.IMP`+和 `gl/Eye.kt` 注释里那几个数**是对的**。它不能证明的:它好不好看。这条界要一直+说清楚,否则一次"跑绿了"很容易被读成"屏幕上看过了"。++⚠️ 它验的是**实验室那一份** `eyeMesh`。Kotlin 的 `gl/Eye.kt` 是它的原件,两边+不共用代码 —— 所以这个脚本只保证实验室画的形状是对的,两边还一致要靠人读。+真机上的几何没有等价的自动检查(那需要设备或模拟器)。+"""+import os+import pathlib+import subprocess+import tempfile++# 仓库根按脚本自身位置推断,别写死绝对路径 —— 写死过一次,别人 clone 下来+# 第一行就指向不存在的目录,且报错是"文件读不到",看不出是路径问题。+ROOT = pathlib.Path(__file__).resolve().parent.parent+html = (ROOT / "reports/ball-cutie-lab.html").read_text()++i = html.index("function eyeMesh(")+j = html.index("{", i)+depth, k = 0, j+while True:+ if html[k] == "{":+ depth += 1+ elif html[k] == "}":+ depth -= 1+ if depth == 0:+ break+ k += 1+fn = html[i:k + 1]++harness = fn + r"""+const DEG = 57.29578;+function audit(label, radius, cylH, topCut){+ const m = eyeMesh(radius, cylH, topCut, 4, 12);+ const {pos, nrm, uv, idx} = m;+ const bad = [];+ const finite = a => Array.prototype.every.call(a, Number.isFinite);+ if(!finite(pos)) bad.push("pos 里有 NaN/Inf");+ if(!finite(nrm)) bad.push("nrm 里有 NaN/Inf");+ if(!finite(uv)) bad.push("uv 里有 NaN/Inf");+ let maxIdx = -1;+ for(const v of idx) if(v > maxIdx) maxIdx = v;+ const vertCount = pos.length/3;+ if(maxIdx >= vertCount) bad.push(`索引越界 ${maxIdx} >= ${vertCount}`);+ // 每个被索引到的顶点都必须是有限值(没被索引到的尾部必须是全 0 —— 那正是+ // 类型化数组"多留了几格"的样子,不该出现非零垃圾)+ const used = new Set(Array.from(idx));+ for(const v of used) for(let c=0;c<3;c++)+ if(!Number.isFinite(pos[v*3+c])) bad.push(`顶点 ${v} 非有限`);+ let usedMax = -1;+ for(const v of used) if(v > usedMax) usedMax = v;+ for(let v=usedMax+1; v<vertCount; v++) for(let c=0;c<3;c++)+ if(pos[v*3+c] !== 0) bad.push(`未使用的顶点 ${v} 有非零位置`);++ // 环表:从位置里把 y 恢复出来(每个环有 stride 个点)+ const half = cylH/2, halfHeight = radius + half;+ const cut = Math.min(0.85, Math.max(0, topCut));+ const yCut = halfHeight*(1-cut);+ // 环的 y 序列:取每环第 0 个顶点+ const ys = [];+ for(let v=0; v<pos.length/3; v+=13){+ const y = pos[v*3+1];+ if(Number.isFinite(y) && y !== 0 && v < pos.length/3) {}+ }+ // 严格一点:按 stride 走+ const stride = 13;+ // 找出实际用到的环数:索引里最大顶点所在的环+ const ringCount = Math.floor((usedMax - 13) / stride) + 1; // 圆心在最后一环的 stride 上+ let rings = 0;+ for(let r=0; r<40; r++){+ const v = r*stride;+ if(v > usedMax) break;+ if(!used.has(v)) break;+ rings++;+ }++ // 结论数:眼睛的净形状+ let minY = Infinity, maxY = -Infinity, maxAbsX = 0, topWidth = 0;+ for(const v of used){+ const x = pos[v*3], y = pos[v*3+1], z = pos[v*3+2];+ if(y < minY) minY = y;+ if(y > maxY) maxY = y;+ if(Math.abs(x) > maxAbsX) maxAbsX = Math.abs(x);+ }+ // 平边的宽度:y 最接近 yCut 的那些顶点里 |x| 的最大值 ×2+ let edgeR = 0;+ for(const v of used){+ if(Math.abs(pos[v*3+1] - yCut) < 1e-5){+ const r = Math.hypot(pos[v*3], pos[v*3+2]);+ if(r > edgeR) edgeR = r;+ }+ }+ return {label, radius, cylH, topCut, cut, yCut, rings, vertCount, maxIdx,+ height: maxY-minY, topEdgeWidth: 2*edgeR, maxAbsX,+ bad,+ // 注释里声称的数+ lost: 1 - (maxY-minY)/(2*halfHeight)};+}++const IMP = audit("IMP eyeCut 0.48", 0.148, 0.055, 0.48);+const IMP_MAX = audit("IMP eyeCut 0.85 (走 else 支)", 0.148, 0.055, 0.85);+const IMP_CEIL = audit("IMP eyeCut 0.63 (瞳孔天花板)", 0.148, 0.055, 0.63);+const CREAM = audit("CREAM 未切", 0.122, 0.045, 0);+const TINY = audit("极小切口 0.02", 0.122, 0.045, 0.02);+const OVER = audit("超上限 1.5(应被夹到 0.85)", 0.148, 0.055, 1.5);+const NEG = audit("负值 -3(应被夹到 0)", 0.148, 0.055, -3);++const all = [IMP, IMP_MAX, IMP_CEIL, CREAM, TINY, OVER, NEG];+for(const a of all){+ console.log(+ `${a.label.padEnd(34)} rings=${String(a.rings).padStart(2)} verts=${String(a.vertCount).padStart(4)} ` ++ `maxIdx=${String(a.maxIdx).padStart(4)} 高=${a.height.toFixed(4)} 平边宽=${a.topEdgeWidth.toFixed(4)} ` ++ `切掉高度=${(a.lost*100).toFixed(1)}%` + (a.bad.length ? " ✗ " + a.bad.join(";") : " ✓"));+}++// ── 对照 BallLook.IMP / gl/Eye.kt 里写下的数 ──────────────────────────────+// 这些数写进了源码注释,也就是写进了**别人以后会相信的东西**。所以这里不是+// 打印一张"差不多"的表给人看 —— 对不上就退出码 1。容差 5e-4:注释里保留三到+// 四位小数,差过这个量级就说明注释该改了,或者代码改坏了。+const halfH = 0.148 + 0.055/2;+const pupilTop = 0.8*0.082;+const claims = [+ ["半高", 0.1755, halfH, 1e-4],+ ["0.48 的切面 y", 0.0913, IMP.yCut, 5e-5],+ ["瞳孔顶", 0.0656, pupilTop, 5e-5],+ ["切面上方的白边", 0.0257, IMP.yCut - pupilTop, 5e-5],+ ["平边宽", 0.267, IMP.topEdgeWidth, 5e-4],+ ["眼宽", 0.296, 2*0.148, 5e-4],+ ["切完的眼高", 0.267, IMP.height, 5e-4],+ ["12° 下平边落差", 0.056, IMP.topEdgeWidth*Math.sin(12/DEG), 5e-4],+ ["瞳孔天花板", 0.626, 1 - pupilTop/halfH, 5e-4],+];+let claimFails = 0;+console.log("\n── 注释里的数对不对 ──");+for(const [name, claimed, actual, tol] of claims){+ const ok = Math.abs(claimed - actual) <= tol;+ if(!ok) claimFails++;+ console.log(` ${ok ? "✓" : "✗"} ${name.padEnd(16)} 注释 ${String(claimed).padEnd(8)} 实测 ${actual.toFixed(4)}`);+}++// 两条分支的边界:yCut <= half 时走 else 支。IMP 的眼睛算出来是 0.8433。+const branchCut = 1 - (0.055/2)/halfH;+const inElse = c => (halfH*(1-c)) <= 0.055/2;+console.log("\n── 两条分支都在真实取值下够得到吗 ──");+console.log(` 分支边界 cut=${branchCut.toFixed(4)} ` ++ (inElse(branchCut + 0.001) && !inElse(branchCut - 0.001)+ ? "✓ 0.843 以上走 else、以下走 if —— 两支都活"+ : "✗ 边界算错了"));+console.log(` eyeCut 0.85 落在: ${IMP_MAX.yCut <= 0.055/2 ? "else 支(正确,会被 0.85 夹住)" : "if 支"}`);+console.log(` eyeCut 0.48 落在: ${IMP.yCut > 0.055/2 ? "if 支(上半球,正常路径)" : "else 支"}`);++const fails = all.filter(a => a.bad.length);+const bad = fails.length + claimFails;+console.log(`\n${bad ? "❌ " + bad + " 项不合格" : "✅ 7 个网格合法,9 个注释数字对得上"}`);+process.exit(bad ? 1 : 0);+"""++p = tempfile.NamedTemporaryFile("w", suffix=".js", delete=False)+p.write(harness); p.close()+r = subprocess.run(["node", p.name], capture_output=True, text=True)+print(r.stdout)+if r.stderr: print("STDERR:", r.stderr[:2000])+os.unlink(p.name)+raise SystemExit(r.returncode)diff --git a/tools/eye_parity_check.py b/tools/eye_parity_check.pynew file mode 100644index 0000000..1dd8063--- /dev/null+++ b/tools/eye_parity_check.py@@ -0,0 +1,218 @@+"""Eye 网格的两个副本是不是同一份 —— gl/Eye.kt 对 实验室的 eyeMesh。++为什么需要这个:`check_look_sync.py` 管的是**参数表、shader 语义、配色归属**,+它看不见网格生成器。于是"Kotlin 里改成 0.48、实验室忘了改"这类漂移没有任何+东西会拦。++⚠️ 这里原来写的是"这台机器上 headless Chrome 用不了,所以'两幅画看起来一样'+这条退路也是断的,只剩结构比对"。**前半句 2026-09-30 被证伪了**:headless WebGL+在这台机器上是能跑的(见 `tools/ball_shot.mjs` 的文件头,关键是**不要加+`--disable-gpu`**)。所以现在其实有两条路:这一份结构比对(仍然是最便宜、+最不依赖环境的一条),以及真去渲一张图。**但别把这条更正当成"这个脚本可以退休了"**+—— 它验证的是"两串源文本在剥掉注释后一致",那和"渲染结果一致"是两个不同的+命题,前者更弱但更早失败。++比对的是两串东西,都在剥掉注释之后取:++ A. **数字字面量序列**,按出现顺序。转写时最常错的是常数(0.85 / 2 / 1 / 3),+ 而常数错不会报错,只会让两边画出不同的形状。+ B. **算术记号序列**(cos / sin / acos / PI / 以及 clamp 的方向)。它管的是+ 公式的形状,比如"切的是 yCut 还是 half"。++两种语言的写法不同(`PI.toFloat()` vs `Math.PI`、`coerceIn(a,b)` vs+`Math.min(b,Math.max(a,·))`),所以有一个**显式的等价替换表**,见 SUBS。表短+且是穷尽的,替换表本身也会被打印出来 —— 免得"归一化到能对上"这件事被藏起来。++## 抓不到什么(别把它当全量证明)++1. **裸的符号翻转**。`1 - i/(ringCount-1)` 改成 `1 + i/...` 三个序列全绿 ——+ 数字没变、函数没变、标识符没变,变的只是那个 `+`。试过加一条"运算符序列",+ 不行:Kotlin 的 `if (y <= -half) -half else …` 与 JS 的三元 `y<=-half?-half:…`+ 是**同义改写**,运算符出现次数天然不同,实测 12 处假差异。收益盖不住成本,+ 所以这一项留白,靠人眼。(真要在意,`tools/eye_mesh_check.py` 里加一条+ 数值断言比在这里抠符号便宜。)+2. **语句顺序**。把"先算 uv 再算法线"调成反序,序列照样对得上。这个不影响+ 顶点数据,只影响可读性。+3. 这只比 `Eye.build` 与 `eyeMesh`。**调用方**(`EmotionBallRenderer` 选哪个+ 生成器、`BallLook.eyeCut` 的值)归 `tools/check_look_sync.py` 管。++## 它会被自己骗吗 —— 负向测试++写完先拿 5 处故意的改动试了一遍,4 处抓到(换常数 / 切面取反 / min-max 互换 /+轴向互换 `r*nx`↔`r*nz`),第 5 处就是上面第 1 条那个符号翻转。改这个脚本的+匹配规则之后**要重新做一遍这个负向测试** —— 一个永远返回 0 的检查比没有检查+更坏,因为它会让人放心。++退出码:0 = 两边同一份;1 = 有差异(差异逐条打印)。+"""++import pathlib+import re+import sys++ROOT = pathlib.Path(__file__).resolve().parent.parent+KT = ROOT / "app/src/main/java/net/kuafuai/andee/ui/ball/gl/Eye.kt"+LAB = ROOT / "reports/ball-cutie-lab.html"+++def slice_block(src: str, start: str) -> str:+ """从 `start` 起,按花括号配对切出整段(含首尾花括号)。"""+ i = src.index(start)+ j = src.index("{", i)+ depth, k = 0, j+ while True:+ if src[k] == "{":+ depth += 1+ elif src[k] == "}":+ depth -= 1+ if depth == 0:+ break+ k += 1+ return src[i : k + 1]+++def strip_comments(text: str) -> str:+ # 块注释、行注释。两边语法一样。+ text = re.sub(r"/\*.*?\*/", " ", text, flags=re.S)+ text = re.sub(r"//[^\n]*", " ", text)+ return text+++# 显式等价替换表:把两种语言里同一件事的不同写法归一。+# 每条都要能一眼看懂它在替什么,否则这个检查就变成"归一化到能对上"。+SUBS = [+ # 常量与类型+ (r"\bMath\s*\.\s*PI\b", "PI"),+ (r"\bPI\s*\.\s*toFloat\s*\(\s*\)", "PI"),+ # 数学函数+ (r"\bMath\s*\.\s*(cos|sin|acos|abs|min|max|hypot|floor)\b", r"\1"),+ # min/max 统一成大写,好和 coerceIn 归一后的 MIN/MAX 同框 —— 否则"夹取方向"+ # 这一项根本没被比到(Kotlin 一侧是大写、实验室一侧是小写,两组永远不相交)。+ (r"\bmin\s*\(", "MIN("),+ (r"\bmax\s*\(", "MAX("),+ # Kotlin 的夹取 / JS 的手写夹取,成对写法不同但语义相同+ (r"topCut\s*\.\s*coerceIn\s*\(\s*0f?\s*,\s*MAX_TOP_CUT\s*\)",+ "MIN(MAX_CUT,MAX(0,topCut))"),+ (r"\(\s*\(\s*yCut\s*-\s*half\s*\)\s*/\s*radius\s*\)\s*\.\s*coerceIn\s*\(\s*-1f?\s*,\s*1f?\s*\)",+ "MIN(1,MAX(-1,(yCut-half)/radius))"),+ (r"\(\s*ringCount\s*-\s*1\s*\)\s*\.\s*coerceAtLeast\s*\(\s*1\s*\)",+ "MAX(1,ringCount-1)"),+ # `MAX_TOP_CUT` / `MAX_CUT` 是同一个常数的两个名字+ (r"\bMAX_TOP_CUT\b", "MAX_CUT"),+ # `ArrayList<Float>(2 * capSegs + 3)` 的容量提示是 Kotlin 特有的、纯性能的+ # 参数,实验室的 `[]` 没有对应物 —— 它不影响一个顶点。整段吃掉,否则那两个+ # 数字(2、3)会把后面所有项错开。+ (r"ArrayList\s*<\s*Float\s*>\s*\(\s*[^()]*\)", "[]"),+ # Kotlin 的区间算符 `0..capSegs`:`..` 紧跟在数字后面,数字正则的边界断言会+ # 认为那是小数点在失败、于是这个 0 凭空消失(实验室写的是 `i=0;i<=capSegs`,+ # 它的 0 是独立记号)。四条区间正好差四项,长度对不上就是这么来的。+ (r"(?<=[\w.)])\s*\.\.\s*", " "),+ # 同一个变量的两种拼法+ (r"\bcylH\b", "cylinderHeight"),+ (r"\bonCyl\b", "onCylinder"),+ (r"\bth\b", "theta"),+ # 数字后缀+ (r"\b(\d+(?:\.\d+)?)[fF]\b", r"\1"),+]++NUM = re.compile(r"(?<![\w.])\d+(?:\.\d+)?(?![\w.])")+# 只认会改变几何形状的记号:函数与夹取方向。+MARK = re.compile(r"\b(cos|sin|acos|PI|MIN|MAX)\b")+# 第三个序列:**决定方向的那些标识符**,按出现顺序。+#+# 为什么数字序列不够:把 `r * nx` 写成 `r * nz` 是这颗网格最容易犯、也最难看出来+# 的错 —— 数字一项没变、函数一项没变,形状却整个转了个向(左右的法线/位置互换)。+# 只挑"轴与尺寸"这一小撮名字比,不把循环计数器 `i`/`j` 拉进来:Kotlin 的+# `for (i in 0 until n)` 与 JS 的 `for(i=0;i<n;i++)` 的出现次数本来就不一样,+# 混进来只会淹掉真正的差异。+AXIS = re.compile(+ r"\b(nx|nz|cy|radius|cylinderHeight|halfHeight|half|yCut|rCut|phiCut|"+ r"ringCount|stride|vertCount|capRing|capCenter)\b"+)+++def normalize(text: str) -> str:+ text = strip_comments(text)+ for pat, rep in SUBS:+ text = re.sub(pat, rep, text)+ # 把空白折成一个空格,让"换行/缩进不同"不算差异。+ #+ # 但**不能删光**:删光会把 `0.85` 和后面的 `fun` 粘成 `0.85fun`,数字正则的+ # 边界断言随之失效、那个常数就凭空消失了(这个检查自己踩过一次 —— 于是+ # "两边一致"和"这一项没被比"看起来一模一样)。提取是按记号走的,空格本来+ # 就不参与比对,留着它只赚不错。+ return re.sub(r"\s+", " ", text)+++def nums(text: str) -> list:+ return NUM.findall(text)+++def marks(text: str) -> list:+ return MARK.findall(text)+++def axis(text: str) -> list:+ return AXIS.findall(text)+++def diff_seq(label: str, a: list, b: list) -> int:+ """逐位比两条序列;返回差异条数。"""+ if a == b:+ print(f" {label}: 一致({len(a)} 项)")+ return 0+ print(f" {label}: ✗ 不一致 kotlin {len(a)} 项 / 实验室 {len(b)} 项")+ shown = 0+ for i in range(max(len(a), len(b))):+ x = a[i] if i < len(a) else "<无>"+ y = b[i] if i < len(b) else "<无>"+ if x != y:+ print(f" 第 {i} 项 kotlin={x} 实验室={y}")+ shown += 1+ if shown >= 12:+ print(" …(只列前 12 处)")+ break+ return 1+++def kotlin_span() -> str:+ """Kotlin 那边要拼两段才等于实验室的一个函数。++ `MAX_TOP_CUT` 在 Kotlin 里是 object 级的 `const val`(因为 [BallLook.eyeCut]+ 的 KDoc 要引它、而它又是 [Eye] 自己的上限),在实验室里是 `eyeMesh` 函数体里+ 的第一条 `const MAX_CUT`。位置不同、值必须同源,所以把那段声明插进函数体的+ 开头 —— 而且必须**插在同一个位置**:拼在最前面的话,两条数字序列会整体错开+ 一位,后面看到的全是假差异。+ """+ src = KT.read_text()+ m = re.search(r"const val MAX_TOP_CUT\s*=\s*[\d.]+f", src)+ if not m:+ raise SystemExit("Eye.kt 里找不到 MAX_TOP_CUT 的声明 —— 检查本身失效了")+ block = slice_block(src, "fun build(")+ brace = block.index("{") # 函数体的开括号(形参里没有花括号)+ return block[: brace + 1] + "\n" + m.group(0) + "\n" + block[brace + 1 :]+++def main() -> int:+ kt = normalize(kotlin_span())+ lab = normalize(slice_block(LAB.read_text(), "function eyeMesh("))++ print("归一化替换表(两种语言同一件事的不同写法):")+ for pat, rep in SUBS:+ print(f" {pat} → {rep}")+ print()++ bad = 0+ bad += diff_seq("数字字面量序列", nums(kt), nums(lab))+ bad += diff_seq("算术记号序列", marks(kt), marks(lab))+ bad += diff_seq("轴与尺寸标识符序列", axis(kt), axis(lab))++ if not bad:+ print("\n✅ gl/Eye.kt 与实验室 eyeMesh 是同一份(数字、算术、轴向逐项相同)")+ else:+ print("\n❌ 两边已经漂了 —— 改一处必须改两处")+ return 1 if bad else 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/faces_lab_ascii.js b/tools/faces_lab_ascii.jsnew file mode 100644index 0000000..0ae8119--- /dev/null+++ b/tools/faces_lab_ascii.js@@ -0,0 +1,123 @@+/* ASCII 验脸器 —— 在没有视觉的情况下"看"这张脸。+ *+ * 这个项目的一贯做法:把画布上的像素解码成字符画,用形状说话。+ * 这里额外做一件事:把渲染器的每个绘制函数包一层计数器,+ * 这样"这块件到底画了没有"就不用猜了。+ *+ * 输出写进 <pre id="zh-ascii">,由 tools/dump_faces_ascii.py 抽出来。+ */+(function(){+ const OUT={rows:[], notes:[], counts:{}};+ const P=document.createElement('pre'); P.id='zh-ascii'; P.style.display='none';+ document.body.appendChild(P);++ function fail(e){+ OUT.fatal=String(e&&e.stack||e);+ P.textContent=JSON.stringify(OUT);+ }++ try{+ const LAB=window.LAB;+ const css=document.createElement('style');+ css.textContent='.grid{grid-template-columns:repeat(3,1fr)!important}';+ document.head.appendChild(css);++ /* ---- 绘制计数器:包住 Ball.prototype 上的那批 draw* ---- */+ const B=LAB.Ball.prototype;+ const METHODS=['drawBrows','drawMouth','drawVein','drawSteam','drawEars','drawWedge','drawBody'];+ for(const m of METHODS){+ if(typeof B[m]!=='function'){ OUT.notes.push('缺少方法 Ball.'+m); continue; }+ OUT.counts[m]=0;+ const orig=B[m];+ B[m]=function(){ OUT.counts[m]++; return orig.apply(this,arguments); };+ }++ function stageOf(k){ return LAB.stages.find(s=>s.key===k); }++ /** 把一张画布变成字符画。alpha 低的当空白。+ *+ * **行序**:gl.readPixels 返回的是**从下往上**的行,所以第 0 行是屏幕底部。+ * 这里统一翻过来,让字符画的第 0 行就是屏幕顶部 —— 否则每看一次就要在脑子里+ * 倒一次,而"眉在上、牙在下"这类判断全靠方向。+ */+ function art(im,cols,rows){+ const {px,w,h}=im;+ const ramp=' .:-=+*#%@';+ const lines=[];+ for(let r=0;r<rows;r++){+ let s='';+ const y=Math.min(h-1,Math.floor((rows-1-r+0.5)*h/rows));+ for(let c=0;c<cols;c++){+ const x=Math.min(w-1,Math.floor((c+0.5)*w/cols));+ const i=(y*w+x)*4;+ const a=px[i+3]/255;+ if(a<0.06){ s+=' '; continue; }+ const L=(0.2126*px[i]+0.7152*px[i+1]+0.0722*px[i+2])/255;+ s+=ramp[1+Math.min(7,Math.floor(L*7.999))];+ }+ lines.push(s);+ }+ return lines;+ }++ /** 只画 alpha,用来看"哪些像素存在"(忽略亮度)。 */+ function alphaArt(im,cols,rows){+ const {px,w,h}=im;+ const lines=[];+ for(let r=0;r<rows;r++){+ let s='';+ const y=Math.min(h-1,Math.floor((rows-1-r+0.5)*h/rows));+ for(let c=0;c<cols;c++){+ const x=Math.min(w-1,Math.floor((c+0.5)*w/cols));+ const a=px[(y*w+x)*4+3]/255;+ s += a<0.06?' ': a<0.5?'.':'#';+ }+ lines.push(s);+ }+ return lines;+ }++ function render(key,over,mood,frames){+ const s=stageOf(key), b=s.ball;+ const params=Object.assign({}, JSON.parse(JSON.stringify(LAB.edited[key])), over||{});+ b.resize();+ b.setParams(params);+ b.mood=mood||(LAB.PRESETS.find(p=>p.key===key).demoMood||'CALM');+ b.autoFidget=false;+ let t=4.0;+ for(let i=0;i<(frames||60);i++){ t+=1/60; b.frame(1/60,t); }+ const w=b.cv.width,h=b.cv.height;+ const px=new Uint8Array(w*h*4);+ b.gl.readPixels(0,0,w,h,b.gl.RGBA,b.gl.UNSIGNED_BYTE,px);+ b.setParams(LAB.edited[key]);+ return {px,w,h};+ }++ const WANT=[+ ['imp', {}, 'CALM', '小恶魔 · 原样'],+ ['imp', {brow:0,fang:0,ticks:0}, 'CALM', '小恶魔 · 拆掉眉和牙'],+ ['imp', {brow:0}, 'CALM', '小恶魔 · 只拆眉'],+ ['imp', {browTilt:-0.30}, 'CALM', '小恶魔 · 眉反过来(委屈)'],+ ['grumpy', {}, 'TENSE', '暴躁球 · 原样'],+ ['grumpy', {steam:0,vein:0}, 'TENSE', '暴躁球 · 拆掉蒸汽和怒符'],+ ['grumpy', {teeth:0,gawp:0.20}, 'TENSE', '暴躁球 · 牙少一点、嘴小一圈'],+ ['vamp', {}, 'CALM', '吸血鬼球 · 原样'],+ ['vamp', {fang:0}, 'CALM', '吸血鬼球 · 拆掉獠牙'],+ ['neko', {}, 'HAPPY', '猫妖 · 原样'],+ ['cream', {}, 'CALM', '奶油球(基线)'],+ ];++ for(const [key,over,mood,label] of WANT){+ const before=Object.assign({},OUT.counts);+ const im=render(key,over,mood);+ const calls={};+ for(const m of METHODS) calls[m]=OUT.counts[m]-before[m];+ OUT.notes.push({label:label, key:key, mood:mood, size:[im.w,im.h],+ over:Object.keys(over), calls:calls});+ OUT.rows.push({label:label, ascii:art(im,68,30), alpha:alphaArt(im,68,30)});+ }++ for(const m of METHODS) delete OUT.counts[m];+ P.textContent=JSON.stringify(OUT);+ }catch(e){ fail(e); }+})();diff --git a/tools/faces_lab_probe.js b/tools/faces_lab_probe.jsnew file mode 100644index 0000000..abde8ce--- /dev/null+++ b/tools/faces_lab_probe.js@@ -0,0 +1,260 @@+/* =========================================================================+ 球球性格实验室 —— 像素级验收探针+ =========================================================================+ 把它注入到 lab 页面的 `</body>` 之前,用 headless Chrome 跑一遍,从页面上+ **量**出结果,而不是靠看。结果写进一个探针结果块(id = zh-probe),由+ `tools/verify_faces_lab.py` 抽出来。那个取结果的脚本要留意:探针的源码本身+ 也会出现在 --dump-dom 的结果里,所以它按"能解析成 JSON 的那一段"去取,+ 而不是取第一个匹配。++ 设计原则跟这个项目里别的地方一样:**每一句设计主张,都要有一个数。**+ 这里验收的是这几句:++ 1. 页面跑起来了,没有异常。(硬失败)+ 2. "可爱住在骨头上" —— 五套新预设的眼宽 / 眼睛高低 / 眼距 / 面平面 /+ 腮红大小,跟已经装机的奶油球**逐位相同**。+ 3. 眉真的画出来了 —— brow 从 0 拉到 1,眼睛上方那片区域必须有像素变化。+ 4. 牙真的画出来了 —— fang 0→1,嘴附近必须有像素变化。+ 5. 张嘴是**一块面**不是一根线 —— open 嘴的着色像素数必须远多于 arc 嘴。+ 6. "很邪恶但也很可爱" —— 眉/牙/怒符这些暗色件不能把整颗球吃掉:+ 球体自身的像素量不能被改变超过一个很小的比例。+ 7. 腹黑的**不对称**是真的 —— 左右镜像之后像素必须对不上。+ 8. 蒸汽在球顶**外面** —— 球顶以上必须出现非透明像素。+ ========================================================================= */+(function(){+ const R={version:(window.LAB&&window.LAB.version)||'?', checks:[], facts:{}, fatal:null};+ const F=document.createElement('pre');+ F.id='zh-probe';+ F.style.display='none';+ document.body.appendChild(F);++ function add(name,ok,detail,value){+ R.checks.push({name:name, ok:!!ok, detail:detail, value:(value===undefined?null:value)});+ }+ function done(){+ F.textContent=JSON.stringify(R);+ }++ try{+ if(!window.LAB) throw new Error('window.LAB 没出现:build() 大概抛异常了。err 框内容='+ + JSON.stringify((document.getElementById('err')||{}).textContent||''));+ const LAB=window.LAB, PRESETS=LAB.PRESETS;++ /* 把画布放大一点,不然 168px 上数不出东西。+ 改的是 CSS 而不是 canvas.width —— Ball 是读 clientWidth 的。 */+ const css=document.createElement('style');+ css.textContent=+ '.grid{grid-template-columns:repeat(3,1fr)!important}'++ '.cell .stage{aspect-ratio:1!important}'++ '.hstage canvas{width:520px!important}';+ document.head.appendChild(css);++ R.facts.presets=PRESETS.map(p=>p.key);++ /* ---------- 取样工具 ---------- */+ function stage(key){+ const s=LAB.stages.find(x=>x.key===key);+ if(!s) throw new Error('没有这个预设: '+key);+ return s;+ }+ /** 把一套参数装上去,settle 若干帧(tick 里有平滑,一帧不够),+ * 然后读回像素。读必须紧跟在 frame() 之后、在同一个任务里 ——+ * 上下文没开 preserveDrawingBuffer,一旦让出控制权缓冲就没了。 */+ function grab(key, params, mood, frames){+ const s=stage(key), b=s.ball;+ b.resize();+ if(params){ b.setParams(params); }+ b.mood=mood||(PRESETS.find(p=>p.key===key).demoMood||'CALM');+ b.autoFidget=false;+ const gl=b.gl, w=b.cv.width, h=b.cv.height;+ let tt=3.0;+ for(let i=0;i<(frames||48);i++){ tt+=1/60; b.frame(1/60,tt); }+ const px=new Uint8Array(w*h*4);+ gl.readPixels(0,0,w,h,gl.RGBA,gl.UNSIGNED_BYTE,px);+ return {px:px,w:w,h:h,ball:b};+ }+ function luma(p,i){ return 0.2126*p[i]+0.7152*p[i+1]+0.0722*p[i+2]; }+ function cloneParams(o){ return JSON.parse(JSON.stringify(o)); }+ /** 两层之间的差异统计。返回改变的像素数、最大差、以及改变区域的质心。 */+ function diff(a,bq,minDelta){+ const n=a.px.length; let cnt=0, mx=0, sx=0, sy=0;+ for(let i=0;i<n;i+=4){+ const dv=Math.abs(luma(a.px,i)-luma(bq.px,i));+ if(dv>=(minDelta||14)){+ cnt++; if(dv>mx) mx=dv;+ const q=i>>2; sx+=(q%a.w); sy+=Math.floor(q/a.w);+ }+ }+ return {count:cnt, max:mx,+ cx:cnt?sx/cnt:0, cy:cnt?sy/cnt:0,+ frac:cnt/(a.w*a.h)};+ }+ function opaque(im){+ let c=0, sx=0, sy=0;+ for(let i=3;i<im.px.length;i+=4) if(im.px[i]>28){ c++; const q=(i-3)>>2; sx+=q%im.w; sy+=Math.floor(q/im.w); }+ return {count:c, cx:c?sx/c:c/2, cy:c?sy/c:0, frac:c/(im.w*im.h)};+ }++ /* ---------- 1. 每套预设都能画出东西 ---------- */+ for(const p of PRESETS){+ const im=grab(p.key, LAB.edited[p.key], p.demoMood);+ const o=opaque(im);+ add('① '+p.label+' 画出了一颗球',+ o.frac>0.06 && o.frac<0.85,+ '不透明像素占画布 '+ (o.frac*100).toFixed(1) +'%',+ {frac:+o.frac.toFixed(4), cx:+(o.cx/im.w).toFixed(3), cy:+(o.cy/im.h).toFixed(3)});+ }++ /* ---------- 2. 骨架没动:可爱度是共享的 ---------- */+ const base=(LAB.edited.cream||{});+ const SKEL=['eyeR','eyeY','eyeX','faceZ','blushSize','blushX','blushY','eyeH'];+ const drift=[];+ for(const p of PRESETS){+ if(p.key==='cream') continue;+ const q=LAB.edited[p.key];+ for(const k of SKEL){+ if(base[k]===undefined) continue;+ if(Math.abs((q[k]||0)-(base[k]||0))>1e-9){+ drift.push(p.key+'.'+k+' '+q[k]+'≠'+base[k]);+ }+ }+ }+ const SKEL_KEYS=SKEL.map(k=>k+'='+base[k]).join(' ');+ add('② 五套新骨架 === 奶油球',+ drift.length===0,+ drift.length? ('漂了: '+drift.join(', ')) : ('逐位相同:'+SKEL_KEYS),+ {checked:SKEL.length*(PRESETS.length-1)});++ /* ---------- 3. 眉:拉起来必须真的多出像素 ---------- */+ {+ const im0=grab('imp', Object.assign(cloneParams(LAB.edited.imp),{brow:0}),'CALM');+ const im1=grab('imp', Object.assign(cloneParams(LAB.edited.imp),{brow:1}),'CALM');+ const d=diff(im0,im1,14);+ // 眉在眼睛**上方**:变化像素的质心必须在画面中线以上,而且大致居中+ const cyN=d.cy/im0.h, cxN=d.cx/im0.w;+ const up = cyN < 0.47;+ const mid= Math.abs(cxN-0.5) < 0.16;+ add('③ 眉画出来了,而且在眼睛上方',+ d.count>150 && up && mid,+ 'brow 0→1 改了 '+d.count+' px('+d.max.toFixed(0)+'/255),质心 y='+cyN.toFixed(3)+' x='+cxN.toFixed(3)++ '(要在中线以上、水平居中)',+ {count:d.count, cy:+cyN.toFixed(3), cx:+cxN.toFixed(3)});+ }++ /* ---------- 4. 牙:一颗和两颗都要有 ---------- */+ for(const [key,lbl] of [['imp','小恶魔(一颗虎牙)'],['vamp','吸血鬼(两颗獠牙)'],['neko','猫妖(ω 里的两颗)']]){+ const q=cloneParams(LAB.edited[key]);+ const im0=grab(key, Object.assign(cloneParams(q),{fang:0}), q.demoMood);+ const im1=grab(key, Object.assign(cloneParams(q),{fang:q.fang||2}), q.demoMood);+ const d=diff(im0,im1,14);+ const cyN=d.cy/im0.h;+ add('④ '+lbl+' 的牙画出来了',+ d.count>60 && cyN > 0.5,+ 'fang 0→'+(q.fang||2)+' 改了 '+d.count+' px,质心 y='+cyN.toFixed(3)+'(要在中线以下)',+ {count:d.count, cy:+cyN.toFixed(3)});+ }++ /* ---------- 5. 张嘴是一块面,不是一根线 ---------- */+ {+ // 用同一套"很厚"的嘴比:arc 嘴把管径加到 0.09,看谁面积大+ const arc=Object.assign(cloneParams(LAB.edited.grumpy),+ {mouthStyle:'arc', mouthTube:0.090, mouthMode:'mood'});+ const opn=Object.assign(cloneParams(LAB.edited.grumpy),+ {mouthStyle:'open', mouthMode:'mood', teeth:0, teethRows:1});+ const ia=grab('grumpy', arc,'CALM');+ const io=grab('grumpy', opn,'CALM');+ // 嘴的着色像素:用"深色且在下半脸"的像素数近似+ function mouthPixels(im){+ let c=0;+ for(let y=Math.floor(im.h*0.5); y<im.h; y++){+ for(let x=0;x<im.w;x++){+ const i=(y*im.w+x)*4;+ if(im.px[i+3]>128 && luma(im.px,i)<70) c++;+ }+ }+ return c;+ }+ const ma=mouthPixels(ia), mo=mouthPixels(io);+ add('⑤ 张嘴是一块面(不是一根粗线)',+ mo > ma*2.0 && mo>400,+ '同样"很厚"的设定下:arc 嘴 '+ma+' px,open 嘴 '+mo+' px,比 '+(mo/Math.max(1,ma)).toFixed(1)+'×',+ {arc:ma, open:mo, ratio:+(mo/Math.max(1,ma)).toFixed(2)});+ }++ /* ---------- 6. 邪恶件没有把可爱吃掉 ---------- */+ {+ const q=cloneParams(LAB.edited.imp);+ const stripped=Object.assign(cloneParams(q),{brow:0,fang:0,ticks:0,ears:false});+ const i0=grab('imp',stripped,'CALM');+ const i1=grab('imp',q,'CALM');+ const o0=opaque(i0), o1=opaque(i1);+ const d=Math.abs(o1.frac-o0.frac)/Math.max(1e-6,o0.frac);+ add('⑥ 加上眉/牙/勾之后,球体自身没有被改掉',+ d<0.06,+ '不透明像素占比 '+(o0.frac*100).toFixed(2)+'% → '+(o1.frac*100).toFixed(2)+'%(变了 '+(d*100).toFixed(2)+'%)',+ {before:+o0.frac.toFixed(4), after:+o1.frac.toFixed(4)});+ }++ /* ---------- 7. 腹黑的不对称是真的 ---------- */+ {+ const q=cloneParams(LAB.edited.smug);+ const sym=Object.assign(cloneParams(q),{browSkew:0});+ const is=grab('smug',q,'CURIOUS');+ const iy=grab('smug',sym,'CURIOUS');+ function mirrorDiff(im){+ let c=0;+ for(let y=0;y<im.h;y++){+ for(let x=0;x<Math.floor(im.w/2);x++){+ const a=(y*im.w+x)*4, b=(y*im.w+(im.w-1-x))*4;+ const da=im.px[a+3], db=im.px[b+3];+ if(da>128 && db>128 && Math.abs(luma(im.px,a)-luma(im.px,b))>18) c++;+ }+ }+ return c;+ }+ const ms=mirrorDiff(is), my=mirrorDiff(iy);+ add('⑦ 腹黑:左右镜像之后确实对不上',+ ms > my*2.0 && ms>120,+ 'skew 0.62 时镜像差异 '+ms+' px,skew 0 时 '+my+' px —— 不对称是这对眉做出来的',+ {skewed:ms, symmetric:my});+ }++ /* ---------- 8. 蒸汽在球顶外面 ---------- */+ {+ const q=cloneParams(LAB.edited.grumpy);+ const i0=grab('grumpy', Object.assign(cloneParams(q),{steam:0,vein:0}), q.demoMood);+ const i1=grab('grumpy', Object.assign(cloneParams(q),{steam:1,vein:1}), q.demoMood);+ // 只看画面上方 22% 那条带:球顶在那里已经结束,那里出现的不透明像素只能是蒸汽+ function topBand(im){+ let c=0;+ for(let y=0;y<Math.floor(im.h*0.22);y++)+ for(let x=0;x<im.w;x++) if(im.px[(y*im.w+x)*4+3]>60) c++;+ return c;+ }+ const t0=topBand(i0), t1=topBand(i1);+ add('⑧ 蒸汽出现在球顶以上的空白里',+ t1 > 60 && t1 > t0*3,+ '画面顶部 22% 那一条带:无蒸汽 '+t0+' px,有蒸汽 '+t1+' px',+ {off:t0, on:t1});+ }++ /* ---------- 附:怒符单独一格 ---------- */+ {+ const q=cloneParams(LAB.edited.grumpy);+ const i0=grab('grumpy', Object.assign(cloneParams(q),{vein:0,steam:0}), q.demoMood);+ const i1=grab('grumpy', Object.assign(cloneParams(q),{vein:1,steam:0}), q.demoMood);+ const d=diff(i0,i1,14);+ add('⑨ 怒符画出来了',+ d.count>80,+ 'vein 0→1 改了 '+d.count+' px,质心 y='+(d.cy/i0.h).toFixed(3),+ {count:d.count});+ }++ // 复原现场,免得后面的截图看到被改过的参数+ for(const p of PRESETS) stage(p.key).ball.setParams(LAB.edited[p.key]);++ }catch(e){+ R.fatal=String(e&&e.stack||e);+ }+ done();+})();diff --git a/tools/grounding_pair.py b/tools/grounding_pair.pynew file mode 100755index 0000000..80dacfa--- /dev/null+++ b/tools/grounding_pair.py@@ -0,0 +1,341 @@+#!/usr/bin/env python3+"""Turn the device's grounding log into labelled mis-tap / hit pairs.++ adb pull /sdcard/Android/data/net.kuafuai.andee/files/grounding ./grounding+ ./tools/grounding_pair.py ./grounding # summary+ ./tools/grounding_pair.py ./grounding -o pairs.jsonl # write the samples+ ./tools/grounding_pair.py ./grounding --pkg com.tencent.mm++## What it is looking for++The body's creed is that the model never does coordinate arithmetic: given an+element list it taps by e-number and the device resolves the pixels. Raw+0-1000 coordinates come out only where there is no list — mini-programs,+WebViews, game canvases — and that is exactly where a general-purpose model is+weakest. What it does there is miss, retry a little to the side, and hit. Each+of those sequences is a free training example: the screenshot it was looking+at, the point that was wrong, the point that was right.++## Why the labelling lives here and not on the device++Deciding *which* tap was a miss is a heuristic, and the first version of a+heuristic is wrong. On the device a wrong rule can only be fixed by shipping an+APK and re-collecting; here it can be fixed and re-run over logs already on+disk. So the device writes down only what it observed, and every judgement+below is this script's.++## The three labels, strongest first++`guard` The device's own stuck-loop guard refused the tap (`refused:true`).+ Its scope is exactly this data set — it counts coordinate taps in+ 120px buckets and exempts e-taps — so a bucket it tripped contains+ nothing but misses. Nothing here is inferred; the device had already+ worked it out and simply never wrote it down before.++`retry` Two coordinate taps at different points on the same screen with no+ navigation between them, and then the episode moves on. The first+ missed, the last hit. Weaker than `guard` because "moved on" is read+ from the event stream, not from the screen.++`diff` NOT IMPLEMENTED — there is no `--diff` flag, and no sample is ever+ emitted with this label. It is described here because the reason it+ was dropped is the reason not to add it back naively. The obvious+ test ("screen unchanged ⇒ the tap did nothing") does not work on this+ device: TapMarkerUi's cyan ring stays up for six seconds and is+ deliberately NOT blanked during a capture, so consecutive screenshots+ always differ by at least a ring. Worse, the ring sits on the tap+ point, which is where a successful tap's change would first appear.+ Anyone adding it has to mask that box first — which is what+ `marker_px` in the log and the `mask` field below are for. Even then+ it is corroboration, never a label on its own.++## One image for both points++Every sample carries a single screenshot — the last one before the *first*+tap of the sequence — and both the miss and the hit are expressed against it.+That is sound precisely because the misses missed: a tap that hit nothing left+the screen as it was, so the picture the brain was looking at when it chose the+wrong point is still the picture that was true when it chose the right one.+Where that assumption fails the sample is already excluded — a tap that *did*+change something is either followed by navigation (which cuts the run) or is+the last tap in it (the hit).++Stdlib only.+"""+import argparse+import json+import os+import sys+from collections import defaultdict++# A new episode when the screen has been idle this long. Matched to the+# device's own LOOP_WINDOW_MS: the stuck-loop guard remembers a bucket for five+# minutes, and cutting an episode sooner would strand a refusal in a different+# episode from the taps it is evidence about.+EPISODE_GAP_MS = 300_000++# The guard's bucket size, in device pixels — must stay equal to+# ScreenController.LOOP_RADIUS_PX. Bucketing is done on the logged `px`/`py`+# rather than on the 0-1000 values so this script reproduces the device's+# grouping exactly; 120px is a different number of relative units on every+# screen width, and guessing it wrong silently splits one hammered bucket into+# two and mislabels half of it.+LOOP_RADIUS_PX = 120++# Two taps closer than this are the same aim, not a correction — used only for+# the weaker `retry` label, which has no device-side bucket to match.+SAME_SPOT_NORM = 45++# Side of the square masked out around the last tap mark, in 0-1000 units.+# Sized against what TapMarkerUi actually DRAWS — a 30dp glow around the centre+# and a caption chip hanging ~55dp below it — not against its 144dp window,+# most of which is empty. There is no honest constant here: dp maps to a+# different number of relative units on every screen, so this covers the mark+# comfortably on the tablet this was written for and is the first thing to+# widen if a `diff` implementation starts seeing the ring as a change.+MARKER_MASK_NORM = 120+++def read_events(root):+ """Every JSONL generation, oldest first, in sequence order."""+ names = []+ for i in range(9, 0, -1):+ p = os.path.join(root, f"events.jsonl.{i}")+ if os.path.exists(p):+ names.append(p)+ p = os.path.join(root, "events.jsonl")+ if os.path.exists(p):+ names.append(p)+ if not names:+ sys.exit(f"no events.jsonl under {root}")++ out = []+ for name in names:+ with open(name, encoding="utf-8") as fh:+ for lineno, line in enumerate(fh, 1):+ line = line.strip()+ if not line:+ continue+ try:+ out.append(json.loads(line))+ except json.JSONDecodeError:+ # A power-off mid-append truncates the last line. One bad+ # line is not a reason to discard the file.+ print(f"skipping malformed {name}:{lineno}", file=sys.stderr)+ out.sort(key=lambda e: (e.get("ts", 0), e.get("seq", 0)))+ return out+++def split_episodes(events):+ """Contiguous runs on one package, cut at long gaps and at package changes."""+ episodes = []+ cur = []+ last_ts = None+ last_pkg = None+ for e in events:+ pkg = e.get("pkg")+ ts = e.get("ts", 0)+ gap = last_ts is not None and ts - last_ts > EPISODE_GAP_MS+ moved = pkg and last_pkg and pkg != last_pkg+ if cur and (gap or moved):+ episodes.append(cur)+ cur = []+ cur.append(e)+ last_ts = ts+ if pkg:+ last_pkg = pkg+ if cur:+ episodes.append(cur)+ return episodes+++def human_touched(episode):+ """Any user-driven navigation poisons the whole episode.++ Not squeamishness: the brain's next tap after a human pressed back is+ aimed at a screen the human chose, so reading it as a correction of the+ brain's previous tap would label a point that was never wrong.+ """+ return any(e.get("actor") == "user" for e in episode)+++def near(a, b):+ return abs(a[0] - b[0]) <= SAME_SPOT_NORM and abs(a[1] - b[1]) <= SAME_SPOT_NORM+++def last_shot_before(episode, idx):+ """The image the brain was looking at when it chose event `idx`."""+ for j in range(idx - 1, -1, -1):+ if episode[j].get("ev") == "shot":+ return episode[j]+ return None+++def pair_episode(episode):+ """Emit {image, miss[], hit, label_src} for each correction in one episode."""+ samples = []++ # --- label 1: the guard already knows -----------------------------------+ # Group coordinate taps into the same 120px-equivalent buckets the device+ # used. A bucket holding a refusal was hammered, so every coordinate tap in+ # it missed. The hit, if any, is whatever coordinate tap came next+ # elsewhere on the same screen.+ buckets = defaultdict(list)+ for i, e in enumerate(episode):+ if e.get("ev") == "tap" and e.get("mode") == "coord" and "px" in e:+ key = (e["px"] // LOOP_RADIUS_PX, e["py"] // LOOP_RADIUS_PX)+ buckets[key].append(i)+ refused_idx = set()+ for key, idxs in buckets.items():+ if not any(episode[i].get("refused") for i in idxs):+ continue+ refused_idx.update(idxs)+ shot = last_shot_before(episode, idxs[0])+ if not shot:+ continue+ miss = []+ for i in idxs:+ p = [episode[i]["nx"], episode[i]["ny"]]+ # The same point four times is one wrong answer repeated, not four+ # wrong answers — weight it once.+ if p not in miss:+ miss.append(p)+ hit = None+ for j in range(idxs[-1] + 1, len(episode)):+ ev = episode[j]+ if ev.get("ev") in ("swipe", "nav"):+ break+ if ev.get("ev") == "tap" and ev.get("mode") == "coord":+ if not near((ev["nx"], ev["ny"]), (miss[0][0], miss[0][1])):+ hit = [ev["nx"], ev["ny"]]+ break+ samples.append(make_sample(shot, miss, hit, "guard", episode))++ # --- label 2: it moved, then it moved on --------------------------------+ # A run of coordinate taps at distinct points with no navigation between+ # them. The brain kept aiming at the same thing and only the last one can+ # have worked, so everything before it is a miss.+ run = []+ for i, e in enumerate(episode):+ ev = e.get("ev")+ if ev == "tap" and e.get("mode") == "coord":+ if i in refused_idx:+ # Already emitted above under the stronger `guard` label. Cut+ # the run here rather than reaching across it: joining the taps+ # either side would re-label those same points, and the tap+ # that follows a hammered bucket is very often another miss,+ # not the hit this label assumes.+ samples.extend(flush_run(episode, run))+ run = []+ else:+ run.append(i)+ continue+ # Anything that changes the screen ends the run: after a scroll or a+ # back the earlier point does not refer to the same pixels.+ if ev in ("swipe", "nav") or (ev == "tap" and e.get("mode") == "eid"):+ samples.extend(flush_run(episode, run))+ run = []+ samples.extend(flush_run(episode, run))++ return [s for s in samples if s]+++def flush_run(episode, run):+ if len(run) < 2:+ return []+ pts = [(episode[i]["nx"], episode[i]["ny"]) for i in run]+ # All in one spot is a model re-pressing the same button, not correcting+ # its aim — no information about where the target actually was.+ if all(near(p, pts[0]) for p in pts[1:]):+ return []+ shot = last_shot_before(episode, run[0])+ if not shot:+ return []+ miss = [list(p) for p in pts[:-1]]+ hit = list(pts[-1])+ return [make_sample(shot, miss, hit, "retry", episode)]+++def make_sample(shot, miss, hit, label_src, episode):+ return {+ "image": shot.get("file"),+ "shot_id": shot.get("shot_id"),+ "image_w": shot.get("w"),+ "image_h": shot.get("h"),+ "pkg": shot.get("pkg") or next((e.get("pkg") for e in episode if e.get("pkg")), None),+ "miss": miss,+ "hit": hit,+ "label_src": label_src,+ # Where our own tap ring sits in the image, in 0-1000 units. Anything+ # comparing two screenshots must blank this box first — see the module+ # docstring.+ "mask": marker_mask(shot),+ }+++def marker_mask(shot):+ px = shot.get("marker_px")+ if not px or not shot.get("src_w") or not shot.get("src_h"):+ return None+ nx = px[0] * 1000 // shot["src_w"]+ ny = px[1] * 1000 // shot["src_h"]+ half = MARKER_MASK_NORM // 2+ return [max(0, nx - half), max(0, ny - half),+ min(1000, nx + half), min(1000, ny + half)]+++def main():+ ap = argparse.ArgumentParser(description=__doc__,+ formatter_class=argparse.RawDescriptionHelpFormatter)+ ap.add_argument("root", help="the pulled grounding/ directory")+ ap.add_argument("-o", "--out", help="write samples as JSONL here")+ ap.add_argument("--pkg", help="only this package")+ ap.add_argument("--keep-user", action="store_true",+ help="keep episodes a human steered (normally dropped)")+ args = ap.parse_args()++ events = read_events(args.root)+ episodes = split_episodes(events)++ kept, dropped_user, dropped_pkg = [], 0, 0+ for ep in episodes:+ pkg = next((e.get("pkg") for e in ep if e.get("pkg")), None)+ if args.pkg and pkg != args.pkg:+ dropped_pkg += 1+ continue+ if human_touched(ep) and not args.keep_user:+ dropped_user += 1+ continue+ kept.append(ep)++ samples = []+ for ep in kept:+ samples.extend(pair_episode(ep))++ by_src = defaultdict(int)+ for s in samples:+ by_src[s["label_src"]] += 1+ missing_image = sum(+ 1 for s in samples+ if s["image"] and not os.path.exists(os.path.join(args.root, s["image"]))+ )++ print(f"events {len(events)}")+ print(f"episodes {len(episodes)} "+ f"(kept {len(kept)}, user-steered {dropped_user}, other pkg {dropped_pkg})")+ print(f"samples {len(samples)} " ++ " ".join(f"{k}={v}" for k, v in sorted(by_src.items())))+ if missing_image:+ # Expected, not a bug: the PNG rotation is much tighter than the+ # JSONL's, so old events outlive their pictures.+ print(f" ({missing_image} reference a PNG that has since rotated away)")++ if args.out:+ with open(args.out, "w", encoding="utf-8") as fh:+ for s in samples:+ fh.write(json.dumps(s, ensure_ascii=False) + "\n")+ print(f"wrote {args.out}")+++if __name__ == "__main__":+ main()diff --git a/tools/license_audit.py b/tools/license_audit.pynew file mode 100644index 0000000..7adead8--- /dev/null+++ b/tools/license_audit.py@@ -0,0 +1,204 @@+#!/usr/bin/env python3+"""生成 THIRD_PARTY_NOTICES.md —— 依赖许可证清单。++为什么要脚本而不是手抄:ML Kit 会拖进 60 多个传递依赖,手工列必然漏,漏一个就是+一份不诚实的合规文件。++许可证来源优先级(都不猜,读不到就报出来):+ 1. 本地 Gradle 缓存里各 artifact 的 POM `<licenses>`+ 2. MANUAL 表 —— POM 里没写、已经到上游仓库 LICENSE 原文逐个核实过的+ 3. 都没有 → 列进 "Unresolved",不写进正文++用法:+ ./gradlew :app:dependencies --configuration debugRuntimeClasspath > /tmp/deps-runtime.txt+ ./gradlew :app:dependencies --configuration debugCompileClasspath > /tmp/deps-compile.txt+ python3 tools/license_audit.py /tmp/deps-runtime.txt /tmp/deps-compile.txt++输出写到一个路径由 -o 指定的文件,默认是仓库根目录的 THIRD_PARTY_NOTICES.md。++注意:依赖树里的版本要取 `->` **右边**那个(Gradle 会把冲突解析后的版本写在箭头后),+取错会把一堆旧版本写进清单。+"""+import argparse+import re+import sys+import xml.etree.ElementTree as ET+from collections import defaultdict+from pathlib import Path++HERE = Path(__file__).resolve().parent+DEFAULT_OUT = HERE.parent / "THIRD_PARTY_NOTICES.md"+CACHE = Path.home() / ".gradle" / "caches" / "modules-2" / "files-2.1"++COORD = re.compile(r"(?:[|+\\\- ]{4,})?([A-Za-z0-9_.\-]+):([A-Za-z0-9_.\-]+):([0-9][A-Za-z0-9_.\-]*)")++# POM 里没有 <licenses> 的,逐个到上游 LICENSE 原文核实过(核实日期 2026-09-29)+MANUAL = {+ ("com.github.mik3y", "usb-serial-for-android"): (+ "MIT", "https://github.com/mik3y/usb-serial-for-android/blob/master/LICENSE.txt"),+ ("com.google.auto.value", "auto-value-annotations"): (+ "Apache-2.0", "https://github.com/google/auto/blob/main/LICENSE"),+ ("com.google.guava", "listenablefuture"): (+ "Apache-2.0", "https://repo1.maven.org/maven2/com/google/guava/guava-parent/26.0-android/guava-parent-26.0-android.pom"),+ ("org.slf4j", "slf4j-api"): (+ "MIT", "https://github.com/qos-ch/slf4j/blob/master/LICENSE.txt"),+}++# 把 POM 里五花八门的写法归一到规范名+CANON = {+ "the apache software license, version 2.0": "Apache-2.0",+ "the apache license, version 2.0": "Apache-2.0",+ "apache 2.0": "Apache-2.0",+ "apache-2.0": "Apache-2.0",+ "mit license": "MIT",+ "mit": "MIT",+ "bsd license": "BSD-3-Clause",+ "ml kit terms of service": "ML Kit Terms of Service",+ "android software development kit license": "Android SDK License",+}++NOTES = {+ "Apache-2.0": "Permissive. Requires keeping the license text and any `NOTICE` file, and stating what you changed.",+ "MIT": "Permissive. Requires keeping the copyright notice and license text.",+ "BSD-3-Clause": "Permissive. Requires keeping the copyright notice; do not use the project's name to endorse your fork.",+ "Android SDK License": "**Not an open-source license.** Google's SDK terms govern redistribution of these `play-services-*` stubs.",+ "ML Kit Terms of Service": "**Not an open-source license.** The barcode-scanning SDK and its model are governed by Google's ML Kit Terms of Service, and the model is fetched at runtime from Google. Confirm this is acceptable for your redistribution.",+}++# 不随包发布、因此不进清单的(测试用)+TEST_ONLY = ("`junit:junit:4.13.2` (EPL-1.0)", "`androidx.test.espresso:espresso-core:3.5.1` (Apache-2.0)",+ "`androidx.test.ext:junit:1.1.5` (Apache-2.0)")+++def parse_tree(path: Path) -> dict:+ found = {}+ for line in path.read_text(encoding="utf-8", errors="replace").splitlines():+ if "(c)" in line: # 依赖约束,不是实际依赖+ continue+ m = COORD.search(line)+ if not m:+ continue+ group, artifact, version = m.groups()+ arrow = re.search(r"->\s*([0-9][A-Za-z0-9_.\-]*)", line)+ if arrow:+ version = arrow.group(1)+ found[(group, artifact)] = version+ return found+++def pom_path(group: str, artifact: str, version: str) -> Path | None:+ base = CACHE / group / artifact / version+ if not base.is_dir():+ return None+ poms = sorted(base.rglob(f"{artifact}-{version}.pom")) or sorted(base.rglob("*.pom"))+ return poms[0] if poms else None+++def licenses_of(pom: Path) -> list[tuple[str, str]]:+ try:+ root = ET.parse(pom).getroot()+ except ET.ParseError:+ return []+ ns = {"m": "http://maven.apache.org/POM/4.0.0"}+ out = []+ for lic in root.findall(".//m:licenses/m:license", ns) or root.findall(".//licenses/license"):+ name = (lic.findtext("m:name", default="", namespaces=ns) or lic.findtext("name", default="")).strip()+ url = (lic.findtext("m:url", default="", namespaces=ns) or lic.findtext("url", default="")).strip()+ if name:+ out.append((name, url))+ return out+++def render(coords: dict, out: Path) -> tuple[int, int, list[str]]:+ by_license = defaultdict(list)+ unresolved = []+ for (group, artifact), version in sorted(coords.items()):+ coord = f"{group}:{artifact}:{version}"+ manual = MANUAL.get((group, artifact))+ if manual:+ by_license[manual[0]].append((coord, manual[1]))+ continue+ p = pom_path(group, artifact, version)+ lics = licenses_of(p) if p else []+ if not lics:+ unresolved.append(coord)+ continue+ for raw, url in lics:+ by_license[CANON.get(raw.strip().lower(), raw.strip())].append((coord, url))++ total = sum(len(v) for v in by_license.values())+ order = sorted(by_license.items(), key=lambda kv: (-len(kv[1]), kv[0]))++ L = []+ L.append("# Third-party notices\n")+ L.append("Andee bundles the following third-party components. The list is generated, not hand-written — "+ "it enumerates the **resolved** dependency graph, so it includes transitive dependencies.\n")+ L.append("Regenerate it with:\n")+ L.append("```bash")+ L.append("./gradlew :app:dependencies --configuration debugRuntimeClasspath > /tmp/deps-runtime.txt")+ L.append("./gradlew :app:dependencies --configuration debugCompileClasspath > /tmp/deps-compile.txt")+ L.append("python3 tools/license_audit.py /tmp/deps-runtime.txt /tmp/deps-compile.txt")+ L.append("```\n")+ L.append("Licenses are read from each artifact's POM. Four artifacts declare nothing in their POM, and one "+ "inherits it from a parent; those were verified against the upstream repository instead and are "+ "included on that basis.\n")+ L.append("**Test-only dependencies are not listed here** because they are not shipped: "+ + ", ".join(TEST_ONLY) + ".\n")++ L.append("---\n")+ L.append(f"## Summary — {total} components\n")+ L.append("| License | Count | What it requires |")+ L.append("|---|---|---|")+ for lic, items in order:+ L.append(f"| {lic} | {len(items)} | {NOTES.get(lic, '')} |")+ L.append("")++ L.append("---\n")+ L.append("## Components by license\n")+ for lic, items in order:+ L.append(f"### {lic} ({len(items)})\n")+ if NOTES.get(lic):+ L.append(f"{NOTES[lic]}\n")+ L.append("| Component | License reference |")+ L.append("|---|---|")+ for coord, url in sorted(items):+ L.append(f"| `{coord}` | {url or '—'} |")+ L.append("")++ if unresolved:+ L.append("---\n")+ L.append("## Unresolved\n")+ L.append("Present in the dependency graph, but no license could be confirmed — so they are **not** "+ "claimed above. Resolve these before relying on this document:\n")+ for c in unresolved:+ L.append(f"- `{c}`")+ L.append("")++ out.write_text("\n".join(L) + "\n", encoding="utf-8")+ return total, len(by_license), unresolved+++def main() -> int:+ ap = argparse.ArgumentParser(description="生成依赖许可证清单")+ ap.add_argument("trees", nargs="+", help="`gradlew :app:dependencies` 的输出文件")+ ap.add_argument("-o", "--out", default=str(DEFAULT_OUT), help="输出路径")+ args = ap.parse_args()++ coords = {}+ for f in args.trees:+ coords.update(parse_tree(Path(f)))+ if not coords:+ print("没解析到任何坐标 —— 依赖树文件是不是空的?", file=sys.stderr)+ return 2++ out = Path(args.out)+ total, n_lic, unresolved = render(coords, out)+ print(f"{out}: {total} 个组件 / {n_lic} 种许可证", end="")+ print(f" / ⚠ {len(unresolved)} 个未决" if unresolved else " / 无未决")+ for c in unresolved:+ print(f" ⚠ {c}", file=sys.stderr)+ return 0+++if __name__ == "__main__":+ raise SystemExit(main())diff --git a/tools/make_launcher_icon.py b/tools/make_launcher_icon.pynew file mode 100644index 0000000..b89cf37--- /dev/null+++ b/tools/make_launcher_icon.py@@ -0,0 +1,526 @@+#!/usr/bin/env python3+"""从一张**透明底的角色渲染图**生成整套 Android 启动图标。++## 和「从美术稿做图标」那套流程的区别++那套流程的对象是别人画好的一张**不透明**方图,难点全在"把主体从背景里抠+出来":背景色是猜的、几何圆遮罩会糊边、五官要找亮度阈值。这一版不一样 ——+源图由 `tools/ball_shot.mjs` 从实验室那一页渲出来,**自带一路 alpha**:++ * **不用抠图**。alpha 就是剪影,边界就是美术自己的边界,没有"几何圆 vs+ 按背景抠"这个取舍,也不会有 1px 的羽化假边。+ * **不用猜背景色**。背景层是单独填的纯色,源图里根本没有背景。+ * **五官的阈值有干净的峰**。实测球内亮度直方图是三个分离的峰:眼睛 33±2+ (21070px)、嘴 78±2(2905px)、主体 120–215,峰与峰之间是空的。阈值落在+ 空隙里是量出来的,不是试出来的。++## 几何怎么定(没有一个是拍的)++**构图下移。** 角色的外接圆是歪的:耳朵在斜上方把它撑开,而**可见圆是以画布+中心为圆心的**。把角色整体下移,让"到画布中心的最大距离"最小 —— 实测最优+下移 43px,最大半径 357.9 → 323.6,**白赚 10.6%**。不做这一步,球还要再小+10%。++**球体尺寸不是自由变量。** 外接圆半径 / 球半径 = 1.154,因为耳朵伸在球外。+要让**整个角色**落进保证可见的 72dp 圆里,球就只能是角色高度的一个固定比例。+这是几何结论,不是缩水的选择 —— 肉眼看到的是"整个角色",而角色占可见圆的+比例几乎没变(95.8% → 96.5%)。++**两层用两套占比,不能互相缩。** 自适应层沿用被替换那套的 CONTENT=276/432+(先量旧图再沿用,换图才不会改变桌面上的视觉大小);传统光栅图**单独构建**+—— 从自适应画布缩下来会把"为遮罩预留的死环"一起烘进去,48px 下就是贴纸。+传统图按满幅惯例,角色占 88%(实测旧图 169/192 = 88.0%)。++## 单色层的语义++**黑 = 实体,透明 = 镂空**,不是"画一个白色剪影"。眼睛和嘴是 alpha=0 的洞。++### 剪影绝不二值化(这条踩过,而且错得不明显)++第一版写的是 `sil = alpha > 128`,结果单色层的剪影和美术层的 IoU 只有+**0.8611**,单色层还比美术层宽 16px(@432)。原因是把剪影当成了"一个实心形状",+而它其实不是 —— 球最外那一圈描边(`rimDark`)的 alpha 是一条**斜坡**:++ 球心行剖面,从最外沿往内:+ 202 168 109 86 65 53 44 37 30 24 20 16 14 11 9 8 7 | 118 231 233 ...++最外沿 202,向内一路衰减到 **7**,然后球体本体的 231 才开始。也就是说描边那一圈+(18px @1152)的 alpha 从 79% 掉到 3% —— 描边是靠最外那 1~2px 撑着的。++任何阈值都会把这条斜坡切成两段:阈值以上留一个细环、阈值以下丢掉整条斜坡,+中间断开。所以**单色层的 alpha 直接沿用源图的 alpha**,不二值化 ——+alpha 本来就是这个像素"被角色覆盖了多少",那正是 mask 要的语义。++眼睛和嘴**必须用 bbox 拟合椭圆 / 真实形状,不能用阈值 mask**:++ * 眼球里有一个亮度 252–255 的高光白点,阈值 mask 会在洞里留一个疙瘩。+ * 嘴是两端收细的弧,矩形框要么切掉嘴角,要么把背景一起拖走。+ * **耳朵会把嘴的阈值一起命中的坑**:耳朵亮度中位数 90,和嘴的 78 只差 12,+ 光靠亮度分不开。分开它们靠**位置** —— 耳朵整片都在球心上方。所以判据是+ `y > 球心` 而不是调阈值。++## 一次变换,不是每层各缩一次++所有层都从**同一张源图**、用**同一个 scale 和同一个整数 offset** 变换过去。+每层各缩一次(尤其遮罩单独缩)会让单色层挖掉的眼睛和美术层的眼睛差 1px ——+432px 上看不出来,108px 上一眼就歪。所以这里先在源图分辨率上把要合成的东西+合成完,再整张变换一次。++用法:+ python tools/make_launcher_icon.py --src a.alpha.png --check-only+ python tools/make_launcher_icon.py --src a.alpha.png --res app/src/main/res \\+ --dump-masks /tmp/masks.png+"""+import argparse+import sys+from pathlib import Path++import numpy as np+from PIL import Image, ImageDraw, ImageFilter++import cosmic_backdrop++# ---------------------------------------------------------------- 常量+DENSITIES = [("mdpi", 1.0), ("hdpi", 1.5), ("xhdpi", 2.0), ("xxhdpi", 3.0), ("xxxhdpi", 4.0)]+ADAPTIVE_DP = 108.0+LEGACY_DP = 48.0++# 保证可见圆:系统把画布中央 72/108 铺满图标位,外面那圈是给遮罩预留的。+VISIBLE_FRACTION = 72.0 / 108.0++# 角色到画布中心的最大半径,占可见圆半径的比例。**基准** 139/144 = 96.5%+# (沿用被替换那套的视觉大小,留的余量是给锐化的 —— 锐化会让主体略微外扩)。+#+# ⚠️ 这个 96.5% 有一个副作用,加了装饰之后才暴露出来:**装饰是加在球周围的**,+# 球占满可见圆的时候,底片只剩一圈细边,等于白加。所以 `--content` 是一个乘在+# 这个基准上的因子,用来给配景让出可见空间。缩小时**必须连传统光栅图一起缩**+# (见下),否则同一个设备上桌面图标和圆形图标会显示成两种大小。+CONTENT_OF_VISIBLE = 139.0 / 144.0++# 传统光栅图上"角色"占方形的比例。实测旧图 169/192 = 88.0%。+# 同样乘 `--content` —— 这两条路的角色大小在视觉上必须一致(见上)。+LEGACY_CONTENT = 0.88++BG = (14, 23, 43) # 实测旧图标 background 的纯色 #0E172B++# 源图上的阈值,落在实测的峰之间的空隙里。**空隙在哪要量,不能看着像就定。**+EYE_LUM = 60.0 # 眼睛 33±2,下一个峰在 76 → 60 落在空隙里+# ⚠️ 嘴这里踩过一次:原来写 120,而**球体自身的暗部底就是 120.6**(下半部亮度+# 分位 5%=78.9、10%=120.6、50%=124.1)。阈值贴着球体暗部底的结果是把它一起吞+# 进来当嘴:阈值 115→120 时"嘴"的像素从 589 涨到 725、被挖空覆盖从 89.3% 掉到+# 72.6%;120→125 更是从 725 暴涨到 6131。真正的谷底在 79 和 120 之间,取 100。+MOUTH_LUM = 100.0 # 嘴 78±2,球体暗部底 120.6 → 100 落在谷底正中+BALL_INNER = 0.90 # 只在球体半径的这个比例内找五官,把耳根挡在外面++SHARPEN_RADIUS = 1.0+SHARPEN_PERCENT = 60++# ---- 装饰:星尘宇宙 ----+# 球缘那圈紫粉边缘光,让它"接住"背景的辉光。宽度按球半径取比例(不是固定像素),+# 否则 432 画布上调好的宽度到 108 画布上就成了一整块糊。+RIM_COLOR = (188, 96, 226)+RIM_OF_R = 0.030+RIM_STRENGTH = 0.50+++def rim_tint(im: Image.Image, color, ball_r: float, of_r: float) -> Image.Image:+ """把剪影最外那薄薄一圈染向 color —— 模拟环境光打在边缘。++ **只改 RGB,不动 alpha。** 这既是"剪影绝不能碰"的老规矩,也是这次能加装饰+ 的前提:边缘光在剪影内部,所以可见圆的预算一点没动(前景层的最大半径不变)。++ 做法:`alpha>200` 腐蚀掉 of_r*ball_r 得到内芯,`实心 & ~内芯` 就是贴着边界的+ 那圈;再高斯模糊出柔和过渡。模糊会让它溢到剪影外,但那里 alpha=0,看不见。+ """+ a = np.asarray(im).astype(np.float64)+ # ⚠️ 必须转成 Python float —— ball_r 一路从 numpy 运算下来是 np.float64,+ # 而 PIL 的 GaussianBlur 里要 `if xy == (0, 0)`,np.float64 和元组比较会返回+ # 一个数组,于是抛 "truth value of an array is ambiguous"。报错点还落在 PIL 里,+ # 看着像是 PIL 的毛病。+ ball_r = float(ball_r)+ solid = a[:, :, 3] > 200+ core = solid.copy()+ for _ in range(max(1, int(round(ball_r * of_r)))):+ core = (core & np.roll(core, 1, 0) & np.roll(core, -1, 0)+ & np.roll(core, 1, 1) & np.roll(core, -1, 1))+ ring = solid & ~core+ w = Image.fromarray((ring * 255).astype(np.uint8), "L").filter(+ ImageFilter.GaussianBlur(max(0.6, ball_r * of_r * 0.55)))+ ww = np.asarray(w).astype(np.float64) / 255.0 * RIM_STRENGTH+ out = a.copy()+ for i in range(3):+ out[:, :, i] = a[:, :, i] * (1.0 - ww) + color[i] * ww+ return Image.fromarray(np.clip(out, 0, 255).astype(np.uint8), "RGBA")+++def backdrop(size: int, ball_cx: float, ball_cy: float, ball_r: float, decor: str):+ """底片。`none` 就是原来的纯色 —— 老样式要能原样复现,不能只有新样式。"""+ if decor == "cosmic":+ return cosmic_backdrop.render(size, ball_cx, ball_cy, ball_r)+ return Image.new("RGB", (size, size), BG)+++# ---------------------------------------------------------------- 工具+def resize_rgba(im: Image.Image, size: int) -> Image.Image:+ """缩放 RGBA。**必须走预乘**,否则透明像素的黑会渗进半透明边缘变成暗边。++ 这不是保险起见:源图是"深色描边 + 抗锯齿过渡",直通缩放时插值会把透明的+ 黑混进边缘像素,48px 上就是一圈脏边。+ """+ a = np.asarray(im).astype(np.float64)+ al = a[:, :, 3:4] / 255.0+ pm = Image.fromarray(np.dstack([a[:, :, :3] * al, a[:, :, 3]]).astype(np.uint8), "RGBA")+ b = np.asarray(pm.resize((size, size), Image.LANCZOS)).astype(np.float64)+ al2 = b[:, :, 3:4] / 255.0+ rgb = np.where(al2 > 1e-6, b[:, :, :3] / np.maximum(al2, 1e-6), 0.0)+ return Image.fromarray(np.dstack([np.clip(rgb, 0, 255), b[:, :, 3]]).astype(np.uint8), "RGBA")+++def resize_mask(im: Image.Image, size: int) -> Image.Image:+ """缩放灰度遮罩。用 BOX(面积平均)—— 遮罩是"覆盖了多少",面积平均才是它的+ 定义;LANCZOS 会在细笔画两侧振铃,把 5px 宽的嘴拆成一串亮点。"""+ return im.resize((size, size), Image.BOX)+++def sharpen_rgb(im: Image.Image) -> Image.Image:+ """只锐化 RGB,**绝不碰 alpha** —— 碰 alpha 会把剪影弄毛,而剪影是整图对比度+ 最高的地方,一锐就是一圈亮边。"""+ r, g, b, a = im.split()+ rgb = Image.merge("RGB", (r, g, b)).filter(+ ImageFilter.UnsharpMask(radius=SHARPEN_RADIUS, percent=SHARPEN_PERCENT, threshold=0)+ )+ r2, g2, b2 = rgb.split()+ return Image.merge("RGBA", (r2, g2, b2, a))+++def find_ball(alpha: np.ndarray):+ """从 alpha 求球体的圆心与半径。++ 球是最大的圆,且耳朵**不超出球在水平方向上的宽度**(耳朵往上长、往外撇,+ 但撇得没球宽),所以"最宽那一行的宽度"就是球直径。球底就是整张图的最下沿+ —— 耳朵只往上长。+ """+ ys, xs = np.where(alpha > 8)+ if len(ys) == 0:+ raise SystemExit("源图是空的 —— 全是透明像素")+ best = (0, None)+ for y in range(ys.min(), ys.max() + 1):+ xr = np.where(alpha[y] > 8)[0]+ if len(xr):+ w = xr.max() - xr.min() + 1+ if w > best[0]:+ best = (w, (int(xr.min()), int(xr.max())))+ d, (x0, x1) = best+ r = d / 2.0+ return (x0 + x1) / 2.0, float(ys.max()) - r, r, (+ int(ys.min()), int(ys.max()), int(xs.min()), int(xs.max()))+++def best_shift(ys, xs, H, W, search=140):+ """扫出那个让"到画布中心的最大距离"最小的下移量。++ 可见圆以画布中心为圆心,所以要最小化的就是"角色到画布中心的最大距离"+ —— 不是到球心的距离,也不是 bbox 的居中。+ """+ ccx, ccy = (W - 1) / 2.0, (H - 1) / 2.0+ base = float(np.hypot(xs - ccx, ys - ccy).max())+ best = (base, 0)+ for dy in range(-search, search + 1):+ d = float(np.hypot(xs - ccx, (ys + dy) - ccy).max())+ if d < best[0]:+ best = (d, dy)+ return base, best[1], best[0]+++def _components(mask: np.ndarray, min_px: int, limit: int = 6):+ """4-邻域连通域。不引 scipy —— 这个仓库的图标链路只依赖 numpy + PIL。"""+ H, W = mask.shape+ lab = np.zeros(mask.shape, np.int32)+ out = []+ cur = 0+ for sy, sx in zip(*np.where(mask)):+ if lab[sy, sx]:+ continue+ cur += 1+ stack = [(sy, sx)]+ lab[sy, sx] = cur+ pix = []+ while stack:+ y, x = stack.pop()+ pix.append((y, x))+ for ny, nx in ((y - 1, x), (y + 1, x), (y, x - 1), (y, x + 1)):+ if 0 <= ny < H and 0 <= nx < W and mask[ny, nx] and not lab[ny, nx]:+ lab[ny, nx] = cur+ stack.append((ny, nx))+ if len(pix) >= min_px:+ out.append(np.array(pix))+ out.sort(key=len, reverse=True)+ return out[:limit]+++def ellipse_mask(shape, box, grow=1.0):+ x0, y0, x1, y1 = box+ cxe, cye = (x0 + x1) / 2.0, (y0 + y1) / 2.0+ axe, aye = (x1 - x0 + 1) / 2.0 * grow, (y1 - y0 + 1) / 2.0 * grow+ H, W = shape+ yy, xx = np.mgrid[0:H, 0:W]+ return ((xx - cxe) / axe) ** 2 + ((yy - cye) / aye) ** 2 <= 1.0+++def close_mask(mask, iters=2):+ """形态学闭运算(先膨胀再腐蚀),把嘴的暗像素连成一片 —— 嘴是弧,内部亮度+ 并不完全均匀,直接取阈值会得到几段。"""+ m = mask+ for _ in range(iters):+ m = m | np.roll(m, 1, 0) | np.roll(m, -1, 0) | np.roll(m, 1, 1) | np.roll(m, -1, 1)+ for _ in range(iters):+ m = m & np.roll(m, 1, 0) & np.roll(m, -1, 0) & np.roll(m, 1, 1) & np.roll(m, -1, 1)+ return m+++def blit(dst: Image.Image, src: Image.Image, ox: int, oy: int):+ """把 src 贴到 dst 的 (ox, oy),负坐标也正确裁剪。++ **所有层都必须走这一个函数**,这是"各层共用同一次变换"在结构上的保证:+ 只要 offset 是同一组整数,单色层挖掉的眼睛就不可能和美术层的眼睛差+ 1px。每个调用点各写一遍 paste(或各自 round 一次),那个 1px 的偏差就会+ 回来 —— 432px 上看不出来,108px 上一眼就歪。+ """+ dw, dh = dst.size+ sw, sh = src.size+ x0, y0 = max(0, ox), max(0, oy)+ x1, y1 = min(dw, ox + sw), min(dh, oy + sh)+ if x1 <= x0 or y1 <= y0:+ return+ patch = src.crop((x0 - ox, y0 - oy, x1 - ox, y1 - oy))+ dst.paste(patch, (x0, y0), patch)+++def analyse(arr, quiet=False):+ """量出球体、最优下移、五官遮罩。返回一个 dict。"""+ alpha = arr[:, :, 3]+ H, W = alpha.shape+ cx, cy, r, bbox = find_ball(alpha)+ ys, xs = np.where(alpha > 8)+ base, dy, opt = best_shift(ys, xs, H, W)++ rgb = arr[:, :, :3]+ lum = 0.2126 * rgb[:, :, 0] + 0.7152 * rgb[:, :, 1] + 0.0722 * rgb[:, :, 2]+ yy, xx = np.mgrid[0:H, 0:W]+ d2c = np.hypot(xx - cx, yy - cy)+ inside = (alpha > 200) & (d2c < r * BALL_INNER)++ eye_hits = _components(inside & (yy < cy) & (lum < EYE_LUM), int(r * r * 0.004))+ eyes = sorted(+ [(int(p[:, 1].min()), int(p[:, 0].min()), int(p[:, 1].max()), int(p[:, 0].max()))+ for p in eye_hits[:2]],+ key=lambda e: (e[0] + e[2]) / 2,+ )+ # 嘴:靠**位置**和耳朵分开,不是靠亮度 —— 耳朵中位亮度 90 vs 嘴 78,阈值分不开;+ # 而耳朵整片在球心上方。+ mouth_hits = _components(inside & (yy > cy) & (lum < MOUTH_LUM), int(r * r * 0.004))+ mouth = mouth_hits[0] if mouth_hits else None++ eye_masks = [ellipse_mask((H, W), e, grow=1.06) for e in eyes]+ if mouth is not None:+ m = (lum < MOUTH_LUM) & (yy > cy) & (d2c < r * BALL_INNER)+ mouth_mask = close_mask(m, 2)+ else:+ mouth_mask = None++ if not quiet:+ print(f" 球体 球心 ({cx:.1f},{cy:.1f}) 半径 {r:.1f} 直径 {2*r:.0f}")+ print(f" bbox y[{bbox[0]},{bbox[1]}] x[{bbox[2]},{bbox[3]}]"+ f" 宽 {bbox[3]-bbox[2]+1} 高 {bbox[1]-bbox[0]+1}")+ print(f" 构图 画布中心最大半径 {base:.1f} → 下移 {dy}px 后 {opt:.1f}"+ f" (白赚 {(base-opt)/opt*100:.1f}%)")+ print(f" 外接圆/球半径 = {opt/r:.3f} —— 球在图标里的尺寸上限"+ f"(耳朵伸在球外,占掉这部分)")+ print(f" 五官 (眼<{EYE_LUM:.0f} / 嘴<{MOUTH_LUM:.0f}, 球内 {BALL_INNER:.0%}):")+ for i, e in enumerate(eyes):+ print(f" 眼{i+1} bbox x[{e[0]},{e[2]}] y[{e[1]},{e[3]}]"+ f" {e[2]-e[0]+1}×{e[3]-e[1]+1}")+ if mouth is not None:+ y0, x0 = mouth.min(axis=0)+ y1, x1 = mouth.max(axis=0)+ print(f" 嘴 {len(mouth)}px bbox x[{x0},{x1}] y[{y0},{y1}] {x1-x0+1}×{y1-y0+1}")+ else:+ print(" 嘴 没找到")+ v = lum[inside]+ print(f" 阈值余量 眼({EYE_LUM:.0f}) 与下一峰之间 {int(((v>EYE_LUM)&(v<70)).sum())}px")++ return dict(cx=cx, cy=cy, r=r, bbox=bbox, dy=dy, opt=opt, base=base,+ eyes=eyes, eye_masks=eye_masks, mouth_mask=mouth_mask,+ H=H, W=W, alpha=alpha)+++def dump_masks(arr, info, path):+ """把人眼核对用的遮罩图写出来。++ 这一步不是装饰:阈值落在空隙里是算出来的,但"算出来的洞是不是真的盖住了+ 眼睛"只有看图才知道。耳朵那一带尤其要看 —— 它的亮度和嘴只差 12。+ """+ H, W = info["H"], info["W"]+ base = Image.fromarray(np.clip(arr[:, :, :3], 0, 255).astype(np.uint8)).convert("RGB")+ ov = np.array(base).copy()+ for m in info["eye_masks"]:+ ov[m] = (ov[m] * 0.25 + np.array([255, 40, 40]) * 0.75).astype(np.uint8)+ if info["mouth_mask"] is not None:+ m = info["mouth_mask"]+ ov[m] = (ov[m] * 0.25 + np.array([40, 255, 90]) * 0.75).astype(np.uint8)+ # 球心十字 + 球轮廓+ im = Image.fromarray(ov)+ d = ImageDraw.Draw(im)+ cx, cy, r = info["cx"], info["cy"], info["r"]+ d.line([(cx - 40, cy), (cx + 40, cy)], fill=(255, 255, 0), width=4)+ d.line([(cx, cy - 40), (cx, cy + 40)], fill=(255, 255, 0), width=4)+ d.ellipse([cx - r, cy - r, cx + r, cy + r], outline=(0, 255, 255), width=3)+ d.ellipse([cx - r * BALL_INNER, cy - r * BALL_INNER,+ cx + r * BALL_INNER, cy + r * BALL_INNER], outline=(255, 0, 255), width=3)+ im.resize((760, 760), Image.LANCZOS).save(path)+ print(f" 遮罩核对图 -> {path}")+++# ---------------------------------------------------------------- 主流程+def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--src", required=True, help="透明底源图(ball_shot.mjs 的 .alpha.png)")+ ap.add_argument("--res", default="app/src/main/res")+ ap.add_argument("--check-only", action="store_true")+ ap.add_argument("--dump-masks", default=None)+ ap.add_argument("--no-sharpen", action="store_true")+ ap.add_argument("--decor", default="none", choices=("none", "cosmic"),+ help="背景装饰样式。none=原来的纯色底(默认,便于复现旧样式)")+ ap.add_argument("--dump-backdrop", default=None,+ help="把纯底片另存到该目录 —— 校验器要靠它把角色从渐变底上量出来")+ ap.add_argument("--content", type=float, default=1.0,+ help="角色大小因子(乘在 CONTENT_OF_VISIBLE / LEGACY_CONTENT 上)。"+ "1.0 = 沿用旧图视觉大小;调小给配景留空间。自适应层与传统层"+ "同比例缩,保证两种图标视觉一致")+ args = ap.parse_args()++ src = Image.open(args.src).convert("RGBA")+ arr = np.asarray(src).astype(np.float64)+ print(f"源图 {src.size}")+ info = analyse(arr)+ if args.dump_masks:+ dump_masks(arr, info, args.dump_masks)+ if args.check_only:+ return 0++ H, W, r, dy = info["H"], info["W"], info["r"], info["dy"]+ cx, cy = info["cx"], info["cy"]+ ccx, ccy = (W - 1) / 2.0, (H - 1) / 2.0+ bbox = info["bbox"]++ # ---- 在源图分辨率上把要合成的东西都做好,之后每层只做同一次变换 ----+ # 单色层的 alpha = **源图的 alpha 原样**,只把五官挖掉。+ # 不二值化,理由见文件头"剪影绝不二值化"那一段 —— 描边的 alpha 是一条+ # 从 202 衰减到 7 的斜坡,阈值会把它切成断开的两段。+ holes = np.zeros((H, W), bool)+ for m in info["eye_masks"]:+ holes |= m+ if info["mouth_mask"] is not None:+ holes |= info["mouth_mask"]+ mono = info["alpha"].copy()+ mono[holes] = 0.0+ mono_img = Image.fromarray(np.clip(mono, 0, 255).astype(np.uint8), "L")++ res = Path(args.res)+ outdirs = {n: res / f"mipmap-{n}" for n, _ in DENSITIES}+ for d in outdirs.values():+ d.mkdir(parents=True, exist_ok=True)+ bdir = Path(args.dump_backdrop) if args.dump_backdrop else None+ if bdir:+ bdir.mkdir(parents=True, exist_ok=True)++ written = []+ content = CONTENT_OF_VISIBLE * args.content+ legacy_content = LEGACY_CONTENT * args.content+ if args.content != 1.0:+ print(f"角色缩放因子 {args.content:.3g}"+ f" (自适应 {CONTENT_OF_VISIBLE*100:.1f}% → {content*100:.1f}% 可见圆,"+ f"传统 {LEGACY_CONTENT*100:.0f}% → {legacy_content*100:.1f}% 方形)")+ for name, dpi in DENSITIES:+ cvs = int(round(ADAPTIVE_DP * dpi))+ lgs = int(round(LEGACY_DP * dpi))++ # ============ 自适应层 ============+ # 变换的全部内容:把源图上那个**外接圆的圆心**(也就是源图中心+ # (ccx,ccy))对齐到画布中心,再整体下移 dy —— 那个 dy 就是上面为了+ # 让外接圆最小而算出来的量,不在这里用上它就等于白算。+ #+ # ⚠️ 这里踩过一次:写成 `cvs/2 + dy*scale - (cy+dy)*scale`,数学上+ # 化简成 `cvs/2 - cy*scale`,**dy 整个消失了** —— 432 画布上等于把+ # 角色上移了 17px,耳朵被顶出可见圆。所以下面把两项分开写。+ vis_r = cvs * VISIBLE_FRACTION / 2.0+ scale = (vis_r * content) / info["opt"]+ n = max(8, int(round(W * scale)))+ off_x = int(round(cvs / 2.0 - ccx * scale))+ off_y = int(round(cvs / 2.0 - ccy * scale + dy * scale))++ fg = Image.new("RGBA", (cvs, cvs), (0, 0, 0, 0))+ small = resize_rgba(src, n)+ blit(fg, small, off_x, off_y)+ bcx, bcy, br = off_x + cx * scale, off_y + cy * scale, r * scale+ if args.decor != "none":+ fg = rim_tint(fg, RIM_COLOR, br, RIM_OF_R)+ fg = fg if args.no_sharpen else sharpen_rgb(fg)+ bg_im = backdrop(cvs, bcx, bcy, br, args.decor)+ bg_im.save(outdirs[name] / "ic_launcher_background.webp", "WEBP", lossless=True, method=6)+ if bdir:+ bg_im.save(bdir / f"adaptive-{name}.png")+ fg.save(outdirs[name] / "ic_launcher_foreground.webp", "WEBP", lossless=True, method=6)++ mono_c = Image.new("L", (cvs, cvs), 0)+ blit(mono_c, resize_mask(mono_img, n), off_x, off_y)+ mc = np.asarray(mono_c)+ Image.fromarray(np.dstack([np.zeros_like(mc)] * 3 + [mc]), "RGBA").save(+ outdirs[name] / "ic_launcher_monochrome.webp", "WEBP", lossless=True, method=6)+ written += [outdirs[name] / f for f in+ ("ic_launcher_background.webp", "ic_launcher_foreground.webp",+ "ic_launcher_monochrome.webp")]++ # ============ 传统光栅(单独构建,不从自适应画布缩) ============+ char_h = bbox[1] - bbox[0] + 1+ char_w = bbox[3] - bbox[2] + 1+ lscale = lgs * legacy_content / max(char_h, char_w)+ ln = max(8, int(round(W * lscale)))+ lx = int(round(lgs / 2.0 - (bbox[2] + bbox[3] + 1) / 2.0 * lscale))+ ly = int(round(lgs / 2.0 - (bbox[0] + bbox[1] + 1) / 2.0 * lscale))+ lsmall = resize_rgba(src, ln)+ lc = Image.new("RGBA", (lgs, lgs), (0, 0, 0, 0))+ blit(lc, lsmall, lx, ly)+ if args.decor != "none":+ lc = rim_tint(lc, RIM_COLOR, r * lscale, RIM_OF_R)+ lc = lc if args.no_sharpen else sharpen_rgb(lc)+ flat = backdrop(lgs, lx + cx * lscale, ly + cy * lscale, r * lscale, args.decor)+ if bdir:+ flat.save(bdir / f"legacy-{name}.png")+ flat.paste(lc, (0, 0), lc)+ flat.save(outdirs[name] / "ic_launcher.webp", "WEBP", lossless=True, method=6)++ # 圆形版:同一个内容,按圆形给 alpha(旧图实测就是满幅圆)+ yy, xx = np.mgrid[0:lgs, 0:lgs]+ cc = (lgs - 1) / 2.0+ circ = Image.fromarray(+ ((np.hypot(xx - cc, yy - cc) <= cc + 0.5) * 255).astype(np.uint8), "L")+ rnd = Image.new("RGBA", (lgs, lgs), (0, 0, 0, 0))+ rnd.paste(flat, (0, 0), circ)+ rnd.save(outdirs[name] / "ic_launcher_round.webp", "WEBP", lossless=True, method=6)+ written += [outdirs[name] / "ic_launcher.webp", outdirs[name] / "ic_launcher_round.webp"]++ print(f" {name:8s} 画布 {cvs:3d} 传统 {lgs:3d} 球直径 {2*r*scale:5.1f}"+ f" ({2*r*scale/cvs*100:4.1f}% 画布) 角色最大半径 "+ f"{vis_r*content:5.1f} / 可见 {vis_r:.0f}")++ # 中间产物不留,免得残留的 .png 在某些路径上盖住 .webp+ for p in res.glob("mipmap-*/*.png"):+ p.unlink()+ total = sum(p.stat().st_size for p in written)+ print(f"\n共 {len(written)} 个文件,{total/1024:.1f} KiB")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/preview_launcher_icon.py b/tools/preview_launcher_icon.pynew file mode 100644index 0000000..9ed1cbc--- /dev/null+++ b/tools/preview_launcher_icon.py@@ -0,0 +1,172 @@+#!/usr/bin/env python3+"""把生成好的图标拼成一张预览图 —— 在**真实像素尺寸**下看,不是放大看。++为什么预览要单独做一步、而不是直接读那张 432px 的源:+ * 图标最终是**在 48px 上被看见**的,432px 上好看不代表 48px 上能认。+ * 系统会给自适应图标套遮罩(圆/方/圆角),套上之后会不会切到主体,只有+ 套一遍才知道。+ * 主题图标(单色层)与美术层是不是同一位置,要在**小尺寸**上对比才看得出来+ —— 差 1px 在 432 上看不见,在 108 上一眼就歪。++所以这张图上有三行:真实尺寸、套遮罩、单色层;每行都同时给出 192/144/96/72/48+五档,末尾把 48px 放大 6 倍(最近邻)供核对细节。++用法:+ python tools/preview_launcher_icon.py --res app/src/main/res --out /tmp/preview.png+"""+import argparse+from pathlib import Path++import numpy as np+from PIL import Image, ImageDraw++SIZES = [192, 144, 96, 72, 48]+CHECKER = 8+MASK_KINDS = ["circle", "squircle", "square"]++# 标签是中文,必须有中文字体。PIL 的默认位图字体只有 ASCII,不给字体的话+# 打出来是一串豆腐块 —— 预览图就白做了。+FONT_CANDIDATES = [+ "/System/Library/Fonts/Hiragino Sans GB.ttc",+ "/System/Library/Fonts/STHeiti Medium.ttc",+ "/System/Library/Fonts/Supplemental/Songti.ttc",+ "/Library/Fonts/Arial Unicode.ttf",+]+++def load_font(size=15):+ from PIL import ImageFont+ for p in FONT_CANDIDATES:+ if Path(p).exists():+ try:+ return ImageFont.truetype(p, size)+ except Exception:+ continue+ return ImageFont.load_default()+++def checkerboard(w, h, a=(58, 58, 64), b=(78, 78, 86)):+ img = Image.new("RGB", (w, h), a)+ d = ImageDraw.Draw(img)+ for y in range(0, h, CHECKER):+ for x in range(0, w, CHECKER):+ if ((x // CHECKER) + (y // CHECKER)) % 2:+ d.rectangle([x, y, x + CHECKER - 1, y + CHECKER - 1], fill=b)+ return img+++def mask_for(kind, n, ss=4):+ """在 4 倍上画遮罩再降采样 —— ImageDraw 没有抗锯齿。"""+ big = n * ss+ m = Image.new("L", (big, big), 0)+ d = ImageDraw.Draw(m)+ if kind == "circle":+ d.ellipse([0, 0, big - 1, big - 1], fill=255)+ elif kind == "squircle":+ # 圆角方形,半径取边长的 ~22% —— 各家 launcher 的常见值+ r = int(big * 0.22)+ d.rounded_rectangle([0, 0, big - 1, big - 1], radius=r, fill=255)+ else:+ d.rectangle([0, 0, big - 1, big - 1], fill=255)+ return m.resize((n, n), Image.LANCZOS)+++def compose(res: Path, density: str, size: int) -> Image.Image:+ """按系统的方式合成:**中央 72/108 铺满图标位**。++ 这一步最容易算错。系统不是把整张 108dp 画布缩到图标位,而是把画布的中央+ 72dp 铺满图标位、外面裁掉 —— 也就是画布要按 slot × 1.5 渲染。+ """+ fg = Image.open(res / f"mipmap-{density}/ic_launcher_foreground.webp").convert("RGBA")+ bg = Image.open(res / f"mipmap-{density}/ic_launcher_background.webp").convert("RGB")+ cvs = fg.size[0]+ full = bg.convert("RGBA")+ full.alpha_composite(fg)+ big = int(round(size * 108.0 / 72.0))+ big = max(big, 1)+ s = full.resize((big, big), Image.LANCZOS)+ c = big // 2+ return s.crop((c - size // 2, c - size // 2, c + size // 2, c + size // 2))+++def legacy(res: Path, density: str, name: str) -> Image.Image:+ return Image.open(res / f"mipmap-{density}/{name}.webp").convert("RGBA")+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--res", default="app/src/main/res")+ ap.add_argument("--out", default="/tmp/launcher-preview.png")+ ap.add_argument("--density", default="xxxhdpi")+ args = ap.parse_args()+ res = Path(args.res)++ ZOOM = 6+ PAD = 18+ row_h = 192 + 46+ labels = ["自适应层 · 真实尺寸(按中央 72/108 铺满)", "套圆形遮罩", "套圆角方形遮罩",+ "传统光栅图(满幅,系统再裁)", "主题图标(单色层)"]+ W = PAD + sum(s + PAD for s in SIZES) + 48 + 48 * ZOOM + PAD+ # ⚠️ 这里是 len(labels),不是写死的 4 —— 写死的时候第五行被画到了画布+ # 外面,单色层在预览里整个不见了,而数据其实是好的(差点去查错的方向)。+ H = PAD + row_h * len(labels) + PAD+ canvas = checkerboard(W, H)+ d = ImageDraw.Draw(canvas)+ font = load_font(15)++ y = PAD+ for label in labels:+ d.text((PAD, y - 16), label, fill=(240, 240, 245), font=font)+ x = PAD+ for sz in SIZES:+ base = Image.new("RGBA", (192, 192), (0, 0, 0, 0))+ if label.startswith("自适应层"):+ tile = compose(res, args.density, sz).convert("RGBA")+ elif label.startswith("传统"):+ tile = legacy(res, args.density, "ic_launcher")+ tile = tile.resize((sz, sz), Image.LANCZOS)+ elif label.startswith("主题"):+ m = legacy(res, args.density, "ic_launcher_monochrome")+ # 单色层的语义是"黑=实体、透明=镂空",在深色底上要反相才看得见+ arr = np.asarray(m).astype(np.uint8).copy()+ arr[:, :, :3] = 235+ m = Image.fromarray(arr, "RGBA")+ tile = m.resize((sz, sz), Image.LANCZOS)+ else:+ kind = "circle" if "圆形" in label else "squircle"+ tile = compose(res, args.density, sz).convert("RGBA")+ tile.putalpha(mask_for(kind, sz))+ base.alpha_composite(tile, (0, 0))+ canvas.paste(base, (x, y), base)+ d.rectangle([x, y, x + 191, y + 191], outline=(96, 96, 106))+ d.text((x, y + 198), f"{sz}px", fill=(200, 200, 210), font=font)+ x += sz + PAD+ # 最后一列:48px 放大 ZOOM 倍,最近邻,用来核对边缘细节+ x += 30+ big = Image.new("RGBA", (48, 48), (0, 0, 0, 0))+ if label.startswith("自适应层"):+ t = compose(res, args.density, 48).convert("RGBA")+ elif label.startswith("传统"):+ t = legacy(res, args.density, "ic_launcher").resize((48, 48), Image.LANCZOS)+ elif label.startswith("主题"):+ m = legacy(res, args.density, "ic_launcher_monochrome")+ arr = np.asarray(m).astype(np.uint8).copy()+ arr[:, :, :3] = 235+ t = Image.fromarray(arr, "RGBA").resize((48, 48), Image.LANCZOS)+ else:+ kind = "circle" if "圆形" in label else "squircle"+ t = compose(res, args.density, 48).convert("RGBA")+ t.putalpha(mask_for(kind, 48))+ big.alpha_composite(t)+ z = big.resize((48 * ZOOM, 48 * ZOOM), Image.NEAREST)+ canvas.paste(z, (x, y), z)+ d.text((x, y + 198), f"48px ×{ZOOM}(最近邻)", fill=(200, 200, 210), font=font)+ y += row_h++ canvas.save(args.out)+ print(f"预览 -> {args.out} ({canvas.size[0]}×{canvas.size[1]})")+ return 0+++if __name__ == "__main__":+ raise SystemExit(main())diff --git a/tools/screen_ascii.py b/tools/screen_ascii.pynew file mode 100644index 0000000..cc0a0b8--- /dev/null+++ b/tools/screen_ascii.py@@ -0,0 +1,64 @@+#!/usr/bin/env python3+"""把真机屏幕解码成字符画 —— "看不见图"时的眼睛。++和 dump_faces_ascii.py 同一个理由:这套球球是画出来的几何体,改参数不看效果+等于盲调。那个工具看的是实验室 HTML;这个看的是**真机** —— adb 截屏里就是+设备上实际合成出来的那一帧,零漂移。++ python3 tools/screen_ascii.py # 全屏,100 列+ python3 tools/screen_ascii.py 200 # 全屏,200 列+ python3 tools/screen_ascii.py 400 100 1400 950 # 裁剪 (左,上,右,下),120 列++只读不改。裁剪坐标是原始像素。+"""+import subprocess+import sys+from pathlib import Path++ADB = Path.home() / "Library/Android/sdk/platform-tools/adb"+RAMP = " .:-=+*#%@"+CHUNK = None # 每个字符覆盖的像素,按宽高比 1:2 采样+++def shot() -> bytes:+ p = subprocess.run([str(ADB), "exec-out", "screencap", "-p"],+ capture_output=True, timeout=30)+ if p.returncode != 0:+ sys.exit(f"adb screencap 失败: {p.stderr.decode(errors='replace')[:500]}")+ return p.stdout+++def main() -> int:+ args = [int(a) for a in sys.argv[1:]]+ crop = None+ if len(args) == 4:+ crop, args = args, []+ elif len(args) == 5:+ crop, args = args[1:], args[:1]+ width = args[0] if args else 100++ from PIL import Image+ import io+ img = Image.open(io.BytesIO(shot())).convert("L")+ if crop:+ img = img.crop(crop)+ w, h = img.size+ cols = min(width, w)+ # 终端字符约 1:2 高宽比,行数减半才是"不变形"+ rows = max(1, int(cols * h / w / 2))+ img = img.resize((cols, rows))+ # 自动拉伸对比:球上的五官(眼/鼻孔/嘴)只占身体亮度的一小段,不拉伸就+ # 全被冲进一两个灰阶里 —— 这个工具读的是形状,不是绝对亮度。+ px = img.load()+ lo, hi = img.getextrema()+ span = max(1, hi - lo)+ for y in range(rows):+ print("|" + "".join(+ RAMP[min(len(RAMP) - 1, (px[x, y] - lo) * (len(RAMP) - 1) // span)]+ for x in range(cols)) + "|")+ print(f"({w}x{h}px → {cols}x{rows} 字符)")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/verify_cutie_lab.py b/tools/verify_cutie_lab.pynew file mode 100644index 0000000..39c1919--- /dev/null+++ b/tools/verify_cutie_lab.py@@ -0,0 +1,223 @@+#!/usr/bin/env python3+"""把 cutie_lab_probe.js 注入奶油球实验室,用 headless Chrome 跑一遍,打印验收表。++⚠️ **这份工具还没在本机跑通过一次。** 写它的时候 headless Chrome 在这台机器上+连一个空白页都 dump 不出来(`--dump-dom` 90 秒超时,加不加沙箱都一样),所以+下面的判据阈值是**照着参数推的,不是量出来的**。第一次跑通之前,不要把它的+输出当作验收结论,也不要在提交信息里引用它。参数本身另有 `check_look_sync.py`+逐位核对,那一份是可靠的。++和 verify_faces_lab.py 同一套路(同一个 Chrome 定位逻辑、同一套+`--enable-unsafe-swiftshader`,**不要**加 `--disable-gpu`,那会让+getContext('webgl') 返回 null)。差别只在判据:这里量的是耳朵(和眉,如果这个+形象还有眉的话 —— 奶油球 2026-09-30 删掉了,见 `_brow_checks`)。++判据写在**这一侧**,不在探针里 —— 阈值是要反复调的,改 python 比改 JS 省事,+而且探针只负责"把数拿出来",职责单一。++用法:+ python3 tools/verify_cutie_lab.py [页面路径]+"""+import json+import re+import subprocess+import sys+import tempfile+from pathlib import Path++ROOT = Path(__file__).resolve().parent.parent+PAGE = Path(sys.argv[1]) if len(sys.argv) > 1 else ROOT / "reports" / "ball-cutie-lab.html"+PROBE = ROOT / "tools" / "cutie_lab_probe.js"++# 验收线。改了造型就得回来看这三个数,别让它们默默漂走。+MIN_EAR_OUT_FRAC = 0.08 # 耳朵出轮廓的宽度 ÷ 球直径+MIN_BROW_PX = 300 # 眉毛造成差异的像素数(68 列字符画下约 5 笔)+MAX_UNEXPECTED_PX = 40 # 轮廓内、眉毛带以下的差异 —— 球体本身不许被动+MIN_BROW_ABOVE_EYE_PX = 2.0 # 眉差异的质心必须比眼睛中心高多少像素+++def _find_chrome() -> str:+ import os+ import shutil++ fallback = "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"+ candidates = [+ os.environ.get("CHROME_PATH"),+ fallback,+ "/Applications/Chromium.app/Contents/MacOS/Chromium",+ "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge",+ "/usr/bin/google-chrome",+ "/usr/bin/chromium",+ "/usr/bin/chromium-browser",+ "/snap/bin/chromium",+ ]+ candidates += [+ shutil.which(name)+ for name in ("google-chrome", "google-chrome-stable", "chromium", "chromium-browser", "chrome")+ ]+ for path in candidates:+ if path and Path(path).exists():+ return path+ return fallback+++CHROME = _find_chrome()++FLAGS = [+ "--headless=new",+ "--no-sandbox",+ "--enable-unsafe-swiftshader",+ "--window-size=1500,3600",+ "--hide-scrollbars",+]+++def run_probe() -> dict:+ html = PAGE.read_text(encoding="utf-8")+ probe = PROBE.read_text(encoding="utf-8")+ if "</body>" not in html:+ raise SystemExit("页面里没有 </body>,不知道该把探针插在哪。")+ html = html.replace("</body>", f"<script>\n{probe}\n</script>\n</body>", 1)++ tmp = Path(tempfile.mkdtemp(prefix="cutie-lab-"))+ harness = tmp / "lab-verify.html"+ harness.write_text(html, encoding="utf-8")++ # Chrome 不给 --user-data-dir 就会去 ~/Library/Application Support/Google/+ # Chrome-headless 建一个临时 profile,跑完还要删掉它 —— 在受限环境里那一+ # 步会被删除规则拦下,表现为"整条命令没有任何输出"。给它一个自己的目录,+ # 写和清都在 temp 里完成。+ profile = tmp / "chrome-profile"+ proc = subprocess.run(+ [CHROME, *FLAGS, f"--user-data-dir={profile}",+ "--virtual-time-budget=20000", "--dump-dom", harness.as_uri()],+ capture_output=True, text=True, timeout=240,+ )+ dom = proc.stdout++ # 探针源码自己就在 DOM 里,注释里也有 `<pre id="zh-probe"` 这个字符串,+ # 所以不能取第一个匹配 —— 要取**能解析成 JSON** 的那一个。+ for raw in re.findall(r'<pre id="zh-probe"[^>]*>(.*?)</pre>', dom, re.S):+ raw = raw.strip()+ if not raw.startswith("{"):+ continue+ try:+ return json.loads(raw)+ except json.JSONDecodeError:+ continue++ print("探针没有留下结果 —— 页面大概在 build() 阶段就挂了。", file=sys.stderr)+ err = re.search(r'<div id="err"[^>]*>(.*?)</div>', dom, re.S)+ if err and err.group(1).strip():+ print("页面自己的错误框:", file=sys.stderr)+ print(err.group(1)[:4000], file=sys.stderr)+ raise SystemExit(2)+++def _brow_checks(m: dict, f: dict) -> list:+ """眉的那两条断言,按"这个形象到底有没有眉"来选。++ 奶油球 2026-09-30 把眉删掉了(browW 0)。探针的两组 A/B 里,`browOff` 是+ `{browW:0}`,而预设本身现在也是 0 —— 两次拍的是**同一个配置**,browDiffPx+ 恒为 0。原来那两条"看得见、且高于眼睛"的断言拿 0 去比 300px 的阈值,会稳定+ FAIL。阈值没写错,是世界变了,所以要改的是断言不是数。++ 分成两支:有眉查"看得见且位置对";没眉查反向的"关掉前后必须一模一样"。++ ⚠️ 说清第二支有多弱:它是**接近重言式**的 —— 两次拍同一配置,本来就该相等。+ 它挡不住"某个分支在偷偷画一笔",只挡得住"渲染本身不确定"(未初始化状态、+ 随机抖动)。真正的"删干净了没有"要靠字符画那一栏人眼看。别把它当强验证。+ """+ if f["browW"] > 0:+ return [+ ("眉毛在眼睛上方留下可见的一笔",+ m["browDiffPx"] >= MIN_BROW_PX,+ f"{m['browDiffPx']}px ≥ {MIN_BROW_PX}px,最大通道差 {m['browDiffMaxDelta']}"),+ ("眉差异的质心高于眼睛中心",+ (m["browDiffCentroidFromTop"] or 0) <= f["eyeRowFromTop"] - MIN_BROW_ABOVE_EYE_PX,+ f"质心行 {m['browDiffCentroidFromTop']:.1f} ≤ 眼睛行 {f['eyeRowFromTop']:.1f} "+ f"− {MIN_BROW_ABOVE_EYE_PX:.0f}"),+ ]+ return [+ ("本形象没有眉:关眉前后应完全一致(弱检查,见 docstring)",+ m["browDiffPx"] == 0,+ f"关眉前后差异 {m['browDiffPx']}px,应为 0"),+ ]+++def main() -> int:+ if not PAGE.exists():+ print(f"找不到页面:{PAGE}", file=sys.stderr)+ return 2+ if not Path(CHROME).exists():+ print(f"找不到 Chrome:{CHROME}", file=sys.stderr)+ return 2++ R = run_probe()+ if R.get("fatal"):+ print("页面/探针抛异常:", file=sys.stderr)+ print(R["fatal"], file=sys.stderr)+ return 2++ print(f"奶油球 · 耳朵验收 (探针 {R.get('version')})")+ print("=" * 72)++ m, f = R["measure"], R["facts"]+ print("─ 参数 ─")+ print(f" 球半径(含描边) {f['bodyRadiusPx']:.1f}px 1 世界单位 = {f['pxPerUnit']:.1f}px")+ print(f" 眼睛中心在第 {f['eyeRowFromTop']:.1f} 行(自上而下)")+ print(f" earH={f['earH']} earX={f['earX']} earY={f['earY']} earTilt={f['earTilt']}° "+ f"earBend={f['earBend']}° earShade={f['earShade']}")+ print(f" browW={f['browW']} browH={f['browH']} browY={f['browY']} browTilt={f['browTilt']}°")+ print()+ print("─ 量 ─")+ rp = m["radiusPx"]+ print(f" 最大半径 全开 {rp['both']:.1f} 只关眉 {rp['browOff']:.1f} "+ f"只关耳 {rp['earOff']:.1f} 基线 {rp['none']:.1f}")+ print(f" 耳朵伸出轮廓 {m['earOutPx']:.1f}px = 球直径的 {m['earOutFracOfDiameter']*100:.1f}%"+ f" (为它多出的像素 {m['earNewPx']})")+ print(f" 眉毛差异 {m['browDiffPx']}px 最大通道差 {m['browDiffMaxDelta']}"+ f" 质心在第 {m['browDiffCentroidFromTop']:.1f} 行(眼睛在第 {f['eyeRowFromTop']:.1f} 行)")+ print(f" 轮廓内 / 眉毛带以下的意外差异 {m['unexpectedPx']}px (总差异 {m['allDiffPx']}px)")+ print()+ print("─ 差异逐行分布(自上而下,28 行)─")+ mk = max(1, max(m["allDiffRows"]))+ for i, v in enumerate(m["allDiffRows"]):+ bar = "▉" * int(round(v / mk * 46))+ print(f" {i:2d} {v:5d} {bar}")+ print()++ checks = [+ ("耳朵伸出轮廓",+ m["earOutFracOfDiameter"] >= MIN_EAR_OUT_FRAC,+ f"{m['earOutFracOfDiameter']*100:.1f}% ≥ {MIN_EAR_OUT_FRAC*100:.0f}%(球直径)"),+ ("耳朵确实落在轮廓外(不是只在里面换色)",+ m["earNewPx"] >= 100,+ f"新增像素 {m['earNewPx']} ≥ 100"),+ ] + _brow_checks(m, f) + [+ ("球体本身没有被意外改动",+ m["unexpectedPx"] <= MAX_UNEXPECTED_PX,+ f"意外差异 {m['unexpectedPx']}px ≤ {MAX_UNEXPECTED_PX}px"),+ ]++ failed = 0+ for name, ok, detail in checks:+ if not ok:+ failed += 1+ print(f"[{'PASS' if ok else 'FAIL'}] {name}")+ print(f" {detail}")+ print("=" * 72)+ print(f"{len(checks) - failed}/{len(checks)} 通过")++ if R.get("rows"):+ print()+ for row in R["rows"]:+ print(f"── {row['label']} ──")+ for line in row["ascii"]:+ print(" " + line)+ print()+ return 1 if failed else 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/verify_faces_lab.py b/tools/verify_faces_lab.pynew file mode 100644index 0000000..c520e24--- /dev/null+++ b/tools/verify_faces_lab.py@@ -0,0 +1,139 @@+#!/usr/bin/env python3+"""把 faces_lab_probe.js 注入设计稿,用 headless Chrome 跑一遍,打印验收表。++为什么不用 agent-browser:它在本机偶尔会因为 daemon 卡住而整条命令超时,+而这里要的只是"跑一段 JS,把结果读回来"。headless Chrome 加+`--enable-unsafe-swiftshader` 就能在无 GPU 的情况下给出 WebGL 上下文+(`--disable-gpu` 会让 getContext('webgl') 直接返回 null,别加),+`--dump-dom` 再把探针写进 DOM 的 JSON 取出来 —— 两步都是确定性的。++用法:+ python3 tools/verify_faces_lab.py [页面路径]++退出码:0 = 全部通过;1 = 有断言没过;2 = 页面本身就挂了。+"""+import json+import re+import subprocess+import sys+import tempfile+from pathlib import Path++ROOT = Path(__file__).resolve().parent.parent+PAGE = Path(sys.argv[1]) if len(sys.argv) > 1 else ROOT / "reports" / "ball-faces-lab.html"+PROBE = ROOT / "tools" / "faces_lab_probe.js"+++def _find_chrome() -> str:+ """定位一个可用的 Chrome/Chromium。++ 不写死 macOS 路径:可用 CHROME_PATH 覆盖,否则按"macOS 常见位置 → PATH"依次找,+ 都找不到就退回 macOS 默认位置,好让"找不到 Chrome"的报错指向一个具体路径。+ """+ import os+ import shutil++ fallback = "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"+ candidates = [+ os.environ.get("CHROME_PATH"),+ fallback,+ "/Applications/Chromium.app/Contents/MacOS/Chromium",+ "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge",+ "/usr/bin/google-chrome",+ "/usr/bin/chromium",+ "/usr/bin/chromium-browser",+ "/snap/bin/chromium",+ ]+ candidates += [+ shutil.which(name)+ for name in ("google-chrome", "google-chrome-stable", "chromium", "chromium-browser", "chrome")+ ]+ for path in candidates:+ if path and Path(path).exists():+ return path+ return fallback+++CHROME = _find_chrome()++FLAGS = [+ "--headless=new",+ "--no-sandbox",+ "--enable-unsafe-swiftshader",+ "--window-size=1500,3600",+ "--hide-scrollbars",+]+++def main() -> int:+ if not PAGE.exists():+ print(f"找不到页面:{PAGE}", file=sys.stderr)+ return 2+ if not Path(CHROME).exists():+ print(f"找不到 Chrome:{CHROME}", file=sys.stderr)+ return 2++ html = PAGE.read_text(encoding="utf-8")+ probe = PROBE.read_text(encoding="utf-8")+ if "</body>" not in html:+ print("页面里没有 </body>,不知道该把探针插在哪。", file=sys.stderr)+ return 2+ html = html.replace("</body>", f"<script>\n{probe}\n</script>\n</body>", 1)++ tmp = Path(tempfile.mkdtemp(prefix="faces-lab-"))+ harness = tmp / "lab-verify.html"+ harness.write_text(html, encoding="utf-8")++ proc = subprocess.run(+ [CHROME, *FLAGS, "--virtual-time-budget=15000", "--dump-dom", harness.as_uri()],+ capture_output=True, text=True, timeout=180,+ )+ dom = proc.stdout++ # 注意:探针自己的源码就在 DOM 里,而那行注释里有一模一样的 `<pre id="zh-probe">`+ # 字符串。所以不能取第一个匹配 —— 要取**能解析成 JSON** 的那一个。+ R = None+ for raw in re.findall(r'<pre id="zh-probe"[^>]*>(.*?)</pre>', dom, re.S):+ raw = raw.strip()+ if not raw.startswith("{"):+ continue+ try:+ R = json.loads(raw)+ except json.JSONDecodeError:+ continue+ if R is None:+ print("探针没有留下结果 —— 页面大概在 build() 阶段就挂了。", file=sys.stderr)+ err = re.search(r'<div id="err"[^>]*>(.*?)</div>', dom, re.S)+ if err and err.group(1).strip():+ print("页面自己的错误框:", file=sys.stderr)+ print(err.group(1)[:4000], file=sys.stderr)+ for raw in re.findall(r'<pre id="zh-probe"[^>]*>(.*?)</pre>', dom, re.S):+ if raw.strip():+ print("探针留下的原文:", raw.strip()[:2000], file=sys.stderr)+ break+ return 2+ print(f"球球性格实验室 · 验收 (探针 {R.get('version')})")+ print("=" * 72)++ if R.get("fatal"):+ print("页面/探针抛异常:")+ print(R["fatal"])+ return 2++ failed = 0+ for c in R["checks"]:+ mark = "PASS" if c["ok"] else "FAIL"+ if not c["ok"]:+ failed += 1+ print(f"[{mark}] {c['name']}")+ print(f" {c['detail']}")++ print("=" * 72)+ print(f"{len(R['checks']) - failed}/{len(R['checks'])} 通过")+ if R.get("facts"):+ print("facts:", json.dumps(R["facts"], ensure_ascii=False))+ return 1 if failed else 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/view_dump.py b/tools/view_dump.pynew file mode 100755index 0000000..2fede55--- /dev/null+++ b/tools/view_dump.py@@ -0,0 +1,185 @@+#!/usr/bin/env python3+"""Dump another app's View Hierarchy over adb — for apps accessibility can't see.++Some app screens can return an empty accessibility root while their visible+content is rendered by a different window or by a custom surface. The first+case should be handled by enumerating accessibility windows; the second has no+semantic tree and needs pixels. This host-side script is a diagnostic fallback,+not proof that an app blocks accessibility.++So reach for this script when the tree is genuinely absent — for example a+mini-program or a WebView article rendering into its own surface — after the+service has already inspected all accessible windows.++The *view* hierarchy is still there, because that is the data Layout Inspector+uses. `dumpsys activity top` prints it — class, resource id, bounds and+visibility, ~700 lines for WeChat's main screen. It carries **no text**, so pair+the two channels:++ this script → where things are, and which stable id they have+ screenshot → what they actually say (feed it to a vision model)++ ./tools/view_dump.py # the focused app+ ./tools/view_dump.py --pkg com.tencent.mm # a specific one+ ./tools/view_dump.py --grep j8g # only nodes whose id matches+ ./tools/view_dump.py --json # machine-readable+ ./tools/view_dump.py --all # keep invisible/zero-size nodes++Requires a running `adb` and a device that already has the app on screen.+"""++import argparse+import json+import os+import re+import shutil+import subprocess+import sys++# `android.widget.FrameLayout{c1f673c V.E...... ........ 0,0-1600,2524 #7f0a1c9d app:id/v6n}`+VIEW_RE = re.compile(r"^(?P<indent> *)(?P<cls>[A-Za-z_][\w.$]*)\{(?P<hash>[0-9a-fA-F]+)\s+(?P<flags>\S+)(?P<rest>.*)\}$")+BOUNDS_RE = re.compile(r"(-?\d+),(-?\d+)-(-?\d+),(-?\d+)")+ID_RE = re.compile(r"#([0-9a-fA-F]+)\s+(\S+)?")+FOCUS_RE = re.compile(r"mCurrentFocus=Window\{[^}]*\s+([\w.]+)/")+++def adb_path(explicit):+ if explicit:+ return explicit+ if os.environ.get("ADB"):+ return os.environ["ADB"]+ found = shutil.which("adb")+ if found:+ return found+ # Android Studio / SDK default on macOS+ guess = os.path.expanduser("~/Library/Android/sdk/platform-tools/adb")+ return guess if os.path.exists(guess) else "adb"+++def run(adb, *args):+ proc = subprocess.run([adb, *args], capture_output=True, text=True)+ if proc.returncode != 0:+ sys.exit(f"adb {' '.join(args)} failed: {proc.stderr.strip()}")+ return proc.stdout+++def focused_package(adb):+ out = run(adb, "shell", "dumpsys", "window")+ m = FOCUS_RE.search(out)+ return m.group(1) if m else None+++def hierarchy_block(dump, pkg):+ """The `View Hierarchy:` block of `pkg`'s top activity, as raw lines."""+ lines = dump.splitlines()+ start = None+ for i, line in enumerate(lines):+ if line.startswith(" ACTIVITY ") and f" {pkg}/" in line:+ start = i+ break+ if start is None:+ return []+ vh = None+ for i in range(start, len(lines)):+ # A new task/section ends the search.+ if i > start and lines[i].startswith("TASK "):+ break+ if lines[i].strip() == "View Hierarchy:":+ vh = i + 1+ break+ if vh is None:+ return []+ block = []+ for line in lines[vh:]:+ if line.strip() == "":+ continue+ # The block opens with a `DecorView@hash[ActivityName]` line at 6 spaces,+ # then the tree proper at 8+. Anything shallower than 6 is the next+ # section of the activity dump.+ if len(line) - len(line.lstrip(" ")) < 6:+ break+ block.append(line)+ return block+++def parse(lines):+ """Raw dump lines -> flat list of nodes with depth/class/id/bounds."""+ nodes = []+ for line in lines:+ m = VIEW_RE.match(line)+ if not m:+ continue # "(nothing)", "(missing)", malformed title rows+ rest = m.group("rest")+ b = BOUNDS_RE.search(rest)+ i = ID_RE.search(rest)+ x1 = y1 = x2 = y2 = 0+ if b:+ x1, y1, x2, y2 = (int(v) for v in b.groups())+ nodes.append(+ {+ "depth": len(m.group("indent")) // 2,+ "class": m.group("cls"),+ "flags": m.group("flags"),+ "id": i.group(2) if i and i.group(2) else "",+ "bounds": [x1, y1, x2, y2],+ "w": x2 - x1,+ "h": y2 - y1,+ }+ )+ return nodes+++def short_id(raw):+ if not raw:+ return ""+ return raw.rsplit("/", 1)[-1].replace(":", "_")+++def keep(node, show_all, min_size, grep):+ if not show_all:+ # Layout Inspector's flags: index 0 is 'V' when the view is visible.+ if not node["flags"].startswith("V"):+ return False+ if node["w"] < min_size or node["h"] < min_size:+ return False+ if grep and grep.lower() not in f"{node['class']} {node['id']}".lower():+ return False+ return True+++def main():+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)+ ap.add_argument("--pkg", help="package to dump (default: whatever is focused)")+ ap.add_argument("--adb", help="path to adb")+ ap.add_argument("--grep", help="only nodes whose class or id contains this")+ ap.add_argument("--min-size", type=int, default=2, help="drop nodes thinner than this many px (default 2)")+ ap.add_argument("--all", action="store_true", help="keep invisible and zero-size nodes")+ ap.add_argument("--json", action="store_true", help="emit JSON instead of a tree")+ opts = ap.parse_args()++ adb = adb_path(opts.adb)+ pkg = opts.pkg or focused_package(adb)+ if not pkg:+ sys.exit("could not determine the focused package; pass --pkg")++ dump = run(adb, "shell", "dumpsys", "activity", "top")+ block = hierarchy_block(dump, pkg)+ if not block:+ sys.exit(f"no view hierarchy for {pkg} in `dumpsys activity top` (is it on screen and resumed?)")++ nodes = [n for n in parse(block) if keep(n, opts.all, opts.min_size, opts.grep)]++ if opts.json:+ print(json.dumps({"pkg": pkg, "nodes": nodes}, ensure_ascii=False, indent=2))+ return++ print(f"# {pkg} — view hierarchy ({len(nodes)} nodes, no text by design)")+ for n in nodes:+ cls = n["class"].rsplit(".", 1)[-1]+ ident = short_id(n["id"])+ box = ",".join(str(v) for v in n["bounds"])+ print(f"{' ' * n['depth']}{cls}{(' #' + ident) if ident else ''} [{box}]")+++if __name__ == "__main__":+ main()