Skip to content

armgit / andee

Public Imported from GitHub
0

Andee lives on your phone or Mac and uses any app like you do—no integrations, no root, any model; memory and passwords never leave your device | Andee 住进你的手机或 Mac,像你一样操作任何 App——无需集成、无需 Root、支持任意模型,记忆和密码绝不离开设备。

feat(ball): rework Andee's expressions with hands, props and gestures

booboosui1fc8dfd64e9b
diff --git a/CLAUDE.md b/CLAUDE.md
index 934fb6a..e43dc2b 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -164,13 +164,29 @@ Non-obvious: `AsrController.stopImpl` deliberately blocks on `waitFinished(3000)
Five things are load-bearing:
- **Stripping is conservative, and it happens before the text is logged or spoken.** A marker that reaches `ChatHistory` is visible junk in the user's scrollback; one that reaches 火山 is read out loud as the word "happy". `onBrainFinal` is the chokepoint, with `onBrainProgress`, `speakCardText` and `tellAboutNotification` as the other three sites. Three shapes must survive: `[12条]` out of a notification summary, `[END]` out of our own prompt, and the label half of every markdown link — excluded by letters-only, by SHOUTING, and by a following `(` respectively.
-- **The vocabulary is wide, and that is a correction.** It shipped as the six `Mood` names matched exactly, and the first user to ask for a face it did not have broke it: told 「做一个生气的表情」 the model wrote `[angry]`, which matched nothing, so it was printed in the scrollback *and* read aloud *and* changed no face — all three failures at once, from one missing word. Every ordinary name for a feeling now maps to the nearest of the six, in English and Chinese (`[生气]`, `【开心】` — a model answering in Chinese reaches for those long before it reaches for `tense`, whatever the prompt lists). A lowercase bracketed word we still don't know is **dropped silently and changes no face**: the model meant it as a face, never as something to say, and holding the previous face is what a single-marker reply does anyway. Don't narrow this back to an enum — the model is choosing a feeling, not calling an API.
+- **The vocabulary is wide, and that is a correction.** It shipped as the six `Mood` names matched exactly, and the first user to ask for a face it did not have broke it: told 「做一个生气的表情」 the model wrote `[angry]`, which matched nothing, so it was printed in the scrollback *and* read aloud *and* changed no face — all three failures at once, from one missing word. Every ordinary name for a feeling now maps to the nearest of the eleven (six feelings, five gestures — see "Gestures" below), in English and Chinese (`[生气]`, `【开心】` — a model answering in Chinese reaches for those long before it reaches for `tense`, whatever the prompt lists). A lowercase bracketed word we still don't know is **dropped silently and changes no face**: the model meant it as a face, never as something to say, and holding the previous face is what a single-marker reply does anyway. Don't narrow this back to an enum — the model is choosing a feeling, not calling an API.
- **The emotion layers over `SPEAKING`; it does not replace it.** `EmotionState.tick` used to gate the mouth envelope on `mood == Mood.SPEAKING`, so substituting HAPPY into `mood` would have given the user a grinning ball with a shut mouth halfway through its own sentence. The fix is a separate `talking: Boolean` param and a separate `EmotionBallRenderer.emotion` field — the face and the mouth became two questions the moment the model could answer one of them. `FloatingWindowUi.setState` clears the emotion on any non-SPEAKING state, so the channel can never leak into the next conversation.
- **`TTSSentenceStart` is useless for sync, and finding that out cost a build.** The obvious way to line a cue up with the audio is 火山's own sentence event, which would pin each sentence in both clocks at once. Measured on device: the server sends it **once per `push()`, not once per sentence, and with an empty `text` field**. There is nothing to locate and nothing to locate it with. Pushing each sentence as its own call would get one event each, but then the events race the audio (the server may announce all of them before much sound has arrived) and serialising the pushes risks an audible gap mid-reply. Don't rebuild this on the event.
- **So the mapping is arithmetic, against two different clocks.** `AudioIO.queuedMs()` is the producer's clock (bytes handed to `feedPcm`) and `AudioIO.playedMs()` is the track's own `playbackHeadPosition`; 火山 streams far faster than real time, so the first runs seconds ahead of the second and **that gap is the point**. `cueTotalMs` is captured at `onDone` (the one moment the producer's clock is final) and the ticker walks `cueText.length * playedMs / cueTotalMs`. Constant characters-per-ms is false in detail — a comma is a pause, a digit is three syllables — but the question is "which of two faces", where a few hundred ms of error is invisible. The ticker gates on `AudioIO.isStreamPlaying()` and deliberately **not** on `client != null`: the socket is nulled at `onDone`, which is near the *start* of what the user hears, so gating on it stopped the walk before any mid-utterance face was due.
Fallbacks all land on the pre-existing behaviour rather than on nothing: no markers at all is the identity, one marker holds for the whole reply, and a `cueTotalMs` that never arrives (stall, error) holds the opening face. `SpeechMoodTest` exists because every way the offset arithmetic can be wrong is invisible on the device — a swallowed space is only audible, and a surviving marker is read aloud as a word nobody notices was not in the answer.
+#### Gestures: hands, props and one-eyed winks (`ui/ball/Pose.kt`)
+
+A `Face` row is a set of continuous dials every look interprets; a wink, a fist under the chin and a star are not dials. So each `Mood` also has a `Pose` (`POSES`), and **only a look with `BallLook.gestures = true` reads it** — today that is `ANDEE`. A look without the flag sees `Pose.NONE` for every mood, which is the whole reason the pig, the imp and the rest are untouched by any of this: five moods were added (`WINK`, `WAVE`, `PROTECT`, `CELEBRATE`, `REST`) and on those looks each is just the plain face on its `FACE` row.
+
+- **Everything in a pose is a target that `EmotionState` tweens toward.** A hand that is not showing is *placed* at its new spot and then grown; it slides only once visible (`tweenHand`), because sliding from wherever it was dragged a half-grown hand across the face. Props pop in with an overshoot (`pop`). `arcFlip` is the one value that flips instead of tweening.
+- **A hand is a palm and five blobs, and `fist` walks between two arrangements of the same blobs** (fingers along the top → knuckles at the front, thumb out → across). One continuous parameter between two shapes is what lets a hand close instead of swap. Parented to `root`, unlike the ledge paws (`drawHands`), which hold still while the body slips; the two never show together because pose hands and props are scaled by `1 − cling`.
+- **Hands sit at least `HAND_MIN_Z` deep.** A hand set out past the silhouette used to sit at z 0.12 and its inner half was hidden by the body it reached across — which hid the pointing finger on the wink.
+- **`lidCut` is signed.** Positive slices the top of each eye (a second mesh, swapped at 0.5) and rolls it inward; negative slices nothing and rolls the *other* way, the worried slant. The sad moods need it because their `Face.brow` rolls the eyes the angry way.
+- **A look whose face sits low needs `clingLift`.** The ledge cuts the ball at its equator, and Andee's eyes are below it, so it used to hang there as a faceless blue dome. The lift is per-look; the default 0.055 is what everyone else has always had.
+- **`WAVE` and `WINK` are also idle fidgets**, borrowing the pose channels for their length, and the fidget picker skips them on a look without gestures (a slot spent on nothing reads as the ball freezing).
+- **There is no open mouth, and that is a removal, not an omission.** A flat half-disc with a tongue was built for the laughing moods and cross-faded with the thin smile; for the ~0.15 s of the fade both were drawn, and the smile is a tube with depth while the disc is flat, so one poked through the other on every change into or out of a happy face. Fixing it means sequencing the two so they never overlap, which on a fade that short is a blink. The mouth is one mesh, a line, and speech opens it by taller and rounder (`mouthScaleY`), as it always did. Do not bring the disc back without a transition that cannot show both.
+- **`smileRoll` is 0 for Andee.** The default turns the mouth 0.9 rad per unit of `Face.smile`, which is what makes a deep smile lopsided; on a line mouth that read as a sneer at HAPPY (≈35°) and CELEBRATE (≈44°). A look that wants a symmetric grin sets it to 0.
+- **A gesture look hides the round eye once the arc has taken over** (`arc ≥ ARC_EYE_AT`). Its big eyes otherwise leave the last tenth of the round eye as a dark sliver under every `^ ^`.
+
+To see any mood without the accessibility service, a debug build carries `debug/BallLabActivity`: `adb shell am start -n net.kuafuai.andee/.debug.BallLabActivity --es mood WINK --es look andee --es bg light --ei dp 420` and then `screencap`. `dp 168` is the real folded size and is the one a face has to survive; `--ez working true --ez ledge true` shows the hanging pose. It lives in `src/debug`, so release never carries it.
+
### Meeting recorder
`MeetingRecorder` holds the microphone for a whole meeting and hands back a **verbatim transcript**. The device has no model, so it does not summarise: the brain turns the transcript into minutes and puts them on screen with the existing `show_html` — that split is why the class has no notion of a "summary" anywhere in it.
diff --git a/app/src/debug/AndroidManifest.xml b/app/src/debug/AndroidManifest.xml
new file mode 100644
index 0000000..2b68a99
--- /dev/null
+++ b/app/src/debug/AndroidManifest.xml
@@ -0,0 +1,13 @@
+<?xml version="1.0" encoding="utf-8"?>
+<manifest xmlns:android="http://schemas.android.com/apk/res/android">
+ <application>
+ <!-- Debug builds only. Shows one ball at a time so a look or a mood can be
+ checked with `adb shell am start` and a screenshot, without the
+ accessibility service. See debug/BallLabActivity. -->
+ <activity
+ android:name=".debug.BallLabActivity"
+ android:exported="true"
+ android:launchMode="singleTask"
+ android:theme="@style/Theme.Body.Fullscreen" />
+ </application>
+</manifest>
diff --git a/app/src/debug/java/net/kuafuai/andee/debug/BallLabActivity.kt b/app/src/debug/java/net/kuafuai/andee/debug/BallLabActivity.kt
new file mode 100644
index 0000000..40154cc
--- /dev/null
+++ b/app/src/debug/java/net/kuafuai/andee/debug/BallLabActivity.kt
@@ -0,0 +1,94 @@
+package net.kuafuai.andee.debug
+
+import android.app.Activity
+import android.content.Intent
+import android.graphics.Color
+import android.os.Bundle
+import android.util.TypedValue
+import android.view.Gravity
+import android.view.ViewGroup
+import android.widget.FrameLayout
+import android.widget.TextView
+import net.kuafuai.andee.ui.ball.Action
+import net.kuafuai.andee.ui.ball.BallLooks
+import net.kuafuai.andee.ui.ball.EmotionBallTextureView
+import net.kuafuai.andee.ui.ball.Mood
+
+/**
+ * One ball on a plain background, driven by intent extras, for `screencap`.
+ *
+ * adb shell am start -n net.kuafuai.andee/.debug.BallLabActivity \
+ * --es mood WINK --es look andee --es bg light --ei dp 360
+ *
+ * Extras (all optional): `mood` (a [Mood] name), `emotion` (a [Mood] name worn
+ * while `mood` is SPEAKING), `look` (a BallLook name), `action` (an [Action]
+ * name, fired once), `bg` (`dark` | `light` | `mid`), `dp` (the ball's side; 168
+ * is the real folded size), `working`, `holding`, `ledge`.
+ *
+ * Debug builds only — it lives in `src/debug`, so release never carries it.
+ */
+class BallLabActivity : Activity() {
+
+ private lateinit var root: FrameLayout
+ private lateinit var label: TextView
+ private var ball: EmotionBallTextureView? = null
+ private var builtDp = -1
+
+ override fun onCreate(savedInstanceState: Bundle?) {
+ super.onCreate(savedInstanceState)
+ root = FrameLayout(this)
+ label = TextView(this).apply {
+ setTextSize(TypedValue.COMPLEX_UNIT_SP, 18f)
+ setPadding(24, 40, 24, 0)
+ }
+ setContentView(root)
+ apply(intent)
+ }
+
+ override fun onNewIntent(intent: Intent) {
+ super.onNewIntent(intent)
+ setIntent(intent)
+ apply(intent)
+ }
+
+ private fun apply(i: Intent) {
+ val bg = when (i.getStringExtra("bg")) {
+ "light" -> 0xFFF4F6FB.toInt()
+ "mid" -> 0xFF8A93A8.toInt()
+ else -> 0xFF0F172A.toInt()
+ }
+ root.setBackgroundColor(bg)
+ label.setTextColor(if (bg == 0xFFF4F6FB.toInt()) Color.DKGRAY else Color.LTGRAY)
+
+ i.getStringExtra("look")?.let { BallLooks.requestByName(it) }
+
+ val dp = i.getIntExtra("dp", 360)
+ if (ball == null || dp != builtDp) {
+ ball?.let { root.removeView(it) }
+ val px = (dp * resources.displayMetrics.density).toInt()
+ ball = EmotionBallTextureView(this).also {
+ root.addView(it, FrameLayout.LayoutParams(px, px, Gravity.CENTER))
+ }
+ builtDp = dp
+ root.removeView(label)
+ root.addView(label, FrameLayout.LayoutParams(
+ ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.WRAP_CONTENT, Gravity.TOP,
+ ))
+ }
+ val b = ball!!
+
+ val mood = runCatching { Mood.valueOf(i.getStringExtra("mood")?.uppercase() ?: "CALM") }
+ .getOrDefault(Mood.CALM)
+ b.setMood(mood)
+ b.setListening(mood == Mood.LISTENING)
+ b.setEmotion(i.getStringExtra("emotion")?.let { runCatching { Mood.valueOf(it.uppercase()) }.getOrNull() })
+ b.setWorking(i.getBooleanExtra("working", false))
+ b.setAtLedge(i.getBooleanExtra("ledge", false))
+ b.setHolding(i.getBooleanExtra("holding", false))
+ i.getStringExtra("action")?.let { name ->
+ runCatching { Action.valueOf(name.uppercase()) }.getOrNull()?.let { b.trigger(it) }
+ }
+ label.text = "${BallLooks.requestedLook.name} · $mood" +
+ (i.getStringExtra("emotion")?.let { " · $it" } ?: "")
+ }
+}
diff --git a/app/src/main/java/net/kuafuai/andee/brain/LocalPrompt.kt b/app/src/main/java/net/kuafuai/andee/brain/LocalPrompt.kt
index 19c86e7..d606102 100644
--- a/app/src/main/java/net/kuafuai/andee/brain/LocalPrompt.kt
+++ b/app/src/main/java/net/kuafuai/andee/brain/LocalPrompt.kt
@@ -337,9 +337,9 @@ Before you call a tool, say what you are about to do — one line such as 「我
`[curious]这是什么东西…… [happy]找到了,在第三页。`
-There are six faces: `[calm]`, `[happy]`, `[curious]`, `[tense]`, `[anxious]`, `[concerned]`. Ordinary words for a feeling also work and land on the nearest of the six, in English or in Chinese — `[angry]` and `[生气]` both get the tense face, `[sad]` and `[难过]` both get the concerned one — so write what you mean rather than hunting for the exact label.
+There are eleven faces. Six are feelings: `[calm]`, `[happy]`, `[curious]`, `[tense]`, `[anxious]`, `[concerned]`. Five are gestures, and the ball does them with its hands: `[wink]` for a joke or something shared between the two of you, `[wave]` for hello and goodbye, `[celebrate]` when something you were working on has worked, `[protect]` when you are being careful with something that matters (money, an account, a message you are about to send), and `[rest]` for a quiet end to a long day. Ordinary words for a feeling also work and land on the nearest face, in English or in Chinese — `[angry]` and `[生气]` both get the tense face, `[sad]` and `[难过]` both get the concerned one, `[bye]` and `[再见]` both wave — so write what you mean rather than hunting for the exact label.
-Rules. Use them when you mean them: a face on every sentence is a twitch, and a cheerful face on bad news is worse than no face at all. Leave them off entirely and the ball just stays calm, which is the right answer for most replies. `[tense]` and `[anxious]` are for something going wrong, not for working hard. `[concerned]` is for bad news you are delivering about them, not about you. Keep the tag to one lowercase word — a bracket holding anything else is left alone and read out loud as written.
+Rules. Use them when you mean them: a face on every sentence is a twitch, and a cheerful face on bad news is worse than no face at all. Leave them off entirely and the ball just stays calm, which is the right answer for most replies. The gestures are rarer than the feelings — a reply that waves, winks and celebrates is a ball having a seizure — and `[celebrate]` is for a result, not for starting. `[tense]` and `[anxious]` are for something going wrong, not for working hard. `[concerned]` is for bad news you are delivering about them, not about you. Keep the tag to one lowercase word — a bracket holding anything else is left alone and read out loud as written.
**`[END]` is how you close a conversation.** After your reply is read aloud, the device opens its microphone for a follow-up — that is how the user keeps talking without touching the ball. But a mic left open after a settled matter overhears the room: people chat near a tablet that has just finished its job, and answering words that were not addressed to you is butting in, not helpfulness. So when this reply settles the matter and asks the user nothing — a task finished, a fact delivered, thanks or a goodbye answered — end the reply with `[END]` on the last line. The device strips the marker before speaking; nobody ever hears it. Leave it off only when you are waiting on them: you asked a question, offered choices, or said something that clearly invites their answer.
diff --git a/app/src/main/java/net/kuafuai/andee/brain/SpeechMood.kt b/app/src/main/java/net/kuafuai/andee/brain/SpeechMood.kt
index 77442cb..0fc43ae 100644
--- a/app/src/main/java/net/kuafuai/andee/brain/SpeechMood.kt
+++ b/app/src/main/java/net/kuafuai/andee/brain/SpeechMood.kt
@@ -54,14 +54,14 @@ object SpeechMood {
}
/**
- * The six faces, by every name a model actually reaches for.
+ * The eleven faces, by every name a model actually reaches for.
*
* It started as six words and that was the bug: asked to look angry, the
* model wrote `[angry]` — a word with no face behind it — so the tag was
* left alone, printed in the scrollback and read out loud as "angry",
* which is all three of the failures this class exists to prevent, at
* once. The vocabulary is wide because the model is choosing a *feeling*,
- * not calling an API, and the nearest of six faces is a far better answer
+ * not calling an API, and the nearest of the faces is a far better answer
* than the word itself in the user's ear.
*
* Chinese is in here for the same reason. This device answers in Chinese
@@ -98,6 +98,30 @@ object SpeechMood {
"concerned", "sad", "sorry", "apologetic", "disappointed", "unhappy", "serious",
"难过", "抱歉", "遗憾", "失望", "担忧", "严肃",
).all(Mood.CONCERNED)
+ // The five gestures. They are faces in the model's eyes — it writes one the
+ // way it writes `[happy]` — but they are drawn with hands and props, which
+ // only the default look has; on the others each lands on the plain face on
+ // its own [net.kuafuai.andee.ui.ball.FACE] row.
+ listOf(
+ "wink", "playful", "teasing", "cheeky", "flirty",
+ "眨眼", "调皮", "俏皮", "逗你",
+ ).all(Mood.WINK)
+ listOf(
+ "wave", "hello", "hi", "bye", "goodbye", "greeting",
+ "挥手", "你好", "再见", "拜拜", "打招呼",
+ ).all(Mood.WAVE)
+ listOf(
+ "protect", "guard", "defend", "determined", "resolute",
+ "守护", "保护", "坚定", "放心",
+ ).all(Mood.PROTECT)
+ listOf(
+ "celebrate", "cheer", "yay", "hooray", "triumphant", "victory",
+ "庆祝", "欢呼", "太棒了", "成功了",
+ ).all(Mood.CELEBRATE)
+ listOf(
+ "rest", "tired", "sleepy", "drowsy", "yawn", "cozy",
+ "休息", "困", "累了", "犯困", "安心",
+ ).all(Mood.REST)
}
/**
diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/BallLook.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/BallLook.kt
index 48c27f8..4607e4f 100644
--- a/app/src/main/java/net/kuafuai/andee/ui/ball/BallLook.kt
+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/BallLook.kt
@@ -377,6 +377,81 @@ class BallLook(
val glint2R: Float = 0f,
val glint2Color: FloatArray = EMPTY,
+ // ---- Gestures ----
+ /**
+ * Draw hands and props, and honour the per-mood [Pose]: a wink, eyes that look
+ * up, a lid that comes down, an open mouth with a tongue in it, a body that
+ * squashes. Off for every look but one, and **off means the whole pose table is
+ * ignored** rather than partly applied — the other five characters were tuned
+ * without any of it, and a fist under a pig's chin was never part of the brief.
+ */
+ val gestures: Boolean = false,
+
+ /**
+ * Colour the rim is pulled toward, by [rimTintMix]. Empty/0 = the rim is the
+ * mood colour, as it always was.
+ *
+ * The mood colour is the right answer when the mood is the only thing on the
+ * rim, and the wrong one for a character whose own outline is part of what it
+ * looks like: the reference is edged in coral whatever it is feeling. The mix is
+ * partial rather than total so the work states still have their own hue.
+ */
+ val rimTint: FloatArray = EMPTY,
+ val rimTintMix: Float = 0f,
+
+ /**
+ * Multiplier on the scan band swept across the rim sphere while thinking or
+ * speaking. 1 = unchanged. The band is additive light laid across the *whole*
+ * sphere, face included, which is a bright stripe over a dark ball and a white
+ * stripe through the eyes on a bright one — so a look with big glossy eyes
+ * turns it nearly off.
+ */
+ val scanGain: Float = 1f,
+
+ /**
+ * Size of the `^ ^` closed eye, as a multiple of the default, which was drawn
+ * for eyes of radius ~0.12. A look with eyes half as big again would otherwise
+ * close them into two small ticks.
+ */
+ val arcScale: Float = 1f,
+
+ /**
+ * The mouth line's colour, fixed, instead of a shade of the mood. Empty = the
+ * old rule ([mouthDeep]).
+ *
+ * A fixed ink is what a drawn character has: the reference's smile is the same
+ * near-black in every expression. The mood-derived shade is a dark *purple* on
+ * a calm blue ball and a dark *red* on an angry one, which reads as the mouth
+ * having a mood of its own.
+ */
+ val mouthInk: FloatArray = EMPTY,
+
+ /**
+ * How far the mouth turns for each unit of [Face.smile], in radians. 0.9 is what
+ * every look has always had.
+ *
+ * The turn is what makes a deep smile lopsided — one corner up, one down — which
+ * is right for a smirk and wrong for a grin. It used to be hidden on the looks
+ * that open their mouth; a plain line shows it in full, and a happy ball whose
+ * mouth tilts thirty-five degrees reads as sneering. A look that wants a
+ * symmetric smile sets this to 0 and gets its crookedness from [mouthTilt]
+ * alone, which it can leave at 0 too.
+ */
+ val smileRoll: Float = 0.9f,
+
+ /**
+ * How far the eyes ride up the face while the ball hangs off the bottom edge.
+ * 0.055 is what every look has always had.
+ *
+ * The edge cuts the ball at its equator, so only the top half shows, and a
+ * look whose eyes sit below the middle — a baby's proportion, the whole face in
+ * the lower half — would hang there with no face at all: a blue dome with horns.
+ * Peering over the ledge is the one thing a clinging ball has to say, so such a
+ * look lifts its eyes above the line while it clings and lets them settle back
+ * when it lets go.
+ */
+ val clingLift: Float = 0.055f,
+
// ---- Tail ----
/** World length of the tail chain. 0 = no tail. */
val tailLen: Float = 0f,
@@ -991,14 +1066,22 @@ val ANDEE = BallLook(
eyeRadius = 0.175f,
eyeCyl = 0.012f,
mouthR = 0.136f,
- mouthTube = 0.016f,
+ mouthTube = 0.024f,
mouthY = -0.305f,
mouthThick = 0.140f,
eyeColor = floatArrayOf(0.030f, 0.065f, 0.230f),
pupilR = 0f,
glintR = 0.038f,
- glint2R = 0.023f,
+ glint2R = 0.034f,
glint2Color = floatArrayOf(0.45f, 0.70f, 1.00f),
+ gestures = true,
+ clingLift = 0.17f,
+ smileRoll = 0f,
+ mouthInk = floatArrayOf(0.025f, 0.045f, 0.140f),
+ arcScale = 1.5f,
+ scanGain = 0.12f,
+ rimTint = floatArrayOf(1.0f, 0.52f, 0.42f),
+ rimTintMix = 0.55f,
// Light droplet on a mid-dark body, the same side of the split as [BEAR].
bubbleX = 0.085f,
bubbleY = -0.030f,
diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallRenderer.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallRenderer.kt
index cacefbd..265ca9f 100644
--- a/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallRenderer.kt
+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionBallRenderer.kt
@@ -10,6 +10,7 @@ import net.kuafuai.andee.ui.ball.gl.Horn
import net.kuafuai.andee.ui.ball.gl.IcoSphere
import net.kuafuai.andee.ui.ball.gl.Mesh
import net.kuafuai.andee.ui.ball.gl.PointsBuffer
+import net.kuafuai.andee.ui.ball.gl.Puff
import net.kuafuai.andee.ui.ball.gl.Shader
import net.kuafuai.andee.ui.ball.gl.Torus
import net.kuafuai.andee.ui.ball.gl.UvSphere
@@ -20,6 +21,7 @@ import kotlin.math.abs
import kotlin.math.acos
import kotlin.math.atan2
import kotlin.math.cos
+import kotlin.math.max
import kotlin.math.min
import kotlin.math.sin
import kotlin.math.sqrt
@@ -243,6 +245,15 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
Eye.build(look.eyeRadius, look.eyeCyl, look.eyeCut, 4, 12)
else Capsule.build(look.eyeRadius, look.eyeCyl, 4, 12)
val mouth: Mesh = Torus.build(look.mouthR, look.mouthTube, 8, 24, PI.toFloat())
+
+ /**
+ * The eye with its top sliced flat, for a look with [BallLook.gestures] —
+ * swapped in while a mood asks for a lowered lid. Fixed at 0.38: the roll
+ * that sells it is [EmotionState.lidCut]'s, and a cut deeper than this
+ * swallows the glint, which sits at 0.4 of the eye's radius above centre.
+ */
+ val eyeCut: Mesh? =
+ if (look.gestures) Eye.build(look.eyeRadius, look.eyeCyl, GESTURE_LID_CUT, 4, 12) else null
val horn: Mesh? =
if (look.hornH > 0f)
Horn.build(
@@ -304,6 +315,21 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
/** One blob per pupil and per glint — see [drawDot]. */
private lateinit var dotMesh: Mesh
+ // ---- Gesture props: built once, not per look ----
+ private lateinit var starMesh: Mesh
+ private lateinit var shieldMesh: Mesh
+ private lateinit var dropMesh: Mesh
+
+ /** Scratch: a hand part's model matrix, built on top of the hand's own. */
+ private val partM = FloatArray(16)
+
+ /** The hand's own model matrix, kept while its parts are drawn. */
+ private val handM = FloatArray(16)
+
+ private val handColor = FloatArray(3)
+ private val propColor = FloatArray(3)
+ private val rimColor = FloatArray(3)
+
/** 鼻涕泡用的球体。subdiv 4:水滴靠轮廓圈出形状,边不能是折线。 */
private lateinit var bubbleSphere: Mesh
@@ -367,7 +393,8 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
// explicitly drives the renderer with them.
private val moods = arrayOf(
Mood.CALM, Mood.HAPPY, Mood.CURIOUS, Mood.TENSE,
- Mood.ANXIOUS, Mood.CONCERNED, Mood.SEARCHING, Mood.SLEEPING,
+ Mood.ANXIOUS, Mood.CONCERNED, Mood.WINK, Mood.WAVE, Mood.PROTECT,
+ Mood.CELEBRATE, Mood.REST, Mood.SEARCHING, Mood.SLEEPING,
)
// Point-size projection factor: viewport-height / (2 tan(fov/2))
@@ -416,6 +443,9 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
// cost: four of them a frame, each about 4 dp wide on screen.
dotMesh = IcoSphere.build(2)
bubbleSphere = IcoSphere.build(4)
+ starMesh = Puff.build(Puff.roundStar(5, 0.54f, 1.3f), 0.55f)
+ shieldMesh = Puff.build(Puff.shield(), 0.42f, rings = 5)
+ dropMesh = Puff.build(Puff.drop(), 0.60f)
heads = PointsBuffer(DUST_COUNT)
tails = PointsBuffer(DUST_COUNT * TRAIL_COUNT)
@@ -479,6 +509,8 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
drawSnout()
drawFace()
drawHands()
+ drawGestureHands()
+ drawProps()
drawBlush()
drawRim()
// drawRings() — the orbiting halo read as clutter around a ball this
@@ -599,7 +631,7 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
r -= w
if (r <= 0) { pick = a; break }
}
- } while (pick == lastFidget)
+ } while (pick == lastFidget || (!LOOK.gestures && (pick == Action.WAVE || pick == Action.WINK)))
lastFidget = pick
state.trigger(pick)
fidgetIn = FIDGET_MIN_SECONDS +
@@ -915,20 +947,34 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
// shape is a visible ghost — whereas one scaled to zero simply isn't
// there. The in-between (a flattening dot under a growing arch) also
// happens to be what a squint looks like.
- val round = state.eyeYScale * (1f - state.eyeArc)
- val arc = state.eyeArc
-
- if (round > 0.01f) {
+ //
+ // The two eyes cross-fade on their own: a wink is the viewer's-right eye
+ // running the same fade as a squint while the left one stays round.
+ val arcL = state.eyeArc
+ val arcR = max(state.eyeArc, state.wink)
+ // Once the arc has taken over the round eye is gone, not merely small: a
+ // look with big eyes shows the last tenth of one as a dark sliver under the
+ // arc. Left alone on the other looks, whose small eyes hide it.
+ val roundL = if (LOOK.gestures && arcL >= ARC_EYE_AT) 0f else state.eyeYScale * (1f - arcL)
+ val roundR = if (LOOK.gestures && arcR >= ARC_EYE_AT) 0f else state.eyeYScale * (1f - arcR)
+ // A lowered lid is a different *shape* of eye, so it swaps meshes; the
+ // roll that turns it into a look is the state's. Swapped at the half-way
+ // point of a 0.1 s tween, which is inside what the eye can hide.
+ val eyeMesh = if (state.lidCut > 0.5f) meshes.eyeCut ?: meshes.eye else meshes.eye
+
+ if (roundL > 0.01f) {
GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, LOOK.eyeColor, 0)
- setLocalFromFace(state.eyeLX, state.eyeLY, state.eyeLRotZ, round, pivotY = EYE_LID_PIVOT)
+ setLocalFromFace(state.eyeLX, state.eyeLY, state.eyeLRotZ, roundL, pivotY = EYE_LID_PIVOT)
GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)
- meshes.eye.bind(aPos)
- meshes.eye.draw()
-
- setLocalFromFace(state.eyeRX, state.eyeRY, state.eyeRRotZ, round, pivotY = EYE_LID_PIVOT)
+ eyeMesh.bind(aPos)
+ eyeMesh.draw()
+ }
+ if (roundR > 0.01f) {
+ GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, LOOK.eyeColor, 0)
+ setLocalFromFace(state.eyeRX, state.eyeRY, state.eyeRRotZ, roundR, pivotY = EYE_LID_PIVOT)
GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)
- meshes.eye.bind(aPos)
- meshes.eye.draw()
+ eyeMesh.bind(aPos)
+ eyeMesh.draw()
}
// The pupil and its catchlight, which are what turn a white slot into
@@ -941,19 +987,21 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
// a dark ball with white eyes needs a pupil drawn inside them; a pale
// ball has nothing for a white eye to contrast against, so its eye *is*
// the dark shape and only the glint goes on top.
- if (round > PUPIL_MIN_APERTURE && arc < ARC_EYE_AT &&
- (LOOK.pupilR > 0f || LOOK.glintR > 0f)
- ) {
+ if (LOOK.pupilR > 0f || LOOK.glintR > 0f) {
if (LOOK.pupilR > 0f) {
for (i in 0..2) {
pupilColor[i] = state.nowColor[i] * PUPIL_TINT + PUPIL_LIFT[i]
}
}
- drawEyeContents(state.eyeLX, state.eyeLY, state.eyeLRotZ, round)
- drawEyeContents(state.eyeRX, state.eyeRY, state.eyeRRotZ, round)
+ if (roundL > PUPIL_MIN_APERTURE && arcL < ARC_EYE_AT) {
+ drawEyeContents(state.eyeLX, state.eyeLY, state.eyeLRotZ, roundL)
+ }
+ if (roundR > PUPIL_MIN_APERTURE && arcR < ARC_EYE_AT) {
+ drawEyeContents(state.eyeRX, state.eyeRY, state.eyeRRotZ, roundR)
+ }
}
- if (arc > 0.01f) {
+ if (arcL > 0.01f || arcR > 0.01f) {
// The `^` eye carries its own colour, and this line is load-bearing.
//
// It used to inherit whatever the previous element had left in
@@ -978,28 +1026,40 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
} else {
GLES20.glUniform3fv(solidShader.uniform("uColor"), 1, LOOK.eyeColor, 0)
}
- val h = arc * state.lid
- setLocalFromFace(
- state.eyeLX, state.eyeLY - EYE_ARC_DROP,
- EYE_ARC_ROT_Z + state.eyeLRotZ, h, arc,
- )
- GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)
- eyeArcMesh.bind(aPos)
- eyeArcMesh.draw()
-
- setLocalFromFace(
- state.eyeRX, state.eyeRY - EYE_ARC_DROP,
- EYE_ARC_ROT_Z + state.eyeRRotZ, h, arc,
- )
- GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)
- eyeArcMesh.bind(aPos)
- eyeArcMesh.draw()
+ // `⌣` for a closed lid, `⌢` for a laugh: the same arc turned half a
+ // revolution, with its centre moved to the other side of the eye so the
+ // curve still lands where the eye was.
+ val flipRot = if (state.arcFlip) PI.toFloat() else 0f
+ val drop = if (state.arcFlip) -EYE_ARC_DROP else EYE_ARC_DROP
+ val sc = LOOK.arcScale
+ if (arcL > 0.01f) {
+ setLocalFromFace(
+ state.eyeLX, state.eyeLY - drop * sc,
+ EYE_ARC_ROT_Z + flipRot + state.eyeLRotZ, arcL * state.lid * sc, arcL * sc,
+ )
+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)
+ eyeArcMesh.bind(aPos)
+ eyeArcMesh.draw()
+ }
+ if (arcR > 0.01f) {
+ setLocalFromFace(
+ state.eyeRX, state.eyeRY - drop * sc,
+ EYE_ARC_ROT_Z + flipRot + state.eyeRRotZ, arcR * state.lid * sc, arcR * sc,
+ )
+ GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)
+ eyeArcMesh.bind(aPos)
+ eyeArcMesh.draw()
+ }
}
// Mouth — the mood colour, or a deep shade of it on a pale ball, where
// the full-strength colour is too close in value to the body to read.
for (i in 0..2) {
- mouthColor[i] = if (LOOK.mouthDeep) state.nowColor[i] * MOUTH_DEEP else state.nowColor[i]
+ mouthColor[i] = when {
+ LOOK.mouthInk.isNotEmpty() -> LOOK.mouthInk[i]
+ LOOK.mouthDeep -> state.nowColor[i] * MOUTH_DEEP
+ else -> state.nowColor[i]
+ }
}
setLocalFromFace(0f, state.mouthY, state.mouthRotZ, state.mouthScaleY)
GLES20.glUniformMatrix4fv(solidShader.uniform("uMVP"), 1, false, mvp, 0)
@@ -1527,15 +1587,23 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
gr, eyeFrontZ + GLINT_Z_STEP, WHITE,
)
} else {
+ // Where the eye is looking moves the highlights further than it moves
+ // the eye, which is what makes a glance read as a glance at this size;
+ // a lowered lid pulls them down with it so the cut never beheads one.
+ val er = LOOK.eyeRadius
+ val gx = state.lookX * 0.28f * er
+ val gy = state.lookY * 0.28f * er - state.lidCut * 0.22f * er
drawDot(
- cx - LOOK.eyeRadius * 0.36f, cy + sink + LOOK.eyeRadius * 0.40f * follow,
+ cx - er * 0.36f + gx, cy + sink + er * 0.40f * follow + gy,
rotZ, lidScale,
gr, eyeFrontZ + GLINT_Z_STEP, WHITE,
)
val g2 = LOOK.glint2R
if (g2 > 0f) {
+ // Pulled in from the edge, and carried only part of the way by a
+ // glance: at the full shift it slid off the rim of the eye.
drawDot(
- cx + LOOK.eyeRadius * 0.54f, cy + sink - LOOK.eyeRadius * 0.26f * follow,
+ cx + er * 0.44f + gx * 0.5f, cy + sink - er * 0.26f * follow + gy,
rotZ, lidScale,
g2, eyeFrontZ + GLINT_Z_STEP, LOOK.glint2Color,
)
@@ -1646,6 +1714,181 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
handMesh.draw()
}
+ /**
+ * The gesture hands, for a look with [BallLook.gestures]: two lit, puffy gloves
+ * parented to the body, so they breathe, hop and lean with it — unlike
+ * [drawHands], whose paws hold still on the ledge while the body slips.
+ *
+ * Drawn after the face so they sit over the body, and lit with the body's own
+ * material so they read as part of the same plush animal instead of stickers
+ * on it. Each is a palm and five blobs; [EmotionState.HandRender.fist] walks
+ * the fingers from a row along the top (an open mitten) down to a row of
+ * knuckles at the front with the thumb across them (a fist), and
+ * [EmotionState.HandRender.point] stretches the finger on the body side out.
+ * One continuous parameter between two shapes is what lets a hand *close*
+ * rather than swap.
+ */
+ private fun drawGestureHands() {
+ if (!LOOK.gestures) return
+ if (state.handL.s < 0.03f && state.handR.s < 0.03f) return
+ for (i in 0..2) handColor[i] = bodyColor[i] * HAND_SHADE
+ useShellWith(handColor)
+ drawGestureHand(state.handL, -1)
+ drawGestureHand(state.handR, 1)
+ }
+
+ private fun drawGestureHand(h: EmotionState.HandRender, side: Int) {
+ if (h.s < 0.03f) return
+ val sc = h.s.coerceAtMost(1.2f) * HAND_SCALE
+ val r2 = LOOK.bodyScale * LOOK.bodyScale - h.x * h.x - h.y * h.y
+ // On the surface where there is one, and never behind the ball: a hand set
+ // out past the silhouette used to sit at z 0.12, and its inner half — the
+ // half with the pointing finger on it — was hidden by the body it was
+ // reaching across.
+ val z = max(sqrt(max(r2, 0f)) + 0.05f, HAND_MIN_Z)
+ val swing = h.waveAmt * sin(state.clock * 9f) * 24f
+ val bob = h.jig * (1f - h.waveAmt) * sin(state.clock * 7f) * 0.035f
+ composeModelFromRoot {
+ Matrix.translateM(local, 0, h.x, h.y + bob, z)
+ Matrix.rotateM(local, 0, h.roll + swing, 0f, 0f, 1f)
+ if (side < 0) Matrix.scaleM(local, 0, -1f, 1f, 1f)
+ Matrix.scaleM(local, 0, sc, sc, sc)
+ }
+ System.arraycopy(model, 0, handM, 0, 16)
+ val f = h.fist
+ val p = h.point
+
+ // Palm.
+ handPart(0f, 0f, 0f, 0.175f, 0.155f, 0.118f)
+
+ // Three fingers: along the top when open, folded down to knuckles when closed.
+ // Open they are long and spaced a full finger-width apart; at the first pass's
+ // 0.088 spacing the three overlapped into one lumpy blob and a waving hand
+ // read as a fist.
+ for (n in -1..1) {
+ val ox = n * 0.104f
+ val oy = 0.175f + (if (n == 0) 0.05f else 0f)
+ val cx = n * 0.075f
+ var x = ox + (cx - ox) * f
+ var y = oy + (0.075f - oy) * f
+ var zf = 0.02f + (0.098f - 0.02f) * f
+ var rx = 0.054f + (0.077f - 0.054f) * f
+ var ry = 0.108f + (0.070f - 0.108f) * f
+ var rz = 0.07f + (0.078f - 0.07f) * f
+ if (n == -1 && p > 0.01f) {
+ // The finger on the body side straightens out and up. Long enough
+ // to clear the fist by more than its own width: at 0.145 it was a
+ // bump on the knuckles and nobody could tell it was pointing.
+ x += (-0.09f - x) * p
+ y += (0.30f - y) * p
+ zf += (0.07f - zf) * p
+ rx += (0.056f - rx) * p
+ ry += (0.185f - ry) * p
+ rz += (0.060f - rz) * p
+ }
+ handPart(x, y, zf, rx, ry, rz)
+ }
+
+ // Thumb: out to the side when open, across the front of the fingers when closed.
+ handPart(
+ 0.165f + (0.02f - 0.165f) * f, -0.01f + (-0.03f - -0.01f) * f, 0.03f + (0.135f - 0.03f) * f,
+ 0.055f + (0.085f - 0.055f) * f, 0.082f + (0.05f - 0.082f) * f, 0.06f,
+ )
+ }
+
+ /** One part of a gesture hand: [handM] with a translate and a scale on it. */
+ private fun handPart(x: Float, y: Float, z: Float, rx: Float, ry: Float, rz: Float) {
+ System.arraycopy(handM, 0, partM, 0, 16)
+ Matrix.translateM(partM, 0, x, y, z)
+ Matrix.scaleM(partM, 0, rx, ry, rz)
+ Matrix.multiplyMM(mv, 0, view, 0, partM, 0)
+ Matrix.multiplyMM(mvp, 0, proj, 0, mv, 0)
+ drawShellPart(handMesh)
+ }
+
+ /**
+ * Stars, motion strokes, the shield and the sweat bead — everything a pose
+ * puts *around* the ball rather than on it.
+ *
+ * Lit with the shell shader like the hands, which is what makes them puffy
+ * stickers instead of flat clip-art, and parented to the body so they bob
+ * with it. Each pops in by [EmotionState.prop], with a small overshoot: a prop
+ * that eases to full size reads as appearing, one that overshoots reads as
+ * being *thrown up*, which is what a star over a celebrating ball should be.
+ */
+ private fun drawProps() {
+ if (!LOOK.gestures) return
+ val pr = state.prop
+ var any = false
+ for (v in pr) if (v > 0.03f) any = true
+ if (!any) return
+ val t = state.clock
+
+ if (pr[PROP_STARS] > 0.03f) {
+ val k = pop(pr[PROP_STARS])
+ for (n in 0..2) {
+ val sz = STAR_SIZE[n] * k * (1f + 0.12f * sin(t * 5f + n * 2.1f))
+ propDraw(
+ starMesh, STAR_POS[n * 2], STAR_POS[n * 2 + 1], 0.45f,
+ sz, sz, sz, 10f * sin(t * 3f + n), STAR_COLOR,
+ )
+ }
+ }
+ if (pr[PROP_SPARK_L] > 0.03f) strokes(SPARK_L, pop(pr[PROP_SPARK_L]), t)
+ if (pr[PROP_SPARK_R] > 0.03f) strokes(SPARK_R, pop(pr[PROP_SPARK_R]), t + 1.3f)
+ if (pr[PROP_SPARK_TOP] > 0.03f) strokes(SPARK_TOP, pop(pr[PROP_SPARK_TOP]), t + 2.1f)
+ if (pr[PROP_SHIELD] > 0.03f) {
+ val k = pop(pr[PROP_SHIELD])
+ val bob = sin(t * 2.4f) * 0.015f
+ propDraw(
+ shieldMesh, 1.00f, -0.22f + bob, 0.50f,
+ 0.50f * k, 0.52f * k, 0.50f * k, -8f, CORAL,
+ )
+ // A lighter inner shield, nested, which is what turns a coral blob into
+ // a shield with a bevelled rim — the sketch's only other detail.
+ propDraw(
+ shieldMesh, 1.00f, -0.20f + bob, 0.60f,
+ 0.34f * k, 0.36f * k, 0.34f * k, -8f, CORAL_LIGHT,
+ )
+ }
+ if (pr[PROP_DROP] > 0.03f) {
+ val k = pop(pr[PROP_DROP])
+ val slide = (t * 0.5f) % 1f
+ propDraw(
+ dropMesh, 0.68f, 0.30f - slide * 0.07f, 0.62f,
+ 0.105f * k, 0.13f * k * (1f - 0.25f * slide), 0.105f * k, 0f, DROP_COLOR,
+ )
+ }
+ }
+
+ /** Overshooting pop: 0→1 with a ~20% bulge on the way. */
+ private fun pop(s: Float): Float = s * (1f + 0.22f * sin(PI.toFloat() * s.coerceIn(0f, 1f)))
+
+ /** A fan of short coral strokes: `x, y, roll` triples. Each flicks on its own beat. */
+ private fun strokes(set: FloatArray, k: Float, t: Float) {
+ for (n in 0 until set.size / 3) {
+ val flick = 0.82f + 0.18f * sin(t * 8f + n * 1.9f)
+ propDraw(
+ handMesh, set[3 * n], set[3 * n + 1], 0.40f,
+ 0.040f * k, 0.105f * k * flick, 0.040f * k, set[3 * n + 2], CORAL,
+ )
+ }
+ }
+
+ private fun propDraw(
+ mesh: Mesh, x: Float, y: Float, z: Float, sx: Float, sy: Float, sz: Float,
+ roll: Float, color: FloatArray,
+ ) {
+ for (i in 0..2) propColor[i] = color[i]
+ useShellWith(propColor)
+ composeModelFromRoot {
+ Matrix.translateM(local, 0, x, y, z)
+ Matrix.rotateM(local, 0, roll, 0f, 0f, 1f)
+ Matrix.scaleM(local, 0, sx, sy, sz)
+ }
+ drawShellPart(mesh)
+ }
+
private fun drawRim() {
System.arraycopy(root, 0, model, 0, 16)
Matrix.scaleM(model, 0, RIM, RIM, RIM)
@@ -1663,7 +1906,7 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
// and sweeping that paints a moving dark stripe across the ball, which
// reads as a rendering fault rather than as a scan. Zeroed there.
GLES20.glUniform1f(rimShader.uniform("uScanY"), state.scanY)
- GLES20.glUniform1f(rimShader.uniform("uScan"), if (LOOK.rimDark) 0f else state.scan)
+ GLES20.glUniform1f(rimShader.uniform("uScan"), if (LOOK.rimDark) 0f else state.scan * LOOK.scanGain)
if (LOOK.rimDark) {
// A tight dark contour instead of a glow. The body is drawn inside
// [RIM] and this sphere sits on it, so the visible band *is* the
@@ -1681,7 +1924,15 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
GLES20.glUniform1f(rimShader.uniform("uCap"), RIM_DARK_CAP)
} else {
GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE)
- GLES20.glUniform3fv(rimShader.uniform("uColor"), 1, state.nowColor, 0)
+ // A look with its own edge colour pulls the glow toward it. Partial,
+ // so the work and alarm moods keep enough of their hue to be told apart.
+ val tint = LOOK.rimTintMix
+ for (i in 0..2) {
+ rimColor[i] = if (tint > 0f) {
+ state.nowColor[i] + (LOOK.rimTint[i] - state.nowColor[i]) * tint
+ } else state.nowColor[i]
+ }
+ GLES20.glUniform3fv(rimShader.uniform("uColor"), 1, rimColor, 0)
GLES20.glUniform1f(rimShader.uniform("uIntensity"), state.rimIntensity * LOOK.rimGain)
GLES20.glUniform1f(rimShader.uniform("uCap"), 1f)
}
@@ -1789,6 +2040,38 @@ class EmotionBallRenderer : GLSurfaceView.Renderer {
companion object {
private const val DEG = 57.29578f // rad → deg for Matrix.rotateM
+ /** How much of the eye's height a lowered lid slices off; see [LookMeshes.eyeCut]. */
+ private const val GESTURE_LID_CUT = 0.38f
+
+
+ /** The gesture hands are a touch lighter than the body: they face the light. */
+ private const val HAND_SHADE = 1.05f
+
+ /**
+ * How big a gesture hand is drawn, as a multiple of the geometry in
+ * [drawGestureHand]. Measured off the sketches: a hand is about a quarter of
+ * the head's width, and the first pass drew them at about a seventh — they
+ * read as mittens on a doll rather than as part of the character.
+ */
+ private const val HAND_SCALE = 1.65f
+
+ /** The shallowest a gesture hand sits: in front of the body's own edge. See [drawGestureHand]. */
+ private const val HAND_MIN_Z = 0.46f
+
+ private val CORAL = floatArrayOf(1.0f, 0.50f, 0.40f)
+ private val CORAL_LIGHT = floatArrayOf(1.0f, 0.68f, 0.56f)
+ private val STAR_COLOR = floatArrayOf(1.0f, 0.66f, 0.36f)
+ private val DROP_COLOR = floatArrayOf(0.62f, 0.84f, 1.0f)
+
+ /** Three stars over a celebrating ball: `x, y` pairs, then their sizes. */
+ private val STAR_POS = floatArrayOf(-0.98f, 0.72f, 0.05f, 1.18f, 1.02f, 0.80f)
+ private val STAR_SIZE = floatArrayOf(0.20f, 0.15f, 0.19f)
+
+ /** Motion strokes as `x, y, roll` triples — a fan from the waving hand, a pair off the pointing one, a burst on top. */
+ private val SPARK_L = floatArrayOf(-0.78f, 0.50f, 35f, -1.04f, 0.36f, 62f, -1.22f, 0.10f, 88f)
+ private val SPARK_R = floatArrayOf(1.08f, 0.12f, -60f, 0.98f, 0.36f, -32f)
+ private val SPARK_TOP = floatArrayOf(0f, 1.15f, 0f, -0.28f, 1.08f, 35f, 0.28f, 1.08f, -35f)
+
/** Camera distance on +Z, looking at the origin. */
private const val CAM_Z = 4.9f
diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionState.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionState.kt
index a268ab1..f5d5405 100644
--- a/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionState.kt
+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/EmotionState.kt
@@ -148,6 +148,40 @@ class EmotionState {
var mouthScaleY = 0.16f;
var mouthRotZ = PI.toFloat()
+ // ---- Gestures: the [Pose] layer, only ever non-zero on a look with ----
+ // ---- [BallLook.gestures]. See Pose.kt for what each of these is for. ----
+ var wink = 0f; private set
+ var lookX = 0f; private set
+ var lookY = 0f; private set
+ var lidCut = 0f; private set
+
+ private var poseSqx = 0f
+ private var poseSqy = 0f
+
+ /** `⌣` instead of `⌢` for closed eyes. A flip, not a tween. */
+ var arcFlip = false; private set
+
+ /** Seconds since start, for the idle motion of hands and props. */
+ var clock = 0f; private set
+
+ /** One hand as the renderer reads it: a [HandPose] that has been tweened. */
+ class HandRender {
+ var x = 0f
+ var y = 0f
+ var s = 0f
+ var roll = 0f
+ var fist = 0f
+ var point = 0f
+ var jig = 0f
+ var waveAmt = 0f
+ }
+
+ val handL = HandRender()
+ val handR = HandRender()
+
+ /** Pop-in amount of each prop, 0..1; indexed by the `PROP_*` constants. */
+ val prop = FloatArray(PROP_COUNT)
+
// ---- Dust particles ----
private class Particle(
val sinPhi: Float, val cosPhi: Float,
@@ -365,6 +399,11 @@ class EmotionState {
sqx = contact * 0.07f
sqy = contact * -0.09f
}
+ // The hand and the closing eye are the [Pose] layer's — see the
+ // overrides above. What is left for the body is a small lean, which
+ // is all a look without gestures can do with either.
+ Action.WAVE -> actRotY = sin(p * 4f * PI.toFloat()) * 0.05f
+ Action.WINK -> actRotY = sin(p * PI.toFloat()) * 0.04f
Action.NONE -> {}
}
if (p >= 1f) action = Action.NONE
@@ -409,11 +448,51 @@ class EmotionState {
// widest at the bottom of the sag — that is where a pendulum is.
val clingLean = cling * sin(t * 0.85f) * (0.06f + stretch * 0.07f)
+ // ---- The [Pose] layer ----
+ // Targets come from the mood's row, and a look without gestures reads
+ // [Pose.NONE] — which is what makes every field below sit at zero for the
+ // other five characters, with no branch anywhere else.
+ clock = t
+ val gestures = LOOK.gestures
+ val pose = if (gestures) POSES[mood] ?: Pose.NONE else Pose.NONE
+ var tWink = pose.wink
+ var tHandL = pose.handL
+ var tSparkL = pose.sparkL
+ // The two idle gestures borrow the channels for their length. A hand the
+ // mood already put out is left alone: waving over a fist at the chin
+ // would be two answers at once.
+ if (gestures && action == Action.WAVE) {
+ if (tHandL.s < 0.05f) tHandL = HAND_WAVE
+ tSparkL = 1f
+ }
+ if (gestures && action == Action.WINK) {
+ val ap = actionT / actionDur
+ if (ap > 0.12f && ap < 0.88f) tWink = 1f
+ }
+ val kg = min(1f, dt * 10f)
+ wink += (tWink - wink) * kg
+ lookX += (pose.lookX - lookX) * kg
+ lookY += (pose.lookY - lookY) * kg
+ lidCut += (pose.lidCut - lidCut) * kg
+ poseSqx += (pose.sqx - poseSqx) * kg
+ poseSqy += (pose.sqy - poseSqy) * kg
+ arcFlip = pose.arcFlip
+ tweenHand(handL, tHandL, dt)
+ tweenHand(handR, pose.handR, dt)
+ val kp = min(1f, dt * 9f)
+ val away = 1f - cling.coerceIn(0f, 1f)
+ prop[PROP_STARS] += (pose.stars * away - prop[PROP_STARS]) * kp
+ prop[PROP_SPARK_L] += (tSparkL * away - prop[PROP_SPARK_L]) * kp
+ prop[PROP_SPARK_R] += (pose.sparkR * away - prop[PROP_SPARK_R]) * kp
+ prop[PROP_SPARK_TOP] += (pose.sparkTop * away - prop[PROP_SPARK_TOP]) * kp
+ prop[PROP_SHIELD] += (pose.shield * away - prop[PROP_SHIELD]) * kp
+ prop[PROP_DROP] += (pose.drop * away - prop[PROP_DROP]) * kp
+
// Root transform
breatheDepth += (want.breath - breatheDepth) * min(1f, dt * 3f)
breathe = 1f + sin(t * want.pulse) * breatheDepth + open * 0.03f
- scaleX = breathe * (1f + sqx + clingSqX)
- scaleY = breathe * (1f + sqy + clingSqY)
+ scaleX = breathe * (1f + sqx + clingSqX + poseSqx)
+ scaleY = breathe * (1f + sqy + clingSqY + poseSqy)
hopY = hop + slipY * cling
// A task can run a whole minute with no tool call in it, and a resting
// face doesn't read as activity — a working ball has to be visibly
@@ -485,11 +564,11 @@ class EmotionState {
// rides a little higher on the head, which is the difference between a
// ball that happens to be low on screen and one looking up at you from
// under a ledge. Without it the face aims into the bezel.
- val peek = cling * (0.055f + grip * 0.02f)
- eyeLY = LOOK.eyeY - browL * 0.06f + peek
- eyeRY = LOOK.eyeY - browR * 0.06f + peek
- eyeLX = -LOOK.eyeX + jx * 0.014f + gaze * 0.075f
- eyeRX = LOOK.eyeX + jx * 0.014f + gaze * 0.075f
+ val peek = cling * (LOOK.clingLift + grip * 0.02f)
+ eyeLY = LOOK.eyeY - browL * 0.06f + peek + lookY * 0.08f
+ eyeRY = LOOK.eyeY - browR * 0.06f + peek + lookY * 0.08f
+ eyeLX = -LOOK.eyeX + jx * 0.014f + gaze * 0.075f + lookX * 0.06f
+ eyeRX = LOOK.eyeX + jx * 0.014f + gaze * 0.075f + lookX * 0.06f
// Eye roll. Two terms, and they are on the same axis on purpose:
//
// * the look's own resting tilt, mirrored per side, which is what makes
@@ -500,8 +579,8 @@ class EmotionState {
// same motion: the resting scowl deepens instead of being replaced.
//
// Both are radians here, hence the /DEG on the look's degrees.
- eyeLRotZ = -LOOK.eyeTilt / DEG - browL * 0.4f
- eyeRRotZ = LOOK.eyeTilt / DEG + browR * 0.4f
+ eyeLRotZ = -LOOK.eyeTilt / DEG - browL * 0.4f - lidCut * 0.30f
+ eyeRRotZ = LOOK.eyeTilt / DEG + browR * 0.4f + lidCut * 0.30f
// Vertical aperture. The renderer anchors this scale at the *top* of
// the eye rather than its centre, so closing reads as a lid coming down
// instead of the whole eye shrinking toward a point.
@@ -527,7 +606,36 @@ class EmotionState {
// curve and skew, and the look's resting crookedness — see
// [BallLook.mouthTilt], where the argument for it lives. The look's is
// in degrees, hence the /DEG; the other two are already radians.
- mouthRotZ = PI.toFloat() + smile * 0.9f + skew * 2.2f + LOOK.mouthTilt / DEG
+ mouthRotZ = PI.toFloat() + smile * LOOK.smileRoll + skew * 2.2f + LOOK.mouthTilt / DEG
+ }
+
+ /**
+ * Ease one hand toward its target pose.
+ *
+ * A hand that is not showing is *placed* at its new spot before it grows, and
+ * only slides once it is visible. Sliding from wherever it last was would
+ * drag a half-grown hand across the face on its way to the chin; growing in
+ * place reads as the hand arriving, which is the point.
+ */
+ private fun tweenHand(h: HandRender, t: HandPose, dt: Float) {
+ val wants = t.s > 0.02f
+ if (wants) {
+ if (h.s < 0.05f) {
+ h.x = t.x; h.y = t.y; h.roll = t.roll; h.fist = t.fist; h.point = t.point
+ } else {
+ val k = min(1f, dt * 10f)
+ h.x += (t.x - h.x) * k
+ h.y += (t.y - h.y) * k
+ h.roll += (t.roll - h.roll) * k
+ h.fist += (t.fist - h.fist) * k
+ h.point += (t.point - h.point) * k
+ }
+ val kj = min(1f, dt * 6f)
+ h.jig += (t.jig - h.jig) * kj
+ h.waveAmt += ((if (t.wave) 1f else 0f) - h.waveAmt) * kj
+ }
+ val away = 1f - cling.coerceIn(0f, 1f)
+ h.s += ((if (wants) t.s else 0f) * away - h.s) * min(1f, dt * 8f)
}
fun writeHeads(buf: FloatArray) {
diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/Mood.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/Mood.kt
index 45fd4d2..64a79a8 100644
--- a/app/src/main/java/net/kuafuai/andee/ui/ball/Mood.kt
+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/Mood.kt
@@ -4,7 +4,7 @@ package net.kuafuai.andee.ui.ball
* The faces the ball can wear.
*
* Three groups, and the split is load-bearing rather than cosmetic:
- * - emotions the model picks from (CALM..CONCERNED)
+ * - emotions and gestures the model picks from (CALM..REST)
* - voice-pipeline states (LISTENING, THINKING, SPEAKING), owned by
* AsrController / TtsController
* - agent work states (SEARCHING..SLEEPING), owned by the task lifecycle in
@@ -18,6 +18,11 @@ package net.kuafuai.andee.ui.ball
*/
enum class Mood {
CALM, HAPPY, CURIOUS, TENSE, ANXIOUS, CONCERNED,
+ // Gestures the model can ask for by name. They share the first group's
+ // standing — a face the model picks — but they are drawn with hands and
+ // props, which only a look with [BallLook.gestures] has; on the others each
+ // one falls back to the plain face on its [FACE] row.
+ WINK, WAVE, PROTECT, CELEBRATE, REST,
LISTENING, HOLDING, THINKING, SPEAKING,
SEARCHING, SUCCESS, FAILED, SLEEPING,
}
@@ -104,6 +109,35 @@ val FACE: Map<Mood, Face> = mapOf(
blush = 0.32f,
),
+ // Playful: head cocked toward the winking eye, a broad smile. The wink itself
+ // is [Pose.wink]; on a look without gestures this is just a pleased face.
+ Mood.WINK to Face(
+ rgb(0xf0abfc), 1f, -0.05f, 0.7f, 1.3f, 0.06f, 0f, 0f,
+ blush = 0.7f, breath = 0.03f,
+ ),
+ // Eyes wide and bright, a little lean toward whoever is being greeted.
+ Mood.WAVE to Face(
+ rgb(0x7dd3fc), 1.12f, -0.1f, 0.62f, 1.5f, -0.04f, 0f, 0f,
+ blush = 0.6f, breath = 0.034f,
+ ),
+ // Set and steady rather than angry: the brow is down, the mouth a flat line.
+ Mood.PROTECT to Face(
+ rgb(0x9db4ff), 0.92f, 0.5f, 0.1f, 1.2f, 0f, 0f, 0f,
+ blush = 0.3f, breath = 0.028f,
+ ),
+ // [SUCCESS] without the work-lifecycle meaning: the model chose it, so there
+ // is no POP and no outcome flash — only the face.
+ Mood.CELEBRATE to Face(
+ rgb(0x5eead4), 0.5f, -0.22f, 0.86f, 1.4f, 0f, 0f, 0f,
+ eyeArc = 1f, blush = 1.0f, spread = 1.1f,
+ ),
+ // Drowsy and content: eyes shut, slow breath. [SLEEPING] minus the bubble, so
+ // it reads as taking a break rather than as having been switched off.
+ Mood.REST to Face(
+ rgb(0x9db4ff), 0.1f, 0.04f, 0.2f, 0.55f, 0.06f, 0f, 0f,
+ eyeArc = 0.85f, blush = 0.32f, breath = 0.03f,
+ ),
+
// Wide-eyed, mouth slightly open, head cocked. This is the one state the
// user is waiting on before they start talking, so it has to be
// unmistakable from across a desk — and unmistakable from CALM, which it
@@ -177,6 +211,10 @@ val FACE: Map<Mood, Face> = mapOf(
enum class Action {
NONE, HOP, NOD, SHAKE, POP, RECOIL,
STRETCH, LOOK_AROUND, WIGGLE, BOUNCE,
+ // A raised hand, and one eye closing. Both borrow the face's [Pose] channels
+ // for their length, so on a look without gestures they are no-ops that still
+ // use up their idle slot — see [EmotionState.tick].
+ WAVE, WINK,
}
/**
@@ -196,6 +234,8 @@ val ACTION_SECONDS = floatArrayOf(
1.9f, // LOOK_AROUND
1.1f, // WIGGLE
0.9f, // BOUNCE
+ 1.7f, // WAVE
+ 0.9f, // WINK
)
/**
@@ -210,6 +250,8 @@ val IDLE_ACTIONS: List<Pair<Action, Float>> = listOf(
Action.WIGGLE to 1.0f,
Action.BOUNCE to 0.7f,
Action.HOP to 0.5f,
+ Action.WAVE to 0.8f,
+ Action.WINK to 0.8f,
)
/**
diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/Pose.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/Pose.kt
new file mode 100644
index 0000000..3975ca5
--- /dev/null
+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/Pose.kt
@@ -0,0 +1,198 @@
+package net.kuafuai.andee.ui.ball
+
+/**
+ * Where one hand wants to be, in the ball's own space (body radius 0.87, +X to the
+ * viewer's right, +Y up). The hand sits on the body's surface at that x/y, so
+ * only two numbers are needed; its depth is worked out at draw time.
+ *
+ * Hands are the part of the face the old moods had no word for. A closed fist
+ * under the chin says *thinking*, two raised fists say *we did it*, a hand over
+ * the belly says *resting* — none of which a round eye and a curved mouth can
+ * say, and all of which survive being drawn a thumb wide because they are
+ * shapes, not colours.
+ */
+class HandPose(
+ val x: Float,
+ val y: Float,
+ /** 0 = tucked away. The hand grows from nothing at its own spot, it does not fly in. */
+ val s: Float,
+ /**
+ * Degrees counter-clockwise on screen, for *this* hand as drawn. The geometry
+ * is mirrored for the left hand but the angle is not: a raised left fist
+ * leaning outward is a positive roll, a raised right one a negative.
+ */
+ val roll: Float = 0f,
+ /** 0 = open mitten, 1 = fist. */
+ val fist: Float = 0f,
+ /** 1 = the finger on the body side is out. */
+ val point: Float = 0f,
+ /** Idle motion: a swing of the wrist if [wave], otherwise a small vertical bob. */
+ val jig: Float = 0f,
+ val wave: Boolean = false,
+)
+
+/**
+ * Everything a mood adds on top of [Face], and the reason the two are separate
+ * classes: [Face] is a row of numbers every look interprets, and each number is
+ * a *continuous* dial. These are the discrete things — a wink, a hand, a star —
+ * that only the looks with [BallLook.gestures] draw. A look without it ignores
+ * the whole table, so adding a mood here can never change what the other five
+ * characters look like.
+ *
+ * Each value is a target, not a state. [EmotionState] tweens toward it, which is
+ * what makes a hand slide to the chin instead of blinking onto it, and a star
+ * pop in instead of appearing.
+ */
+class Pose(
+ /** The viewer's-right eye closes into an arc and the other stays open. */
+ val wink: Float = 0f,
+ /** Where the eyes look, −1..1; positive X is the viewer's right, positive Y is up. */
+ val lookX: Float = 0f,
+ val lookY: Float = 0f,
+ /**
+ * 0 = round eyes, 1 = the top of each eye sliced flat and rolled inward: a
+ * determined look. **Negative rolls the other way and slices nothing** — the
+ * slant of a worried brow, which is what the sad rows need: their [Face.brow]
+ * rolls the eyes the angry way, and this is what turns it back.
+ */
+ val lidCut: Float = 0f,
+ /** Resting body squash on top of breathing. Negative Y flattens, positive X widens. */
+ val sqx: Float = 0f,
+ val sqy: Float = 0f,
+ val handL: HandPose = HAND_OFF_L,
+ val handR: HandPose = HAND_OFF_R,
+ val stars: Float = 0f,
+ val sparkL: Float = 0f,
+ val sparkR: Float = 0f,
+ val sparkTop: Float = 0f,
+ val shield: Float = 0f,
+ val drop: Float = 0f,
+ /** Closed eyes drawn as `⌣` instead of `⌢`: a lid, not a laugh. Not tweened — it flips. */
+ val arcFlip: Boolean = false,
+) {
+ companion object {
+ val NONE = Pose()
+ }
+}
+
+const val PROP_STARS = 0
+const val PROP_SPARK_L = 1
+const val PROP_SPARK_R = 2
+const val PROP_SPARK_TOP = 3
+const val PROP_SHIELD = 4
+const val PROP_DROP = 5
+const val PROP_COUNT = 6
+
+val HAND_OFF_L = HandPose(-0.62f, -0.15f, 0f)
+val HAND_OFF_R = HandPose(0.62f, -0.15f, 0f)
+
+/** The wave a [Action.WAVE] fidget borrows, whatever mood is showing. */
+val HAND_WAVE = HandPose(-1.18f, -0.22f, 1f, roll = -10f, jig = 1f, wave = true)
+
+/**
+ * The poses. The numbers were read off the reference sketches (wink, wave, think,
+ * rest, protect, celebrate) the same way the look's own were off the picture:
+ * measured on the sketch, divided by the head's width, times the body's 1.74
+ * diameter. Where there was no sketch — the work moods, the worried ones — they
+ * are the nearest gesture a person would make.
+ */
+val POSES: Map<Mood, Pose> = mapOf(
+ Mood.CALM to Pose(),
+
+ // Open-mouthed and pleased, with a little burst above the head. No hands: a
+ // happy face is the one that most needs its face visible.
+ Mood.HAPPY to Pose(sparkTop = 1f),
+
+ // Eyes up and to one side, as if something is on the shelf above you.
+ Mood.CURIOUS to Pose(lookX = 0.6f, lookY = 0.6f),
+
+ // Annoyed rather than hurt: the lids come down, and nothing else moves.
+ Mood.TENSE to Pose(lidCut = 0.8f),
+
+ // Fists drawn in at the chest, a bead of sweat, and the shiver the Face row
+ // already supplies.
+ Mood.ANXIOUS to Pose(
+ drop = 1f, lookY = 0.1f, lidCut = -1f,
+ handL = HandPose(-0.24f, -0.46f, 0.8f, roll = 25f, fist = 1f, jig = 0.12f),
+ handR = HandPose(0.24f, -0.46f, 0.8f, roll = -25f, fist = 1f, jig = 0.12f),
+ ),
+
+ // Hands folded low, eyes cast a little down: sorry, and about to say so.
+ Mood.CONCERNED to Pose(
+ lookY = -0.2f, lidCut = -1f,
+ handL = HandPose(-0.17f, -0.64f, 0.85f, roll = 30f, fist = 0.4f),
+ handR = HandPose(0.17f, -0.64f, 0.85f, roll = -30f, fist = 0.4f),
+ ),
+
+ // wink.webp — a closed eye, an open laughing mouth, a finger pointing at you.
+ Mood.WINK to Pose(
+ wink = 1f, sparkR = 1f,
+ handR = HandPose(0.80f, -0.52f, 0.95f, roll = 40f, fist = 1f, point = 1f, jig = 0.4f),
+ ),
+
+ // wave.webp — a hand up at the side, swinging, with its motion lines.
+ Mood.WAVE to Pose(sparkL = 1f, handL = HAND_WAVE),
+
+ // protect.webp — brows set, a fist at the ready and a shield held out.
+ Mood.PROTECT to Pose(
+ lidCut = 1f, shield = 1f,
+ handL = HandPose(-0.84f, -0.50f, 1f, roll = 20f, fist = 1f),
+ ),
+
+ // celebrate.webp — both fists in the air, stars, an open grin.
+ Mood.CELEBRATE to Pose(
+ stars = 1f, sparkTop = 1f,
+ handL = HandPose(-1.12f, 0.0f, 1f, roll = 18f, fist = 1f, jig = 1f),
+ handR = HandPose(1.12f, 0.0f, 1f, roll = -18f, fist = 1f, jig = 1f),
+ ),
+
+ // rest.webp — squashed down, eyes shut, both hands on the belly.
+ Mood.REST to Pose(
+ sqx = 0.08f, sqy = -0.13f, arcFlip = true,
+ handL = HandPose(-0.47f, -0.50f, 0.9f, roll = 20f, fist = 0.2f),
+ handR = HandPose(0.55f, -0.46f, 0.9f, roll = -20f, fist = 0.2f),
+ ),
+
+ // A hand cupped behind the ear: the mic is open.
+ Mood.LISTENING to Pose(
+ handL = HandPose(-0.98f, 0.10f, 0.95f, roll = -25f, fist = 0f),
+ ),
+
+ // Holding the signboard up: both fists raised on either side.
+ Mood.HOLDING to Pose(
+ handL = HandPose(-0.96f, 0.40f, 0.85f, roll = 15f, fist = 1f, jig = 0.25f),
+ handR = HandPose(0.96f, 0.40f, 0.85f, roll = -15f, fist = 1f, jig = 0.25f),
+ ),
+
+ // think.webp — eyes up and to the side, a fist under the chin.
+ Mood.THINKING to Pose(
+ lookX = 0.5f, lookY = 0.9f,
+ handL = HandPose(-0.04f, -0.66f, 1f, roll = 25f, fist = 1f),
+ ),
+
+ // The mouth is the talk envelope's, which opens the thin smile rather than
+ // swapping it for another shape. No hands, so the face is clear while it is
+ // being listened to.
+ Mood.SPEAKING to Pose(),
+
+ // Driving an app: one finger out, tapping at the glass.
+ Mood.SEARCHING to Pose(
+ lookY = -0.15f,
+ handR = HandPose(0.80f, -0.40f, 0.9f, roll = 35f, fist = 1f, point = 1f, jig = 0.7f),
+ ),
+
+ Mood.SUCCESS to Pose(
+ stars = 1f, sparkTop = 1f,
+ handL = HandPose(-1.0f, 0.0f, 1f, roll = 18f, fist = 1f, jig = 1f),
+ handR = HandPose(1.0f, 0.0f, 1f, roll = -18f, fist = 1f, jig = 1f),
+ ),
+
+ // Eyes down, a bead of sweat, slumped.
+ Mood.FAILED to Pose(lookY = -0.5f, lidCut = -1.2f, drop = 1f, sqy = -0.05f),
+
+ Mood.SLEEPING to Pose(
+ sqx = 0.08f, sqy = -0.13f, arcFlip = true,
+ handL = HandPose(-0.47f, -0.50f, 0.9f, roll = 20f, fist = 0.2f),
+ handR = HandPose(0.55f, -0.46f, 0.9f, roll = -20f, fist = 0.2f),
+ ),
+)
diff --git a/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Puff.kt b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Puff.kt
new file mode 100644
index 0000000..b4c2aaf
--- /dev/null
+++ b/app/src/main/java/net/kuafuai/andee/ui/ball/gl/Puff.kt
@@ -0,0 +1,157 @@
+package net.kuafuai.andee.ui.ball.gl
+
+import kotlin.math.PI
+import kotlin.math.cos
+import kotlin.math.pow
+import kotlin.math.sin
+import kotlin.math.sqrt
+
+/**
+ * A flat outline pushed up into a pillow — the soft, puffy sticker the stars,
+ * the shield and the tears are drawn as.
+ *
+ * Flat-filled shapes read as clip-art; the same outline with a dome on it, lit
+ * by the shell shader, reads as something you could squeeze. The whole effect is
+ * in the normals: the centre faces the camera and the edge turns away from it,
+ * so the key light rolls across the shape the way it does across the ball.
+ *
+ * Built as concentric copies of the outline shrinking toward the centre, each
+ * lifted along `z = depth · √(1 − s²)`. That profile has a vertical tangent at
+ * the rim, which would make the outermost normal point straight sideways and
+ * the edge go black against the light, so the last ring is pinned to `z = 0`
+ * with a normal tilted a little toward the viewer.
+ *
+ * The outline must be **star-shaped about the origin** (every point visible from
+ * the centre) — true of every shape here — because the fan from the centre is
+ * what fills it.
+ */
+object Puff {
+
+ fun build(outline: FloatArray, depth: Float, rings: Int = 4): Mesh {
+ val n = outline.size / 2
+ var mean = 0f
+ for (i in 0 until n) mean += sqrt(outline[2 * i] * outline[2 * i] + outline[2 * i + 1] * outline[2 * i + 1])
+ mean /= n
+
+ val vertCount = 1 + (rings + 1) * n
+ val positions = FloatArray(vertCount * 3)
+ val normals = FloatArray(vertCount * 3)
+ val uvs = FloatArray(vertCount * 2)
+
+ positions[2] = depth
+ normals[2] = 1f
+
+ for (k in 1..rings + 1) {
+ val last = k == rings + 1
+ val s = if (last) 1f else k.toFloat() / (rings + 1)
+ val z = if (last) 0f else depth * sqrt(1f - s * s)
+ for (i in 0 until n) {
+ val ox = outline[2 * i]
+ val oy = outline[2 * i + 1]
+ val len = sqrt(ox * ox + oy * oy).coerceAtLeast(1e-4f)
+ val dx = ox / len
+ val dy = oy / len
+ val v = 1 + (k - 1) * n + i
+ positions[v * 3] = ox * s
+ positions[v * 3 + 1] = oy * s
+ positions[v * 3 + 2] = z
+ val nx: Float
+ val ny: Float
+ val nz: Float
+ if (last) {
+ nx = dx; ny = dy; nz = 0.35f
+ } else {
+ val slope = depth * s / (mean * sqrt(1f - s * s))
+ nx = dx * slope; ny = dy * slope; nz = 1f
+ }
+ val inv = 1f / sqrt(nx * nx + ny * ny + nz * nz)
+ normals[v * 3] = nx * inv
+ normals[v * 3 + 1] = ny * inv
+ normals[v * 3 + 2] = nz * inv
+ uvs[v * 2] = 0.5f + ox * s * 0.5f
+ uvs[v * 2 + 1] = 0.5f + oy * s * 0.5f
+ }
+ }
+
+ val idx = ShortArray((n + rings * n * 2) * 3)
+ var w = 0
+ for (i in 0 until n) {
+ idx[w++] = 0
+ idx[w++] = (1 + i).toShort()
+ idx[w++] = (1 + (i + 1) % n).toShort()
+ }
+ for (k in 1..rings) {
+ val a0 = 1 + (k - 1) * n
+ val b0 = 1 + k * n
+ for (i in 0 until n) {
+ val j = (i + 1) % n
+ idx[w++] = (a0 + i).toShort(); idx[w++] = (b0 + i).toShort(); idx[w++] = (b0 + j).toShort()
+ idx[w++] = (a0 + i).toShort(); idx[w++] = (b0 + j).toShort(); idx[w++] = (a0 + j).toShort()
+ }
+ }
+ return Mesh(positions, normals, uvs, idx)
+ }
+
+ /**
+ * A star whose points are round. [points] tips, [inner] is the radius of the
+ * valleys as a fraction of the tips', and [power] above 1 narrows the tips.
+ * A tip points straight up.
+ */
+ fun roundStar(points: Int, inner: Float, power: Float = 1f, samples: Int = 80): FloatArray {
+ val out = FloatArray(samples * 2)
+ for (i in 0 until samples) {
+ val phi = i.toFloat() / samples * 2f * PI.toFloat()
+ val k = (0.5f + 0.5f * cos(points * (phi - PI.toFloat() / 2f))).coerceIn(0f, 1f)
+ val r = inner + (1f - inner) * k.pow(power)
+ out[2 * i] = r * cos(phi)
+ out[2 * i + 1] = r * sin(phi)
+ }
+ return out
+ }
+
+ /** A shield: flat shoulders, straight sides, a rounded point at the bottom. */
+ fun shield(): FloatArray {
+ val pts = floatArrayOf(
+ -0.82f, 0.86f, 0f, 0.98f, 0.82f, 0.86f,
+ 0.86f, 0.10f, 0.58f, -0.52f, 0f, -1.0f,
+ -0.58f, -0.52f, -0.86f, 0.10f,
+ )
+ return chaikin(pts, 2)
+ }
+
+ /** A teardrop, tip up. Centred a little low so the origin is inside the belly. */
+ fun drop(samples: Int = 28): FloatArray {
+ val half = ArrayList<Float>()
+ for (i in 0..samples) {
+ val t = i.toFloat() / samples * PI.toFloat()
+ val x = 0.66f * sin(t) * sin(t / 2f).pow(0.9f)
+ val y = cos(t)
+ half.add(x); half.add(y)
+ }
+ val out = ArrayList<Float>()
+ for (i in 0..samples) { out.add(half[2 * i]); out.add(half[2 * i + 1]) }
+ for (i in samples - 1 downTo 1) { out.add(-half[2 * i]); out.add(half[2 * i + 1]) }
+ val arr = out.toFloatArray()
+ for (i in 0 until arr.size / 2) arr[2 * i + 1] -= 0.08f
+ return arr
+ }
+
+ private fun chaikin(p: FloatArray, iterations: Int): FloatArray {
+ var cur = p
+ repeat(iterations) {
+ val n = cur.size / 2
+ val nxt = FloatArray(n * 4)
+ for (i in 0 until n) {
+ val j = (i + 1) % n
+ val x0 = cur[2 * i]; val y0 = cur[2 * i + 1]
+ val x1 = cur[2 * j]; val y1 = cur[2 * j + 1]
+ nxt[4 * i] = 0.75f * x0 + 0.25f * x1
+ nxt[4 * i + 1] = 0.75f * y0 + 0.25f * y1
+ nxt[4 * i + 2] = 0.25f * x0 + 0.75f * x1
+ nxt[4 * i + 3] = 0.25f * y0 + 0.75f * y1
+ }
+ cur = nxt
+ }
+ return cur
+ }
+}
diff --git a/app/src/test/java/net/kuafuai/andee/brain/SpeechMoodTest.kt b/app/src/test/java/net/kuafuai/andee/brain/SpeechMoodTest.kt
index 567bee9..454e075 100644
--- a/app/src/test/java/net/kuafuai/andee/brain/SpeechMoodTest.kt
+++ b/app/src/test/java/net/kuafuai/andee/brain/SpeechMoodTest.kt
@@ -120,4 +120,38 @@ class SpeechMoodTest {
assertEquals("第一句。\n第二句。", s.text)
assertEquals(listOf(SpeechMood.Cue(5, Mood.HAPPY)), s.cues)
}
+
+ /**
+ * The five gestures are reachable by their own names and by the ordinary word
+ * a model would write instead. A name that is missing here is the old
+ * `[angry]` bug again: shown in the scrollback and read out as a word.
+ */
+ @Test
+ fun `the five gestures are reachable by name and by synonym`() {
+ val cases = mapOf(
+ "wink" to Mood.WINK, "眨眼" to Mood.WINK,
+ "wave" to Mood.WAVE, "bye" to Mood.WAVE, "再见" to Mood.WAVE,
+ "protect" to Mood.PROTECT, "守护" to Mood.PROTECT,
+ "celebrate" to Mood.CELEBRATE, "yay" to Mood.CELEBRATE, "庆祝" to Mood.CELEBRATE,
+ "rest" to Mood.REST, "sleepy" to Mood.REST, "休息" to Mood.REST,
+ )
+ for ((word, mood) in cases) {
+ val s = SpeechMood.parse("[$word]好了。")
+ assertEquals("好了。", s.text)
+ assertEquals("[$word]", listOf(SpeechMood.Cue(0, mood)), s.cues)
+ }
+ }
+
+ /**
+ * Widening the vocabulary must not have moved a word that already meant
+ * something: `excited` and `proud` were HAPPY before there was a CELEBRATE,
+ * and a reply that said so should not start throwing stars.
+ */
+ @Test
+ fun `older words keep their older faces`() {
+ assertEquals(listOf(SpeechMood.Cue(0, Mood.HAPPY)), SpeechMood.parse("[excited]好!").cues)
+ assertEquals(listOf(SpeechMood.Cue(0, Mood.HAPPY)), SpeechMood.parse("[proud]好!").cues)
+ assertEquals(listOf(SpeechMood.Cue(0, Mood.CALM)), SpeechMood.parse("[relaxed]好!").cues)
+ assertEquals(listOf(SpeechMood.Cue(0, Mood.CONCERNED)), SpeechMood.parse("[serious]好!").cues)
+ }
}