Skip to content

Turn one illustration into an animated 2D mesh avatar with a coding agent and a local editor

Switch to drawn eyes for angry, sad and surprised expressions

shinshin86478a8ffc6b14
diff --git a/.changeset/expression-drawn-eyes.md b/.changeset/expression-drawn-eyes.md
new file mode 100644
index 0000000..05a0426
--- /dev/null
+++ b/.changeset/expression-drawn-eyes.md
@@ -0,0 +1,5 @@
+---
+"mesh-avatar": minor
+---
+
+Switch to drawn expression eyes (`eyes_angry`, `eyes_sad`, `eyes_surprise` sprites) while the angry, sad or surprised expression or emotion tag is active. Blinking still closes them, and avatars without these sprites look the same as before.
diff --git a/README.ja.md b/README.ja.md
index 51ca065..a8911a5 100644
--- a/README.ja.md
+++ b/README.ja.md
@@ -68,6 +68,8 @@ npm run dev
描き分け画像がない間は、目と口をメッシュの変形だけで動かしています。閉じた目や母音ごとの口の絵を用意すると、まばたきと口の動きがかなり自然になります。「描き分け画像」の欄で「目」「口」の片方または両方にチェックを入れ、表示された文章をコピーして、このリポジトリのフォルダで開いた Codex に貼り付けてください。Codex が組み込みの画像生成で絵を作って取り込み、エディタが自動で読み込み直します。Claude Code は画像を生成できないため、Claude Code 向けの文章では、マスクと依頼文を用意して画像生成に渡すファイルを案内するところまでを頼みます。
+「表情の目」は任意です。怒った目・悲しい目・驚いた目を描き分けると、その表情や感情タグのときに描いた目へ切り替わります。まばたきはそのまま動きます。描いていないアバターの見た目は今までと変わりません。
+
![口の描き分けの依頼](docs/images/ja/05-variants.png)
## アバターを共有する
diff --git a/README.md b/README.md
index 90e30f3..275e03e 100644
--- a/README.md
+++ b/README.md
@@ -94,6 +94,10 @@ repository folder. Codex draws them with its built-in image generation and impor
editor reloads them automatically. (Claude Code cannot generate images; its message prepares
the masks and prompts for you to hand to an image generator.)
+**Expression eyes** is optional: it adds drawn angry, sad and surprised eyes. The avatar
+switches to them for those expressions and emotion tags and still blinks; avatars without
+them look the same as before.
+
![Requesting drawn mouths](docs/images/en/05-variants.png)
## Share your avatar
diff --git a/docs/agent-guide.md b/docs/agent-guide.md
index c4b00e4..c4fa477 100644
--- a/docs/agent-guide.md
+++ b/docs/agent-guide.md
@@ -131,8 +131,9 @@ npm run render-poses -- projects/<name>
```
This renders the avatar in fixed poses (rest, half and closed eyes, mouth open, turn left and
-right, look up and down, tilt, body tilt, and a hair sway sequence) into
-`projects/<name>/review/`, with a contact sheet and zoomed eye and mouth crops.
+right, look up and down, tilt, body tilt, the angry, sad and surprised expressions, and a hair
+sway sequence) into `projects/<name>/review/`, with a contact sheet and zoomed eye and mouth
+crops.
Look at every frame. Pass when:
- closed eyes are fully closed, with one clean lash line and no leftover iris or white;
@@ -150,7 +151,7 @@ Without drawn variants, a closed eye can keep faint fragments of the original up
crease above the closed line. If iris and white are gone and only such faint lines remain,
the rig is good enough: continue to Step 6, which removes them.
-## Step 6 — Drawn variants (closed eyes, mouth shapes)
+## Step 6 — Drawn variants (closed eyes, mouth shapes, expression eyes)
Codex uses its built-in image generation (imagegen) for these variants.
@@ -161,7 +162,13 @@ uv run tools/variant-requests.py projects/<name>
```
This writes `projects/<name>/variant-requests/` with, for each variant (closed / half /
-smiling eyes; mouths あ, half あ, い, お): an edit mask, the prompt, and a README.
+smiling eyes; mouths あ, half あ, い, お; angry / sad / surprised expression eyes): an edit
+mask, the prompt, and a README.
+
+The expression eyes (`eyes_angry`, `eyes_sad`, `eyes_surprise`) are optional; create them
+only when the person asks for them. They use the eye mask, which leaves the eyebrows out: the
+rig already moves the brows for each expression, and a drawn brow would show twice. The
+half-lidded expression uses `eyes_half`, so it needs no separate drawing.
Ask the person before generating: image generation sends `source.png` and a mask to the
generator. If they decline, stop here; the avatar still works without variants.
@@ -173,16 +180,20 @@ generator. If they decline, stop here; the avatar still works without variants.
and where to put the results. Continue when they say the files are in place.
In each mask, transparent pixels may change and opaque pixels must stay identical. Many
-generators recompress or shift the whole image; if a variant is rejected for changes outside
-the mask, generate it again, asking explicitly for identical size and untouched pixels outside
-the masked area.
+generators recompress the whole image, fill in the transparent background or shift it by a
+pixel. `build-sprites` then imports the masked area only: it corrects a shift of up to 3 px,
+resizes a result with the source's aspect ratio, and logs `importing the mask area only`. It
+still rejects, with the reason, a result whose whole image is shifted further, whose picture
+outside the mask was redrawn, whose drawing spills over the mask edge, or whose aspect ratio
+differs. Generate a rejected variant again, asking for identical size and untouched pixels
+outside the masked area.
Then:
```sh
uv run tools/build-sprites.py projects/<name>
```
-It rejects variants whose size differs from the source or that change pixels outside the
-mask, and cuts accepted variants into `built/sprites/`. Render the poses again (Step 5).
+It cuts accepted variants into `built/sprites/`. Render the poses again (Step 5); the
+`angry`, `sad` and `surprise` frames show the expression eyes.
## Step 7 — Hand over
diff --git a/docs/format.md b/docs/format.md
index efc9b81..9be2b42 100644
--- a/docs/format.md
+++ b/docs/format.md
@@ -123,12 +123,20 @@ individually; readers must not require a complete set.
| `eyes_closed_0`, `eyes_closed_1` | Closed eyes with relaxed lids. |
| `eyes_half_0`, `eyes_half_1` | Half-open eyes. |
| `eyes_smile_0`, `eyes_smile_1` | Closed eyes in a smile. |
+| `eyes_angry_0`, `eyes_angry_1` | Open eyes for the angry expression. |
+| `eyes_sad_0`, `eyes_sad_1` | Open eyes for the sad expression. |
+| `eyes_surprise_0`, `eyes_surprise_1` | Wide-open eyes for the surprised expression. |
| `mouth_a` | Open mouth for “a”. |
| `mouth_a_half` | Partly open “a”; also reused for “e”. |
| `mouth_i` | Wide mouth for “i”. |
| `mouth_o` | Rounded mouth for “o”; also reused, with reduced width, for “u”. |
The eye suffix is the index in `rig.eyes`, not a separate left/right convention.
+The expression eyes replace the open and half-open eye while the live expression
+(`setExpression`: `angry`, `sad`, `surprise`) or the emotion tag (`setEmotion`: `angry`,
+`sad`, `surprised`) is active; a blink still switches to `eyes_closed`. They cover the
+same area as the other eye drawings and do not include the eyebrows, which the rig moves.
+Without them, those expressions use the other drawings as before.
The original base drawing provides the closed mouth.
## Archive rules and compatibility
diff --git a/e2e/workflow.spec.ts b/e2e/workflow.spec.ts
index e6e862f..243c697 100644
--- a/e2e/workflow.spec.ts
+++ b/e2e/workflow.spec.ts
@@ -71,7 +71,7 @@ test('a failed build leaves the previous layers and view, with a useful reason a
});
test('requests provide masks, root handoff and prompts; drops accept valid drawings and reject outside changes atomically', async ({ page, context }) => {
test.setTimeout(120000); await context.grantPermissions(['clipboard-read', 'clipboard-write']); await open(page);
- const panel = page.getByTestId('variants-panel'); await expect(panel.getByRole('checkbox')).toHaveCount(2);
+ const panel = page.getByTestId('variants-panel'); await expect(panel.getByRole('checkbox')).toHaveCount(3);
await expect(panel.getByTestId('ask-agent-variants')).toHaveCount(0);
await expect(panel.locator('.manual-variants')).not.toHaveAttribute('open', '');
await panel.getByRole('checkbox', { name: 'Mouth', exact: true }).check();
@@ -133,6 +133,9 @@ test('eye and mouth checkboxes immediately adapt the prompt and lip fallback ope
const prompt = panel.getByRole('textbox', { name: 'Copy message', exact: true });
await panel.getByRole('checkbox', { name: 'Eyes', exact: true }).check(); await expect(prompt).toContainText('eye drawn variants');
await panel.getByRole('checkbox', { name: 'Mouth', exact: true }).check(); await expect(prompt).toContainText('eye and mouth drawn variants');
+ await panel.getByRole('checkbox', { name: 'Expression eyes', exact: true }).check();
+ await expect(prompt).toContainText('eye, expression eye (angry, sad, surprised) and mouth drawn variants');
+ await panel.getByRole('checkbox', { name: 'Expression eyes', exact: true }).uncheck();
await panel.getByRole('checkbox', { name: 'Eyes', exact: true }).uncheck(); await expect(prompt).toContainText('mouth drawn variants');
await panel.getByRole('checkbox', { name: 'Mouth', exact: true }).uncheck(); await expect(prompt).toHaveCount(0);
await panel.locator(':scope > summary').click();
diff --git a/packages/runtime/src/engine/createMeshAvatar.js b/packages/runtime/src/engine/createMeshAvatar.js
index beb3e2b..a2700eb 100644
--- a/packages/runtime/src/engine/createMeshAvatar.js
+++ b/packages/runtime/src/engine/createMeshAvatar.js
@@ -4,7 +4,7 @@
import { createRig } from './rig.js';
import { createRenderer } from './renderer.js';
import { createPhysics } from './physics.js';
-import { createSpriteModule } from './sprites.js';
+import { createSpriteModule, eyeMood } from './sprites.js';
import { Motion } from './motion.js';
import { ExpressionOverlay } from './expression-overlay.js';
import { MOTIONS, IDLE_MOTIONS } from './motions.js';
@@ -182,7 +182,8 @@ export async function createMeshAvatarImpl(canvas, options) {
o[k * 2] = tmp[0]; o[k * 2 + 1] = tmp[1];
}
}
- const cover = sprites?.update(P, phys, dt);
+ // the motion expression is not applied in manual mode, so only the live one counts there
+ const cover = sprites?.update(P, phys, dt, eyeMood(expression.name, motion.mode === 'manual' ? null : motion.expr));
if (cover) for (const ep of eyeParts) if (cover.eyes[ep.eye]) ep.layer.alpha = 0;
const hp = handMesh.pos, hr = handMesh.rest, HF = handFrame(P);
for (let i = 0; i < handW.length; i++) {
diff --git a/packages/runtime/src/engine/sprites.js b/packages/runtime/src/engine/sprites.js
index 9e77312..4feb4f6 100644
--- a/packages/runtime/src/engine/sprites.js
+++ b/packages/runtime/src/engine/sprites.js
@@ -44,7 +44,10 @@ export function createSpriteModule(engine, rig) {
// and the drawn closed / smiling eye.
const EYE_OPEN_MIN = 0.75, EYE_CLOSED_MAX = 0.3, EYE_FADE_SEC = 0.06;
- function eyeSprite(open, smile) {
+ // Open eyes drawn for an expression (eyes_angry / eyes_sad / eyes_surprise) replace the open
+ // and half-open eye while it lasts; a blink still switches to the closed drawing.
+ function eyeSprite(open, smile, mood = null) {
+ if (mood && open >= EYE_CLOSED_MAX) return mood;
if (open >= EYE_OPEN_MIN) return null;
if (open < EYE_CLOSED_MAX) return smile >= 0.5 ? 'eyes_smile' : 'eyes_closed';
return 'eyes_half';
@@ -78,8 +81,9 @@ export function createSpriteModule(engine, rig) {
*/
function createSprites(R, sheet, imgs, buildGrid, alphaOf) {
const items = {};
- // draw order matters for the cross-fades: half-closed under closed / smiling (the fade-in frame is drawn on top)
- const order = n => (n.startsWith('eyes_half') ? 0 : n.startsWith('eyes_') ? 1 : 2);
+ // draw order matters for the cross-fades: half-closed and expression eyes under closed / smiling (the fade-in frame is drawn on top)
+ const shut = n => n.startsWith('eyes_closed') || n.startsWith('eyes_smile');
+ const order = n => (shut(n) ? 1 : n.startsWith('eyes_') ? 0 : 2);
const entries = Object.entries(sheet.layers).sort((x, y) => order(x[0]) - order(y[0]));
for (const [name, rect] of entries) {
const mesh = buildGrid(rect, rig.mesh.spriteCell, alphaOf(imgs[name]), imgs[name].width);
@@ -117,8 +121,9 @@ export function createSpriteModule(engine, rig) {
/**
* Show / hide and deform the sprites for this frame.
* Returns which eyes are covered by a sprite, so the layered eye parts can be hidden.
+ * `mood` (angry / sad / surprise) picks drawn expression eyes when the avatar has them.
*/
- update(P, phys, dt = 1 / 60) {
+ update(P, phys, dt = 1 / 60, mood = null) {
for (const it of Object.values(items)) it.layer.visible = false;
const covered = [false, false];
const eyes = [
@@ -129,7 +134,8 @@ export function createSpriteModule(engine, rig) {
// EYE_FADE_SEC (a time-based cross-fade, so no double image is ever held)
eyes.forEach(([open, smile], i) => {
const st = eyeState[i];
- const want = eyeSprite(open, smile) ?? 'open';
+ const drawn = mood && items[`eyes_${mood}_${i}`] ? `eyes_${mood}` : null;
+ const want = eyeSprite(open, smile, drawn) ?? 'open';
if (want !== st.cur) { st.prev = st.cur; st.cur = want; st.t = 0; }
// A redraw without elapsed time (a paused preview moving a slider) shows the target
// frame at once; otherwise the fade would stay at its first frame.
@@ -145,9 +151,9 @@ export function createSpriteModule(engine, rig) {
};
// the outgoing frame stays fully drawn underneath while the new one fades in on top;
// fading out to the open eye works the other way round
- // whichever frame is drawn higher up (closed/smile above half, both above the open eye)
+ // whichever frame is drawn higher up (closed/smile above half or expression eyes, all above the open eye)
// does the fading, the lower one stays solid, so the eye is never see-through
- const rank = n => (n === 'open' ? -1 : n === 'eyes_half' ? 0 : 1);
+ const rank = n => (n === 'open' ? -1 : shut(n) ? 1 : 0);
if (st.t >= 1) show(st.cur, 1);
else if (rank(st.cur) > rank(st.prev)) { show(st.prev, 1); show(st.cur, k); }
else { show(st.cur, 1); show(st.prev, 1 - k); }
@@ -171,3 +177,12 @@ export function createSpriteModule(engine, rig) {
}
return { createSprites, mouthShape, eyeSprite };
}
+
+// Expressions (live overlay names and motion expression keys) that have drawn open eyes.
+const EYE_MOODS = { angry: 'angry', angryTalk: 'angry', sad: 'sad', sadTalk: 'sad', surprise: 'surprise', surprisedTalk: 'surprise' };
+
+/** The drawn-eye mood for the live expression, else for the motion expression (if it is applied). */
+export function eyeMood(overlay, expression) {
+ if (overlay && overlay !== 'neutral') return EYE_MOODS[overlay] ?? null;
+ return (expression && EYE_MOODS[expression]) ?? null;
+}
diff --git a/src/editor/AskAgent.tsx b/src/editor/AskAgent.tsx
index 8993d85..98090fa 100644
--- a/src/editor/AskAgent.tsx
+++ b/src/editor/AskAgent.tsx
@@ -2,12 +2,12 @@ import { useState } from 'react';
import { readPreference, savePreference } from './preferences';
import { useI18n } from './i18n';
import { CopyButton } from './CopyButton';
-import { revealRepository } from './project';
+import { revealRepository, type VariantGroup } from './project';
-export function AskAgent({ rootPath, projectPath, target = 'both', newProject = false, readOnlySource }: {
- rootPath?: string; projectPath?: string; target?: 'eyes' | 'mouth' | 'both'; newProject?: boolean; readOnlySource?: string;
+export function AskAgent({ rootPath, projectPath, target = ['eyes', 'mouth'], newProject = false, readOnlySource }: {
+ rootPath?: string; projectPath?: string; target?: VariantGroup[]; newProject?: boolean; readOnlySource?: string;
}) {
- const { t } = useI18n();
+ const { t, locale } = useI18n();
const [recipient, setRecipient] = useState(() => readPreference('mesh-avatar-agent-recipient') === 'claude' ? 'claude' : 'codex');
const [revealError, setRevealError] = useState(false);
const reveal = async () => {
@@ -16,7 +16,7 @@ export function AskAgent({ rootPath, projectPath, target = 'both', newProject =
};
const agent = newProject || recipient === 'codex' ? 'Codex' : 'Claude Code';
const path = projectPath ?? 'projects/my-avatar';
- const prompt = (readOnlySource ? t.agentCopySample.replace('SOURCE', readOnlySource).replace('PROJECT', path) : '') + (newProject ? t.agentNewPrompt : recipient === 'codex' ? t.agentCodexPrompt : t.agentClaudePrompt).replace('PROJECT', path).replace('TARGET', target === 'eyes' ? t.agentEyes : target === 'mouth' ? t.agentMouth : t.agentBoth);
+ const prompt = (readOnlySource ? t.agentCopySample.replace('SOURCE', readOnlySource).replace('PROJECT', path) : '') + (newProject ? t.agentNewPrompt : recipient === 'codex' ? t.agentCodexPrompt : t.agentClaudePrompt).replace('PROJECT', path).replace('TARGET', new Intl.ListFormat(locale).format(target.map(group => ({ eyes: t.agentEyes, expression: t.agentExpression, mouth: t.agentMouth })[group])));
return <section className="ask-agent" data-testid={newProject ? 'ask-agent-new' : 'ask-agent-variants'} aria-label={t.askAgent}>
<p className="agent-instruction">{t.agentInstruction.replace('AGENT', agent)}</p>
{rootPath ? <div className="agent-folder-actions"><CopyButton value={rootPath} label={t.copyFolderPath} /><button type="button" onClick={() => { void reveal(); }}>{t.showFolder}</button></div> : <p className="workflow-note">{t.rootUnavailable}</p>}
diff --git a/src/editor/JobFeedback.tsx b/src/editor/JobFeedback.tsx
index 6c7b412..3c04b08 100644
--- a/src/editor/JobFeedback.tsx
+++ b/src/editor/JobFeedback.tsx
@@ -10,7 +10,7 @@ export function JobFeedback({ error, log = '', accepted = false, importing = fal
const metrics = [...details.matchAll(/(eyes_\w+|mouth_\w+): outside-mask max ([\d.]+)\/255, mean ([\d.]+)\/255, (\d+) pixels over tolerance ([\d.]+)/g)];
return <div className="job-feedback" data-testid="job-feedback">
{error && <p role="alert">{error.code === 'reloadFailed' ? t.reloadFailed : <>{importing ? t.importRejected : t.jobFailed}{reason !== t.jobFailed && ` ${reason}`}</>}</p>}
- {accepted && <p role="status">{t.importAccepted}</p>}
+ {accepted && <p role="status">{t.importAccepted}{/importing the mask area only/.test(details) && ` ${t.importMaskOnly}`}</p>}
{metrics.length > 0 && <details><summary>{t.metricsTitle}</summary>{metrics.map(metric => <p key={metric[1]}><code>{metric[1]}</code>: {t.outsideMax} {metric[2]}/255 · {t.outsideMean} {metric[3]}/255 · {t.outsidePixels} {metric[4]} ({metric[5]}/255)</p>)}</details>}
{details && <details><summary>{t.showLog}</summary><pre>{details}</pre></details>}
</div>;
diff --git a/src/editor/VariantsPanel.tsx b/src/editor/VariantsPanel.tsx
index 6b3178b..c9cc1eb 100644
--- a/src/editor/VariantsPanel.tsx
+++ b/src/editor/VariantsPanel.tsx
@@ -1,20 +1,21 @@
import { useEffect, useRef, useState } from 'react';
import { useI18n } from './i18n';
-import { variantNames, projectVariantRequests, openSampleAvatar, ProjectJobError, type LocalProject, type ProjectJob, type JobResult, type VariantRequest } from './project';
+import { variantGroups, variantNames, projectVariantRequests, openSampleAvatar, ProjectJobError, type LocalProject, type ProjectJob, type VariantGroup, type JobResult, type VariantRequest } from './project';
import { AskAgent } from './AskAgent';
import { JobFeedback } from './JobFeedback';
import { CopyButton } from './CopyButton';
import { Icon } from './Icon';
-const labels = ['eyesClosed', 'eyesHalf', 'eyesSmile', 'mouthA', 'mouthAHalf', 'mouthI', 'mouthO'] as const;
+const labels = ['eyesClosed', 'eyesHalf', 'eyesSmile', 'eyesAngry', 'eyesSad', 'eyesSurprise', 'mouthA', 'mouthAHalf', 'mouthI', 'mouthO'] as const;
+const groupOf = (name: string) => (Object.keys(variantGroups) as VariantGroup[]).find(group => (variantGroups[group] as readonly string[]).includes(name));
export function VariantsPanel({ project, projectPath, assets, rootPath, busy, stale, onRun, mouthRequest, onMouthPresence }: {
project: LocalProject | null; projectPath?: string; assets?: Record<string, string>; rootPath?: string; busy: boolean; stale: boolean;
onRun: (action: ProjectJob, files?: File[]) => Promise<JobResult>; mouthRequest: number; onMouthPresence: (present: boolean) => void;
}) {
const { t } = useI18n();
const [present, setPresent] = useState<string[]>([]), [requests, setRequests] = useState<VariantRequest[]>([]);
- const [eyes, setEyes] = useState(false), [mouth, setMouth] = useState(false);
- const target = eyes && mouth ? 'both' : eyes ? 'eyes' : 'mouth';
+ const [eyes, setEyes] = useState(false), [expression, setExpression] = useState(false), [mouth, setMouth] = useState(false);
+ const target = ([['eyes', eyes], ['expression', expression], ['mouth', mouth]] as const).filter(([, on]) => on).map(([group]) => group);
const panel = useRef<HTMLDetailsElement>(null), mouthCheckbox = useRef<HTMLInputElement>(null);
useEffect(() => {
if (!mouthRequest) return;
@@ -28,7 +29,7 @@ export function VariantsPanel({ project, projectPath, assets, rootPath, busy, st
const writable = !!project && !project.readOnly;
// Loading metadata for the current project must not reset choices made during loading.
const identity = project?.relativePath ?? projectPath;
- useEffect(() => { setEyes(false); setMouth(false); setError(null); setLog(''); setAccepted(false); setNotice(''); }, [identity]);
+ useEffect(() => { setEyes(false); setExpression(false); setMouth(false); setError(null); setLog(''); setAccepted(false); setNotice(''); }, [identity]);
useEffect(() => {
const revision = ++current.current;
setPresent([]); setRequests([]);
@@ -51,23 +52,24 @@ export function VariantsPanel({ project, projectPath, assets, rootPath, busy, st
setAccepted(action === 'import-variants');
} catch (error) { setError(error instanceof ProjectJobError ? error : new ProjectJobError('toolFailed')); }
};
- const count = (prefix: string, total: number) => {
- const value = present.filter(name => name.startsWith(prefix)).length;
- return value ? t.variantCount.replace('COUNT', String(value)).replace('TOTAL', String(total)) : t.variantMissing;
+ const count = (group: VariantGroup) => {
+ const value = present.filter(name => groupOf(name) === group).length;
+ return value ? t.variantCount.replace('COUNT', String(value)).replace('TOTAL', String(variantGroups[group].length)) : t.variantMissing;
};
return <details ref={panel} className="panel variants-panel" data-testid="variants-panel" open>
<summary><Icon name="brush" />{t.variantsTitle}</summary>
<div className="variants-content"><p>{t.variantsHelp}</p>
<div className="variant-choices">
- <label><input type="checkbox" checked={eyes} onChange={event => setEyes(event.target.checked)} aria-label={t.targetEyes} /><span>{t.targetEyes}<small>{t.eyesKinds}</small></span><span className="variant-count">{count('eyes_', 3)}</span></label>
- <label><input ref={mouthCheckbox} type="checkbox" checked={mouth} onChange={event => setMouth(event.target.checked)} aria-label={t.targetMouth} /><span>{t.targetMouth}<small>{t.mouthKinds}</small></span><span className="variant-count">{count('mouth_', 4)}</span></label>
+ <label><input type="checkbox" checked={eyes} onChange={event => setEyes(event.target.checked)} aria-label={t.targetEyes} /><span>{t.targetEyes}<small>{t.eyesKinds}</small></span><span className="variant-count">{count('eyes')}</span></label>
+ <label><input type="checkbox" checked={expression} onChange={event => setExpression(event.target.checked)} aria-label={t.targetExpression} /><span>{t.targetExpression}<small>{t.expressionKinds}</small></span><span className="variant-count">{count('expression')}</span></label>
+ <label><input ref={mouthCheckbox} type="checkbox" checked={mouth} onChange={event => setMouth(event.target.checked)} aria-label={t.targetMouth} /><span>{t.targetMouth}<small>{t.mouthKinds}</small></span><span className="variant-count">{count('mouth')}</span></label>
</div>
- {(eyes || mouth) && projectPath && (projectPath.startsWith('<') ? <p className="workflow-note">{t.agentUnknownPath}</p> : <AskAgent rootPath={rootPath} projectPath={project?.readOnly || projectPath === 'samples/miko-qipao' ? 'projects/miko-qipao-variants' : projectPath} readOnlySource={project?.readOnly || projectPath === 'samples/miko-qipao' ? projectPath : undefined} target={target} />)}
+ {target.length > 0 && projectPath && (projectPath.startsWith('<') ? <p className="workflow-note">{t.agentUnknownPath}</p> : <AskAgent rootPath={rootPath} projectPath={project?.readOnly || projectPath === 'samples/miko-qipao' ? 'projects/miko-qipao-variants' : projectPath} readOnlySource={project?.readOnly || projectPath === 'samples/miko-qipao' ? projectPath : undefined} target={target} />)}
<details className="manual-variants"><summary>{t.manualVariants}</summary>
<p>{t.manualHelp}</p>
{writable ? <button disabled={busy || stale} onClick={() => { void run('variant-requests'); }}>{busy ? t.operationBusy : t.createRequests}</button> : <p className="workflow-note">{t.variantsUnavailable}</p>}
{stale && <p className="workflow-note">{t.variantsNeedRebuild}</p>}
- {requests.length > 0 && <div className="variant-requests">{requests.filter(request => (!eyes && !mouth) || target === 'both' || request.name.startsWith(target === 'eyes' ? 'eyes_' : 'mouth_')).map(request => <details key={request.name}>
+ {requests.length > 0 && <div className="variant-requests">{requests.filter(request => !target.length || target.includes(groupOf(request.name)!)).map(request => <details key={request.name}>
<summary>{t[labels[variantNames.indexOf(request.name)]]} · <code>{request.name}.png</code></summary>
<img src={request.maskUrl} alt={`${t.requestFiles}: ${t[labels[variantNames.indexOf(request.name)]]}`} />
<p><code>{project?.relativePath}/source.png</code><br /><code>{project?.relativePath}/variant-requests/{request.name}/mask.png</code></p>
diff --git a/src/editor/i18n-zh.ts b/src/editor/i18n-zh.ts
index ae61f80..2e57acd 100644
--- a/src/editor/i18n-zh.ts
+++ b/src/editor/i18n-zh.ts
@@ -47,7 +47,7 @@ export const uiZh = {
export const workflowZh: typeof workflowEn = {
agentInstruction: '在本项目文件夹中打开 AGENT,然后粘贴下面的请求。',
variantsReloaded: '已加载表情差分。', mouthFallback: '尚无嘴型差分图片,正在使用网格变形。', drawMouth: '→ 制作嘴型差分',
- eyesKinds: '闭眼、半闭眼、笑眼', mouthKinds: 'あ(a)、半张嘴あ(a)、い(i)、お(o)', variantCount: 'COUNT / TOTAL 张',
+ eyesKinds: '闭眼、半闭眼、笑眼', expressionKinds: '生气、悲伤、惊讶(可选)', mouthKinds: 'あ(a)、半张嘴あ(a)、い(i)、お(o)', variantCount: 'COUNT / TOTAL 张',
askAgent: '请编程助手制作',
copyPrompt: '复制请求', copyDone: '已复制', rootUnavailable: '请在包含本应用和 docs/agent-guide.md 的文件夹中打开编程助手。启动本地开发服务后,可在此复制完整路径。',
newIllustration: '从新插画创建角色',
@@ -57,7 +57,7 @@ export const workflowZh: typeof workflowEn = {
agentClaudePrompt: '请按照 docs/agent-guide.md 的步骤 6,为 PROJECT 的 TARGET差分准备制作请求。运行 variant-requests 生成蒙版和提示词。Claude Code 不能生成图片:请明确告诉我,需要将哪些 source.png、蒙版和提示词交给 Codex 或其他生图工具,以及应以哪些完整尺寸的 PNG 文件名保存到 variants/,然后停止。暂时不要生成或发送图片,也不要运行 build-sprites。所有文件都保存在本仓库内。',
agentCopySample: '请先将只读项目 SOURCE 复制到 PROJECT;如果目标已存在,请使用一个未占用的新名称。保留原项目。',
agentUnknownPath: '浏览器无法读取该文件夹的完整路径。请从本地开发服务的项目列表中打开它,以生成包含准确路径的请求。',
- agentEyes: '眼睛', agentMouth: '嘴型', agentBoth: '眼睛和嘴型',
+ agentEyes: '眼睛', agentExpression: '表情眼睛(生气、悲伤、惊讶)', agentMouth: '嘴型',
rebuild: '保存并重建图层', rebuilding: '正在保存并重建图层…', staleLocal: '轮廓已修改,重建图层后即可更新预览。',
staleSample: '示例项目是只读的。请复制为自己的项目后继续编辑。', copyContinue: '复制并继续编辑',
staleListed: '此项目已在本地项目列表中。', reopenListed: '从列表中重新打开',
@@ -75,14 +75,14 @@ export const workflowZh: typeof workflowEn = {
jobMeshRegion: '请检查网格单元大小和精细网格矩形,单元大小应为正整数,矩形应具有宽度和高度。',
jobFailed: '本地工具未能完成操作,之前的图层已保留。', dependenciesHelp: '请安装 uv 和 Python 3.10 或更高版本后重试。安装说明见 docs/agent-guide.md。',
unsafeFilesHelp: '项目中包含指向其他位置的文件。请改用此项目内的普通文件后重试。', showLog: '查看日志', operationBusy: '正在本地处理…', reloadFailed: '无法重新加载项目,当前视图仍保留。',
- variantsTitle: '表情差分', variantMissing: '暂无', variantsHelp: '为闭眼和说话绘制不同的表情图片,可让动作更自然。',
- eyesClosed: '闭眼', eyesHalf: '半闭眼', eyesSmile: '笑眼', mouthA: '嘴型「あ」', mouthAHalf: '半张嘴「あ」', mouthI: '嘴型「い」', mouthO: '嘴型「お」',
- targetEyes: '眼睛', targetMouth: '嘴型',
+ variantsTitle: '表情差分', variantMissing: '暂无', variantsHelp: '为闭眼、说话以及生气、悲伤、惊讶的表情绘制不同的图片,可让动作更自然。',
+ eyesClosed: '闭眼', eyesHalf: '半闭眼', eyesSmile: '笑眼', eyesAngry: '生气的眼睛', eyesSad: '悲伤的眼睛', eyesSurprise: '惊讶的眼睛', mouthA: '嘴型「あ」', mouthAHalf: '半张嘴「あ」', mouthI: '嘴型「い」', mouthO: '嘴型「お」',
+ targetEyes: '眼睛', targetExpression: '表情眼睛', targetMouth: '嘴型',
createRequests: '导出蒙版和提示词', requestsReady: '已在本地生成蒙版和提示词。', requestFiles: '蒙版和提示词', copyVariantPrompt: '复制生图提示词', copyFilePaths: '复制文件路径',
manualVariants: '自己准备差分图片', manualHelp: '将 source.png、蒙版和提示词交给图片编辑工具。按下方文件名保存与原图尺寸相同的 PNG,再拖入此处。仅可修改蒙版的透明区域,不透明区域必须与原图完全一致。',
variantOutput: '将返回的图片保存为', dropVariants: '将生成的 PNG 图片拖入此处', chooseVariants: '选择差分图片', variantFiles: '生成的差分图片',
- variantFilenameHelp: '使用 eyes_closed.png、eyes_half.png、eyes_smile.png、mouth_a.png、mouth_a_half.png、mouth_i.png 或 mouth_o.png。图片会保存在本地。',
- importAccepted: '导入成功,预览已更新。', importRejected: '导入未通过,之前的差分和图层已保留。', importOutside: '编辑蒙版以外的像素被修改了。', importSize: '请使用与 source.png 尺寸相同的 PNG 图片。', importMask: '该差分的编辑蒙版不符合要求。',
+ variantFilenameHelp: '使用 eyes_closed.png、eyes_half.png、eyes_smile.png、eyes_angry.png、eyes_sad.png、eyes_surprise.png、mouth_a.png、mouth_a_half.png、mouth_i.png 或 mouth_o.png。图片会保存在本地。',
+ importAccepted: '导入成功,预览已更新。', importMaskOnly: '蒙版外的像素与原图不同(重新压缩或轻微偏移),因此只导入了蒙版内的区域。', importRejected: '导入未通过,之前的差分和图层已保留。', importOutside: '编辑蒙版以外的像素被修改了。', importSize: '请使用与 source.png 尺寸相同的 PNG 图片。', importMask: '该差分的编辑蒙版不符合要求。',
importInvalid: '请使用列出的 PNG 文件名,每个名称最多一张;单张不超过 24 MB,总计不超过 26 MB。', metricsTitle: '蒙版外像素对比', outsideMax: '最大差异', outsideMean: '平均差异', outsidePixels: '超出容差的像素数',
variantsUnavailable: '若要在此制作请求或导入图片,请从本地开发服务的列表中打开一个可编辑项目。其他文件夹可让编程助手按照 docs/agent-guide.md 的步骤 6 处理。',
variantsNeedRebuild: '请先重建修改过的轮廓,让蒙版与当前图层一致。',
diff --git a/src/editor/project.ts b/src/editor/project.ts
index d9a0a9c..d147a72 100644
--- a/src/editor/project.ts
+++ b/src/editor/project.ts
@@ -101,7 +101,13 @@ export async function revealRepository() {
const response = await fetch('/__studio/reveal', { method: 'POST' });
if (!response.ok) throw new Error('Could not open repository.');
}
-export const variantNames = ['eyes_closed', 'eyes_half', 'eyes_smile', 'mouth_a', 'mouth_a_half', 'mouth_i', 'mouth_o'] as const;
+export const variantGroups = {
+ eyes: ['eyes_closed', 'eyes_half', 'eyes_smile'],
+ expression: ['eyes_angry', 'eyes_sad', 'eyes_surprise'],
+ mouth: ['mouth_a', 'mouth_a_half', 'mouth_i', 'mouth_o'],
+} as const;
+export type VariantGroup = keyof typeof variantGroups;
+export const variantNames = [...variantGroups.eyes, ...variantGroups.expression, ...variantGroups.mouth] as const;
export type VariantName = typeof variantNames[number];
export interface VariantRequest { name: VariantName; prompt: string; maskUrl: string }
export interface JobResult { path: string; log: string; requests?: VariantRequest[] }
@@ -112,7 +118,7 @@ export class ProjectJobError extends Error {
export async function runProjectJob(project: LocalProject, action: ProjectJob, rig: Rig, files: File[] = []): Promise<JobResult> {
let body: object = { rig };
if (action === 'import-variants') {
- if (!files.length || files.length > 7 || new Set(files.map(file => file.name)).size !== files.length || files.some(file => !variantNames.some(name => file.name === `${name}.png`) || file.size > 24 * 1024 * 1024) || files.reduce((sum, file) => sum + file.size, 0) > 26 * 1024 * 1024) throw new ProjectJobError('invalidImages');
+ if (!files.length || files.length > variantNames.length || new Set(files.map(file => file.name)).size !== files.length || files.some(file => !variantNames.some(name => file.name === `${name}.png`) || file.size > 24 * 1024 * 1024) || files.reduce((sum, file) => sum + file.size, 0) > 26 * 1024 * 1024) throw new ProjectJobError('invalidImages');
body = { files: await Promise.all(files.map(file => new Promise<{ name: string; data: string }>((done, reject) => {
const reader = new FileReader(); reader.onload = () => done({ name: file.name, data: String(reader.result).split(',')[1] }); reader.onerror = () => reject(new ProjectJobError('invalidImages')); reader.readAsDataURL(file);
}))) };
diff --git a/src/editor/workflow-i18n.ts b/src/editor/workflow-i18n.ts
index 795e8a4..c3ccc18 100644
--- a/src/editor/workflow-i18n.ts
+++ b/src/editor/workflow-i18n.ts
@@ -1,7 +1,7 @@
export const workflowEn = {
agentInstruction: 'Open AGENT in this repository folder and paste the following message.',
variantsReloaded: 'Drawn variants loaded.', mouthFallback: 'No drawn mouth images; using mesh deformation.', drawMouth: '→ Draw mouth variants',
- eyesKinds: 'Closed, half-closed, smiling', mouthKinds: 'あ, half-open あ, い, お', variantCount: 'COUNT of TOTAL images',
+ eyesKinds: 'Closed, half-closed, smiling', expressionKinds: 'Angry, sad, surprised (optional)', mouthKinds: 'あ, half-open あ, い, お', variantCount: 'COUNT of TOTAL images',
askAgent: 'Ask your agent',
copyPrompt: 'Copy message', copyDone: 'Copied', rootUnavailable: 'Open the folder containing this app and docs/agent-guide.md. Start the development server to copy its full path here.',
newIllustration: 'Create from a new illustration',
@@ -11,7 +11,7 @@ export const workflowEn = {
agentClaudePrompt: 'Prepare TARGET drawn variant requests for PROJECT following docs/agent-guide.md Step 6. Run variant-requests to prepare the masks and prompts. Claude Code cannot generate images: tell me exactly which source.png, masks and prompts to give to Codex or another image generator and which full-size PNG filenames to save in variants/, then stop. Do not generate or send images, or run build-sprites yet. Keep all files inside this repository.',
agentCopySample: 'First copy the read-only project at SOURCE into PROJECT; if the destination exists, use an unused name. Keep the original project unchanged. ',
agentUnknownPath: 'The browser cannot read this folder’s full path. Open it from the development server’s project list to prepare an exact request.',
- agentEyes: 'eye', agentMouth: 'mouth', agentBoth: 'eye and mouth',
+ agentEyes: 'eye', agentExpression: 'expression eye (angry, sad, surprised)', agentMouth: 'mouth',
rebuild: 'Save and rebuild layers', rebuilding: 'Saving and rebuilding layers…', staleLocal: 'Outlines changed. Rebuild layers to update the preview.',
staleSample: 'The sample is read-only. To continue editing, copy it into a project.', copyContinue: 'Copy and continue editing',
staleListed: 'This project is available in the project list.', reopenListed: 'Reopen from the list',
@@ -29,14 +29,14 @@ export const workflowEn = {
jobMeshRegion: 'Check the mesh cell sizes and fine rectangle. Use positive integer cell sizes and a rectangle with width and height.',
jobFailed: 'The local tool could not finish. Previous layers were kept.', dependenciesHelp: 'Install uv and Python 3.10 or later, then try again. With Homebrew: brew install uv python. Setup: docs/agent-guide.md.',
unsafeFilesHelp: 'The project contains redirected files. Use regular files inside this project and try again.', showLog: 'Show log', operationBusy: 'Working locally…', reloadFailed: 'Could not reload the project. The current view is still open.',
- variantsTitle: 'Drawn variants', variantMissing: 'Missing', variantsHelp: 'Drawn expressions make closed eyes and speaking look more natural.',
- eyesClosed: 'Closed eyes', eyesHalf: 'Half-closed eyes', eyesSmile: 'Smiling eyes', mouthA: 'Mouth あ', mouthAHalf: 'Half-open あ', mouthI: 'Mouth い', mouthO: 'Mouth お',
- targetEyes: 'Eyes', targetMouth: 'Mouth',
+ variantsTitle: 'Drawn variants', variantMissing: 'Missing', variantsHelp: 'Drawn expressions make closed eyes, speaking and angry, sad or surprised faces look more natural.',
+ eyesClosed: 'Closed eyes', eyesHalf: 'Half-closed eyes', eyesSmile: 'Smiling eyes', eyesAngry: 'Angry eyes', eyesSad: 'Sad eyes', eyesSurprise: 'Surprised eyes', mouthA: 'Mouth あ', mouthAHalf: 'Half-open あ', mouthI: 'Mouth い', mouthO: 'Mouth お',
+ targetEyes: 'Eyes', targetExpression: 'Expression eyes', targetMouth: 'Mouth',
createRequests: 'Export masks and prompts', requestsReady: 'Local masks and prompts are ready.', requestFiles: 'Masks and prompts', copyVariantPrompt: 'Copy image prompt', copyFilePaths: 'Copy file paths',
manualVariants: 'If you prepare images yourself', manualHelp: 'Give your image editor source.png, the mask and its prompt. Save a full-size PNG with the filename below, then drop it here. Transparent mask pixels may change; opaque pixels must stay identical.',
variantOutput: 'Save the returned drawing as', dropVariants: 'Drop returned PNG images here', chooseVariants: 'Choose returned images', variantFiles: 'Returned variant images',
- variantFilenameHelp: 'Use eyes_closed.png, eyes_half.png, eyes_smile.png, mouth_a.png, mouth_a_half.png, mouth_i.png or mouth_o.png. The app keeps the images local.',
- importAccepted: 'Accepted; preview updated.', importRejected: 'Rejected; previous drawings and layers were kept.', importOutside: 'Pixels outside the edit mask changed.', importSize: 'The returned image must be a PNG with the same dimensions as source.png.', importMask: 'The edit mask is not valid for this drawing.',
+ variantFilenameHelp: 'Use eyes_closed.png, eyes_half.png, eyes_smile.png, eyes_angry.png, eyes_sad.png, eyes_surprise.png, mouth_a.png, mouth_a_half.png, mouth_i.png or mouth_o.png. The app keeps the images local.',
+ importAccepted: 'Accepted; preview updated.', importMaskOnly: 'The image differed outside the mask (re-encoded or slightly shifted), so only the masked area was used.', importRejected: 'Rejected; previous drawings and layers were kept.', importOutside: 'Pixels outside the edit mask changed.', importSize: 'The returned image must be a PNG with the same dimensions as source.png.', importMask: 'The edit mask is not valid for this drawing.',
importInvalid: 'Use the listed PNG filenames (one of each), up to 24 MB per file and 26 MB total.', metricsTitle: 'Measured comparison outside the mask', outsideMax: 'Maximum difference', outsideMean: 'Mean difference', outsidePixels: 'Pixels over tolerance',
variantsUnavailable: 'To create requests or import images here, open a writable project from the local development server. An agent can use docs/agent-guide.md Step 6 for other folders.',
variantsNeedRebuild: 'Rebuild the changed outlines first, so the masks match the current layers.',
@@ -44,7 +44,7 @@ export const workflowEn = {
export const workflowJa: typeof workflowEn = {
agentInstruction: 'このリポジトリのフォルダで AGENT を開いて、次の文章を貼ってください。',
variantsReloaded: '描き分け画像を読み込みました', mouthFallback: '描き分け画像がないため、メッシュの変形で代用しています', drawMouth: '→ 口を描き分ける',
- eyesKinds: '閉じた目・半目・笑った目', mouthKinds: 'あ・半分のあ・い・お', variantCount: 'TOTAL枚中COUNT枚',
+ eyesKinds: '閉じた目・半目・笑った目', expressionKinds: '怒り・悲しみ・驚き(任意)', mouthKinds: 'あ・半分のあ・い・お', variantCount: 'TOTAL枚中COUNT枚',
askAgent: 'エージェントに依頼',
copyPrompt: '依頼文をコピー', copyDone: 'コピーしました', rootUnavailable: 'このアプリと docs/agent-guide.md があるフォルダで開いてください。開発サーバーを起動すると、ここでフルパスをコピーできます。',
newIllustration: '新しいイラストから作る',
@@ -54,7 +54,7 @@ export const workflowJa: typeof workflowEn = {
agentClaudePrompt: 'PROJECT の TARGETの描き分けについて、docs/agent-guide.md の Step 6 に沿って variant-requests でマスクと依頼文を用意してください。Claude Code は画像を生成できません。source.png と各マスク・依頼文のどのファイルを Codex などの画像生成に渡すか、結果を variants/ のどの PNG 名で保存するかを具体的に案内して、そこで止まってください。画像の生成・送信や build-sprites の実行はまだ行わないでください。ファイルはすべてこのリポジトリ内に保存してください。',
agentCopySample: '最初に読み取り専用の SOURCE を PROJECT にコピーしてください。作成先が存在する場合は未使用の名前を選び、元のプロジェクトは変更しないでください。',
agentUnknownPath: 'ブラウザで選んだフォルダの実際の場所は取得できません。正しい場所を含む依頼文を作るには、開発サーバーのプロジェクト一覧から開いてください。',
- agentEyes: '目', agentMouth: '口', agentBoth: '目と口',
+ agentEyes: '目', agentExpression: '表情の目(怒り・悲しみ・驚き)', agentMouth: '口',
rebuild: '保存してレイヤーを作り直す', rebuilding: '保存してレイヤーを作り直しています…', staleLocal: '輪郭が変更されました。レイヤーを作り直すとプレビューに反映されます。',
staleSample: 'サンプルは読み取り専用です。編集を続けるには、コピーしてプロジェクトにしてください。', copyContinue: 'コピーして編集を続ける',
staleListed: 'このプロジェクトは一覧から開けます', reopenListed: '一覧から開き直す',
@@ -72,14 +72,14 @@ export const workflowJa: typeof workflowEn = {
jobMeshRegion: '格子幅と細かい格子の範囲を確認してください。格子幅は正の整数にし、範囲に幅と高さを持たせてください。',
jobFailed: '作業を完了できませんでした。以前のレイヤーは残しています。', dependenciesHelp: 'uv と Python 3.10 以降を用意してから、もう一度お試しください。Homebrew の場合: brew install uv python。準備手順: docs/agent-guide.md。',
unsafeFilesHelp: 'プロジェクトに別の場所へ転送するファイルがあります。このプロジェクト内の通常のファイルにしてからお試しください。', showLog: 'ログを表示', operationBusy: 'この端末で処理中…', reloadFailed: 'プロジェクトを読み込み直せませんでした。現在の画面は残っています。',
- variantsTitle: '描き分け画像', variantMissing: 'なし', variantsHelp: '目を閉じたときや話すときの絵を描き分けると、動きが自然になります。',
- eyesClosed: '閉じた目', eyesHalf: '半分閉じた目', eyesSmile: '笑った目', mouthA: '口「あ」', mouthAHalf: '半分開いた「あ」', mouthI: '口「い」', mouthO: '口「お」',
- targetEyes: '目', targetMouth: '口',
+ variantsTitle: '描き分け画像', variantMissing: 'なし', variantsHelp: '目を閉じたとき・話すとき・怒りや悲しみや驚きの表情の絵を描き分けると、動きが自然になります。',
+ eyesClosed: '閉じた目', eyesHalf: '半分閉じた目', eyesSmile: '笑った目', eyesAngry: '怒った目', eyesSad: '悲しい目', eyesSurprise: '驚いた目', mouthA: '口「あ」', mouthAHalf: '半分開いた「あ」', mouthI: '口「い」', mouthO: '口「お」',
+ targetEyes: '目', targetExpression: '表情の目', targetMouth: '口',
createRequests: 'マスクと依頼文を書き出す', requestsReady: 'マスクと依頼文をこの端末に作りました。', requestFiles: 'マスクと依頼文', copyVariantPrompt: '画像の依頼文をコピー', copyFilePaths: 'ファイルの場所をコピー',
manualVariants: '自分で画像を用意する場合', manualHelp: '元画像 source.png・マスク・依頼文を画像編集ツールに渡してください。同じ大きさの PNG を下記の名前で保存し、この欄にドロップします。マスクの透明部分だけ変更でき、不透明部分は元画像と同じにします。',
variantOutput: '戻ってきた画像の保存先', dropVariants: '戻ってきた PNG 画像をここにドロップ', chooseVariants: '戻ってきた画像を選ぶ', variantFiles: '戻ってきた描き分け画像',
- variantFilenameHelp: 'eyes_closed.png・eyes_half.png・eyes_smile.png・mouth_a.png・mouth_a_half.png・mouth_i.png・mouth_o.png の名前を使います。画像はこの端末内に保存します。',
- importAccepted: '合格しました。プレビューに反映しました。', importRejected: '不合格でした。以前の画像とレイヤーは残しています。', importOutside: 'マスクの外側の画素が変更されています。', importSize: '元画像 source.png と同じ大きさの PNG にしてください。', importMask: 'この描き分けに使うマスクの形が正しくありません。',
+ variantFilenameHelp: 'eyes_closed.png・eyes_half.png・eyes_smile.png・eyes_angry.png・eyes_sad.png・eyes_surprise.png・mouth_a.png・mouth_a_half.png・mouth_i.png・mouth_o.png の名前を使います。画像はこの端末内に保存します。',
+ importAccepted: '合格しました。プレビューに反映しました。', importMaskOnly: 'マスクの外側が元画像と違っていた(再圧縮や小さなずれ)ため、マスクの中だけを取り込みました。', importRejected: '不合格でした。以前の画像とレイヤーは残しています。', importOutside: 'マスクの外側の画素が変更されています。', importSize: '元画像 source.png と同じ大きさの PNG にしてください。', importMask: 'この描き分けに使うマスクの形が正しくありません。',
importInvalid: '指定された PNG 名を使ってください(同じ名前は1枚まで)。1枚24 MB・合計26 MBまで取り込めます。', metricsTitle: 'マスクの外側の比較結果', outsideMax: '差の最大値', outsideMean: '差の平均値', outsidePixels: '許容差を超えた画素数',
variantsUnavailable: 'ここで依頼ファイルを作ったり画像を取り込んだりするには、開発サーバーの一覧から編集できるプロジェクトを開いてください。ほかのフォルダは、エージェントに docs/agent-guide.md の Step 6 を依頼できます。',
variantsNeedRebuild: '先に輪郭の変更を反映して、マスクを現在のレイヤーに合わせてください。',
diff --git a/src/server/project-jobs.ts b/src/server/project-jobs.ts
index 4b5efa2..5011841 100644
--- a/src/server/project-jobs.ts
+++ b/src/server/project-jobs.ts
@@ -3,7 +3,7 @@ import { execFile } from 'node:child_process';
import { copyFile, lstat, mkdir, mkdtemp, readdir, readFile, realpath, rename, rm, writeFile } from 'node:fs/promises';
import { join, resolve, relative, dirname } from 'node:path';
-export const VARIANTS = ['eyes_closed', 'eyes_half', 'eyes_smile', 'mouth_a', 'mouth_a_half', 'mouth_i', 'mouth_o'] as const;
+export const VARIANTS = ['eyes_closed', 'eyes_half', 'eyes_smile', 'eyes_angry', 'eyes_sad', 'eyes_surprise', 'mouth_a', 'mouth_a_half', 'mouth_i', 'mouth_o'] as const;
export type Tool = 'build-layers' | 'variant-requests' | 'build-sprites';
export class JobError extends Error {
constructor(public code: 'dependencies' | 'toolFailed' | 'unsafeFiles', public log: string) { super(code); }
@@ -59,10 +59,10 @@ async function publish(project: string, stage: string, names: string[]) {
export interface VariantInput { name: string; data: string }
export function decodeVariants(value: unknown): { name: string; bytes: Buffer }[] {
const files = (value as { files?: VariantInput[] })?.files;
- if (!Array.isArray(files) || !files.length || files.length > VARIANTS.length) throw new Error('Choose one to seven named PNG files.');
+ if (!Array.isArray(files) || !files.length || files.length > VARIANTS.length) throw new Error(`Choose one to ${VARIANTS.length} named PNG files.`);
const names = new Set<string>();
return files.map(file => {
- if (!file || !VARIANTS.some(name => file.name === `${name}.png`) || names.has(file.name) || typeof file.data !== 'string' || !/^[A-Za-z0-9+/]+={0,2}$/.test(file.data)) throw new Error('Use the seven variant filenames and PNG images.');
+ if (!file || !VARIANTS.some(name => file.name === `${name}.png`) || names.has(file.name) || typeof file.data !== 'string' || !/^[A-Za-z0-9+/]+={0,2}$/.test(file.data)) throw new Error('Use the variant filenames and PNG images.');
names.add(file.name);
const bytes = Buffer.from(file.data, 'base64');
if (bytes.length > 24 * 1024 * 1024 || !bytes.subarray(0, 8).equals(Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]))) throw new Error('Use PNG images smaller than 24 MB each.');
diff --git a/tests/expression-eyes.test.ts b/tests/expression-eyes.test.ts
new file mode 100644
index 0000000..d16599a
--- /dev/null
+++ b/tests/expression-eyes.test.ts
@@ -0,0 +1,64 @@
+import { expect, test } from 'vitest';
+import fixture from '../samples/miko-qipao/rig.json';
+import { parseRig, PARAMS } from 'mesh-avatar';
+import { createRig } from '../packages/runtime/src/engine/rig.js';
+import { createSpriteModule, eyeMood } from '../packages/runtime/src/engine/sprites.js';
+
+const rig = parseRig(fixture);
+const engine = createRig(rig);
+const defaults = Object.fromEntries(PARAMS.map(p => [p.id, p.def]));
+const physics = { strands: [], bunL: [0, 0], bunR: [0, 0] };
+
+// Minimal renderer: records each sprite layer so the visible drawings can be read back.
+function sprites(names: string[]) {
+ const layers = new Map<string, { visible: boolean; alpha: number }>();
+ const renderer = { addLayer(name: string) { const layer = { visible: false, alpha: 1 }; layers.set(name, layer); return layer; } };
+ const sheet = { build: 'test', layers: Object.fromEntries(names.map(name => [name, [500, 400, 10, 10]])) };
+ const images = Object.fromEntries(names.map(name => [name, { width: 10 }]));
+ const grid = (rect: number[]) => ({ rest: [rect[0], rect[1]], pos: [0, 0] });
+ type Create = (...args: unknown[]) => { update(P: Record<string, number>, phys: unknown, dt: number, mood?: string | null): { eyes: boolean[] } };
+ const update = (createSpriteModule(engine, rig).createSprites as unknown as Create)(renderer, sheet, images, grid, () => new Uint8Array(1));
+ return (open: number, mood: string | null = null, dt = 1) => {
+ const result = update.update({ ...defaults, eyeROpen: open, eyeLOpen: open, eyeSmile: 0, eyeSmileL: 0 }, physics, dt, mood);
+ return { covered: result.eyes, shown: [...layers].filter(([, layer]) => layer.visible && layer.alpha > 0).map(([name]) => name).sort() };
+ };
+}
+
+const eyes = (...kinds: string[]) => kinds.flatMap(kind => [`eyes_${kind}_0`, `eyes_${kind}_1`]);
+
+test('expression names map to drawn-eye moods, and the live expression wins over the motion one', () => {
+ expect(eyeMood('angry', null)).toBe('angry');
+ expect(eyeMood('neutral', 'sadTalk')).toBe('sad');
+ expect(eyeMood('neutral', 'surprisedTalk')).toBe('surprise');
+ expect(eyeMood('neutral', 'surprise')).toBe('surprise');
+ expect(eyeMood('smile', 'angryTalk')).toBeNull();
+ expect(eyeMood('neutral', 'happyTalk')).toBeNull();
+ expect(eyeMood('neutral', null)).toBeNull();
+});
+
+test('drawn expression eyes replace open and half eyes, and blinks still close them', () => {
+ const frame = sprites(eyes('closed', 'half', 'smile', 'angry', 'sad', 'surprise'));
+ expect(frame(0.72, 'angry')).toEqual({ covered: [true, true], shown: eyes('angry') });
+ expect(frame(1.22, 'surprise')).toEqual({ covered: [true, true], shown: eyes('surprise') });
+ expect(frame(0.1, 'surprise')).toEqual({ covered: [true, true], shown: eyes('closed') });
+ expect(frame(0.78, 'sad').shown).toEqual(eyes('sad'));
+ // Back to neutral: the plain open eye returns (no sprite drawn).
+ expect(frame(1, null)).toEqual({ covered: [false, false], shown: [] });
+});
+
+test('the blink fade keeps one drawing solid underneath, so no eye turns see-through', () => {
+ const frame = sprites(eyes('closed', 'half', 'angry'));
+ frame(0.72, 'angry');
+ // Mid-fade both drawings are shown and the layered eye stays hidden underneath.
+ expect(frame(0.1, 'angry', 0.02)).toEqual({ covered: [false, false], shown: eyes('angry', 'closed').sort() });
+ frame(0.1, 'angry', 1);
+ expect(frame(0.72, 'angry', 0.02).shown).toEqual(eyes('angry', 'closed').sort());
+ expect(frame(0.72, 'angry', 1).shown).toEqual(eyes('angry'));
+});
+
+test('avatars without drawn expression eyes keep the previous drawings', () => {
+ const frame = sprites(eyes('closed', 'half', 'smile'));
+ expect(frame(0.72, 'angry').shown).toEqual(eyes('half'));
+ expect(frame(1.22, 'surprise')).toEqual({ covered: [false, false], shown: [] });
+ expect(frame(0.1, 'sad').shown).toEqual(eyes('closed'));
+});
diff --git a/tools/agent_common.py b/tools/agent_common.py
index c08ff44..f075190 100644
--- a/tools/agent_common.py
+++ b/tools/agent_common.py
@@ -109,6 +109,9 @@ def grid_image(image, region=None, step=50):
EYE_VARIANTS = ("eyes_closed", "eyes_half", "eyes_smile")
+# Open eyes drawn for an expression; the brows stay outside the mask (the runtime moves them).
+EXPRESSION_EYE_VARIANTS = ("eyes_angry", "eyes_sad", "eyes_surprise")
+ALL_EYE_VARIANTS = (*EYE_VARIANTS, *EXPRESSION_EYE_VARIANTS)
MOUTH_VARIANTS = ("mouth_a", "mouth_a_half", "mouth_i", "mouth_o")
diff --git a/tools/build-sprites.py b/tools/build-sprites.py
index 3de139b..94c1787 100644
--- a/tools/build-sprites.py
+++ b/tools/build-sprites.py
@@ -14,7 +14,7 @@ from pathlib import Path
import cv2
import numpy as np
from agent_common import (
- EYE_VARIANTS,
+ ALL_EYE_VARIANTS,
MOUTH_VARIANTS,
edit_masks,
inside_repo,
@@ -41,11 +41,11 @@ def regions(project, rig, size, name):
legacy = (
project
/ "variant-masks"
- / ("eyes_edit_mask.png" if name in EYE_VARIANTS else "mouth_edit_mask.png")
+ / ("eyes_edit_mask.png" if name in ALL_EYE_VARIANTS else "mouth_edit_mask.png")
)
if mask_path.exists() or legacy.exists():
region = load_mask(mask_path if mask_path.exists() else legacy, size)
- if name in EYE_VARIANTS:
+ if name in ALL_EYE_VARIANTS:
count, labels = cv2.connectedComponents(region)
if count != 3:
raise ValueError(
@@ -57,7 +57,7 @@ def regions(project, rig, size, name):
return [(labels == i).astype(np.uint8) for i in components]
return [region]
eyes, mouth = edit_masks(rig, size)
- return eyes if name in EYE_VARIANTS else [mouth]
+ return eyes if name in ALL_EYE_VARIANTS else [mouth]
def visible(image):
@@ -66,6 +66,104 @@ def visible(image):
return value
+# Image generators often return the whole picture re-encoded or nudged by a pixel, with the
+# transparent background filled in. Sprites only use pixels inside the mask, so such a drawing
+# can still be imported from the mask alone, as long as it is the same picture: limits for the
+# whole-image shift, the opaque-pixel difference, and the band just outside the mask.
+MAX_SHIFT = 3
+MAX_OPAQUE_MEAN = 16
+RING = 12
+MAX_RING_MEAN = 20
+MAX_RING_SHARE = 0.12
+RING_LEVEL = 48
+
+
+def grey(image):
+ """Edges of the picture over mid grey, each image with its own alpha."""
+ value = visible(image)
+ value = value[..., :3].mean(axis=-1) + 128 * (1 - value[..., 3] / 255)
+ return cv2.magnitude(cv2.Sobel(value, cv2.CV_32F, 1, 0), cv2.Sobel(value, cv2.CV_32F, 0, 1))
+
+
+def mask_only(name, variant, source, region):
+ """Return the variant aligned to the source, or raise with the reason it differs."""
+ opaque = source[..., 3] == 255
+ (dx, dy), response = cv2.phaseCorrelate(
+ grey(source), grey(variant), cv2.createHanningWindow(source.shape[1::-1], cv2.CV_32F)
+ )
+ if response < 0.05:
+ dx = dy = 0.0 # no clear match (e.g. a flat image): judge the differences unshifted
+ if max(abs(dx), abs(dy)) > MAX_SHIFT:
+ raise ValueError(
+ f"{name}: rejected outside-mask changes: the whole image is shifted by "
+ f"{dx:+.1f}, {dy:+.1f} px (limit {MAX_SHIFT} px); return it unshifted"
+ )
+ def difference(image):
+ return np.abs(image[..., :3].astype(np.float32) - source[..., :3]).max(axis=-1)
+
+ rest = opaque & (region == 0)
+ delta = difference(variant)
+ sx, sy = round(dx), round(dy)
+ if sx or sy:
+ # Keep the correction only when it actually brings the picture closer.
+ moved = cv2.warpAffine(
+ variant,
+ np.float32([[1, 0, -sx], [0, 1, -sy]]),
+ source.shape[1::-1],
+ flags=cv2.INTER_NEAREST,
+ borderMode=cv2.BORDER_REPLICATE,
+ )
+ moved_delta = difference(moved)
+ if moved_delta[rest].mean() < delta[rest].mean():
+ variant, delta = moved, moved_delta
+ else:
+ sx = sy = 0
+ outside = cv2.distanceTransform((region == 0).astype(np.uint8), cv2.DIST_L2, 5)
+ ring = rest & (outside <= RING)
+ if not np.any(ring):
+ raise ValueError(f"{name}: rejected outside-mask changes: no opaque pixels border the mask")
+ opaque_mean = float(delta[rest].mean())
+ ring_mean = float(delta[ring].mean())
+ ring_share = float((delta[ring] > RING_LEVEL).mean())
+ print(
+ f"{name}: mask-only check: shift {dx:+.2f}, {dy:+.2f} px, opaque mean {opaque_mean:.2f}/255, "
+ f"edge band mean {ring_mean:.2f}/255, {ring_share:.1%} of edge band over {RING_LEVEL}"
+ )
+ if opaque_mean > MAX_OPAQUE_MEAN:
+ raise ValueError(
+ f"{name}: rejected outside-mask changes: the picture outside the mask was redrawn "
+ f"(opaque mean {opaque_mean:.1f}/255, limit {MAX_OPAQUE_MEAN})"
+ )
+ if ring_mean > MAX_RING_MEAN or ring_share > MAX_RING_SHARE:
+ raise ValueError(
+ f"{name}: rejected outside-mask changes: the drawing spills over the mask edge "
+ f"(within {RING} px: mean {ring_mean:.1f}/255, {ring_share:.0%} over {RING_LEVEL}; "
+ f"limits {MAX_RING_MEAN} and {MAX_RING_SHARE:.0%})"
+ )
+ # Generators flatten transparency; the masked face keeps the source alpha.
+ variant = variant.copy()
+ variant[..., 3] = source[..., 3]
+ print(f"{name}: importing the mask area only (corrected shift {sx:+d}, {sy:+d} px)")
+ return variant
+
+
+def read_variant(name, path, size):
+ with Image.open(path) as image:
+ if image.format != "PNG":
+ raise ValueError(f"{name}: expected PNG size {size[0]} x {size[1]}, got {image.format}")
+ resized = image.size != size
+ if resized:
+ # A uniformly scaled result is the same picture; a different aspect ratio is not.
+ if abs(image.width * size[1] / (image.height * size[0]) - 1) > 0.005:
+ raise ValueError(
+ f"{name}: expected PNG size {size[0]} x {size[1]}, got {image.size} "
+ "(different aspect ratio)"
+ )
+ print(f"{name}: resizing {image.width} x {image.height} to {size[0]} x {size[1]}")
+ image = image.convert("RGBA").resize(size, Image.Resampling.LANCZOS)
+ return np.array(image.convert("RGBA")), resized
+
+
def build(project, output=None, tolerance=8):
project = inside_repo(project)
if not np.isfinite(tolerance) or not 0 <= tolerance <= 255:
@@ -75,17 +173,12 @@ def build(project, output=None, tolerance=8):
if rig["image"] != {"width": source_image.width, "height": source_image.height}:
raise ValueError("rig.image must match source.png")
images, meta = {}, {}
- for name in (*EYE_VARIANTS, *MOUTH_VARIANTS):
+ for name in (*ALL_EYE_VARIANTS, *MOUTH_VARIANTS):
path = project / "variants" / f"{name}.png"
if not path.exists():
print(f"Skipping missing variant: {name}")
continue
- with Image.open(path) as image:
- if image.format != "PNG" or image.size != source_image.size:
- raise ValueError(
- f"{name}: expected PNG size {source_image.width} x {source_image.height}, got {image.size}"
- )
- variant = np.array(image.convert("RGBA"))
+ variant, resized = read_variant(name, path, source_image.size)
parts = regions(project, rig, source_image.size, name)
region = np.maximum.reduce(parts)
if not np.any(region) or np.all(region):
@@ -98,10 +191,8 @@ def build(project, output=None, tolerance=8):
print(
f"{name}: outside-mask max {maximum:.3f}/255, mean {mean:.6f}/255, {changed} pixels over tolerance {tolerance:g}"
)
- if changed:
- raise ValueError(
- f"{name}: rejected outside-mask changes; preserve source pixels outside the edit region"
- )
+ if changed or resized:
+ variant = mask_only(name, variant, source, region)
for i, part in enumerate(parts):
weight = np.clip(cv2.distanceTransform(part, cv2.DIST_L2, 5) / 4, 0, 1)
ys, xs = np.where(weight > 0)
@@ -110,7 +201,7 @@ def build(project, output=None, tolerance=8):
x0, y0, x1, y1 = xs.min(), ys.min(), xs.max() + 1, ys.max() + 1
crop = variant[y0:y1, x0:x1].copy()
crop[..., 3] = (crop[..., 3] * weight[y0:y1, x0:x1]).astype(np.uint8)
- layer = f"{name}_{i}" if name in EYE_VARIANTS else name
+ layer = f"{name}_{i}" if name in ALL_EYE_VARIANTS else name
images[layer] = crop
meta[layer] = [int(x0), int(y0), int(x1 - x0), int(y1 - y0)]
if not images:
diff --git a/tools/render-poses.mjs b/tools/render-poses.mjs
index 12986a0..ceed620 100644
--- a/tools/render-poses.mjs
+++ b/tools/render-poses.mjs
@@ -13,6 +13,8 @@ const poses = [
['look-up', { angleY: 15 }], ['look-down', { angleY: -15 }],
['tilt-left', { angleZ: -15 }], ['tilt-right', { angleZ: 15 }],
['body-left', { bodyAngleZ: -8 }], ['body-right', { bodyAngleZ: 8 }],
+ // live expressions; they show the drawn expression eyes when the avatar has them
+ ['angry', {}, 'angry'], ['sad', {}, 'sad'], ['surprise', {}, 'surprise'],
];
async function main() {
@@ -81,9 +83,11 @@ async function main() {
av.setAutoIdle(false); av.setAutoMotion(false); av.setParameters(defaults);
return av;
}
- for (const [name, values] of poses) {
+ for (const [name, values, expression] of poses) {
const av = await avatar();
- av.setParameters({ ...defaults, ...values }); av.advance(1, 60);
+ av.setParameters({ ...defaults, ...values });
+ if (expression) av.setExpression(expression);
+ av.advance(1, 60);
pictures.push([name, canvas.toDataURL()]); av.destroy();
}
const av = await avatar();
@@ -135,7 +139,7 @@ async function main() {
if (relative(project, await realpath(review)).startsWith('..')) throw new Error('review folder must stay inside the project');
for (const [name, url] of result.images) await writeFile(resolve(review, `${name}.png`), Buffer.from(url.split(',')[1], 'base64'));
const { images, ...metadata } = result;
- await writeFile(resolve(review, 'poses.json'), JSON.stringify({ ...metadata, project: basename(project), fixedFps: 60, poseSeconds: 1, poses: poses.map(([name, parameters]) => ({ name, parameters })), hairFrames: 6, files: images.map(([name]) => `${name}.png`) }, null, 2) + '\n');
+ await writeFile(resolve(review, 'poses.json'), JSON.stringify({ ...metadata, project: basename(project), fixedFps: 60, poseSeconds: 1, poses: poses.map(([name, parameters, expression]) => ({ name, parameters, ...(expression && { expression }) })), hairFrames: 6, files: images.map(([name]) => `${name}.png`) }, null, 2) + '\n');
console.log(`Rendered ${poses.length + 6} poses, eye/mouth crops and contact-sheet.png in ${relative(root, review)}/`);
} finally {
await browser?.close();
diff --git a/tools/test_agent_tools.py b/tools/test_agent_tools.py
index dc27ec8..44c35e5 100644
--- a/tools/test_agent_tools.py
+++ b/tools/test_agent_tools.py
@@ -126,7 +126,15 @@ class AgentToolsTest(unittest.TestCase):
result = self.cli("variant-requests", self.project)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(
- len(list((self.project / "variant-requests").glob("*/mask.png"))), 7
+ len(list((self.project / "variant-requests").glob("*/mask.png"))), 10
+ )
+ self.assertEqual(
+ (self.project / "variant-requests/eyes_angry/mask.png").read_bytes(),
+ (self.project / "variant-requests/eyes_closed/mask.png").read_bytes(),
+ )
+ self.assertIn(
+ "eyebrows unchanged",
+ (self.project / "variant-requests/eyes_sad/prompt.md").read_text(),
)
self.assertEqual(self.cli("build-sprites", self.project).returncode, 0)
self.assertFalse((self.project / "built").exists())
@@ -148,10 +156,12 @@ class AgentToolsTest(unittest.TestCase):
self.assertEqual(json.loads(before)["version"], 1)
self.assertEqual(list(json.loads(before)), ["build", "layers", "version"])
self.assertEqual(json.loads(before)["layers"]["mouth_a"][2] > 0, True)
- invalid = valid.copy()
- invalid.putpixel((0, 0), (0, 0, 0, 255))
- invalid.save(path)
- with self.assertRaisesRegex(ValueError, "outside-mask"):
+ spill = valid.copy()
+ for x in range(30, 70):
+ for y in range(52, 78):
+ spill.putpixel((x, y), (0, 0, 0, 255))
+ spill.save(path)
+ with self.assertRaisesRegex(ValueError, "outside-mask changes: the drawing spills"):
sprites.build(self.project)
self.assertEqual(metadata.read_bytes(), before)
Image.new("RGBA", (99, 100)).save(path)
@@ -159,6 +169,50 @@ class AgentToolsTest(unittest.TestCase):
sprites.build(self.project)
self.assertEqual(metadata.read_bytes(), before)
+ def test_reencoded_or_shifted_drawings_import_only_the_mask_area(self):
+ import numpy as np
+
+ rng = np.random.default_rng(1)
+ texture = rng.integers(60, 200, (100, 100, 3), dtype=np.uint8)
+ source = np.dstack([texture, np.full((100, 100), 255, np.uint8)])
+ source[:, :6, 3] = 0
+ Image.fromarray(source).save(self.project / "source.png")
+ variants = self.project / "variants"
+ variants.mkdir()
+ path = variants / "eyes_angry.png"
+
+ def save(image):
+ Image.fromarray(image).save(path)
+
+ drawn = source.copy()
+ drawn[25:36, 15:26, :3] = 20
+ # Re-encoded: small noise everywhere, the transparent edge filled in.
+ noisy = drawn.astype(int) + rng.integers(-12, 13, drawn.shape)
+ noisy[..., 3] = 255
+ save(np.clip(noisy, 0, 255).astype(np.uint8))
+ sprites.build(self.project)
+ layers = json.loads((self.project / "built/sprites/sprites.json").read_text())["layers"]
+ self.assertEqual(sorted(layers), ["eyes_angry_0", "eyes_angry_1"])
+ with Image.open(self.project / "built/sprites/eyes_angry_0.png") as image:
+ x, y = layers["eyes_angry_0"][:2]
+ self.assertLess(image.getpixel((20 - x, 30 - y))[0], 40)
+ # A one-pixel shift is corrected; a uniformly scaled image is resized.
+ save(np.roll(drawn, 1, axis=1))
+ sprites.build(self.project)
+ Image.fromarray(drawn).resize((200, 200), Image.Resampling.NEAREST).save(path)
+ sprites.build(self.project)
+ save(np.roll(drawn, 6, axis=1))
+ with self.assertRaisesRegex(ValueError, "shifted by"):
+ sprites.build(self.project)
+ redrawn = drawn.copy()
+ redrawn[..., :3] = 255 - redrawn[..., :3]
+ save(redrawn)
+ with self.assertRaisesRegex(ValueError, "outside-mask changes"):
+ sprites.build(self.project)
+ Image.fromarray(drawn).resize((100, 120)).save(path)
+ with self.assertRaisesRegex(ValueError, "aspect ratio"):
+ sprites.build(self.project)
+
def module_cleanup(path):
import shutil
diff --git a/tools/variant-requests.py b/tools/variant-requests.py
index fa0773e..f27b4a0 100644
--- a/tools/variant-requests.py
+++ b/tools/variant-requests.py
@@ -9,7 +9,7 @@ import sys
from pathlib import Path
from agent_common import (
- EYE_VARIANTS,
+ ALL_EYE_VARIANTS,
MOUTH_VARIANTS,
edit_masks,
inside_repo,
@@ -23,6 +23,9 @@ CHANGES = {
"eyes_closed": "Close both eyes naturally, using one clean upper lash line per eye. Remove all visible iris and eye white. Keep the eyebrows unchanged.",
"eyes_half": "Draw both eyes half closed, with the upper lids halfway down. Keep the iris visible only below the upper lid and preserve the original gaze.",
"eyes_smile": "Draw both eyes closed in a gentle smiling upward arc, with clean lashes and no visible iris or eye white. Keep the eyebrows unchanged.",
+ "eyes_angry": "Redraw both eyes as angry open eyes: the upper lids lowered into a narrowed glare that slants down toward the nose, with the upper part of each iris hidden under the lid and the gaze straight ahead. Keep each eye in place and the eyebrows unchanged; they are moved separately.",
+ "eyes_sad": "Redraw both eyes as sad open eyes: the upper lids drooping at the outer corners and raised toward the nose, softer and slightly moist, with the gaze a little downward. Do not draw tears on the cheeks. Keep each eye in place and the eyebrows unchanged; they are moved separately.",
+ "eyes_surprise": "Redraw both eyes as surprised wide-open eyes: the upper lids raised to show eye white above the iris, slightly smaller irises with clear highlights, and the gaze straight ahead. Keep each eye in place and the eyebrows unchanged; they are moved separately.",
"mouth_a": "Open the mouth in the Japanese vowel あ (a): a natural vertically open mouth, with a subtle tongue and upper teeth.",
"mouth_a_half": "Draw the Japanese vowel あ (a) at half openness: a smaller, gently open mouth.",
"mouth_i": "Draw the Japanese vowel い (i): a wide, narrow open smile with subtle upper teeth.",
@@ -38,11 +41,11 @@ def generate(project):
eyes, mouth = edit_masks(rig, image.size)
output = project / "variant-requests"
output.mkdir(exist_ok=True)
- names = (*EYE_VARIANTS, *MOUTH_VARIANTS)
+ names = (*ALL_EYE_VARIANTS, *MOUTH_VARIANTS)
for name in names:
folder = output / name
save_image(
- rgba_mask((eyes[0] | eyes[1]) if name in EYE_VARIANTS else mouth),
+ rgba_mask((eyes[0] | eyes[1]) if name in ALL_EYE_VARIANTS else mouth),
folder / "mask.png",
)
prompt = (
@@ -62,7 +65,8 @@ def generate(project):
"# Drawn variant requests\n\n"
f"Required output size: **{image.width} x {image.height}** pixels, RGBA PNG.\n\n"
"For each variant below, supply `../source.png`, the variant's `mask.png`, and its "
- "`prompt.md` to an image editor you trust. Transparent mask pixels may change; opaque "
+ "`prompt.md` to an image editor you trust. The expression eyes (`eyes_angry`, `eyes_sad`, "
+ "`eyes_surprise`) are optional; generate only the ones you want. Transparent mask pixels may change; opaque "
"mask pixels must stay identical. Generating these files is a separate action: this "
"command uploads nothing. If the image must stay local, use a local editor.\n\n"
+ "\n".join(