Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

fix(editor): a write the layout will overwrite says so, in its own reply

yhh28b23de6ed40
diff --git a/desktop/electron/agent/prompt.ts b/desktop/electron/agent/prompt.ts
index 15ec8beb9..0fe020df6 100644
--- a/desktop/electron/agent/prompt.ts
+++ b/desktop/electron/agent/prompt.ts
@@ -33,6 +33,7 @@ Working style:
- You can see the result: capture_viewport renders what the user is looking at. Use it when the question is visual — layout, whether something is on screen, why the scene looks wrong.
- Prefer one apply_scene_ops over many small calls when building something: it is one undo step for the user and one round trip for you, and it is the ONE write the user sees BEFORE it lands — they get a line-by-line preview and can strike parts of it out. A dozen set_field calls give them no such chance, so build with a batch and save the single writes for genuinely single changes.
- Units are the editor's: 1 world unit = 1 design pixel. Y is up in the world, and UI is laid out top-down.
+- Game content lives in the WORLD: a board, its pieces, a character are Sprite / ShapeRenderer entities you place with Transform.position. A Canvas and its UINodes are for HUD and menus, and their placement is the layout's OUTPUT — a Transform.position written on a UI node is overwritten at the next relayout, which looks like nothing happened. Move those with layout inputs instead (UINode.position = Absolute, then left/top), or let the flow place them.
- Say what you did in a sentence or two. The user watched it happen — a summary of every call is noise. Speak the user's language: answer in the language they wrote to you in.`;
/**
diff --git a/desktop/shared/toolCatalog.mjs b/desktop/shared/toolCatalog.mjs
index ee0662a6a..21526f298 100644
--- a/desktop/shared/toolCatalog.mjs
+++ b/desktop/shared/toolCatalog.mjs
@@ -77,7 +77,9 @@ const CAPTURE_PNG_JS = `(() => {
/** name → { description, schema (JSON Schema), method (surface), args(input)→[] }. */
export const TOOLS = [
{ name: 'load_scene',
- description: 'Load a scene (and optional asset manifest) into the headless World; returns the spawned entity count. Call first to set up a known scene.',
+ description: 'FIXTURES ONLY: fetch a scene by URL into the headless World; returns the spawned entity count. '
+ + 'With a project open this is the wrong door — a project path is not a URL and the fetch 404s: use open_scene, '
+ + 'which the project already knows how to resolve.',
schema: obj({ sceneUrl: { type: 'string' }, manifestUrl: { type: 'string' } }, ['sceneUrl']),
method: 'loadScene', args: (i) => [i.sceneUrl, i.manifestUrl] },
{ name: 'get_scene_tree',
@@ -298,9 +300,10 @@ export const TOOLS = [
+ 'or give `components` as ["Transform","UINode","UIVisual"] / [{type,data}]; '
+ '{op:"set", entity, fields}; {op:"add_component"|"remove_component", entity, component}; {op:"rename", entity, name}; {op:"parent", entity, parent}; {op:"delete", entity}. '
+ 'ENTITY ADDRESSING: an entity is a live numeric id, or "$name" naming an earlier create\'s `ref` — so a parent/child tree is one call with no round trip to learn ids. '
+ + 'A `$ref` lives for THIS call only; the returned `refs` map is how its ids reach the next one. '
+ 'FIELDS is a flat map of "Component.key" → value, e.g. {"Transform.position.x": 10, "UIVisual.color": "#ff0000ff", "Text.content": "Hi"} — same coercion and validation as set_field, '
+ 'including one member of a structural field ("FlexContainer.gap.x", "UINode.marginLeft"). '
- + 'Give either `ops` inline or `opsPath` (a JSON file), never both. Returns { refs: {name: id}, created: [id], applied: n }. Check get_diagnostics afterwards.',
+ + 'Give either `ops` inline or `opsPath` (a JSON file), never both. Returns { refs: {name: id}, created: [id], applied: n, warnings? } — `warnings` names writes that were accepted and will NOT survive (a field the layout owns), so read it. Check get_diagnostics afterwards.',
schema: obj({
ops: { type: 'array', description: 'the op program, executed in order as one undo step' },
opsPath: { type: 'string', description: 'project-relative path to a JSON file holding the op array, INSTEAD of `ops` — a panel of a few hundred entities is a few hundred KB of program, which does not belong in a message. Write it with write_project_file, then name it here.' },
diff --git a/desktop/src/engine/sceneOps.ts b/desktop/src/engine/sceneOps.ts
index 57b0ebd71..4afbb880c 100644
--- a/desktop/src/engine/sceneOps.ts
+++ b/desktop/src/engine/sceneOps.ts
@@ -23,6 +23,7 @@
*/
import type { EntityId } from '@/types';
import type { PrefabData } from 'esengine';
+import { SceneModel } from './SceneModel';
import { EditorControlSurface } from './EditorSession';
import { prefabFromSpecs, sourceById } from './entitySources';
@@ -62,6 +63,37 @@ export interface SceneOpsResult {
created: EntityId[];
/** How many ops ran. */
applied: number;
+ /**
+ * Writes that were accepted and will not survive — a field something else
+ * owns. Present only when there are any.
+ *
+ * A write that silently does nothing is worse than one that fails: an agent
+ * built a chess board out of forty UI nodes, set `Transform.position` on each,
+ * and got an empty viewport with no error anywhere. The inspector already says
+ * this on the entity card and the diagnostics sweep repeats it, but both are
+ * places you have to go LOOK — the answer belongs in the reply to the call
+ * that made the mistake.
+ */
+ warnings?: string[];
+}
+
+/**
+ * Why a field write will not stick, or null when it will.
+ *
+ * A UINode's placement is the LAYOUT's output: the layout pass writes the
+ * resolved box into `Transform.position` on every relayout, so an authored one
+ * holds until the next UI change and then vanishes. Moving a UI element means
+ * changing a layout INPUT — the anchor/offset in Absolute mode, or the flow it
+ * sits in.
+ */
+function layoutOwnedWarning(entity: EntityId, component: string, key: string): string | null {
+ if (component !== 'Transform' || !/^(position|rotation|scale)/.test(key)) return null;
+ const e = SceneModel.entityBySource(entity);
+ if (!e?.components.some((c) => c.type === 'UINode')) return null;
+ return `entity ${entity}: Transform.${key.split('.')[0]} is owned by the UI layout and will be overwritten `
+ + 'at the next relayout. Move a UI element with its layout INPUTS instead — set UINode.position to '
+ + 'Absolute (1) and give UINode.left / UINode.top — or place game content in the world as a Sprite, '
+ + 'where Transform.position is exactly how it is placed.';
}
/**
@@ -113,11 +145,14 @@ export async function applySceneOps(
return id;
};
+ const warnings: string[] = [];
const setFields = (entity: EntityId, fields: Record<string, unknown>): void => {
for (const [path, value] of Object.entries(fields)) {
const { component, key } = splitFieldPath(path);
// The declared inspector type wins inside setField; this argument is advisory.
EditorControlSurface.setField(entity, component, key, 'string', value as never);
+ const warning = layoutOwnedWarning(entity, component, key);
+ if (warning && !warnings.includes(warning)) warnings.push(warning);
}
};
@@ -184,5 +219,5 @@ export async function applySceneOps(
});
});
- return { refs, created, applied: ops.length };
+ return { refs, created, applied: ops.length, ...(warnings.length ? { warnings } : {}) };
}
diff --git a/desktop/tests/scene-ops-warnings.test.ts b/desktop/tests/scene-ops-warnings.test.ts
new file mode 100644
index 000000000..7cf6cf15b
--- /dev/null
+++ b/desktop/tests/scene-ops-warnings.test.ts
@@ -0,0 +1,92 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file A write that will not stick says so, in the reply to the write.
+ *
+ * From a dogfood run: an agent built a chess board out of forty UI nodes, set
+ * `Transform.position` on every one of them, and got an empty viewport with no
+ * error anywhere — the layout owns that field and overwrites it. The inspector
+ * card says so and the diagnostics sweep repeats it, but both are places you
+ * have to go LOOK; the answer belongs in the reply to the call that made the
+ * mistake.
+ */
+import { describe, it, expect, vi, beforeEach } from 'vitest';
+
+const h = {
+ entities: new Map<number, { components: Array<{ type: string }> }>(),
+ nextId: 100,
+};
+
+vi.mock('../src/engine/SceneModel', () => ({
+ SceneModel: {
+ entityBySource: (id: number) => h.entities.get(id) ?? null,
+ },
+}));
+
+vi.mock('../src/engine/EditorSession', () => ({
+ EditorControlSurface: {
+ atomic: (_label: string, fn: () => void) => fn(),
+ addEntity: () => {
+ const id = ++h.nextId;
+ h.entities.set(id, { components: [{ type: 'Transform' }] });
+ return id;
+ },
+ setField: () => {},
+ renameEntity: () => {},
+ setParent: () => {},
+ deleteEntity: () => {},
+ addComponent: (id: number, type: string) => h.entities.get(id)?.components.push({ type }),
+ removeComponent: () => {},
+ },
+}));
+
+vi.mock('../src/engine/entitySources', () => ({
+ prefabFromSpecs: () => null,
+ sourceById: () => null,
+}));
+
+const { applySceneOps } = await import('@/engine/sceneOps');
+
+beforeEach(() => {
+ h.entities.clear();
+ h.nextId = 100;
+});
+
+describe('a field the layout owns', () => {
+ it('warns when Transform.position is written on a UI node', async () => {
+ h.entities.set(1, { components: [{ type: 'Transform' }, { type: 'UINode' }] });
+ const res = await applySceneOps([
+ { op: 'set', entity: 1, fields: { 'Transform.position.x': 120, 'UINode.width.value': 46 } },
+ ]);
+ expect(res.applied).toBe(1);
+ expect(res.warnings).toHaveLength(1);
+ expect(res.warnings?.[0]).toContain('entity 1');
+ // The message has to name the way that DOES work, or it is just a scolding.
+ expect(res.warnings?.[0]).toContain('UINode.left');
+ });
+
+ it('says it once per entity and field, not once per member written', async () => {
+ h.entities.set(1, { components: [{ type: 'Transform' }, { type: 'UINode' }] });
+ const res = await applySceneOps([
+ { op: 'set', entity: 1, fields: { 'Transform.position.x': 1, 'Transform.position.y': 2, 'Transform.rotation': 45 } },
+ ]);
+ // x and y are one field between them; rotation is another.
+ expect(res.warnings).toHaveLength(2);
+ expect(res.warnings?.some((w) => w.includes('Transform.position'))).toBe(true);
+ expect(res.warnings?.some((w) => w.includes('Transform.rotation'))).toBe(true);
+ });
+
+ it('says nothing about a world entity — the same write is how you place one', async () => {
+ h.entities.set(2, { components: [{ type: 'Transform' }, { type: 'Sprite' }] });
+ const res = await applySceneOps([{ op: 'set', entity: 2, fields: { 'Transform.position.x': 120 } }]);
+ expect(res.warnings).toBeUndefined();
+ });
+
+ it('says nothing about the fields a UI node is actually moved with', async () => {
+ h.entities.set(1, { components: [{ type: 'Transform' }, { type: 'UINode' }] });
+ const res = await applySceneOps([
+ { op: 'set', entity: 1, fields: { 'UINode.position': 1, 'UINode.left.value': 40, 'UINode.top.value': 12 } },
+ ]);
+ expect(res.warnings).toBeUndefined();
+ });
+});