Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

fix(agent): the brief taught an API that does not exist, and nothing compiled it

esengine0dd39a6b8b43
diff --git a/desktop/electron/agent/prompt.ts b/desktop/electron/agent/prompt.ts
index a6425cc08..4a55bd254 100644
--- a/desktop/electron/agent/prompt.ts
+++ b/desktop/electron/agent/prompt.ts
@@ -35,7 +35,8 @@ Working style:
- Units are the editor's: 1 world unit = 1 design pixel. Y is up in the world, and UI is laid out top-down.
- Game content lives in the WORLD: a board, its pieces, a character are Sprite / ShapeRenderer entities you place with Transform.position. A Canvas and its UINodes are for HUD and menus, and their placement is the layout's OUTPUT — a Transform.position written on a UI node is overwritten at the next relayout, which looks like nothing happened. Move those with layout inputs instead (UINode.position = Absolute, then left/top), or let the flow place them.
- Behaviour is a system and data is a component, both declared through the front door: \`defineSystem([Query(Mut(Transform)), Res(Time)], (q, t) => {...})\` and \`defineComponent('Name', { speed: 100 })\`, registered with \`addSystemToSchedule\`. create_script writes either one already wired into the entry that loads it. The underscore-prefixed fields you can see on those objects (\`_params\`, \`_fn\`, \`_id\`) are the scheduler's, not an authoring surface: assembling one by hand happens to run and leaves a project nobody else's tools understand.
-- A game reads input from the ENGINE, never from the DOM. In a system, ask for \`Res(Input)\` and \`Res(CameraView)\`: \`input.isMouseButtonPressed(MouseButton.Left)\`, \`input.mouseX/mouseY\` (SCREEN pixels, canvas-relative, y down), and \`camera.getWorldMousePosition()\` / \`camera.screenToWorld(x, y)\` for where that is in the world. \`document.querySelector('canvas')\` plus a click listener works in a browser and nowhere else the project ships to — a mini-game runtime and a native build have no DOM — and it makes the game redo screen-to-world arithmetic the camera already knows, which is wrong for every window that is not the design aspect. Read the shape of an API before calling it (read_project_file takes \`offset\` and \`limit\` to page a big .d.ts) rather than guessing method names into a try/catch that hides the mistake.
+- A game reads input from the ENGINE, never from the DOM. In a system, ask for \`Res(Input)\` and \`Res(CameraView)\`: \`input.isMouseButtonPressed(0)\` (0 = left, 1 = right, 2 = middle — plain numbers; \`MouseButton\` is the InputMap BINDING builder, \`MouseButton(0)\`, and has no \`.Left\`), \`input.mouseX/mouseY\` (SCREEN pixels, canvas-relative, y down), and \`camera.getWorldMousePosition()\` / \`camera.screenToWorld(x, y)\` for where that is in the world — both return \`null\` when there is no camera, so check before using it. \`document.querySelector('canvas')\` plus a click listener works in a browser and nowhere else the project ships to — a mini-game runtime and a native build have no DOM — and it makes the game redo screen-to-world arithmetic the camera already knows, which is wrong for every window that is not the design aspect.
+- Ask the compiler, do not guess and do not page the .d.ts. \`lookup_symbol\` gives a name's real signature, its doc and where it is declared, in one call; \`search_project_files\` finds where something is written. Writing a .ts file REPORTS ITS TYPE ERRORS in the reply, and \`check_scripts\` asks for them at any time. A game whose script does not compile does not run at all, so an error there is the first thing to fix — not something to discover after the screen comes up black.
- Say what you did in a sentence or two. The user watched it happen — a summary of every call is noise. Speak the user's language: answer in the language they wrote to you in.`;
/**
diff --git a/desktop/tests/agent-brief-compiles.test.ts b/desktop/tests/agent-brief-compiles.test.ts
new file mode 100644
index 000000000..21c02820f
--- /dev/null
+++ b/desktop/tests/agent-brief-compiles.test.ts
@@ -0,0 +1,72 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file The built-in agent's brief teaches APIs; this compiles them.
+ *
+ * A brief is prose, so nothing checked that the code inside it existed — and for
+ * six months it taught `input.isMouseButtonPressed(MouseButton.Left)`, which
+ * type-checks nowhere: `MouseButton` is the InputMap binding builder. The agent
+ * copied it into every game it built and the mouse silently did nothing.
+ *
+ * Two halves, and both are needed. The fixture must COMPILE against the real SDK
+ * .d.ts, and the brief must still be TEACHING what the fixture does — a fixture
+ * that drifts from the prose guards nothing.
+ */
+import { describe, it, expect } from 'vitest';
+import ts from 'typescript';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { existsSync } from 'node:fs';
+import { SYSTEM_PROMPT } from '../electron/agent/prompt';
+
+const HERE = path.dirname(fileURLToPath(import.meta.url));
+const FIXTURE = path.join(HERE, 'fixtures', 'agent-brief-usage.ts');
+const SDK_TYPES = path.join(HERE, '..', '..', 'sdk', 'dist', 'index.d.ts');
+
+describe("the agent's brief", () => {
+ it('teaches APIs that compile against the SDK', () => {
+ // A missing sdk/dist is a stale checkout, not a passing test: say so instead
+ // of reporting green on a compile that never happened.
+ expect([SDK_TYPES, existsSync(SDK_TYPES)]).toEqual([SDK_TYPES, true]);
+
+ const program = ts.createProgram([FIXTURE], {
+ target: ts.ScriptTarget.ES2020,
+ module: ts.ModuleKind.ESNext,
+ moduleResolution: ts.ModuleResolutionKind.Bundler,
+ strict: true,
+ skipLibCheck: true,
+ noEmit: true,
+ paths: { esengine: [SDK_TYPES] },
+ });
+ const errors = ts.getPreEmitDiagnostics(program)
+ .filter((d) => d.category === ts.DiagnosticCategory.Error)
+ .map((d) => {
+ const where = d.file && d.start !== undefined
+ ? `${path.basename(d.file.fileName)}:${d.file.getLineAndCharacterOfPosition(d.start).line + 1}`
+ : '(project)';
+ return `${where} TS${d.code}: ${ts.flattenDiagnosticMessageText(d.messageText, ' ')}`;
+ });
+ expect(errors).toEqual([]);
+ });
+
+ it('still says what the fixture proves', () => {
+ // Positive coupling, not a blacklist of past mistakes: each of these appears
+ // in the brief AND is exercised by the fixture, so the two cannot drift
+ // apart silently. A brief that stops teaching one of them fails here, and
+ // whoever removes it either updates the fixture or reconsiders.
+ for (const taught of [
+ 'isMouseButtonPressed(0)',
+ 'input.mouseX/mouseY',
+ 'camera.screenToWorld(x, y)',
+ 'camera.getWorldMousePosition()',
+ 'defineComponent(',
+ 'addSystemToSchedule',
+ 'lookup_symbol',
+ 'check_scripts',
+ ]) {
+ expect([taught, SYSTEM_PROMPT.includes(taught)]).toEqual([taught, true]);
+ }
+ // The wrong form must not come back. Named, because this one shipped.
+ expect(SYSTEM_PROMPT).not.toContain('MouseButton.Left');
+ });
+});
diff --git a/desktop/tests/agent-prompt.test.ts b/desktop/tests/agent-prompt.test.ts
index 4164053a5..c4a21529e 100644
--- a/desktop/tests/agent-prompt.test.ts
+++ b/desktop/tests/agent-prompt.test.ts
@@ -34,7 +34,13 @@ describe('the agent is told where game input comes from', () => {
});
it('points at the way to read an API instead of guessing at it', () => {
- expect(SYSTEM_PROMPT).toMatch(/offset.*limit|limit.*offset/);
+ // It used to point at paging the .d.ts with offset/limit, which is what the
+ // agent then did: 50k lines, a hundred at a time, most of a context window
+ // spent to learn one method name. The compiler answers the same question in
+ // one call, and knows whether the answer was used correctly.
+ expect(SYSTEM_PROMPT).toContain('lookup_symbol');
+ expect(SYSTEM_PROMPT).toContain('check_scripts');
+ expect(SYSTEM_PROMPT).toMatch(/do not guess/i);
});
it('names the two front doors, and that the underscore fields are not one', () => {
diff --git a/desktop/tests/fixtures/agent-brief-usage.ts b/desktop/tests/fixtures/agent-brief-usage.ts
new file mode 100644
index 000000000..eeb238b89
--- /dev/null
+++ b/desktop/tests/fixtures/agent-brief-usage.ts
@@ -0,0 +1,52 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file Every API the built-in agent's brief teaches, written the way the brief
+ * teaches it — compiled by agent-brief-compiles.test.ts.
+ *
+ * The brief told the agent to call `input.isMouseButtonPressed(MouseButton.Left)`
+ * for six months. `MouseButton` is the InputMap binding builder and has no
+ * `.Left`, so the call passed `undefined` and read false forever: every game the
+ * agent built ignored the mouse, silently, and the agent had no way to find out.
+ * Prose about an API is code that nothing compiles. This file is that code.
+ */
+import {
+ addSystemToSchedule,
+ defineComponent,
+ defineSystem,
+ CameraView,
+ Commands,
+ Input,
+ MouseButton,
+ Mut,
+ Query,
+ Res,
+ Schedule,
+ Time,
+ Transform,
+} from 'esengine';
+
+export const Marker = defineComponent('AgentBriefMarker', { speed: 100 });
+
+const drift = defineSystem([Query(Mut(Transform)), Res(Time)], (q, t) => {
+ for (const [, transform] of q) {
+ transform.position.x += t.delta;
+ }
+});
+
+const pointer = defineSystem(
+ [Res(Input), Res(CameraView), Commands()],
+ (input, camera, commands) => {
+ if (!input.isMouseButtonPressed(0)) return;
+ const screen: [number, number] = [input.mouseX, input.mouseY];
+ const world = camera.screenToWorld(screen[0], screen[1]) ?? camera.getWorldMousePosition();
+ if (!world) return;
+ commands.spawn('Placed').insert(Transform, { position: { x: world.x, y: world.y, z: 0 } });
+ },
+);
+
+// The binding builder the brief distinguishes from the raw button number.
+export const leftClickBinding = MouseButton(0);
+
+addSystemToSchedule(Schedule.Update, drift);
+addSystemToSchedule(Schedule.Update, pointer);