Build and revise procedural 3D assets with your coding agent. Local MCP server, CLI and TypeScript engine with rendered review, editable source and GLB export.
JavaScript 3d3d-assets3d-modelingagent-skillsagent-toolsai-agentsasset-pipelinecligame-assetsglbgltfmcpmcp-servermodel-context-protocolprocedural-generationprocedural-modelingtext-to-3dthreejstypescript
feat(composer): scene_set_environment + scene_set_backdrop tools + reuse steering
Matt Kissingere7517aa0e982
diff --git a/src/__tests__/composer-agent-tools.test.ts b/src/__tests__/composer-agent-tools.test.tsindex 68a613d3..f563c08c 100644--- a/src/__tests__/composer-agent-tools.test.ts+++ b/src/__tests__/composer-agent-tools.test.ts@@ -63,7 +63,7 @@ function call(tools: Tool[], name: string, input: unknown = {}): Promise<AnyResu}describe('surface', () => {- test('exposes the 14 scene_* tools by name', () => {+ test('exposes the 16 scene_* tools by name', () => {const { tools } = setup();const names = tools.map((t) => t.name).sort();expect(names).toEqual(@@ -81,10 +81,30 @@ describe('surface', () => {'scene_face','scene_group','scene_remove',+ 'scene_set_environment',+ 'scene_set_backdrop','scene_finalize',].sort(),);});++ test('scene_set_environment + scene_set_backdrop set scene-level theme on the model', async () => {+ const { model, tools } = setup();+ const env = await call(tools, 'scene_set_environment', { ground: 'edo' });+ expect(env.ok).toBe(true);+ expect(env.environment).toBe('edo');+ expect(model.environment()).toBe('edo');++ const bg = await call(tools, 'scene_set_backdrop', { kind: 'fuji', scale: 2 });+ expect(bg.ok).toBe(true);+ expect(model.sceneBackdrop()).toEqual({ kind: 'fuji', pos: [0, 20, -160], scale: 2 });++ // Both round-trip through the model JSON (so compose.ts reads them off result.scene).+ const json = model.toJSON();+ expect(json.environment).toBe('edo');+ expect(json.backdrop).toEqual({ kind: 'fuji', pos: [0, 20, -160], scale: 2 });+ expect(PlacementModel.fromJSON(json).sceneBackdrop()).toEqual({ kind: 'fuji', pos: [0, 20, -160], scale: 2 });+ });});describe('read tools', () => {diff --git a/src/composer/agent/prompt.ts b/src/composer/agent/prompt.tsindex 709c130c..9ebd9c5c 100644--- a/src/composer/agent/prompt.ts+++ b/src/composer/agent/prompt.ts@@ -17,7 +17,7 @@ export const COMPOSER_SYSTEM_PROMPT = `You are Kiln's scene composer. You arrang- Footprint overlaps are tracked and must be resolved: a finished scene has zero overlaps. The placement primitives (layout / cluster / ring) are overlap-free by construction; hand placements can collide.## The catalog is fixed-You may only place assets from this scene's catalog (call scene_list_assets to see every generationId, its name, and its footprint size). You cannot invent or generate new assets in this loop — compose with what you have, reusing an asset as many times as the scene needs.+You may only place assets from this scene's catalog (call scene_list_assets to see every generationId, its name, and its footprint size). You cannot invent or generate new assets in this loop — compose with what you have, reusing assets HEAVILY: most of a believable scene is the same few assets repeated.## Composition principles (make it read as intentional, not scattered)- Spacing is deliberate, not uniform. Give hero/landmark assets breathing room; let supporting and fill assets cluster and frame them. Open up sightlines and negative space.@@ -26,15 +26,29 @@ You may only place assets from this scene's catalog (call scene_list_assets to s- Use roles: hero (anchors the scene), support (frames + structures it), fill (density + texture).- Depth and rhythm beat symmetry. Vary distances; avoid a flat even grid unless the scene is literally a grid.+## Fill a believable place — reuse heavily, repeat architecture+A real town is mostly REPETITION. Place a handful of distinct assets MANY times rather than each asset once. Two levers, both overlap-free and budget-cheap (one tool call = many instances):+- Fill density via scene_cluster: scatter the small / natural fill assets in generous counts — e.g. 10-16 cherry-blossom trees in loose groves, 8-12 lanterns lining an approach, 6-10 fence segments around a precinct. Reuse the SAME asset across several clusters in different spots; vary spread + count so it reads organic, not tiled. Distinct fill variety is good (several different tree assets) but density comes from repetition.+- Repeated architecture via rows: a town is rows of houses, not one of each. Repeat each building asset 3-6 times to build a street — a line with consistent spacing + a shared facing — then a second parallel row to make a block. Use a few different building assets across the rows for rhythm. Heroes (a castle keep, a great gate, a pagoda) stay singular and get breathing room; SUPPORT + FILL carry the repetition.++## Stay under the placement budget (HARD cap: 200 instances)+Every cluster / ring instance and every laid-out asset counts toward a hard 200-instance ceiling — exceed it and the whole scene is REJECTED. A cluster of count:12 spends 12 of your 200. Budget before you place (e.g. ~40 houses across rows + ~30 trees + ~24 lanterns + heroes ≈ 100 — comfortably under). Prefer fewer, denser clusters over many tiny ones; scene_view reports the live placement count — check it after a big batch and stop adding fill before you hit the wall.++## Scene theme + atmosphere (scene-level, not placements)+- scene_set_environment sets the ground + sky THEME (terrain colour, sun, fog). Allowed: meadow, desert, egypt, plaza, snow, arctic, edo, night, studio. Match it to the setting — use "edo" for a Japanese / Edo castle-town scene.+- scene_set_backdrop adds ONE horizon billboard. Allowed: mushroom-cloud, sun-disc, aurora, fuji. Use "fuji" (Mt. Fuji) for a Japanese / Edo scene; place it far behind the composition on the view axis (e.g. pos [0, 20, -160]) and scale it up to own the skyline. Optional — omit if none fits.+- Set both EARLY (right after scene_list_assets), once each. They don't affect placement or overlaps.+## Tools- Plan: scene_list_assets (always first), scene_view (the current program + counts).+- Theme: scene_set_environment (ground/sky theme), scene_set_backdrop (one horizon billboard) — scene-level, set early, optional.- Build: scene_layout (place the WHOLE catalog at once into an overlap-free baseline — your default first move), scene_place (one asset), scene_cluster (N scattered around a point), scene_ring (N on a circle).- Refine: scene_move (reposition / use a separation vector / rescale), scene_face (re-orient), scene_group (bind + move together), scene_remove.- See + check: scene_render (the whole scene from three angles — LOOK at it), scene_screenshot_camera (one vantage you choose), scene_validate (overlaps + the vector to separate each).- Commit: scene_finalize (exactly once, at the end).## Working loop (you have a BOUNDED tool-call budget — finish within it)-1. scene_list_assets, and read the scene prompt: decide the layout idea (what anchors the scene, what frames it, where the groupings go).+1. scene_list_assets, and read the scene prompt: decide the layout idea (what anchors the scene, what frames it, where the groupings go). Then set the theme: scene_set_environment (and scene_set_backdrop if one fits the setting).2. START with scene_layout to place the WHOLE catalog at once into an overlap-free baseline: omit \`assets\` to include everything; anchor "zonedCenters" spreads them across districts (a hub + four satellites), "single" packs them around the origin; pick a facing; pass a \`scale\` up if the assets read small on the ground. This is ONE call for the whole scene — do NOT hand-place dozens of assets one at a time, which exhausts the budget before you can finish.3. scene_render and JUDGE it: spacing, facing, groupings, silhouette, overlaps. Use scene_screenshot_camera to check a key vantage (e.g. how it reads on approach).4. Refine with a HANDFUL of targeted edits, not a fresh placement per asset: scene_move / scene_face / scene_group the heroes, and build one or two deliberate clusters or rings for the spaces that matter (a courtyard, a motor-court, a row). scene_validate for overlaps (move by the mtv). Re-render once to confirm.diff --git a/src/composer/agent/tools.ts b/src/composer/agent/tools.tsindex 93995041..656f2913 100644--- a/src/composer/agent/tools.ts+++ b/src/composer/agent/tools.ts@@ -13,12 +13,12 @@* `tool()`. No THREE — rendering is the injected host port. Errors are in-band* `{ ok:false, error, hint }` (tools never throw at the model).*- * Surface (14): read — scene_list_assets, scene_view, scene_validate,+ * Surface (16): read — scene_list_assets, scene_view, scene_validate,* scene_render, scene_screenshot_camera · mutate — scene_layout, scene_place,* scene_cluster, scene_ring, scene_move, scene_face, scene_group, scene_remove ·- * commit — scene_finalize. (Extends the design doc's 12 with scene_cluster /- * scene_ring so the agent can author the grouping primitives the program already- * serializes.)+ * theme — scene_set_environment, scene_set_backdrop · commit — scene_finalize.+ * (Extends the design doc's 12 with scene_cluster / scene_ring so the agent can+ * author the grouping primitives the program already serializes.)*/import { ImageBlock, JsonBlock, type JSONValue, type Tool, tool } from '@strands-agents/sdk';import { z } from 'zod';@@ -62,6 +62,19 @@ const roleEnum = z.enum(['hero', 'support', 'fill']);/** center (face the scene origin) · out (face away) · [x,z] (face a point) · degrees. */const facing = z.union([z.literal('center'), z.literal('out'), vec2, z.number()]);const empty = z.object({});+/** Scene-level ground/sky theme ids + atmosphere backdrop kinds (mirror Studio). */+const groundThemeEnum = z.enum([+ 'meadow',+ 'desert',+ 'egypt',+ 'plaza',+ 'snow',+ 'arctic',+ 'edo',+ 'night',+ 'studio',+]);+const backdropKindEnum = z.enum(['mushroom-cloud', 'sun-disc', 'aurora', 'fuji']);// ── render-result → content blocks ────────────────────────────────────────────@@ -569,6 +582,62 @@ export function makeSceneComposerTools(opts: MakeComposerToolsOptions): Tool[] {},});+ const setEnvironmentTool: Tool = tool({+ name: 'scene_set_environment',+ description:+ "Set the scene's ground + sky THEME (terrain colour, sun, fog) — a scene-level choice, not a " ++ 'placement. Match it to the setting: "edo" for a Japanese / Edo castle-town scene (warm earth ' ++ 'ground, soft hanami sky), "desert"/"egypt" for arid, "snow"/"arctic" for cold, "meadow" for ' ++ 'green, "plaza"/"studio"/"night" for neutral or dark. Call it once, early. Does not affect ' ++ 'placements or overlaps.',+ inputSchema: z.object({+ ground: groundThemeEnum.describe('The ground/sky theme id.'),+ }),+ callback: (input) => {+ const i = input as { ground: z.infer<typeof groundThemeEnum> };+ model.setEnvironment(i.ground);+ const ev = capture();+ return {+ ok: true,+ environment: i.ground,+ placements: ev.placements.length,+ overlaps: ev.overlaps.length,+ } as JSONValue;+ },+ });++ const setBackdropTool: Tool = tool({+ name: 'scene_set_backdrop',+ description:+ 'Set ONE atmosphere backdrop rendered as a sky billboard on the horizon (not a placed asset): ' ++ '"fuji" a snow-capped Mt. Fuji (use it for a Japanese / Edo scene), "sun-disc" a low sun, ' ++ '"aurora" northern lights, "mushroom-cloud". Place it far behind the composition on the view ' ++ 'axis (default pos [0, 20, -160]); scale it up to dominate the skyline. Optional — omit if no ' ++ 'backdrop fits. Call it once, early.',+ inputSchema: z.object({+ kind: backdropKindEnum.describe('The backdrop kind.'),+ pos: vec3.optional().describe('Base position [x,y,z] on the horizon (default [0,20,-160]).'),+ scale: z.number().optional().describe('Visual scale multiplier (default 1).'),+ }),+ callback: (input) => {+ const i = input as {+ kind: z.infer<typeof backdropKindEnum>;+ pos?: [number, number, number];+ scale?: number;+ };+ // Default pos so the persisted backdrop always satisfies Studio's required pos field.+ const pos: [number, number, number] = i.pos ?? [0, 20, -160];+ model.setBackdrop({ kind: i.kind, pos, ...(i.scale != null ? { scale: i.scale } : {}) });+ const ev = capture();+ return {+ ok: true,+ backdrop: { kind: i.kind, pos, ...(i.scale != null ? { scale: i.scale } : {}) },+ placements: ev.placements.length,+ overlaps: ev.overlaps.length,+ } as JSONValue;+ },+ });+const finalizeTool: Tool = tool({name: 'scene_finalize',description:@@ -603,6 +672,8 @@ export function makeSceneComposerTools(opts: MakeComposerToolsOptions): Tool[] {faceTool,groupTool,removeTool,+ setEnvironmentTool,+ setBackdropTool,finalizeTool,];}diff --git a/src/composer/model.ts b/src/composer/model.tsindex 246e086f..3a4720c6 100644--- a/src/composer/model.ts+++ b/src/composer/model.ts@@ -43,6 +43,26 @@ export interface CatalogEntry {export type Role = 'hero' | 'support' | 'fill';+/** Scene-level terrain/sky theme id — mirrors Studio's GroundThemeId (kept local so+ * the engine carries no studio dependency; compose.ts casts across the wire). */+export type SceneGroundTheme =+ | 'meadow'+ | 'desert'+ | 'egypt'+ | 'plaza'+ | 'snow'+ | 'arctic'+ | 'edo'+ | 'night'+ | 'studio';++/** Scene atmosphere backdrop — mirrors Studio's SceneBackdrop. */+export interface SceneBackdropSpec {+ kind: 'mushroom-cloud' | 'sun-disc' | 'aurora' | 'fuji';+ pos?: Vec3;+ scale?: number;+}+/** One evaluated instance: a generation placed in the world, linked to its statement. */export interface Placement {instanceId: string;@@ -118,6 +138,10 @@ export interface SceneModelJSON {seed: number;catalog: CatalogEntry[];statements: Statement[];+ /** Scene-level ground/sky theme the agent chose (scene_set_environment). */+ environment?: SceneGroundTheme;+ /** Scene-level atmosphere backdrop the agent chose (scene_set_backdrop). */+ backdrop?: SceneBackdropSpec;}export interface PlaceArgs {@@ -199,6 +223,8 @@ export class PlacementModel {private readonly byAlias = new Map<string, Statement>();private seq = 0;private groupSeq = 0;+ private env?: SceneGroundTheme;+ private bg?: SceneBackdropSpec;constructor(name: string,@@ -432,6 +458,20 @@ export class PlacementModel {return this.stmts;}+ /** Scene-level theme/backdrop (not placements) — set once, early. */+ setEnvironment(ground: SceneGroundTheme): void {+ this.env = ground;+ }+ setBackdrop(b: SceneBackdropSpec): void {+ this.bg = b;+ }+ environment(): SceneGroundTheme | undefined {+ return this.env;+ }+ sceneBackdrop(): SceneBackdropSpec | undefined {+ return this.bg;+ }+// ── evaluate ────────────────────────────────────────────────────────────────/** Expand every statement to world placements, then gate on footprint overlap. */@@ -530,6 +570,8 @@ export class PlacementModel {seed: this.seed,catalog: [...this.catalog.values()],statements: this.stmts,+ ...(this.env ? { environment: this.env } : {}),+ ...(this.bg ? { backdrop: this.bg } : {}),}),);}@@ -547,6 +589,8 @@ export class PlacementModel {if (Number.isFinite(n)) maxSeq = Math.max(maxSeq, n);}m.seq = maxSeq;+ if (j.environment) m.env = j.environment;+ if (j.backdrop) m.bg = j.backdrop;return m;}}