Skip to content

armgit / kiln

Public Imported from GitHub
0

Build and revise procedural 3D assets with your coding agent. Local MCP server, CLI and TypeScript engine with rendered review, editable source and GLB export.

feat(composer): scene_set_environment + scene_set_backdrop tools + reuse steering

Matt Kissingere7517aa0e982
diff --git a/src/__tests__/composer-agent-tools.test.ts b/src/__tests__/composer-agent-tools.test.ts
index 68a613d3..f563c08c 100644
--- a/src/__tests__/composer-agent-tools.test.ts
+++ b/src/__tests__/composer-agent-tools.test.ts
@@ -63,7 +63,7 @@ function call(tools: Tool[], name: string, input: unknown = {}): Promise<AnyResu
}
describe('surface', () => {
- test('exposes the 14 scene_* tools by name', () => {
+ test('exposes the 16 scene_* tools by name', () => {
const { tools } = setup();
const names = tools.map((t) => t.name).sort();
expect(names).toEqual(
@@ -81,10 +81,30 @@ describe('surface', () => {
'scene_face',
'scene_group',
'scene_remove',
+ 'scene_set_environment',
+ 'scene_set_backdrop',
'scene_finalize',
].sort(),
);
});
+
+ test('scene_set_environment + scene_set_backdrop set scene-level theme on the model', async () => {
+ const { model, tools } = setup();
+ const env = await call(tools, 'scene_set_environment', { ground: 'edo' });
+ expect(env.ok).toBe(true);
+ expect(env.environment).toBe('edo');
+ expect(model.environment()).toBe('edo');
+
+ const bg = await call(tools, 'scene_set_backdrop', { kind: 'fuji', scale: 2 });
+ expect(bg.ok).toBe(true);
+ expect(model.sceneBackdrop()).toEqual({ kind: 'fuji', pos: [0, 20, -160], scale: 2 });
+
+ // Both round-trip through the model JSON (so compose.ts reads them off result.scene).
+ const json = model.toJSON();
+ expect(json.environment).toBe('edo');
+ expect(json.backdrop).toEqual({ kind: 'fuji', pos: [0, 20, -160], scale: 2 });
+ expect(PlacementModel.fromJSON(json).sceneBackdrop()).toEqual({ kind: 'fuji', pos: [0, 20, -160], scale: 2 });
+ });
});
describe('read tools', () => {
diff --git a/src/composer/agent/prompt.ts b/src/composer/agent/prompt.ts
index 709c130c..9ebd9c5c 100644
--- a/src/composer/agent/prompt.ts
+++ b/src/composer/agent/prompt.ts
@@ -17,7 +17,7 @@ export const COMPOSER_SYSTEM_PROMPT = `You are Kiln's scene composer. You arrang
- Footprint overlaps are tracked and must be resolved: a finished scene has zero overlaps. The placement primitives (layout / cluster / ring) are overlap-free by construction; hand placements can collide.
## The catalog is fixed
-You may only place assets from this scene's catalog (call scene_list_assets to see every generationId, its name, and its footprint size). You cannot invent or generate new assets in this loop — compose with what you have, reusing an asset as many times as the scene needs.
+You may only place assets from this scene's catalog (call scene_list_assets to see every generationId, its name, and its footprint size). You cannot invent or generate new assets in this loop — compose with what you have, reusing assets HEAVILY: most of a believable scene is the same few assets repeated.
## Composition principles (make it read as intentional, not scattered)
- Spacing is deliberate, not uniform. Give hero/landmark assets breathing room; let supporting and fill assets cluster and frame them. Open up sightlines and negative space.
@@ -26,15 +26,29 @@ You may only place assets from this scene's catalog (call scene_list_assets to s
- Use roles: hero (anchors the scene), support (frames + structures it), fill (density + texture).
- Depth and rhythm beat symmetry. Vary distances; avoid a flat even grid unless the scene is literally a grid.
+## Fill a believable place — reuse heavily, repeat architecture
+A real town is mostly REPETITION. Place a handful of distinct assets MANY times rather than each asset once. Two levers, both overlap-free and budget-cheap (one tool call = many instances):
+- Fill density via scene_cluster: scatter the small / natural fill assets in generous counts — e.g. 10-16 cherry-blossom trees in loose groves, 8-12 lanterns lining an approach, 6-10 fence segments around a precinct. Reuse the SAME asset across several clusters in different spots; vary spread + count so it reads organic, not tiled. Distinct fill variety is good (several different tree assets) but density comes from repetition.
+- Repeated architecture via rows: a town is rows of houses, not one of each. Repeat each building asset 3-6 times to build a street — a line with consistent spacing + a shared facing — then a second parallel row to make a block. Use a few different building assets across the rows for rhythm. Heroes (a castle keep, a great gate, a pagoda) stay singular and get breathing room; SUPPORT + FILL carry the repetition.
+
+## Stay under the placement budget (HARD cap: 200 instances)
+Every cluster / ring instance and every laid-out asset counts toward a hard 200-instance ceiling — exceed it and the whole scene is REJECTED. A cluster of count:12 spends 12 of your 200. Budget before you place (e.g. ~40 houses across rows + ~30 trees + ~24 lanterns + heroes ≈ 100 — comfortably under). Prefer fewer, denser clusters over many tiny ones; scene_view reports the live placement count — check it after a big batch and stop adding fill before you hit the wall.
+
+## Scene theme + atmosphere (scene-level, not placements)
+- scene_set_environment sets the ground + sky THEME (terrain colour, sun, fog). Allowed: meadow, desert, egypt, plaza, snow, arctic, edo, night, studio. Match it to the setting — use "edo" for a Japanese / Edo castle-town scene.
+- scene_set_backdrop adds ONE horizon billboard. Allowed: mushroom-cloud, sun-disc, aurora, fuji. Use "fuji" (Mt. Fuji) for a Japanese / Edo scene; place it far behind the composition on the view axis (e.g. pos [0, 20, -160]) and scale it up to own the skyline. Optional — omit if none fits.
+- Set both EARLY (right after scene_list_assets), once each. They don't affect placement or overlaps.
+
## Tools
- Plan: scene_list_assets (always first), scene_view (the current program + counts).
+- Theme: scene_set_environment (ground/sky theme), scene_set_backdrop (one horizon billboard) — scene-level, set early, optional.
- Build: scene_layout (place the WHOLE catalog at once into an overlap-free baseline — your default first move), scene_place (one asset), scene_cluster (N scattered around a point), scene_ring (N on a circle).
- Refine: scene_move (reposition / use a separation vector / rescale), scene_face (re-orient), scene_group (bind + move together), scene_remove.
- See + check: scene_render (the whole scene from three angles — LOOK at it), scene_screenshot_camera (one vantage you choose), scene_validate (overlaps + the vector to separate each).
- Commit: scene_finalize (exactly once, at the end).
## Working loop (you have a BOUNDED tool-call budget — finish within it)
-1. scene_list_assets, and read the scene prompt: decide the layout idea (what anchors the scene, what frames it, where the groupings go).
+1. scene_list_assets, and read the scene prompt: decide the layout idea (what anchors the scene, what frames it, where the groupings go). Then set the theme: scene_set_environment (and scene_set_backdrop if one fits the setting).
2. START with scene_layout to place the WHOLE catalog at once into an overlap-free baseline: omit \`assets\` to include everything; anchor "zonedCenters" spreads them across districts (a hub + four satellites), "single" packs them around the origin; pick a facing; pass a \`scale\` up if the assets read small on the ground. This is ONE call for the whole scene — do NOT hand-place dozens of assets one at a time, which exhausts the budget before you can finish.
3. scene_render and JUDGE it: spacing, facing, groupings, silhouette, overlaps. Use scene_screenshot_camera to check a key vantage (e.g. how it reads on approach).
4. Refine with a HANDFUL of targeted edits, not a fresh placement per asset: scene_move / scene_face / scene_group the heroes, and build one or two deliberate clusters or rings for the spaces that matter (a courtyard, a motor-court, a row). scene_validate for overlaps (move by the mtv). Re-render once to confirm.
diff --git a/src/composer/agent/tools.ts b/src/composer/agent/tools.ts
index 93995041..656f2913 100644
--- a/src/composer/agent/tools.ts
+++ b/src/composer/agent/tools.ts
@@ -13,12 +13,12 @@
* `tool()`. No THREE — rendering is the injected host port. Errors are in-band
* `{ ok:false, error, hint }` (tools never throw at the model).
*
- * Surface (14): read — scene_list_assets, scene_view, scene_validate,
+ * Surface (16): read — scene_list_assets, scene_view, scene_validate,
* scene_render, scene_screenshot_camera · mutate — scene_layout, scene_place,
* scene_cluster, scene_ring, scene_move, scene_face, scene_group, scene_remove ·
- * commit — scene_finalize. (Extends the design doc's 12 with scene_cluster /
- * scene_ring so the agent can author the grouping primitives the program already
- * serializes.)
+ * theme — scene_set_environment, scene_set_backdrop · commit — scene_finalize.
+ * (Extends the design doc's 12 with scene_cluster / scene_ring so the agent can
+ * author the grouping primitives the program already serializes.)
*/
import { ImageBlock, JsonBlock, type JSONValue, type Tool, tool } from '@strands-agents/sdk';
import { z } from 'zod';
@@ -62,6 +62,19 @@ const roleEnum = z.enum(['hero', 'support', 'fill']);
/** center (face the scene origin) · out (face away) · [x,z] (face a point) · degrees. */
const facing = z.union([z.literal('center'), z.literal('out'), vec2, z.number()]);
const empty = z.object({});
+/** Scene-level ground/sky theme ids + atmosphere backdrop kinds (mirror Studio). */
+const groundThemeEnum = z.enum([
+ 'meadow',
+ 'desert',
+ 'egypt',
+ 'plaza',
+ 'snow',
+ 'arctic',
+ 'edo',
+ 'night',
+ 'studio',
+]);
+const backdropKindEnum = z.enum(['mushroom-cloud', 'sun-disc', 'aurora', 'fuji']);
// ── render-result → content blocks ────────────────────────────────────────────
@@ -569,6 +582,62 @@ export function makeSceneComposerTools(opts: MakeComposerToolsOptions): Tool[] {
},
});
+ const setEnvironmentTool: Tool = tool({
+ name: 'scene_set_environment',
+ description:
+ "Set the scene's ground + sky THEME (terrain colour, sun, fog) — a scene-level choice, not a " +
+ 'placement. Match it to the setting: "edo" for a Japanese / Edo castle-town scene (warm earth ' +
+ 'ground, soft hanami sky), "desert"/"egypt" for arid, "snow"/"arctic" for cold, "meadow" for ' +
+ 'green, "plaza"/"studio"/"night" for neutral or dark. Call it once, early. Does not affect ' +
+ 'placements or overlaps.',
+ inputSchema: z.object({
+ ground: groundThemeEnum.describe('The ground/sky theme id.'),
+ }),
+ callback: (input) => {
+ const i = input as { ground: z.infer<typeof groundThemeEnum> };
+ model.setEnvironment(i.ground);
+ const ev = capture();
+ return {
+ ok: true,
+ environment: i.ground,
+ placements: ev.placements.length,
+ overlaps: ev.overlaps.length,
+ } as JSONValue;
+ },
+ });
+
+ const setBackdropTool: Tool = tool({
+ name: 'scene_set_backdrop',
+ description:
+ 'Set ONE atmosphere backdrop rendered as a sky billboard on the horizon (not a placed asset): ' +
+ '"fuji" a snow-capped Mt. Fuji (use it for a Japanese / Edo scene), "sun-disc" a low sun, ' +
+ '"aurora" northern lights, "mushroom-cloud". Place it far behind the composition on the view ' +
+ 'axis (default pos [0, 20, -160]); scale it up to dominate the skyline. Optional — omit if no ' +
+ 'backdrop fits. Call it once, early.',
+ inputSchema: z.object({
+ kind: backdropKindEnum.describe('The backdrop kind.'),
+ pos: vec3.optional().describe('Base position [x,y,z] on the horizon (default [0,20,-160]).'),
+ scale: z.number().optional().describe('Visual scale multiplier (default 1).'),
+ }),
+ callback: (input) => {
+ const i = input as {
+ kind: z.infer<typeof backdropKindEnum>;
+ pos?: [number, number, number];
+ scale?: number;
+ };
+ // Default pos so the persisted backdrop always satisfies Studio's required pos field.
+ const pos: [number, number, number] = i.pos ?? [0, 20, -160];
+ model.setBackdrop({ kind: i.kind, pos, ...(i.scale != null ? { scale: i.scale } : {}) });
+ const ev = capture();
+ return {
+ ok: true,
+ backdrop: { kind: i.kind, pos, ...(i.scale != null ? { scale: i.scale } : {}) },
+ placements: ev.placements.length,
+ overlaps: ev.overlaps.length,
+ } as JSONValue;
+ },
+ });
+
const finalizeTool: Tool = tool({
name: 'scene_finalize',
description:
@@ -603,6 +672,8 @@ export function makeSceneComposerTools(opts: MakeComposerToolsOptions): Tool[] {
faceTool,
groupTool,
removeTool,
+ setEnvironmentTool,
+ setBackdropTool,
finalizeTool,
];
}
diff --git a/src/composer/model.ts b/src/composer/model.ts
index 246e086f..3a4720c6 100644
--- a/src/composer/model.ts
+++ b/src/composer/model.ts
@@ -43,6 +43,26 @@ export interface CatalogEntry {
export type Role = 'hero' | 'support' | 'fill';
+/** Scene-level terrain/sky theme id — mirrors Studio's GroundThemeId (kept local so
+ * the engine carries no studio dependency; compose.ts casts across the wire). */
+export type SceneGroundTheme =
+ | 'meadow'
+ | 'desert'
+ | 'egypt'
+ | 'plaza'
+ | 'snow'
+ | 'arctic'
+ | 'edo'
+ | 'night'
+ | 'studio';
+
+/** Scene atmosphere backdrop — mirrors Studio's SceneBackdrop. */
+export interface SceneBackdropSpec {
+ kind: 'mushroom-cloud' | 'sun-disc' | 'aurora' | 'fuji';
+ pos?: Vec3;
+ scale?: number;
+}
+
/** One evaluated instance: a generation placed in the world, linked to its statement. */
export interface Placement {
instanceId: string;
@@ -118,6 +138,10 @@ export interface SceneModelJSON {
seed: number;
catalog: CatalogEntry[];
statements: Statement[];
+ /** Scene-level ground/sky theme the agent chose (scene_set_environment). */
+ environment?: SceneGroundTheme;
+ /** Scene-level atmosphere backdrop the agent chose (scene_set_backdrop). */
+ backdrop?: SceneBackdropSpec;
}
export interface PlaceArgs {
@@ -199,6 +223,8 @@ export class PlacementModel {
private readonly byAlias = new Map<string, Statement>();
private seq = 0;
private groupSeq = 0;
+ private env?: SceneGroundTheme;
+ private bg?: SceneBackdropSpec;
constructor(
name: string,
@@ -432,6 +458,20 @@ export class PlacementModel {
return this.stmts;
}
+ /** Scene-level theme/backdrop (not placements) — set once, early. */
+ setEnvironment(ground: SceneGroundTheme): void {
+ this.env = ground;
+ }
+ setBackdrop(b: SceneBackdropSpec): void {
+ this.bg = b;
+ }
+ environment(): SceneGroundTheme | undefined {
+ return this.env;
+ }
+ sceneBackdrop(): SceneBackdropSpec | undefined {
+ return this.bg;
+ }
+
// ── evaluate ────────────────────────────────────────────────────────────────
/** Expand every statement to world placements, then gate on footprint overlap. */
@@ -530,6 +570,8 @@ export class PlacementModel {
seed: this.seed,
catalog: [...this.catalog.values()],
statements: this.stmts,
+ ...(this.env ? { environment: this.env } : {}),
+ ...(this.bg ? { backdrop: this.bg } : {}),
}),
);
}
@@ -547,6 +589,8 @@ export class PlacementModel {
if (Number.isFinite(n)) maxSeq = Math.max(maxSeq, n);
}
m.seq = maxSeq;
+ if (j.environment) m.env = j.environment;
+ if (j.backdrop) m.bg = j.backdrop;
return m;
}
}