Skip to content

armgit / kiln

Public Imported from GitHub
0

Build and revise procedural 3D assets with your coding agent. Local MCP server, CLI and TypeScript engine with rendered review, editable source and GLB export.

Fix blind-run MCP receipts and apply dogfooding guidance (#144)

Matthew Kissinger65ce2f815a70
diff --git a/.agents/skills/kiln-setup-workspace/SKILL.md b/.agents/skills/kiln-setup-workspace/SKILL.md
index 193b3753..d675789f 100644
--- a/.agents/skills/kiln-setup-workspace/SKILL.md
+++ b/.agents/skills/kiln-setup-workspace/SKILL.md
@@ -58,6 +58,13 @@ A `KILN_RENDER_PORT_URL` whose origin is exactly the shared local socket (`http:
Accept the project and MCP trust prompts, then confirm the server is actually live rather than assuming it from configuration. Call `kiln_discover` on `kiln_workspace` with `{ capabilities: true }`; it returns the runtime, source, export and camera contract and proves the tools resolved. For OpenCode, `opencode debug config` from the workspace is the configuration check: it shows the resolved `kiln_workspace` entry. `opencode mcp list` reports what OpenCode's shared background service loaded, so it can print "No MCP servers configured" in a correctly configured workspace. `node kiln.mjs discover --capabilities --json` from the workspace is the harness-independent proof; it reports the same runtime, collections and source storage as the MCP server. Compare `capabilities.engine.installUrl` with `runtime` in `.kiln/workspace.json`. A server named `kiln` from a global installation is a different thing. Do not substitute it silently; report the setup problem instead. When another agent is to do the authoring, start it as a separate harness process in the workspace directory (for OpenCode, `opencode run --standalone "<brief>"` after changing into the workspace); a subagent of the current session inherits this session's tools and never sees the workspace's MCP server, so it would be left with the CLI alone.
+When the task includes that separate author's result, retain its process/session
+identifier and wait for its terminal result. A launch message, session ID or empty
+log proves no authoring work. Use the harness's noninteractive mode with closed
+stdin for an unattended child, preserve stdout/stderr, and check the requested
+files and saved revision before reporting completion. If it stops early, report
+the incomplete handoff; confirm it has stopped before launching a replacement.
+
For modeling, `kiln_discover({})` supplies a compact orientation and starting signatures. Use natural-language `query` for related operations, assemblies, and optional recipes; use exact `ids` for complete contracts. Discovery needs no separate search model or asset-category selection. The local equivalent is `node kiln.mjs discover --query "curved hollow tube"`, followed by `node kiln.mjs discover --id RETURNED_ID`.
Confirm the installed skills are readable at `skills/` in the workspace, and read the relevant one from there rather than a global copy. Whether the harness also registers them natively depends on the harness.
diff --git a/.claude/skills/kiln-setup-workspace/SKILL.md b/.claude/skills/kiln-setup-workspace/SKILL.md
index 193b3753..d675789f 100644
--- a/.claude/skills/kiln-setup-workspace/SKILL.md
+++ b/.claude/skills/kiln-setup-workspace/SKILL.md
@@ -58,6 +58,13 @@ A `KILN_RENDER_PORT_URL` whose origin is exactly the shared local socket (`http:
Accept the project and MCP trust prompts, then confirm the server is actually live rather than assuming it from configuration. Call `kiln_discover` on `kiln_workspace` with `{ capabilities: true }`; it returns the runtime, source, export and camera contract and proves the tools resolved. For OpenCode, `opencode debug config` from the workspace is the configuration check: it shows the resolved `kiln_workspace` entry. `opencode mcp list` reports what OpenCode's shared background service loaded, so it can print "No MCP servers configured" in a correctly configured workspace. `node kiln.mjs discover --capabilities --json` from the workspace is the harness-independent proof; it reports the same runtime, collections and source storage as the MCP server. Compare `capabilities.engine.installUrl` with `runtime` in `.kiln/workspace.json`. A server named `kiln` from a global installation is a different thing. Do not substitute it silently; report the setup problem instead. When another agent is to do the authoring, start it as a separate harness process in the workspace directory (for OpenCode, `opencode run --standalone "<brief>"` after changing into the workspace); a subagent of the current session inherits this session's tools and never sees the workspace's MCP server, so it would be left with the CLI alone.
+When the task includes that separate author's result, retain its process/session
+identifier and wait for its terminal result. A launch message, session ID or empty
+log proves no authoring work. Use the harness's noninteractive mode with closed
+stdin for an unattended child, preserve stdout/stderr, and check the requested
+files and saved revision before reporting completion. If it stops early, report
+the incomplete handoff; confirm it has stopped before launching a replacement.
+
For modeling, `kiln_discover({})` supplies a compact orientation and starting signatures. Use natural-language `query` for related operations, assemblies, and optional recipes; use exact `ids` for complete contracts. Discovery needs no separate search model or asset-category selection. The local equivalent is `node kiln.mjs discover --query "curved hollow tube"`, followed by `node kiln.mjs discover --id RETURNED_ID`.
Confirm the installed skills are readable at `skills/` in the workspace, and read the relevant one from there rather than a global copy. Whether the harness also registers them natively depends on the harness.
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9f672a7c..0a6484c4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,13 @@ Changes since the 0.10.0 tagged package release. The package version stays 0.10.
until the next package release; source/main and website deployment are separate
from publishing a new installable package.
+- Blind-run receipts recognize OpenCode's recorded code-mode MCP calls and AGY's
+ generic MCP dispatcher without inferring execution from source or output text.
+ Setup and authoring guidance now covers waiting for independent authors, checking
+ sloping panel fit, retaining repeated parts during edits and saving child revisions
+ before exporting. The eight-asset campaign records its incomplete handoffs and
+ visual defects separately from valid exports.
+
- **Breaking (planned 0.11):** `optimize: 'full'` replaces flatten/join with rigid-group
merging. Its rigid-merge pass preserves node names, hierarchy and transforms, including animated targets,
named joint pivots, semantic nodes, composition placements and LOD levels. Ordinary
diff --git a/docs/dogfooding.md b/docs/dogfooding.md
index d3ada965..640b5098 100644
--- a/docs/dogfooding.md
+++ b/docs/dogfooding.md
@@ -11,6 +11,13 @@ Authorized free models and existing subscriptions may be used, with the model,
runtime and guidance fixed within each trial. Historical run receipts below keep
the versions, tool names and outcomes that were actually observed.
+The [October 5 eight-asset campaign](reviews/2026-10-05-blind-assets.md) adds fresh
+AGY/Gemini 3.8 Flash High and OpenCode/Muse Spark Contributor 1.3 trials: four public
+clone/setup attempts, four prepared authorings and four localized edits. Its two
+assisted OpenCode recoveries remain distinct from the incomplete blind handoffs.
+The report covers native traces, independent export checks, visual defects and
+the resulting evaluation-tool and skill changes.
+
| tier | question it answers | cost |
| --- | --- | --- |
| 0 · wiring smoke | can this harness reach the tools at all? | one short turn |
diff --git a/docs/reviews/2026-10-05-blind-assets.md b/docs/reviews/2026-10-05-blind-assets.md
new file mode 100644
index 00000000..8abf226c
--- /dev/null
+++ b/docs/reviews/2026-10-05-blind-assets.md
@@ -0,0 +1,274 @@
+# Blind setup and eight-asset dogfooding, October 5, 2026
+
+Eight different asset types produced editable source, GLBs and preview sheets.
+The two AGY public-clone trials completed setup and independent authoring without
+operator intervention. Both OpenCode public-clone trials completed workspace setup
+but stopped at the author handoff. Separate, explicitly assisted authoring sessions
+then completed those two assets. All four prepared-workspace authorings and their
+four edit turns completed. This is evidence of useful engine breadth and several
+workflow and review weaknesses, not blanket asset or release acceptance.
+
+The most consequential findings are incomplete child-process handoffs, confident
+visual claims contradicted by geometry, and loss of repeated parts during a
+localized edit. No core geometry regression was isolated by this campaign.
+One evaluation-tool defect was reproduced and fixed: native MCP calls were being
+missed in the current harness trace formats. Targeted skill guidance was updated
+after the trials; those changes have delivery checks but no new blind-model
+qualification yet.
+
+## Candidate, models and evidence
+
+- Public GitHub clone and prepared runtime head:
+ `42fab36d700ef86eed6581ca993d838dfb23af3c`, version `0.10.0`.
+- All eight workspace runtimes were checked after authoring and report that head
+ and build identity
+ `sha256:58b1b1784701b3b04bac6e7a429ea5dd9d1aeac646e5189bbb21f251e9f1a295`.
+- AGY `1.2.17`: `gemini-3.8-flash-high`, explicitly `--effort high`.
+- OpenCode `2.0.14`: `opencode/muse-spark-1.3-contributor-free`, the available
+ Muse Spark Contributor 1.3 route. A real READY probe and exported native session
+ metadata verified it; the model-list command alone was insufficient. No model
+ substitution or paid API fallback was used.
+- Maintainer toolchain: Bun `1.4.2`, Node `22.23.3`, npm `12.2.0`.
+- Evidence root: `C:\Users\Mattm\X\kiln-dogfood\blind-assets-2026-10-05`.
+ `review/index.html` contains all eight preview sheets and source/GLB downloads,
+ with separate baseline and edited files. It is a local artifact, not a publication.
+
+The protocol came from the maintained setup and batch-dispatch clean-room guidance,
+`docs/harnesses.md`, `docs/dogfooding.md`, and the existing evaluation records. The
+earlier October readiness campaign was background for the operator only. Agents
+received no prior results, engine implementation, examples or asset repair hints.
+The prepared runtime was a fresh local clone, fresh frozen dependency installation,
+and fresh viewer/runtime build. It remained frozen while the repository fixes were
+made. The four public-clone agents performed their own installation and setup.
+
+There were four public-clone trial cells, four prepared authoring cells, four
+predeclared edit turns, and two assisted recovery sessions. Child setup/author
+processes and READY probes are additional episodes, not extra successful trials.
+The recovery driver records `stage: null` because its special argument was parsed
+numerically; the `*-assisted/stage-0-*` paths and prompts identify those sessions
+unambiguously. Original receipts were preserved; `derived-receipts.json` is the
+separate recount using the fixed trace parser.
+
+## Outcomes
+
+Times are observed process wall times rounded to minutes. Runs overlapped and
+tasks differ, so these are neither benchmarks nor a model ranking.
+
+| Trial | Route and setup | First attempt | Artifact and review |
+| --- | --- | --- | --- |
+| Brass astronomical telescope | AGY, public clone, ordinary path | Complete, 18.7 min | Recognizable brass telescope, nested tubes, hood, yoke and tripod. Saved revision, source, GLB and sheet. Elevation clip has one exported track with matching endpoint poses. |
+| Victorian greenhouse | OpenCode, public clone, ordinary path | Incomplete handoff, 3.7 min | Separate author completed in 6.0 min. Saved/exported, but roof panes are perpendicular to the sloping frame despite a claim of correct seating. Door track closes at its endpoints. |
+| Canvas mountaineering backpack | AGY, public clone, path with spaces | Complete, 23.9 min | Saved/exported; routed straps, buckles and blanket are recognizable. At sheet scale the canvas looks comparatively rigid and the requested wear/folds are weak. |
+| Reef manta ray | OpenCode, public clone, path with spaces | Incomplete handoff, 4.3 min | Separate author completed in 7.3 min. Seven tracks have matching start/end poses over four seconds. Visibly segmented, overlapping wing pieces fall short of the smooth continuous-body brief. |
+| Hollow ventilation transition duct | AGY, prepared workspace | Complete, 12.3 min | Convincing rounded-rectangular to circular hollow transition, flanges and panel. Saved/exported. The panel edit completed in 2.7 min. |
+| Sword fern clump | OpenCode, prepared workspace | Complete, 3.3 min | Saved/exported. Reads better from above than edge-on. The 1.4 min edit shortened the target frond but deleted 12 leaflets; duplicate names prevented built-in revision comparison. |
+| Celadon teapot | AGY, prepared workspace | Complete, 13.5 min | Saved/exported, coherent glaze/handle/lid, visible body faceting. The hollow spout overlaps an uncut body wall at its junction. A 3.7 min tip edit preserved unrelated geometry and motion. |
+| Lunar salvage crawler | OpenCode, prepared workspace | Complete, 2.7 min | Saved/exported baseline; functional but blocky mechanical treatment. The 1.5 min rack edit preserved other components but was exported without saving a child revision. |
+
+All eight final baseline GLBs and all four edited GLBs pass the independent
+Khronos glTF validator with **zero errors and zero warnings**. The wider scan also
+checked saved duplicates and retained draft exports: 31 files, all passing. This
+establishes file validity, not visual quality, functional clearances, consumer
+performance or owner acceptance.
+
+The source set exercises custom mesh topology, lofted/hollow forms, Boolean work,
+repeated foliage, hierarchy, animation, procedural textures and packaged recipes.
+Concrete recipe use includes rubber, glass, cloth, painted metal, emissive material
+and packaged brown-leather texture maps. It does not qualify every pack or the
+Projects, shared material-library, LOD and destination-scene workflows.
+
+## Findings from the traces and exported geometry
+
+### 1. OpenCode stopped at the independent-author handoff
+
+Both setup parents returned exit code zero while their final messages described
+waiting for an author. That is incomplete task execution. For greenhouse, the
+launched author session contained only its user prompt and no model tokens. The
+first manta author reached ten messages and Discovery, then stopped; its later
+replacement also contained only the initial prompt. Neither initial trial had a
+finished asset before intervention.
+
+Separate operator-launched authors, in the workspaces the original trials created,
+completed the original asset briefs with the same model and no geometry hints.
+Their outputs are useful asset evidence, but do not convert the original onboarding
+attempts into passes. The evidence implicates process/session handoff, rather than
+a missing engine capability. It does not isolate a single OpenCode termination
+cause. The maintained setup skill now requires waiting for the child terminal
+result, retaining logs, checking artifacts and the saved revision, and verifying
+that an earlier child stopped before launching a replacement.
+
+### 2. The greenhouse review missed a ninety-degree frame error
+
+The final source defines `TILT = 90 - PITCH`. The rafter long axis uses
+`[-s*TILT, 0, 0]`, while an XY pane uses `[s*PITCH, 0, 0]`. Their relevant axes
+therefore differ by 90 degrees. The exported six-view sheet visibly shows panes
+standing out of the roof. Matching centers and the right nominal pitch were not
+enough, and the agent's claim that the panes were seated in their frames is false.
+
+The shared geometry reference now directs authors to check all transformed panel
+corners against the frame plane and opening, as well as an oblique view. This is
+general shape/fit guidance; greenhouse dimensions remain in its asset workspace.
+
+### 3. A localized fern edit preserved the wrong inventory
+
+The edit changed the tallest frond length from 1.06 to 0.80. Its length-dependent
+construction also reduced the exported node count from 389 to 377. Independent
+comparison found 336 unchanged nodes and all changes confined to the intended
+frond, but twelve leaflets disappeared. The final response claimed that all 38
+leaflets had been redistributed. That claim is contradicted by the exported files.
+
+Repeated sibling names also caused Kiln's comparison to reject the structure. The
+agent disclosed this rejection, then relied on a small source diff and bounds.
+Those are insufficient preservation evidence. The geometry reference now makes
+assembly/element indices and retained child counts explicit; the refinement skill
+calls for an inventory check when resizing repeated structures.
+
+### 4. Exact preservation can coexist with an incomplete asset
+
+Independent comparison decoded the baseline and revised GLBs, compared accessor
+bytes, material records, world matrices rounded to eight decimal places, and
+exported animation targets/times/values. Duplicate sibling names in the fern were
+disambiguated by stable occurrence index for reviewer analysis only; this does
+not imply that Kiln can address those ambiguous parts.
+
+| Edit | Independently observed change | Preserved |
+| --- | --- | --- |
+| Duct panel out by 0.15 m | Five world transforms: assembly and four child meshes | Seven nodes; all geometry/material records |
+| Tallest fern frond shorter | 53 changed/removed nodes within that frond, including twelve removed leaflets | 336 nodes outside the changes; no animation present |
+| Teapot tip bore +20% | One mesh's geometry changed | Twelve nodes and both lid animation tracks |
+| Crawler rack up by 0.2 m | Fifteen rack/support nodes; support geometry extended | 97 nodes and all three crane tracks |
+
+The teapot's body is a continuous shell without a spout cutout. A raycast against
+its exported body triangles, along the start of the authored spout centerline,
+hits the wall at approximately `[0.08114, 0.04919, 0]`. A hollow tube and visible
+join do not establish a usable pouring passage. The tip edit is localized and
+correct in isolation; it does not repair that baseline limitation.
+
+The crawler's revised source/GLB/PNG exist, but there is no saved child revision.
+The agent interpreted the requested filenames as sufficient delivery despite the
+collection refinement contract. The refinement skill's closing export example now
+states explicitly that file writes follow the child save and do not create one.
+
+### 5. Errors were often recoverable, but consumed avoidable work
+
+- OpenCode repeatedly tried POSIX `head`, `which` and `python3` in PowerShell.
+ The tools failed and the agents recovered. This is harness/platform friction,
+ not an engine defect.
+- AGY's isolated home triggered a missing PowerShell-profile path repeatedly.
+ These messages did not establish failure of the following command. They should
+ be separated from engine errors when mining transcripts.
+- The telescope tried the nonexistent `differenceGeo` name and recovered through
+ Discovery. The duct hit a precise rounded-box radius rejection, an unsupported
+ capture `preset`, and a guessed part name, then corrected them.
+- The duct called `kiln_inspect` with `image: false` but no numeric selector. The
+ error text correctly named the missing selector, while the generic `next` field
+ incorrectly recommended editing source. Improving that diagnostic is a bounded
+ engine follow-up; it was not changed here without a dedicated reproduction.
+- The teapot encountered zero normals rejected by glTF validation, investigated
+ its authored geometry, and repaired the source. The final exports validate.
+ This is evidence that the guard caught a real invalid draft, not an isolated
+ regression in an engine primitive.
+- AGY's streamed parent trace could stop updating while a child continued working.
+ Native conversation records showed progress. A quiet stdout stream alone is
+ insufficient evidence of a hung run.
+
+The regex error candidates in `trace-audit.json` include documentation excerpts
+and profile warnings. Their raw count must not be presented as an engine error rate.
+
+### 6. The evaluator missed actual MCP use
+
+OpenCode 2 records calls inside its `execute` wrapper at
+`part.state.metadata.metadata.toolCalls`. AGY can use `call_mcp_tool`, naming the
+server and tool in its parameters only on a later step update. The existing
+counter missed both forms and could report no workspace MCP usage despite real
+calls in native traces.
+
+`scripts/tier2-dogfood.mjs` now counts observed OpenCode nested calls alongside the
+wrapper and merges AGY step updates before resolving the dispatcher. It does not
+infer execution from generated code or tool-output prose. Focused tests first
+failed on both formats and then passed after the change. Original campaign
+receipts remain untouched; the derived recount is separate.
+
+Parent-process counts exclude independent child sessions. The telescope author
+made 25 observed workspace MCP calls; the backpack author made 35, separately from
+its one-call setup probe. `child-mcp-receipts.json` records those native child
+counts. Zero parent MCP calls must not be interpreted as zero author MCP use.
+
+## Isolation and evidence limits
+
+AGY used a fresh home/config/data directory and retained OS-keyring sign-in without
+copying credentials. OpenCode used fresh XDG configuration and private standalone
+servers while retaining its account data. An inherited global `kiln` tool remained
+visible in nested OpenCode context. Authors were instructed to use
+`kiln_workspace`; this is a documented inherited-context limitation, not a claim
+of an empty OS account or filesystem sandbox.
+
+No engine implementation/example reads were found in the audited AGY tool inputs;
+the backpack setup did execute the documented render-service entrypoint. The
+teapot wrote diagnostic math scripts to AGY's own scratch directory outside the
+requested workspace. Those files and commands are retained as a boundary deviation.
+The initial OpenCode parents inspected ambient configuration and exposed unrelated
+tool context. Consequently the public-clone trials should be described as fresh
+task/workspace tests with audited inheritance, not perfectly hermetic experiments.
+
+All final sheets were independently viewed. Native image reads and MCP calls were
+retained, but this campaign did not capture provider request payloads. Prior image
+transport qualification does not replace that missing measurement here. Endpoint
+animation equality proves matching endpoint poses only; it does not prove velocity
+continuity, collision freedom or pleasing motion. Quality assessments above are
+reviewer observations, not owner acceptance.
+
+No unrelated workspace instructions were upgraded. No credentials were copied,
+and no commit, push, publication or release was performed. The final shared renderer
+status reported no listener on port 8000; no process was killed to obtain that state.
+
+## Changes and validation
+
+The repository changes are the trace-counter repair and two focused tests;
+setup-skill handoff guidance (including its two registered copies); shared
+geometry guidance on frame alignment and repeated-part preservation; refinement
+guidance on inventory checks and child-save-before-export; and this report.
+
+Validation completed:
+
+- Focused trace-counter TDD: 27 pass / 2 expected failures, then 29 pass / 0 fail.
+- `check:toolchain`, `check:skills`, `typecheck`, and `lint`: pass.
+- Full offline suite: 3,190 pass, 2 skip, 0 fail across 403 files.
+- Coverage run: same test totals; 95.20% functions and 92.52% lines, ratchet passed.
+- Follow-up skill/setup checks: 6 pass; managed-upgrade checks: 10 pass.
+- Fresh AGY/OpenCode workspaces received matching updated guidance. Managed
+ upgrades refreshed unchanged copies and preserved an owner file. Customized
+ skill copies refused upgrade and remained unchanged.
+- `npm pack --ignore-scripts --dry-run --json` includes all three changed maintained
+ skill resources. It is a package-content check, not an installed-package or
+ published-release qualification.
+
+There were no runtime or render-service source changes, so no new engine bundle
+or render-service regression claim is attached to the evaluation-tool fix.
+The next useful experiment is a fresh, matched repeat of the failed handoffs and
+preservation tasks with these skills, followed by broader pack-specific sampling.
+Do not count this campaign's unchanged outputs as acceptance of the new guidance.
+
+## Evidence map
+
+Paths below are relative to the private evidence root:
+
+| File or directory | Contents |
+| --- | --- |
+| `authorization.json`, `preflight.json`, `candidate.json` | Authorized scope, live route probes, exact candidate and dependency/build identity |
+| `evidence/*/stage-*-input.json` | Exact prompts, arguments, versions, workspace paths and initial instruction hashes |
+| `evidence/*/stage-*-events.jsonl`, `*-stderr.log`, `*-process.json` | Native stream and terminal/process receipts |
+| `evidence/*/native/`, `*-native.json` | AGY native rows/transcripts and OpenCode native session exports, including child episodes |
+| `derived-receipts.json`, `trace-audit.json`, `native-transcript-index.json` | Recount and trace navigation; error candidates require interpretation |
+| `final-artifact-manifest.json` | Final source/GLB/PNG hashes, exact runtime heads/build identities, copied skill hashes, recipes and animation endpoints |
+| `saved-revision-verification.json`, `child-mcp-receipts.json` | Exact source/GLB matches to all eight saved baselines and three saved edits; separate native author MCP counts |
+| `artifact-validation.json` | Independent glTF validation for all 31 retained GLBs |
+| `independent-preservation.json` | Independent baseline/edit structural comparisons |
+| `teapot-junction-check.json` | Exported-body intersection at the spout junction |
+| `skill-delivery-verification.json`, `package-dry-run.json` | Fresh setup, upgrade/customization and package-content evidence |
+| `receipt-tests-*.log`, `full-test.log`, `coverage.log`, `*-followup-tests.log` | Repository validation evidence |
+| `review/index.html` | Local visual review with the eight sheets and four edited results |
+
+The external `run.mjs` and analysis scripts are retained with the evidence. Preserve
+that directory alongside this report; the repository alone does not contain the
+private native traces or generated assets.
diff --git a/scripts/tier2-dogfood.mjs b/scripts/tier2-dogfood.mjs
index 3d397853..92ac0fe2 100644
--- a/scripts/tier2-dogfood.mjs
+++ b/scripts/tier2-dogfood.mjs
@@ -476,13 +476,16 @@ const KILN_MCP_TOOL = new RegExp(
* JSON (`{type:'tool_use', name}` inside message content), Codex JSONL
* (`{type:'mcp_tool_call', server, tool}` and `command_execution` items), Agy stream
* JSON (`{event:'step_update', step_update:{step_index, tool_name}}`, emitted when a
- * tool step starts and again when it ends, so one step index is one call). Anything
+ * tool step starts and again when it ends, so one step index is one call). OpenCode
+ * code-mode metadata records nested calls; count those alongside the execute wrapper,
+ * never infer execution from source or returned text. Agy's generic MCP dispatcher
+ * identifies the server and tool in its step parameters. Anything
* else counts as no tool calls, which the receipt reports as `unknown` rather than
* as `not-exercised`.
*/
export function toolUsageFromEvents(events) {
const calls = {};
- const agySteps = new Set();
+ const agySteps = new Map();
const record = (name) => {
if (typeof name !== 'string' || !name) return;
calls[name] = (calls[name] ?? 0) + 1;
@@ -497,15 +500,21 @@ export function toolUsageFromEvents(events) {
const step = node.step_update;
if (typeof step.tool_name === 'string') {
const key = `${step.conversation_id ?? ''}:${step.step_index}`;
- if (!agySteps.has(key)) {
- agySteps.add(key);
- record(step.tool_name);
- }
+ const previous = agySteps.get(key);
+ agySteps.set(key, {
+ ...previous,
+ ...step,
+ tool_info: step.tool_info ?? previous?.tool_info,
+ });
}
return;
}
if (node.type === 'tool_use') {
record(node.name ?? node.tool ?? node.part?.tool);
+ if (node.part?.tool === 'execute') {
+ const nested = node.part.state?.metadata?.metadata?.toolCalls;
+ if (Array.isArray(nested)) for (const call of nested) record(call?.tool);
+ }
return;
}
if (node.type === 'mcp_tool_call') {
@@ -519,6 +528,16 @@ export function toolUsageFromEvents(events) {
for (const value of Object.values(node)) if (value && typeof value === 'object') visit(value);
};
for (const event of events) visit(event);
+ for (const step of agySteps.values()) {
+ const params = step.tool_info?.parameters;
+ record(
+ step.tool_name === 'call_mcp_tool' &&
+ typeof params?.ServerName === 'string' &&
+ typeof params?.ToolName === 'string'
+ ? `${params.ServerName}__${params.ToolName}`
+ : step.tool_name,
+ );
+ }
const total = Object.values(calls).reduce((sum, count) => sum + count, 0);
const mcpCalls = Object.entries(calls)
.filter(([name]) => KILN_MCP_TOOL.test(name))
diff --git a/scripts/tier2-dogfood.test.mjs b/scripts/tier2-dogfood.test.mjs
index 905422e8..d75d433c 100644
--- a/scripts/tier2-dogfood.test.mjs
+++ b/scripts/tier2-dogfood.test.mjs
@@ -492,6 +492,61 @@ describe('Tier 2 blind dogfood driver', () => {
expect(toolUsageFromEvents([{ type: 'text' }]).workspaceMcp).toBe('unknown');
});
+ test('counts observed OpenCode code-mode calls without interpreting source or tool output', () => {
+ const event = {
+ type: 'tool_use',
+ part: {
+ tool: 'execute',
+ state: {
+ status: 'completed',
+ input: { code: 'tools.kiln_workspace.kiln_save({}); // not observed executing' },
+ output: '{"toolCalls":[{"tool":"kiln_workspace.kiln_save"}]}',
+ metadata: {
+ metadata: {
+ toolCalls: [
+ { tool: 'search', status: 'completed' },
+ { tool: 'kiln_workspace.kiln_discover', status: 'completed' },
+ { tool: 'kiln_workspace.kiln_render', status: 'completed' },
+ ],
+ },
+ },
+ },
+ },
+ };
+ expect(toolUsageFromEvents([event])).toEqual({
+ calls: {
+ execute: 1,
+ search: 1,
+ 'kiln_workspace.kiln_discover': 1,
+ 'kiln_workspace.kiln_render': 1,
+ },
+ total: 4,
+ mcpCalls: 2,
+ workspaceMcp: 'exercised',
+ });
+ delete event.part.state.metadata;
+ expect(toolUsageFromEvents([event]).mcpCalls).toBe(0);
+ });
+
+ test('resolves Agy MCP dispatcher names from the final step update once', () => {
+ const step = (conversation_id, tool_info) => ({
+ event: 'step_update',
+ step_update: { conversation_id, step_index: 12, tool_name: 'call_mcp_tool', tool_info },
+ });
+ const info = {
+ parameters: { ServerName: 'kiln_workspace', ToolName: 'kiln_render', Arguments: {} },
+ };
+ expect(
+ toolUsageFromEvents([step('c1'), step('c1', info), step('c1', info), step('c2', info)]),
+ ).toEqual({
+ calls: { kiln_workspace__kiln_render: 2 },
+ total: 2,
+ mcpCalls: 2,
+ workspaceMcp: 'exercised',
+ });
+ expect(toolUsageFromEvents([step('c1', { output: JSON.stringify(info) })]).mcpCalls).toBe(0);
+ });
+
test('a source and GLB are necessary but remain pending human quality review', () => {
expect(
classifyOutcome({
diff --git a/skills/kiln-author-asset/references/geometry-recipes.md b/skills/kiln-author-asset/references/geometry-recipes.md
index ed852d26..32551883 100644
--- a/skills/kiln-author-asset/references/geometry-recipes.md
+++ b/skills/kiln-author-asset/references/geometry-recipes.md
@@ -62,6 +62,13 @@ winding or missing geometry rather than hiding the defect with double-sided shad
deliberate thin sheets may need it. Check ceiling and trim extents against the actual sloping roof, including
the eaves, rather than its bounding box.
+For a pane or panel fitted into a sloping frame, derive its corners and surface
+normal in the frame's coordinates. A bar's long axis and a sheet's plane may use
+different local axes, so copying a slope angle does not align them. Check all
+four transformed corners against the intended frame plane and opening, then
+inspect an oblique view where an out-of-plane pane is visible. Matching centers
+or bounding boxes alone cannot establish this fit.
+
For attached veins, ribs or seams, discover `recipe:surface-detail-v1`. Its strip
shares the carrier mesh's sampled boundary vertices and rises along local normals.
A shared equation followed by a separately interpolated curve does not establish
@@ -245,6 +252,12 @@ choice belongs to export settings, not asset source.
Put repeated assemblies in ordinary functions with JSDoc parameters and named return values. Pass parent/material explicitly; return the assembly root and attachment markers. Change one parameter at the call site for a variant. Give such a function a name prefix or suffix and use it for every part it creates (`Leg_L`, `Leg_R`); calling it twice with the same names makes duplicate node names, which QA reports as `UNIVERSAL_DUPLICATE_NODE_NAME`. The runnable [reusable-frame recipe](reusable-frame.kiln.js) demonstrates this without imports or hidden dependencies.
+Include the assembly and element index in repeated child names as well, so each
+leaflet, slat or fastener remains addressable during revision comparison. When a
+resize must retain those children, keep their count explicit and redistribute
+their attachment positions; deriving the count from the new length can silently
+delete parts. Check the exported child inventory as well as the changed outline.
+
When components must move or be replaced together, parent them under a real root
at the assembly's placement datum. Name suffixes alone leave independent siblings.
`recipe:editable-assembly-v1` demonstrates hierarchy replication and replacing one
diff --git a/skills/kiln-refine-asset/SKILL.md b/skills/kiln-refine-asset/SKILL.md
index 97088b70..b639d75e 100644
--- a/skills/kiln-refine-asset/SKILL.md
+++ b/skills/kiln-refine-asset/SKILL.md
@@ -86,6 +86,11 @@ mesh contact and final dimensions after reshaping; shared equations or control
points alone do not prove attachment. Discovery offers optional editable-assembly
and surface-detail recipes when these patterns fit.
+When resizing an assembly while retaining its children, compare their before/after
+inventory too. A length-dependent loop can remove leaflets, slats or fasteners even
+when every remaining part stays attached. Preserve the requested count and
+attachment points; see the shared geometry recipes for repeated-part naming and fit.
+
Check the retained parts that connected to the replaced part, including their
other ends and intermediate mounts. Unchanged geometry can become unsupported
after its neighbor moves or disappears. Preserve the requested connection points;
@@ -112,7 +117,8 @@ dimensions separately when they differ from the usable space.
Check the part in context before hiding neighbors. Isolation can reveal a surface but cannot prove its attachment. `viewFidelity.materialFaithful: false` leaves PBR appearance unverified. Animation and cutaway inspection should target the motion or occlusion in question, not add a fixed number of routine images. For CLI motion review, use `node kiln.mjs animation RETURNED_REF --clip CLIP_NAME --phases 0,0.017,0.31,0.68,1 --views motion.png --json`; add `--render gpu` when material fidelity matters. Compare `poseBounds` before and after the edit for requested ground clearance and travel, including protrusions. Sampled bounds do not prove continuous contact; include phases inside a geometric repeat, even when regular samples look identical. The [motion reference](references/revision-and-views.md#motion-and-interiors) covers phase units, cameras and separate frames.
-Save source and GLB from the same final reference:
+After saving the reviewed child revision of a collection asset, export source and
+GLB from that same final reference. These file writes do not create a saved revision:
```sh
node kiln.mjs source RETURNED_REF --out revised-v1.kiln.js
diff --git a/skills/kiln-setup-workspace/SKILL.md b/skills/kiln-setup-workspace/SKILL.md
index 193b3753..d675789f 100644
--- a/skills/kiln-setup-workspace/SKILL.md
+++ b/skills/kiln-setup-workspace/SKILL.md
@@ -58,6 +58,13 @@ A `KILN_RENDER_PORT_URL` whose origin is exactly the shared local socket (`http:
Accept the project and MCP trust prompts, then confirm the server is actually live rather than assuming it from configuration. Call `kiln_discover` on `kiln_workspace` with `{ capabilities: true }`; it returns the runtime, source, export and camera contract and proves the tools resolved. For OpenCode, `opencode debug config` from the workspace is the configuration check: it shows the resolved `kiln_workspace` entry. `opencode mcp list` reports what OpenCode's shared background service loaded, so it can print "No MCP servers configured" in a correctly configured workspace. `node kiln.mjs discover --capabilities --json` from the workspace is the harness-independent proof; it reports the same runtime, collections and source storage as the MCP server. Compare `capabilities.engine.installUrl` with `runtime` in `.kiln/workspace.json`. A server named `kiln` from a global installation is a different thing. Do not substitute it silently; report the setup problem instead. When another agent is to do the authoring, start it as a separate harness process in the workspace directory (for OpenCode, `opencode run --standalone "<brief>"` after changing into the workspace); a subagent of the current session inherits this session's tools and never sees the workspace's MCP server, so it would be left with the CLI alone.
+When the task includes that separate author's result, retain its process/session
+identifier and wait for its terminal result. A launch message, session ID or empty
+log proves no authoring work. Use the harness's noninteractive mode with closed
+stdin for an unattended child, preserve stdout/stderr, and check the requested
+files and saved revision before reporting completion. If it stops early, report
+the incomplete handoff; confirm it has stopped before launching a replacement.
+
For modeling, `kiln_discover({})` supplies a compact orientation and starting signatures. Use natural-language `query` for related operations, assemblies, and optional recipes; use exact `ids` for complete contracts. Discovery needs no separate search model or asset-category selection. The local equivalent is `node kiln.mjs discover --query "curved hollow tube"`, followed by `node kiln.mjs discover --id RETURNED_ID`.
Confirm the installed skills are readable at `skills/` in the workspace, and read the relevant one from there rather than a global copy. Whether the harness also registers them natively depends on the harness.