3D engine that AI agents drive
Bench: compare the official Blender MCP server with 67js and three.js (part of #31)
Nguyen Xuan An9569fcea0d7f
diff --git a/bench/agent-modeling/README.md b/bench/agent-modeling/README.mdindex 1cb0db80..4d240a05 100644--- a/bench/agent-modeling/README.md+++ b/bench/agent-modeling/README.md@@ -397,7 +397,7 @@ Not run:## Agent friendliness: 67js MCP against three.js by code (2026-11-07, issue #31)-The three.js half of issue #31 and the 67js side of the same table. The Blender MCP half needs a VM and is not run. Model `sonnet`, headless `claude -p` on the subscription, 6 suite tasks (`crate`, `bolt`, `window`, `table`, `fence`, `bracket`), 1 run each per system, so 12 records in `results/2026-11-07-threejs-vs-67js.json`. Tokens and tool calls are reported; there is no API dollar figure.+The three.js half of issue #31 and the 67js side of the same table. The Blender MCP half is in the next section. Model `sonnet`, headless `claude -p` on the subscription, 6 suite tasks (`crate`, `bolt`, `window`, `table`, `fence`, `bracket`), 1 run each per system, so 12 records in `results/2026-11-07-threejs-vs-67js.json`. Tokens and tool calls are reported; there is no API dollar figure.- 67js MCP: `run.mjs`, the task prompt behind the usual MCP preamble.- three.js: `threejs.mjs`, the same task text behind a different preamble. The agent gets an empty folder with `three` installed from npm, Bash, Read, Write, Edit, Glob and Grep, no web and no MCP. It writes `build.mjs`, which builds the object, writes `geometry.json` (world-space triangles per mesh) and renders `scene.png` with headless Chrome on SwiftShader.@@ -407,6 +407,7 @@ The three.js half of issue #31 and the 67js side of the same table. The Blender| --- | --- | --- | --- | --- | --- || 67js MCP | 6/6 | 3.2 | 375k | 0.932 | 24 || three.js by code | 5/6 | 8.8 | 269k | 0.926 | 359 |+| Blender MCP (official server) | 6/6 | 3.8 | 177k | 0.926 | 517 || Task | System | Pass | Tool calls (errors) | Look | Failure || --- | --- | --- | --- | --- | --- |@@ -428,3 +429,21 @@ The three.js half of issue #31 and the 67js side of the same table. The Blender- 67js failures: none in the checks. One harness fault: the `bracket` look images failed because the look page could not fetch the boolean wasm (`both async and sync fetching of the wasm failed`), which threw away the whole look record, silhouette included. The silhouette was scored again afterwards from the exported recipe, and the record carries `imagesError`. `scoreLook` should score the silhouette before it renders images.- A first attempt of the 67js run failed on every task because the checkout had no `build/mcp` folder (`npm run build:lib` creates it); those results were discarded. The first three.js run stopped after `bolt` when its process disappeared, so `window`, `table`, `fence` and `bracket` were run again as a second run and joined with `compare-threejs.mjs`.- Limits: 1 run per cell, one model, tasks the 67js suite was built around (they favour 67js's checks, not three.js). The `window` three.js failure label came from an earlier wording of the same check.++### Blender MCP half (2026-11-07, issue #31)++The official Blender MCP server (Blender Lab, v1.0.3, add-on and server) driving Blender 5.1.2, same model, same six tasks, 1 run each, in `results/2026-11-07-blender-mcp.json`. The server runs model-written Python with no guards, so `blender.mjs` never runs it on the benchmark machine: Blender (headless, `--command blender_mcp`) and the HTTP MCP server run inside a throwaway Docker container (`blender-container/`, x86_64 under Rosetta on an Apple silicon laptop, one scratch folder mounted, a fresh container per task). `claude -p` reaches the server through an SSH port forward and gets only the MCP tools: no Bash and no file tools on its own machine. The prompt asks the agent to leave `geometry.json` (world-space triangles, Y up) and `scene.png` in the container's `/work`; the script copies them back and grades them exactly like the three.js run.++| Task | Pass | Tool calls (errors) | Tokens in / out | Look | Wall seconds |+| --- | --- | --- | --- | --- | --- |+| crate | yes, 8/8 | 4 (1) | 179k / 3k | 1.000 | 590 |+| bolt | yes, 7/7 | 4 (1) | 179k / 3k | 0.964 | 609 |+| window | yes, 5/5 | 4 (1) | 181k / 4k | 1.000 | 611 |+| table | yes, 6/6 | 5 (1) | 207k / 3k | 0.592 | 625 |+| fence | yes, 6/6 | 2 (0) | 116k / 3k | 1.000 | 38 |+| bracket | yes, 6/6 | 4 (1) | 182k / 4k | 1.000 | 626 |++- Measured: no failures. Tool calls exclude the `ToolSearch` call that loads the one tool the agent uses (`execute_blender_code`), as in the other runs. Look matches the other two systems (`table` 0.592 again, so it is the reference).+- Wall time is not a fair figure. Measured in the transcripts: `crate` hit Claude Code's 300 s no-response abort on its render call while the files were still written; the other four runs with an error and about 600 s show the same wall time. Inferred: the render is a CPU Cycles render of an x86 Blender under emulation, which a native Blender would finish much faster; `fence`, whose render stayed under the limit, took 38 s. Treat the 517 s mean as environment cost, not as Blender MCP latency.+- Every run but `fence` spent one call on a failed first attempt (a Python traceback or the timeout) and then repaired it. The tool takes raw `bpy` code, so the agent writes the mesh code, the Z-up to Y-up conversion and the export itself, as in the three.js run; unlike three.js it needs no browser for the picture.+- Limits: 1 run per cell, one model, tasks built around 67js. The server's own screenshot and render tools were never used; the agents rendered from their own code. The container has outbound network access (the model's code could reach the internet); only the scratch folder is shared with the host.diff --git a/bench/agent-modeling/blender-container/.gitattributes b/bench/agent-modeling/blender-container/.gitattributesnew file mode 100644index 00000000..fcadb2cf--- /dev/null+++ b/bench/agent-modeling/blender-container/.gitattributes@@ -0,0 +1 @@+* text eol=lfdiff --git a/bench/agent-modeling/blender-container/Dockerfile b/bench/agent-modeling/blender-container/Dockerfilenew file mode 100644index 00000000..b985a955--- /dev/null+++ b/bench/agent-modeling/blender-container/Dockerfile@@ -0,0 +1,12 @@+FROM --platform=linux/amd64 debian:bookworm-slim+RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl xz-utils git \+ libegl1 libgl1 libgl1-mesa-dri libxi6 libxkbcommon0 libxxf86vm1 libxfixes3 libxrender1 libsm6 libx11-6 libxext6 libgomp1 libwayland-client0 \+ && rm -rf /var/lib/apt/lists/*+RUN curl -fsSL https://download.blender.org/release/Blender5.1/blender-5.1.2-linux-x64.tar.xz | tar -xJ -C /opt && mv /opt/blender-5.1.2-linux-x64 /opt/blender+RUN curl -fsSL https://astral.sh/uv/install.sh | UV_INSTALL_DIR=/usr/local/bin sh+RUN git clone -q --branch v1.0.3 --depth 1 https://projects.blender.org/lab/blender_mcp.git /opt/blender_mcp \+ && curl -fsSL -o /opt/mcp-1.0.3.zip "https://projects.blender.org/lab/blender_mcp/releases/download/v1.0.3/mcp-1.0.3.zip"+ENV BLENDER_PATH=/opt/blender/blender+RUN /opt/blender/blender -b --command extension install-file --enable -r user_default /opt/mcp-1.0.3.zip 2>&1 | tail -5; \+ cd /opt/blender_mcp/mcp && uv sync 2>&1 | tail -2+WORKDIR /workdiff --git a/bench/agent-modeling/blender-container/entry.sh b/bench/agent-modeling/blender-container/entry.shnew file mode 100644index 00000000..645c22b6--- /dev/null+++ b/bench/agent-modeling/blender-container/entry.sh@@ -0,0 +1,5 @@+#!/bin/sh+# Blender add-on server in background mode (port 9876, container-local) plus the official MCP server over HTTP (9191).+/opt/blender/blender --online-mode -b --command blender_mcp --host 127.0.0.1 --port 9876 > /work/blender-addon.log 2>&1 &+sleep 5+cd /opt/blender_mcp/mcp && exec uv run blender-mcp --transport http --host 0.0.0.0 --port 9191diff --git a/bench/agent-modeling/blender.mjs b/bench/agent-modeling/blender.mjsnew file mode 100644index 00000000..d7932d76--- /dev/null+++ b/bench/agent-modeling/blender.mjs@@ -0,0 +1,202 @@+// The Blender MCP side of the agent friendliness comparison (issue #31): the same task prompts and model as run.mjs and+// threejs.mjs, but the agent only has the official Blender MCP server (https://www.blender.org/lab/mcp-server/).+//+// The official server runs model-written Python without guards, so Blender and the server run ONLY inside a disposable+// Docker container on a separate machine (see blender-container/). This script drives it over SSH and an SSH port forward:+//+// 1. one-time on the remote machine: build the image from blender-container/ (`docker build --platform linux/amd64`)+// 2. open the forward: ssh -N -L 127.0.0.1:9191:127.0.0.1:9191 <host>+// 3. node bench/agent-modeling/blender.mjs --ssh-host <host> [--claude <executable>] [--model sonnet] [--tasks crate,bolt]+//+// Per task the script starts a fresh container (a fresh Blender), mounts only a scratch folder, lets `claude -p` use the+// MCP server and nothing else (no Bash, no file tools on this machine), then copies geometry.json and scene.png back and+// grades them like threejs.mjs does: the task checks on the world-space triangles plus the silhouette look score.+import { execFileSync, spawn } from 'node:child_process';+import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';+import { tmpdir } from 'node:os';+import { join, resolve } from 'node:path';+import { fileURLToPath } from 'node:url';+import { parseArgs } from 'node:util';+import { removeLocalPaths, totalsByModel, totalsOf } from './aggregate.mjs';+import { checkPng } from './png-check.mjs';+import { gradeGeometries, readTranscript, scoreSilhouetteOfGeometries } from './threejs.mjs';+import { taskById } from './tasks.mjs';++const repository = resolve(import.meta.dirname, '../..');+const MCP_SERVER_NAME = 'blender';+const SERVER_URL = 'http://127.0.0.1:9191/';+const BUILT_IN_TOOLS_DENIED = ['Bash', 'Read', 'Write', 'Edit', 'Glob', 'Grep', 'WebFetch', 'WebSearch', 'Agent', 'Task', 'NotebookEdit'];+const DOCKER = 'export PATH=/opt/homebrew/bin:$PATH;';++export const BLENDER_PREAMBLE = `You are modeling in Blender through the Blender MCP server. Blender 5.1 runs headless in a container with no GPU and no+internet; the folder /work inside it is the only place files persist. Build the object described below as a closed,+watertight solid, in metres, Y up as in glTF (Blender itself is Z up, so convert when you export). Rest the object on the+ground (lowest point at Y = 0), centred on X = 0 and Z = 0.+Leave two files in /work and then reply with one line saying so:+1. geometry.json: a JSON array with one entry { "positions": [...], "indices": [...] } per mesh, with every vertex in world+ space (apply each object's world matrix and any modifiers, then the Z-up to Y-up change) and indices as a flat triangle list.+2. scene.png: a render of the scene, 512 by 512 pixels, any render engine that works headless (Cycles on the CPU works).`;++export function parseBlenderArguments(argv) {+ return parseArgs({+ args: argv,+ options: {+ 'ssh-host': { type: 'string' },+ image: { type: 'string', default: 'blender67' },+ claude: { type: 'string', default: process.env.CLAUDE_EXECUTABLE ?? 'claude' },+ model: { type: 'string', default: 'sonnet' },+ tasks: { type: 'string', default: 'crate,bolt,window,table,fence,bracket' },+ runs: { type: 'string', default: '1' },+ budget: { type: 'string', default: '3' },+ timeout: { type: 'string', default: '1200' },+ label: { type: 'string', default: 'blender-mcp' },+ },+ }).values;+}++/** The scratch folder and the container are both removed by the script that created them. */+function remote(options, command) {+ return execFileSync('ssh', [options['ssh-host'], `${DOCKER} ${command}`], { encoding: 'utf8', timeout: 300_000 });+}++async function callServer(code) {+ const response = await fetch(SERVER_URL, {+ method: 'POST',+ headers: { 'content-type': 'application/json', accept: 'application/json, text/event-stream' },+ body: JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'tools/call', params: { name: 'execute_blender_code', arguments: { code } } }),+ signal: AbortSignal.timeout(20_000),+ });+ return (await response.text()).includes('"status\\": \\"ok\\"');+}++async function waitForBlender() {+ for (let attempt = 0; attempt < 60; attempt++) {+ try {+ if (await callServer('result = {}')) return;+ } catch {+ // The container is still starting.+ }+ await new Promise(resolvePromise => setTimeout(resolvePromise, 2000));+ }+ throw new Error('the Blender MCP server never answered');+}++function runAgent(options, prompt, workingDirectory, configPath) {+ return new Promise(resolvePromise => {+ const started = performance.now();+ const child = spawn(options.claude, [+ '-p', prompt, '--model', options.model, '--output-format', 'stream-json', '--verbose',+ '--mcp-config', configPath, '--strict-mcp-config',+ '--allowedTools', `mcp__${MCP_SERVER_NAME}`,+ '--disallowedTools', ...BUILT_IN_TOOLS_DENIED,+ '--permission-mode', 'dontAsk', '--setting-sources', 'project', '--no-session-persistence', '--max-budget-usd', options.budget,+ ], { cwd: workingDirectory, windowsHide: true, stdio: ['ignore', 'pipe', 'pipe'] });+ let output = '';+ child.stdout.on('data', chunk => { output += chunk; });+ child.stderr.on('data', () => {});+ const timer = setTimeout(() => child.kill(), Number(options.timeout) * 1000);+ child.on('close', code => {+ clearTimeout(timer);+ resolvePromise({ output, code, wallMs: Math.round(performance.now() - started) });+ });+ });+}++if (process.argv[1] === fileURLToPath(import.meta.url)) {+ const options = parseBlenderArguments(process.argv.slice(2));+ if (!options['ssh-host']) throw new Error('--ssh-host is required: the machine that runs the disposable container');+ const runName = `${new Date().toISOString().slice(0, 16).replace(/[:T]/g, '-')}-${options.model}-${options.label}`;+ const rawDirectory = join(repository, 'artifacts', 'agent-modeling', runName);+ const resultsDirectory = join(repository, 'bench', 'agent-modeling', 'results');+ await mkdir(rawDirectory, { recursive: true });+ await mkdir(resultsDirectory, { recursive: true });+ execFileSync('scp', ['-q', join(import.meta.dirname, 'blender-container', 'entry.sh'), `${options['ssh-host']}:bench-scratch/entry.sh`]);+ const blenderVersion = remote(options, `docker run --rm --platform linux/amd64 ${options.image} /opt/blender/blender --version | head -1`).trim();++ const records = [];+ const summarize = () => ({+ run: runName,+ system: 'Blender MCP (official server in a container)',+ blenderVersion,+ model: records.find(record => record.model)?.model ?? options.model,+ date: new Date().toISOString(),+ runsPerTask: Number(options.runs),+ passed: records.filter(record => record.pass).length,+ tasks: records.length,+ toolCalls: records.reduce((sum, record) => sum + record.toolCalls, 0),+ byModel: totalsByModel(records),+ records,+ });++ const workingDirectory = await mkdtemp(join(tmpdir(), '67js-blender-agent-'));+ const configPath = join(workingDirectory, 'mcp.json');+ await writeFile(configPath, JSON.stringify({ mcpServers: { [MCP_SERVER_NAME]: { type: 'http', url: SERVER_URL } } }));+ try {+ for (let attempt = 1; attempt <= Number(options.runs); attempt++) {+ for (const task of options.tasks.split(',').map(taskById)) {+ const container = `blender67-${task.id}-${attempt}`;+ const scratch = `bench-scratch/${runName}/${task.id}-${attempt}`;+ const localCopy = join(rawDirectory, `${task.id}-${attempt}`);+ await mkdir(localCopy, { recursive: true });+ let run = { output: '', code: null, wallMs: 0 };+ let startError = null;+ try {+ remote(options, `mkdir -p "$HOME/${scratch}" && docker run -d --name ${container} --platform linux/amd64 --memory 6g -p 127.0.0.1:9191:9191 -v "$HOME/${scratch}:/work" -v "$HOME/bench-scratch/entry.sh:/entry.sh:ro" ${options.image} sh /entry.sh`);+ await waitForBlender();+ run = await runAgent(options, `${BLENDER_PREAMBLE}\n\n${task.prompt.replace(/\n/g, ' ')}`, workingDirectory, configPath);+ } catch (error) {+ startError = error.message.split('\n')[0];+ } finally {+ try {+ execFileSync('scp', ['-q', `${options['ssh-host']}:${scratch}/geometry.json`, `${options['ssh-host']}:${scratch}/scene.png`, localCopy], { stdio: 'ignore' });+ } catch {+ // A file the agent never wrote is a failed check below.+ }+ remote(options, `docker rm -f ${container} >/dev/null; rm -rf "$HOME/${scratch}"`);+ }+ const transcript = readTranscript(run.output);+ await writeFile(join(rawDirectory, `${task.id}-${attempt}.jsonl`), run.output);+ let geometries = null;+ let grade = { pass: false, passed: 0, total: task.checks.length, results: [{ kind: 'geometry', pass: false, detail: startError ?? 'no geometry.json' }] };+ try {+ geometries = JSON.parse(await readFile(join(localCopy, 'geometry.json'), 'utf8'));+ grade = gradeGeometries(task, geometries);+ } catch (error) {+ grade.results[0].detail = startError ?? `geometry.json unreadable: ${error.message.split('\n')[0]}`;+ }+ const png = await checkPng(join(localCopy, 'scene.png'));+ const silhouette = geometries ? await scoreSilhouetteOfGeometries(task.id, geometries).catch(error => ({ error: error.message.split('\n')[0] })) : null;+ const usage = transcript.result?.usage ?? {};+ const failed = grade.results.filter(result => !result.pass).map(result => `${result.label ?? result.kind}: ${result.detail}`);+ if (!png.pass) failed.push(`scene.png: ${png.detail}`);+ const record = {+ task: task.id,+ attempt,+ pass: grade.pass && png.pass,+ passed: grade.passed,+ total: grade.total,+ pngPass: png.pass,+ failed: failed.map(removeLocalPaths),+ model: transcript.model,+ toolCalls: transcript.toolCalls,+ toolErrors: transcript.toolErrors,+ toolsByName: transcript.toolsByName,+ look: silhouette ? { silhouette } : null,+ turns: transcript.result?.num_turns ?? null,+ costUsd: null,+ inputTokens: (usage.input_tokens ?? 0) + (usage.cache_read_input_tokens ?? 0) + (usage.cache_creation_input_tokens ?? 0),+ outputTokens: usage.output_tokens ?? 0,+ wallSeconds: Math.round(run.wallMs / 1000),+ exitCode: run.code,+ stopped: transcript.result?.subtype ?? 'no result',+ };+ records.push(record);+ console.log(`${record.task.padEnd(10)} #${attempt} ${record.pass ? 'PASS' : 'FAIL'} ${record.passed}/${record.total} png ${png.pass} calls ${record.toolCalls} (errors ${record.toolErrors}) look ${silhouette?.mean ?? '-'} ${record.wallSeconds}s${failed.length ? ` ${failed.join(' | ')}` : ''}`);+ await writeFile(join(resultsDirectory, `${runName}.json`), `${JSON.stringify(summarize(), null, 2)}\n`);+ }+ }+ } finally {+ await rm(workingDirectory, { recursive: true, force: true }).catch(() => {});+ }+ console.log(`${records.filter(record => record.pass).length}/${records.length} passed, ${totalsOf(records).toolCalls} tool calls. Summary: bench/agent-modeling/results/${runName}.json`);+}diff --git a/bench/agent-modeling/results/2026-11-07-blender-mcp.json b/bench/agent-modeling/results/2026-11-07-blender-mcp.jsonnew file mode 100644index 00000000..dfceeed4--- /dev/null+++ b/bench/agent-modeling/results/2026-11-07-blender-mcp.json@@ -0,0 +1,279 @@+{+ "run": "2026-11-07-blender-mcp",+ "system": "Blender MCP (official server in a container)",+ "blenderVersion": "Blender 5.1.2",+ "model": "claude-sonnet-5-5",+ "date": "2026-11-07T18:04:20.919Z",+ "runsPerTask": 1,+ "passed": 6,+ "tasks": 6,+ "toolCalls": 23,+ "byModel": {+ "claude-sonnet-5-5": {+ "runs": 6,+ "passed": 6,+ "successRate": 1,+ "toolCalls": 23,+ "toolErrors": 5,+ "inputTokens": 1043319,+ "outputTokens": 18972,+ "costUsd": 0,+ "tasks": {+ "crate": {+ "runs": 1,+ "passed": 1,+ "successRate": 1,+ "toolCalls": 4,+ "toolErrors": 1,+ "inputTokens": 178557,+ "outputTokens": 3026,+ "costUsd": 0+ },+ "bolt": {+ "runs": 1,+ "passed": 1,+ "successRate": 1,+ "toolCalls": 4,+ "toolErrors": 1,+ "inputTokens": 178812,+ "outputTokens": 3081,+ "costUsd": 0+ },+ "window": {+ "runs": 1,+ "passed": 1,+ "successRate": 1,+ "toolCalls": 4,+ "toolErrors": 1,+ "inputTokens": 181015,+ "outputTokens": 3619,+ "costUsd": 0+ },+ "table": {+ "runs": 1,+ "passed": 1,+ "successRate": 1,+ "toolCalls": 5,+ "toolErrors": 1,+ "inputTokens": 206569,+ "outputTokens": 2655,+ "costUsd": 0+ },+ "fence": {+ "runs": 1,+ "passed": 1,+ "successRate": 1,+ "toolCalls": 2,+ "toolErrors": 0,+ "inputTokens": 116074,+ "outputTokens": 2525,+ "costUsd": 0+ },+ "bracket": {+ "runs": 1,+ "passed": 1,+ "successRate": 1,+ "toolCalls": 4,+ "toolErrors": 1,+ "inputTokens": 182292,+ "outputTokens": 4066,+ "costUsd": 0+ }+ }+ }+ },+ "records": [+ {+ "task": "crate",+ "attempt": 1,+ "pass": true,+ "passed": 8,+ "total": 8,+ "pngPass": true,+ "failed": [],+ "model": "claude-sonnet-5-5",+ "toolCalls": 4,+ "toolErrors": 1,+ "toolsByName": {+ "mcp__blender__execute_blender_code": 4+ },+ "look": {+ "silhouette": {+ "views": {+ "front": 1,+ "side": 1,+ "three-quarter": 1+ },+ "mean": 1+ }+ },+ "turns": 6,+ "costUsd": null,+ "inputTokens": 178557,+ "outputTokens": 3026,+ "wallSeconds": 590,+ "exitCode": 0,+ "stopped": "success"+ },+ {+ "task": "bolt",+ "attempt": 1,+ "pass": true,+ "passed": 7,+ "total": 7,+ "pngPass": true,+ "failed": [],+ "model": "claude-sonnet-5-5",+ "toolCalls": 4,+ "toolErrors": 1,+ "toolsByName": {+ "mcp__blender__execute_blender_code": 4+ },+ "look": {+ "silhouette": {+ "views": {+ "front": 0.9621,+ "side": 0.9621,+ "three-quarter": 0.9666+ },+ "mean": 0.9636+ }+ },+ "turns": 6,+ "costUsd": null,+ "inputTokens": 178812,+ "outputTokens": 3081,+ "wallSeconds": 609,+ "exitCode": 0,+ "stopped": "success"+ },+ {+ "task": "window",+ "attempt": 1,+ "pass": true,+ "passed": 5,+ "total": 5,+ "pngPass": true,+ "failed": [],+ "model": "claude-sonnet-5-5",+ "toolCalls": 4,+ "toolErrors": 1,+ "toolsByName": {+ "mcp__blender__execute_blender_code": 4+ },+ "look": {+ "silhouette": {+ "views": {+ "front": 1,+ "side": 1,+ "three-quarter": 1+ },+ "mean": 1+ }+ },+ "turns": 6,+ "costUsd": null,+ "inputTokens": 181015,+ "outputTokens": 3619,+ "wallSeconds": 611,+ "exitCode": 0,+ "stopped": "success"+ },+ {+ "task": "table",+ "attempt": 1,+ "pass": true,+ "passed": 6,+ "total": 6,+ "pngPass": true,+ "failed": [],+ "model": "claude-sonnet-5-5",+ "toolCalls": 5,+ "toolErrors": 1,+ "toolsByName": {+ "mcp__blender__execute_blender_code": 5+ },+ "look": {+ "silhouette": {+ "views": {+ "front": 1,+ "side": 0.1684,+ "three-quarter": 0.6073+ },+ "mean": 0.5919+ }+ },+ "turns": 7,+ "costUsd": null,+ "inputTokens": 206569,+ "outputTokens": 2655,+ "wallSeconds": 625,+ "exitCode": 0,+ "stopped": "success"+ },+ {+ "task": "fence",+ "attempt": 1,+ "pass": true,+ "passed": 6,+ "total": 6,+ "pngPass": true,+ "failed": [],+ "model": "claude-sonnet-5-5",+ "toolCalls": 2,+ "toolErrors": 0,+ "toolsByName": {+ "mcp__blender__execute_blender_code": 2+ },+ "look": {+ "silhouette": {+ "views": {+ "front": 1,+ "side": 1,+ "three-quarter": 1+ },+ "mean": 1+ }+ },+ "turns": 4,+ "costUsd": null,+ "inputTokens": 116074,+ "outputTokens": 2525,+ "wallSeconds": 38,+ "exitCode": 0,+ "stopped": "success"+ },+ {+ "task": "bracket",+ "attempt": 1,+ "pass": true,+ "passed": 6,+ "total": 6,+ "pngPass": true,+ "failed": [],+ "model": "claude-sonnet-5-5",+ "toolCalls": 4,+ "toolErrors": 1,+ "toolsByName": {+ "mcp__blender__execute_blender_code": 4+ },+ "look": {+ "silhouette": {+ "views": {+ "front": 1,+ "side": 1,+ "three-quarter": 1+ },+ "mean": 1+ }+ },+ "turns": 6,+ "costUsd": null,+ "inputTokens": 182292,+ "outputTokens": 4066,+ "wallSeconds": 626,+ "exitCode": 0,+ "stopped": "success"+ }+ ]+}diff --git a/bench/agent-modeling/threejs.mjs b/bench/agent-modeling/threejs.mjsindex 83fed2c3..f7586e58 100644--- a/bench/agent-modeling/threejs.mjs+++ b/bench/agent-modeling/threejs.mjs@@ -134,7 +134,7 @@ export async function scoreSilhouetteOfGeometries(taskId, rawGeometries) {return { views: overlap, mean: Number((values.reduce((sum, value) => sum + value, 0) / values.length).toFixed(4)) };}-function readTranscript(output) {+export function readTranscript(output) {const summary = { model: null, toolCalls: 0, toolErrors: 0, toolsByName: {}, result: null };for (const line of output.split('\n')) {if (!line.trim().startsWith('{')) continue;diff --git a/changes/31.md b/changes/31.mdindex 5e9be688..b5034902 100644--- a/changes/31.md+++ b/changes/31.md@@ -1 +1,2 @@- Compare agent friendliness of 67js MCP against three.js written as code on six suite tasks with the same model and the same checks and look score (issue #31, three.js half; the Blender MCP half still needs a VM): `bench/agent-modeling/threejs.mjs` runs the three.js side and `results/2026-11-07-threejs-vs-67js.json` holds both.+- Add the Blender MCP half of issue #31: `bench/agent-modeling/blender.mjs` runs Sonnet against the official Blender MCP server inside a disposable container (`blender-container/`) on the same six tasks and the same grading; 6 of 6 pass, 3.8 tool calls, look 0.926 (`results/2026-11-07-blender-mcp.json`), so the issue now has a row for 67js, three.js and Blender.