Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Bench: compare the official Blender MCP server with 67js and three.js (part of #31)

Nguyen Xuan An9569fcea0d7f
diff --git a/bench/agent-modeling/README.md b/bench/agent-modeling/README.md
index 1cb0db80..4d240a05 100644
--- a/bench/agent-modeling/README.md
+++ b/bench/agent-modeling/README.md
@@ -397,7 +397,7 @@ Not run:
## Agent friendliness: 67js MCP against three.js by code (2026-11-07, issue #31)
-The three.js half of issue #31 and the 67js side of the same table. The Blender MCP half needs a VM and is not run. Model `sonnet`, headless `claude -p` on the subscription, 6 suite tasks (`crate`, `bolt`, `window`, `table`, `fence`, `bracket`), 1 run each per system, so 12 records in `results/2026-11-07-threejs-vs-67js.json`. Tokens and tool calls are reported; there is no API dollar figure.
+The three.js half of issue #31 and the 67js side of the same table. The Blender MCP half is in the next section. Model `sonnet`, headless `claude -p` on the subscription, 6 suite tasks (`crate`, `bolt`, `window`, `table`, `fence`, `bracket`), 1 run each per system, so 12 records in `results/2026-11-07-threejs-vs-67js.json`. Tokens and tool calls are reported; there is no API dollar figure.
- 67js MCP: `run.mjs`, the task prompt behind the usual MCP preamble.
- three.js: `threejs.mjs`, the same task text behind a different preamble. The agent gets an empty folder with `three` installed from npm, Bash, Read, Write, Edit, Glob and Grep, no web and no MCP. It writes `build.mjs`, which builds the object, writes `geometry.json` (world-space triangles per mesh) and renders `scene.png` with headless Chrome on SwiftShader.
@@ -407,6 +407,7 @@ The three.js half of issue #31 and the 67js side of the same table. The Blender
| --- | --- | --- | --- | --- | --- |
| 67js MCP | 6/6 | 3.2 | 375k | 0.932 | 24 |
| three.js by code | 5/6 | 8.8 | 269k | 0.926 | 359 |
+| Blender MCP (official server) | 6/6 | 3.8 | 177k | 0.926 | 517 |
| Task | System | Pass | Tool calls (errors) | Look | Failure |
| --- | --- | --- | --- | --- | --- |
@@ -428,3 +429,21 @@ The three.js half of issue #31 and the 67js side of the same table. The Blender
- 67js failures: none in the checks. One harness fault: the `bracket` look images failed because the look page could not fetch the boolean wasm (`both async and sync fetching of the wasm failed`), which threw away the whole look record, silhouette included. The silhouette was scored again afterwards from the exported recipe, and the record carries `imagesError`. `scoreLook` should score the silhouette before it renders images.
- A first attempt of the 67js run failed on every task because the checkout had no `build/mcp` folder (`npm run build:lib` creates it); those results were discarded. The first three.js run stopped after `bolt` when its process disappeared, so `window`, `table`, `fence` and `bracket` were run again as a second run and joined with `compare-threejs.mjs`.
- Limits: 1 run per cell, one model, tasks the 67js suite was built around (they favour 67js's checks, not three.js). The `window` three.js failure label came from an earlier wording of the same check.
+
+### Blender MCP half (2026-11-07, issue #31)
+
+The official Blender MCP server (Blender Lab, v1.0.3, add-on and server) driving Blender 5.1.2, same model, same six tasks, 1 run each, in `results/2026-11-07-blender-mcp.json`. The server runs model-written Python with no guards, so `blender.mjs` never runs it on the benchmark machine: Blender (headless, `--command blender_mcp`) and the HTTP MCP server run inside a throwaway Docker container (`blender-container/`, x86_64 under Rosetta on an Apple silicon laptop, one scratch folder mounted, a fresh container per task). `claude -p` reaches the server through an SSH port forward and gets only the MCP tools: no Bash and no file tools on its own machine. The prompt asks the agent to leave `geometry.json` (world-space triangles, Y up) and `scene.png` in the container's `/work`; the script copies them back and grades them exactly like the three.js run.
+
+| Task | Pass | Tool calls (errors) | Tokens in / out | Look | Wall seconds |
+| --- | --- | --- | --- | --- | --- |
+| crate | yes, 8/8 | 4 (1) | 179k / 3k | 1.000 | 590 |
+| bolt | yes, 7/7 | 4 (1) | 179k / 3k | 0.964 | 609 |
+| window | yes, 5/5 | 4 (1) | 181k / 4k | 1.000 | 611 |
+| table | yes, 6/6 | 5 (1) | 207k / 3k | 0.592 | 625 |
+| fence | yes, 6/6 | 2 (0) | 116k / 3k | 1.000 | 38 |
+| bracket | yes, 6/6 | 4 (1) | 182k / 4k | 1.000 | 626 |
+
+- Measured: no failures. Tool calls exclude the `ToolSearch` call that loads the one tool the agent uses (`execute_blender_code`), as in the other runs. Look matches the other two systems (`table` 0.592 again, so it is the reference).
+- Wall time is not a fair figure. Measured in the transcripts: `crate` hit Claude Code's 300 s no-response abort on its render call while the files were still written; the other four runs with an error and about 600 s show the same wall time. Inferred: the render is a CPU Cycles render of an x86 Blender under emulation, which a native Blender would finish much faster; `fence`, whose render stayed under the limit, took 38 s. Treat the 517 s mean as environment cost, not as Blender MCP latency.
+- Every run but `fence` spent one call on a failed first attempt (a Python traceback or the timeout) and then repaired it. The tool takes raw `bpy` code, so the agent writes the mesh code, the Z-up to Y-up conversion and the export itself, as in the three.js run; unlike three.js it needs no browser for the picture.
+- Limits: 1 run per cell, one model, tasks built around 67js. The server's own screenshot and render tools were never used; the agents rendered from their own code. The container has outbound network access (the model's code could reach the internet); only the scratch folder is shared with the host.
diff --git a/bench/agent-modeling/blender-container/.gitattributes b/bench/agent-modeling/blender-container/.gitattributes
new file mode 100644
index 00000000..fcadb2cf
--- /dev/null
+++ b/bench/agent-modeling/blender-container/.gitattributes
@@ -0,0 +1 @@
+* text eol=lf
diff --git a/bench/agent-modeling/blender-container/Dockerfile b/bench/agent-modeling/blender-container/Dockerfile
new file mode 100644
index 00000000..b985a955
--- /dev/null
+++ b/bench/agent-modeling/blender-container/Dockerfile
@@ -0,0 +1,12 @@
+FROM --platform=linux/amd64 debian:bookworm-slim
+RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl xz-utils git \
+ libegl1 libgl1 libgl1-mesa-dri libxi6 libxkbcommon0 libxxf86vm1 libxfixes3 libxrender1 libsm6 libx11-6 libxext6 libgomp1 libwayland-client0 \
+ && rm -rf /var/lib/apt/lists/*
+RUN curl -fsSL https://download.blender.org/release/Blender5.1/blender-5.1.2-linux-x64.tar.xz | tar -xJ -C /opt && mv /opt/blender-5.1.2-linux-x64 /opt/blender
+RUN curl -fsSL https://astral.sh/uv/install.sh | UV_INSTALL_DIR=/usr/local/bin sh
+RUN git clone -q --branch v1.0.3 --depth 1 https://projects.blender.org/lab/blender_mcp.git /opt/blender_mcp \
+ && curl -fsSL -o /opt/mcp-1.0.3.zip "https://projects.blender.org/lab/blender_mcp/releases/download/v1.0.3/mcp-1.0.3.zip"
+ENV BLENDER_PATH=/opt/blender/blender
+RUN /opt/blender/blender -b --command extension install-file --enable -r user_default /opt/mcp-1.0.3.zip 2>&1 | tail -5; \
+ cd /opt/blender_mcp/mcp && uv sync 2>&1 | tail -2
+WORKDIR /work
diff --git a/bench/agent-modeling/blender-container/entry.sh b/bench/agent-modeling/blender-container/entry.sh
new file mode 100644
index 00000000..645c22b6
--- /dev/null
+++ b/bench/agent-modeling/blender-container/entry.sh
@@ -0,0 +1,5 @@
+#!/bin/sh
+# Blender add-on server in background mode (port 9876, container-local) plus the official MCP server over HTTP (9191).
+/opt/blender/blender --online-mode -b --command blender_mcp --host 127.0.0.1 --port 9876 > /work/blender-addon.log 2>&1 &
+sleep 5
+cd /opt/blender_mcp/mcp && exec uv run blender-mcp --transport http --host 0.0.0.0 --port 9191
diff --git a/bench/agent-modeling/blender.mjs b/bench/agent-modeling/blender.mjs
new file mode 100644
index 00000000..d7932d76
--- /dev/null
+++ b/bench/agent-modeling/blender.mjs
@@ -0,0 +1,202 @@
+// The Blender MCP side of the agent friendliness comparison (issue #31): the same task prompts and model as run.mjs and
+// threejs.mjs, but the agent only has the official Blender MCP server (https://www.blender.org/lab/mcp-server/).
+//
+// The official server runs model-written Python without guards, so Blender and the server run ONLY inside a disposable
+// Docker container on a separate machine (see blender-container/). This script drives it over SSH and an SSH port forward:
+//
+// 1. one-time on the remote machine: build the image from blender-container/ (`docker build --platform linux/amd64`)
+// 2. open the forward: ssh -N -L 127.0.0.1:9191:127.0.0.1:9191 <host>
+// 3. node bench/agent-modeling/blender.mjs --ssh-host <host> [--claude <executable>] [--model sonnet] [--tasks crate,bolt]
+//
+// Per task the script starts a fresh container (a fresh Blender), mounts only a scratch folder, lets `claude -p` use the
+// MCP server and nothing else (no Bash, no file tools on this machine), then copies geometry.json and scene.png back and
+// grades them like threejs.mjs does: the task checks on the world-space triangles plus the silhouette look score.
+import { execFileSync, spawn } from 'node:child_process';
+import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { parseArgs } from 'node:util';
+import { removeLocalPaths, totalsByModel, totalsOf } from './aggregate.mjs';
+import { checkPng } from './png-check.mjs';
+import { gradeGeometries, readTranscript, scoreSilhouetteOfGeometries } from './threejs.mjs';
+import { taskById } from './tasks.mjs';
+
+const repository = resolve(import.meta.dirname, '../..');
+const MCP_SERVER_NAME = 'blender';
+const SERVER_URL = 'http://127.0.0.1:9191/';
+const BUILT_IN_TOOLS_DENIED = ['Bash', 'Read', 'Write', 'Edit', 'Glob', 'Grep', 'WebFetch', 'WebSearch', 'Agent', 'Task', 'NotebookEdit'];
+const DOCKER = 'export PATH=/opt/homebrew/bin:$PATH;';
+
+export const BLENDER_PREAMBLE = `You are modeling in Blender through the Blender MCP server. Blender 5.1 runs headless in a container with no GPU and no
+internet; the folder /work inside it is the only place files persist. Build the object described below as a closed,
+watertight solid, in metres, Y up as in glTF (Blender itself is Z up, so convert when you export). Rest the object on the
+ground (lowest point at Y = 0), centred on X = 0 and Z = 0.
+Leave two files in /work and then reply with one line saying so:
+1. geometry.json: a JSON array with one entry { "positions": [...], "indices": [...] } per mesh, with every vertex in world
+ space (apply each object's world matrix and any modifiers, then the Z-up to Y-up change) and indices as a flat triangle list.
+2. scene.png: a render of the scene, 512 by 512 pixels, any render engine that works headless (Cycles on the CPU works).`;
+
+export function parseBlenderArguments(argv) {
+ return parseArgs({
+ args: argv,
+ options: {
+ 'ssh-host': { type: 'string' },
+ image: { type: 'string', default: 'blender67' },
+ claude: { type: 'string', default: process.env.CLAUDE_EXECUTABLE ?? 'claude' },
+ model: { type: 'string', default: 'sonnet' },
+ tasks: { type: 'string', default: 'crate,bolt,window,table,fence,bracket' },
+ runs: { type: 'string', default: '1' },
+ budget: { type: 'string', default: '3' },
+ timeout: { type: 'string', default: '1200' },
+ label: { type: 'string', default: 'blender-mcp' },
+ },
+ }).values;
+}
+
+/** The scratch folder and the container are both removed by the script that created them. */
+function remote(options, command) {
+ return execFileSync('ssh', [options['ssh-host'], `${DOCKER} ${command}`], { encoding: 'utf8', timeout: 300_000 });
+}
+
+async function callServer(code) {
+ const response = await fetch(SERVER_URL, {
+ method: 'POST',
+ headers: { 'content-type': 'application/json', accept: 'application/json, text/event-stream' },
+ body: JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'tools/call', params: { name: 'execute_blender_code', arguments: { code } } }),
+ signal: AbortSignal.timeout(20_000),
+ });
+ return (await response.text()).includes('"status\\": \\"ok\\"');
+}
+
+async function waitForBlender() {
+ for (let attempt = 0; attempt < 60; attempt++) {
+ try {
+ if (await callServer('result = {}')) return;
+ } catch {
+ // The container is still starting.
+ }
+ await new Promise(resolvePromise => setTimeout(resolvePromise, 2000));
+ }
+ throw new Error('the Blender MCP server never answered');
+}
+
+function runAgent(options, prompt, workingDirectory, configPath) {
+ return new Promise(resolvePromise => {
+ const started = performance.now();
+ const child = spawn(options.claude, [
+ '-p', prompt, '--model', options.model, '--output-format', 'stream-json', '--verbose',
+ '--mcp-config', configPath, '--strict-mcp-config',
+ '--allowedTools', `mcp__${MCP_SERVER_NAME}`,
+ '--disallowedTools', ...BUILT_IN_TOOLS_DENIED,
+ '--permission-mode', 'dontAsk', '--setting-sources', 'project', '--no-session-persistence', '--max-budget-usd', options.budget,
+ ], { cwd: workingDirectory, windowsHide: true, stdio: ['ignore', 'pipe', 'pipe'] });
+ let output = '';
+ child.stdout.on('data', chunk => { output += chunk; });
+ child.stderr.on('data', () => {});
+ const timer = setTimeout(() => child.kill(), Number(options.timeout) * 1000);
+ child.on('close', code => {
+ clearTimeout(timer);
+ resolvePromise({ output, code, wallMs: Math.round(performance.now() - started) });
+ });
+ });
+}
+
+if (process.argv[1] === fileURLToPath(import.meta.url)) {
+ const options = parseBlenderArguments(process.argv.slice(2));
+ if (!options['ssh-host']) throw new Error('--ssh-host is required: the machine that runs the disposable container');
+ const runName = `${new Date().toISOString().slice(0, 16).replace(/[:T]/g, '-')}-${options.model}-${options.label}`;
+ const rawDirectory = join(repository, 'artifacts', 'agent-modeling', runName);
+ const resultsDirectory = join(repository, 'bench', 'agent-modeling', 'results');
+ await mkdir(rawDirectory, { recursive: true });
+ await mkdir(resultsDirectory, { recursive: true });
+ execFileSync('scp', ['-q', join(import.meta.dirname, 'blender-container', 'entry.sh'), `${options['ssh-host']}:bench-scratch/entry.sh`]);
+ const blenderVersion = remote(options, `docker run --rm --platform linux/amd64 ${options.image} /opt/blender/blender --version | head -1`).trim();
+
+ const records = [];
+ const summarize = () => ({
+ run: runName,
+ system: 'Blender MCP (official server in a container)',
+ blenderVersion,
+ model: records.find(record => record.model)?.model ?? options.model,
+ date: new Date().toISOString(),
+ runsPerTask: Number(options.runs),
+ passed: records.filter(record => record.pass).length,
+ tasks: records.length,
+ toolCalls: records.reduce((sum, record) => sum + record.toolCalls, 0),
+ byModel: totalsByModel(records),
+ records,
+ });
+
+ const workingDirectory = await mkdtemp(join(tmpdir(), '67js-blender-agent-'));
+ const configPath = join(workingDirectory, 'mcp.json');
+ await writeFile(configPath, JSON.stringify({ mcpServers: { [MCP_SERVER_NAME]: { type: 'http', url: SERVER_URL } } }));
+ try {
+ for (let attempt = 1; attempt <= Number(options.runs); attempt++) {
+ for (const task of options.tasks.split(',').map(taskById)) {
+ const container = `blender67-${task.id}-${attempt}`;
+ const scratch = `bench-scratch/${runName}/${task.id}-${attempt}`;
+ const localCopy = join(rawDirectory, `${task.id}-${attempt}`);
+ await mkdir(localCopy, { recursive: true });
+ let run = { output: '', code: null, wallMs: 0 };
+ let startError = null;
+ try {
+ remote(options, `mkdir -p "$HOME/${scratch}" && docker run -d --name ${container} --platform linux/amd64 --memory 6g -p 127.0.0.1:9191:9191 -v "$HOME/${scratch}:/work" -v "$HOME/bench-scratch/entry.sh:/entry.sh:ro" ${options.image} sh /entry.sh`);
+ await waitForBlender();
+ run = await runAgent(options, `${BLENDER_PREAMBLE}\n\n${task.prompt.replace(/\n/g, ' ')}`, workingDirectory, configPath);
+ } catch (error) {
+ startError = error.message.split('\n')[0];
+ } finally {
+ try {
+ execFileSync('scp', ['-q', `${options['ssh-host']}:${scratch}/geometry.json`, `${options['ssh-host']}:${scratch}/scene.png`, localCopy], { stdio: 'ignore' });
+ } catch {
+ // A file the agent never wrote is a failed check below.
+ }
+ remote(options, `docker rm -f ${container} >/dev/null; rm -rf "$HOME/${scratch}"`);
+ }
+ const transcript = readTranscript(run.output);
+ await writeFile(join(rawDirectory, `${task.id}-${attempt}.jsonl`), run.output);
+ let geometries = null;
+ let grade = { pass: false, passed: 0, total: task.checks.length, results: [{ kind: 'geometry', pass: false, detail: startError ?? 'no geometry.json' }] };
+ try {
+ geometries = JSON.parse(await readFile(join(localCopy, 'geometry.json'), 'utf8'));
+ grade = gradeGeometries(task, geometries);
+ } catch (error) {
+ grade.results[0].detail = startError ?? `geometry.json unreadable: ${error.message.split('\n')[0]}`;
+ }
+ const png = await checkPng(join(localCopy, 'scene.png'));
+ const silhouette = geometries ? await scoreSilhouetteOfGeometries(task.id, geometries).catch(error => ({ error: error.message.split('\n')[0] })) : null;
+ const usage = transcript.result?.usage ?? {};
+ const failed = grade.results.filter(result => !result.pass).map(result => `${result.label ?? result.kind}: ${result.detail}`);
+ if (!png.pass) failed.push(`scene.png: ${png.detail}`);
+ const record = {
+ task: task.id,
+ attempt,
+ pass: grade.pass && png.pass,
+ passed: grade.passed,
+ total: grade.total,
+ pngPass: png.pass,
+ failed: failed.map(removeLocalPaths),
+ model: transcript.model,
+ toolCalls: transcript.toolCalls,
+ toolErrors: transcript.toolErrors,
+ toolsByName: transcript.toolsByName,
+ look: silhouette ? { silhouette } : null,
+ turns: transcript.result?.num_turns ?? null,
+ costUsd: null,
+ inputTokens: (usage.input_tokens ?? 0) + (usage.cache_read_input_tokens ?? 0) + (usage.cache_creation_input_tokens ?? 0),
+ outputTokens: usage.output_tokens ?? 0,
+ wallSeconds: Math.round(run.wallMs / 1000),
+ exitCode: run.code,
+ stopped: transcript.result?.subtype ?? 'no result',
+ };
+ records.push(record);
+ console.log(`${record.task.padEnd(10)} #${attempt} ${record.pass ? 'PASS' : 'FAIL'} ${record.passed}/${record.total} png ${png.pass} calls ${record.toolCalls} (errors ${record.toolErrors}) look ${silhouette?.mean ?? '-'} ${record.wallSeconds}s${failed.length ? ` ${failed.join(' | ')}` : ''}`);
+ await writeFile(join(resultsDirectory, `${runName}.json`), `${JSON.stringify(summarize(), null, 2)}\n`);
+ }
+ }
+ } finally {
+ await rm(workingDirectory, { recursive: true, force: true }).catch(() => {});
+ }
+ console.log(`${records.filter(record => record.pass).length}/${records.length} passed, ${totalsOf(records).toolCalls} tool calls. Summary: bench/agent-modeling/results/${runName}.json`);
+}
diff --git a/bench/agent-modeling/results/2026-11-07-blender-mcp.json b/bench/agent-modeling/results/2026-11-07-blender-mcp.json
new file mode 100644
index 00000000..dfceeed4
--- /dev/null
+++ b/bench/agent-modeling/results/2026-11-07-blender-mcp.json
@@ -0,0 +1,279 @@
+{
+ "run": "2026-11-07-blender-mcp",
+ "system": "Blender MCP (official server in a container)",
+ "blenderVersion": "Blender 5.1.2",
+ "model": "claude-sonnet-5-5",
+ "date": "2026-11-07T18:04:20.919Z",
+ "runsPerTask": 1,
+ "passed": 6,
+ "tasks": 6,
+ "toolCalls": 23,
+ "byModel": {
+ "claude-sonnet-5-5": {
+ "runs": 6,
+ "passed": 6,
+ "successRate": 1,
+ "toolCalls": 23,
+ "toolErrors": 5,
+ "inputTokens": 1043319,
+ "outputTokens": 18972,
+ "costUsd": 0,
+ "tasks": {
+ "crate": {
+ "runs": 1,
+ "passed": 1,
+ "successRate": 1,
+ "toolCalls": 4,
+ "toolErrors": 1,
+ "inputTokens": 178557,
+ "outputTokens": 3026,
+ "costUsd": 0
+ },
+ "bolt": {
+ "runs": 1,
+ "passed": 1,
+ "successRate": 1,
+ "toolCalls": 4,
+ "toolErrors": 1,
+ "inputTokens": 178812,
+ "outputTokens": 3081,
+ "costUsd": 0
+ },
+ "window": {
+ "runs": 1,
+ "passed": 1,
+ "successRate": 1,
+ "toolCalls": 4,
+ "toolErrors": 1,
+ "inputTokens": 181015,
+ "outputTokens": 3619,
+ "costUsd": 0
+ },
+ "table": {
+ "runs": 1,
+ "passed": 1,
+ "successRate": 1,
+ "toolCalls": 5,
+ "toolErrors": 1,
+ "inputTokens": 206569,
+ "outputTokens": 2655,
+ "costUsd": 0
+ },
+ "fence": {
+ "runs": 1,
+ "passed": 1,
+ "successRate": 1,
+ "toolCalls": 2,
+ "toolErrors": 0,
+ "inputTokens": 116074,
+ "outputTokens": 2525,
+ "costUsd": 0
+ },
+ "bracket": {
+ "runs": 1,
+ "passed": 1,
+ "successRate": 1,
+ "toolCalls": 4,
+ "toolErrors": 1,
+ "inputTokens": 182292,
+ "outputTokens": 4066,
+ "costUsd": 0
+ }
+ }
+ }
+ },
+ "records": [
+ {
+ "task": "crate",
+ "attempt": 1,
+ "pass": true,
+ "passed": 8,
+ "total": 8,
+ "pngPass": true,
+ "failed": [],
+ "model": "claude-sonnet-5-5",
+ "toolCalls": 4,
+ "toolErrors": 1,
+ "toolsByName": {
+ "mcp__blender__execute_blender_code": 4
+ },
+ "look": {
+ "silhouette": {
+ "views": {
+ "front": 1,
+ "side": 1,
+ "three-quarter": 1
+ },
+ "mean": 1
+ }
+ },
+ "turns": 6,
+ "costUsd": null,
+ "inputTokens": 178557,
+ "outputTokens": 3026,
+ "wallSeconds": 590,
+ "exitCode": 0,
+ "stopped": "success"
+ },
+ {
+ "task": "bolt",
+ "attempt": 1,
+ "pass": true,
+ "passed": 7,
+ "total": 7,
+ "pngPass": true,
+ "failed": [],
+ "model": "claude-sonnet-5-5",
+ "toolCalls": 4,
+ "toolErrors": 1,
+ "toolsByName": {
+ "mcp__blender__execute_blender_code": 4
+ },
+ "look": {
+ "silhouette": {
+ "views": {
+ "front": 0.9621,
+ "side": 0.9621,
+ "three-quarter": 0.9666
+ },
+ "mean": 0.9636
+ }
+ },
+ "turns": 6,
+ "costUsd": null,
+ "inputTokens": 178812,
+ "outputTokens": 3081,
+ "wallSeconds": 609,
+ "exitCode": 0,
+ "stopped": "success"
+ },
+ {
+ "task": "window",
+ "attempt": 1,
+ "pass": true,
+ "passed": 5,
+ "total": 5,
+ "pngPass": true,
+ "failed": [],
+ "model": "claude-sonnet-5-5",
+ "toolCalls": 4,
+ "toolErrors": 1,
+ "toolsByName": {
+ "mcp__blender__execute_blender_code": 4
+ },
+ "look": {
+ "silhouette": {
+ "views": {
+ "front": 1,
+ "side": 1,
+ "three-quarter": 1
+ },
+ "mean": 1
+ }
+ },
+ "turns": 6,
+ "costUsd": null,
+ "inputTokens": 181015,
+ "outputTokens": 3619,
+ "wallSeconds": 611,
+ "exitCode": 0,
+ "stopped": "success"
+ },
+ {
+ "task": "table",
+ "attempt": 1,
+ "pass": true,
+ "passed": 6,
+ "total": 6,
+ "pngPass": true,
+ "failed": [],
+ "model": "claude-sonnet-5-5",
+ "toolCalls": 5,
+ "toolErrors": 1,
+ "toolsByName": {
+ "mcp__blender__execute_blender_code": 5
+ },
+ "look": {
+ "silhouette": {
+ "views": {
+ "front": 1,
+ "side": 0.1684,
+ "three-quarter": 0.6073
+ },
+ "mean": 0.5919
+ }
+ },
+ "turns": 7,
+ "costUsd": null,
+ "inputTokens": 206569,
+ "outputTokens": 2655,
+ "wallSeconds": 625,
+ "exitCode": 0,
+ "stopped": "success"
+ },
+ {
+ "task": "fence",
+ "attempt": 1,
+ "pass": true,
+ "passed": 6,
+ "total": 6,
+ "pngPass": true,
+ "failed": [],
+ "model": "claude-sonnet-5-5",
+ "toolCalls": 2,
+ "toolErrors": 0,
+ "toolsByName": {
+ "mcp__blender__execute_blender_code": 2
+ },
+ "look": {
+ "silhouette": {
+ "views": {
+ "front": 1,
+ "side": 1,
+ "three-quarter": 1
+ },
+ "mean": 1
+ }
+ },
+ "turns": 4,
+ "costUsd": null,
+ "inputTokens": 116074,
+ "outputTokens": 2525,
+ "wallSeconds": 38,
+ "exitCode": 0,
+ "stopped": "success"
+ },
+ {
+ "task": "bracket",
+ "attempt": 1,
+ "pass": true,
+ "passed": 6,
+ "total": 6,
+ "pngPass": true,
+ "failed": [],
+ "model": "claude-sonnet-5-5",
+ "toolCalls": 4,
+ "toolErrors": 1,
+ "toolsByName": {
+ "mcp__blender__execute_blender_code": 4
+ },
+ "look": {
+ "silhouette": {
+ "views": {
+ "front": 1,
+ "side": 1,
+ "three-quarter": 1
+ },
+ "mean": 1
+ }
+ },
+ "turns": 6,
+ "costUsd": null,
+ "inputTokens": 182292,
+ "outputTokens": 4066,
+ "wallSeconds": 626,
+ "exitCode": 0,
+ "stopped": "success"
+ }
+ ]
+}
diff --git a/bench/agent-modeling/threejs.mjs b/bench/agent-modeling/threejs.mjs
index 83fed2c3..f7586e58 100644
--- a/bench/agent-modeling/threejs.mjs
+++ b/bench/agent-modeling/threejs.mjs
@@ -134,7 +134,7 @@ export async function scoreSilhouetteOfGeometries(taskId, rawGeometries) {
return { views: overlap, mean: Number((values.reduce((sum, value) => sum + value, 0) / values.length).toFixed(4)) };
}
-function readTranscript(output) {
+export function readTranscript(output) {
const summary = { model: null, toolCalls: 0, toolErrors: 0, toolsByName: {}, result: null };
for (const line of output.split('\n')) {
if (!line.trim().startsWith('{')) continue;
diff --git a/changes/31.md b/changes/31.md
index 5e9be688..b5034902 100644
--- a/changes/31.md
+++ b/changes/31.md
@@ -1 +1,2 @@
- Compare agent friendliness of 67js MCP against three.js written as code on six suite tasks with the same model and the same checks and look score (issue #31, three.js half; the Blender MCP half still needs a VM): `bench/agent-modeling/threejs.mjs` runs the three.js side and `results/2026-11-07-threejs-vs-67js.json` holds both.
+- Add the Blender MCP half of issue #31: `bench/agent-modeling/blender.mjs` runs Sonnet against the official Blender MCP server inside a disposable container (`blender-container/`) on the same six tasks and the same grading; 6 of 6 pass, 3.8 tool calls, look 0.926 (`results/2026-11-07-blender-mcp.json`), so the issue now has a row for 67js, three.js and Blender.