Official Codebase of 3DCodeVerse
| Name | Last commit message | Last commit date |
|---|---|---|
| .claude-plugin | ||
| assets | ||
| eval | ||
| finetune | ||
| flywheel | ||
| harness | ||
| toolkits | ||
| .gitignore 1.1 KB | ||
| CLAUDE.md 1.2 KB | ||
| LICENSE 11.1 KB | ||
| README.md 7.3 KB |
Exploring 3D Worlds as Code
Yipeng Gao1*†, Ziyao Zeng2*, Jinfa Huang3*, Yijiang Li4*
Haolin Xiong1, Wang Qin1, Xinlei Yu1, Youheng Yao2, Jie Zhang5, Kun-Yu Lin6, Rong Liu1, Yinyuan Zhao7, Yajie Zhao1
Meiqi Guo8, Jiebo Luo3, Lei Shu8, Yunhao Ge1, Xinyu Hu9, Laurent Itti1
1University of Southern California · 2Yale University · 3University of Rochester
4University of California, San Diego · 5Columbia University · 6Nanyang Technological University
7University of California, Santa Cruz · 8Google DeepMind · 9Philo Labs
*Equal contribution · †Project lead
Awesome Showcase
Films rebuilt entirely in code, with full source and agent logs on Hugging Face.
| Code | Reference | Result |
|---|---|---|
| link | ![]() OpenAI on X |
![]() Opus 5.5 by Code |
Overview
https://github.com/user-attachments/assets/3b0b5e5f-eed0-4ffd-8ab7-f3c709e15fad
3DCodeVerse Harness

An LLM planner defines the target asset, and a DAG of sub-agents codes and assembles a 3D program. Every round, the harness collects three kinds of feedback:
- Execution — build and runtime logs.
- Measurement — dimensions, contacts, collisions, kinematics and placement, reported per part with numerical correction hints.
- Vision — a rubric-guided VLM judge on labelled multi-view renders.
The score is computed in code from rubric weights, defect penalties and measured caps, and the measurements veto visual claims they rule out. The feedback becomes localized repair tasks for a fix agent, and the best-scoring round is the deliverable. Because every round is recorded, including successes, failures, repairs and preference pairs, the harness also serves as the corpus's data engine.
3DCodeVerse-2M Dataset

News
- [10/15/2026] Our paper is now available on arXiv.
Results

Fine-tuning on 3DCodeVerse-2M makes open models execute. The table reports execution rate (%) on 805 fixed tasks:
| Model | 3DCodeBench | Blender | CadQuery | OpenSCAD | GLSL | Three.js |
|---|---|---|---|---|---|---|
| Qwen3.5-9B | 0.0 | 1.9 | 10.5 | 50.0 | 33.0 | 72.5 |
| + SFT | 84.9 | 90.3 | 98.5 | 86.0 | 56.0 | 74.5 |
| Qwen3.8-27B | 51.9 | 74.8 | 65.5 | 80.0 | 57.0 | 72.5 |
| + SFT + DPO | 92.9 | 93.2 | 98.5 | 90.0 | 71.5 | 92.5 |
Quick Start
The harness ships as 3dcodeverse, a plugin for Claude Code, Codex and Antigravity: the host agent
writes the code with its own model, and the plugin's skills and 3dcode instruments build, measure,
render, gate and score it (a fresh-context subagent judges). It needs Python 3.13, Node 20.6+ and Git;
Blender 5.2 LTS for the blender, urdf_blender and scene_blender languages.
Full guide: harness/README.md and harness/docs/INSTALL.md.
git clone https://github.com/gaoypeng/3dcodeverse && cd 3dcodeverse
bash harness/setup.sh # the `3dcode` instruments + runtime_js + headless Chrome, then doctor
claude plugin marketplace add . && claude plugin install 3dcodeverse@3dcodeverse # Claude Code
codex plugin marketplace add . && codex plugin add 3dcodeverse@3dcodeverse # Codex
agy plugin install harness # Antigravity
Then ask your agent for a 3D thing, e.g. "Make a mid-century wooden dining chair in Blender bpy."
Repository Structure
| Directory | Description | Documentation |
|---|---|---|
harness/ |
The 3dcodeverse agent plugin: skills, judge subagent, and the 3dcode build / measure / render / gate / score instruments. |
Plugin guide |
flywheel/ |
Reader for runs recorded by the v0.0.x CLI harness: gallery, cost, dataset export, pick. | Flywheel guide |
toolkits/ |
General data CLI and per-source corpus curation. | Data toolkit guide |
finetune/ |
LLaMA-Factory training-data preparation, supervised fine-tuning, and preference optimization for open models. | Training guide |
eval/ |
Evaluation of standalone LLMs / VLMs on 3D coding. | Evaluation guide |
Citation
If you find 3DCodeVerse useful, please cite:
@article{gao20263dcodeverse,
title={3DCodeVerse: Exploring 3D Worlds as Code},
author={Gao, Yipeng and Zeng, Ziyao and Huang, Jinfa and Li, Yijiang and Xiong, Haolin and Qin, Wang and Yu, Xinlei and Yao, Youheng and Zhang, Jie and Lin, Kun-Yu and others},
journal={arXiv preprint arXiv:2610.22341},
year={2026}
}
License
The code is licensed under Apache-2.0. The published datasets carry their own terms (CC-BY-NC-SA, non-commercial); see each dataset card.

