Official Codebase of 3DCodeVerse
Python
toolkits/: one home for the data tooling — per-source curation scripts from 3dcode_toolkit, plus the corpus-to-LLaMA-Factory pipeline (per-subdir conversion + QC, compiler/renderer verification, multi-file flattening, URDF recovery, multimodal render+instruction variants, mix sampling)
gaoypengbcdbcff738d4
diff --git a/README.md b/README.mdindex 8eab8e44..a7056282 100644--- a/README.md+++ b/README.md@@ -15,6 +15,10 @@ The 3dcodeverse project. Top-level layout:`harness/docs/INSTALL.md`. Needs **python 3.10+** and **node 20.6+** on Linux(developed on 3.13 / node 24; CI runs both ends — `harness/docs/INSTALL.md` §2.1).+- **`toolkits/`** — the data tooling: per-source curation scripts, and the pipeline that turns the+ corpus into verified LLaMA-Factory datasets (per-subdir conversion, compiler/renderer verification,+ multi-file flattening, multimodal variants, mix sampling). See `toolkits/README.md`.+- **`finetune/`** — LLM finetuning recipes (LoRA / full-parameter SFT, execution-and geometry-feedback DPO via unpatched LLaMA-Factory) and the execution-basedevaluation stack (3DCodeBench + per-dialect executors) used to train opendiff --git a/toolkits/README.md b/toolkits/README.mdnew file mode 100644index 00000000..5248089e--- /dev/null+++ b/toolkits/README.md@@ -0,0 +1,87 @@+# toolkits/ — the tools that turn raw 3D projects into trainable data++Everything here operates on **data**, not on models. The pipeline they cover, end to end:++```+contributor's project ──▶ curation ──▶ 3DCodeVerse on the Hub ──▶ LLaMA-Factory datasets ──▶ finetune/+ code + renders validate, per-source subdirs per-subdir ShareGPT, training runs+ + meta.json dedupe, (metadata.parquet + tars) verified & flagged and evaluation+ render, pack+```++| folder | stage | what it does |+|---|---|---|+| [`curation/`](curation/) | raw → corpus | per-source curation: convert a source's native format to runnable projects, execute/render them, compute ground-truth meshes, dedupe, pack to tar+parquet, upload to the Hub |+| [`llamafactory/`](llamafactory/) | corpus → training data | turn every corpus subdirectory into a LLaMA-Factory dataset, verify it with the real compilers/runtimes, flatten multi-file samples, build multimodal (render + instruction) variants, and sample training mixes |++The contributor-facing CLI (`3dcode validate / check / exec / render / push`) lives in its own installable+package, **[gaoypeng/3dcode_toolkit](https://github.com/gaoypeng/3dcode_toolkit)**, and is intentionally *not*+vendored here — it is `pipx install`-ed and would drift if copied. `curation/` holds the per-source scripts from+that repo, which are not part of the package and would otherwise have no home.++---++## curation/ — per source++| source | scripts | notes |+|---|---|---|+| `deepcad/` | `convert_vec_to_cq.py`, `batch_convert.py`, `cq_check{,_batch}.py`, `gen_gt_stl.py`, `iou_compute.py`, `render_all.py`, `dedup_check.py`, `attach_text.py`, `pack_tar_parquet.py`, `finalize.py`, `upload_to_hf.py` | DeepCAD vector sequences → CadQuery programs, executed and checked, GT meshes + IoU, captions attached, packed and uploaded |+| `articraft/` | `articraft_cq.py`, `assemble_shim.py`, … | articulated assets → CadQuery / URDF projects |++The shape of a curation pass, which any new source should follow: **convert → execute → render → compute a+ground truth → dedupe → attach text → pack → upload**. Every step writes files, so a failed step can be rerun+without redoing the pipeline.++## llamafactory/ — corpus → trainable datasets++| script | what it does |+|---|---|+| `convert_subdirs.py` | one `<subdir>_llamafactory/` per corpus subdirectory: `train.parquet` (ShareGPT) + `dataset_info.json` + `qc.json` + README. Runs the per-sample quality checks (empty/short/mojibake code, in-subdir duplicates, missing captions, **benchmark overlap**, syntax, over-length, literal media placeholders) |+| `flatten_multifile.py` | makes multi-file samples single-file: merges a Shadertoy shader's **Common tab** into each pass exactly as Shadertoy does, and recovers `code.urdf` for the articraft URDF subdirs (whose `code` column is empty in the metadata) |+| `qc_repair.py` | validates with the **real** compiler (glslangValidator for GLSL, `ast.parse` for Python dialects), attempts minimal semantics-preserving repairs, re-validates each one, and reports exact-duplicate groups within and across subdirs |+| `glsl_render_batch.py`, `glsl_render.py`, `render_verify_glsl.py` | compiling is not drawing: render every shader in a real headless WebGL2 context, sample two frames, and record whether the image is non-uniform and whether it animates |+| `finalize_glsl.py` | consolidates: prefer the author's untouched program, fall back to a repair only if it compiles, otherwise mark `compiles=False`. Writes `compiles` / `repair` columns |+| `build_mm_dataset.py` | the multimodal datasets — `mm_img` (renders only), `mm_img_text` (renders + instruction), `mm_text` (control), from the same samples with the same split so they can be compared |+| `build_image_dataset.py` | earlier single-variant image builder (kept for reproducing the first pilot) |+| `build_md_max.py`, `sample_mix.py` | build the full corpus (every source, every caption variant) and sample training mixes from it with a per-dialect token budget and a captions-per-sample cap |+| `to_llamafactory.py` | jsonl → ShareGPT JSON + a `dataset_info.json` entry |+| `build_hf_dataset.py` | package corpus + preference sets + held-out tests as a Hub dataset |+| `cache_ntok.py`, `ntok_cache.py` | cache per-sample token counts so re-mixing is seconds, not half an hour |+| `extract.py` | dialect-aware code extraction from free-form model output (used by evaluation, included here because dataset QC uses the same rules) |+| `analyze_3dcodeverse.py`, `fetch_hf_files.py` | corpus inventory; selective file download from the Hub |++### Two rules these tools encode++1. **Suspect the checker before the data.** Of 4,192 shaders that "failed", only 588 actually needed a code+ change; the rest exposed defects in the validation harness (guessing sampler types from syntax, a missing+ `iFrameRate` uniform, a regex that deleted real code from minified lines, the wrong GLSL ES profile).+ Every checker in here has a known-good control that must pass.+2. **Mark, don't silently drop.** Rows that cannot be used for a particular purpose — `benchmark_factory`+ (3DCodeBench contamination), `compiles=False`, over-length — stay in the dataset with a column that lets a+ training mix filter them out. The one exception is exact duplicates, which are removed and listed in+ `duplicates_<source>.json` so the upstream corpus can be cleaned too.++### Typical use++```bash+# 1. corpus -> one dataset per subdirectory, with QC+python convert_subdirs.py --out /path/to/out --workers 16++# 2. make multi-file samples single-file, recover code that lives in tars+python flatten_multifile.py --shadertoy --out /path/to/out+python flatten_multifile.py --urdf_tars /path/to/tars --out /path/to/out++# 3. verify with the real toolchain, repair minimally, consolidate+python qc_repair.py --pass all --root /path/to/out+python render_verify_glsl.py --workers 10 --write+python finalize_glsl.py++# 4. build a training mix (per-dialect token budget, one caption per sample)+python sample_mix.py --out mix/ --budget "blender=all,cadquery=25M,glsl=25M" --captions_per_sample 1++# 5. multimodal variants (renders + instruction)+python build_mm_dataset.py --views 4 --out mm/+```++Results produced with these tools: [`finetune/llm_finetune_exps.md`](../finetune/llm_finetune_exps.md) and+[`finetune/docs/REPORT.md`](../finetune/docs/REPORT.md).diff --git a/toolkits/curation/articraft/README.md b/toolkits/curation/articraft/README.mdnew file mode 100644index 00000000..27e7a129--- /dev/null+++ b/toolkits/curation/articraft/README.md@@ -0,0 +1,108 @@+# Articraft → CadQuery curation tools++Tools used to curate the **Articraft** subset of [`ilabai/3dcodeverse`](https://huggingface.co/datasets/ilabai/3dcodeverse).++[Articraft](https://github.com/mattzh72/articraft) (Apache-2.0) is a **mesh-based**+SDK: its models build triangle meshes. We convert each model into a+**self-contained, parametric CadQuery (B-rep) program** — one `code.py` per sample+that only needs `import cadquery` to run — then verify, render, caption, and upload.++**Result: 6878 / 6882 samples converted (99.94%).**++## Approach++1. **Clean-room CadQuery shim** reimplementing the Articraft SDK public API, producing+ exact B-rep instead of triangle meshes. `shim/` = a core (`articraft_cq.py`) ++ one module per geometry family (`shim_parts/`).+2. **Tree-shaking generator** (`gen_code.py`): for each model it inlines *only* the+ shim symbols the model actually uses (symbol-level dependency closure over core ++ families + the model body), giving a minimal self-contained `code.py`+ (median ~900 lines vs ~1600 for whole-shim inlining).+3. **Verify** each output against the SDK's ground-truth mesh via normalized+ **chamfer distance** (pass = chamfer < 0.02 and bbox ratio ∈ [0.9, 1.1]).+4. **Render** 8 azimuth views + a colored GLB with Blender (Cycles, GPU).+5. **Caption** a natural "user instruction" per sample with Gemini (image-primary ++ a structured prior of parts/joints).+6. **Upload** to Hugging Face in small low-concurrency chunks.++## Layout++```+shim:+ articraft_cq.py core shim: model API (ArticulatedObject + rest-pose FK ->+ cq.Assembly), primitives, transforms, MeshGeometry base,+ profile/builder helpers, booleans+ shim_parts/*.py 10 geometry families (primitives, lathe_extrude_loft, sweeps,+ section_loft, knobs, wheels_tires, fans, panels_vents,+ brackets_hinges_bezels, gears)+ assemble_shim.py combine core + families; name -> family map++convert:+ gen_code.py tree-shaking self-contained code.py generator++verify:+ verify_cadquery.py exec code.py -> chamfer vs HF ground-truth mesh (+ --glb export)+ geom_chamfer.py isolated per-geometry chamfer (real sdk vs candidate)++render:+ cq_to_glb.py exec code.py -> renders/object.glb (colored cq.Assembly)+ blender_glb.py Blender: load GLB -> 8 views (material / clay)+ blender_urdf.py Blender: render a URDF -> 8 views+ render_glb_only.py GPU consumer: render every pending GLB in parallel+ driver_urdf_render.py URDF render driver++caption:+ caption_cadquery.py Gemini "generate this as CadQuery code" user-instruction+ caption_urdf.py Gemini "generate this as URDF" user-instruction++batch / pipeline:+ batch_verify.py parallel gen + verify + meta update (resumable)+ regen_glb.py regenerate code.py + export GLB (no chamfer), parallel+ exec_validate.py regen + exec-validate ALL samples (catch regressions)+ integrate_recovered.py re-run failed samples through fixes and fold the passes back in++upload:+ upload_articraft.py chunked HF upload (low-concurrency create_commit to respect+ rate limits; hf_transfer bursts trip 429s)+```++## Configuration (env vars)++| var | meaning | default |+|-----|---------|---------|+| `ARTICRAFT_ROOT` | dataset root containing `data/articraft_with_mesh/` etc. | repo-relative guess |+| `ARTICRAFT_FORK` | path to the real Articraft sdk (ground truth, for `geom_chamfer`) | — |+| `BLENDER` | Blender executable | `blender` |+| `CONDA_PREFIX` | used to set `LD_LIBRARY_PATH` so Blender finds conda libs | — |+| `GEMINI_API_KEY` or `~/.gemini_api_key` | Gemini key (captioning) | — |+| `~/.hf_token` | Hugging Face write token (verify download + upload) | — |++Credentials are **always read from env / dotfiles — never hard-coded or committed.**++## Typical pipeline++```bash+export ARTICRAFT_ROOT=/path/to/dataset+python assemble_shim.py # sanity: combine core + all families+python batch_verify.py all --workers 12 # gen self-contained code.py + chamfer verify+python regen_glb.py --workers 16 # export renders/object.glb for passing samples+python render_glb_only.py --workers 12 # render 8 views per GLB+python caption_cadquery.py --workers 256 --rpm 900 # Gemini captions+python upload_articraft.py mesh # chunked, low-concurrency HF upload+```++## Gotchas learned (baked into the tools)++- **mesh vs B-rep**: some models read `.vertices/.faces` off a geometry and rebuild it+ (axis swaps) — the shim tessellates the B-rep on demand for those (`MeshGeometry`+ `.vertices/.faces` are read/write properties).+- **`unit_scale`**: models authored in mm pass `unit_scale=0.001`; the shim honors it+ (otherwise output is 1000× too large).+- **`cq.Workplane.gear` plugin**: cq_gears-style `wp.gear(SpurGear(...))` is registered+ when the gears family is inlined.+- **name collisions**: inlining puts shim helpers and model helpers in one namespace;+ private shim helpers that clash with a model's are auto-renamed (`_shim*`).+- **OCC robustness**: `merge()` falls back to a Compound when a boolean fuse returns a+ null shape (e.g. many thin wire loops).+- **HF rate limits**: upload uses `create_commit(num_threads=5)` + backoff, not+ `hf_transfer` (whose parallel bursts trip HF's request-rate limit and then hang).diff --git a/toolkits/curation/articraft/articraft_cq.py b/toolkits/curation/articraft/articraft_cq.pynew file mode 100644index 00000000..1175eaae--- /dev/null+++ b/toolkits/curation/articraft/articraft_cq.py@@ -0,0 +1,914 @@+"""Self-contained CadQuery reimplementation of the Articraft `sdk` public API.++This is a clean-room shim: it exposes the SAME names that generated Articraft+model scripts import via `from sdk import (...)`, but every geometry call+produces real parametric CadQuery B-rep instead of triangle meshes. A model+script can run unchanged (minus the sdk import) and then:++ result = object_model.to_cq() # whole model, REST pose, cq.Assembly++The geometry semantics mirror the original Apache-2.0 Articraft SDK+(github mattzh72/articraft); only cadquery + the stdlib are required.++Modular: each public name is grouped so a generator can inline only the+pieces a given model actually uses (see tools/gen_code.py).+"""+from __future__ import annotations++import math+import cadquery as cq++# ===========================================================================+# @section core-transform (always required)+# ===========================================================================+def _rpy_R(r, p, y):+ """Rotation matrix Rz(yaw)@Ry(pitch)@Rx(roll) as rows (URDF convention)."""+ cr, sr = math.cos(r), math.sin(r)+ cp, sp = math.cos(p), math.sin(p)+ cy, sy = math.cos(y), math.sin(y)+ return (+ (cy * cp, cy * sp * sr - sy * cr, cy * sp * cr + sy * sr),+ (sy * cp, sy * sp * sr + cy * cr, sy * sp * cr - cy * sr),+ (-sp, cp * sr, cp * cr),+ )+++def _mat_mul(A, B):+ return tuple(+ tuple(sum(A[i][k] * B[k][j] for k in range(3)) for j in range(3))+ for i in range(3)+ )+++def _mat_vec(A, v):+ return tuple(sum(A[i][k] * v[k] for k in range(3)) for i in range(3))+++class _Tf:+ """Rigid transform: rotation matrix R (rows) + translation t."""+ __slots__ = ("R", "t")+ _I = ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))++ def __init__(self, R=None, t=(0.0, 0.0, 0.0)):+ self.R = R if R is not None else _Tf._I+ self.t = (float(t[0]), float(t[1]), float(t[2]))++ @staticmethod+ def from_origin(origin):+ if origin is None:+ return _Tf()+ xyz = getattr(origin, "xyz", (0.0, 0.0, 0.0))+ rpy = getattr(origin, "rpy", (0.0, 0.0, 0.0))+ return _Tf(_rpy_R(rpy[0], rpy[1], rpy[2]), xyz)++ def __matmul__(self, o):+ return _Tf(_mat_mul(self.R, o.R),+ tuple(a + b for a, b in zip(_mat_vec(self.R, o.t), self.t)))++ def location(self):+ R, t = self.R, self.t+ xdir = cq.Vector(R[0][0], R[1][0], R[2][0])+ zdir = cq.Vector(R[0][2], R[1][2], R[2][2])+ return cq.Location(cq.Plane(origin=cq.Vector(*t), xDir=xdir, normal=zdir))+++# ===========================================================================+# @section core-errors+# ===========================================================================+class ValidationError(ValueError):+ pass+++# ===========================================================================+# @section core-types (always required: Origin/Material/limits/enums)+# ===========================================================================+class ArticulationType:+ REVOLUTE = "revolute"+ CONTINUOUS = "continuous"+ PRISMATIC = "prismatic"+ FIXED = "fixed"+ FLOATING = "floating"+++class Origin:+ __slots__ = ("xyz", "rpy")++ def __init__(self, xyz=(0.0, 0.0, 0.0), rpy=(0.0, 0.0, 0.0)):+ self.xyz = (float(xyz[0]), float(xyz[1]), float(xyz[2]))+ self.rpy = (float(rpy[0]), float(rpy[1]), float(rpy[2]))+++class Material:+ def __init__(self, name, rgba=None, texture=None, *, color=None):+ self.name = str(name)+ c = rgba if rgba is not None else color+ if c is not None:+ c = tuple(float(v) for v in c)+ if len(c) == 3:+ c = c + (1.0,)+ self.rgba = c+ self.texture = texture+++class MotionLimits:+ def __init__(self, effort=1.0, velocity=1.0, lower=None, upper=None):+ self.effort = float(effort)+ self.velocity = float(velocity)+ self.lower = None if lower is None else float(lower)+ self.upper = None if upper is None else float(upper)+++class MotionProperties:+ def __init__(self, damping=None, friction=None):+ self.damping = damping+ self.friction = friction+++class Mimic:+ def __init__(self, joint, multiplier=1.0, offset=0.0):+ self.joint = str(joint)+ self.multiplier = float(multiplier)+ self.offset = float(offset)+++class Inertia:+ def __init__(self, ixx=0.0, ixy=0.0, ixz=0.0, iyy=0.0, iyz=0.0, izz=0.0):+ self.ixx, self.ixy, self.ixz = ixx, ixy, ixz+ self.iyy, self.iyz, self.izz = iyy, iyz, izz+++class Inertial:+ def __init__(self, mass=1.0, inertia=None, origin=None):+ self.mass = float(mass)+ self.inertia = inertia+ self.origin = origin or Origin()++ @staticmethod+ def from_geometry(geometry, mass, *, origin=None):+ return Inertial(mass=mass, inertia=Inertia(), origin=origin or Origin())+++# ===========================================================================+# @section core-primitives (Box / Cylinder / Sphere data + cq realization)+# ===========================================================================+class Box:+ __slots__ = ("size",)++ def __init__(self, size):+ self.size = (float(size[0]), float(size[1]), float(size[2]))++ def _cq(self):+ sx, sy, sz = self.size+ return cq.Solid.makeBox(sx, sy, sz, cq.Vector(-sx / 2, -sy / 2, -sz / 2))+++class Cylinder:+ __slots__ = ("radius", "length")++ def __init__(self, radius, length=None, *, height=None):+ self.radius = float(radius)+ self.length = float(height if length is None else length)++ @property+ def height(self):+ return self.length++ def _cq(self):+ return cq.Solid.makeCylinder(+ self.radius, self.length, cq.Vector(0, 0, -self.length / 2), cq.Vector(0, 0, 1)+ )+++class Sphere:+ __slots__ = ("radius",)++ def __init__(self, radius):+ self.radius = float(radius)++ def _cq(self):+ return cq.Solid.makeSphere(self.radius, angleDegrees1=-90, angleDegrees2=90,+ angleDegrees3=360)+++# ===========================================================================+# @section core-mesh (mesh_from_cadquery / mesh_from_geometry passthrough)+# ===========================================================================+def _to_shape(model):+ """Coerce a cadquery object (Workplane/Shape/Assembly) or a shim geometry+ wrapper into a single cq.Shape."""+ if isinstance(model, _CQMesh):+ return model.shape+ if hasattr(model, "_cq"):+ return model._cq()+ if isinstance(model, cq.Assembly):+ return model.toCompound()+ if isinstance(model, cq.Workplane):+ vals = [v for v in model.vals() if isinstance(v, cq.Shape)]+ if not vals:+ raise ValidationError("cadquery Workplane has no shape values")+ return vals[0] if len(vals) == 1 else cq.Compound.makeCompound(vals)+ if isinstance(model, cq.Shape):+ return model+ raise ValidationError("unsupported cadquery model type: %r" % type(model))+++class _CQMesh:+ """Mesh stand-in that simply carries an exact cadquery B-rep shape."""+ __slots__ = ("shape", "name")++ def __init__(self, shape, name=None):+ self.shape = shape+ self.name = name++ def _cq(self):+ return self.shape+++# alias name some models expect+Mesh = _CQMesh+++def _scale_shape(shape, s):+ """Honor unit_scale (e.g. models authored in mm, exported to meters with+ unit_scale=0.001): uniformly scale the B-rep about the origin."""+ if not s or s == 1.0 or shape is None:+ return shape+ try:+ return shape.transformGeometry(cq.Matrix([[s, 0, 0, 0], [0, s, 0, 0], [0, 0, s, 0]]))+ except Exception:+ return shape+++def mesh_from_cadquery(model, name=None, **kwargs):+ return _CQMesh(_scale_shape(_to_shape(model), kwargs.get("unit_scale", 1.0)),+ name=name if isinstance(name, str) else None)+++def mesh_from_geometry(geometry, name=None, **kwargs):+ return _CQMesh(_scale_shape(_to_shape(geometry), kwargs.get("unit_scale", 1.0)),+ name=name if isinstance(name, str) else None)+++def mesh_from_input(name):+ raise ValidationError("mesh_from_input (external asset) is not supported in self-contained mode")+++# ===========================================================================+# @section core-model (Visual / Part / Articulation / ArticulatedObject)+# ===========================================================================+class Visual:+ __slots__ = ("geometry", "origin", "material", "name")++ def __init__(self, geometry, origin=None, material=None, name=None):+ self.geometry = geometry+ self.origin = origin or Origin()+ self.material = material+ self.name = name+++class Part:+ def __init__(self, name, visuals=None, inertial=None, meta=None):+ self.name = name+ self.visuals = list(visuals or [])+ self.collisions = []+ self.inertial = inertial+ self.meta = dict(meta or {})++ def visual(self, geometry, *, origin=None, material=None, color=None, name=None):+ if color is not None and material is None:+ material = color if isinstance(color, (str, Material)) else Material("inline", rgba=color)+ v = Visual(geometry, origin=origin, material=material, name=name)+ self.visuals.append(v)+ return v++ def get_visual(self, name):+ for v in self.visuals:+ if v.name == name:+ return v+ raise ValidationError("unknown visual %r" % name)+++class Articulation:+ def __init__(self, name, articulation_type, parent, child, origin=None,+ axis=(0.0, 0.0, 1.0), motion_limits=None, motion_properties=None,+ mimic=None, meta=None):+ self.name = name+ self.articulation_type = articulation_type+ self.parent = parent if isinstance(parent, str) else parent.name+ self.child = child if isinstance(child, str) else child.name+ self.origin = origin or Origin()+ self.axis = tuple(float(a) for a in axis)+ self.motion_limits = motion_limits+ self.motion_properties = motion_properties+ self.mimic = mimic+ self.meta = dict(meta or {})++ # aliases used by some scripts+ @property+ def joint_type(self):+ return self.articulation_type++ @property+ def limit(self):+ return self.motion_limits+++class ArticulatedObject:+ def __init__(self, name, parts=None, articulations=None, materials=None, meta=None, assets=None):+ self.name = name+ self.parts = list(parts or [])+ self.articulations = list(articulations or [])+ self.materials = list(materials or [])+ self.meta = dict(meta or {})+ self.assets = assets++ # ---- builders ----+ def part(self, name, *, visuals=None, inertial=None, meta=None):+ p = Part(name, visuals=visuals, inertial=inertial, meta=meta)+ self.parts.append(p)+ return p++ link = part++ def articulation(self, name, articulation_type, parent, child, *, origin=None,+ axis=None, motion_limits=None, motion_properties=None, mimic=None, meta=None):+ a = Articulation(name, articulation_type, parent, child, origin=origin,+ axis=axis or (0.0, 0.0, 1.0), motion_limits=motion_limits,+ motion_properties=motion_properties, mimic=mimic, meta=meta)+ self.articulations.append(a)+ return a++ def joint(self, name, joint_type, parent, child, *, origin=None, axis=None,+ limit=None, dynamics=None, mimic=None, meta=None):+ return self.articulation(name, joint_type, parent, child, origin=origin, axis=axis,+ motion_limits=limit, motion_properties=dynamics, mimic=mimic, meta=meta)++ def material(self, name, *, rgba=None, color=None, texture=None):+ m = Material(name, rgba=rgba, color=color, texture=texture)+ self.materials.append(m)+ return m++ def set_assets(self, assets):+ self.assets = assets+ return assets++ # ---- queries ----+ def get_part(self, name):+ key = name if isinstance(name, str) else name.name+ for p in self.parts:+ if p.name == key:+ return p+ raise ValidationError("unknown part %r" % name)++ get_link = get_part++ def get_articulation(self, name):+ key = name if isinstance(name, str) else getattr(name, "name", name)+ for a in self.articulations:+ if a.name == key:+ return a+ raise ValidationError("unknown articulation %r" % name)++ get_joint = get_articulation++ def root_parts(self):+ children = {a.child for a in self.articulations}+ return [p for p in self.parts if p.name not in children]++ root_links = root_parts++ def validate(self, *a, **k):+ return None++ def to_urdf(self, *a, **k):+ raise ValidationError("URDF export is not available in self-contained CadQuery mode")++ # ---- rest-pose world transform per part (URDF forward kinematics, q=0) ----+ def _world_transforms(self):+ parent_of, joint_tf = {}, {}+ for a in self.articulations:+ parent_of[a.child] = a.parent+ joint_tf[a.child] = _Tf.from_origin(a.origin)+ cache = {}++ def world(name):+ if name in cache:+ return cache[name]+ if name in parent_of:+ tf = world(parent_of[name]) @ joint_tf[name]+ else:+ tf = _Tf()+ cache[name] = tf+ return tf++ return {p.name: world(p.name) for p in self.parts}++ # ---- assemble to a CadQuery Assembly in rest pose ----+ def to_cq(self):+ worlds = self._world_transforms()+ asm = cq.Assembly(name=self.name or "model")+ idx = 0+ for part in self.parts:+ wt = worlds.get(part.name, _Tf())+ for vis in part.visuals:+ try:+ shape = _to_shape(vis.geometry)+ except Exception as exc:+ raise ValidationError(+ "part %r visual %r: %s" % (part.name, vis.name, exc)+ )+ loc = (wt @ _Tf.from_origin(vis.origin)).location()+ rgba = None+ mat = vis.material+ if isinstance(mat, str):+ mat = next((m for m in self.materials if m.name == mat), None)+ if isinstance(mat, Material) and mat.rgba:+ rgba = mat.rgba+ kw = {"name": "%s_%d" % (part.name, idx)}+ if rgba:+ kw["color"] = cq.Color(*rgba)+ asm.add(shape, loc=loc, **kw)+ idx += 1+ return asm++ # convenience aliases a few scripts might call+ def to_assembly(self):+ return self.to_cq()++ def compound(self):+ return self.to_cq().toCompound()+++# ===========================================================================+# @section core-testing-stubs (TestContext/TestReport: never executed for+# geometry, but must be importable; run_tests() is not called by the epilogue)+# ===========================================================================+class _Nullctx:+ def __enter__(self):+ return self++ def __exit__(self, *a):+ return False+++class TestReport:+ def __init__(self, *a, **k):+ self.checks = []+ self.passed = True+++class TestContext:+ def __init__(self, *a, **k):+ pass++ def __getattr__(self, _name):+ def _noop(*a, **k):+ return None+ return _noop++ def pose(self, *a, **k):+ return _Nullctx()++ def report(self, *a, **k):+ return TestReport()+++class ValidationReport(TestReport):+ pass+++# ===========================================================================+# @section core-assets-stubs (AssetContext etc.: not needed for geometry)+# ===========================================================================+class _NoOp:+ """Universal no-op sentinel that is callable AND Path-like, so asset/IO+ expressions in model scripts (e.g. ASSETS.mesh_dir.mkdir(); dir / 'x.obj')+ are harmless in self-contained mode."""+ def __call__(self, *a, **k):+ return self++ def __getattr__(self, _n):+ return self++ def __truediv__(self, _o):+ return self++ def __rtruediv__(self, _o):+ return self++ def __fspath__(self):+ return "/tmp/_articraft_noop"++ def mkdir(self, *a, **k):+ return None+++_NOOP = _NoOp()+++class _NoopMeta(type):+ def __getattr__(cls, _name):+ def _noop(*a, **k):+ return cls()+ return _noop+++class AssetContext(metaclass=_NoopMeta):+ def __init__(self, *a, **k):+ pass++ def __getattr__(self, _n):+ return _NOOP+++class AssetSession(AssetContext):+ pass+++def coerce_asset_context(*a, **k):+ return None+++def resolve_asset_context(*a, **k):+ return None+++class AssetContextLike(AssetContext):+ pass+++# ===========================================================================+# @section geom-base (MeshGeometry base shared by all geometry families)+# ===========================================================================+def _solid_from_triangles(vertices, faces):+ """Fallback: wrap a raw triangle soup as a B-rep solid (for MeshGeometry+ instances that are authored vertex/face directly)."""+ tri_faces = []+ for (a, b, c) in faces:+ va, vb, vc = vertices[a], vertices[b], vertices[c]+ if va == vb or vb == vc or vc == va:+ continue+ w = cq.Wire.makePolygon([cq.Vector(*va), cq.Vector(*vb), cq.Vector(*vc)], close=True)+ try:+ tri_faces.append(cq.Face.makeFromWires(w))+ except Exception:+ continue+ if not tri_faces:+ raise ValidationError("MeshGeometry: no valid triangles to build a solid")+ shell = cq.Shell.makeShell(tri_faces)+ try:+ return cq.Solid.makeSolid(shell)+ except Exception:+ return cq.Compound.makeCompound(tri_faces)+++class MeshGeometry:+ """Base for all geometry families. Subclasses build parametric CadQuery+ B-rep and call ``self._set(shape)``. Supports the mesh-style manipulation+ API (translate/rotate/scale/merge/copy) used by Articraft model scripts,+ applied to the underlying CadQuery shape."""++ def __init__(self, vertices=None, faces=None):+ self._raw_verts = [tuple(v) for v in (vertices or [])]+ self._raw_faces = [tuple(f) for f in (faces or [])]+ self._shape = None+ self._tess = None++ # subclasses call this with their cadquery shape+ def _set(self, shape):+ self._shape = _to_shape(shape) if not isinstance(shape, cq.Shape) else shape+ self._tess = None+ return self++ def _tessellate(self):+ """Triangle mesh (vertices, faces) of the B-rep — some model scripts read+ .vertices/.faces to manipulate raw geometry (e.g. axis swaps)."""+ if self._tess is None:+ if self._shape is None:+ self._tess = (list(self._raw_verts), list(self._raw_faces))+ else:+ try:+ vs, ts = self._shape.tessellate(0.0005)+ self._tess = ([(v.x, v.y, v.z) for v in vs],+ [tuple(int(i) for i in t) for t in ts])+ except Exception:+ self._tess = (list(self._raw_verts), list(self._raw_faces))+ return self._tess++ @property+ def vertices(self):+ return self._raw_verts if self._raw_verts else self._tessellate()[0]++ @vertices.setter+ def vertices(self, v):+ # capture the B-rep's faces first so a vertices-ONLY reassignment+ # (read mesh -> transform verts -> assign back) doesn't lose the faces+ if self._shape is not None and not self._raw_faces:+ self._raw_faces = list(self._tessellate()[1])+ self._raw_verts = [tuple(x) for x in (v or [])]+ self._shape = None; self._tess = None++ @property+ def faces(self):+ return self._raw_faces if self._raw_faces else self._tessellate()[1]++ @faces.setter+ def faces(self, f):+ if self._shape is not None and not self._raw_verts:+ self._raw_verts = list(self._tessellate()[0])+ self._raw_faces = [tuple(x) for x in (f or [])]+ self._shape = None; self._tess = None++ def _cq(self):+ if self._shape is not None:+ return self._shape+ return _solid_from_triangles(self._raw_verts, self._raw_faces)++ # ---- mesh-style authoring (raw triangle fallback path) ----+ def add_vertex(self, x, y, z):+ self._raw_verts.append((float(x), float(y), float(z)))+ self._shape = None; self._tess = None+ return len(self._raw_verts) - 1++ def add_face(self, a, b, c):+ self._raw_faces.append((int(a), int(b), int(c)))+ self._shape = None; self._tess = None++ def copy(self):+ g = MeshGeometry(self._raw_verts, self._raw_faces)+ g._shape = self._shape+ return g++ clone = copy++ # ---- rigid / affine transforms (operate on the cadquery shape) ----+ def translate(self, dx, dy=None, dz=None):+ if dy is None:+ dx, dy, dz = dx+ self._shape = self._cq().translate(cq.Vector(float(dx), float(dy), float(dz)))+ return self++ def rotate(self, axis, angle, *, origin=(0.0, 0.0, 0.0)):+ # Articraft angles are RADIANS; cadquery rotate takes degrees about a line.+ o = cq.Vector(float(origin[0]), float(origin[1]), float(origin[2]))+ a = cq.Vector(float(axis[0]), float(axis[1]), float(axis[2]))+ self._shape = self._cq().rotate(o, o + a, math.degrees(float(angle)))+ return self++ def scale(self, sx, sy=None, sz=None):+ if sy is None:+ sy = sz = sx+ if sz is None:+ sz = sx+ m = cq.Matrix([[float(sx), 0, 0, 0], [0, float(sy), 0, 0], [0, 0, float(sz), 0]])+ self._shape = self._cq().transformGeometry(m)+ return self++ def rotate_x(self, angle):+ return self.rotate((1.0, 0.0, 0.0), angle)++ def rotate_y(self, angle):+ return self.rotate((0.0, 1.0, 0.0), angle)++ def rotate_z(self, angle):+ return self.rotate((0.0, 0.0, 1.0), angle)++ def merge(self, other):+ o = _to_shape(other)+ if self._shape is None and not self._raw_verts:+ self._shape = o # empty accumulator: adopt other's shape+ else:+ a = self._cq()+ try:+ fused = a.fuse(o)+ if fused is None or fused.wrapped is None or fused.wrapped.IsNull():+ raise ValueError("null fuse result")+ self._shape = fused+ except Exception:+ # OCC fuse can fail/return null on many thin/degenerate parts+ # (e.g. a balloon whisk's wire loops) — fall back to a non-boolean+ # Compound, which is equivalent for surface sampling / rendering.+ self._shape = cq.Compound.makeCompound([a, o])+ self._tess = None+ return self++ union = merge++ def cut(self, other):+ self._shape = self._cq().cut(_to_shape(other))+ return self++ def difference(self, other):+ return self.cut(other)++ def intersect(self, other):+ self._shape = self._cq().intersect(_to_shape(other))+ return self+++# boolean free-functions used by some scripts+def boolean_difference(a, b, *more):+ s = _to_shape(a).cut(_to_shape(b))+ for m in more:+ s = s.cut(_to_shape(m))+ return MeshGeometry()._set(s)+++def boolean_union(a, *more):+ s = _to_shape(a)+ for m in more:+ s = s.fuse(_to_shape(m))+ return MeshGeometry()._set(s)+++def boolean_intersection(a, b, *more):+ s = _to_shape(a).intersect(_to_shape(b))+ for m in more:+ s = s.intersect(_to_shape(m))+ return MeshGeometry()._set(s)+++def repair_loft(geometry, *a, **k):+ return geometry+++# ===========================================================================+# @section geom-profiles (pure-python 2D profile generators, verbatim from+# the Apache-2.0 Articraft SDK _mesh/common.py)+# ===========================================================================+_EPS = 1e-9+++def _points_match_2d(a, b, tol=1e-9):+ return abs(a[0] - b[0]) <= tol and abs(a[1] - b[1]) <= tol+++def _polygon_area(points):+ area = 0.0+ n = len(points)+ for i in range(n):+ x1, y1 = points[i]+ x2, y2 = points[(i + 1) % n]+ area += x1 * y2 - x2 * y1+ return area * 0.5+++def _ensure_ccw(points):+ area = _polygon_area(points)+ if abs(area) <= 1e-9:+ raise ValidationError("Profile area must be non-zero")+ return list(reversed(points)) if area < 0 else points+++def _profile_points_2d(profile):+ points = [(float(x), float(y)) for (x, y) in profile]+ cleaned = []+ for pt in points:+ if not cleaned or not _points_match_2d(cleaned[-1], pt):+ cleaned.append(pt)+ if len(cleaned) >= 2 and _points_match_2d(cleaned[0], cleaned[-1]):+ cleaned = cleaned[:-1]+ if len(cleaned) < 3:+ raise ValidationError("Profile must have at least 3 unique points")+ return cleaned+++def _polyline_points_2d(profile):+ points = [(float(x), float(y)) for (x, y) in profile]+ cleaned = []+ for pt in points:+ if not cleaned or not _points_match_2d(cleaned[-1], pt):+ cleaned.append(pt)+ if len(cleaned) < 2:+ raise ValidationError("Profile must have at least 2 unique points")+ return cleaned+++def _profile_points_3d(profile):+ points = [(float(x), float(y), float(z)) for (x, y, z) in profile]+ cleaned = []+ for pt in points:+ if not cleaned or not (abs(cleaned[-1][0] - pt[0]) <= 1e-9 and abs(cleaned[-1][1] - pt[1]) <= 1e-9 and abs(cleaned[-1][2] - pt[2]) <= 1e-9):+ cleaned.append(pt)+ if len(cleaned) >= 2 and abs(cleaned[0][0] - cleaned[-1][0]) <= 1e-9 and abs(cleaned[0][1] - cleaned[-1][1]) <= 1e-9 and abs(cleaned[0][2] - cleaned[-1][2]) <= 1e-9:+ cleaned = cleaned[:-1]+ if len(cleaned) < 2:+ raise ValidationError("Profile must have at least 2 unique points")+ return cleaned+++def _polygon_centroid(points):+ a = _polygon_area(points)+ if abs(a) <= _EPS:+ n = len(points)+ return (sum(p[0] for p in points) / n, sum(p[1] for p in points) / n)+ cx = cy = 0.0+ n = len(points)+ for i in range(n):+ x1, y1 = points[i]+ x2, y2 = points[(i + 1) % n]+ cr = x1 * y2 - x2 * y1+ cx += (x1 + x2) * cr+ cy += (y1 + y2) * cr+ return (cx / (6.0 * a), cy / (6.0 * a))+++def _point_in_polygon(pt, poly):+ x, y = pt+ inside = False+ n = len(poly)+ j = n - 1+ for i in range(n):+ xi, yi = poly[i]+ xj, yj = poly[j]+ if ((yi > y) != (yj > y)) and (x < (xj - xi) * (y - yi) / (yj - yi + 1e-30) + xi):+ inside = not inside+ j = i+ return inside+++def superellipse_profile(width, height, exponent=2.6, *, segments=48):+ width = float(width); height = float(height); exponent = float(exponent)+ segments = max(12, int(segments))+ if width <= 0 or height <= 0:+ raise ValidationError("width/height must be positive")+ if exponent <= 0:+ raise ValidationError("exponent must be positive")+ a = width * 0.5; b = height * 0.5; power = 2.0 / exponent+ pts = []+ for i in range(segments):+ t = 2.0 * math.pi * i / segments+ ct = math.cos(t); st = math.sin(t)+ x = a * (abs(ct) ** power); y = b * (abs(st) ** power)+ if ct < 0:+ x = -x+ if st < 0:+ y = -y+ pts.append((x, y))+ return pts+++def rounded_rect_profile(width, height, radius, *, corner_segments=6):+ width = float(width); height = float(height); radius = float(radius)+ if width <= 0 or height <= 0:+ raise ValidationError("width/height must be positive")+ corner_segments = max(1, int(corner_segments))+ max_r = 0.5 * min(width, height)+ if radius < 0 or radius > max_r:+ raise ValidationError("radius must be within [0, %s]" % max_r)+ hw = width * 0.5; hh = height * 0.5+ if radius <= 1e-12:+ return [(-hw, -hh), (hw, -hh), (hw, hh), (-hw, hh)]+ r = radius++ def arc(cx, cy, a0, a1):+ out = []+ for i in range(corner_segments + 1):+ t = i / corner_segments+ a = a0 + (a1 - a0) * t+ out.append((cx + r * math.cos(a), cy + r * math.sin(a)))+ return out++ pts = []+ pts.extend(arc(hw - r, -hh + r, -math.pi / 2.0, 0.0))+ pts.extend(arc(hw - r, hh - r, 0.0, math.pi / 2.0)[1:])+ pts.extend(arc(-hw + r, hh - r, math.pi / 2.0, math.pi)[1:])+ pts.extend(arc(-hw + r, -hh + r, math.pi, 3.0 * math.pi / 2.0)[1:])+ return pts+++# ---- cadquery profile/solid builders shared by geometry families ----+def _wire_2d(points, plane="XY", close=True):+ """Build a cq.Wire from a list of 2D points placed on the given plane."""+ pts = [(float(x), float(y)) for (x, y) in points]+ wp = cq.Workplane(plane).polyline(pts)+ if close:+ wp = wp.close()+ return wp.wires().val()+++def _extrude_profile(points, height, *, center=True, z0=None):+ """Extrude a closed 2D profile (XY) by height along +Z. Returns cq.Solid."""+ pts = _ensure_ccw(_profile_points_2d(points))+ face = cq.Face.makeFromWires(cq.Wire.makePolygon([cq.Vector(x, y, 0) for (x, y) in pts], close=True))+ solid = cq.Solid.extrudeLinear(face, cq.Vector(0, 0, float(height)))+ dz = (-height / 2.0 if center else 0.0) if z0 is None else z0+ if dz:+ solid = solid.translate(cq.Vector(0, 0, dz))+ return solid+++def _revolve_profile(profile_rz, *, angle=360.0):+ """Revolve a closed (r,z) profile about the Z axis. r=radius>=0, z=height.+ Matches Articraft LatheGeometry (profile point (r,z) -> circle of radius r+ at height z)."""+ pts = [(float(r), float(z)) for (r, z) in profile_rz]+ wire = cq.Wire.makePolygon([cq.Vector(r, 0.0, z) for (r, z) in pts], close=True)+ return cq.Solid.revolve(wire, [], float(angle), cq.Vector(0, 0, 0), cq.Vector(0, 0, 1))+++def _loft_rings(rings, *, cap=True):+ """Loft through a list of rings; each ring is a list of 3D points (closed+ loop). Returns cq.Solid (capped) or shell."""+ wires = [cq.Wire.makePolygon([cq.Vector(*p) for p in ring], close=True) for ring in rings]+ return cq.Solid.makeLoft(wires, ruled=False)diff --git a/toolkits/curation/articraft/assemble_shim.py b/toolkits/curation/articraft/assemble_shim.pynew file mode 100644index 00000000..5434485f--- /dev/null+++ b/toolkits/curation/articraft/assemble_shim.py@@ -0,0 +1,113 @@+#!/usr/bin/env python3+"""Assemble the self-contained CadQuery shim from the core + per-family modules.++- assemble(families=None) -> str : combined shim source (core + selected family+ modules), with each family module's `from articraft_cq import (...)` and+ `from __future__` lines stripped (the core already provides those names).+ families=None -> ALL families found in tools/shim_parts/.+- name2family : map of every geometry name -> its family module stem.++CLI: python tools/assemble_shim.py # write tools/_shimpkg/sdk.py (all families)+ python tools/assemble_shim.py --names # print name -> family map+"""+import os, re, ast, sys++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+TOOLS = _HERE+CORE = TOOLS + "/articraft_cq.py"+PARTS = TOOLS + "/shim_parts"++_IMPORT_CORE_RE = re.compile(r"^from articraft_cq import\b")+_FUTURE_RE = re.compile(r"^from __future__ import\b")+++def _strip_core_imports(src):+ """Remove `from articraft_cq import ...` (incl. multi-line parenthesised)+ and `from __future__ ...` lines from a family module."""+ lines = src.splitlines()+ out = []+ i = 0+ while i < len(lines):+ ln = lines[i]+ if _FUTURE_RE.match(ln):+ i += 1+ continue+ if _IMPORT_CORE_RE.match(ln):+ # consume continuation lines if parenthesised import+ if "(" in ln and ")" not in ln:+ i += 1+ while i < len(lines) and ")" not in lines[i]:+ i += 1+ i += 1 # skip the line with ')'+ else:+ i += 1+ continue+ out.append(ln)+ i += 1+ return "\n".join(out)+++def _family_files(families=None):+ if not os.path.isdir(PARTS):+ return []+ stems = sorted(f[:-3] for f in os.listdir(PARTS) if f.endswith(".py") and not f.startswith("_"))+ if families is not None:+ stems = [s for s in stems if s in set(families)]+ return [(s, os.path.join(PARTS, s + ".py")) for s in stems]+++def defined_names(src):+ """Top-level class/def/assignment names defined in a source string."""+ names = []+ try:+ tree = ast.parse(src)+ except Exception:+ return names+ for node in tree.body:+ if isinstance(node, (ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef)):+ names.append(node.name)+ elif isinstance(node, ast.Assign):+ for t in node.targets:+ if isinstance(t, ast.Name):+ names.append(t.id)+ return names+++def name2family():+ m = {}+ for stem, path in _family_files():+ src = open(path).read()+ for n in defined_names(src):+ if not n.startswith("_"):+ m[n] = stem+ return m+++def assemble(families=None):+ core = open(CORE).read()+ # core keeps its own __future__ at the very top; ensure it's first+ parts = [core]+ for stem, path in _family_files(families):+ body = _strip_core_imports(open(path).read())+ parts.append("\n\n# ===== family: %s =====\n%s" % (stem, body))+ return "\n".join(parts)+++def main():+ if "--names" in sys.argv:+ import json+ print(json.dumps(name2family(), indent=1, ensure_ascii=False))+ return+ os.makedirs(TOOLS + "/_shimpkg", exist_ok=True)+ text = assemble()+ with open(TOOLS + "/_shimpkg/sdk.py", "w") as f:+ f.write(text)+ nm = name2family()+ print("wrote tools/_shimpkg/sdk.py (%d chars)" % len(text))+ print("families:", sorted(set(nm.values())))+ print("geometry names exposed:", len(nm))+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/articraft/batch_verify.py b/toolkits/curation/articraft/batch_verify.pynew file mode 100644index 00000000..1e01ab0d--- /dev/null+++ b/toolkits/curation/articraft/batch_verify.py@@ -0,0 +1,131 @@+#!/usr/bin/env python3+"""Batch: generate self-contained CadQuery code.py for every with_mesh sample,+verify chamfer vs HF ground truth, update meta.json. Resumable & parallel.++For each sample:+ 1. If articraft_source_code.py missing -> cp code.py to it (preserve SDK source).+ 2. gen_code -> write self-contained code.py (only import cadquery + inlined shim).+ 3. verify_cadquery.py (subprocess) -> chamfer / bbox / pass.+ 4. update meta.json: language/entry/environment/aux_source/conversion/status.++Usage: python tools/batch_verify.py [N|all] [--workers W] [--force] [--shard i/k]+ default: process only samples not yet verifier_pass (resumable).+"""+import os, sys, json, subprocess, shutil, time, hashlib+from concurrent.futures import ThreadPoolExecutor, as_completed++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+DD = ROOT + "/data/articraft_with_mesh"+sys.path.insert(0, _HERE)+import gen_code++# cap per-subprocess thread fan-out so N workers don't each grab all cores+_ENV = dict(os.environ, OMP_NUM_THREADS="1", OPENBLAS_NUM_THREADS="1",+ MKL_NUM_THREADS="1", NUMEXPR_NUM_THREADS="1", VECLIB_MAXIMUM_THREADS="1")++argv = sys.argv[1:]+N = 0+WORKERS = 10+FORCE = False+SHARD = None+_i = 0+while _i < len(argv):+ a = argv[_i]+ if a == "--workers":+ WORKERS = int(argv[_i + 1]); _i += 2; continue+ if a == "--shard":+ _x, _k = argv[_i + 1].split("/"); SHARD = (int(_x), int(_k)); _i += 2; continue+ if a == "--force":+ FORCE = True; _i += 1; continue+ if a.isdigit():+ N = int(a)+ _i += 1+++def done(name):+ try:+ m = json.load(open(f"{DD}/{name}/meta.json"))+ return (m.get("conversion") or {}).get("verifier_pass") is True+ except Exception:+ return False+++def process(name):+ d = f"{DD}/{name}"+ src = f"{d}/articraft_source_code.py"+ try:+ if not os.path.exists(src):+ shutil.copy(f"{d}/code.py", src) # preserve original SDK source+ txt, fams = gen_code.gen_code_text(d)+ with open(f"{d}/code.py", "w") as f:+ f.write(txt)+ except Exception as e:+ return name, {"pass": False, "error": "gen:%s" % e, "families": []}+ try:+ r = subprocess.run([sys.executable, f"{ROOT}/tools/verify_cadquery.py", d, f"{d}/code.py",+ "--glb", f"{d}/renders/object.glb"],+ capture_output=True, text=True, timeout=420, env=_ENV)+ res = json.loads(r.stdout.strip().splitlines()[-1])+ except Exception as e:+ res = {"pass": False, "chamfer": None, "bbox_ratio": None, "error": "verify:%s" % e}+ try:+ mp = f"{d}/meta.json"+ meta = json.load(open(mp))+ meta["language"] = "CadQuery (Python)"+ meta["entry"] = "code.py"+ meta["environment"] = "CadQuery (pip install cadquery)"+ meta["aux_source"] = "articraft_source_code.py (Articraft SDK source)"+ meta["conversion"] = {+ "chamfer": res.get("chamfer"), "bbox_ratio": res.get("bbox_ratio"),+ "volume_ratio": res.get("volume_ratio"), "verifier_pass": bool(res.get("pass")),+ "families": fams, "error": res.get("error"),+ }+ meta["status"] = "curated" if res.get("pass") else "revise"+ json.dump(meta, open(mp, "w"), indent=2, ensure_ascii=False)+ except Exception as e:+ res["meta_error"] = str(e)+ res["families"] = fams+ return name, res+++def main():+ names = sorted(os.listdir(DD))+ if SHARD:+ i, k = SHARD+ names = [n for n in names if int(hashlib.md5(n.encode()).hexdigest(), 16) % k == i]+ if not FORCE:+ names = [n for n in names if not done(n)]+ if N:+ names = names[:N]+ total = len(names)+ gen_code._load() # pre-load shim sources ONCE (avoid thread race on lazy init)+ print(f"processing {total} samples, workers={WORKERS}", flush=True)+ t0 = time.time()+ npass = nfail = 0+ chs = []+ failcauses = {}+ with ThreadPoolExecutor(max_workers=WORKERS) as ex:+ futs = {ex.submit(process, n): n for n in names}+ for k, fut in enumerate(as_completed(futs), 1):+ name, res = fut.result()+ if res.get("pass"):+ npass += 1+ if res.get("chamfer") is not None:+ chs.append(res["chamfer"])+ else:+ nfail += 1+ err = (res.get("error") or "chamfer/bbox_fail")[:45]+ failcauses[err] = failcauses.get(err, 0) + 1+ if k % 25 == 0 or k == total:+ el = time.time() - t0+ avg = sum(chs) / len(chs) if chs else 0+ print(f"[{k}/{total}] pass={npass} fail={nfail} avgChamfer={avg:.4f} "+ f"{el:.0f}s ETA{el/k*(total-k):.0f}s", flush=True)+ el = time.time() - t0+ print(f"\nDONE pass={npass} fail={nfail} ({100*npass/max(total,1):.1f}%) in {el:.0f}s", flush=True)+ print("top fail causes:", sorted(failcauses.items(), key=lambda x: -x[1])[:15], flush=True)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/articraft/blender_glb.py b/toolkits/curation/articraft/blender_glb.pynew file mode 100644index 00000000..5abb02e1--- /dev/null+++ b/toolkits/curation/articraft/blender_glb.py@@ -0,0 +1,73 @@+"""Render a GLB (exported from a CadQuery Assembly) to N azimuth views on a+transparent background, GPU Cycles. Mirrors blender_urdf.py lighting/camera.+Usage: blender --background --threads 4 --python blender_glb.py -- <glb> <out_dir> [res] [samples] [n_azim] [material|clay]+"""+import bpy, sys, os, math+from mathutils import Vector++argv = sys.argv[sys.argv.index("--") + 1:]+GLB, OUT = argv[0], argv[1]+RES = int(argv[2]) if len(argv) > 2 else 1024+SAMPLES = int(argv[3]) if len(argv) > 3 else 16+NAZ = int(argv[4]) if len(argv) > 4 else 8+MODE = argv[5] if len(argv) > 5 else "material"+os.makedirs(OUT, exist_ok=True)++bpy.ops.wm.read_factory_settings(use_empty=True)+scene = bpy.context.scene+bpy.ops.import_scene.gltf(filepath=GLB)+objs = [o for o in scene.objects if o.type == "MESH"]+if not objs:+ print("GLB_RENDER_EMPTY")+ sys.exit(0)++mn = Vector((1e18,) * 3); mx = Vector((-1e18,) * 3)+for ob in objs:+ for c in ob.bound_box:+ w = ob.matrix_world @ Vector(c)+ for k in range(3):+ mn[k] = min(mn[k], w[k]); mx[k] = max(mx[k], w[k])+center = (mn + mx) / 2; radius = max((mx - mn).length / 2, 1e-4)++CLAY = bpy.data.materials.new("clay"); CLAY.use_nodes = True+CLAY.node_tree.nodes["Principled BSDF"].inputs["Base Color"].default_value = (0.82, 0.82, 0.84, 1.0)+CLAY.node_tree.nodes["Principled BSDF"].inputs["Roughness"].default_value = 0.5+if MODE == "clay":+ for ob in objs:+ ob.data.materials.clear(); ob.data.materials.append(CLAY)++w = bpy.data.worlds.new("w"); scene.world = w; w.use_nodes = True+bg = w.node_tree.nodes["Background"]; bg.inputs[0].default_value = (1, 1, 1, 1); bg.inputs[1].default_value = 0.55+sd = bpy.data.lights.new("s", type="SUN"); sd.energy = 3.0; sd.angle = math.radians(8)+so = bpy.data.objects.new("s", sd); scene.collection.objects.link(so)+so.rotation_euler = (math.radians(52), math.radians(12), math.radians(-50))+FOV = math.radians(45)+cd = bpy.data.cameras.new("c"); cd.sensor_fit = "AUTO"; cd.angle = FOV+co = bpy.data.objects.new("c", cd); scene.collection.objects.link(co); scene.camera = co++scene.render.engine = "CYCLES"+prefs = bpy.context.preferences.addons["cycles"].preferences+prefs.compute_device_type = "CUDA"; prefs.refresh_devices()+gpu = any(d.type == "CUDA" for d in prefs.devices)+for d in prefs.devices:+ d.use = (d.type == "CUDA")+scene.cycles.device = "GPU" if gpu else "CPU"+scene.cycles.samples = SAMPLES; scene.cycles.use_denoising = True+try:+ scene.cycles.denoiser = "OPENIMAGEDENOISE"; scene.cycles.denoising_use_gpu = True+except Exception:+ pass+scene.render.resolution_x = RES; scene.render.resolution_y = RES+scene.render.film_transparent = True+scene.render.image_settings.file_format = "PNG"; scene.render.image_settings.color_mode = "RGBA"++dist = radius / math.sin(FOV / 2) * 1.15+elev = math.radians(25)+for i in range(NAZ):+ a = math.radians(i * 360.0 / NAZ)+ eye = center + Vector((dist * math.cos(elev) * math.cos(a), dist * math.cos(elev) * math.sin(a), dist * math.sin(elev)))+ co.location = eye+ co.rotation_euler = (center - eye).to_track_quat("-Z", "Y").to_euler()+ scene.render.filepath = os.path.join(OUT, "view_%02d.png" % i)+ bpy.ops.render.render(write_still=True)+print("GLB_RENDER_DONE objs=%d views=%d mode=%s" % (len(objs), NAZ, MODE))diff --git a/toolkits/curation/articraft/blender_urdf.py b/toolkits/curation/articraft/blender_urdf.pynew file mode 100644index 00000000..975729a1--- /dev/null+++ b/toolkits/curation/articraft/blender_urdf.py@@ -0,0 +1,158 @@+"""解析自包含 URDF(primitive),FK归零位摆放,Blender GPU 渲染多视角,透明背景。+渲两套: view_XX.png(URDF材质彩色) + clay_XX.png(白模)。+用法: blender --background --python blender_urdf.py -- <urdf> <out_dir> [res] [samples] [n_azim]"""+import bpy, sys, os, math, random+import xml.etree.ElementTree as ET+from mathutils import Matrix, Vector++argv=sys.argv[sys.argv.index("--")+1:]+URDF, OUT = argv[0], argv[1]+RES=int(argv[2]) if len(argv)>2 else 1024+SAMPLES=int(argv[3]) if len(argv)>3 else 48+NAZ=int(argv[4]) if len(argv)>4 else 8+MODE=argv[5] if len(argv)>5 else "material" # material | clay | both+os.makedirs(OUT,exist_ok=True)+bpy.ops.wm.read_factory_settings(use_empty=True)+scene=bpy.context.scene++def rpy_m(r,p,y): return Matrix.Rotation(y,4,'Z')@Matrix.Rotation(p,4,'Y')@Matrix.Rotation(r,4,'X')+def origin_m(o):+ if o is None: return Matrix.Identity(4)+ xyz=[float(x) for x in o.get('xyz','0 0 0').split()]+ rpy=[float(x) for x in o.get('rpy','0 0 0').split()]+ return Matrix.Translation(Vector(xyz))@rpy_m(*rpy)++root=ET.parse(URDF).getroot()+# 顶层 material 定义 name->rgba+TOP={}+for m in root.findall('material'):+ c=m.find('color')+ if c is not None and c.get('rgba'): TOP[m.get('name')]=[float(x) for x in c.get('rgba').split()]++def visual_color(v):+ mat=v.find('material')+ if mat is None: return None+ c=mat.find('color')+ if c is not None and c.get('rgba'): return [float(x) for x in c.get('rgba').split()]+ return TOP.get(mat.get('name'))++links={}+for link in root.findall('link'):+ vs=[]+ for v in link.findall('visual'):+ g=v.find('geometry')+ if g is None or len(g)==0: continue+ vs.append((g[0], v.find('origin'), visual_color(v)))+ links[link.get('name')]=vs+joints=[]; children=set()+for j in root.findall('joint'):+ p=j.find('parent').get('link'); c=j.find('child').get('link')+ joints.append((p,c,j.find('origin'))); children.add(c)+roots=[n for n in links if n not in children]+world={}+def dfs(n,T):+ world[n]=T+ for p,c,o in joints:+ if p==n and c not in world: dfs(c,T@origin_m(o))+for r in roots: dfs(r,Matrix.Identity(4))+for n in links: world.setdefault(n,Matrix.Identity(4))++def mk_mat(rgba):+ m=bpy.data.materials.new("m"); m.use_nodes=True+ b=m.node_tree.nodes.get("Principled BSDF")+ b.inputs["Base Color"].default_value=tuple(rgba)+ b.inputs["Roughness"].default_value=0.5+ return m+matcache={}+def color_mat(color):+ key=tuple(color) if color else (0.8,0.8,0.82,1.0)+ if len(key)==3: key=key+(1.0,)+ if key not in matcache: matcache[key]=mk_mat(key)+ return matcache[key]+CLAY=mk_mat((0.82,0.82,0.84,1.0))++objs=[] # (obj, color_material)+for ln,vs in links.items():+ LW=world[ln]+ for geo,o,color in vs:+ VW=LW@origin_m(o); t=geo.tag+ if t=='box':+ s=[float(x) for x in geo.get('size').split()]+ bpy.ops.mesh.primitive_cube_add(size=1); ob=bpy.context.active_object+ [email protected]((s[0],s[1],s[2],1))+ elif t=='cylinder':+ r=float(geo.get('radius')); l=float(geo.get('length'))+ bpy.ops.mesh.primitive_cylinder_add(radius=1,depth=1,vertices=48); ob=bpy.context.active_object+ [email protected]((r,r,l,1))+ elif t=='sphere':+ r=float(geo.get('radius'))+ bpy.ops.mesh.primitive_uv_sphere_add(radius=1,segments=48,ring_count=24); ob=bpy.context.active_object+ [email protected]((r,r,r,1))+ else: continue+ cm=color_mat(color); ob.data.materials.append(cm)+ objs.append((ob,cm))++mn=Vector((1e18,)*3); mx=Vector((-1e18,)*3)+for ob,_ in objs:+ for c in ob.bound_box:+ w=ob.matrix_world@Vector(c)+ for k in range(3): mn[k]=min(mn[k],w[k]); mx[k]=max(mx[k],w[k])+center=(mn+mx)/2; radius=max((mx-mn).length/2, 1e-4)++# 破除"精确共面/重合面"导致的黑块: 给每个 visual 加微小确定性位移抖动。+# URDF 由大量 box 堆叠拼装, 相邻部件常共享完全重合的面; Cycles 下深度竞争中+# "胜出"的面会落在相邻实体内部而被自遮挡 → 渲成纯黑方块。亚毫米级抖动让重合面+# 明确分到"内部(被正常遮挡隐藏)"或"外部(正常受光)", 实测可消除 ~90% 黑块。+# 幅度按模型尺度自适应(对小物体按比例更小), 视觉上不可见。固定 seed 保证可复现。+random.seed(12345)+JIT=radius*0.002+for ob,_ in objs:+ ob.matrix_world=Matrix.Translation(Vector((random.uniform(-JIT,JIT) for _ in range(3))))@ob.matrix_world++w=bpy.data.worlds.new("w"); scene.world=w; w.use_nodes=True+bg=w.node_tree.nodes["Background"]; bg.inputs[0].default_value=(1,1,1,1); bg.inputs[1].default_value=0.55+sd=bpy.data.lights.new("s",type='SUN'); sd.energy=3.0; sd.angle=math.radians(8)+so=bpy.data.objects.new("s",sd); scene.collection.objects.link(so)+so.rotation_euler=(math.radians(52),math.radians(12),math.radians(-50))+FOV=math.radians(45)+cd=bpy.data.cameras.new("c"); cd.sensor_fit='AUTO'; cd.angle=FOV+co=bpy.data.objects.new("c",cd); scene.collection.objects.link(co); scene.camera=co++scene.render.engine='CYCLES'+prefs=bpy.context.preferences.addons['cycles'].preferences+prefs.compute_device_type='CUDA'; prefs.refresh_devices()+gpu=any(d.type=='CUDA' for d in prefs.devices)+for d in prefs.devices: d.use=(d.type=='CUDA')+scene.cycles.device='GPU' if gpu else 'CPU'+scene.cycles.samples=SAMPLES; scene.cycles.use_denoising=True+try:+ scene.cycles.denoiser='OPENIMAGEDENOISE'; scene.cycles.denoising_use_gpu=True # GPU降噪,省CPU+except Exception: pass+scene.render.resolution_x=RES; scene.render.resolution_y=RES+scene.render.film_transparent=True+scene.render.image_settings.file_format='PNG'; scene.render.image_settings.color_mode='RGBA'++dist=radius/math.sin(FOV/2)*1.15 # bounding-sphere fit, 完整入画+elev=math.radians(25)+poses=[]+for i in range(NAZ):+ a=math.radians(i*360.0/NAZ)+ eye=center+Vector((dist*math.cos(elev)*math.cos(a),dist*math.cos(elev)*math.sin(a),dist*math.sin(elev)))+ poses.append(eye)++def render_set(prefix):+ for i,eye in enumerate(poses):+ co.location=eye; co.rotation_euler=(center-eye).to_track_quat('-Z','Y').to_euler()+ scene.render.filepath=os.path.join(OUT,f"{prefix}_{i:02d}.png")+ bpy.ops.render.render(write_still=True)++if MODE=="clay":+ for ob,_ in objs: ob.data.materials.clear(); ob.data.materials.append(CLAY)+ render_set("view") # 白模, 命名 view_XX+elif MODE=="both":+ render_set("view") # 彩色+ for ob,_ in objs: ob.data.materials.clear(); ob.data.materials.append(CLAY)+ render_set("clay") # 白模+else: # material+ render_set("view") # 彩色(URDF材质), 命名 view_XX+print(f"URDF_RENDER_DONE objs={len(objs)} views={NAZ} mode={MODE} colored_mats={len(matcache)}")diff --git a/toolkits/curation/articraft/caption_cadquery.py b/toolkits/curation/articraft/caption_cadquery.pynew file mode 100644index 00000000..6d328dd0--- /dev/null+++ b/toolkits/curation/articraft/caption_cadquery.py@@ -0,0 +1,241 @@+#!/usr/bin/env python3+"""Generate a natural USER-INSTRUCTION caption for each with_mesh (CadQuery-code)+sample with Gemini Flash, and add it to that sample's captions.json.++The instruction is phrased as what a user would type to ask an AI to GENERATE+this 3D object as parametric CadQuery (Python) code (the with_mesh deliverable is+a self-contained CadQuery program; the rendered object keeps per-part colors, so+it INCLUDES materials/colors). Multimodal: 2 rendered views + the model-body+source (the geometry/build code, not the inlined shim). Cover object identity,+key parts, articulation, and materials.++Parallel (network-bound), resumable (skips samples that already have the field),+retries on rate-limit. Writes captions.json[FIELD].++Usage:+ GEMINI_API_KEY=... python tools/caption_cadquery.py [N] [--workers W] [--model M]+ [--field F] [--force]+"""+import os, sys, json, time, re, threading+from concurrent.futures import ThreadPoolExecutor, as_completed++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+DD = ROOT + "/data/articraft_with_mesh"++argv = sys.argv[1:]+N = 0; WORKERS = 600; MODEL = "gemini-3.5-flash"; FIELD = "user_instruction_gemini_3_5_flash"; FORCE = False; RPM = 900+_i = 0+while _i < len(argv):+ a = argv[_i]+ if a == "--workers": WORKERS = int(argv[_i + 1]); _i += 2; continue+ if a == "--model": MODEL = argv[_i + 1]; _i += 2; continue+ if a == "--field": FIELD = argv[_i + 1]; _i += 2; continue+ if a == "--rpm": RPM = int(argv[_i + 1]); _i += 2; continue+ if a == "--force": FORCE = True; _i += 1; continue+ if a.isdigit(): N = int(a)+ _i += 1++KEY = os.environ.get("GEMINI_API_KEY") or os.environ.get("GOOGLE_API_KEY")+if not KEY:+ kp = os.path.expanduser("~/.gemini_api_key")+ if os.path.exists(kp):+ KEY = open(kp).read().strip()+if not KEY:+ sys.exit("ERROR: no GEMINI_API_KEY (env or ~/.gemini_api_key)")++import httpx+from google import genai+from google.genai import types as gtypes++# big httpx connection pool so high worker counts actually run concurrently+# (default pool is ~100 connections -> caps throughput regardless of workers)+_POOL = max(WORKERS + 64, 128)+client = genai.Client(api_key=KEY, http_options=gtypes.HttpOptions(+ timeout=120000, # ms; flash+images can take ~30-50s+ client_args={"limits": httpx.Limits(max_connections=_POOL, max_keepalive_connections=_POOL)},+))+++class RateLimiter:+ """Token-spacing limiter: at most RPM requests per minute, shared across all+ worker threads. Lets us push high concurrency up to the model's RPM quota+ (gemini-3.5-flash ~1000/min) without tripping 429s."""+ def __init__(self, rpm):+ self.interval = 60.0 / max(rpm, 1)+ self.lock = threading.Lock()+ self.next_t = 0.0++ def acquire(self):+ with self.lock:+ now = time.time()+ t = self.next_t if self.next_t > now else now+ self.next_t = t + self.interval+ wait = t - now+ if wait > 0:+ time.sleep(wait)+++_limiter = RateLimiter(RPM)++PROMPT_HEAD = """You are writing a realistic USER INSTRUCTION: the prompt a person would type to ask an AI assistant to GENERATE this 3D model as parametric CadQuery (Python) code.++PRIMARY SOURCE = the RENDERED IMAGES. Judge from the images what the object IS, its shape and key parts, and its colors/materials.++The STRUCTURED SUMMARY below gives the object's name and its ARTICULATION (which parts move and how) — motion is invisible in a static image, so rely on it for the movement, and use the part names as hints. (Ignore it where it conflicts with what you clearly see in the images.)++Write ONE natural, concise instruction (1-2 sentences, first person / imperative, e.g. "Generate CadQuery code for ..." or "Create a parametric ... model in CadQuery with ..."). Cover: WHAT the object is, its key parts, its ARTICULATION (which parts move and how), and that it should have appropriate materials/colors.++Constraints:+- It must read like a user's request to produce CadQuery code, but do NOT mention internal API or class names and no raw geometry-kernel jargon. Describe the object and its motion naturally.+- Say it should be parametric CadQuery (Python) code.+- This model INCLUDES materials and colors — phrase it so the user asks for appropriate materials/colors on the parts (not bare geometry).+- Output ONLY the instruction text, no quotes, no preamble.++STRUCTURED SUMMARY:+%s+"""++_lock = threading.Lock()+_counts = {"done": 0, "skip": 0, "fail": 0}++try:+ from PIL import Image+ import io+ _HAVE_PIL = True+except Exception:+ _HAVE_PIL = False+++def _small_jpeg(path, maxdim=512):+ """Downscale a render to <=512px on a white background as JPEG — cuts the+ per-request payload from ~1-2MB PNG to ~40KB, which sharply lowers latency+ (the real throughput bottleneck) so we can hit the RPM quota."""+ try:+ if not (os.path.exists(path) and os.path.getsize(path) > 1000):+ return None+ if not _HAVE_PIL:+ return open(path, "rb").read()+ im = Image.open(path).convert("RGBA")+ bg = Image.new("RGBA", im.size, (255, 255, 255, 255))+ im = Image.alpha_composite(bg, im).convert("RGB")+ im.thumbnail((maxdim, maxdim))+ buf = io.BytesIO()+ im.save(buf, "JPEG", quality=85)+ return buf.getvalue()+ except Exception:+ try:+ return open(path, "rb").read()+ except Exception:+ return None+++def passed(n):+ try:+ return (json.load(open(f"{DD}/{n}/meta.json")).get("conversion") or {}).get("verifier_pass") is True+ except Exception:+ return False+++def extract_prior(src):+ """Compact, semantically-dense summary from the source (NOT the geometry math):+ object name + part names + movable joints & their type. Most of the code is+ B-rep/shim with no caption value, so we feed only this prior + the images."""+ lines = []+ m = re.search(r'ArticulatedObject\(\s*name\s*=\s*["\']([^"\']+)', src)+ if m:+ lines.append("Object name: " + m.group(1).replace("_", " "))+ parts = re.findall(r'\.part\(\s*["\']([^"\']+)', src)+ if parts:+ lines.append("Parts: " + ", ".join(p.replace("_", " ") for p in parts))+ arts = re.findall(r'\.articulation\(\s*["\']([^"\']+)["\']\s*,\s*ArticulationType\.(\w+)', src)+ moving = [(n, t) for n, t in arts if t.upper() != "FIXED"]+ if moving:+ lines.append("Movable joints: " + "; ".join(+ f"{n.replace('_', ' ')} ({t.lower()})" for n, t in moving))+ elif arts:+ lines.append("All joints are fixed (rigid assembly, no moving parts).")+ return "\n".join(lines) or "(no structured info available)"+++def samples():+ return [n for n in sorted(os.listdir(DD))+ if passed(n) and os.path.exists(f"{DD}/{n}/renders/view_00.png")]+++def gen_one(name):+ d = f"{DD}/{name}"+ cp = f"{d}/captions.json"+ try:+ caps = json.load(open(cp)) if os.path.exists(cp) else {}+ except Exception:+ caps = {}+ if not FORCE and caps.get(FIELD):+ with _lock: _counts["skip"] += 1+ return name, "skip"+ try:+ sp = f"{d}/articraft_source_code.py" # clean structure (ArticulationType, model.part)+ if not os.path.exists(sp):+ sp = f"{d}/code.py"+ prior = extract_prior(open(sp).read())+ except Exception as e:+ with _lock: _counts["fail"] += 1+ return name, f"no-code:{e}"+ contents = [PROMPT_HEAD % prior]+ for v in ("view_00.png", "view_04.png"):+ b = _small_jpeg(f"{d}/renders/{v}")+ if b:+ contents.append(gtypes.Part.from_bytes(data=b, mime_type="image/jpeg"))+ for attempt in range(5):+ try:+ _limiter.acquire()+ r = client.models.generate_content(+ model=MODEL, contents=contents,+ config=gtypes.GenerateContentConfig(+ temperature=0.7, max_output_tokens=400,+ thinking_config=gtypes.ThinkingConfig(thinking_budget=0),+ ),+ )+ txt = (r.text or "").strip().strip('"').strip()+ if not txt:+ raise RuntimeError("empty response")+ caps[FIELD] = txt+ with open(cp, "w") as f:+ json.dump(caps, f, ensure_ascii=False, indent=2)+ with _lock: _counts["done"] += 1+ return name, "ok"+ except Exception as e:+ msg = str(e)+ if any(s in msg for s in ("429", "RESOURCE_EXHAUSTED", "503", "UNAVAILABLE", "500")):+ time.sleep(2 * (attempt + 1) + (hash(name) % 3))+ continue+ with _lock: _counts["fail"] += 1+ return name, f"err:{msg[:80]}"+ with _lock: _counts["fail"] += 1+ return name, "rate-limited-giveup"+++def main():+ items = samples()+ if N:+ items = items[:N]+ total = len(items)+ print(f"captioning {total} with_mesh samples | model={MODEL} field={FIELD} workers={WORKERS}", flush=True)+ t0 = time.time(); fails = []+ with ThreadPoolExecutor(max_workers=WORKERS) as ex:+ futs = [ex.submit(gen_one, it) for it in items]+ for k, fut in enumerate(as_completed(futs), 1):+ name, st = fut.result()+ if st not in ("ok", "skip"):+ fails.append((name, st))+ if k % 50 == 0 or k == total:+ el = time.time() - t0; c = _counts+ print(f"[{k}/{total}] done={c['done']} skip={c['skip']} fail={c['fail']} "+ f"{el:.0f}s ETA{el/k*(total-k):.0f}s", flush=True)+ print(f"\nCAPTION_DONE done={_counts['done']} skip={_counts['skip']} fail={_counts['fail']} in {time.time()-t0:.0f}s", flush=True)+ for n, s in fails[:20]:+ print(" FAIL", n[:45], s)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/articraft/caption_urdf.py b/toolkits/curation/articraft/caption_urdf.pynew file mode 100644index 00000000..34cd3d2e--- /dev/null+++ b/toolkits/curation/articraft/caption_urdf.py@@ -0,0 +1,169 @@+#!/usr/bin/env python3+"""Generate a natural USER-INSTRUCTION caption for each URDF-only sample with+Gemini Flash, and add it to that sample's captions.json.++The instruction is phrased as what a user would type to ask an AI to GENERATE+this model as URDF code. It is folder-aware:+ - articraft_urdf_geo_only -> geometry-only (box/cylinder/sphere primitives,+ articulated joints, NO colors/materials)+ - articraft_urdf_tex -> includes materials/colors++Parallel (network-bound), resumable (skips samples that already have the field),+retries on rate-limit. Writes captions.json[FIELD].++Usage:+ GEMINI_API_KEY=... python tools/caption_urdf.py [N] [--workers W] [--model M]+ [--field F] [--scope geo|tex|all] [--force]+"""+import os, sys, json, time, threading+from concurrent.futures import ThreadPoolExecutor, as_completed++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+DATA = ROOT + "/data"+GEO = "articraft_urdf_geo_only"+TEX = "articraft_urdf_tex"++argv = sys.argv[1:]+N = 0; WORKERS = 16; MODEL = "gemini-2.5-flash"; FIELD = "user_instruction"; SCOPE = "all"; FORCE = False+_i = 0+while _i < len(argv):+ a = argv[_i]+ if a == "--workers": WORKERS = int(argv[_i + 1]); _i += 2; continue+ if a == "--model": MODEL = argv[_i + 1]; _i += 2; continue+ if a == "--field": FIELD = argv[_i + 1]; _i += 2; continue+ if a == "--scope": SCOPE = argv[_i + 1]; _i += 2; continue+ if a == "--force": FORCE = True; _i += 1; continue+ if a.isdigit(): N = int(a)+ _i += 1++KEY = os.environ.get("GEMINI_API_KEY") or os.environ.get("GOOGLE_API_KEY")+if not KEY:+ kp = os.path.expanduser("~/.gemini_api_key")+ if os.path.exists(kp):+ KEY = open(kp).read().strip()+if not KEY:+ sys.exit("ERROR: no GEMINI_API_KEY (env or ~/.gemini_api_key)")++from google import genai+from google.genai import types as gtypes+client = genai.Client(api_key=KEY)++PROMPT_HEAD = """You are writing a realistic USER INSTRUCTION: the prompt a person would type to ask an AI assistant to GENERATE this 3D model as a URDF file. You are given rendered views of the model AND its URDF source.++Use the RENDERED IMAGES to judge what the object looks like (shape, parts, and — if present — colors/materials). Use the URDF SOURCE to determine the ARTICULATION (which parts move and how: hinges=revolute, sliders=prismatic, spinning=continuous, from the <joint> elements), since motion is not visible in a static image.++Write ONE natural, concise instruction (1-2 sentences, first person / imperative, e.g. "Generate a URDF of ..." or "Create a ... model with ..."). Cover: WHAT the object is, its key parts, and its ARTICULATION (which parts move and how).++Constraints:+- It must read like a user's request to produce a URDF, but do NOT mention raw XML mechanics (no "link tag", "joint element", "<visual>", etc.). Describe the object and its motion naturally.+- Say it should be a URDF model.+- %s+- Output ONLY the instruction text, no quotes, no preamble.++URDF SOURCE:+%s+"""++GEO_LINE = ("This model is GEOMETRY-ONLY: built purely from box/cylinder/sphere PRIMITIVES "+ "with NO colors or materials. Phrase the instruction so the user explicitly asks for a "+ "geometry-only / primitive-only URDF (just shapes and joints, no colors or materials).")+TEX_LINE = ("This model INCLUDES materials and colors. Phrase the instruction so the user asks for a "+ "URDF with appropriate materials/colors on the parts (not just bare geometry).")++_lock = threading.Lock()+_counts = {"done": 0, "skip": 0, "fail": 0}+++def samples(scope):+ out = []+ if scope in ("all", "geo"):+ out += [(GEO, n, GEO_LINE) for n in sorted(os.listdir(f"{DATA}/{GEO}"))]+ if scope in ("all", "tex"):+ out += [(TEX, n, TEX_LINE) for n in sorted(os.listdir(f"{DATA}/{TEX}"))]+ return out+++def gen_one(item):+ folder, name, line = item+ d = f"{DATA}/{folder}/{name}"+ cp = f"{d}/captions.json"+ try:+ caps = json.load(open(cp)) if os.path.exists(cp) else {}+ except Exception:+ caps = {}+ if not FORCE and caps.get(FIELD):+ with _lock: _counts["skip"] += 1+ return name, "skip"+ try:+ urdf = open(f"{d}/code.urdf").read()+ except Exception as e:+ with _lock: _counts["fail"] += 1+ return name, f"no-urdf:{e}"+ urdf = urdf[:8000]+ prompt = PROMPT_HEAD % (line, urdf)+ contents = [prompt]+ for v in ("view_00.png", "view_04.png"):+ ip = f"{d}/renders/{v}"+ try:+ if os.path.exists(ip) and os.path.getsize(ip) > 1000:+ contents.append(gtypes.Part.from_bytes(data=open(ip, "rb").read(), mime_type="image/png"))+ except Exception:+ pass+ for attempt in range(5):+ try:+ r = client.models.generate_content(+ model=MODEL, contents=contents,+ config=gtypes.GenerateContentConfig(+ temperature=0.7, max_output_tokens=400,+ thinking_config=gtypes.ThinkingConfig(thinking_budget=0), # 关闭thinking,省token给正文+ ),+ )+ txt = (r.text or "").strip().strip('"').strip()+ if not txt:+ raise RuntimeError("empty response")+ caps[FIELD] = txt+ with open(cp, "w") as f:+ json.dump(caps, f, ensure_ascii=False, indent=2)+ with _lock: _counts["done"] += 1+ return name, "ok"+ except Exception as e:+ msg = str(e)+ if any(s in msg for s in ("429", "RESOURCE_EXHAUSTED", "503", "UNAVAILABLE", "500")):+ time.sleep(2 * (attempt + 1) + (hash(name) % 3))+ continue+ with _lock: _counts["fail"] += 1+ return name, f"err:{msg[:80]}"+ with _lock: _counts["fail"] += 1+ return name, "rate-limited-giveup"+++def main():+ items = samples(SCOPE)+ if not FORCE:+ # quick prefilter to estimate+ pass+ if N:+ items = items[:N]+ total = len(items)+ print(f"captioning {total} samples | model={MODEL} field={FIELD} workers={WORKERS}", flush=True)+ t0 = time.time()+ fails = []+ with ThreadPoolExecutor(max_workers=WORKERS) as ex:+ futs = [ex.submit(gen_one, it) for it in items]+ for k, fut in enumerate(as_completed(futs), 1):+ name, st = fut.result()+ if st not in ("ok", "skip"):+ fails.append((name, st))+ if k % 50 == 0 or k == total:+ el = time.time() - t0+ c = _counts+ print(f"[{k}/{total}] done={c['done']} skip={c['skip']} fail={c['fail']} "+ f"{el:.0f}s ETA{el/k*(total-k):.0f}s", flush=True)+ print(f"\nDONE done={_counts['done']} skip={_counts['skip']} fail={_counts['fail']} in {time.time()-t0:.0f}s", flush=True)+ for n, s in fails[:20]:+ print(" FAIL", n[:45], s)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/articraft/cq_to_glb.py b/toolkits/curation/articraft/cq_to_glb.pynew file mode 100644index 00000000..c393785d--- /dev/null+++ b/toolkits/curation/articraft/cq_to_glb.py@@ -0,0 +1,24 @@+#!/usr/bin/env python3+"""Exec a self-contained with_mesh code.py and export its `result` (cq.Assembly,+with material colors) to <sample>/renders/object.glb. Usage: cq_to_glb.py <sample_dir>"""+import sys, os, types, warnings+warnings.filterwarnings("ignore")++d = sys.argv[1]+src = open(f"{d}/code.py").read()+mod = types.ModuleType("cqglb_mod")+mod.__dict__["__builtins__"] = __builtins__+mod.__dict__["__file__"] = f"{d}/code.py" # model body may ref AssetContext.from_script(__file__)+sys.modules["cqglb_mod"] = mod+exec(compile(src, f"{d}/code.py", "exec"), mod.__dict__)+result = mod.__dict__.get("result")+if result is None:+ print("NO_RESULT"); sys.exit(1)+os.makedirs(f"{d}/renders", exist_ok=True)+out = f"{d}/renders/object.glb"+try:+ result.save(out) # cq.Assembly -> GLB (keeps per-part colors)+except Exception:+ import cadquery as cq+ cq.exporters.export(result.toCompound(), out)+print("GLB_OK", os.path.getsize(out))diff --git a/toolkits/curation/articraft/driver_urdf_render.py b/toolkits/curation/articraft/driver_urdf_render.pynew file mode 100644index 00000000..ece0fb21--- /dev/null+++ b/toolkits/curation/articraft/driver_urdf_render.py@@ -0,0 +1,41 @@+"""并行渲染所有 URDF 样本(透明背景8视角): geo_only→白模(clay), tex→彩色(material)。+每物体独立 Blender 进程。用法: python driver_urdf_render.py [workers] [limit]"""+import os, subprocess, glob, sys, time+from concurrent.futures import ThreadPoolExecutor++BL="./tools/blender-5.0.1-linux-x64/blender"+N=int(sys.argv[1]) if len(sys.argv)>1 else 8+SCOPE=sys.argv[2] if len(sys.argv)>2 else "all" # all | geo | tex+FORCE=len(sys.argv)>3 and sys.argv[3]=="force" # 重渲已完成的+ENV=os.environ.copy(); ENV["LD_LIBRARY_PATH"]=(os.environ.get("CONDA_PREFIX","") + "/lib:")+ENV.get("LD_LIBRARY_PATH","")++tasks=[]+if SCOPE in ("all","geo"):+ for n in sorted(os.listdir("data/articraft_urdf_geo_only")):+ d=f"data/articraft_urdf_geo_only/{n}"+ if not FORCE and os.path.exists(f"{d}/renders/view_07.png"): continue+ tasks.append((f"{d}/code.urdf", f"{d}/renders", "clay"))+if SCOPE in ("all","tex"):+ for n in sorted(os.listdir("data/articraft_urdf_tex")):+ d=f"data/articraft_urdf_tex/{n}"+ if not FORCE and os.path.exists(f"{d}/renders/view_07.png"): continue+ tasks.append((f"{d}/code.urdf", f"{d}/renders", "material"))++def render(t):+ urdf,out,mode=t+ try:+ r=subprocess.run([BL,"--background","--threads","4","--python","blender_urdf.py","--",urdf,out,"1024","16","8",mode],+ env=ENV,capture_output=True,text=True,timeout=300)+ return "URDF_RENDER_DONE" in r.stdout+ except Exception:+ return False++t0=time.time(); done=fail=0+print(f"workers={N} tasks={len(tasks)} (geo_only白模 + tex彩色)",flush=True)+with ThreadPoolExecutor(max_workers=N) as ex:+ for ok in ex.map(render,tasks):+ done+=1; fail+=(0 if ok else 1)+ if done%100==0 or done==len(tasks):+ el=time.time()-t0+ print(f"[{done}/{len(tasks)}] {el:.0f}s {el/done:.1f}s/ea fail{fail} ETA{el/done*(len(tasks)-done):.0f}s",flush=True)+print(f"RENDER_ALL_DONE fail{fail} 用时{time.time()-t0:.0f}s",flush=True)diff --git a/toolkits/curation/articraft/exec_validate.py b/toolkits/curation/articraft/exec_validate.pynew file mode 100644index 00000000..f16a6ce7--- /dev/null+++ b/toolkits/curation/articraft/exec_validate.py@@ -0,0 +1,73 @@+#!/usr/bin/env python3+"""Regenerate every PASSING with_mesh code.py (current gen_code) and EXEC-validate+all of them: exec code.py -> result must be a cq.Assembly/Workplane/Shape. Catches+NameError / type / dedup-precedence regressions across the WHOLE set (not a sample).+Then chamfer-verify a random subset for geometry fidelity."""+import os, sys, time, json, statistics, subprocess, random+from concurrent.futures import ThreadPoolExecutor, as_completed++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+DD = ROOT + "/data/articraft_with_mesh"+sys.path.insert(0, _HERE)+import gen_code; gen_code._load()++names = [n for n in os.listdir(DD)+ if (json.load(open(f"{DD}/{n}/meta.json")).get("conversion") or {}).get("verifier_pass") is True]++# 1) regenerate all (last-wins dedup)+t0 = time.time()+for n in names:+ try:+ txt, _ = gen_code.gen_code_text(f"{DD}/{n}")+ open(f"{DD}/{n}/code.py", "w").write(txt)+ except Exception as e:+ print("GENFAIL", n, str(e)[:60], flush=True)+ls = [sum(1 for _ in open(f"{DD}/{n}/code.py")) for n in names]+print(f"regen {len(names)} {time.time()-t0:.0f}s | median {statistics.median(ls):.0f} "+ f"mean {statistics.mean(ls):.0f} max {max(ls)} | >1500 {sum(x>1500 for x in ls)} "+ f">1000 {sum(x>1000 for x in ls)}", flush=True)++CHECK = ROOT + "/tools/_execchk.py"+open(CHECK, "w").write(+ 'import sys,types,warnings\n'+ 'warnings.filterwarnings("ignore")\n'+ 'd=sys.argv[1]; src=open(d+"/code.py").read()\n'+ 'm=types.ModuleType("v"); m.__dict__["__builtins__"]=__builtins__; m.__dict__["__file__"]=d+"/code.py"\n'+ 'sys.modules["v"]=m\n'+ 'try:\n'+ ' exec(compile(src,"c","exec"),m.__dict__)\n'+ ' r=m.__dict__.get("result")\n'+ ' import cadquery as cq\n'+ ' if r is None: print("NO_RESULT")\n'+ ' elif isinstance(r,(cq.Assembly,cq.Workplane,cq.Shape)): print("OK")\n'+ ' else: print("BADTYPE:"+type(r).__name__)\n'+ 'except Exception as e:\n'+ ' print("EXECERR:"+type(e).__name__+":"+str(e)[:70])\n'+)+env = dict(os.environ, OMP_NUM_THREADS="1", MKL_NUM_THREADS="1", OPENBLAS_NUM_THREADS="1", NUMEXPR_NUM_THREADS="1")+++def chk(n):+ r = subprocess.run([sys.executable, CHECK, f"{DD}/{n}"], env=env, capture_output=True, text=True, timeout=300)+ return n, (r.stdout.strip().splitlines()[-1] if r.stdout.strip() else "EMPTY:" + r.stderr[-60:])+++t1 = time.time(); ok = 0; fails = []+with ThreadPoolExecutor(max_workers=16) as ex:+ futs = [ex.submit(chk, n) for n in names]+ for k, f in enumerate(as_completed(futs), 1):+ n, st = f.result()+ if st == "OK":+ ok += 1+ else:+ fails.append((n, st))+ if k % 1000 == 0:+ print(f" exec {k}/{len(names)} ok={ok} fail={len(fails)} {time.time()-t1:.0f}s", flush=True)+print(f"EXEC_VALIDATE OK {ok}/{len(names)} fail {len(fails)} in {time.time()-t1:.0f}s", flush=True)+from collections import Counter+c = Counter(":".join(st.split(":")[:2]) for _, st in fails)+for k, v in c.most_common(20):+ print(f" {v}x {k}", flush=True)+for n, st in fails[:30]:+ print(" FAIL", n[:44], st[:60], flush=True)diff --git a/toolkits/curation/articraft/gen_code.py b/toolkits/curation/articraft/gen_code.pynew file mode 100644index 00000000..5396b174--- /dev/null+++ b/toolkits/curation/articraft/gen_code.py@@ -0,0 +1,360 @@+#!/usr/bin/env python3+"""Generate a MINIMAL self-contained CadQuery code.py for an Articraft with_mesh+sample (tree-shaken: only the shim symbols the model actually uses are inlined).++code.py = future import + base imports + (any extra imports the used family+symbols need) + CORE shim + only-used family symbols (dependency closure) ++the original model body (sdk/future imports removed) + epilogue building+`result = object_model.to_cq()`.++Only needs `import cadquery` (+ numpy/OCP only if a used family — e.g. gears —+requires it). Tree-shaking drops unused geometry classes/helpers, so files are+far smaller than whole-family inlining.++Usage: python tools/gen_code.py <sample_name|dir> [--out PATH] [--inplace]+"""+import os, re, ast, sys++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+DD = ROOT + "/data/articraft_with_mesh"+sys.path.insert(0, _HERE)+import assemble_shim as _asm++_FUTURE = "from __future__ import annotations"+_CORE_SRC = None+_NAME2FAM = None+_CORE_SYMS = {} # sym -> (text, refs) in core file order+_CORE_IMPORTS = []+_FAM_SYMS = {} # fam -> {sym: (text, refs_set)} in file order+_FAM_IMPORTS = {} # fam -> [import source lines]+# always-seed runtime so the to_cq()/build path survives even if only referenced dynamically+_RUNTIME_SEED = {+ "ArticulatedObject", "Part", "Visual", "Articulation", "Origin", "Material",+ "ArticulationType", "_Tf", "_to_shape", "_CQMesh", "ValidationError",+ "mesh_from_cadquery",+}+++def _blocks(src):+ """Split a module source into (import_lines, {sym:(text,refs)}) preserving+ top-level order. Captures decorators via line slicing (get_source_segment+ would DROP @dataclass/@property)."""+ tree = ast.parse(src)+ lines = src.split("\n")+ imports, syms = [], {}+ for node in tree.body:+ if isinstance(node, (ast.Import, ast.ImportFrom)):+ imports.append("\n".join(lines[node.lineno - 1:node.end_lineno]))+ continue+ if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):+ names = [node.name]+ elif isinstance(node, ast.Assign):+ names = [t.id for t in node.targets if isinstance(t, ast.Name)]+ else:+ continue+ if not names:+ continue+ start = node.lineno+ for d in getattr(node, "decorator_list", []) or []:+ start = min(start, d.lineno)+ seg = "\n".join(lines[start - 1:node.end_lineno])+ refs = {sub.id for sub in ast.walk(node) if isinstance(sub, ast.Name)}+ for nm in names:+ syms[nm] = (seg, refs)+ return imports, syms+++def _load():+ global _CORE_SRC, _NAME2FAM, _CORE_IMPORTS, _CORE_SYMS+ if _CORE_SRC is not None:+ return+ core = open(_asm.CORE).read()+ core = "\n".join(l for l in core.splitlines() if l.strip() != _FUTURE)+ _CORE_SRC = core+ # core import lines: keep math/cadquery; drop the redundant ones we emit in header+ cimps, csyms = _blocks(core)+ _CORE_IMPORTS = [i for i in cimps if i.strip() not in ("import math", "import cadquery as cq")]+ _CORE_SYMS = csyms+ _NAME2FAM = _asm.name2family()+ for stem, path in _asm._family_files():+ src = _asm._strip_core_imports(open(path).read())+ imps, syms = _blocks(src)+ _FAM_IMPORTS[stem] = imps+ _FAM_SYMS[stem] = syms+++def _closure(fam, seed):+ syms = _FAM_SYMS[fam]+ need, stack = set(), [s for s in seed if s in syms]+ while stack:+ s = stack.pop()+ if s in need:+ continue+ need.add(s)+ for r in syms[s][1]:+ if r in syms and r not in need:+ stack.append(r)+ return need+++def _imported_sdk_names(src):+ out = []+ for node in ast.walk(ast.parse(src)):+ if isinstance(node, ast.ImportFrom) and (node.module == "sdk" or (node.module or "").startswith("sdk.")):+ out += [a.name for a in node.names]+ return out+++def _model_node_names(nd):+ """Top-level names a node defines: def/class name, or assign targets+ (incl. tuple/list unpacking like `A, B = f()` and annotated assigns)."""+ if isinstance(nd, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):+ return [nd.name]+ out = []+ if isinstance(nd, ast.Assign):+ for t in nd.targets:+ if isinstance(t, ast.Name):+ out.append(t.id)+ elif isinstance(t, (ast.Tuple, ast.List)):+ out += [e.id for e in t.elts if isinstance(e, ast.Name)]+ elif isinstance(nd, ast.AnnAssign) and isinstance(nd.target, ast.Name):+ out.append(nd.target.id)+ return out+++def _treeshake_model(body):+ """Keep only build_object_model and its FULL closure — geometry helpers AND+ the module-level constants it uses AND any functions those constants call+ (e.g. `KNOB_CENTER_X, KNOB_CENTER_Y = _key_center(...)`). Drops run_tests()+ and other test-only / dead top-level code. The epilogue rebuilds+ object_model = build_object_model()."""+ try:+ tree = ast.parse(body)+ except Exception:+ return body+ lines = body.split("\n")+ sym_nodes = {} # name -> [defining nodes in order] (a name may be (re)defined+ refs_union = {} # multiple times, e.g. SHAFT_Y built in two steps where the+ for nd in tree.body: # second references the first — keep ALL, not just the last+ nms = _model_node_names(nd)+ if not nms:+ continue+ r = {x.id for x in ast.walk(nd) if isinstance(x, ast.Name)}+ for nm in nms:+ sym_nodes.setdefault(nm, []).append(nd)+ refs_union.setdefault(nm, set()).update(r)+ if "build_object_model" not in sym_nodes:+ return body # unusual structure -> keep whole (safe)+ keep, st = set(), ["build_object_model"]+ while st:+ s = st.pop()+ if s in keep or s not in sym_nodes:+ continue+ keep.add(s)+ st += [r for r in refs_union[s] if r in sym_nodes and r not in keep]+ kept_ids = {id(nd) for n in keep for nd in sym_nodes[n]}+ dropped = set(sym_nodes) - keep+ out = []+ for nd in tree.body:+ if isinstance(nd, (ast.Import, ast.ImportFrom)):+ pass # always keep imports+ elif _model_node_names(nd): # a definition / assignment+ if id(nd) not in kept_ids:+ continue+ else: # bare statement (Expr / If __main__ / call)+ names = {x.id for x in ast.walk(nd) if isinstance(x, ast.Name)}+ if names & dropped: # e.g. bare run_tests() / report = run_tests()+ continue+ a = nd.lineno+ for d in getattr(nd, "decorator_list", []) or []:+ a = min(a, d.lineno)+ out.append("\n".join(lines[a - 1:nd.end_lineno]))+ return "\n".join(out)+++def _strip_model_imports(src):+ lines = src.splitlines()+ out, i = [], 0+ while i < len(lines):+ ln = lines[i]+ s = ln.strip()+ if s.startswith("from __future__ import"):+ i += 1+ continue+ if s.startswith("import sdk") or re.match(r"^from sdk(\.|\s)", s):+ if "(" in ln and ")" not in ln:+ i += 1+ while i < len(lines) and ")" not in lines[i]:+ i += 1+ i += 1+ else:+ i += 1+ continue+ out.append(ln)+ i += 1+ return "\n".join(out)+++def gen_code_text(sample_dir):+ _load()+ src_path = os.path.join(sample_dir, "articraft_source_code.py")+ if not os.path.exists(src_path):+ src_path = os.path.join(sample_dir, "code.py")+ model_src = open(src_path).read()++ # unified symbol graph over core + all families. Some names exist in BOTH+ # core and a family (or two families). Use LAST-DEFINITION-WINS, matching+ # Python runtime if every copy were emitted in order core -> sorted(families):+ # a name in core AND a family -> the family copy is the real one (core may+ # hold only a stub, e.g. repair_loft); a name in two families -> identical+ # copies, last sorted family wins. The closure refs MUST come from the SAME+ # copy that gets emitted, so build combined + last_owner with that order.+ combined, last_owner = {}, {}+ for s, (t, r) in _CORE_SYMS.items():+ combined[s] = r; last_owner[s] = "core"+ for fam in sorted(_FAM_SYMS):+ for s, (t, r) in _FAM_SYMS[fam].items():+ combined[s] = r; last_owner[s] = fam++ body = _treeshake_model(_strip_model_imports(model_src))+ try:+ seed = {nd.id for nd in ast.walk(ast.parse(body)) if isinstance(nd, ast.Name)} | set(_RUNTIME_SEED)+ except Exception:+ seed = set(_imported_sdk_names(model_src)) | set(_RUNTIME_SEED)+ need, stack = set(), [s for s in seed if s in combined]+ while stack:+ s = stack.pop()+ if s in need:+ continue+ need.add(s)+ for r in combined[s]:+ if r in combined and r not in need:+ stack.append(r)++ # emit each needed symbol ONCE, from its last_owner copy (the one Python would+ # actually use) — drops the shadowed earlier duplicate (core stub / first copy).+ emitted_shim = set()+ core_blocks = []+ for s, (t, r) in _CORE_SYMS.items():+ if s in need and last_owner[s] == "core":+ core_blocks.append(t); emitted_shim.add(s)+ fam_chunks, extra_imports, used_fams = [], [], []+ for fam in sorted(_FAM_SYMS):+ blocks = []+ for s, (t, r) in _FAM_SYMS[fam].items():+ if s in need and last_owner[s] == fam:+ blocks.append(t); emitted_shim.add(s)+ if not blocks:+ continue+ used_fams.append((fam, len(blocks)))+ for imp in _FAM_IMPORTS[fam]:+ if imp not in extra_imports:+ extra_imports.append(imp)+ fam_chunks.append("\n# ===== %s (%d syms) =====\n%s" % (fam, len(blocks), "\n\n".join(blocks)))++ # cq_gears' `cq.Workplane.gear(SpurGear(...))` plugin: some models call it+ # directly. It's a module-level attribute-assign (not a tree-shakeable symbol),+ # so emit it explicitly whenever the gears family is inlined.+ if any(f == "gears" for f, _ in used_fams):+ fam_chunks.append(_GEAR_PLUGIN)++ # Inlining puts shim helpers AND the model's own helpers in ONE namespace. If a+ # model defines a PRIVATE helper with the same name as a shim helper, the model+ # copy (emitted last) shadows the shim one and breaks the shim's internal calls+ # (seen: model `_v_norm`->tuple shadowing sweeps `_v_norm`->float). Rename the+ # colliding shim-private helpers (+ their refs within the shim) out of the way.+ model_names = set()+ try:+ for nd in ast.parse(body).body:+ model_names.update(_model_node_names(nd))+ except Exception:+ pass+ collisions = {n for n in emitted_shim if n.startswith("_") and n in model_names}+ if collisions:+ rmap = {c: "_shim" + c for c in collisions}++ def _ren(txt):+ for c, nw in rmap.items():+ txt = re.sub(r"\b" + re.escape(c) + r"\b", nw, txt)+ return txt+ core_blocks = [_ren(t) for t in core_blocks]+ fam_chunks = [_ren(t) for t in fam_chunks]++ epilogue = (+ "\n\n# ---- self-contained REST-pose assembly ----\n"+ "try:\n object_model\nexcept NameError:\n object_model = build_object_model()\n"+ "result = object_model.to_cq()\n"+ )+ code_parts = ["\n# ===== shim core =====\n" + "\n\n".join(core_blocks)] + fam_chunks+ code_parts += ["\n# ===== model =====\n" + body, epilogue]+ code_body = "\n".join(code_parts)+ header = [_FUTURE, "", "import cadquery as cq", "import math"]+ # some models do `pathlib.os.X` (worked pre-3.13 when pathlib re-exported os)+ if "pathlib.os" in code_body:+ header += ["import os as _os_c, pathlib as _pl_c",+ "if not hasattr(_pl_c, 'os'): _pl_c.os = _os_c"]+ # prune import lines whose names are never referenced in the body+ header += _prune_imports(_CORE_IMPORTS + extra_imports, code_body)+ return "\n".join(["\n".join(header)] + code_parts), [f for f, _ in used_fams]+++_GEAR_PLUGIN = '''+# ===== gears cq.Workplane plugin (cq_gears workflow) =====+def _gear_plugin(self, gear_, *a, **k):+ body = gear_.build(*a, **k)+ if hasattr(body, "val"):+ body = body.val() # Workplane -> Shape+ return self.eachpoint(lambda loc: body.located(loc), True)+++def _addgear_plugin(self, gear_, *a, **k):+ return self.union(_gear_plugin(self, gear_, *a, **k))+++cq.Workplane.gear = _gear_plugin+cq.Workplane.addGear = _addgear_plugin+'''+++def _prune_imports(imports, body):+ out = []+ for line in imports:+ try:+ node = ast.parse(line.strip()).body[0]+ except Exception:+ out.append(line); continue+ if isinstance(node, ast.Import):+ names = [a.asname or a.name.split(".")[0] for a in node.names]+ elif isinstance(node, ast.ImportFrom):+ names = [a.asname or a.name for a in node.names]+ else:+ out.append(line); continue+ if not names or any(re.search(r"\b" + re.escape(n) + r"\b", body) for n in names):+ out.append(line)+ return out+++def main():+ if len(sys.argv) < 2:+ print("usage: gen_code.py <sample_name|dir> [--out PATH] [--inplace]")+ sys.exit(1)+ arg = sys.argv[1]+ sample_dir = arg if os.path.isdir(arg) else os.path.join(DD, arg)+ name = os.path.basename(sample_dir.rstrip("/"))+ txt, fams = gen_code_text(sample_dir)+ if "--inplace" in sys.argv:+ out = os.path.join(sample_dir, "code.py")+ elif "--out" in sys.argv:+ out = sys.argv[sys.argv.index("--out") + 1]+ else:+ d = os.path.join(os.environ.get("CLAUDE_JOB_DIR", "/tmp"), "tmp", "gen")+ os.makedirs(d, exist_ok=True)+ out = os.path.join(d, name + ".code.py")+ with open(out, "w") as f:+ f.write(txt)+ print("wrote", out, "| families:", fams, "| lines:", txt.count(chr(10)) + 1)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/articraft/geom_chamfer.py b/toolkits/curation/articraft/geom_chamfer.pynew file mode 100644index 00000000..36f11543--- /dev/null+++ b/toolkits/curation/articraft/geom_chamfer.py@@ -0,0 +1,86 @@+"""Isolated per-geometry verification: compare a REAL Articraft sdk geometry+(triangle mesh) against a candidate CadQuery shape via normalized chamfer.++Usage inside a family agent:++ import sys; sys.path.insert(0, os.path.dirname(__file__)) # this folder+ import sys; sys.path.insert(0, os.environ["ARTICRAFT_FORK"]) # the real sdk (Apache-2.0 fork)+ import sdk # the REAL sdk (ground truth meshes)+ import geom_chamfer as gc+ import mymodule # your cadquery shim family++ real = sdk.TorusGeometry(0.1, 0.02)+ mine = mymodule.TorusGeometry(0.1, 0.02)+ print(gc.chamfer(real, mine)) # {'chamfer':..., 'bbox_ratio':[...]} pass if chamfer<0.02 & bbox in [0.9,1.1]+"""+import os, sys, tempfile+import numpy as np++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+++def _tm_from_vf(verts, faces):+ import trimesh+ return trimesh.Trimesh(vertices=np.asarray(verts, float),+ faces=np.asarray(faces, int), process=False)+++def _tm_from_sdk(geom):+ """Real sdk geometry -> trimesh. sdk MeshGeometry exposes .vertices/.faces."""+ import trimesh+ if hasattr(geom, "vertices") and hasattr(geom, "faces") and len(getattr(geom, "faces")):+ return _tm_from_vf(geom.vertices, geom.faces)+ # some sdk helpers return a Mesh referencing an obj file+ raise RuntimeError("object is not a triangle-mesh sdk geometry: %r" % type(geom))+++def _tm_from_cq(shape):+ import cadquery as cq+ import trimesh+ if hasattr(shape, "_cq"):+ shape = shape._cq()+ if isinstance(shape, cq.Assembly):+ shape = shape.toCompound()+ if isinstance(shape, cq.Workplane):+ vals = [v for v in shape.vals() if isinstance(v, cq.Shape)]+ shape = vals[0] if len(vals) == 1 else cq.Compound.makeCompound(vals)+ fd, tmp = tempfile.mkstemp(suffix=".stl")+ os.close(fd)+ try:+ cq.exporters.export(shape, tmp, tolerance=1e-4, angularTolerance=0.1)+ m = trimesh.load(tmp, process=False)+ finally:+ if os.path.exists(tmp):+ os.remove(tmp)+ return m+++def _norm(m):+ m = m.copy()+ m.apply_translation(-m.bounds.mean(axis=0))+ e = m.extents+ s = e.max()+ if s <= 0:+ raise RuntimeError("degenerate mesh")+ m.apply_scale(1.0 / s)+ return m, e+++def chamfer(real_geom, mine, n=20000):+ """Return {'chamfer','bbox_ratio','pass'} comparing a real sdk geometry to a+ candidate cadquery geometry/shape (both normalized to unit bbox)."""+ import trimesh+ from scipy.spatial import cKDTree+ gt = real_geom if hasattr(real_geom, "extents") else _tm_from_sdk(real_geom)+ mn = _tm_from_cq(mine)+ gtn, ge = _norm(gt)+ mnn, me = _norm(mn)+ pg = trimesh.sample.sample_surface(gtn, n, seed=0)[0]+ pm = trimesh.sample.sample_surface(mnn, n, seed=0)[0]+ d1, _ = cKDTree(pg).query(pm)+ d2, _ = cKDTree(pm).query(pg)+ ch = float((d1.mean() + d2.mean()) / 2.0)+ br = [float(x) for x in (me / ge)]+ return {"chamfer": round(ch, 5), "bbox_ratio": [round(x, 4) for x in br],+ "pass": ch < 0.02 and all(0.9 <= r <= 1.1 for r in br)}diff --git a/toolkits/curation/articraft/integrate_recovered.py b/toolkits/curation/articraft/integrate_recovered.pynew file mode 100644index 00000000..9209b74a--- /dev/null+++ b/toolkits/curation/articraft/integrate_recovered.py@@ -0,0 +1,83 @@+#!/usr/bin/env python3+"""Phase A of integrating debug-recovered samples: for every currently-failing+sample (verifier_pass != true) plus globe_0003 (passed but never rendered),+regenerate code.py with the fixed shim, verify+export glb in one exec, and if it+executes, mark it included (verifier_pass=true, real chamfer/bbox, geom_approx flag+for chamfer>0.02 or bbox out of [0.9,1.1]). Samples that still fail to exec are left+untouched. Phases B/C/D (render, caption, upload) run after."""+import os, sys, json, subprocess+from concurrent.futures import ThreadPoolExecutor, as_completed++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+DD = ROOT + "/data/articraft_with_mesh"+sys.path.insert(0, _HERE)+import gen_code; gen_code._load()+env = dict(os.environ, OMP_NUM_THREADS="1", MKL_NUM_THREADS="1", OPENBLAS_NUM_THREADS="1")+++def cv(meta):+ return (meta.get("conversion") or {})+++def bbox_ok(bb):+ if bb is None:+ return False+ vals = bb if isinstance(bb, list) else [bb]+ return all(0.9 <= v <= 1.1 for v in vals if isinstance(v, (int, float)))+++def proc(n):+ d = f"{DD}/{n}"+ try:+ txt, _ = gen_code.gen_code_text(d)+ open(f"{d}/code.py", "w").write(txt)+ except Exception as e:+ return n, "gen_fail", str(e)[:50]+ glb = f"{d}/renders/object.glb"+ try:+ r = subprocess.run([sys.executable, "tools/verify_cadquery.py", d, f"{d}/code.py", "--glb", glb],+ env=env, capture_output=True, text=True, timeout=400)+ j = json.loads(r.stdout.strip().splitlines()[-1])+ except Exception as e:+ return n, "exec_fail", str(e)[:50]+ ch, bb = j.get("chamfer"), j.get("bbox_ratio")+ if ch is None and not os.path.exists(glb):+ return n, "exec_fail", str(j.get("error"))[:50]+ # executed + produced glb -> include it+ meta = json.load(open(f"{d}/meta.json"))+ conv = meta.get("conversion") or {}+ conv["verifier_pass"] = True+ conv["chamfer"] = ch+ conv["bbox_ratio"] = bb+ conv["error"] = None+ strict = (ch is not None and ch < 0.02) and bbox_ok(bb)+ conv["geom_approx"] = (not strict) # accepted despite slightly-off chamfer/bbox+ meta["conversion"] = conv+ json.dump(meta, open(f"{d}/meta.json", "w"), ensure_ascii=False, indent=2)+ return n, ("ok_strict" if strict else "ok_approx"), ch+++def main():+ targets = sorted(n for n in os.listdir(DD) if not cv(json.load(open(f"{DD}/{n}/meta.json"))).get("verifier_pass") is True)+ # also retry globe_0003 (passed but no glb -> slow exec)+ g = [n for n in os.listdir(DD) if n.startswith("rec_globe_0003")]+ for n in g:+ if n not in targets:+ targets.append(n)+ print(f"Phase A: processing {len(targets)} samples", flush=True)+ res = {"ok_strict": 0, "ok_approx": 0, "exec_fail": 0, "gen_fail": 0}+ skipped = []+ with ThreadPoolExecutor(max_workers=5) as ex:+ for n, st, info in [f.result() for f in as_completed([ex.submit(proc, n) for n in targets])]:+ res[st] = res.get(st, 0) + 1+ if st in ("exec_fail", "gen_fail"):+ skipped.append((n, info))+ print(f"PHASE_A_DONE included={res['ok_strict']+res['ok_approx']} "+ f"(strict={res['ok_strict']} approx={res['ok_approx']}) skipped={len(skipped)}", flush=True)+ for n, i in skipped:+ print(" SKIP", n[:46], i)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/articraft/regen_glb.py b/toolkits/curation/articraft/regen_glb.pynew file mode 100644index 00000000..c963b0fa--- /dev/null+++ b/toolkits/curation/articraft/regen_glb.py@@ -0,0 +1,74 @@+#!/usr/bin/env python3+"""Light pass: for every PASSING with_mesh sample, regenerate the tree-shaken+code.py and export object.glb (exec ONCE) — NO chamfer (tree-shaking can't+change the geometry of used symbols, and a successful glb export already+validates the B-rep, so re-chamfering is redundant). Keeps the existing+verifier_pass verdict. The GPU render consumer renders the glbs in parallel.++Usage: python tools/regen_glb.py [--workers W] [--force]+"""+import os, sys, json, subprocess, time+from concurrent.futures import ThreadPoolExecutor, as_completed++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+DD = ROOT + "/data/articraft_with_mesh"+sys.path.insert(0, _HERE)+import gen_code++argv = sys.argv[1:]+WORKERS = int(argv[argv.index("--workers") + 1]) if "--workers" in argv else 16+FORCE = "--force" in argv+_ENV = dict(os.environ, OMP_NUM_THREADS="1", OPENBLAS_NUM_THREADS="1",+ MKL_NUM_THREADS="1", NUMEXPR_NUM_THREADS="1")+++def passed(n):+ try:+ return (json.load(open(f"{DD}/{n}/meta.json")).get("conversion") or {}).get("verifier_pass") is True+ except Exception:+ return False+++def proc(n):+ d = f"{DD}/{n}"+ try:+ txt, _ = gen_code.gen_code_text(d)+ with open(f"{d}/code.py", "w") as f:+ f.write(txt)+ except Exception as e:+ return n, "gen_fail:%s" % str(e)[:50]+ try:+ r = subprocess.run([sys.executable, f"{ROOT}/tools/cq_to_glb.py", d],+ env=_ENV, capture_output=True, text=True, timeout=300)+ return n, ("ok" if "GLB_OK" in r.stdout else "glb_fail")+ except Exception as e:+ return n, "err:%s" % str(e)[:40]+++def main():+ gen_code._load()+ names = [n for n in sorted(os.listdir(DD)) if passed(n)]+ if not FORCE:+ names = [n for n in names if not os.path.exists(f"{DD}/{n}/renders/object.glb")]+ total = len(names)+ print(f"regen+glb {total} passing samples | workers={WORKERS} (no chamfer)", flush=True)+ t0 = time.time(); ok = fail = 0; fails = []+ with ThreadPoolExecutor(max_workers=WORKERS) as ex:+ futs = {ex.submit(proc, n): n for n in names}+ for k, fut in enumerate(as_completed(futs), 1):+ n, st = fut.result()+ if st == "ok":+ ok += 1+ else:+ fail += 1; fails.append((n, st))+ if k % 100 == 0 or k == total:+ el = time.time() - t0+ print(f"[{k}/{total}] ok={ok} fail={fail} {el:.0f}s {el/k:.2f}s/ea ETA{el/k*(total-k):.0f}s", flush=True)+ print(f"REGEN_GLB_DONE ok={ok} fail={fail} in {time.time()-t0:.0f}s", flush=True)+ for n, s in fails[:20]:+ print(" FAIL", n[:45], s)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/articraft/render_glb_only.py b/toolkits/curation/articraft/render_glb_only.pynew file mode 100644index 00000000..507da636--- /dev/null+++ b/toolkits/curation/articraft/render_glb_only.py@@ -0,0 +1,84 @@+#!/usr/bin/env python3+"""GPU-only consumer: render any with_mesh sample that has renders/object.glb+(produced by the fused batch_verify --glb) but no view_07.png yet. Loops until+the producer (batch_verify) is gone and nothing is left. Pure blender (GPU);+no cadquery exec, so it overlaps the CPU-bound gen+verify perfectly.++Usage: python tools/render_glb_only.py [--workers W] [--mode material|clay]+"""+import os, sys, json, glob, subprocess, time+from concurrent.futures import ThreadPoolExecutor, as_completed++_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+DD = ROOT + "/data/articraft_with_mesh"+BL = os.environ.get("BLENDER", "blender")+argv = sys.argv[1:]+WORKERS = int(argv[argv.index("--workers") + 1]) if "--workers" in argv else 6+MODE = argv[argv.index("--mode") + 1] if "--mode" in argv else "material"++ENV = os.environ.copy()+ENV["LD_LIBRARY_PATH"] = (os.environ.get("CONDA_PREFIX","") + "/lib:") + ENV.get("LD_LIBRARY_PATH", "")+++def bv_running():+ for p in glob.glob("/proc/[0-9]*"):+ try:+ c = open(p + "/cmdline", "rb").read().decode("u8", "replace")+ except Exception:+ continue+ if " -c " in c:+ continue+ if "batch_verify.py" in c or "regen_glb.py" in c:+ return True+ return False+++def render(n):+ d = f"{DD}/{n}"+ glbp = f"{d}/renders/object.glb"+ try:+ r = subprocess.run([BL, "--background", "--threads", "3", "--python", f"{ROOT}/blender_glb.py",+ "--", glbp, f"{d}/renders", "1024", "16", "8", MODE],+ env=ENV, capture_output=True, text=True, timeout=300)+ return n, ("ok" if "GLB_RENDER_DONE" in r.stdout else "render_fail")+ except Exception as e:+ return n, "err:%s" % str(e)[:40]+++def pending():+ out = []+ for n in os.listdir(DD):+ d = f"{DD}/{n}"+ if os.path.exists(f"{d}/renders/object.glb") and not os.path.exists(f"{d}/renders/view_07.png"):+ out.append(n)+ return out+++def main():+ print(f"glb-render consumer | workers={WORKERS} mode={MODE}", flush=True)+ t0 = time.time(); total_ok = total_fail = 0; idle = 0+ while True:+ ps = pending()+ if not ps:+ if not bv_running():+ idle += 1+ if idle >= 2:+ break+ time.sleep(15)+ continue+ idle = 0+ with ThreadPoolExecutor(max_workers=WORKERS) as ex:+ futs = {ex.submit(render, n): n for n in ps}+ for fut in as_completed(futs):+ n, st = fut.result()+ if st == "ok":+ total_ok += 1+ else:+ total_fail += 1+ print(f" rendered batch: ok={total_ok} fail={total_fail} elapsed {time.time()-t0:.0f}s", flush=True)+ print(f"GLB_RENDER_ALL_DONE ok={total_ok} fail={total_fail} in {time.time()-t0:.0f}s", flush=True)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/articraft/shim_parts/brackets_hinges_bezels.py b/toolkits/curation/articraft/shim_parts/brackets_hinges_bezels.pynew file mode 100644index 00000000..f6c96e85--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/brackets_hinges_bezels.py@@ -0,0 +1,2239 @@+from __future__ import annotations++from articraft_cq import (cq, math, MeshGeometry, _CQMesh, _to_shape, mesh_from_geometry,+ _revolve_profile, _extrude_profile, _loft_rings, rounded_rect_profile,+ _profile_points_2d, _ensure_ccw, boolean_difference, boolean_union, boolean_intersection,+ Box, Cylinder, Sphere, ValidationError,+ superellipse_profile, Origin, Part, Visual)++# ===========================================================================+# @section brackets_hinges_bezels+#+# Clean-room CadQuery B-rep reimplementation of the Articraft sdk+# brackets / yokes / forks / bezels / piano-hinge family plus the placement+# helpers (place_on_*, surface_frame, wrap_*, cut_opening_on_face, ...).+# Constructor signatures mirror the real sdk EXACTLY; the produced shapes are+# parametric solids (the sdk builds these with cadquery then tessellates --+# here we keep the B-rep). Placement helpers reproduce the sdk's analytic+# geometry math so model scripts get identical Origins / frames.+# ===========================================================================+from contextlib import suppress+from dataclasses import dataclass+from typing import Iterable, Literal, Optional, Sequence, Tuple, Union++_EPS = 1e-9+++# ---------------------------------------------------------------------------+# shared finalize helpers (center=False -> rest the part's min-z on z=0)+# ---------------------------------------------------------------------------+def _brackets_hinges_bezels_shift_to_z0(shape):+ bb = shape.BoundingBox()+ return shape.translate(cq.Vector(0.0, 0.0, -bb.zmin))+++def _brackets_hinges_bezels_finish(target, model, center):+ shape = _to_shape(model)+ if not center:+ shape = _brackets_hinges_bezels_shift_to_z0(shape)+ target._set(shape)+++# ===========================================================================+# @section brackets (ClevisBracket / PivotFork / TrunnionYoke)+# ===========================================================================+class ClevisBracketGeometry(MeshGeometry):+ """Build a U-shaped clevis bracket with a bottom base and a transverse pin bore."""++ def __init__(+ self,+ overall_size: Sequence[float],+ *,+ gap_width: float,+ bore_diameter: float,+ bore_center_z: float,+ base_thickness: float,+ corner_radius: float = 0.0,+ center: bool = True,+ ):+ super().__init__()+ width = float(overall_size[0])+ depth = float(overall_size[1])+ height = float(overall_size[2])+ gap_width = float(gap_width)+ bore_diameter = float(bore_diameter)+ bore_center_z = float(bore_center_z)+ base_thickness = float(base_thickness)+ corner_radius = max(0.0, float(corner_radius))++ if width <= 0.0 or depth <= 0.0 or height <= 0.0:+ raise ValueError("overall_size values must be positive")+ if gap_width <= 0.0 or gap_width >= width:+ raise ValueError("gap_width must be positive and less than overall_size[0]")+ cheek_thickness = 0.5 * (width - gap_width)+ if cheek_thickness <= 1e-6:+ raise ValueError("gap_width leaves no side wall material")+ if bore_diameter <= 0.0 or bore_diameter >= min(cheek_thickness * 2.0, depth, height):+ raise ValueError("bore_diameter is too large for the clevis envelope")+ if base_thickness <= 0.0 or base_thickness >= height:+ raise ValueError("base_thickness must be positive and less than overall_size[2]")+ bore_radius = bore_diameter * 0.5+ if bore_center_z - bore_radius <= base_thickness or bore_center_z + bore_radius >= height:+ raise ValueError(+ "bore_center_z must leave material above the base and below the top edge"+ )++ shape = cq.Workplane("XY").box(width, depth, height)+ slot_cut = (+ cq.Workplane("XY")+ .box(gap_width, depth + 0.004, height - base_thickness)+ .translate((0.0, 0.0, base_thickness * 0.5))+ )+ shape = shape.cut(slot_cut)+ if corner_radius > 0.0:+ shape = shape.edges("|Z").fillet(+ min(corner_radius, cheek_thickness * 0.6, depth * 0.25, height * 0.25)+ )++ bore_z = -height * 0.5 + bore_center_z+ bore = (+ cq.Workplane("YZ")+ .circle(bore_radius)+ .extrude(width + 0.01, both=True)+ .translate((0.0, 0.0, bore_z))+ )+ shape = shape.cut(bore)++ _brackets_hinges_bezels_finish(self, shape, center)+++class PivotForkGeometry(MeshGeometry):+ """Build an open-front pivot fork with a rear bridge and a transverse pin bore."""++ def __init__(+ self,+ overall_size: Sequence[float],+ *,+ gap_width: float,+ bore_diameter: float,+ bore_center_z: float,+ bridge_thickness: float,+ corner_radius: float = 0.0,+ center: bool = True,+ ):+ super().__init__()+ width = float(overall_size[0])+ depth = float(overall_size[1])+ height = float(overall_size[2])+ gap_width = float(gap_width)+ bore_diameter = float(bore_diameter)+ bore_center_z = float(bore_center_z)+ bridge_thickness = float(bridge_thickness)+ corner_radius = max(0.0, float(corner_radius))++ if width <= 0.0 or depth <= 0.0 or height <= 0.0:+ raise ValueError("overall_size values must be positive")+ if gap_width <= 0.0 or gap_width >= width:+ raise ValueError("gap_width must be positive and less than overall_size[0]")+ cheek_thickness = 0.5 * (width - gap_width)+ if cheek_thickness <= 1e-6:+ raise ValueError("gap_width leaves no side wall material")+ if bridge_thickness <= 0.0 or bridge_thickness >= depth:+ raise ValueError("bridge_thickness must be positive and less than overall_size[1]")+ if bore_diameter <= 0.0 or bore_diameter >= min(cheek_thickness * 2.0, depth, height):+ raise ValueError("bore_diameter is too large for the pivot fork envelope")+ bore_radius = bore_diameter * 0.5+ if bore_center_z - bore_radius <= 0.0 or bore_center_z + bore_radius >= height:+ raise ValueError("bore_center_z must keep the bore inside the fork cheeks")++ tine_depth = depth+ left_tine = (+ cq.Workplane("XY")+ .box(cheek_thickness, tine_depth, height)+ .translate((-(gap_width * 0.5 + cheek_thickness * 0.5), 0.0, 0.0))+ )+ right_tine = (+ cq.Workplane("XY")+ .box(cheek_thickness, tine_depth, height)+ .translate(((gap_width * 0.5 + cheek_thickness * 0.5), 0.0, 0.0))+ )+ rear_bridge = (+ cq.Workplane("XY")+ .box(width, bridge_thickness, height)+ .translate((0.0, -depth * 0.5 + bridge_thickness * 0.5, 0.0))+ )+ shape = left_tine.union(right_tine).union(rear_bridge)+ if corner_radius > 0.0:+ shape = shape.edges("|Z").fillet(+ min(corner_radius, cheek_thickness * 0.6, bridge_thickness * 0.6, height * 0.25)+ )++ bore_z = -height * 0.5 + bore_center_z+ bore = (+ cq.Workplane("YZ")+ .circle(bore_radius)+ .extrude(width + 0.01, both=True)+ .translate((0.0, 0.0, bore_z))+ )+ shape = shape.cut(bore)++ _brackets_hinges_bezels_finish(self, shape, center)+++class TrunnionYokeGeometry(MeshGeometry):+ """Build a trunnion support yoke with a bottom base and cheek-mounted trunnion bores."""++ def __init__(+ self,+ overall_size: Sequence[float],+ *,+ span_width: float,+ trunnion_diameter: float,+ trunnion_center_z: float,+ base_thickness: float,+ corner_radius: float = 0.0,+ center: bool = True,+ ):+ super().__init__()+ width = float(overall_size[0])+ depth = float(overall_size[1])+ height = float(overall_size[2])+ span_width = float(span_width)+ trunnion_diameter = float(trunnion_diameter)+ trunnion_center_z = float(trunnion_center_z)+ base_thickness = float(base_thickness)+ corner_radius = max(0.0, float(corner_radius))++ if width <= 0.0 or depth <= 0.0 or height <= 0.0:+ raise ValueError("overall_size values must be positive")+ if span_width <= 0.0 or span_width >= width:+ raise ValueError("span_width must be positive and less than overall_size[0]")+ cheek_thickness = 0.5 * (width - span_width)+ if cheek_thickness <= 1e-6:+ raise ValueError("span_width leaves no side wall material")+ if base_thickness <= 0.0 or base_thickness >= height:+ raise ValueError("base_thickness must be positive and less than overall_size[2]")+ if trunnion_diameter <= 0.0 or trunnion_diameter >= min(+ cheek_thickness * 2.0, depth, height+ ):+ raise ValueError("trunnion_diameter is too large for the yoke envelope")+ trunnion_radius = trunnion_diameter * 0.5+ if (+ trunnion_center_z - trunnion_radius <= base_thickness+ or trunnion_center_z + trunnion_radius >= height+ ):+ raise ValueError(+ "trunnion_center_z must leave material above the base and below the top edge"+ )++ base = (+ cq.Workplane("XY")+ .box(width, depth, base_thickness)+ .translate((0.0, 0.0, -height * 0.5 + base_thickness * 0.5))+ )+ cheek_height = height - base_thickness+ cheek_z = -height * 0.5 + base_thickness + cheek_height * 0.5+ left_cheek = (+ cq.Workplane("XY")+ .box(cheek_thickness, depth, cheek_height)+ .translate((-(span_width * 0.5 + cheek_thickness * 0.5), 0.0, cheek_z))+ )+ right_cheek = (+ cq.Workplane("XY")+ .box(cheek_thickness, depth, cheek_height)+ .translate(((span_width * 0.5 + cheek_thickness * 0.5), 0.0, cheek_z))+ )+ boss_radius = max(trunnion_radius * 1.4, cheek_thickness * 0.55)+ boss_length = min(cheek_thickness * 0.75, depth * 0.35)+ boss_z = -height * 0.5 + trunnion_center_z+ left_boss = (+ cq.Workplane("YZ")+ .circle(boss_radius)+ .extrude(boss_length, both=False)+ .translate((-(span_width * 0.5 + cheek_thickness), 0.0, boss_z))+ )+ right_boss = (+ cq.Workplane("YZ")+ .circle(boss_radius)+ .extrude(-boss_length, both=False)+ .translate(((span_width * 0.5 + cheek_thickness), 0.0, boss_z))+ )+ shape = base.union(left_cheek).union(right_cheek).union(left_boss).union(right_boss)+ if corner_radius > 0.0:+ shape = shape.edges("|Z").fillet(+ min(corner_radius, cheek_thickness * 0.5, depth * 0.2, height * 0.2)+ )++ trunnion_bore = (+ cq.Workplane("YZ")+ .circle(trunnion_radius)+ .extrude(+ width + boss_length * 2.0 + 0.01,+ both=True,+ )+ .translate((0.0, 0.0, boss_z))+ )+ shape = shape.cut(trunnion_bore)++ _brackets_hinges_bezels_finish(self, shape, center)+++# ===========================================================================+# @section bezels (BezelGeometry + sub-feature specs)+# ===========================================================================+Vec2 = Tuple[float, float]+++@dataclass(frozen=True)+class BezelFace:+ style: Literal["flat", "rounded", "chamfered", "radiused_step"] = "flat"+ front_lip: float = 0.0+ chamfer: float = 0.0+ fillet: float = 0.0+++@dataclass(frozen=True)+class BezelRecess:+ depth: float+ inset: float+ floor_radius: float = 0.0+ wall_draft_deg: float = 0.0+++@dataclass(frozen=True)+class BezelVisor:+ top_extension: float = 0.0+ side_extension: float = 0.0+ thickness: float = 0.0+ draft_deg: float = 0.0+++@dataclass(frozen=True)+class BezelFlange:+ width: float = 0.0+ thickness: float = 0.0+ offset: float = 0.0+++@dataclass(frozen=True)+class BezelMounts:+ style: Literal["none", "bosses", "tabs", "rear_flange"] = "none"+ hole_count: int = 0+ hole_diameter: Optional[float] = None+ boss_diameter: Optional[float] = None+ setback: float = 0.0+++@dataclass(frozen=True)+class BezelCutout:+ edge: Literal["top", "bottom", "left", "right"]+ width: float+ depth: float+ offset: float = 0.0+++@dataclass(frozen=True)+class BezelEdgeFeature:+ style: Literal["bead", "step", "notch"] = "bead"+ edge: Literal["top", "bottom", "left", "right"] = "top"+ size: float = 0.0+ offset: float = 0.0+ extent: float = 0.0+++# ---- profile helpers copied from the sdk cadquery_helpers --------------------+def _sample_ellipse_profile(width: float, height: float, *, segments: int = 64):+ rx = float(width) * 0.5+ ry = float(height) * 0.5+ return [+ (rx * math.cos(2.0 * math.pi * index / float(segments)),+ ry * math.sin(2.0 * math.pi * index / float(segments)))+ for index in range(segments)+ ]+++def _sample_rect_profile(width: float, height: float):+ hw = float(width) * 0.5+ hh = float(height) * 0.5+ return [(-hw, -hh), (hw, -hh), (hw, hh), (-hw, hh)]+++def _shape_profile_points_2d(shape, size, *, corner_radius: float = 0.0, segments: int = 64):+ width = float(size[0])+ height = float(size[1])+ if width <= 0.0 or height <= 0.0:+ raise ValueError("shape size values must be positive")+ if shape == "rect":+ return _sample_rect_profile(width, height)+ if shape == "rounded_rect":+ return rounded_rect_profile(+ width, height, max(0.0, corner_radius), corner_segments=max(4, segments // 16)+ )+ if shape == "circle":+ diameter = min(width, height)+ return _sample_ellipse_profile(diameter, diameter, segments=segments)+ if shape == "ellipse":+ return _sample_ellipse_profile(width, height, segments=segments)+ if shape == "superellipse":+ return superellipse_profile(width, height, exponent=2.8, segments=segments)+ raise ValueError(f"Unsupported shape {shape!r}")+++def _shape_size_from_wall(inner_size, wall):+ inner_w = float(inner_size[0])+ inner_h = float(inner_size[1])+ if isinstance(wall, (int, float)):+ wall_left = wall_right = wall_bottom = wall_top = float(wall)+ else:+ if len(wall) != 4:+ raise ValueError("wall must be a float or a 4-sequence")+ wall_left, wall_right, wall_bottom, wall_top = (float(value) for value in wall)+ if min(wall_left, wall_right, wall_bottom, wall_top) < 0.0:+ raise ValueError("wall values must be non-negative")+ return (inner_w + wall_left + wall_right, inner_h + wall_bottom + wall_top)+++def _cq_polyline_wire(cq_module, points, plane: str = "XY"):+ return cq_module.Workplane(plane).polyline(points).close()+++def _cq_ring_solid(cq_module, outer_points, inner_points, depth: float, *, center: bool = True):+ shape = _cq_polyline_wire(cq_module, outer_points).extrude(float(depth) * 0.5, both=center)+ inner_cut = _cq_polyline_wire(cq_module, inner_points).extrude(+ float(depth) + max(0.002, float(depth) * 0.5),+ both=center,+ )+ return shape.cut(inner_cut)+++def _brackets_hinges_bezels_centered_pattern_positions(count: int, spacing: float):+ if count <= 0:+ return []+ if count == 1:+ return [0.0]+ origin = -0.5 * float(spacing) * float(count - 1)+ return [origin + float(index) * float(spacing) for index in range(count)]+++class BezelGeometry(MeshGeometry):+ """Build a framed opening with optional recess, visor, flange, and rear mounts."""++ def __init__(+ self,+ opening_size: Sequence[float],+ outer_size: Sequence[float],+ depth: float,+ *,+ opening_shape: Literal[+ "rect", "rounded_rect", "circle", "ellipse", "superellipse"+ ] = "rounded_rect",+ outer_shape: Literal[+ "rect", "rounded_rect", "circle", "ellipse", "superellipse"+ ] = "rounded_rect",+ opening_corner_radius: float = 0.0,+ outer_corner_radius: float = 0.0,+ wall: Union[float, tuple[float, float, float, float], None] = None,+ face: Optional[BezelFace] = None,+ recess: Optional[BezelRecess] = None,+ visor: Optional[BezelVisor] = None,+ flange: Optional[BezelFlange] = None,+ mounts: Optional[BezelMounts] = None,+ cutouts: Sequence[BezelCutout] = (),+ edge_features: Sequence[BezelEdgeFeature] = (),+ center: bool = True,+ ):+ super().__init__()+ opening_w = float(opening_size[0])+ opening_h = float(opening_size[1])+ outer_w = float(outer_size[0])+ outer_h = float(outer_size[1])+ depth = float(depth)+ opening_corner_radius = max(0.0, float(opening_corner_radius))+ outer_corner_radius = max(0.0, float(outer_corner_radius))+ face = face or BezelFace()+ visor = visor or BezelVisor()+ flange = flange or BezelFlange()+ mounts = mounts or BezelMounts()++ if min(opening_w, opening_h, outer_w, outer_h, depth) <= 0.0:+ raise ValueError("opening_size, outer_size, and depth must be positive")+ if opening_w >= outer_w or opening_h >= outer_h:+ raise ValueError("opening_size must be smaller than outer_size on both axes")+ if recess is not None and (recess.depth <= 0.0 or recess.inset < 0.0):+ raise ValueError("BezelRecess depth must be positive and inset must be non-negative")++ outer_points = _shape_profile_points_2d(+ outer_shape, (outer_w, outer_h), corner_radius=outer_corner_radius+ )+ opening_points = _shape_profile_points_2d(+ opening_shape, (opening_w, opening_h), corner_radius=opening_corner_radius+ )+ shape = _cq_ring_solid(cq, outer_points, opening_points, depth, center=True)++ if face.style in {"rounded", "radiused_step"} and face.fillet > 1e-6:+ with suppress(Exception):+ shape = shape.edges("|Z").fillet(+ min(face.fillet, depth * 0.25, min(outer_w, outer_h) * 0.1)+ )+ if face.style == "chamfered" and face.chamfer > 1e-6:+ with suppress(Exception):+ shape = shape.edges("|Z").chamfer(+ min(face.chamfer, depth * 0.25, min(outer_w, outer_h) * 0.1)+ )++ derived_wall = (+ (outer_w - opening_w) * 0.5,+ (outer_w - opening_w) * 0.5,+ (outer_h - opening_h) * 0.5,+ (outer_h - opening_h) * 0.5,+ )+ if face.front_lip > 1e-6 or face.style == "radiused_step":+ lip_thickness = max(face.front_lip, depth * 0.08, 0.0015)+ if wall is not None:+ lip_size = _shape_size_from_wall(opening_size, wall)+ else:+ lip_size = (+ min(+ opening_w + max(face.front_lip * 2.0, derived_wall[0] * 1.05),+ outer_w - 0.001,+ ),+ min(+ opening_h + max(face.front_lip * 2.0, derived_wall[2] * 1.05),+ outer_h - 0.001,+ ),+ )+ if lip_size[0] < outer_w and lip_size[1] < outer_h:+ lip_outer = _shape_profile_points_2d(+ opening_shape if outer_shape != "circle" else outer_shape,+ lip_size,+ corner_radius=min(+ opening_corner_radius + face.front_lip,+ lip_size[0] * 0.25,+ lip_size[1] * 0.25,+ ),+ )+ lip = _cq_ring_solid(+ cq, lip_outer, opening_points, lip_thickness, center=True+ ).translate((0.0, 0.0, depth * 0.5))+ shape = shape.union(lip)++ if recess is not None:+ requested_recess_size = (opening_w + recess.inset * 2.0, opening_h + recess.inset * 2.0)+ if wall is None and (+ requested_recess_size[0] >= outer_w or requested_recess_size[1] >= outer_h+ ):+ raise ValueError("recess wall leaves no outer frame material")+ recess_size = (+ _shape_size_from_wall(opening_size, wall)+ if wall is not None+ else (+ min(requested_recess_size[0], outer_w - 0.001),+ min(requested_recess_size[1], outer_h - 0.001),+ )+ )+ if recess_size[0] >= outer_w or recess_size[1] >= outer_h:+ raise ValueError("recess wall leaves no outer frame material")+ recess_points = _shape_profile_points_2d(+ opening_shape if outer_shape != "circle" else outer_shape,+ recess_size,+ corner_radius=min(+ opening_corner_radius + recess.inset,+ recess_size[0] * 0.25,+ recess_size[1] * 0.25,+ ),+ )+ recess_cut = _cq_ring_solid(+ cq, recess_points, opening_points, recess.depth, center=True+ ).translate((0.0, 0.0, depth * 0.5 - recess.depth * 0.5))+ shape = shape.cut(recess_cut)++ if visor.thickness > 1e-6 and (visor.top_extension > 1e-6 or visor.side_extension > 1e-6):+ top_visor = (+ cq.Workplane("XY")+ .box(+ outer_w + visor.side_extension * 2.0,+ max(visor.top_extension, visor.thickness),+ visor.thickness,+ )+ .translate((0.0, outer_h * 0.5 + visor.top_extension * 0.5, depth * 0.5))+ )+ shape = shape.union(top_visor)+ if visor.side_extension > 1e-6:+ cheek_y = max(visor.top_extension, outer_h * 0.5) * 0.5+ cheek = cq.Workplane("XY").box(+ visor.side_extension,+ max(visor.top_extension, outer_h * 0.45),+ visor.thickness,+ )+ shape = shape.union(+ cheek.translate(+ (outer_w * 0.5 + visor.side_extension * 0.5, cheek_y, depth * 0.5)+ )+ )+ shape = shape.union(+ cheek.translate(+ (-(outer_w * 0.5 + visor.side_extension * 0.5), cheek_y, depth * 0.5)+ )+ )++ if flange.width > 1e-6 and flange.thickness > 1e-6:+ flange_outer = _shape_profile_points_2d(+ outer_shape,+ (outer_w + flange.width * 2.0, outer_h + flange.width * 2.0),+ corner_radius=outer_corner_radius + flange.width,+ )+ flange_shape = _cq_ring_solid(+ cq, flange_outer, outer_points, flange.thickness, center=True+ ).translate((0.0, 0.0, -depth * 0.5 - flange.offset))+ shape = shape.union(flange_shape)++ if mounts.style == "bosses" and mounts.hole_count > 0:+ boss_radius = (+ max(float(mounts.boss_diameter) * 0.5, 0.0015)+ if mounts.boss_diameter is not None+ else max(min(outer_w, outer_h) * 0.05, 0.003)+ )+ hole_radius = (+ max(float(mounts.hole_diameter) * 0.5, 0.0008)+ if mounts.hole_diameter is not None+ else boss_radius * 0.38+ )+ boss_thickness = max(depth * 0.18, boss_radius * 0.8)+ margin_x = outer_w * 0.5 - boss_radius - max(mounts.setback, 0.001)+ margin_y = outer_h * 0.5 - boss_radius - max(mounts.setback, 0.001)+ boss_points = [+ (-margin_x, -margin_y),+ (margin_x, -margin_y),+ (margin_x, margin_y),+ (-margin_x, margin_y),+ ][: mounts.hole_count]+ for bx, by in boss_points:+ boss = (+ cq.Workplane("XY")+ .circle(boss_radius)+ .extrude(boss_thickness)+ .translate((bx, by, -depth * 0.5 - boss_thickness))+ )+ hole = (+ cq.Workplane("XY")+ .circle(hole_radius)+ .extrude(boss_thickness + depth + 0.01)+ .translate((bx, by, -depth * 0.5 - boss_thickness))+ )+ shape = shape.union(boss).cut(hole)+ elif mounts.style == "tabs" and mounts.hole_count > 0:+ tab_width = max(outer_w * 0.16, 0.008)+ tab_depth = max(depth * 0.14, 0.002)+ hole_radius = (+ max(float(mounts.hole_diameter) * 0.5, 0.0008)+ if mounts.hole_diameter is not None+ else tab_width * 0.14+ )+ tab_positions = _brackets_hinges_bezels_centered_pattern_positions(+ mounts.hole_count, outer_w / max(mounts.hole_count, 1)+ )+ for px in tab_positions:+ tab = (+ cq.Workplane("XY")+ .box(tab_width, tab_width * 0.6, tab_depth)+ .translate(+ (px, -(outer_h * 0.5 + tab_width * 0.3), -depth * 0.5 - tab_depth * 0.5)+ )+ )+ hole = (+ cq.Workplane("XY")+ .circle(hole_radius)+ .extrude(tab_depth + depth + 0.01)+ .translate((px, -(outer_h * 0.5 + tab_width * 0.3), -depth * 0.5 - tab_depth))+ )+ shape = shape.union(tab).cut(hole)+ elif (+ mounts.style == "rear_flange"+ and mounts.hole_count > 0+ and mounts.hole_diameter is not None+ ):+ flange_width = max(max(mounts.setback, 0.003), min(outer_w, outer_h) * 0.06)+ rear_flange_outer = _shape_profile_points_2d(+ outer_shape,+ (outer_w + flange_width * 2.0, outer_h + flange_width * 2.0),+ corner_radius=outer_corner_radius + flange_width,+ )+ rear_flange = _cq_ring_solid(+ cq, rear_flange_outer, outer_points, max(depth * 0.12, 0.002), center=True+ ).translate((0.0, 0.0, -depth * 0.5 - max(depth * 0.12, 0.002)))+ shape = shape.union(rear_flange)++ for cutout in cutouts:+ if cutout.width <= 0.0 or cutout.depth <= 0.0:+ raise ValueError("BezelCutout width and depth must be positive")+ cut_height = cutout.width+ cut_depth = cutout.depth+ if cutout.edge in {"top", "bottom"}:+ cutter = cq.Workplane("XY").box(+ cutout.width, cut_depth, depth + visor.thickness + 0.02+ )+ y = (+ outer_h * 0.5 - cut_depth * 0.5+ if cutout.edge == "top"+ else -outer_h * 0.5 + cut_depth * 0.5+ )+ cutter = cutter.translate((cutout.offset, y, 0.0))+ else:+ cutter = cq.Workplane("XY").box(+ cut_depth, cut_height, depth + visor.thickness + 0.02+ )+ x = (+ outer_w * 0.5 - cut_depth * 0.5+ if cutout.edge == "right"+ else -outer_w * 0.5 + cut_depth * 0.5+ )+ cutter = cutter.translate((x, cutout.offset, 0.0))+ shape = shape.cut(cutter)++ for feature in edge_features:+ if feature.size <= 0.0:+ continue+ extent = (+ feature.extent+ if feature.extent > 0.0+ else (outer_w if feature.edge in {"top", "bottom"} else outer_h)+ )+ if feature.style == "notch":+ if feature.edge in {"top", "bottom"}:+ cutter = cq.Workplane("XY").box(+ extent, feature.size, depth + visor.thickness + 0.02+ )+ y = (+ outer_h * 0.5 - feature.size * 0.5+ if feature.edge == "top"+ else -outer_h * 0.5 + feature.size * 0.5+ )+ cutter = cutter.translate((feature.offset, y, 0.0))+ else:+ cutter = cq.Workplane("XY").box(+ feature.size, extent, depth + visor.thickness + 0.02+ )+ x = (+ outer_w * 0.5 - feature.size * 0.5+ if feature.edge == "right"+ else -outer_w * 0.5 + feature.size * 0.5+ )+ cutter = cutter.translate((x, feature.offset, 0.0))+ shape = shape.cut(cutter)+ continue+ if feature.edge in {"top", "bottom"}:+ solid = cq.Workplane("XY").box(extent, feature.size, max(depth * 0.10, 0.0015))+ y = (+ outer_h * 0.5 + feature.size * 0.5+ if feature.edge == "top"+ else -(outer_h * 0.5 + feature.size * 0.5)+ )+ solid = solid.translate((feature.offset, y, depth * 0.5))+ else:+ solid = cq.Workplane("XY").box(feature.size, extent, max(depth * 0.10, 0.0015))+ x = (+ outer_w * 0.5 + feature.size * 0.5+ if feature.edge == "right"+ else -(outer_w * 0.5 + feature.size * 0.5)+ )+ solid = solid.translate((x, feature.offset, depth * 0.5))+ shape = (+ shape.union(solid)+ if feature.style == "bead"+ else shape.cut(solid.translate((0.0, 0.0, -max(depth * 0.04, 0.0008))))+ )++ _brackets_hinges_bezels_finish(self, shape, center)+++# ===========================================================================+# @section hinges (PianoHingeGeometry + HingeHolePattern / HingePinStyle specs)+# ===========================================================================+@dataclass(frozen=True)+class HingeHolePattern:+ style: Literal["none", "round", "countersunk", "slotted"] = "none"+ count: int = 0+ diameter: Optional[float] = None+ slot_size: Optional[tuple[float, float]] = None+ edge_margin: float = 0.0+ pitch: Optional[float] = None+++@dataclass(frozen=True)+class HingePinStyle:+ head_style: Literal["plain", "button", "flat", "peened"] = "plain"+ head_height: float = 0.0+ head_diameter: Optional[float] = None+ exposed_end: float = 0.0+++def _brackets_hinges_bezels_rounded_slot_profile(length: float, width: float, *, segments: int = 10):+ radius = width * 0.5+ straight = max(length - width, 0.0)+ half = straight * 0.5+ points = []+ for index in range(segments + 1):+ theta = -math.pi * 0.5 + math.pi * (index / float(segments))+ points.append((half + radius * math.cos(theta), radius * math.sin(theta)))+ for index in range(segments + 1):+ theta = math.pi * 0.5 + math.pi * (index / float(segments))+ points.append((-half + radius * math.cos(theta), radius * math.sin(theta)))+ return points+++class _BarrelHingeGeometry(MeshGeometry):+ """Two-leaf knuckle hinge around a local Z pin axis (sdk BarrelHingeGeometry).++ Kept private; PianoHingeGeometry builds on it (matches the sdk module+ structure where PianoHinge delegates to BarrelHinge)."""++ def __init__(+ self,+ length: float,+ *,+ leaf_width_a: float,+ leaf_width_b: Optional[float] = None,+ leaf_thickness: float,+ pin_diameter: float,+ knuckle_outer_diameter: Optional[float] = None,+ knuckle_count: int = 5,+ clearance: float = 0.0005,+ open_angle_deg: float = 180.0,+ holes_a: Optional[HingeHolePattern] = None,+ holes_b: Optional[HingeHolePattern] = None,+ pin: Optional[HingePinStyle] = None,+ center: bool = True,+ ):+ super().__init__()+ length = float(length)+ leaf_width_a = float(leaf_width_a)+ leaf_width_b = float(leaf_width_b) if leaf_width_b is not None else leaf_width_a+ leaf_thickness = float(leaf_thickness)+ pin_diameter = float(pin_diameter)+ knuckle_outer_diameter = (+ float(knuckle_outer_diameter)+ if knuckle_outer_diameter is not None+ else pin_diameter * 1.75+ )+ clearance = float(clearance)+ open_angle_deg = float(open_angle_deg)+ holes_a = holes_a or HingeHolePattern()+ holes_b = holes_b or HingeHolePattern()+ pin = pin or HingePinStyle()++ if (+ min(length, leaf_width_a, leaf_width_b, leaf_thickness, pin_diameter,+ knuckle_outer_diameter)+ <= 0.0+ ):+ raise ValueError(+ "length, leaf widths, thickness, pin_diameter, and knuckle size must be positive"+ )+ if knuckle_count < 3:+ raise ValueError("knuckle_count must be at least 3")+ if pin_diameter >= knuckle_outer_diameter:+ raise ValueError("pin_diameter must be less than knuckle_outer_diameter")+ segment_length = (length - clearance * float(knuckle_count - 1)) / float(knuckle_count)+ if segment_length <= 0.0:+ raise ValueError("clearance/knuckle_count leave no knuckle length")++ leaf_overlap = min(leaf_thickness * 0.75, knuckle_outer_diameter * 0.12)+ leaf_a = (+ cq.Workplane("XY")+ .box(leaf_width_a, leaf_thickness, length)+ .translate(+ (-(knuckle_outer_diameter * 0.5 + leaf_width_a * 0.5 - leaf_overlap), 0.0, 0.0)+ )+ )+ leaf_b = (+ cq.Workplane("XY")+ .box(leaf_width_b, leaf_thickness, length)+ .translate(+ ((knuckle_outer_diameter * 0.5 + leaf_width_b * 0.5 - leaf_overlap), 0.0, 0.0)+ )+ )+ leaf_b = leaf_b.rotate((0.0, 0.0, 0.0), (0.0, 0.0, 1.0), 180.0 - open_angle_deg)+ shape = leaf_a.union(leaf_b)++ z_start = -length * 0.5+ for index in range(knuckle_count):+ center_z = z_start + segment_length * 0.5 + index * (segment_length + clearance)+ knuckle = (+ cq.Workplane("XY")+ .circle(knuckle_outer_diameter * 0.5)+ .extrude(segment_length)+ .translate((0.0, 0.0, center_z - segment_length * 0.5))+ )+ shape = shape.union(knuckle)++ pin_len = length + pin.exposed_end * 2.0+ pin_bottom = -length * 0.5 - pin.exposed_end+ pin_shape = (+ cq.Workplane("XY")+ .circle(pin_diameter * 0.5)+ .extrude(pin_len)+ .translate((0.0, 0.0, pin_bottom))+ )+ if pin.head_style != "plain" and pin.head_height > 1e-6:+ head_d = pin.head_diameter or pin_diameter * 1.6+ head = cq.Workplane("XY").circle(head_d * 0.5).extrude(pin.head_height)+ pin_shape = pin_shape.union(head.translate((0.0, 0.0, pin_bottom - pin.head_height)))+ if pin.head_style != "peened":+ pin_shape = pin_shape.union(head.translate((0.0, 0.0, pin_bottom + pin_len)))+ shape = shape.union(pin_shape)++ def _apply_holes(base_shape, pattern, side):+ if pattern.style == "none" or pattern.count <= 0:+ return base_shape+ if pattern.diameter is None and pattern.slot_size is None:+ raise ValueError("HingeHolePattern requires diameter or slot_size")+ count = pattern.count+ z_positions = _brackets_hinges_bezels_centered_pattern_positions(+ count,+ pattern.pitch+ if pattern.pitch is not None+ else (length - pattern.edge_margin * 2.0) / max(count - 1, 1),+ )+ x_center = side * (+ knuckle_outer_diameter * 0.5 + (leaf_width_a if side < 0.0 else leaf_width_b) * 0.5+ )+ for z_pos in z_positions:+ if pattern.style == "slotted":+ if pattern.slot_size is None:+ raise ValueError("Slotted hinge holes require slot_size")+ slot_profile = _brackets_hinges_bezels_rounded_slot_profile(pattern.slot_size[0], pattern.slot_size[1])+ cutter = (+ cq.Workplane("XZ")+ .polyline(+ [(x_center + point[0], z_pos + point[1]) for point in slot_profile]+ )+ .close()+ .extrude(leaf_thickness + 0.01, both=True)+ )+ else:+ hole_r = (pattern.diameter or 0.0) * 0.5+ cutter = (+ cq.Workplane("XZ")+ .circle(hole_r)+ .extrude(leaf_thickness + 0.01, both=True)+ .translate((x_center, 0.0, z_pos))+ )+ base_shape = base_shape.cut(cutter)+ return base_shape++ shape = _apply_holes(shape, holes_a, -1.0)+ shape = _apply_holes(shape, holes_b, 1.0)++ _brackets_hinges_bezels_finish(self, shape, center)+++class PianoHingeGeometry(MeshGeometry):+ """Build a continuous piano hinge strip around a local Z pin axis."""++ def __init__(+ self,+ length: float,+ *,+ leaf_width_a: float,+ leaf_width_b: Optional[float] = None,+ leaf_thickness: float,+ pin_diameter: float,+ knuckle_pitch: float,+ clearance: float = 0.0005,+ open_angle_deg: float = 180.0,+ holes_a: Optional[HingeHolePattern] = None,+ holes_b: Optional[HingeHolePattern] = None,+ pin: Optional[HingePinStyle] = None,+ center: bool = True,+ ):+ super().__init__()+ knuckle_pitch = float(knuckle_pitch)+ if knuckle_pitch <= 0.0:+ raise ValueError("knuckle_pitch must be positive")+ knuckle_count = max(3, int(length / knuckle_pitch))+ if knuckle_count % 2 == 0:+ knuckle_count += 1+ knuckle_outer_diameter = pin_diameter * 1.55+ base = _BarrelHingeGeometry(+ length,+ leaf_width_a=leaf_width_a,+ leaf_width_b=leaf_width_b,+ leaf_thickness=leaf_thickness,+ pin_diameter=pin_diameter,+ knuckle_outer_diameter=knuckle_outer_diameter,+ knuckle_count=knuckle_count,+ clearance=clearance,+ open_angle_deg=open_angle_deg,+ holes_a=holes_a,+ holes_b=holes_b,+ pin=pin,+ center=center,+ )+ self._set(base._cq())+++# ===========================================================================+# @section placement-math (Vec/Mat helpers ported from sdk geometry_qc /+# placement -- pure analytic, no triangle meshes)+# ===========================================================================+def _dot(a, b):+ return float(a[0]) * float(b[0]) + float(a[1]) * float(b[1]) + float(a[2]) * float(b[2])+++def _cross(a, b):+ return (a[1] * b[2] - a[2] * b[1], a[2] * b[0] - a[0] * b[2], a[0] * b[1] - a[1] * b[0])+++def _norm(v):+ return math.sqrt(_dot(v, v))+++def _normalize_vec(v, *, name="vector"):+ n = _norm(v)+ if n <= _EPS:+ raise ValidationError(f"{name} must be non-zero")+ return (v[0] / n, v[1] / n, v[2] / n)+++def _brackets_hinges_bezels_clamp(value, lo, hi):+ return min(max(float(value), float(lo)), float(hi))+++def _identity4():+ return (+ (1.0, 0.0, 0.0, 0.0),+ (0.0, 1.0, 0.0, 0.0),+ (0.0, 0.0, 1.0, 0.0),+ (0.0, 0.0, 0.0, 1.0),+ )+++def _rpy_matrix(roll, pitch, yaw):+ cr, sr = math.cos(roll), math.sin(roll)+ cp, sp = math.cos(pitch), math.sin(pitch)+ cy, sy = math.cos(yaw), math.sin(yaw)+ return (+ (cy * cp, cy * sp * sr - sy * cr, cy * sp * cr + sy * sr),+ (sy * cp, sy * sp * sr + cy * cr, sy * sp * cr - cy * sr),+ (-sp, cp * sr, cp * cr),+ )+++def _origin_to_mat4(origin):+ if origin is None:+ return _identity4()+ ox, oy, oz = origin.xyz+ rr, rp, ry = origin.rpy+ r = _rpy_matrix(float(rr), float(rp), float(ry))+ return (+ (r[0][0], r[0][1], r[0][2], float(ox)),+ (r[1][0], r[1][1], r[1][2], float(oy)),+ (r[2][0], r[2][1], r[2][2], float(oz)),+ (0.0, 0.0, 0.0, 1.0),+ )+++def _mat4_mul(a, b):+ out = [[0.0] * 4 for _ in range(4)]+ for i in range(4):+ for j in range(4):+ out[i][j] = (+ a[i][0] * b[0][j] + a[i][1] * b[1][j] + a[i][2] * b[2][j] + a[i][3] * b[3][j]+ )+ return tuple(tuple(row) for row in out)+++def _mat4_vec3(mat, vec):+ x, y, z = vec+ return (+ mat[0][0] * x + mat[0][1] * y + mat[0][2] * z + mat[0][3],+ mat[1][0] * x + mat[1][1] * y + mat[1][2] * z + mat[1][3],+ mat[2][0] * x + mat[2][1] * y + mat[2][2] * z + mat[2][3],+ )+++def _mat3_from_mat4(mat):+ return (+ (mat[0][0], mat[0][1], mat[0][2]),+ (mat[1][0], mat[1][1], mat[1][2]),+ (mat[2][0], mat[2][1], mat[2][2]),+ )+++def _mat3_transpose(mat):+ return (+ (mat[0][0], mat[1][0], mat[2][0]),+ (mat[0][1], mat[1][1], mat[2][1]),+ (mat[0][2], mat[1][2], mat[2][2]),+ )+++def _mat3_vec3(mat, vec):+ return (+ mat[0][0] * vec[0] + mat[0][1] * vec[1] + mat[0][2] * vec[2],+ mat[1][0] * vec[0] + mat[1][1] * vec[1] + mat[1][2] * vec[2],+ mat[2][0] * vec[0] + mat[2][1] * vec[1] + mat[2][2] * vec[2],+ )+++def _mat3_mul(a, b):+ rows = []+ for i in range(3):+ rows.append(tuple(+ a[i][0] * b[0][j] + a[i][1] * b[1][j] + a[i][2] * b[2][j] for j in range(3)+ ))+ return (rows[0], rows[1], rows[2])+++def _mat3_from_columns(a, b, c):+ return ((a[0], b[0], c[0]), (a[1], b[1], c[1]), (a[2], b[2], c[2]))+++def _mat3_to_rpy(mat):+ pitch = math.asin(_brackets_hinges_bezels_clamp(-mat[2][0], -1.0, 1.0))+ cp = math.cos(pitch)+ if abs(cp) > 1e-8:+ roll = math.atan2(mat[2][1], mat[2][2])+ yaw = math.atan2(mat[1][0], mat[0][0])+ return (roll, pitch, yaw)+ roll = 0.0+ if pitch >= 0.0:+ yaw = math.atan2(-mat[0][1], mat[1][1])+ else:+ yaw = math.atan2(mat[0][1], mat[1][1])+ return (roll, pitch, yaw)+++def _mat4_inverse_rigid(tf):+ rot = _mat3_from_mat4(tf)+ rot_t = _mat3_transpose(rot)+ trans = (float(tf[0][3]), float(tf[1][3]), float(tf[2][3]))+ inv_trans = _mat3_vec3(rot_t, (-trans[0], -trans[1], -trans[2]))+ return (+ (rot_t[0][0], rot_t[0][1], rot_t[0][2], inv_trans[0]),+ (rot_t[1][0], rot_t[1][1], rot_t[1][2], inv_trans[1]),+ (rot_t[2][0], rot_t[2][1], rot_t[2][2], inv_trans[2]),+ (0.0, 0.0, 0.0, 1.0),+ )+++def _mat4_transform_direction(tf, vec):+ return _mat3_vec3(_mat3_from_mat4(tf), vec)+++def _axis_angle_matrix(axis, angle):+ ax = _normalize_vec(axis, name="axis")+ x, y, z = ax+ c = math.cos(angle)+ s = math.sin(angle)+ t = 1.0 - c+ return (+ (t * x * x + c, t * x * y - s * z, t * x * z + s * y),+ (t * x * y + s * z, t * y * y + c, t * y * z - s * x),+ (t * x * z - s * y, t * y * z + s * x, t * z * z + c),+ )+++def _transform_aabb(aabb, tf):+ (min_x, min_y, min_z), (max_x, max_y, max_z) = aabb+ corners = [(x, y, z) for x in (min_x, max_x) for y in (min_y, max_y) for z in (min_z, max_z)]+ tx0 = ty0 = tz0 = float("inf")+ tx1 = ty1 = tz1 = float("-inf")+ for c in corners:+ x, y, z = _mat4_vec3(tf, c)+ tx0, ty0, tz0 = min(tx0, x), min(ty0, y), min(tz0, z)+ tx1, ty1, tz1 = max(tx1, x), max(ty1, y), max(tz1, z)+ return (tx0, ty0, tz0), (tx1, ty1, tz1)+++def _aabb_union(items):+ if not items:+ raise ValidationError("Cannot union empty AABB list")+ mins = [float("inf")] * 3+ maxs = [float("-inf")] * 3+ for mn, mx in items:+ for i in range(3):+ mins[i] = min(mins[i], mn[i])+ maxs[i] = max(maxs[i], mx[i])+ return (mins[0], mins[1], mins[2]), (maxs[0], maxs[1], maxs[2])+++def _aabb_center(aabb):+ (mn, mx) = aabb+ return ((mn[0] + mx[0]) * 0.5, (mn[1] + mx[1]) * 0.5, (mn[2] + mx[2]) * 0.5)+++def _aabb_size(aabb):+ (mn, mx) = aabb+ return (mx[0] - mn[0], mx[1] - mn[1], mx[2] - mn[2])+++# ---------------------------------------------------------------------------+# subject iteration / local AABB (Box/Cylinder/Sphere analytic; any other+# geometry falls back to its cadquery B-rep bounding box)+# ---------------------------------------------------------------------------+def _geometry_local_aabb(geometry):+ if isinstance(geometry, Box):+ sx, sy, sz = geometry.size+ return ((-sx / 2.0, -sy / 2.0, -sz / 2.0), (sx / 2.0, sy / 2.0, sz / 2.0))+ if isinstance(geometry, Cylinder):+ r = float(geometry.radius)+ length = float(geometry.length)+ return ((-r, -r, -length / 2.0), (r, r, length / 2.0))+ if isinstance(geometry, Sphere):+ r = float(geometry.radius)+ return ((-r, -r, -r), (r, r, r))+ shape = _to_shape(geometry)+ bb = shape.BoundingBox()+ return ((bb.xmin, bb.ymin, bb.zmin), (bb.xmax, bb.ymax, bb.zmax))+++def _iter_subject_items(subject, *, prefer_collisions):+ if isinstance(subject, Part):+ collisions = list(getattr(subject, "collisions", []) or [])+ if prefer_collisions and collisions:+ items = collisions+ elif subject.visuals:+ items = list(subject.visuals)+ else:+ items = collisions+ return [+ (item.geometry, _origin_to_mat4(getattr(item, "origin", None)))+ for item in items+ if getattr(item, "geometry", None) is not None+ ]+ if isinstance(subject, Visual):+ return [(subject.geometry, _origin_to_mat4(subject.origin))]+ return [(subject, _identity4())]+++def link_local_aabbs(link, *, asset_root=None, prefer_collisions=True):+ items = _iter_subject_items(link, prefer_collisions=prefer_collisions)+ return [_transform_aabb(_geometry_local_aabb(g), tf) for g, tf in items]+++def link_local_aabb(link, *, asset_root=None, prefer_collisions=True):+ aabbs = link_local_aabbs(link, asset_root=asset_root, prefer_collisions=prefer_collisions)+ if not aabbs:+ return None+ return _aabb_union(aabbs)+++def _subject_world_aabb(subject, *, asset_root=None, prefer_collisions):+ aabbs = []+ for geometry, tf in _iter_subject_items(subject, prefer_collisions=prefer_collisions):+ aabbs.append(_transform_aabb(_geometry_local_aabb(geometry), tf))+ if not aabbs:+ raise ValidationError("subject has no geometry to compute AABB")+ return _aabb_union(aabbs)+++# ===========================================================================+# @section placement-surface-query (closest point on box/cylinder/sphere)+# ===========================================================================+@dataclass(frozen=True)+class SurfaceFrame:+ point: Tuple[float, float, float]+ normal: Tuple[float, float, float]+ tangent_u: Tuple[float, float, float]+ tangent_v: Tuple[float, float, float]+++@dataclass(frozen=True)+class _SurfaceHit:+ point: Tuple[float, float, float]+ normal: Tuple[float, float, float]+ distance: float+++def _closest_point_on_box(point, size):+ hx, hy, hz = float(size[0]) * 0.5, float(size[1]) * 0.5, float(size[2]) * 0.5+ x, y, z = point+ clamped = (_brackets_hinges_bezels_clamp(x, -hx, hx), _brackets_hinges_bezels_clamp(y, -hy, hy), _brackets_hinges_bezels_clamp(z, -hz, hz))+ inside = abs(x) <= hx and abs(y) <= hy and abs(z) <= hz+ if inside:+ face_dist = (hx - abs(x), hy - abs(y), hz - abs(z))+ axis = min(range(3), key=lambda idx: face_dist[idx])+ sign = 1.0 if point[axis] >= 0.0 else -1.0+ nearest = [x, y, z]+ nearest[axis] = (hx, hy, hz)[axis] * sign+ normal = [0.0, 0.0, 0.0]+ normal[axis] = sign+ return (nearest[0], nearest[1], nearest[2]), (normal[0], normal[1], normal[2])+ dx = x - clamped[0]+ dy = y - clamped[1]+ dz = z - clamped[2]+ axis = max(range(3), key=lambda idx: abs((dx, dy, dz)[idx]))+ delta = (dx, dy, dz)[axis]+ sign = 1.0 if delta >= 0.0 else -1.0+ normal = [0.0, 0.0, 0.0]+ normal[axis] = sign+ return clamped, (normal[0], normal[1], normal[2])+++def _closest_point_on_cylinder(point, radius, length):+ x, y, z = point+ half = float(length) * 0.5+ radial = math.hypot(x, y)+ if radial <= _EPS:+ rx, ry = 1.0, 0.0+ else:+ rx, ry = x / radial, y / radial+ side = (rx * radius, ry * radius, _brackets_hinges_bezels_clamp(z, -half, half))+ side_d2 = (x - side[0]) ** 2 + (y - side[1]) ** 2 + (z - side[2]) ** 2++ def cap_candidate(sign):+ cap_z = sign * half+ if radial <= radius:+ cap_xy = (x, y)+ else:+ cap_xy = (rx * radius, ry * radius)+ cap = (cap_xy[0], cap_xy[1], cap_z)+ d2 = (x - cap[0]) ** 2 + (y - cap[1]) ** 2 + (z - cap[2]) ** 2+ return cap, d2++ top, top_d2 = cap_candidate(+1.0)+ bottom, bottom_d2 = cap_candidate(-1.0)+ if side_d2 <= top_d2 and side_d2 <= bottom_d2:+ return side, (rx, ry, 0.0)+ if top_d2 <= bottom_d2:+ return top, (0.0, 0.0, 1.0)+ return bottom, (0.0, 0.0, -1.0)+++def _query_surface_on_element(geometry, tf, *, point_hint):+ inv_tf = _mat4_inverse_rigid(tf)+ point_local = _mat4_vec3(inv_tf, point_hint)++ if isinstance(geometry, Sphere):+ radius = float(geometry.radius)+ direction = point_local+ if _norm(direction) <= _EPS:+ direction = (1.0, 0.0, 0.0)+ normal_local = _normalize_vec(direction, name="sphere query direction")+ surface_local = (normal_local[0] * radius, normal_local[1] * radius, normal_local[2] * radius)+ elif isinstance(geometry, Box):+ surface_local, normal_local = _closest_point_on_box(point_local, geometry.size)+ elif isinstance(geometry, Cylinder):+ surface_local, normal_local = _closest_point_on_cylinder(+ point_local, radius=float(geometry.radius), length=float(geometry.length)+ )+ else:+ # generic B-rep: approximate by its local AABB treated as a box+ (mn, mx) = _geometry_local_aabb(geometry)+ ctr = ((mn[0] + mx[0]) * 0.5, (mn[1] + mx[1]) * 0.5, (mn[2] + mx[2]) * 0.5)+ size = (mx[0] - mn[0], mx[1] - mn[1], mx[2] - mn[2])+ rel = (point_local[0] - ctr[0], point_local[1] - ctr[1], point_local[2] - ctr[2])+ sp, normal_local = _closest_point_on_box(rel, size)+ surface_local = (sp[0] + ctr[0], sp[1] + ctr[1], sp[2] + ctr[2])++ distance = _norm(+ (point_local[0] - surface_local[0],+ point_local[1] - surface_local[1],+ point_local[2] - surface_local[2])+ )+ point_world = _mat4_vec3(tf, surface_local)+ normal_world = _normalize_vec(_mat4_transform_direction(tf, normal_local), name="surface normal")+ return _SurfaceHit(point=point_world, normal=normal_world, distance=distance)+++def _query_subject_surface_at_point(subject, *, point_hint, asset_root=None, prefer_collisions):+ best = None+ for geometry, tf in _iter_subject_items(subject, prefer_collisions=prefer_collisions):+ hit = _query_surface_on_element(geometry, tf, point_hint=point_hint)+ if best is None or hit.distance < best.distance:+ best = hit+ if best is None:+ raise ValidationError("target has no geometry to query")+ return best+++def _brackets_hinges_bezels_build_surface_tangents(normal, up_hint):+ up = _normalize_vec(up_hint, name="up_hint")+ projected = (+ up[0] - normal[0] * _dot(up, normal),+ up[1] - normal[1] * _dot(up, normal),+ up[2] - normal[2] * _dot(up, normal),+ )+ if _norm(projected) <= _EPS:+ fallback = (1.0, 0.0, 0.0)+ if abs(_dot(fallback, normal)) >= 0.95:+ fallback = (0.0, 1.0, 0.0)+ projected = _cross(fallback, normal)+ tangent_u = _normalize_vec(projected, name="surface tangent")+ tangent_v = _normalize_vec(_cross(normal, tangent_u), name="surface bitangent")+ return tangent_u, tangent_v+++def surface_frame(+ target,+ *,+ point_hint=None,+ direction=None,+ asset_root=None,+ prefer_collisions=False,+ up_hint=(0.0, 0.0, 1.0),+):+ if (point_hint is None) == (direction is None):+ raise ValidationError("Exactly one of point_hint or direction must be provided")+ query_point = point_hint+ if direction is not None:+ dir_world = _normalize_vec(direction, name="direction")+ aabb = _subject_world_aabb(target, prefer_collisions=prefer_collisions)+ center = _aabb_center(aabb)+ size = _aabb_size(aabb)+ radius = max(_norm(size), 1e-3)+ query_point = (+ center[0] + dir_world[0] * radius * 4.0,+ center[1] + dir_world[1] * radius * 4.0,+ center[2] + dir_world[2] * radius * 4.0,+ )+ best = _query_subject_surface_at_point(+ target, point_hint=query_point, prefer_collisions=prefer_collisions+ )+ tangent_u, tangent_v = _brackets_hinges_bezels_build_surface_tangents(best.normal, up_hint)+ return SurfaceFrame(point=best.point, normal=best.normal, tangent_u=tangent_u, tangent_v=tangent_v)+++# ===========================================================================+# @section placement-mount (place_on_surface / place_on_face / *_uv / proud)+# ===========================================================================+def _axis_vector(axis):+ table = {+ "+x": (1.0, 0.0, 0.0), "-x": (-1.0, 0.0, 0.0),+ "+y": (0.0, 1.0, 0.0), "-y": (0.0, -1.0, 0.0),+ "+z": (0.0, 0.0, 1.0), "-z": (0.0, 0.0, -1.0),+ }+ axis_key = axis.strip().lower()+ if axis_key not in table:+ raise ValidationError(f"Invalid axis {axis!r}; expected one of: {', '.join(sorted(table))}")+ return table[axis_key]+++def _child_local_basis_for_axis(axis):+ basis = {+ "+z": ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0)),+ "-z": ((1.0, 0.0, 0.0), (0.0, -1.0, 0.0), (0.0, 0.0, -1.0)),+ "+x": ((0.0, 1.0, 0.0), (0.0, 0.0, 1.0), (1.0, 0.0, 0.0)),+ "-x": ((0.0, 1.0, 0.0), (0.0, 0.0, -1.0), (-1.0, 0.0, 0.0)),+ "+y": ((0.0, 0.0, 1.0), (1.0, 0.0, 0.0), (0.0, 1.0, 0.0)),+ "-y": ((0.0, 0.0, -1.0), (1.0, 0.0, 0.0), (0.0, -1.0, 0.0)),+ }+ axis_key = axis.strip().lower()+ if axis_key not in basis:+ raise ValidationError(f"Invalid axis {axis!r}; expected one of: {', '.join(sorted(basis))}")+ return basis[axis_key]+++def _rotation_for_surface_frame(frame, *, child_axis, spin):+ local_basis = _child_local_basis_for_axis(child_axis)+ world_basis = _mat3_from_columns(frame.tangent_u, frame.tangent_v, frame.normal)+ local_basis_mat = _mat3_from_columns(local_basis[0], local_basis[1], local_basis[2])+ base = _mat3_mul(world_basis, _mat3_transpose(local_basis_mat))+ if abs(float(spin)) <= _EPS:+ return base+ spin_world = _axis_angle_matrix(frame.normal, float(spin))+ return _mat3_mul(spin_world, base)+++def _support_min_projection_on_element(geometry, tf, *, axis):+ center = (float(tf[0][3]), float(tf[1][3]), float(tf[2][3]))+ rot = _mat3_from_mat4(tf)+ axis_local = _mat3_vec3(_mat3_transpose(rot), axis)++ if isinstance(geometry, Sphere):+ return _dot(center, axis) - float(geometry.radius)+ if isinstance(geometry, Box):+ hx, hy, hz = (float(geometry.size[0]) * 0.5, float(geometry.size[1]) * 0.5,+ float(geometry.size[2]) * 0.5)+ half = hx * abs(axis_local[0]) + hy * abs(axis_local[1]) + hz * abs(axis_local[2])+ return _dot(center, axis) - half+ if isinstance(geometry, Cylinder):+ radial = math.hypot(axis_local[0], axis_local[1])+ half = float(geometry.radius) * radial + (float(geometry.length) * 0.5) * abs(axis_local[2])+ return _dot(center, axis) - half+ # generic B-rep: support of its local AABB box (centered at AABB center)+ (mn, mx) = _geometry_local_aabb(geometry)+ c_local = ((mn[0] + mx[0]) * 0.5, (mn[1] + mx[1]) * 0.5, (mn[2] + mx[2]) * 0.5)+ hx, hy, hz = (mx[0] - mn[0]) * 0.5, (mx[1] - mn[1]) * 0.5, (mx[2] - mn[2]) * 0.5+ world_center = _mat4_vec3(tf, c_local)+ half = hx * abs(axis_local[0]) + hy * abs(axis_local[1]) + hz * abs(axis_local[2])+ return _dot(world_center, axis) - half+++def _subject_min_projection(subject, *, axis, asset_root=None, prefer_collisions):+ projections = [+ _support_min_projection_on_element(geometry, tf, axis=axis)+ for geometry, tf in _iter_subject_items(subject, prefer_collisions=prefer_collisions)+ ]+ if not projections:+ raise ValidationError("child has no geometry to compute support distance")+ return min(projections)+++def place_on_surface(+ child,+ target,+ *,+ point_hint=None,+ direction=None,+ child_axis="+z",+ clearance=0.0,+ spin=0.0,+ asset_root=None,+ prefer_collisions=False,+ child_prefer_collisions=False,+ up_hint=(0.0, 0.0, 1.0),+):+ """Rigidly place a child on a target surface using a tangent-frame mount."""+ frame = surface_frame(+ target, point_hint=point_hint, direction=direction,+ prefer_collisions=prefer_collisions, up_hint=up_hint,+ )+ child_axis_vec = _axis_vector(child_axis)+ mount_proj = _subject_min_projection(+ child, axis=child_axis_vec, prefer_collisions=child_prefer_collisions+ )+ offset = float(clearance) - float(mount_proj)+ xyz = (+ frame.point[0] + frame.normal[0] * offset,+ frame.point[1] + frame.normal[1] * offset,+ frame.point[2] + frame.normal[2] * offset,+ )+ rot = _rotation_for_surface_frame(frame, child_axis=child_axis, spin=float(spin))+ return Origin(xyz=xyz, rpy=_mat3_to_rpy(rot))+++_FACE_TABLE = {+ "+x": (0, +1.0, (0.0, math.pi / 2.0, 0.0)),+ "-x": (0, -1.0, (0.0, -math.pi / 2.0, 0.0)),+ "+y": (1, +1.0, (-math.pi / 2.0, 0.0, 0.0)),+ "-y": (1, -1.0, (math.pi / 2.0, 0.0, 0.0)),+ "+z": (2, +1.0, (0.0, 0.0, 0.0)),+ "-z": (2, -1.0, (math.pi, 0.0, 0.0)),+}++_FACE_TANGENT_AXES = {+ "+x": (1, 2), "-x": (1, 2),+ "+y": (0, 2), "-y": (0, 2),+ "+z": (0, 1), "-z": (0, 1),+}+++def _as_pair(values, *, name):+ try:+ if len(values) != 2:+ raise ValidationError(f"{name} must have 2 elements")+ return (float(values[0]), float(values[1]))+ except TypeError as exc:+ raise ValidationError(f"{name} must have 2 elements") from exc+++def place_on_face(+ parent_link,+ face,+ *,+ face_pos=(0.0, 0.0),+ proud=0.0,+ asset_root=None,+ prefer_collisions=True,+):+ """Compute an Origin that places a child on a face of the parent's AABB."""+ face_key = face.strip().lower()+ if face_key not in _FACE_TABLE:+ raise ValidationError(+ f"Invalid face {face!r}; expected one of: {', '.join(sorted(_FACE_TABLE))}"+ )+ axis_idx, sign, rpy = _FACE_TABLE[face_key]+ tang_a, tang_b = _FACE_TANGENT_AXES[face_key]+ face_pos_pair = _as_pair(face_pos, name="face_pos")++ aabbs = link_local_aabbs(parent_link, prefer_collisions=prefer_collisions)+ if not aabbs:+ raise ValidationError(f"Parent link {parent_link.name!r} has no geometry to compute AABB")+ (mn, mx) = _aabb_union(aabbs)++ surface = float(mx[axis_idx]) if sign > 0 else float(mn[axis_idx])+ normal_coord = surface + sign * float(proud)++ xyz = [0.0, 0.0, 0.0]+ xyz[axis_idx] = normal_coord+ xyz[tang_a] = face_pos_pair[0]+ xyz[tang_b] = face_pos_pair[1]+ return Origin(xyz=(xyz[0], xyz[1], xyz[2]), rpy=rpy)+++def place_on_face_uv(+ parent_link,+ face,+ *,+ uv=(0.5, 0.5),+ uv_margin=0.0,+ proud=0.0,+ asset_root=None,+ prefer_collisions=True,+):+ """Like place_on_face but address the face using normalized (u, v) coordinates."""+ face_key = face.strip().lower()+ if face_key not in _FACE_TABLE:+ raise ValidationError(+ f"Invalid face {face!r}; expected one of: {', '.join(sorted(_FACE_TABLE))}"+ )+ axis_idx, sign, rpy = _FACE_TABLE[face_key]+ tang_a, tang_b = _FACE_TANGENT_AXES[face_key]++ if isinstance(uv_margin, tuple):+ mu, mv = _as_pair(uv_margin, name="uv_margin")+ else:+ mu = mv = float(uv_margin)+ mu = max(0.0, min(0.49, mu))+ mv = max(0.0, min(0.49, mv))++ u_raw, v_raw = _as_pair(uv, name="uv")+ u = max(0.0, min(1.0, u_raw))+ v = max(0.0, min(1.0, v_raw))+ u = mu + (1.0 - 2.0 * mu) * u+ v = mv + (1.0 - 2.0 * mv) * v++ aabbs = link_local_aabbs(parent_link, prefer_collisions=prefer_collisions)+ if not aabbs:+ raise ValidationError(f"Parent link {parent_link.name!r} has no geometry to compute AABB")+ (mn, mx) = _aabb_union(aabbs)++ surface = float(mx[axis_idx]) if sign > 0 else float(mn[axis_idx])+ normal_coord = surface + sign * float(proud)++ ta0, ta1 = float(mn[tang_a]), float(mx[tang_a])+ tb0, tb1 = float(mn[tang_b]), float(mx[tang_b])+ face_pos = (ta0 + (ta1 - ta0) * u, tb0 + (tb1 - tb0) * v)++ xyz = [0.0, 0.0, 0.0]+ xyz[axis_idx] = normal_coord+ xyz[tang_a] = float(face_pos[0])+ xyz[tang_b] = float(face_pos[1])+ return Origin(xyz=(xyz[0], xyz[1], xyz[2]), rpy=rpy)+++def proud_for_flush_mount(child_link, *, axis="z", clearance=0.0, asset_root=None,+ prefer_collisions=True):+ """Return half the child's thickness along ``axis`` (+clearance) for a flush mount."""+ axis_key = axis.strip().lower()+ axis_idx = {"x": 0, "y": 1, "z": 2}.get(axis_key)+ if axis_idx is None:+ raise ValidationError(f"Invalid axis {axis!r}; expected 'x', 'y', or 'z'.")+ aabb = link_local_aabb(child_link, prefer_collisions=prefer_collisions)+ if aabb is None:+ raise ValidationError(f"Child link {child_link.name!r} has no geometry to compute AABB")+ (mn, mx) = aabb+ thickness = float(mx[axis_idx]) - float(mn[axis_idx])+ return 0.5 * thickness + float(clearance)+++# ===========================================================================+# @section placement-aabb-overlap (cadquery_local_aabb / AllowedOverlap)+# ===========================================================================+def cadquery_local_aabb(model, *, tolerance=0.001, angular_tolerance=0.1, unit_scale=1.0):+ """Axis-aligned bounding box of a cadquery model in its local frame."""+ shape = _to_shape(model)+ try:+ verts, _tris = shape.tessellate(float(tolerance), float(angular_tolerance))+ except TypeError:+ verts, _tris = shape.tessellate(float(tolerance))+ except Exception:+ verts = None+ if verts:+ xs = [v.x for v in verts]+ ys = [v.y for v in verts]+ zs = [v.z for v in verts]+ mn = (min(xs), min(ys), min(zs))+ mx = (max(xs), max(ys), max(zs))+ else:+ bb = shape.BoundingBox()+ mn = (bb.xmin, bb.ymin, bb.zmin)+ mx = (bb.xmax, bb.ymax, bb.zmax)+ s = float(unit_scale)+ return ((mn[0] * s, mn[1] * s, mn[2] * s), (mx[0] * s, mx[1] * s, mx[2] * s))+++@dataclass(frozen=True)+class AllowedOverlap:+ link_a: str+ link_b: str+ reason: str+ elem_a: Optional[str] = None+ elem_b: Optional[str] = None+++# ===========================================================================+# @section mesh-openings-and-wraps (cut_opening_on_face / wrap_*_onto_surface)+#+# These operate on triangle data (matching the sdk's mesh-level behaviour).+# A shim geometry that only carries a B-rep is tessellated first.+# ===========================================================================+def _cross_z(a, b, c):+ return (b[0] - a[0]) * (c[1] - a[1]) - (b[1] - a[1]) * (c[0] - a[0])+++def _point_in_triangle(p, a, b, c):+ eps = 1e-9+ c1 = _cross_z(a, b, p)+ c2 = _cross_z(b, c, p)+ c3 = _cross_z(c, a, p)+ return (c1 > eps and c2 > eps and c3 > eps) or (c1 < -eps and c2 < -eps and c3 < -eps)+++def _triangulate_polygon(points):+ if len(points) < 3:+ return []+ indices = list(range(len(points)))+ triangles = []+ guard = 0+ max_guard = len(points) * len(points)+ while len(indices) > 3:+ ear_found = False+ for i in range(len(indices)):+ i_prev = indices[i - 1]+ i_curr = indices[i]+ i_next = indices[(i + 1) % len(indices)]+ if _cross_z(points[i_prev], points[i_curr], points[i_next]) <= 0:+ continue+ ear = True+ for j in indices:+ if j in (i_prev, i_curr, i_next):+ continue+ if _point_in_triangle(points[j], points[i_prev], points[i_curr], points[i_next]):+ ear = False+ break+ if ear:+ triangles.append((i_prev, i_curr, i_next))+ del indices[i]+ ear_found = True+ break+ guard += 1+ if not ear_found or guard > max_guard:+ raise ValidationError("Failed to triangulate profile; ensure it is simple")+ triangles.append((indices[0], indices[1], indices[2]))+ return triangles+++def _polygon_centroid(points):+ sx = sum(p[0] for p in points)+ sy = sum(p[1] for p in points)+ n = max(1, len(points))+ return (sx / float(n), sy / float(n))+++def _geom_vertices_faces(geom):+ """Return (vertices, faces) for a shim geometry, tessellating a B-rep if needed."""+ if getattr(geom, "vertices", None) and getattr(geom, "faces", None):+ return list(geom.vertices), list(geom.faces)+ shape = _to_shape(geom)+ try:+ verts, tris = shape.tessellate(0.001, 0.1)+ except TypeError:+ verts, tris = shape.tessellate(0.001)+ vertices = [(float(v.x), float(v.y), float(v.z)) for v in verts]+ faces = [(int(t[0]), int(t[1]), int(t[2])) for t in tris]+ return vertices, faces+++def cut_opening_on_face(+ shell_geometry,+ *,+ face,+ opening_profile,+ depth,+ offset=(0.0, 0.0),+ taper=0.0,+):+ """Add an opening "throat" (internal side walls) on a mesh AABB face.++ Faithful to the sdk: this does not boolean-subtract; it lofts the side+ walls of a recessed opening and merges them into ``shell_geometry``.+ """+ if not isinstance(shell_geometry, MeshGeometry):+ raise TypeError("shell_geometry must be MeshGeometry")++ vertices, faces = _geom_vertices_faces(shell_geometry)+ if not vertices:+ raise ValueError("shell_geometry has no vertices")++ d = float(depth)+ if d <= 0:+ raise ValueError("depth must be positive")++ f = (face or "").strip().lower()+ if len(f) != 2 or f[0] not in "+-" or f[1] not in "xyz":+ raise ValueError("face must be one of: '+x', '-x', '+y', '-y', '+z', '-z'")++ points = _ensure_ccw(_profile_points_2d(opening_profile))+ ox = float(offset[0])+ oy = float(offset[1])+ taper = float(taper)+ if abs(taper) >= 0.95:+ raise ValueError("abs(taper) must be < 0.95")++ center = _polygon_centroid(points)+ scale = 1.0 - taper+ inner_points = [+ (center[0] + (x - center[0]) * scale, center[1] + (y - center[1]) * scale)+ for (x, y) in points+ ]++ xs = [v[0] for v in vertices]+ ys = [v[1] for v in vertices]+ zs = [v[2] for v in vertices]+ bounds = {"x": (min(xs), max(xs)), "y": (min(ys), max(ys)), "z": (min(zs), max(zs))}++ axis = f[1]+ sign = 1.0 if f[0] == "+" else -1.0+ face_plane = bounds[axis][1] if sign > 0 else bounds[axis][0]++ def map_uv(u, v, depth_local):+ inward = -sign * depth_local+ if axis == "x":+ return (face_plane + inward, u + ox, v + oy)+ if axis == "y":+ return (u + ox, face_plane + inward, v + oy)+ return (u + ox, v + oy, face_plane + inward)++ p0 = [map_uv(u, v, 0.0) for (u, v) in points]+ p1 = [map_uv(u, v, d) for (u, v) in inner_points]++ # open closed loft (no caps): connect ring p0 -> ring p1 with side quads+ base = len(vertices)+ n = len(p0)+ new_vertices = list(vertices) + p0 + p1+ new_faces = list(faces)+ for i in range(n):+ a = base + i+ b = base + (i + 1) % n+ c = base + n + (i + 1) % n+ dd = base + n + i+ new_faces.append((a, b, c))+ new_faces.append((a, c, dd))++ shell_geometry._shape = None+ shell_geometry.vertices = new_vertices+ shell_geometry.faces = new_faces+ return shell_geometry+++# ---------------------------------------------------------------------------+# surface wrapping helpers (port of sdk placement wrap kernels)+# ---------------------------------------------------------------------------+def _subdivide_mesh_geometry_to_size(geometry, *, max_edge):+ if max_edge is None:+ return geometry+ max_edge_value = float(max_edge)+ if max_edge_value <= 0.0:+ raise ValidationError("max_edge must be positive")+ if not geometry.vertices or not geometry.faces:+ return geometry+ import trimesh+ vertices, faces = trimesh.remesh.subdivide_to_size(+ geometry.vertices, geometry.faces, max_edge=max_edge_value+ )+ return MeshGeometry(+ vertices=[(float(v[0]), float(v[1]), float(v[2])) for v in vertices],+ faces=[(int(f[0]), int(f[1]), int(f[2])) for f in faces],+ )+++def _normalize_profile_points(profile):+ return _ensure_ccw(_profile_points_2d(profile))+++def _planar_profile_cap(profile, *, max_edge):+ points = _normalize_profile_points(profile)+ faces = _triangulate_polygon(points)+ cap = MeshGeometry(+ vertices=[(float(x), float(y), 0.0) for x, y in points],+ faces=[(int(a), int(b), int(c)) for a, b, c in faces],+ )+ if not cap.faces:+ raise ValidationError("profile triangulation produced no faces")+ return _subdivide_mesh_geometry_to_size(cap, max_edge=max_edge)+++def _boundary_loop_for_planar_mesh(geometry, *, boundary_profile):+ edge_counts = {}+ for a, b, c in geometry.faces:+ for u, v in ((a, b), (b, c), (c, a)):+ key = (u, v) if u < v else (v, u)+ edge_counts[key] = edge_counts.get(key, 0) + 1+ boundary_vertices = sorted(+ {index for edge, count in edge_counts.items() if count == 1 for index in edge}+ )+ if not boundary_vertices:+ raise ValidationError("profile mesh has no boundary loop")++ cumulative = [0.0]+ for i, a in enumerate(boundary_profile):+ b = boundary_profile[(i + 1) % len(boundary_profile)]+ cumulative.append(+ cumulative[-1] + math.hypot(float(b[0]) - float(a[0]), float(b[1]) - float(a[1]))+ )+ perimeter = cumulative[-1]+ if perimeter <= _EPS:+ raise ValidationError("profile perimeter must be non-zero")++ def _dist_to_seg(point, a, b):+ seg_x = float(b[0]) - float(a[0])+ seg_y = float(b[1]) - float(a[1])+ seg_len2 = seg_x * seg_x + seg_y * seg_y+ if seg_len2 <= _EPS:+ return math.hypot(point[0] - a[0], point[1] - a[1]), 0.0+ rel_x = float(point[0]) - float(a[0])+ rel_y = float(point[1]) - float(a[1])+ t = _brackets_hinges_bezels_clamp((rel_x * seg_x + rel_y * seg_y) / seg_len2, 0.0, 1.0)+ cx = float(a[0]) + seg_x * t+ cy = float(a[1]) + seg_y * t+ return math.hypot(float(point[0]) - cx, float(point[1]) - cy), t++ ordered = []+ for index in boundary_vertices:+ point = (float(geometry.vertices[index][0]), float(geometry.vertices[index][1]))+ best_distance = float("inf")+ best_arclength = 0.0+ for segment_index, a in enumerate(boundary_profile):+ b = boundary_profile[(segment_index + 1) % len(boundary_profile)]+ distance, t = _dist_to_seg(point, a, b)+ if distance + 1e-9 < best_distance:+ best_distance = distance+ best_arclength = (+ cumulative[segment_index]+ + (cumulative[segment_index + 1] - cumulative[segment_index]) * t+ )+ if best_distance > 1e-5:+ raise ValidationError("profile mesh boundary deviated from source profile")+ if best_arclength >= perimeter - 1e-8:+ best_arclength = 0.0+ ordered.append((best_arclength, index))++ ordered.sort(key=lambda item: item[0])+ loop = [index for _, index in ordered]+ area = 0.0+ for i, index in enumerate(loop):+ x1, y1, _ = geometry.vertices[index]+ x2, y2, _ = geometry.vertices[loop[(i + 1) % len(loop)]]+ area += x1 * y2 - x2 * y1+ if area < 0.0:+ loop.reverse()+ return loop+++def _solid_from_planar_profile_cap(cap, *, boundary_profile, thickness):+ thickness_value = float(thickness)+ if thickness_value <= 0.0:+ raise ValidationError("thickness must be positive")+ loop = _boundary_loop_for_planar_mesh(cap, boundary_profile=boundary_profile)+ vertex_count = len(cap.vertices)+ solid = MeshGeometry(+ vertices=[+ *[(float(x), float(y), 0.0) for x, y, _ in cap.vertices],+ *[(float(x), float(y), -thickness_value) for x, y, _ in cap.vertices],+ ],+ faces=[],+ )+ for a, b, c in cap.faces:+ solid.add_face(a, b, c)+ offset = vertex_count+ for a, b, c in cap.faces:+ solid.add_face(offset + c, offset + b, offset + a)+ for i, a in enumerate(loop):+ b = loop[(i + 1) % len(loop)]+ solid.add_face(a, b, offset + b)+ solid.add_face(a, offset + b, offset + a)+ return solid+++def _fallback_profile_solid(profile, *, hole_profiles, thickness):+ """Direct prism builder (visible cap on z=0, thickness toward -z).++ Holes are not supported in this fallback path; they are ignored.+ """+ points = _normalize_profile_points(profile)+ faces = _triangulate_polygon(points)+ cap = MeshGeometry(vertices=[(float(x), float(y), 0.0) for x, y in points],+ faces=[(int(a), int(b), int(c)) for a, b, c in faces])+ return _solid_from_planar_profile_cap(cap, boundary_profile=points, thickness=thickness)+++def _mesh_geometry_from_subject(mesh):+ if isinstance(mesh, MeshGeometry):+ if getattr(mesh, "vertices", None) and getattr(mesh, "faces", None):+ return mesh.copy()+ v, f = _geom_vertices_faces(mesh)+ return MeshGeometry(vertices=v, faces=f)+ if isinstance(mesh, _CQMesh):+ v, f = _geom_vertices_faces(mesh)+ return MeshGeometry(vertices=v, faces=f)+ raise ValidationError(f"mesh must be a MeshGeometry, got {type(mesh).__name__}")+++def _subject_single_world_sphere(subject, *, prefer_collisions):+ items = _iter_subject_items(subject, prefer_collisions=prefer_collisions)+ if len(items) != 1:+ return None+ geometry, tf = items[0]+ if not isinstance(geometry, Sphere):+ return None+ return _mat4_vec3(tf, (0.0, 0.0, 0.0)), float(geometry.radius)+++def _subject_single_world_cylinder(subject, *, prefer_collisions):+ items = _iter_subject_items(subject, prefer_collisions=prefer_collisions)+ if len(items) != 1:+ return None+ geometry, tf = items[0]+ if not isinstance(geometry, Cylinder):+ return None+ center = _mat4_vec3(tf, (0.0, 0.0, 0.0))+ axis_world = _normalize_vec(_mat4_transform_direction(tf, (0.0, 0.0, 1.0)), name="cylinder axis")+ return center, axis_world, float(geometry.radius), float(geometry.length)+++def _wrap_mesh_onto_sphere(geometry, *, frame, sphere_center, sphere_radius, child_axis,+ visible_relief, spin):+ rot = _rotation_for_surface_frame(frame, child_axis=child_axis, spin=float(spin))+ axis_local = _axis_vector(child_axis)+ outer_support = max(_dot(vertex, axis_local) for vertex in geometry.vertices)+ wrapped = []+ for vertex in geometry.vertices:+ local_proj = _dot(vertex, axis_local)+ depth_behind = outer_support - local_proj+ in_plane_local = (+ float(vertex[0]) - axis_local[0] * local_proj,+ float(vertex[1]) - axis_local[1] * local_proj,+ float(vertex[2]) - axis_local[2] * local_proj,+ )+ tangent_offset = _mat3_vec3(rot, in_plane_local)+ tangent_distance = _norm(tangent_offset)+ if tangent_distance <= _EPS:+ surface_normal = frame.normal+ else:+ tangent_dir = (tangent_offset[0] / tangent_distance,+ tangent_offset[1] / tangent_distance,+ tangent_offset[2] / tangent_distance)+ angle = tangent_distance / float(sphere_radius)+ surface_normal = _normalize_vec(+ (math.cos(angle) * frame.normal[0] + math.sin(angle) * tangent_dir[0],+ math.cos(angle) * frame.normal[1] + math.sin(angle) * tangent_dir[1],+ math.cos(angle) * frame.normal[2] + math.sin(angle) * tangent_dir[2]),+ name="sphere wrap normal",+ )+ surface_point = (+ sphere_center[0] + surface_normal[0] * float(sphere_radius),+ sphere_center[1] + surface_normal[1] * float(sphere_radius),+ sphere_center[2] + surface_normal[2] * float(sphere_radius),+ )+ wrapped.append((+ surface_point[0] + surface_normal[0] * (float(visible_relief) - depth_behind),+ surface_point[1] + surface_normal[1] * (float(visible_relief) - depth_behind),+ surface_point[2] + surface_normal[2] * (float(visible_relief) - depth_behind),+ ))+ return MeshGeometry(vertices=wrapped, faces=list(geometry.faces))+++def _wrap_mesh_onto_cylinder(geometry, *, frame, cylinder_center, cylinder_axis, cylinder_radius,+ cylinder_length, child_axis, visible_relief, spin):+ if abs(_dot(frame.normal, cylinder_axis)) >= 0.2:+ return None+ rel_anchor = (frame.point[0] - cylinder_center[0],+ frame.point[1] - cylinder_center[1],+ frame.point[2] - cylinder_center[2])+ anchor_axial = _dot(rel_anchor, cylinder_axis)+ anchor_radial = (rel_anchor[0] - cylinder_axis[0] * anchor_axial,+ rel_anchor[1] - cylinder_axis[1] * anchor_axial,+ rel_anchor[2] - cylinder_axis[2] * anchor_axial)+ if _norm(anchor_radial) <= _EPS:+ return None+ anchor_normal = _normalize_vec(anchor_radial, name="cylinder wrap normal")+ circum_axis = _normalize_vec(_cross(cylinder_axis, anchor_normal),+ name="cylinder circumferential tangent")+ half_length = float(cylinder_length) * 0.5+ rot = _rotation_for_surface_frame(frame, child_axis=child_axis, spin=float(spin))+ axis_local = _axis_vector(child_axis)+ outer_support = max(_dot(vertex, axis_local) for vertex in geometry.vertices)+ wrapped = []+ for vertex in geometry.vertices:+ local_proj = _dot(vertex, axis_local)+ depth_behind = outer_support - local_proj+ in_plane_local = (+ float(vertex[0]) - axis_local[0] * local_proj,+ float(vertex[1]) - axis_local[1] * local_proj,+ float(vertex[2]) - axis_local[2] * local_proj,+ )+ tangent_offset = _mat3_vec3(rot, in_plane_local)+ circum_distance = _dot(tangent_offset, circum_axis)+ axial_distance = _dot(tangent_offset, cylinder_axis)+ axial = anchor_axial + axial_distance+ if abs(axial) > half_length + 1e-9:+ return None+ angle = circum_distance / float(cylinder_radius)+ turn = _axis_angle_matrix(cylinder_axis, angle)+ surface_normal = _normalize_vec(_mat3_vec3(turn, anchor_normal),+ name="cylinder wrap rotated normal")+ surface_point = (+ cylinder_center[0] + cylinder_axis[0] * axial + surface_normal[0] * float(cylinder_radius),+ cylinder_center[1] + cylinder_axis[1] * axial + surface_normal[1] * float(cylinder_radius),+ cylinder_center[2] + cylinder_axis[2] * axial + surface_normal[2] * float(cylinder_radius),+ )+ wrapped.append((+ surface_point[0] + surface_normal[0] * (float(visible_relief) - depth_behind),+ surface_point[1] + surface_normal[1] * (float(visible_relief) - depth_behind),+ surface_point[2] + surface_normal[2] * (float(visible_relief) - depth_behind),+ ))+ return MeshGeometry(vertices=wrapped, faces=list(geometry.faces))+++def _wrap_mesh_onto_surface_nearest(geometry, *, frame, target, child_axis, visible_relief, spin,+ prefer_collisions):+ rot = _rotation_for_surface_frame(frame, child_axis=child_axis, spin=float(spin))+ axis_local = _axis_vector(child_axis)+ outer_support = max(_dot(vertex, axis_local) for vertex in geometry.vertices)+ wrapped = []+ for vertex in geometry.vertices:+ local_proj = _dot(vertex, axis_local)+ depth_behind = outer_support - local_proj+ in_plane_local = (+ float(vertex[0]) - axis_local[0] * local_proj,+ float(vertex[1]) - axis_local[1] * local_proj,+ float(vertex[2]) - axis_local[2] * local_proj,+ )+ plane_offset = _mat3_vec3(rot, in_plane_local)+ query_point = (frame.point[0] + plane_offset[0],+ frame.point[1] + plane_offset[1],+ frame.point[2] + plane_offset[2])+ hit = _query_subject_surface_at_point(target, point_hint=query_point,+ prefer_collisions=prefer_collisions)+ wrapped.append((+ hit.point[0] + hit.normal[0] * (float(visible_relief) - depth_behind),+ hit.point[1] + hit.normal[1] * (float(visible_relief) - depth_behind),+ hit.point[2] + hit.normal[2] * (float(visible_relief) - depth_behind),+ ))+ return MeshGeometry(vertices=wrapped, faces=list(geometry.faces))+++def _normalize_surface_wrap_mapping(mapping):+ mapping_key = str(mapping).strip().lower()+ if mapping_key not in {"auto", "intrinsic", "nearest"}:+ raise ValidationError("mapping must be one of: 'auto', 'intrinsic', 'nearest'")+ return mapping_key+++def _resolve_surface_max_edge(*, surface_max_edge, max_edge):+ if surface_max_edge is None:+ return max_edge+ if max_edge is None:+ return surface_max_edge+ if abs(float(surface_max_edge) - float(max_edge)) > 1e-12:+ raise ValidationError("Provide only one of surface_max_edge or max_edge")+ return surface_max_edge+++def wrap_mesh_onto_surface(+ mesh,+ target,+ *,+ point_hint=None,+ direction=None,+ child_axis="+z",+ visible_relief=0.0,+ mapping="auto",+ surface_max_edge=None,+ max_edge=None,+ spin=0.0,+ asset_root=None,+ prefer_collisions=False,+ up_hint=(0.0, 0.0, 1.0),+):+ """Conform a mesh onto a target surface and return a baked MeshGeometry."""+ mapping_mode = _normalize_surface_wrap_mapping(mapping)+ frame = surface_frame(target, point_hint=point_hint, direction=direction,+ prefer_collisions=prefer_collisions, up_hint=up_hint)+ resolved_max_edge = _resolve_surface_max_edge(surface_max_edge=surface_max_edge,+ max_edge=max_edge)+ geometry = _mesh_geometry_from_subject(mesh)+ geometry = _subdivide_mesh_geometry_to_size(geometry, max_edge=resolved_max_edge)+ if not geometry.vertices:+ raise ValidationError("mesh has no vertices")++ if mapping_mode != "nearest":+ sphere_info = _subject_single_world_sphere(target, prefer_collisions=prefer_collisions)+ if sphere_info is not None:+ sphere_center, sphere_radius = sphere_info+ return _wrap_mesh_onto_sphere(geometry, frame=frame, sphere_center=sphere_center,+ sphere_radius=sphere_radius, child_axis=child_axis,+ visible_relief=float(visible_relief), spin=float(spin))+ cylinder_info = _subject_single_world_cylinder(target, prefer_collisions=prefer_collisions)+ if cylinder_info is not None:+ cc, ca, cr, cl = cylinder_info+ intrinsic = _wrap_mesh_onto_cylinder(geometry, frame=frame, cylinder_center=cc,+ cylinder_axis=ca, cylinder_radius=cr,+ cylinder_length=cl, child_axis=child_axis,+ visible_relief=float(visible_relief),+ spin=float(spin))+ if intrinsic is not None:+ return intrinsic+ if mapping_mode == "intrinsic":+ raise ValidationError(+ "Intrinsic surface wrapping is only supported for spheres and cylinder sidewalls"+ )++ return _wrap_mesh_onto_surface_nearest(geometry, frame=frame, target=target,+ child_axis=child_axis,+ visible_relief=float(visible_relief), spin=float(spin),+ prefer_collisions=prefer_collisions)+++def wrap_profile_onto_surface(+ profile,+ target,+ *,+ thickness,+ hole_profiles=(),+ point_hint=None,+ direction=None,+ visible_relief=0.0,+ mapping="auto",+ surface_max_edge=None,+ max_edge=None,+ spin=0.0,+ asset_root=None,+ prefer_collisions=False,+ up_hint=(0.0, 0.0, 1.0),+):+ """Wrap a 2D profile with thickness directly onto a target surface."""+ thickness_value = float(thickness)+ if thickness_value <= 0.0:+ raise ValidationError("thickness must be positive")+ resolved_max_edge = _resolve_surface_max_edge(surface_max_edge=surface_max_edge,+ max_edge=max_edge)+ points = _normalize_profile_points(profile)+ holes = [_normalize_profile_points(h) for h in hole_profiles]++ if holes:+ fallback = _fallback_profile_solid(points, hole_profiles=holes, thickness=thickness_value)+ return wrap_mesh_onto_surface(fallback, target, point_hint=point_hint, direction=direction,+ child_axis="+z", visible_relief=visible_relief, mapping=mapping,+ surface_max_edge=resolved_max_edge, spin=spin,+ prefer_collisions=prefer_collisions, up_hint=up_hint)++ cap = _planar_profile_cap(points, max_edge=resolved_max_edge)+ try:+ solid = _solid_from_planar_profile_cap(cap, boundary_profile=points,+ thickness=thickness_value)+ return wrap_mesh_onto_surface(solid, target, point_hint=point_hint, direction=direction,+ child_axis="+z", visible_relief=visible_relief, mapping=mapping,+ spin=spin, prefer_collisions=prefer_collisions, up_hint=up_hint)+ except ValidationError as exc:+ if "profile mesh boundary" not in str(exc):+ raise++ fallback = _fallback_profile_solid(points, hole_profiles=[], thickness=thickness_value)+ return wrap_mesh_onto_surface(fallback, target, point_hint=point_hint, direction=direction,+ child_axis="+z", visible_relief=visible_relief, mapping=mapping,+ surface_max_edge=resolved_max_edge, spin=spin,+ prefer_collisions=prefer_collisions, up_hint=up_hint)diff --git a/toolkits/curation/articraft/shim_parts/fans.py b/toolkits/curation/articraft/shim_parts/fans.pynew file mode 100644index 00000000..3c616e84--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/fans.py@@ -0,0 +1,568 @@+from articraft_cq import cq, MeshGeometry, _to_shape, _EPS++# ===========================================================================+# @section geom-fans (FanRotorGeometry / BlowerWheelGeometry + specs)+#+# Clean-room CadQuery B-rep reimplementation of the Articraft sdk fan/rotor+# family. Constructor signatures mirror the real sdk exactly; the produced+# shapes are parametric solids (the sdk itself builds these with cadquery and+# then tessellates -- here we keep the B-rep).+# ===========================================================================+from dataclasses import dataclass+from math import cos, pi, sin, sqrt+from typing import Literal, Optional+++def _fans_shift_to_z0(solid):+ """Place the rear-most face on z=0 (sdk center=False behaviour)."""+ bb = solid.BoundingBox()+ return solid.translate(cq.Vector(0.0, 0.0, -bb.zmin))+++# ---- sub-feature specs (frozen dataclasses, verbatim sdk signatures) -------+@dataclass(frozen=True)+class FanRotorBlade:+ shape: Literal["straight", "scimitar", "broad", "narrow"] = "straight"+ tip_pitch_deg: Optional[float] = None+ camber: float = 0.0+ tip_clearance: float = 0.0+++@dataclass(frozen=True)+class FanRotorHub:+ style: Literal["flat", "domed", "capped", "spinner"] = "domed"+ rear_collar_height: Optional[float] = None+ rear_collar_radius: Optional[float] = None+ bore_diameter: Optional[float] = None+++@dataclass(frozen=True)+class FanRotorShroud:+ thickness: float+ depth: Optional[float] = None+ clearance: float = 0.0+ lip_depth: float = 0.0+++class FanRotorGeometry(MeshGeometry):+ """+ Build an axial fan rotor with a central hub and pitched blades.+ """++ def __init__(+ self,+ outer_radius: float,+ hub_radius: float,+ blade_count: int,+ *,+ thickness: float,+ blade_pitch_deg: float = 28.0,+ blade_sweep_deg: float = 20.0,+ blade_root_chord: Optional[float] = None,+ blade_tip_chord: Optional[float] = None,+ blade: Optional[FanRotorBlade] = None,+ hub: Optional[FanRotorHub] = None,+ shroud: Optional[FanRotorShroud] = None,+ center: bool = True,+ ):+ super().__init__()+ outer_radius = float(outer_radius)+ hub_radius = float(hub_radius)+ thickness = float(thickness)+ blade_pitch_deg = float(blade_pitch_deg)+ blade_sweep_deg = float(blade_sweep_deg)+ blade_count = int(blade_count)+ blade = blade or FanRotorBlade()+ hub = hub or FanRotorHub()++ if outer_radius <= 0.0 or hub_radius <= 0.0 or thickness <= 0.0:+ raise ValueError("outer_radius, hub_radius, and thickness must be positive")+ if blade_count < 2:+ raise ValueError("blade_count must be at least 2")+ if hub_radius >= outer_radius:+ raise ValueError("hub_radius must be less than outer_radius")+ if abs(blade_pitch_deg) >= 85.0:+ raise ValueError("abs(blade_pitch_deg) must be < 85")+ if abs(blade_sweep_deg) >= 85.0:+ raise ValueError("abs(blade_sweep_deg) must be < 85")++ radial_span = outer_radius - hub_radius+ root_chord = (+ float(blade_root_chord)+ if blade_root_chord is not None+ else max(radial_span * 0.32, thickness * 1.5)+ )+ tip_chord = (+ float(blade_tip_chord)+ if blade_tip_chord is not None+ else max(radial_span * 0.20, thickness * 1.2)+ )+ if root_chord <= 0.0 or tip_chord <= 0.0:+ raise ValueError("blade_root_chord and blade_tip_chord must be positive")+ if max(root_chord, tip_chord) >= outer_radius * 1.6:+ raise ValueError("blade chords are too large for the rotor envelope")++ blade_shape = str(blade.shape)+ if blade_shape not in {"straight", "scimitar", "broad", "narrow"}:+ raise ValueError("blade.shape must be one of straight, scimitar, broad, or narrow")+ hub_style = str(hub.style)+ if hub_style not in {"flat", "domed", "capped", "spinner"}:+ raise ValueError("hub.style must be one of flat, domed, capped, or spinner")++ tip_pitch_deg = (+ float(blade.tip_pitch_deg)+ if blade.tip_pitch_deg is not None+ else blade_pitch_deg * 0.56+ )+ if abs(tip_pitch_deg) >= 85.0:+ raise ValueError("abs(blade.tip_pitch_deg) must be < 85")+ camber = float(blade.camber)+ if abs(camber) > 0.50:+ raise ValueError("abs(blade.camber) must be <= 0.5")+ tip_clearance = float(blade.tip_clearance)+ if tip_clearance < 0.0:+ raise ValueError("blade.tip_clearance must be non-negative")++ shroud_thickness = 0.0+ shroud_depth = 0.0+ ring_inner_radius: Optional[float] = None+ ring_outer_radius: Optional[float] = None+ shroud_lip_depth = 0.0+ if shroud is not None:+ shroud_thickness = float(shroud.thickness)+ shroud_depth = (+ float(shroud.depth) if shroud.depth is not None else max(thickness * 0.80, 1.0e-4)+ )+ shroud_clearance = float(shroud.clearance)+ shroud_lip_depth = float(shroud.lip_depth)+ if shroud_thickness <= 0.0:+ raise ValueError("shroud.thickness must be positive")+ if shroud_depth <= 0.0:+ raise ValueError("shroud.depth must be positive")+ if shroud_clearance < 0.0:+ raise ValueError("shroud.clearance must be non-negative")+ if shroud_lip_depth < 0.0:+ raise ValueError("shroud.lip_depth must be non-negative")+ if tip_clearance > 1.0e-9:+ raise ValueError("blade.tip_clearance cannot be used with shroud")+ ring_outer_radius = outer_radius - shroud_clearance+ ring_inner_radius = ring_outer_radius - shroud_thickness+ if ring_outer_radius <= 0.0 or ring_inner_radius <= 0.0:+ raise ValueError("shroud dimensions exceed the rotor envelope")+ if ring_inner_radius <= hub_radius + max(thickness * 0.25, 1.0e-4):+ raise ValueError("shroud leaves no usable blade span")++ def lerp(a: float, b: float, t: float) -> float:+ return float(a) + (float(b) - float(a)) * float(t)++ def chord_scale(t: float) -> float:+ if blade_shape == "straight":+ return 1.00+ if blade_shape == "scimitar":+ return 1.08 - 0.16 * t + 0.05 * sin(pi * t)+ if blade_shape == "broad":+ return 1.05 + 0.10 * sin(pi * t) - 0.03 * t+ return 0.92 - 0.12 * t++ def sweep_factor(t: float) -> float:+ if blade_shape == "straight":+ return t+ if blade_shape == "scimitar":+ return min(1.25, 0.52 * t + 0.76 * t * t)+ if blade_shape == "broad":+ return min(1.08, 0.82 * t + 0.22 * sin(pi * t))+ return 0.82 * t++ def rotate_section_points(+ points: list,+ angle_rad: float,+ ) -> list:+ c = cos(angle_rad)+ s = sin(angle_rad)+ return [(y * c - z * s, y * s + z * c) for (y, z) in points]++ def blade_section_points(+ chord: float,+ section_thickness: float,+ pitch_deg: float,+ sweep_y: float,+ span_t: float,+ ) -> list:+ half_t = max(section_thickness * 0.5, chord * 0.011)+ camber_amplitude = camber * chord * 0.060+ skew_gain = 0.0+ if blade_shape == "scimitar":+ skew_gain = chord * (0.05 + 0.08 * span_t)+ elif blade_shape == "broad":+ skew_gain = chord * 0.03 * span_t+ elif blade_shape == "narrow":+ skew_gain = -chord * 0.04 * span_t++ upper: list = []+ lower: list = []+ for u in (0.00, 0.10, 0.24, 0.42, 0.62, 0.82, 1.00):+ chord_pos = (u - 0.5) * chord+ thickness_scale = max(0.12, sin(pi * u) ** 0.82)+ thickness_z = half_t * thickness_scale+ camber_z = camber_amplitude * sin(pi * u)+ skew_y = skew_gain * sin(pi * u) * (u - 0.20)+ upper.append((chord_pos + skew_y, camber_z + thickness_z))+ lower.append((chord_pos + skew_y, camber_z - thickness_z * 0.90))++ raw = upper + list(reversed(lower[1:-1]))+ rotated = rotate_section_points(raw, pitch_deg * pi / 180.0)+ return [(sweep_y + y, z) for (y, z) in rotated]++ hub_body_height = thickness * 0.36+ rear_collar_height = (+ float(hub.rear_collar_height)+ if hub.rear_collar_height is not None+ else thickness * 0.18+ )+ rear_collar_radius = (+ float(hub.rear_collar_radius)+ if hub.rear_collar_radius is not None+ else hub_radius * 0.78+ )+ if rear_collar_height < 0.0:+ raise ValueError("hub.rear_collar_height must be non-negative")+ if rear_collar_height > 1.0e-9 and rear_collar_radius <= 0.0:+ raise ValueError("hub.rear_collar_radius must be positive when rear_collar_height > 0")+ if rear_collar_radius >= hub_radius * 1.05:+ raise ValueError("hub.rear_collar_radius must fit within the hub radius")++ hub_body = cq.Workplane("XY").circle(hub_radius).extrude(hub_body_height * 0.5, both=True)+ shape = hub_body+ if rear_collar_height > 1.0e-9:+ rear_collar = (+ cq.Workplane("XY")+ .circle(rear_collar_radius)+ .extrude(rear_collar_height * 0.5, both=True)+ .translate((0.0, 0.0, -thickness * 0.24))+ )+ shape = shape.union(rear_collar)++ if hub_style == "capped":+ front_cap = (+ cq.Workplane("XY")+ .circle(hub_radius * 0.82)+ .extrude(max(thickness * 0.18, 1.0e-4))+ .translate((0.0, 0.0, hub_body_height * 0.5))+ )+ shape = shape.union(front_cap)+ elif hub_style in {"domed", "spinner"}:+ front_cap_height = thickness * (0.34 if hub_style == "domed" else 0.52)+ front_top_radius = hub_radius * (0.40 if hub_style == "domed" else 0.05)+ front_cap = (+ cq.Workplane("XY")+ .workplane(offset=hub_body_height * 0.5)+ .circle(hub_radius * 0.92)+ .workplane(offset=front_cap_height)+ .circle(max(front_top_radius, 1.0e-4))+ .loft(combine=True, ruled=False)+ )+ shape = shape.union(front_cap)++ if hub.bore_diameter is not None:+ bore_diameter = float(hub.bore_diameter)+ if bore_diameter <= 0.0:+ raise ValueError("hub.bore_diameter must be positive")+ bore_radius = bore_diameter * 0.5+ bore_limit = min(+ hub_radius, rear_collar_radius if rear_collar_height > 0.0 else hub_radius+ )+ if bore_radius >= bore_limit * 0.92:+ raise ValueError("hub.bore_diameter is too large for the hub")+ bore = (+ cq.Workplane("XY")+ .circle(bore_radius)+ .extrude(+ max(thickness * 1.8, hub_body_height + rear_collar_height + shroud_depth),+ both=True,+ )+ )+ shape = shape.cut(bore)++ root_radius = hub_radius * 0.82+ if shroud is not None:+ assert ring_inner_radius is not None+ tip_bridge = min(shroud_thickness * 0.18, max(thickness * 0.05, 5.0e-4))+ tip_radius = ring_inner_radius + tip_bridge+ else:+ tip_radius = outer_radius - tip_clearance+ if tip_radius <= root_radius + max(thickness * 0.22, 1.0e-4):+ raise ValueError("blade.tip_clearance leaves no usable blade span")++ blade_span = tip_radius - root_radius+ station_fracs = (0.00, 0.18, 0.42, 0.70, 1.00)+ stations = [root_radius + blade_span * frac for frac in station_fracs]+ section_chords = [+ lerp(root_chord, tip_chord, frac) * chord_scale(frac) for frac in station_fracs+ ]+ root_pitch_deg = blade_pitch_deg * 1.16+ section_pitches = [+ lerp(root_pitch_deg, tip_pitch_deg, frac**0.82) for frac in station_fracs+ ]+ root_section_t = max(thickness * 0.18, root_chord * 0.080)+ tip_section_t = max(thickness * 0.08, tip_chord * 0.048)+ thickness_shape_scale = {+ "straight": 1.00,+ "scimitar": 0.94,+ "broad": 1.08,+ "narrow": 0.88,+ }[blade_shape]+ section_thicknesses = [+ lerp(root_section_t, tip_section_t, frac**0.78) * thickness_shape_scale+ for frac in station_fracs+ ]+ sweep_amount = blade_span * sin(blade_sweep_deg * pi / 180.0) * 0.34+ section_sweeps = [sweep_amount * sweep_factor(frac) for frac in station_fracs]++ blade_wp = None+ previous_station = 0.0+ for station, chord, section_t, pitch_deg, sweep_y, span_t in zip(+ stations,+ section_chords,+ section_thicknesses,+ section_pitches,+ section_sweeps,+ station_fracs,+ ):+ section = blade_section_points(chord, section_t, pitch_deg, sweep_y, span_t)+ if blade_wp is None:+ blade_wp = cq.Workplane("YZ").workplane(offset=station).polyline(section).close()+ else:+ blade_wp = (+ blade_wp.workplane(offset=station - previous_station).polyline(section).close()+ )+ previous_station = station+ assert blade_wp is not None+ blade_solid = blade_wp.loft(combine=True, ruled=False)++ angle_step = 360.0 / float(blade_count)+ for blade_index in range(blade_count):+ shape = shape.union(+ blade_solid.rotate(+ (0.0, 0.0, 0.0),+ (0.0, 0.0, 1.0),+ angle_step * float(blade_index),+ )+ )++ if shroud is not None:+ assert ring_outer_radius is not None+ assert ring_inner_radius is not None+ tip_ring = (+ cq.Workplane("XY")+ .circle(ring_outer_radius)+ .circle(ring_inner_radius)+ .extrude(shroud_depth * 0.5, both=True)+ )+ shape = shape.union(tip_ring)+ if shroud_lip_depth > 1.0e-9:+ front_lip = (+ cq.Workplane("XY")+ .workplane(offset=shroud_depth * 0.5)+ .circle(ring_outer_radius)+ .circle(ring_inner_radius)+ .extrude(shroud_lip_depth)+ )+ shape = shape.union(front_lip)++ solid = _to_shape(shape)+ if not center:+ solid = _fans_shift_to_z0(solid)+ self._set(solid)+++class BlowerWheelGeometry(MeshGeometry):+ """+ Build a squirrel-cage blower wheel around a central drum.+ """++ def __init__(+ self,+ outer_radius: float,+ inner_radius: float,+ width: float,+ blade_count: int,+ *,+ blade_thickness: float,+ blade_sweep_deg: float = 35.0,+ backplate: bool = True,+ shroud: bool = True,+ center: bool = True,+ ):+ super().__init__()+ outer_radius = float(outer_radius)+ inner_radius = float(inner_radius)+ width = float(width)+ blade_count = int(blade_count)+ blade_thickness = float(blade_thickness)+ blade_sweep_deg = float(blade_sweep_deg)++ if outer_radius <= 0.0 or inner_radius <= 0.0 or width <= 0.0:+ raise ValueError("outer_radius, inner_radius, and width must be positive")+ if inner_radius >= outer_radius:+ raise ValueError("inner_radius must be less than outer_radius")+ if blade_count < 2:+ raise ValueError("blade_count must be at least 2")+ if blade_thickness <= 0.0:+ raise ValueError("blade_thickness must be positive")+ if abs(blade_sweep_deg) >= 85.0:+ raise ValueError("abs(blade_sweep_deg) must be < 85")++ radial_span = outer_radius - inner_radius+ if blade_thickness >= radial_span * 0.9:+ raise ValueError("blade_thickness is too large for the blower annulus")++ drum_wall = min(max(blade_thickness * 1.8, radial_span * 0.10), inner_radius * 0.55)+ if drum_wall <= 1e-6 or inner_radius - drum_wall <= 1e-6:+ raise ValueError("inner_radius is too small for the blower drum wall")+ drum = (+ cq.Workplane("XY")+ .circle(inner_radius)+ .circle(inner_radius - drum_wall)+ .extrude(width * 0.5, both=True)+ )++ shape = drum+ side_plate_thickness = min(max(blade_thickness * 1.1, width * 0.06), width * 0.16)+ side_plate_inner_radius = max(inner_radius - drum_wall * 0.40, 1.0e-4)+ if backplate:+ rear_plate = (+ cq.Workplane("XY")+ .circle(outer_radius)+ .circle(side_plate_inner_radius)+ .extrude(side_plate_thickness, both=False)+ .translate((0.0, 0.0, -width * 0.5))+ )+ shape = shape.union(rear_plate)+ if shroud:+ front_plate = (+ cq.Workplane("XY")+ .circle(outer_radius)+ .circle(side_plate_inner_radius)+ .extrude(side_plate_thickness, both=False)+ .translate((0.0, 0.0, width * 0.5 - side_plate_thickness))+ )+ shape = shape.union(front_plate)++ sweep_rad = blade_sweep_deg * pi / 180.0+ inner_attach_radius = max(inner_radius - drum_wall * 0.08, 1.0e-4)+ outer_attach_radius = outer_radius - blade_thickness * 0.35+ rear_clear = side_plate_thickness * 0.95 if backplate else blade_thickness * 0.50+ front_clear = side_plate_thickness * 0.95 if shroud else blade_thickness * 0.50+ blade_length = width - rear_clear - front_clear+ if blade_length <= blade_thickness * 1.5:+ raise ValueError("width is too small for the blower blade depth")+ blade_center_z = (rear_clear - front_clear) * 0.5++ def annulus_point(radius: float, angle_rad: float) -> tuple:+ return (radius * cos(angle_rad), radius * sin(angle_rad))++ def thicken_centerline(+ centerline: list,+ strip_thickness: float,+ ) -> list:+ half_t = strip_thickness * 0.5+ positive: list = []+ negative: list = []+ count = len(centerline)+ for index, point in enumerate(centerline):+ if index == 0:+ tangent = (+ centerline[1][0] - point[0],+ centerline[1][1] - point[1],+ )+ elif index == count - 1:+ tangent = (+ point[0] - centerline[index - 1][0],+ point[1] - centerline[index - 1][1],+ )+ else:+ tangent = (+ centerline[index + 1][0] - centerline[index - 1][0],+ centerline[index + 1][1] - centerline[index - 1][1],+ )+ tangent_norm = sqrt(tangent[0] * tangent[0] + tangent[1] * tangent[1])+ if tangent_norm <= _EPS:+ normal = (0.0, 1.0)+ else:+ normal = (-tangent[1] / tangent_norm, tangent[0] / tangent_norm)+ positive.append((point[0] + normal[0] * half_t, point[1] + normal[1] * half_t))+ negative.append((point[0] - normal[0] * half_t, point[1] - normal[1] * half_t))+ return positive + list(reversed(negative))++ def annular_sector_polygon(+ inner_r: float,+ outer_r: float,+ start_angle: float,+ end_angle: float,+ *,+ segments: int = 8,+ ) -> list:+ outer_points = [+ annulus_point(+ outer_r,+ start_angle + (end_angle - start_angle) * (index / float(segments)),+ )+ for index in range(segments + 1)+ ]+ inner_points = [+ annulus_point(+ inner_r,+ end_angle - (end_angle - start_angle) * (index / float(segments)),+ )+ for index in range(segments + 1)+ ]+ return outer_points + inner_points++ angle_step = 2.0 * pi / float(blade_count)+ drum_window_inner_radius = max(inner_radius - drum_wall - 1.0e-4, 1.0e-4)+ drum_window_outer_radius = inner_radius + 1.0e-4+ drum_window_span = angle_step * 0.34+ drum_window_length = width - 2.0 * max(side_plate_thickness * 1.15, blade_thickness * 0.8)+ if drum_window_length > blade_thickness:+ for window_index in range(blade_count):+ passage_center = window_index * angle_step + angle_step * 0.5 - sweep_rad * 0.18+ window_profile = annular_sector_polygon(+ drum_window_inner_radius,+ drum_window_outer_radius,+ passage_center - drum_window_span * 0.5,+ passage_center + drum_window_span * 0.5,+ )+ drum_window = (+ cq.Workplane("XY")+ .polyline(window_profile)+ .close()+ .extrude(drum_window_length * 0.5, both=True)+ )+ shape = shape.cut(drum_window)++ for blade_index in range(blade_count):+ base_angle = blade_index * angle_step+ centerline = [+ annulus_point(inner_attach_radius, base_angle - sweep_rad * 0.18),+ annulus_point(+ inner_attach_radius + radial_span * 0.34, base_angle + sweep_rad * 0.10+ ),+ annulus_point(+ inner_attach_radius + radial_span * 0.68, base_angle + sweep_rad * 0.42+ ),+ annulus_point(outer_attach_radius, base_angle + sweep_rad * 0.72),+ ]+ blade_profile = thicken_centerline(centerline, blade_thickness)+ blade = (+ cq.Workplane("XY")+ .polyline(blade_profile)+ .close()+ .extrude(blade_length * 0.5, both=True)+ .translate((0.0, 0.0, blade_center_z))+ )+ shape = shape.union(blade)++ solid = _to_shape(shape)+ if not center:+ solid = _fans_shift_to_z0(solid)+ self._set(solid)diff --git a/toolkits/curation/articraft/shim_parts/gears.py b/toolkits/curation/articraft/shim_parts/gears.pynew file mode 100644index 00000000..bba5ad3e--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/gears.py@@ -0,0 +1,1106 @@+from articraft_cq import cq, math, MeshGeometry, _to_shape, mesh_from_geometry, ValidationError # noqa: F401++# ===========================================================================+# @section geom-gears (SpurGear / BevelGearPair / Worm)+#+# Clean-room CadQuery reimplementation mirroring the Apache-2.0 Articraft sdk+# gear surface (itself vendored/adapted from `cq_gears`, github meadiode/cq_gears,+# Apache License 2.0). The sdk gears are ALREADY pure parametric CadQuery B-rep+# with a GearBase/.build() API, so we mirror their cq calls exactly: each gear+# class stores parameters at construction and emits a real cq solid/compound from+# .build(). Model scripts run unchanged:+#+# spur = SpurGear(module=0.001, teeth_number=20, width=0.006, bore_d=0.004)+# body = cq.Workplane("XY").gear(spur).val()+# mesh = mesh_from_cadquery(body, "spur_gear")+#+# Only the assigned names (SpurGear, BevelGearPair, Worm) plus the dependency+# classes/helpers they require (GearBase, BevelGear, math + OCP helpers, and the+# gear()/addGear() Workplane plugins) are copied here.+# ===========================================================================+import numpy as np+from OCP.BRepAdaptor import BRepAdaptor_Surface+from OCP.BRepBuilderAPI import (+ BRepBuilderAPI_MakeEdge,+ BRepBuilderAPI_MakeFace,+ BRepBuilderAPI_MakeWire,+ BRepBuilderAPI_Sewing,+)+from OCP.GeomAPI import GeomAPI_IntSS+from OCP.ShapeAnalysis import ShapeAnalysis_FreeBounds+from OCP.ShapeFix import ShapeFix_Face+from OCP.TopoDS import TopoDS+from OCP.TopTools import TopTools_HSequenceOfShape, TopTools_ListOfShape+++# ---------------------------------------------------------------------------+# geometry helpers (verbatim from the sdk gear surface)+# ---------------------------------------------------------------------------+def sphere_to_cartesian(r, gamma, theta):+ """Convert spherical coordinates to cartesian."""++ return (+ r * np.sin(gamma) * np.sin(theta),+ r * np.sin(gamma) * np.cos(theta),+ r * np.cos(gamma),+ )+++def s_arc(sr, c_gamma, c_theta, r_delta, start, end, n=32):+ """Get arc points plotted on a sphere's surface."""++ t = np.expand_dims(np.linspace(start, end, n), axis=1)+ a = sphere_to_cartesian(1.0, c_gamma + r_delta, c_theta)+ k = sphere_to_cartesian(1.0, c_gamma, c_theta)+ c = np.cos(t) * a + np.sin(t) * np.cross(k, a) + np.dot(k, a) * (1.0 - np.cos(t)) * k+ c = c * sr+ return [dim.squeeze() for dim in np.hsplit(c, 3)]+++def s_inv(gamma0, gamma):+ """Spherical involute curve function."""++ phi = np.arccos(np.tan(gamma0) / np.tan(gamma))+ return np.arccos(np.cos(gamma) / np.cos(gamma0)) / np.sin(gamma0) - phi+++def circle3d_by3points(a, b, c):+ """Find a circle in 3D space defined by three points."""++ u = b - a+ w = np.cross(c - a, u)+ u = u / np.linalg.norm(u)+ w = w / np.linalg.norm(w)+ v = np.cross(w, u)++ bx = np.dot(b - a, u)+ cx, cy = np.dot(c - a, u), np.dot(c - a, v)++ h = ((cx - bx / 2.0) ** 2 + cy**2 - (bx / 2.0) ** 2) / (2.0 * cy)+ cc = a + u * (bx / 2.0) + v * h+ r = np.linalg.norm(a - cc)++ return r, cc+++def rotation_matrix(axis, alpha):+ """Construct a 3D rotation transform matrix."""++ ux, uy, uz = axis+ sina, cosa = np.sin(alpha), np.cos(alpha)+ return np.array(+ (+ (+ cosa + (1.0 - cosa) * ux**2,+ ux * uy * (1.0 - cosa) - uz * sina,+ ux * uz * (1.0 - cosa) + uy * sina,+ ),+ (+ uy * ux * (1.0 - cosa) + uz * sina,+ cosa + (1.0 - cosa) * uy**2,+ uy * uz * (1.0 - cosa) - ux * sina,+ ),+ (+ uz * ux * (1.0 - cosa) - uy * sina,+ uz * uy * (1.0 - cosa) + ux * sina,+ cosa + (1.0 - cosa) * uz**2,+ ),+ )+ )+++def angle_between(o, a, b):+ """Find the angle between vectors OA and OB."""++ p = a - o+ q = b - o+ return np.arccos(np.dot(p, q) / (np.linalg.norm(p) * np.linalg.norm(q)))+++def make_shell(faces, tol=1e-2):+ """Like ``cq.Shell.makeShell`` but allows an explicit tolerance."""++ shell_builder = BRepBuilderAPI_Sewing(tol)+ for face in faces:+ shell_builder.Add(face.wrapped)+ shell_builder.Perform()+ return cq.Shell(shell_builder.SewedShape())+++def make_cross_section_face(faces, cut_plane, int_tol=1e-7, wire_con_tol=1e-3):+ ss = GeomAPI_IntSS()+ cps = BRepAdaptor_Surface(cut_plane.wrapped).Surface().Surface()++ curves = []+ for face in faces:+ gfs = BRepAdaptor_Surface(face.wrapped).Surface().Surface()+ ss.Perform(cps, gfs, int_tol)+ if ss.NbLines():+ for i in range(ss.NbLines()):+ curves.append(ss.Line(i + 1))++ edges = []+ for curve in curves:+ eb = BRepBuilderAPI_MakeEdge(curve)+ edges.append(eb.Edge())++ wb = BRepBuilderAPI_MakeWire()+ elist = TopTools_ListOfShape()+ for edge in edges:+ elist.Append(edge)+ wb.Add(elist)++ if not wb.IsDone():+ edges_in = TopTools_HSequenceOfShape()+ wires_out = TopTools_HSequenceOfShape()+ for edge in edges:+ edges_in.Append(edge)+ ShapeAnalysis_FreeBounds.ConnectEdgesToWires_s(+ edges_in,+ wire_con_tol,+ False,+ wires_out,+ )+ wire = TopoDS.Wire_s(wires_out.First())+ else:+ wire = wb.Wire()++ fb = BRepBuilderAPI_MakeFace(wire, True)+ face = fb.Face()+ if not cq.Face(face).isValid():+ fix = ShapeFix_Face(face)+ fix.FixOrientation()+ fix.Perform()+ face = fix.Face()+ return cq.Face(face)+++# ---------------------------------------------------------------------------+# gear classes+# ---------------------------------------------------------------------------+class GearBase:+ ka = 1.0+ kd = 1.25++ curve_points = 20+ surface_splines = 5++ wire_comb_tol = 1e-2+ spline_approx_tol = 1e-2+ shell_sewing_tol = 1e-2+ isection_tol = 1e-7+ spline_approx_min_deg = 3+ spline_approx_max_deg = 8++ def __init__(self, *args, **kv_args):+ raise NotImplementedError("Constructor is not defined")++ def build(self, **kv_params):+ params = {**self.build_params, **kv_params}+ return self._build(**params)+++class SpurGear(GearBase):+ def __init__(+ self,+ module,+ teeth_number,+ width,+ pressure_angle=20.0,+ helix_angle=0.0,+ clearance=0.0,+ backlash=0.0,+ addendum_coeff=None,+ dedendum_coeff=None,+ **build_params,+ ):+ if addendum_coeff is not None and addendum_coeff <= 0:+ raise ValueError("Addendum coefficient (addendum_coeff) must be greater than 0.")+ if dedendum_coeff is not None and dedendum_coeff <= 0:+ raise ValueError("Dedendum coefficient (dedendum_coeff) must be greater than 0.")++ self.ka = addendum_coeff if addendum_coeff is not None else self.ka+ self.kd = dedendum_coeff if dedendum_coeff is not None else self.kd+ self.m = m = module+ self.z = z = teeth_number+ self.a0 = a0 = np.radians(pressure_angle)+ self.clearance = clearance+ self.backlash = backlash+ self.helix_angle = np.radians(helix_angle)+ self.width = width++ d0 = m * z+ adn = self.ka / (z / d0)+ ddn = self.kd / (z / d0)++ if 2.0 * ddn + 2.0 * clearance >= d0:+ raise ValueError(+ "Invalid dedendum or clearance: resulting dedendum circle diameter is negative or zero."+ )++ da = d0 + 2.0 * adn+ dd = d0 - 2.0 * ddn - 2.0 * clearance+ s0 = m * (np.pi / 2.0 - backlash * np.tan(a0))+ inv_a0 = np.tan(a0) - a0++ self.r0 = r0 = d0 / 2.0+ self.ra = ra = da / 2.0+ self.rd = rd = dd / 2.0+ self.rb = rb = np.cos(a0) * d0 / 2.0+ self.rr = rr = max(rb, rd)+ self.tau = tau = np.pi * 2.0 / z++ if helix_angle != 0.0:+ self.twist_angle = width / (r0 * np.tan(np.pi / 2.0 - self.helix_angle))+ else:+ self.surface_splines = 2+ self.twist_angle = 0.0++ self.build_params = build_params++ r = np.linspace(rr, ra, self.curve_points)+ cos_a = r0 / r * np.cos(a0)+ a = np.arccos(np.clip(cos_a, -1.0, 1.0))+ inv_a = np.tan(a) - a+ s = r * (s0 / d0 + inv_a0 - inv_a)+ phi = s / r+ self.t_lflank_pts = np.dstack(+ (np.cos(phi) * r, np.sin(phi) * r, np.zeros(self.curve_points))+ ).squeeze()++ b = np.linspace(phi[-1], -phi[-1], self.curve_points)+ self.t_tip_pts = np.dstack(+ (np.cos(b) * ra, np.sin(b) * ra, np.zeros(self.curve_points))+ ).squeeze()++ self.t_rflank_pts = np.dstack(+ ((np.cos(-phi) * r)[::-1], (np.sin(-phi) * r)[::-1], np.zeros(self.curve_points))+ ).squeeze()++ rho = tau - phi[0] * 2.0+ p1 = np.array((self.t_rflank_pts[-1][0], self.t_rflank_pts[-1][1], 0.0))+ p2 = np.array((np.cos(-phi[0] - rho / 2.0) * rd, np.sin(-phi[0] - rho / 2.0) * rd, 0.0))+ p3 = np.array((np.cos(-phi[0] - rho) * rr, np.sin(-phi[0] - rho) * rr, 0.0))++ bcr, bcxy = circle3d_by3points(p1, p2, p3)+ t1 = np.arctan2(p1[1] - bcxy[1], p1[0] - bcxy[0])+ t2 = np.arctan2(p3[1] - bcxy[1], p3[0] - bcxy[0])+ if t1 < 0.0:+ t1 += np.pi * 2.0+ if t2 < 0.0:+ t2 += np.pi * 2.0+ t1, t2 = min(t1, t2), max(t1, t2)+ t = np.linspace(t1 + np.pi * 2.0, t2 + np.pi * 2.0, self.curve_points)+ self.t_root_pts = np.dstack(+ (bcxy[0] + bcr * np.cos(t), bcxy[1] + bcr * np.sin(t), np.zeros(self.curve_points))+ ).squeeze()++ def tooth_points(self):+ return np.concatenate(+ (self.t_lflank_pts, self.t_tip_pts, self.t_rflank_pts, self.t_root_pts)+ )++ def gear_points(self):+ tpts = self.tooth_points()+ pts = tpts.copy()+ angle = self.tau+ for _ in range(self.z - 1):+ pts = np.concatenate((pts, tpts @ rotation_matrix((0.0, 0.0, 1.0), angle)))+ angle += self.tau+ return pts++ def _build_tooth_faces(self, twist_angle_a, twist_angle_b, z_pos, width):+ surf_splines = int(np.ceil(abs(self.twist_angle) / np.pi))+ surf_splines = max(1, surf_splines) * self.surface_splines+ spline_tf = np.linspace(+ (twist_angle_a, z_pos), (twist_angle_b, z_pos + width), surf_splines+ )+ t_faces = []+ for spline in (+ self.t_lflank_pts,+ self.t_tip_pts,+ self.t_rflank_pts,+ self.t_root_pts,+ ):+ face_pts = []+ for a, z in spline_tf:+ r_mat = rotation_matrix((0.0, 0.0, 1.0), a)+ pts = spline.copy()+ pts[:, 2] = z+ pts = pts @ r_mat+ face_pts.append([cq.Vector(*pt) for pt in pts])+ face = cq.Face.makeSplineApprox(+ face_pts,+ tol=self.spline_approx_tol,+ minDeg=self.spline_approx_min_deg,+ maxDeg=self.spline_approx_max_deg,+ )+ t_faces.append(face)+ return t_faces++ def _build_gear_faces(self):+ t_faces = self._build_tooth_faces(0.0, self.twist_angle, 0.0, self.width)+ faces = []+ for i in range(self.z):+ for tf in t_faces:+ faces.append(+ tf.rotate(+ (0.0, 0.0, 0.0),+ (0.0, 0.0, 1.0),+ np.degrees(self.tau * i),+ )+ )+ wp = cq.Workplane("XY").add(faces)+ topface_wires = cq.Wire.combine(wp.edges("<Z").vals(), tol=self.wire_comb_tol)+ topface = cq.Face.makeFromWires(topface_wires[0])+ botface_wires = cq.Wire.combine(wp.edges(">Z").vals(), tol=self.wire_comb_tol)+ botface = cq.Face.makeFromWires(botface_wires[0])+ wp = wp.add(topface).add(botface)+ return wp.vals()++ def _make_bore(self, body, bore_d):+ if bore_d is None:+ return body+ return (+ cq.Workplane("XY")+ .add(body)+ .faces("<Z")+ .workplane()+ .circle(bore_d / 2.0)+ .cutThruAll()+ .val()+ )++ def _make_teeth_cutout_wire(self, plane, t1, t2, twist_angle):+ at1 = t1 * self.tau + self.tau / 2.0 + twist_angle+ at2 = t2 * self.tau + self.tau / 2.0 + twist_angle+ p1x = np.cos(at1)+ p1y = np.sin(at1)+ p2x = np.cos((at1 + at2) / 2.0)+ p2y = np.sin((at1 + at2) / 2.0)+ p3x = np.cos(at2)+ p3y = np.sin(at2)+ rc = self.ra + 1.0+ rd = self.rd - 0.01+ return (+ plane.moveTo(p1x * rd, p1y * rd)+ .lineTo(p1x * rc, p1y * rc)+ .threePointArc((p2x * rc, p2y * rc), (p3x * rc, p3y * rc))+ .lineTo(p3x * rd, p3y * rd)+ .threePointArc((p2x * rd, p2y * rd), (p1x * rd, p1y * rd))+ .close()+ )++ def _remove_teeth(self, body, t1, t2):+ plane = cq.Workplane("XY").workplane(offset=-0.1).add(body)+ if self.twist_angle == 0.0:+ cutout = self._make_teeth_cutout_wire(plane, t1, t2, 0.0).extrude(+ self.width + 0.2,+ combine=False,+ )+ else:+ cutout = self._make_teeth_cutout_wire(plane, t1, t2, 0.0).twistExtrude(+ self.width + 0.2,+ np.degrees(-self.twist_angle),+ combine=False,+ )+ return cq.Workplane("XY").add(body).cut(cutout).val()++ def _make_missing_teeth(self, body, missing_teeth):+ if missing_teeth is None:+ return body+ if isinstance(missing_teeth[0], (list, tuple)):+ for t1, t2 in missing_teeth:+ body = self._remove_teeth(body, t1, t2)+ else:+ t1, t2 = missing_teeth+ body = self._remove_teeth(body, t1, t2)+ return body++ def _make_recess(+ self,+ body,+ hub_d,+ recess_d,+ recess=None,+ bottom_recess=None,+ bottom_hub_d=None,+ bottom_recess_d=None,+ ):+ if recess is None and bottom_recess is None:+ return body+ if recess is not None:+ assert recess_d is not None, "Top face recess diameter is not set"+ if bottom_recess is not None:+ assert bottom_recess_d is not None or recess_d is not None, (+ "Bottom face recess diameter is not set"+ )+ if recess:+ body = cq.Workplane("XY").add(body).faces(">Z").workplane()+ if hub_d is not None:+ body = body.circle(hub_d / 2.0)+ body = body.circle(recess_d / 2.0).cutBlind(-recess).val()+ if bottom_recess:+ body = cq.Workplane("XY").add(body).faces("<Z").workplane()+ if bottom_hub_d is None:+ bottom_hub_d = hub_d+ if bottom_recess_d is None:+ bottom_recess_d = recess_d+ if bottom_hub_d is not None:+ body = body.circle(bottom_hub_d / 2.0)+ body = body.circle(bottom_recess_d / 2.0).cutBlind(-bottom_recess).val()+ return body++ def _make_hub(self, body, hub_d, hub_length, bore_d):+ if hub_length is None:+ return body+ assert hub_d is not None, "Hub diameter is not set"+ body = cq.Workplane("XY").add(body).faces(">Z").workplane()+ if bore_d is not None:+ body = body.circle(bore_d / 2.0)+ body = body.circle(hub_d / 2.0).extrude(hub_length)+ return body.val()++ def _make_spokes(self, body, spokes_id, spokes_od, n_spokes, spoke_width, spoke_fillet):+ if n_spokes is None:+ return body+ assert n_spokes > 1, "Number of spokes must be > 1"+ assert spoke_width is not None, "Spoke width is not set"+ assert spokes_od is not None, "Outer spokes diameter is not set"+ if spokes_id is None:+ r1 = spoke_width / 2.0+ else:+ r1 = max(spoke_width / 2.0, spokes_id / 2.0)+ r2 = spokes_od / 2.0+ r1 += 0.0001+ r2 -= 0.0001+ tau = np.pi * 2.0 / n_spokes+ a1 = np.arcsin((spoke_width / 2.0) / (spokes_id / 2.0))+ a2 = np.arcsin((spoke_width / 2.0) / (spokes_od / 2.0))+ a3 = tau - a2+ a4 = tau - a1+ cutout = (+ cq.Workplane("XY")+ .workplane(offset=-0.1)+ .moveTo(np.cos(a1) * r1, np.sin(a1) * r1)+ .lineTo(np.cos(a2) * r2, np.sin(a2) * r2)+ .radiusArc((np.cos(a3) * r2, np.sin(a3) * r2), -r2)+ .lineTo(np.cos(a4) * r1, np.sin(a4) * r1)+ .radiusArc((np.cos(a1) * r1, np.sin(a1) * r1), r1)+ .close()+ .extrude(self.width + 1.0)+ )+ if spoke_fillet is not None:+ cutout = cutout.edges("|Z").fillet(spoke_fillet)+ body = cq.Workplane("XY").add(body)+ for i in range(n_spokes):+ body = body.cut(+ cutout.rotate(+ (0.0, 0.0, 0.0),+ (0.0, 0.0, 1.0),+ np.degrees(tau * i),+ )+ )+ return body.val()++ def _make_chamfer(self, body, chamfer=None, chamfer_top=None, chamfer_bottom=None):+ e = 0.01+ if chamfer is None and chamfer_top is None and chamfer_bottom is None:+ return body+ if chamfer is not None:+ if chamfer_top is None:+ chamfer_top = chamfer+ if chamfer_bottom is None:+ chamfer_bottom = chamfer+ if chamfer_top is not None:+ if isinstance(chamfer_top, (list, tuple)):+ wx, wy = chamfer_top+ else:+ wx, wy = chamfer_top, chamfer_top+ cutter = (+ cq.Workplane("XZ")+ .moveTo(self.ra - wx, self.width + e)+ .hLine(wx + e)+ .vLine(-wy - e)+ .close()+ .revolve()+ )+ body = cq.Workplane("XY").add(body).cut(cutter)+ if chamfer_bottom is not None:+ if isinstance(chamfer_bottom, (list, tuple)):+ wx, wy = chamfer_bottom+ else:+ wx, wy = chamfer_bottom, chamfer_bottom+ cutter = (+ cq.Workplane("XZ")+ .moveTo(self.ra + e, wy)+ .vLine(-wy - e)+ .hLine(-wx - e)+ .close()+ .revolve()+ )+ body = cq.Workplane("XY").add(body).cut(cutter)+ return body.val()++ def _build(+ self,+ bore_d=None,+ missing_teeth=None,+ hub_d=None,+ hub_length=None,+ recess_d=None,+ recess=None,+ bottom_recess=None,+ bottom_recess_d=None,+ bottom_hub_d=None,+ n_spokes=None,+ spoke_width=None,+ spoke_fillet=None,+ spokes_id=None,+ spokes_od=None,+ chamfer=None,+ chamfer_top=None,+ chamfer_bottom=None,+ *args,+ **kv_args,+ ):+ faces = self._build_gear_faces()+ shell = make_shell(faces, tol=self.shell_sewing_tol)+ body = cq.Solid.makeSolid(shell)+ body = self._make_chamfer(body, chamfer, chamfer_top, chamfer_bottom)+ body = self._make_bore(body, bore_d)+ body = self._make_missing_teeth(body, missing_teeth)+ body = self._make_recess(+ body,+ hub_d,+ recess_d,+ recess,+ bottom_recess=bottom_recess,+ bottom_hub_d=bottom_hub_d,+ bottom_recess_d=bottom_recess_d,+ )+ body = self._make_hub(body, hub_d, hub_length, bore_d)+ if spokes_id is None:+ spokes_id = hub_d+ if spokes_od is None:+ spokes_od = recess_d+ body = self._make_spokes(body, spokes_id, spokes_od, n_spokes, spoke_width, spoke_fillet)+ return body+++class BevelGear(GearBase):+ surface_splines = 12++ def __init__(+ self,+ module,+ teeth_number,+ cone_angle,+ face_width,+ pressure_angle=20.0,+ helix_angle=0.0,+ clearance=0.0,+ backlash=0.0,+ **build_params,+ ):+ self.m = m = module+ self.z = z = teeth_number+ self.a0 = a0 = np.radians(pressure_angle)+ self.clearance = clearance+ self.backlash = backlash+ self.helix_angle = np.radians(helix_angle)+ self.face_width = face_width++ self.gamma_p = gamma_p = np.radians(cone_angle)+ rp = m * z / 2.0+ self.gs_r = gs_r = rp / np.sin(gamma_p)+ assert gs_r > face_width, f"face_width value is too big, it should be < {gs_r:0.3f}"+ self.gamma_b = gamma_b = np.arcsin(np.cos(a0) * np.sin(gamma_p))+ self.gamma_f = gamma_f = gamma_p + np.arctan(self.ka * m / gs_r)+ self.gamma_r = gamma_r = gamma_p - np.arctan(self.kd * m / gs_r)+ self.tau = tau = np.pi * 2.0 / z++ if helix_angle != 0.0:+ beta = np.arctan(face_width * np.tan(self.helix_angle) / (2.0 * gs_r - face_width))+ self.twist_angle = np.arcsin(gs_r / rp * np.sin(beta)) * 2.0+ else:+ self.surface_splines = 2+ self.twist_angle = 0.0+ assert not np.isnan(self.twist_angle), "Twist angle is NaN"++ self.build_params = build_params+ self.cone_h = np.cos(gamma_r) * gs_r+ phi_r = s_inv(gamma_b, gamma_p)+ self.mp_theta = mp_theta = np.pi / z + 2.0 * phi_r++ gamma_tr = max(gamma_b, gamma_r)+ gamma = np.linspace(gamma_tr, gamma_f, self.curve_points)+ theta = s_inv(gamma_b, gamma) + backlash / (module * teeth_number)+ self.t_lflank_pts = np.dstack(sphere_to_cartesian(1.0, gamma, theta)).squeeze()++ theta_tip = np.linspace(theta[-1], mp_theta - theta[-1], self.curve_points)+ self.t_tip_pts = np.dstack(+ sphere_to_cartesian(1.0, np.full(self.curve_points, gamma_f), theta_tip)+ ).squeeze()++ self.t_rflank_pts = np.dstack(+ sphere_to_cartesian(1.0, gamma[::-1], mp_theta - theta[::-1])+ ).squeeze()++ if gamma_r < gamma_b:+ p1 = self.t_rflank_pts[-1]+ p2 = np.array(sphere_to_cartesian(1.0, gamma_b, theta[0] + tau))+ p3 = np.array(sphere_to_cartesian(1.0, gamma_r, (tau + mp_theta) / 2.0))+ rr, rcc = circle3d_by3points(p1, p2, p3)+ rcc_gamma = np.arccos(np.dot(p3, rcc) / (np.linalg.norm(p3) * np.linalg.norm(rcc)))+ p1p3 = angle_between(rcc, p1, p3)+ a_start = (np.pi - p1p3 * 2.0) / 2.0+ a_end = -a_start + np.pi+ self.t_root_pts = np.dstack(+ s_arc(+ 1.0,+ gamma_r + rcc_gamma,+ (tau + mp_theta) / 2.0,+ rcc_gamma,+ np.pi / 2.0 + a_start,+ np.pi / 2.0 + a_end,+ self.curve_points,+ )+ ).squeeze()+ else:+ r_theta = np.linspace(mp_theta - theta[0], theta[0] + tau, self.curve_points)+ self.t_root_pts = np.dstack(+ sphere_to_cartesian(1.0, np.full(self.curve_points, gamma_tr), r_theta)+ ).squeeze()++ def tooth_points(self):+ return np.concatenate(+ (self.t_lflank_pts, self.t_tip_pts, self.t_rflank_pts, self.t_root_pts)+ )++ def gear_points(self):+ tpts = self.tooth_points()+ pts = tpts.copy()+ angle = self.tau+ for _ in range(self.z - 1):+ pts = np.concatenate((pts, tpts @ rotation_matrix((0.0, 0.0, 1.0), angle)))+ angle += self.tau+ return pts++ def _build_tooth_faces(self):+ pc_h = np.cos(self.gamma_r) * self.gs_r+ pc_f = pc_h / np.cos(self.gamma_f)+ pc_rb = pc_f * np.sin(self.gamma_f)+ tc_h = np.cos(self.gamma_f) * (self.gs_r - self.face_width)+ tc_f = tc_h / np.cos(self.gamma_r)+ tc_rb = tc_f * np.sin(self.gamma_f)+ ta1 = -(pc_f - self.gs_r) / self.face_width * self.twist_angle+ ta2 = (self.gs_r - tc_f) / self.face_width * self.twist_angle++ surf_splines = int(np.ceil(abs(self.twist_angle) / (np.pi * 2.0)))+ surf_splines = max(1, surf_splines) * self.surface_splines+ spline_tf = np.linspace((pc_f, ta1), (tc_f - 0.01, ta2), surf_splines)++ tcp_size = tc_rb * 1000.0+ top_cut_plane = cq.Face.makePlane(+ length=tcp_size, width=tcp_size, basePnt=(0.0, 0.0, tc_h), dir=(0.0, 0.0, 1.0)+ )+ bcp_size = pc_rb * 1000.0+ bott_cut_plane = cq.Face.makePlane(+ length=bcp_size, width=bcp_size, basePnt=(0.0, 0.0, pc_h), dir=(0.0, 0.0, 1.0)+ )++ def get_zmax(face):+ return face.BoundingBox().zmax++ t_faces = []+ for spline in (+ self.t_lflank_pts,+ self.t_tip_pts,+ self.t_rflank_pts,+ self.t_root_pts,+ ):+ face_pts = []+ for r, a in spline_tf:+ r_mat = rotation_matrix((0.0, 0.0, 1.0), a)+ pts = (spline @ r_mat) * r+ face_pts.append([cq.Vector(*pt) for pt in pts])+ face = cq.Face.makeSplineApprox(+ face_pts,+ tol=self.spline_approx_tol,+ minDeg=self.spline_approx_min_deg,+ maxDeg=self.spline_approx_max_deg,+ )+ cpd = face.split(top_cut_plane)+ face = max(list(cpd), key=get_zmax) if isinstance(cpd, cq.Compound) else cpd+ cpd = face.split(bott_cut_plane)+ face = min(list(cpd), key=get_zmax) if isinstance(cpd, cq.Compound) else cpd+ t_faces.append(face)+ return t_faces++ def _build_gear_faces(self):+ t_faces = self._build_tooth_faces()+ faces = []+ for i in range(self.z):+ for tf in t_faces:+ faces.append(+ tf.rotate(+ (0.0, 0.0, 0.0),+ (0.0, 0.0, 1.0),+ np.degrees(self.tau * i),+ )+ )+ wp = cq.Workplane("XY").add(faces)+ topface_wires = cq.Wire.combine(wp.edges("<Z").vals(), tol=self.wire_comb_tol)+ topface = cq.Face.makeFromWires(topface_wires[0])+ botface_wires = cq.Wire.combine(wp.edges(">Z").vals(), tol=self.wire_comb_tol)+ botface = cq.Face.makeFromWires(botface_wires[0])+ wp = wp.add(topface).add(botface)+ return wp.vals()++ def _trim_bottom(self, body, do_trim=False):+ if not do_trim:+ return body+ r = self.gs_r+ p1 = sphere_to_cartesian(r, self.gamma_r * 0.99, np.pi / 2.0)+ p2 = sphere_to_cartesian(r, self.gamma_p, np.pi / 2.0)+ p3 = sphere_to_cartesian(r, self.gamma_f * 1.01, np.pi / 2.0)+ x1 = np.tan(self.gamma_f) * self.cone_h + 1.0+ trimmer = (+ cq.Workplane("XZ")+ .moveTo(p1[0], p1[2])+ .threePointArc((p2[0], p2[2]), (p3[0], p3[2]))+ .lineTo(x1, p3[2])+ .lineTo(x1, p1[2])+ .close()+ .revolve(combine=False)+ )+ return cq.Workplane("XY").add(body).cut(trimmer).val()++ def _trim_top(self, body, do_trim=False):+ if not do_trim:+ return body+ r = self.gs_r - self.face_width+ p1 = sphere_to_cartesian(r, self.gamma_r, np.pi / 2.0)+ p2 = sphere_to_cartesian(r, self.gamma_p, np.pi / 2.0)+ p3 = sphere_to_cartesian(r, self.gamma_f * 1.01, np.pi / 2.0)+ trimmer = (+ cq.Workplane("XZ")+ .moveTo(p1[0], p1[2])+ .threePointArc((p2[0], p2[2]), (p3[0], p3[2]))+ .lineTo(0.0, p3[2])+ .lineTo(0.0, p1[2])+ .close()+ .revolve(combine=False)+ )+ return cq.Workplane("XY").add(body).cut(trimmer).val()++ def _make_bore(self, body, bore_d):+ if bore_d is None:+ return body+ return (+ cq.Workplane("XY")+ .add(body)+ .faces("<Z")+ .workplane()+ .circle(bore_d / 2.0)+ .cutThruAll()+ .val()+ )++ def _build(self, bore_d=None, trim_bottom=True, trim_top=True, **kv_args):+ faces = self._build_gear_faces()+ shell = make_shell(faces)+ body = cq.Solid.makeSolid(shell)+ body = self._trim_bottom(body, trim_bottom)+ body = self._trim_top(body, trim_top)+ t_align_angle = -self.mp_theta / 2.0 - np.pi / 2.0 + np.pi / self.z+ body = (+ cq.Workplane("XY")+ .add(body)+ .rotate((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), 180.0)+ .translate((0.0, 0.0, self.cone_h))+ .rotate((0.0, 0.0, 0.0), (0.0, 0.0, 1.0), np.degrees(t_align_angle))+ .solids()+ .val()+ )+ return self._make_bore(body, bore_d)+++class BevelGearPair(GearBase):+ gear_cls = BevelGear++ asm_gear_color = "goldenrod"+ asm_pinion_color = "lightsteelblue"++ def __init__(+ self,+ module,+ gear_teeth,+ pinion_teeth,+ face_width,+ axis_angle=90.0,+ pressure_angle=20.0,+ helix_angle=0.0,+ clearance=0.0,+ backlash=0.0,+ **build_params,+ ):+ self.axis_angle = axis_angle = np.radians(axis_angle)+ aa_sin = np.sin(axis_angle)+ aa_cos = np.cos(axis_angle)+ delta_gear = np.arctan(aa_sin / (pinion_teeth / gear_teeth + aa_cos))+ delta_pinion = np.arctan(aa_sin / (gear_teeth / pinion_teeth + aa_cos))+ self.gear = self.gear_cls(+ module,+ gear_teeth,+ np.degrees(delta_gear),+ face_width,+ pressure_angle,+ helix_angle,+ clearance,+ backlash,+ )+ self.pinion = self.gear_cls(+ module,+ pinion_teeth,+ np.degrees(delta_pinion),+ face_width,+ pressure_angle,+ -helix_angle,+ backlash=backlash,+ )+ self.build_params = build_params++ def assemble(+ self,+ build_gear=True,+ build_pinion=True,+ transform_pinion=True,+ gear_build_args={},+ pinion_build_args={},+ **kv_args,+ ):+ gearset = cq.Assembly(name="bevel_pair")+ if build_gear:+ in_args = self.build_params.get("gear_build_args", {})+ args = {**self.build_params, **in_args, **kv_args, **gear_build_args}+ gear = self.gear.build(**args)+ gearset.add(gear, name="gear", loc=cq.Location(), color=cq.Color(self.asm_gear_color))+ if build_pinion:+ in_args = self.build_params.get("pinion_build_args", {})+ args = {**self.build_params, **in_args, **kv_args, **pinion_build_args}+ pinion = self.pinion.build(**args)+ loc = cq.Location()+ if transform_pinion:+ loc *= cq.Location(+ cq.Vector(0.0, 0.0, self.gear.cone_h),+ cq.Vector(0.0, 1.0, 0.0),+ np.degrees(self.axis_angle),+ )+ loc *= cq.Location(cq.Vector((0.0, 0.0, -self.pinion.cone_h)))+ if self.pinion.z % 2 == 0:+ loc *= cq.Location(+ cq.Vector(0.0, 0.0, 0.0),+ cq.Vector(0.0, 0.0, 1.0),+ np.degrees(np.pi / self.pinion.z),+ )+ gearset.add(pinion, name="pinion", loc=loc, color=cq.Color(self.asm_pinion_color))+ return gearset++ def _build(self, *args, **kv_args):+ return self.assemble(*args, **kv_args).toCompound()+++class Worm(GearBase):+ surface_splines = 8+ wire_comb_tol = 0.1+ t_face_parts = 2++ def __init__(+ self,+ module,+ lead_angle,+ n_threads,+ length,+ pressure_angle=20.0,+ clearance=0.0,+ backlash=0.0,+ **build_params,+ ):+ self.m = m = module+ self.a0 = a0 = np.radians(pressure_angle)+ self.clearance = clearance+ self.backlash = backlash+ self.lead_angle = np.radians(lead_angle)+ self.length = length+ self.n_threads = n_threads+ self.build_params = build_params++ d0 = self.n_threads * m / np.abs(np.tan(self.lead_angle))+ self.r0 = d0 / 2.0+ adn = self.ka * m+ ddn = self.kd * m+ self.la = la = adn+ self.ld = ld = -(ddn + clearance)+ self.ra = self.r0 + adn+ self.rd = self.r0 - ddn++ s0 = m * (np.pi / 2.0 - backlash * np.tan(a0)) / 2.0+ p1x = np.tan(a0) * abs(ld)+ p1p2 = (abs(la) + abs(ld)) / np.cos(a0)+ p1 = (-s0 - p1x, ld, 0.0)+ p2 = (np.sin(a0) * p1p2 + p1[0], np.cos(a0) * p1p2 + p1[1], 0.0)+ p3 = (-p2[0], p2[1], 0.0)+ p4 = (-p1[0], p1[1], 0.0)+ p5 = (p4[0] + (np.pi * m - p4[0] * 2.0), p4[1], 0.0)+ self.t_lflank_pts = np.array((p1, p2))+ self.t_tip_pts = np.array((p2, p3))+ self.t_rflank_pts = np.array((p3, p4))+ self.t_root_pts = np.array((p4, p5))+ self.tooth_height = abs(la) + abs(ld)++ def tooth_points(self):+ return np.concatenate(+ (self.t_lflank_pts, self.t_tip_pts, self.t_rflank_pts, self.t_root_pts)+ )++ def gear_points(self):+ tpts = self.tooth_points()+ pts = tpts.copy()+ for i in range(10):+ ttpts = tpts.copy()+ ttpts[:, 0] += np.pi * self.m * (i + 1)+ pts = np.concatenate((pts, ttpts))+ return pts++ def _build_tooth_faces(self):+ t_faces = []+ ttx = self.m * np.pi * self.n_threads+ start_x = -ttx / 2.0+ step_x = ttx / self.t_face_parts+ part_turn = np.pi * 2.0 / self.t_face_parts * np.sign(self.lead_angle)+ spline_tf = np.linspace((start_x, 0.0), (start_x + step_x, part_turn), self.surface_splines)+ for spline in (+ self.t_lflank_pts,+ self.t_tip_pts,+ self.t_rflank_pts,+ self.t_root_pts,+ ):+ face_pts = []+ for tx, alpha in spline_tf:+ r_mat = rotation_matrix((1.0, 0.0, 0.0), alpha)+ pts = (spline + (tx, self.r0, 0.0)) @ r_mat+ face_pts.append([cq.Vector(*pt) for pt in pts])+ face = cq.Face.makeSplineApprox(+ face_pts,+ tol=self.spline_approx_tol,+ minDeg=self.spline_approx_min_deg,+ maxDeg=self.spline_approx_max_deg,+ )+ t_faces.append(face)+ faces = []+ for n in range(self.t_face_parts):+ for tf in t_faces:+ faces.append(+ tf.rotate(+ (0.0, 0.0, 0.0),+ (1.0, 0.0, 0.0),+ np.degrees(-n * part_turn),+ ).translate((step_x * n, 0.0, 0.0))+ )+ return faces++ def _build_gear_faces(self):+ step = np.pi * self.m * self.n_threads+ turns = int(np.ceil(self.length / step)) + 2+ x_start = -turns * step / 2.0+ tau = np.pi * 2.0 / self.n_threads+ t_faces = self._build_tooth_faces()+ faces = []+ for th in range(self.n_threads):+ for tf in t_faces:+ faces.append(+ tf.rotate(+ (0.0, 0.0, 0.0),+ (1.0, 0.0, 0.0),+ np.degrees(tau * th),+ )+ )+ nfaces = []+ for i in range(turns):+ for tf in faces:+ nfaces.append(tf.translate((step / 2.0 + x_start + i * step, 0.0, 0.0)))++ cp_size = self.ra * 2.0 + 2.0+ cp_x = self.length / 2.0+ left_cut_plane = cq.Face.makePlane(+ length=cp_size, width=cp_size, basePnt=(-cp_x, 0.0, 0.0), dir=(-1.0, 0.0, 0.0)+ )+ right_cut_plane = cq.Face.makePlane(+ length=cp_size, width=cp_size, basePnt=(cp_x, 0.0, 0.0), dir=(1.0, 0.0, 0.0)+ )+ lface = make_cross_section_face(+ nfaces, left_cut_plane, self.isection_tol, self.wire_comb_tol+ )+ rface = make_cross_section_face(+ nfaces, right_cut_plane, self.isection_tol, self.wire_comb_tol+ )++ def get_xmin(face):+ return face.BoundingBox().xmin++ def get_xmax(face):+ return face.BoundingBox().xmax++ g_faces = []+ for face in nfaces:+ bb = face.BoundingBox()+ if -(self.length / 2.0) < bb.xmin and bb.xmax < (self.length / 2.0):+ g_faces.append(face)+ else:+ cpd = face.split(left_cut_plane)+ if isinstance(cpd, cq.Compound):+ g_faces.append(max(list(cpd), key=get_xmax))+ cpd = face.split(right_cut_plane)+ if isinstance(cpd, cq.Compound):+ g_faces.append(min(list(cpd), key=get_xmin))+ g_faces.append(lface)+ g_faces.append(rface)+ return g_faces++ def _make_bore(self, body, bore_d):+ if bore_d is None:+ return body+ return (+ cq.Workplane("YZ")+ .add(body)+ .faces("<X")+ .workplane()+ .circle(bore_d / 2.0)+ .cutThruAll()+ .val()+ )++ def _build(self, bore_d=None):+ faces = self._build_gear_faces()+ shell = make_shell(faces, tol=self.shell_sewing_tol)+ body = cq.Solid.makeSolid(shell)+ return self._make_bore(body, bore_d)+++# ---------------------------------------------------------------------------+# Workplane plugin helpers (importing this module installs cq.Workplane.gear /+# cq.Workplane.addGear, exactly like the sdk gear surface).+# ---------------------------------------------------------------------------+def gear(self, gear_, *build_args, **build_kv_args):+ gear_body = gear_.build(*build_args, **build_kv_args)+ return self.eachpoint(lambda loc: gear_body.located(loc), True)+++def addGear(self, gear_, *build_args, **build_kv_args):+ return self.union(gear(self, gear_, *build_args, **build_kv_args))+++cq.Workplane.gear = gear+cq.Workplane.addGear = addGeardiff --git a/toolkits/curation/articraft/shim_parts/knobs.py b/toolkits/curation/articraft/shim_parts/knobs.pynew file mode 100644index 00000000..63026870--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/knobs.py@@ -0,0 +1,530 @@+from articraft_cq import (cq, math, MeshGeometry, _CQMesh, _to_shape, mesh_from_geometry,+ _revolve_profile, _extrude_profile, _loft_rings, rounded_rect_profile, superellipse_profile,+ _profile_points_2d, _ensure_ccw, boolean_difference, boolean_union, boolean_intersection,+ Box, Cylinder, Sphere, ValidationError)++# ===========================================================================+# @section geom-knobs (rotary control caps / appliance knobs)+#+# Clean-room CadQuery B-rep reimplementation of the Articraft sdk+# `KnobGeometry` and its detail spec dataclasses. Constructor signatures+# mirror the real sdk exactly; the produced shapes are smooth parametric+# solids (not triangle meshes). Pure cadquery + stdlib only (no numpy).+# ===========================================================================+from contextlib import suppress+from dataclasses import dataclass+from math import cos, pi, sin+from typing import Literal, Optional, Sequence+++# ---------------------------------------------------------------------------+# Detail spec dataclasses (frozen, fields mirror the sdk verbatim). These are+# passed into KnobGeometry; they carry no geometry of their own.+# ---------------------------------------------------------------------------+@dataclass(frozen=True)+class KnobSkirt:+ diameter: float+ height: float+ flare: float = 0.0+ chamfer: float = 0.0+++@dataclass(frozen=True)+class KnobGrip:+ style: Literal[+ "none",+ "fluted",+ "scalloped",+ "knurled",+ "ribbed",+ "diamond_knurl",+ ] = "none"+ count: Optional[int] = None+ depth: float = 0.0+ width: Optional[float] = None+ helix_angle_deg: float = 0.0+++@dataclass(frozen=True)+class KnobIndicator:+ style: Literal["none", "line", "notch", "wedge", "dot"] = "none"+ length: Optional[float] = None+ width: Optional[float] = None+ depth: float = 0.0+ angle_deg: float = 0.0+ mode: Literal["engraved", "raised"] = "engraved"+++@dataclass(frozen=True)+class KnobTopFeature:+ style: Literal["none", "flush_disk", "recessed_disk", "top_insert"] = "none"+ diameter: Optional[float] = None+ depth: float = 0.0+ height: float = 0.0+++@dataclass(frozen=True)+class KnobBore:+ style: Literal["none", "round", "d_shaft", "double_d", "splined", "hex"] = "round"+ diameter: Optional[float] = None+ flat_depth: Optional[float] = None+ spline_count: Optional[int] = None+ spline_depth: float = 0.0+ through: bool = True+++@dataclass(frozen=True)+class KnobRelief:+ style: Literal["side_window", "top_recess", "coin_slot"]+ angle_deg: float = 0.0+ width: Optional[float] = None+ height: Optional[float] = None+ depth: float = 0.0+++# ---------------------------------------------------------------------------+# pure-cadquery helpers (ported verbatim from the sdk cadquery_helpers)+# ---------------------------------------------------------------------------+def _knobs_loft_between_radii_z(cq_module, radii_and_offsets):+ """Loft a stack of circular sections at given (radius, z-offset) pairs."""+ wp = None+ previous_offset = 0.0+ for radius, offset in radii_and_offsets:+ if wp is None:+ wp = cq_module.Workplane("XY").workplane(offset=offset).circle(radius)+ else:+ wp = wp.workplane(offset=offset - previous_offset).circle(radius)+ previous_offset = offset+ if wp is None:+ raise ValidationError("No loft sections were provided")+ return wp.loft(combine=True, ruled=False)+++def _knobs_cut_with_pattern(shape, cutters):+ for cutter in cutters:+ shape = shape.cut(cutter)+ return shape+++class KnobGeometry(MeshGeometry):+ """+ Build a flexible rotary knob aligned to local Z with multiple silhouette families.+ """++ def __init__(+ self,+ diameter: float,+ height: float,+ *,+ body_style: Literal[+ "cylindrical",+ "tapered",+ "domed",+ "mushroom",+ "skirted",+ "hourglass",+ "faceted",+ "lobed",+ ] = "cylindrical",+ top_diameter: Optional[float] = None,+ base_diameter: Optional[float] = None,+ crown_radius: float = 0.0,+ edge_radius: float = 0.0,+ side_draft_deg: float = 0.0,+ skirt: Optional[KnobSkirt] = None,+ grip: Optional[KnobGrip] = None,+ indicator: Optional[KnobIndicator] = None,+ top_feature: Optional[KnobTopFeature] = None,+ bore: Optional[KnobBore] = None,+ body_reliefs: Sequence[KnobRelief] = (),+ center: bool = True,+ ):+ super().__init__()+ diameter = float(diameter)+ height = float(height)+ crown_radius = max(0.0, float(crown_radius))+ edge_radius = max(0.0, float(edge_radius))+ side_draft_deg = float(side_draft_deg)+ if diameter <= 0.0 or height <= 0.0:+ raise ValidationError("diameter and height must be positive")+ if abs(side_draft_deg) >= 45.0:+ raise ValidationError("abs(side_draft_deg) must be < 45")++ base_diameter = float(base_diameter) if base_diameter is not None else diameter+ if base_diameter <= 0.0:+ raise ValidationError("base_diameter must be positive")+ top_diameter = float(top_diameter) if top_diameter is not None else diameter+ if top_diameter <= 0.0:+ raise ValidationError("top_diameter must be positive")++ grip = grip or KnobGrip()+ indicator = indicator or KnobIndicator()+ top_feature = top_feature or KnobTopFeature()+ bore = bore or KnobBore(style="none")+ if skirt is not None and (skirt.diameter <= 0.0 or skirt.height <= 0.0):+ raise ValidationError("KnobSkirt diameter and height must be positive")+ if grip.depth < 0.0:+ raise ValidationError("KnobGrip.depth must be non-negative")+ if indicator.depth < 0.0:+ raise ValidationError("KnobIndicator.depth must be non-negative")+ if top_feature.depth < 0.0 or top_feature.height < 0.0:+ raise ValidationError("KnobTopFeature depth/height must be non-negative")+ if bore.diameter is not None and bore.diameter <= 0.0:+ raise ValidationError("KnobBore.diameter must be positive when provided")+ for relief in body_reliefs:+ if relief.depth < 0.0:+ raise ValidationError("KnobRelief.depth must be non-negative")++ body_height = height+ body_bottom = -height * 0.5+ max_radius = max(base_diameter, top_diameter) * 0.5+ if skirt is not None:+ max_radius = max(max_radius, skirt.diameter * 0.5)++ def body_radius_at(t: float) -> float:+ if body_style == "cylindrical":+ return max(base_diameter, top_diameter) * 0.5+ if body_style == "tapered":+ return (base_diameter * (1.0 - t) + top_diameter * t) * 0.5+ if body_style == "domed":+ if t < 0.72:+ return (base_diameter * 0.5) * (+ 1.0 - min(max(side_draft_deg / 50.0, -0.2), 0.2) * t+ )+ u = (t - 0.72) / 0.28+ return max(+ 0.001,+ top_diameter * 0.5 + (base_diameter * 0.5 - top_diameter * 0.5) * (1.0 - u * u),+ )+ if body_style == "mushroom":+ stem_radius = min(base_diameter, top_diameter, diameter) * 0.28+ cap_radius = max(base_diameter, top_diameter, diameter) * 0.5+ if t < 0.42:+ return stem_radius+ if t < 0.62:+ u = (t - 0.42) / 0.20+ return stem_radius + (cap_radius - stem_radius) * u+ if t < 0.85:+ return cap_radius+ u = (t - 0.85) / 0.15+ return max(cap_radius * (1.0 - 0.20 * u * u), stem_radius)+ if body_style == "skirted":+ skirt_radius = max(base_diameter * 0.55, diameter * 0.52)+ crown_radius_local = top_diameter * 0.5+ if t < 0.40:+ return skirt_radius+ if t < 0.56:+ u = (t - 0.40) / 0.16+ return skirt_radius + (crown_radius_local - skirt_radius) * u+ return crown_radius_local+ if body_style == "hourglass":+ waist_radius = min(base_diameter, top_diameter, diameter) * 0.32+ if t < 0.5:+ u = t / 0.5+ return base_diameter * 0.5 + (waist_radius - base_diameter * 0.5) * u+ u = (t - 0.5) / 0.5+ return waist_radius + (top_diameter * 0.5 - waist_radius) * u+ if body_style == "faceted":+ return (base_diameter * (1.0 - t) + top_diameter * t) * 0.5+ if body_style == "lobed":+ lower_radius = base_diameter * 0.5+ upper_radius = max(top_diameter, diameter * 1.02) * 0.5+ if t < 0.24:+ u = t / 0.24+ return lower_radius + (upper_radius * 0.92 - lower_radius) * u+ if t < 0.80:+ u = (t - 0.24) / 0.56+ return upper_radius * (0.92 + 0.08 * sin(u * pi))+ u = (t - 0.80) / 0.20+ return upper_radius + (top_diameter * 0.5 - upper_radius) * u+ raise ValidationError(f"Unsupported body_style {body_style!r}")++ def section_outline(radius: float, t: float):+ if body_style == "faceted":+ facet_count = 6+ phase = pi / float(facet_count)+ return [+ (+ radius * cos(phase + 2.0 * pi * index / float(facet_count)),+ radius * sin(phase + 2.0 * pi * index / float(facet_count)),+ )+ for index in range(facet_count)+ ]+ if body_style == "lobed":+ lobe_count = 5+ point_count = 72+ blend = min(max((t - 0.10) / 0.30, 0.0), 1.0)+ amplitude = radius * (0.04 + 0.14 * blend)+ valley_floor = radius * 0.62+ points = []+ for index in range(point_count):+ theta = 2.0 * pi * index / float(point_count)+ local_radius = radius - amplitude * (0.5 - 0.5 * cos(lobe_count * theta))+ local_radius = max(local_radius, valley_floor)+ points.append((local_radius * cos(theta), local_radius * sin(theta)))+ return points+ return None++ section_offsets = [+ 0.0,+ body_height * 0.18,+ body_height * 0.42,+ body_height * 0.72,+ body_height,+ ]+ radii_and_offsets = [+ (max(0.001, body_radius_at(offset / body_height)), body_bottom + offset)+ for offset in section_offsets+ ]+ wp = None+ previous_offset = 0.0+ for radius, offset in radii_and_offsets:+ t = (offset - body_bottom) / body_height if body_height > 1e-9 else 0.0+ profile_points = section_outline(radius, t)+ if wp is None:+ wp = cq.Workplane("XY").workplane(offset=offset)+ else:+ wp = wp.workplane(offset=offset - previous_offset)+ if profile_points is None:+ wp = wp.circle(radius)+ else:+ wp = wp.polyline(profile_points).close()+ previous_offset = offset+ if wp is None:+ raise ValidationError("KnobGeometry requires at least one loft section")+ shape = wp.loft(combine=True, ruled=False)++ if skirt is not None:+ skirt_radius = skirt.diameter * 0.5+ skirt_bottom = body_bottom - skirt.height+ skirt_shape = _knobs_loft_between_radii_z(+ cq,+ [+ (max(0.001, skirt_radius * (1.0 + skirt.flare)), skirt_bottom),+ (max(0.001, skirt_radius), body_bottom),+ ],+ )+ shape = shape.union(skirt_shape)+ if skirt.chamfer > 1e-6:+ with suppress(Exception):+ shape = (+ shape.faces("<Z")+ .edges()+ .chamfer(min(skirt.chamfer, skirt.height * 0.7, skirt_radius * 0.25))+ )++ if edge_radius > 0.0:+ with suppress(Exception):+ shape = shape.edges("|Z").fillet(min(edge_radius, max_radius * 0.35, height * 0.18))+ if crown_radius > 0.0:+ with suppress(Exception):+ shape = (+ shape.faces(">Z")+ .edges()+ .fillet(min(crown_radius, max_radius * 0.25, height * 0.16))+ )++ if grip.style != "none" and grip.depth > 1e-6:+ grip_count = grip.count or (28 if grip.style in {"knurled", "diamond_knurl"} else 18)+ if grip_count < 2:+ raise ValidationError("KnobGrip.count must be at least 2")+ grip_width = (+ float(grip.width)+ if grip.width is not None+ else (+ max_radius * 0.18+ if grip.style in {"fluted", "scalloped"}+ else max_radius * 0.10+ )+ )+ if grip_width <= 0.0:+ raise ValidationError("KnobGrip.width must be positive when provided")++ cutters = []+ if grip.style in {"fluted", "scalloped", "ribbed"}:+ cutter_radius = grip_width * (0.55 if grip.style != "ribbed" else 0.38)+ radial_center = max_radius + cutter_radius - grip.depth+ for index in range(grip_count):+ cutter = (+ cq.Workplane("XY")+ .circle(cutter_radius)+ .extrude(+ height + (skirt.height if skirt is not None else 0.0) + 0.01, both=True+ )+ .translate((radial_center, 0.0, body_bottom + height * 0.5))+ .rotate((0.0, 0.0, 0.0), (0.0, 0.0, 1.0), 360.0 * index / float(grip_count))+ )+ cutters.append(cutter)+ else:+ tangential = max(grip_width, max_radius * 0.06)+ radial = max(grip.depth * 1.8, max_radius * 0.06)+ box_height = height * 1.25+ helix_angle = grip.helix_angle_deg if abs(grip.helix_angle_deg) > 1e-6 else 24.0+ for index in range(grip_count):+ base_angle = 360.0 * index / float(grip_count)+ for tilt_sign in (-1.0, 1.0) if grip.style == "diamond_knurl" else (1.0,):+ cutter = (+ cq.Workplane("XY")+ .box(radial, tangential, box_height)+ .translate(+ (max_radius - grip.depth * 0.5, 0.0, body_bottom + height * 0.5)+ )+ .rotate(+ (0.0, 0.0, 0.0), (0.0, 1.0, 0.0), helix_angle * float(tilt_sign)+ )+ .rotate((0.0, 0.0, 0.0), (0.0, 0.0, 1.0), base_angle)+ )+ cutters.append(cutter)+ shape = _knobs_cut_with_pattern(shape, cutters)++ if indicator.style != "none":+ indicator_length = indicator.length or max(diameter * 0.34, 0.003)+ indicator_width = indicator.width or max(diameter * 0.06, 0.0015)+ indicator_depth = max(indicator.depth, max(height * 0.03, 0.0008))+ top_z = body_bottom + height+ if indicator.style in {"line", "notch"}:+ feature = (+ cq.Workplane("XY")+ .box(indicator_length, indicator_width, indicator_depth)+ .translate((indicator_length * 0.18, 0.0, top_z + indicator_depth * 0.5))+ )+ elif indicator.style == "wedge":+ profile = [+ (-indicator_width * 0.5, 0.0),+ (indicator_width * 0.5, 0.0),+ (0.0, indicator_length),+ ]+ feature = (+ cq.Workplane("XY")+ .polyline(profile)+ .close()+ .extrude(indicator_depth)+ .translate((0.0, 0.0, top_z))+ )+ else:+ dot_radius = indicator_width * 0.5+ feature = (+ cq.Workplane("XY")+ .circle(dot_radius)+ .extrude(indicator_depth)+ .translate((indicator_length * 0.22, 0.0, top_z))+ )+ feature = feature.rotate((0.0, 0.0, 0.0), (0.0, 0.0, 1.0), indicator.angle_deg)+ if indicator.mode == "raised" and indicator.style != "notch":+ shape = shape.union(feature)+ else:+ shape = shape.cut(feature.translate((0.0, 0.0, -indicator_depth * 0.5)))++ if top_feature.style != "none":+ feature_diameter = top_feature.diameter or diameter * 0.55+ feature_radius = feature_diameter * 0.5+ top_z = body_bottom + height+ if feature_radius <= 0.0:+ raise ValidationError("KnobTopFeature.diameter must be positive when provided")+ if top_feature.style == "flush_disk":+ feature = (+ cq.Workplane("XY")+ .circle(feature_radius)+ .extrude(max(top_feature.height, height * 0.06))+ .translate((0.0, 0.0, top_z))+ )+ shape = shape.union(feature)+ elif top_feature.style == "top_insert":+ feature = (+ cq.Workplane("XY")+ .circle(feature_radius)+ .extrude(max(top_feature.height, height * 0.04))+ .translate((0.0, 0.0, top_z + height * 0.01))+ )+ shape = shape.union(feature)+ else:+ feature = (+ cq.Workplane("XY")+ .circle(feature_radius)+ .extrude(max(top_feature.depth, height * 0.08))+ .translate((0.0, 0.0, top_z - max(top_feature.depth, height * 0.08)))+ )+ shape = shape.cut(feature)++ if bore.style != "none":+ bore_diameter = bore.diameter or diameter * 0.34+ if bore_diameter <= 0.0 or bore_diameter >= max_radius * 2.0:+ raise ValidationError("KnobBore diameter must fit inside the knob body")+ bore_depth = (+ height + (skirt.height if skirt is not None else 0.0) + 0.01+ if bore.through+ else max(height * 0.7, diameter * 0.22)+ )+ if bore.style == "round":+ bore_cut = cq.Workplane("XY").circle(bore_diameter * 0.5).extrude(bore_depth)+ elif bore.style == "hex":+ bore_cut = cq.Workplane("XY").polygon(6, bore_diameter).extrude(bore_depth)+ elif bore.style in {"d_shaft", "double_d"}:+ flat_depth = (+ float(bore.flat_depth) if bore.flat_depth is not None else bore_diameter * 0.16+ )+ cut_r = bore_diameter * 0.5+ flat_x = cut_r - flat_depth+ circle_points = [+ (cut_r * cos(theta), cut_r * sin(theta))+ for theta in (2.0 * pi * index / 48.0 for index in range(48))+ ]+ if bore.style == "d_shaft":+ profile_points = [(min(point[0], flat_x), point[1]) for point in circle_points]+ else:+ profile_points = [+ (max(min(point[0], flat_x), -flat_x), point[1]) for point in circle_points+ ]+ bore_cut = cq.Workplane("XY").polyline(profile_points).close().extrude(bore_depth)+ else:+ spline_count = bore.spline_count or 8+ if spline_count < 3:+ raise ValidationError("KnobBore.spline_count must be at least 3")+ spline_depth = max(bore.spline_depth, bore_diameter * 0.06)+ outer_r = bore_diameter * 0.5+ inner_r = max(outer_r - spline_depth, outer_r * 0.72)+ points = []+ for index in range(spline_count * 2):+ theta = pi * index / float(spline_count)+ radius = outer_r if index % 2 == 0 else inner_r+ points.append((radius * cos(theta), radius * sin(theta)))+ bore_cut = cq.Workplane("XY").polyline(points).close().extrude(bore_depth)+ bore_cut = bore_cut.translate(+ (0.0, 0.0, body_bottom - (skirt.height if skirt is not None else 0.0))+ )+ shape = shape.cut(bore_cut)++ for relief in body_reliefs:+ relief_depth = max(relief.depth, diameter * 0.04)+ relief_width = relief.width or diameter * 0.22+ relief_height = relief.height or height * 0.18+ if relief.style == "side_window":+ cutter = (+ cq.Workplane("XY")+ .box(relief_depth * 2.2, relief_width, relief_height)+ .translate((max_radius - relief_depth * 0.55, 0.0, body_bottom + height * 0.55))+ .rotate((0.0, 0.0, 0.0), (0.0, 0.0, 1.0), relief.angle_deg)+ )+ shape = shape.cut(cutter)+ elif relief.style == "top_recess":+ cutter = (+ cq.Workplane("XY")+ .circle(relief_width * 0.5)+ .extrude(relief_depth)+ .translate((0.0, 0.0, body_bottom + height - relief_depth))+ )+ shape = shape.cut(cutter)+ else:+ cutter = (+ cq.Workplane("XY")+ .box(relief_width, relief_width * 0.24, relief_depth)+ .translate((0.0, 0.0, body_bottom + height - relief_depth * 0.5))+ .rotate((0.0, 0.0, 0.0), (0.0, 0.0, 1.0), relief.angle_deg)+ )+ shape = shape.cut(cutter)++ solid = _to_shape(shape)+ if not center:+ bb = solid.BoundingBox()+ solid = solid.translate(cq.Vector(0.0, 0.0, -bb.zmin))+ self._set(solid)diff --git a/toolkits/curation/articraft/shim_parts/lathe_extrude_loft.py b/toolkits/curation/articraft/shim_parts/lathe_extrude_loft.pynew file mode 100644index 00000000..6e806a20--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/lathe_extrude_loft.py@@ -0,0 +1,324 @@+from articraft_cq import (cq, math, MeshGeometry, _to_shape, _revolve_profile,+ _extrude_profile, _loft_rings, _profile_points_2d, _polyline_points_2d,+ _ensure_ccw, _profile_points_3d, _polygon_centroid, _point_in_polygon,+ _points_match_2d, _EPS, ValidationError)++# ===========================================================================+# @section geom-lathe-extrude-loft (LatheGeometry / LoftGeometry /+# ExtrudeGeometry / ExtrudeWithHolesGeometry — clean-room CadQuery B-rep+# reimplementation of the Apache-2.0 Articraft sdk primitives.)+#+# Pure-python helpers below (_lathe_extrude_loft_tuple_distance, sample_cubic_bezier_spline_2d,+# _lathe_cap_connector, _lathe_shell_profile) are copied verbatim from the+# Apache-2.0 Articraft sdk `_mesh/common.py`.+# ===========================================================================++LatheCapMode = str # Literal["flat", "round"]+++def _lathe_extrude_loft_tuple_distance(a, b):+ return math.sqrt(sum((float(xa) - float(xb)) * (float(xa) - float(xb))+ for xa, xb in zip(a, b)))+++def sample_cubic_bezier_spline_2d(control_points, *, samples_per_segment=12):+ """Sample a C1-continuous cubic Bezier spline in 2D (verbatim from sdk)."""+ pts = [(float(x), float(y)) for (x, y) in control_points]+ if len(pts) < 4:+ raise ValidationError("Bezier spline requires at least 4 control points")+ if (len(pts) - 1) % 3 != 0:+ raise ValidationError("Bezier spline control points must satisfy (n-1) % 3 == 0")++ seg_count = (len(pts) - 1) // 3+ steps = max(2, int(samples_per_segment))++ def eval_seg(p0, p1, p2, p3, t):+ u = 1.0 - t+ b0 = u * u * u+ b1 = 3.0 * u * u * t+ b2 = 3.0 * u * t * t+ b3 = t * t * t+ x = b0 * p0[0] + b1 * p1[0] + b2 * p2[0] + b3 * p3[0]+ y = b0 * p0[1] + b1 * p1[1] + b2 * p2[1] + b3 * p3[1]+ return (x, y)++ out = []+ for seg in range(seg_count):+ i = seg * 3+ p0, p1, p2, p3 = pts[i], pts[i + 1], pts[i + 2], pts[i + 3]+ for j in range(steps):+ if seg > 0 and j == 0:+ continue+ t = float(j) / float(steps)+ out.append(eval_seg(p0, p1, p2, p3, t))+ out.append(pts[-1])+ return out+++def _lathe_cap_connector(start, end, *, start_tangent, end_tangent, mode, samples):+ """Build a flat or round lip connector between two (r,z) points (verbatim)."""+ if _points_match_2d(start, end):+ return [start]+ if mode == "flat":+ return [start, end]+ if mode != "round":+ raise ValidationError("Lathe cap mode must be 'flat' or 'round'")++ chord_dx = end[0] - start[0]+ chord_dz = end[1] - start[1]+ chord = math.sqrt(chord_dx * chord_dx + chord_dz * chord_dz)+ if chord <= _EPS:+ return [start]++ start_len = math.sqrt(start_tangent[0] * start_tangent[0] + start_tangent[1] * start_tangent[1])+ end_len = math.sqrt(end_tangent[0] * end_tangent[0] + end_tangent[1] * end_tangent[1])+ if start_len <= _EPS or end_len <= _EPS:+ return [start, end]++ handle = min(chord * 0.75, start_len * 0.5, end_len * 0.5)+ if handle <= _EPS:+ return [start, end]++ p1 = (+ start[0] + start_tangent[0] * (handle / start_len),+ start[1] + start_tangent[1] * (handle / start_len),+ )+ p2 = (+ end[0] - end_tangent[0] * (handle / end_len),+ end[1] - end_tangent[1] * (handle / end_len),+ )+ return sample_cubic_bezier_spline_2d(+ [start, p1, p2, end],+ samples_per_segment=max(3, int(samples)),+ )+++def _lathe_shell_profile(outer_profile, inner_profile, *, start_cap, end_cap, lip_samples):+ """Build a closed (r,z) shell cross-section from outer+inner walls (verbatim)."""+ outer = _polyline_points_2d(outer_profile)+ inner = _polyline_points_2d(inner_profile)++ same_alignment = _lathe_extrude_loft_tuple_distance(outer[0], inner[0]) + _lathe_extrude_loft_tuple_distance(outer[-1], inner[-1])+ flipped_alignment = _lathe_extrude_loft_tuple_distance(outer[0], inner[-1]) + _lathe_extrude_loft_tuple_distance(outer[-1], inner[0])+ if flipped_alignment < same_alignment:+ inner = list(reversed(inner))++ end_connector = _lathe_cap_connector(+ outer[-1],+ inner[-1],+ start_tangent=(outer[-1][0] - outer[-2][0], outer[-1][1] - outer[-2][1]),+ end_tangent=(inner[-2][0] - inner[-1][0], inner[-2][1] - inner[-1][1]),+ mode=end_cap,+ samples=lip_samples,+ )+ start_connector = _lathe_cap_connector(+ inner[0],+ outer[0],+ start_tangent=(inner[0][0] - inner[1][0], inner[0][1] - inner[1][1]),+ end_tangent=(outer[1][0] - outer[0][0], outer[1][1] - outer[0][1]),+ mode=start_cap,+ samples=lip_samples,+ )++ profile = list(outer)+ profile.extend(end_connector[1:])+ profile.extend(reversed(inner[:-1]))+ profile.extend(start_connector[1:])+ return profile+++# ---- cadquery loft side-wall helper (open shells, cap=False path) ----+def _loft_side_faces(rings, closed):+ """Ruled side-wall faces between consecutive 3D rings (matched point counts).+ Each quad is split into two planar triangles, matching the sdk mesh exactly."""+ faces = []+ ring_count = len(rings[0])+ seg = ring_count if closed else ring_count - 1+ for i in range(len(rings) - 1):+ a_ring = rings[i]+ b_ring = rings[i + 1]+ for j in range(seg):+ j2 = (j + 1) % ring_count+ quad = (a_ring[j], a_ring[j2], b_ring[j2], b_ring[j])+ for tri in ((quad[0], quad[1], quad[2]), (quad[0], quad[2], quad[3])):+ if (tri[0] == tri[1] or tri[1] == tri[2] or tri[2] == tri[0]):+ continue+ try:+ w = cq.Wire.makePolygon([cq.Vector(*p) for p in tri], close=True)+ faces.append(cq.Face.makeFromWires(w))+ except Exception:+ continue+ return faces+++def _xy_area(ring):+ a = 0.0+ n = len(ring)+ for i in range(n):+ x1, y1 = ring[i][0], ring[i][1]+ x2, y2 = ring[(i + 1) % n][0], ring[(i + 1) % n][1]+ a += x1 * y2 - x2 * y1+ return a * 0.5+++def _loft_solid(rings, *, cap, closed, ruled=True):+ """Loft through a list of 3D rings. cap+closed -> solid via makeLoft;+ otherwise an open side-wall shell. Rings are oriented consistently first."""+ rings = [list(r) for r in rings]+ if len(rings) < 2:+ raise ValidationError("Loft requires at least two profiles")+ base = _xy_area(rings[0])+ if abs(base) > 1e-9:+ if base < 0:+ rings = [list(reversed(r)) for r in rings]+ base = -base+ for idx in range(len(rings)):+ if _xy_area(rings[idx]) * base < 0:+ rings[idx] = list(reversed(rings[idx]))+ if cap and closed:+ wires = [cq.Wire.makePolygon([cq.Vector(*p) for p in r], close=True) for r in rings]+ return cq.Solid.makeLoft(wires, ruled)+ faces = _loft_side_faces(rings, closed)+ if not faces:+ raise ValidationError("Loft produced no valid side faces")+ return cq.Shell.makeShell(faces)+++# ===========================================================================+class LatheGeometry(MeshGeometry):+ """Revolve a closed (r,z) profile 360 deg about the Z axis: a profile point+ (r, z) maps to a circle of radius r at height z."""++ @classmethod+ def from_shell_profiles(cls, outer_profile, inner_profile, *, segments=32,+ start_cap="flat", end_cap="flat", lip_samples=6):+ profile = _lathe_shell_profile(+ outer_profile, inner_profile,+ start_cap=start_cap, end_cap=end_cap, lip_samples=lip_samples,+ )+ return cls(profile, segments=segments, closed=True)++ def __init__(self, profile, *, segments=32, closed=True):+ super().__init__()+ segments = max(3, int(segments)) # noqa: F841 (exact B-rep circle; mesh resolution N/A)+ points = _profile_points_2d(profile) if closed else _polyline_points_2d(profile)+ if closed:+ points = _ensure_ccw(points)++ normalized = []+ for r, z in points:+ if r < -_EPS:+ raise ValidationError("Lathe profile radii must be non-negative")+ normalized.append((0.0 if abs(r) <= _EPS else float(r), float(z)))++ self._set(_revolve_profile(normalized))+++# ===========================================================================+class LoftGeometry(MeshGeometry):+ """Loft through a sequence of closed 3D profiles (each a list of (x,y,z))."""++ def __init__(self, profiles, *, cap=True, closed=True):+ super().__init__()+ profile_list = [_profile_points_3d(p) for p in profiles]+ if len(profile_list) < 2:+ raise ValidationError("Loft requires at least two profiles")++ ring_count = len(profile_list[0])+ if closed and ring_count < 3:+ raise ValidationError("Closed loft profiles must have at least 3 points")+ for profile in profile_list:+ if len(profile) != ring_count:+ raise ValidationError("Loft profiles must have the same point count")++ self._set(_loft_solid(profile_list, cap=cap, closed=closed))+++# ===========================================================================+class ExtrudeGeometry(MeshGeometry):+ """Extrude a 2D profile by `height` along +Z. center=True -> z in+ [-h/2, h/2]; center=False -> z in [0, h]."""++ @classmethod+ def centered(cls, profile, height, *, cap=True, closed=True):+ return cls(profile, height, cap=cap, center=True, closed=closed)++ @classmethod+ def from_z0(cls, profile, height, *, cap=True, closed=True):+ return cls(profile, height, cap=cap, center=False, closed=closed)++ def __init__(self, profile, height, *, cap=True, center=True, closed=True):+ super().__init__()+ height = float(height)+ if height <= 0:+ raise ValidationError("Extrude height must be positive")++ points = _ensure_ccw(_profile_points_2d(profile))+ if not closed:+ cap = False++ if cap:+ self._set(_extrude_profile(points, height, center=center))+ return++ z0 = -height / 2.0 if center else 0.0+ z1 = z0 + height+ rings = [[(x, y, z0) for (x, y) in points], [(x, y, z1) for (x, y) in points]]+ faces = _loft_side_faces(rings, closed)+ if not faces:+ raise ValidationError("Extrude produced no valid side faces")+ self._set(cq.Shell.makeShell(faces))+++# ===========================================================================+class ExtrudeWithHolesGeometry(MeshGeometry):+ """Extrude a 2D outer profile minus one or more through-holes."""++ def __init__(self, outer_profile, hole_profiles, height, *, cap=True,+ center=True, closed=True):+ super().__init__()+ height = float(height)+ if height <= 0:+ raise ValidationError("Extrude height must be positive")++ outer = _ensure_ccw(_profile_points_2d(outer_profile))+ holes = [_ensure_ccw(_profile_points_2d(h)) for h in hole_profiles]+ if not closed:+ cap = False++ for hole in holes:+ c = _polygon_centroid(hole)+ if not _point_in_polygon(c, outer):+ raise ValidationError("Hole profile must lie inside outer profile")++ if not holes:+ self._set(_to_shape(ExtrudeGeometry(outer, height, cap=cap,+ center=center, closed=closed)))+ return++ z0 = -height / 2.0 if center else 0.0+ z1 = z0 + height++ if cap:+ outer_wire = cq.Wire.makePolygon(+ [cq.Vector(x, y, 0.0) for (x, y) in outer], close=True)+ hole_wires = [+ cq.Wire.makePolygon([cq.Vector(x, y, 0.0) for (x, y) in h], close=True)+ for h in holes+ ]+ solid = cq.Solid.extrudeLinear(outer_wire, hole_wires, cq.Vector(0, 0, height))+ if z0:+ solid = solid.translate(cq.Vector(0, 0, z0))+ self._set(solid)+ return++ # Open shell: outer side walls + interior (reversed) hole side walls.+ faces = _loft_side_faces(+ [[(x, y, z0) for (x, y) in outer], [(x, y, z1) for (x, y) in outer]], closed)+ for hole in holes:+ hole_cw = list(reversed(hole))+ faces += _loft_side_faces(+ [[(x, y, z0) for (x, y) in hole_cw], [(x, y, z1) for (x, y) in hole_cw]], closed)+ if not faces:+ raise ValidationError("ExtrudeWithHoles produced no valid side faces")+ self._set(cq.Shell.makeShell(faces))diff --git a/toolkits/curation/articraft/shim_parts/panels_vents.py b/toolkits/curation/articraft/shim_parts/panels_vents.pynew file mode 100644index 00000000..43bcee93--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/panels_vents.py@@ -0,0 +1,536 @@+from articraft_cq import (cq, math, MeshGeometry, _to_shape, ValidationError)++from dataclasses import dataclass+from typing import Optional, Sequence, Union++# ===========================================================================+# @section geom-panels-vents (Perforated / Slot / VentGrille panels + specs)+#+# Clean-room CadQuery B-rep reimplementation of the Articraft sdk+# panels_and_grilles family. Constructor signatures mirror the real sdk+# exactly; the produced shapes are exact parametric B-rep solids (built with+# the same cadquery operations the sdk uses, kept as B-rep rather than+# re-tessellated into a triangle mesh).+# ===========================================================================+++# ---- copied sdk helpers (verbatim semantics) -----------------------------+def _normalize_pitch_2d(pitch, *, name="pitch"):+ """Accept a positive scalar or a positive 2-sequence -> (pitch_x, pitch_y)."""+ if isinstance(pitch, Sequence) and not isinstance(pitch, (str, bytes)):+ values = list(pitch)+ if len(values) != 2:+ raise ValidationError("%s must be a positive scalar or a 2-tuple" % name)+ pitch_x = float(values[0])+ pitch_y = float(values[1])+ else:+ pitch_x = float(pitch)+ pitch_y = float(pitch)+ if pitch_x <= 0.0 or pitch_y <= 0.0:+ raise ValidationError("%s values must be positive" % name)+ return (pitch_x, pitch_y)+++def _centered_axis_positions(limit, pitch):+ """Symmetric positions in [-limit, +limit] spaced by `pitch`, centered on 0."""+ if limit < -1e-9:+ return []+ if limit <= 1e-9:+ return [0.0]+ count = int((2.0 * limit) / pitch) + 1+ if count <= 0:+ return []+ return [((index - (count - 1) * 0.5) * pitch) for index in range(count)]+++def _panels_vents_finish(self, shape, center):+ """Coerce the (Workplane/Shape) result to a solid, optionally drop its+ rear-most face to z=0 (center=False), and store it on the geometry."""+ solid = _to_shape(shape)+ if not center:+ bb = solid.BoundingBox()+ if abs(bb.zmin) > 1e-12:+ solid = solid.translate(cq.Vector(0.0, 0.0, -bb.zmin))+ self._set(solid)+++# ===========================================================================+# @section geom-panels-specs (frozen dataclass feature specs)+# ===========================================================================+@dataclass(frozen=True)+class BoltPattern:+ count: int+ circle_diameter: float+ hole_diameter: float+ countersink: float = 0.0+++@dataclass(frozen=True)+class VentGrilleSlats:+ profile: str = "flat"+ direction: str = "down"+ inset: float = 0.0+ divider_count: int = 0+ divider_width: float = 0.004+++@dataclass(frozen=True)+class VentGrilleFrame:+ style: str = "flush"+ depth: float = 0.0+++@dataclass(frozen=True)+class VentGrilleMounts:+ style: str = "none"+ inset: float = 0.008+ hole_diameter: Optional[float] = None+++@dataclass(frozen=True)+class VentGrilleSleeve:+ style: str = "full"+ depth: Optional[float] = None+ wall: Optional[float] = None+++# ===========================================================================+# @section geom-perforated-panel+# ===========================================================================+class PerforatedPanelGeometry(MeshGeometry):+ """Rectangular plate (XY, extruded along Z) with a grid of round holes."""++ def __init__(+ self,+ panel_size,+ thickness,+ *,+ hole_diameter,+ pitch,+ frame=0.008,+ corner_radius=0.0,+ stagger=False,+ center=True,+ ):+ super().__init__()+ panel_w = float(panel_size[0])+ panel_h = float(panel_size[1])+ thickness = float(thickness)+ hole_diameter = float(hole_diameter)+ frame = float(frame)+ corner_radius = max(0.0, float(corner_radius))+ pitch_x, pitch_y = _normalize_pitch_2d(pitch)++ if panel_w <= 0.0 or panel_h <= 0.0 or thickness <= 0.0:+ raise ValidationError("panel_size and thickness must be positive")+ if hole_diameter <= 0.0:+ raise ValidationError("hole_diameter must be positive")+ if frame < 0.0 or frame >= min(panel_w, panel_h) * 0.5:+ raise ValidationError("frame must be >= 0 and less than half of min(panel_size)")+ if pitch_x <= hole_diameter or pitch_y <= hole_diameter:+ raise ValidationError("pitch must be greater than hole_diameter on both axes")++ hole_radius = hole_diameter * 0.5+ x_limit = panel_w * 0.5 - frame - hole_radius+ y_limit = panel_h * 0.5 - frame - hole_radius+ if x_limit < -1e-9 or y_limit < -1e-9:+ raise ValidationError("frame/hole_diameter leave no usable perforation area")++ y_positions = _centered_axis_positions(y_limit, pitch_y)+ if not y_positions:+ raise ValidationError("No perforation rows fit panel; increase panel size or reduce pitch")++ point_rows = []+ for row_index, y in enumerate(y_positions):+ x_offset = pitch_x * 0.5 if stagger and row_index % 2 == 1 else 0.0+ row_limit = x_limit - abs(x_offset)+ if row_limit < -1e-9:+ continue+ x_positions = _centered_axis_positions(row_limit, pitch_x)+ row_points = [(x + x_offset, y) for x in x_positions]+ if row_points:+ point_rows.append(row_points)+ if not point_rows:+ raise ValidationError("No perforation columns fit panel; increase panel size or reduce pitch")++ shape = cq.Workplane("XY").box(panel_w, panel_h, thickness)+ if corner_radius > 0.0:+ shape = shape.edges("|Z").fillet(+ min(corner_radius, panel_w * 0.5 - 1e-4, panel_h * 0.5 - 1e-4)+ )++ cut_depth = thickness + max(0.002, thickness * 0.5)+ cut_shape = None+ for row_points in point_rows:+ row_cut = (+ cq.Workplane("XY")+ .pushPoints(row_points)+ .circle(hole_radius)+ .extrude(cut_depth, both=True)+ )+ cut_shape = row_cut if cut_shape is None else cut_shape.union(row_cut)+ if cut_shape is not None:+ shape = shape.cut(cut_shape)++ _panels_vents_finish(self, shape, center)+++# ===========================================================================+# @section geom-slot-panel+# ===========================================================================+class SlotPatternPanelGeometry(MeshGeometry):+ """Rectangular plate (XY, extruded along Z) with a grid of rounded slots."""++ def __init__(+ self,+ panel_size,+ thickness,+ *,+ slot_size,+ pitch,+ frame=0.008,+ corner_radius=0.0,+ slot_angle_deg=0.0,+ stagger=False,+ center=True,+ ):+ super().__init__()+ panel_w = float(panel_size[0])+ panel_h = float(panel_size[1])+ thickness = float(thickness)+ slot_length = float(slot_size[0])+ slot_width = float(slot_size[1])+ frame = float(frame)+ corner_radius = max(0.0, float(corner_radius))+ slot_angle_deg = float(slot_angle_deg)+ pitch_x, pitch_y = _normalize_pitch_2d(pitch)++ if panel_w <= 0.0 or panel_h <= 0.0 or thickness <= 0.0:+ raise ValidationError("panel_size and thickness must be positive")+ if slot_length <= 0.0 or slot_width <= 0.0:+ raise ValidationError("slot_size values must be positive")+ if slot_length < slot_width:+ raise ValidationError("slot_size[0] must be greater than or equal to slot_size[1]")+ if frame < 0.0 or frame >= min(panel_w, panel_h) * 0.5:+ raise ValidationError("frame must be >= 0 and less than half of min(panel_size)")+ if abs(slot_angle_deg) >= 90.0:+ raise ValidationError("abs(slot_angle_deg) must be < 90")++ slot_angle_rad = slot_angle_deg * math.pi / 180.0+ slot_half_x = 0.5 * (+ abs(slot_length * math.cos(slot_angle_rad)) + abs(slot_width * math.sin(slot_angle_rad))+ )+ slot_half_y = 0.5 * (+ abs(slot_length * math.sin(slot_angle_rad)) + abs(slot_width * math.cos(slot_angle_rad))+ )+ if pitch_x <= 2.0 * slot_half_x or pitch_y <= 2.0 * slot_half_y:+ raise ValidationError("pitch must be greater than the rotated slot envelope on both axes")++ x_limit = panel_w * 0.5 - frame - slot_half_x+ y_limit = panel_h * 0.5 - frame - slot_half_y+ if x_limit < -1e-9 or y_limit < -1e-9:+ raise ValidationError("frame/slot_size leave no usable slot area")++ y_positions = _centered_axis_positions(y_limit, pitch_y)+ if not y_positions:+ raise ValidationError("No slot rows fit panel; increase panel size or reduce pitch")++ point_rows = []+ for row_index, y in enumerate(y_positions):+ x_offset = pitch_x * 0.5 if stagger and row_index % 2 == 1 else 0.0+ row_limit = x_limit - abs(x_offset)+ if row_limit < -1e-9:+ continue+ x_positions = _centered_axis_positions(row_limit, pitch_x)+ row_points = [(x + x_offset, y) for x in x_positions]+ if row_points:+ point_rows.append(row_points)+ if not point_rows:+ raise ValidationError("No slot columns fit panel; increase panel size or reduce pitch")++ shape = cq.Workplane("XY").box(panel_w, panel_h, thickness)+ if corner_radius > 0.0:+ shape = shape.edges("|Z").fillet(+ min(corner_radius, panel_w * 0.5 - 1e-4, panel_h * 0.5 - 1e-4)+ )++ cut_depth = thickness + max(0.002, thickness * 0.5)+ slot_core_length = max(slot_length - slot_width, 0.0)++ def build_slot_cut(center_xy):+ slot_cut = None+ if slot_core_length > 1e-6:+ slot_cut = cq.Workplane("XY").box(slot_core_length, slot_width, cut_depth)+ cap_radius = slot_width * 0.5+ cap_offset = slot_core_length * 0.5+ left_cap = (+ cq.Workplane("XY")+ .circle(cap_radius)+ .extrude(cut_depth, both=True)+ .translate((-cap_offset, 0.0, 0.0))+ )+ right_cap = (+ cq.Workplane("XY")+ .circle(cap_radius)+ .extrude(cut_depth, both=True)+ .translate((cap_offset, 0.0, 0.0))+ )+ slot_cut = (+ left_cap.union(right_cap)+ if slot_cut is None+ else slot_cut.union(left_cap).union(right_cap)+ )+ slot_cut = slot_cut.rotate((0.0, 0.0, 0.0), (0.0, 0.0, 1.0), slot_angle_deg)+ return slot_cut.translate((center_xy[0], center_xy[1], 0.0))++ cut_shape = None+ for row_points in point_rows:+ for point in row_points:+ slot_cut = build_slot_cut(point)+ cut_shape = slot_cut if cut_shape is None else cut_shape.union(slot_cut)+ if cut_shape is not None:+ shape = shape.cut(cut_shape)++ _panels_vents_finish(self, shape, center)+++# ===========================================================================+# @section geom-vent-grille+# ===========================================================================+class VentGrilleGeometry(MeshGeometry):+ """Framed vent/register face (XY, extruded along Z) with real slats and an+ optional shallow rear sleeve. The front flange is centered on local z=0 and+ the sleeve extends toward -Z."""++ def __init__(+ self,+ panel_size,+ *,+ frame=0.012,+ face_thickness=0.004,+ duct_depth=0.026,+ duct_wall=0.003,+ slat_pitch=0.018,+ slat_width=0.009,+ slat_angle_deg=35.0,+ slat_thickness=None,+ corner_radius=0.0,+ slats=None,+ frame_profile=None,+ mounts=None,+ sleeve=None,+ center=True,+ ):+ super().__init__()+ panel_w = float(panel_size[0])+ panel_h = float(panel_size[1])+ frame = float(frame)+ face_t = float(face_thickness)+ duct_depth = float(duct_depth)+ duct_wall = float(duct_wall)+ slat_pitch = float(slat_pitch)+ slat_w = float(slat_width)+ slat_t = float(slat_thickness) if slat_thickness is not None else max(0.001, face_t * 0.35)+ corner_radius = max(0.0, float(corner_radius))+ slats = slats or VentGrilleSlats()+ frame_profile = frame_profile or VentGrilleFrame()+ mounts = mounts or VentGrilleMounts()+ sleeve = sleeve or VentGrilleSleeve()++ if panel_w <= 0 or panel_h <= 0:+ raise ValidationError("panel_size must be positive")+ if face_t <= 0 or duct_depth <= 0 or duct_wall <= 0:+ raise ValidationError("face_thickness, duct_depth, and duct_wall must be positive")+ if frame <= 0 or frame >= min(panel_w, panel_h) * 0.5:+ raise ValidationError("frame must be > 0 and less than half of min(panel_size)")+ if slat_pitch <= 0 or slat_w <= 0 or slat_t <= 0:+ raise ValidationError("slat_pitch, slat_width, and slat_thickness must be positive")+ if slat_pitch <= slat_w:+ raise ValidationError("slat_pitch must be greater than slat_width")++ slat_profile = str(slats.profile)+ if slat_profile not in {"flat", "airfoil", "boxed"}:+ raise ValidationError("slats.profile must be one of flat, airfoil, or boxed")+ slat_direction = str(slats.direction)+ if slat_direction not in {"down", "up"}:+ raise ValidationError("slats.direction must be one of down or up")+ slat_inset = float(slats.inset)+ divider_count = int(slats.divider_count)+ divider_width = float(slats.divider_width)+ if slat_inset < 0.0:+ raise ValidationError("slats.inset must be non-negative")+ if divider_count < 0:+ raise ValidationError("slats.divider_count must be non-negative")+ if divider_count > 0 and divider_width <= 0.0:+ raise ValidationError("slats.divider_width must be positive when dividers are requested")++ frame_style = str(frame_profile.style)+ if frame_style not in {"flush", "beveled", "radiused"}:+ raise ValidationError("frame_profile.style must be one of flush, beveled, or radiused")+ frame_depth = float(frame_profile.depth)+ if frame_depth < 0.0:+ raise ValidationError("frame_profile.depth must be non-negative")++ mount_style = str(mounts.style)+ if mount_style not in {"none", "holes"}:+ raise ValidationError("mounts.style must be one of none or holes")+ mount_inset = float(mounts.inset)+ if mount_inset < 0.0:+ raise ValidationError("mounts.inset must be non-negative")+ mount_hole_diameter = (+ float(mounts.hole_diameter)+ if mounts.hole_diameter is not None+ else max(frame * 0.36, 0.0032)+ )+ if mount_style != "none" and mount_hole_diameter <= 0.0:+ raise ValidationError("mounts.hole_diameter must be positive when mounts are enabled")++ sleeve_style = str(sleeve.style)+ if sleeve_style not in {"none", "short", "full"}:+ raise ValidationError("sleeve.style must be one of none, short, or full")+ sleeve_depth = float(sleeve.depth) if sleeve.depth is not None else duct_depth+ sleeve_wall = float(sleeve.wall) if sleeve.wall is not None else duct_wall+ if sleeve_style == "short" and sleeve.depth is None:+ sleeve_depth = max(face_t * 1.8, duct_depth * 0.45)+ if sleeve_style == "none":+ sleeve_depth = 0.0+ if sleeve_style != "none" and sleeve_depth <= 0.0:+ raise ValidationError("sleeve.depth must be positive when sleeve.style is not none")+ if sleeve_style != "none" and sleeve_wall <= 0.0:+ raise ValidationError("sleeve.wall must be positive when sleeve.style is not none")++ opening_w = panel_w - 2.0 * frame+ opening_h = panel_h - 2.0 * frame+ if sleeve_style != "none" and (+ opening_w <= 2.0 * sleeve_wall or opening_h <= 2.0 * sleeve_wall+ ):+ raise ValidationError("frame/duct_wall leave no open sleeve area")+ y = -opening_h * 0.5 + slat_pitch * 0.5+ limit = opening_h * 0.5 - slat_pitch * 0.5+ slat_rows = []+ while y <= limit + 1e-9:+ slat_rows.append(y)+ y += slat_pitch+ if not slat_rows:+ raise ValidationError("No slat rows fit panel; increase panel height or reduce slat_pitch")++ if divider_count > 0:+ usable_divider_span = opening_w - divider_width+ if usable_divider_span <= 0.0:+ raise ValidationError("slats.divider_width leaves no grille opening")+ divider_pitch = usable_divider_span / float(divider_count + 1)+ if divider_pitch <= divider_width:+ raise ValidationError("slats.divider_count/divider_width leave no usable openings")++ mount_positions = [+ (panel_w * 0.5 - mount_inset, panel_h * 0.5 - mount_inset),+ (-panel_w * 0.5 + mount_inset, panel_h * 0.5 - mount_inset),+ (panel_w * 0.5 - mount_inset, -panel_h * 0.5 + mount_inset),+ (-panel_w * 0.5 + mount_inset, -panel_h * 0.5 + mount_inset),+ ]+ if mount_style != "none":+ hole_radius = mount_hole_diameter * 0.5+ if hole_radius >= frame * 0.80:+ raise ValidationError("mounts.hole_diameter is too large for the frame width")+ for hole_x, hole_y in mount_positions:+ if (+ abs(hole_x) + hole_radius > panel_w * 0.5+ or abs(hole_y) + hole_radius > panel_h * 0.5+ ):+ raise ValidationError("mounts.inset places holes outside the face")++ shape = cq.Workplane("XY").box(panel_w, panel_h, face_t)+ if corner_radius > 0.0:+ shape = shape.edges("|Z").fillet(+ min(corner_radius, panel_w * 0.5 - frame, panel_h * 0.5 - frame)+ )+ if frame_style == "beveled" and frame_depth > 1.0e-9:+ shape = shape.edges(">Z").chamfer(min(frame_depth, frame * 0.7, face_t * 0.7))+ elif frame_style == "radiused" and frame_depth > 1.0e-9:+ shape = shape.edges(">Z").fillet(min(frame_depth, frame * 0.7, face_t * 0.48))++ shape = shape.cut(+ cq.Workplane("XY").box(+ opening_w,+ opening_h,+ face_t + max(0.002, face_t * 0.5),+ )+ )++ if sleeve_style != "none":+ duct_outer = cq.Workplane("XY").box(opening_w, opening_h, sleeve_depth)+ duct_inner = cq.Workplane("XY").box(+ opening_w - 2.0 * sleeve_wall,+ opening_h - 2.0 * sleeve_wall,+ sleeve_depth + face_t + 0.004,+ )+ duct_shell = duct_outer.cut(duct_inner).translate(+ (0.0, 0.0, -face_t * 0.5 - sleeve_depth * 0.5 + min(face_t * 0.25, 0.001))+ )+ shape = shape.union(duct_shell)++ slat_embed = min(frame * 0.5, 0.002)+ slat_clear_w = opening_w - max(0.0, divider_count * divider_width)+ slat_chord = max(1.0e-4, slat_clear_w + 2.0 * slat_embed)+ slat_z = -face_t * 0.25 - slat_inset+ slat_angle = abs(float(slat_angle_deg)) * (-1.0 if slat_direction == "down" else 1.0)++ def _make_slat():+ if slat_profile == "flat":+ return cq.Workplane("XY").box(slat_chord, slat_w, slat_t)+ if slat_profile == "boxed":+ box_t = max(slat_t * 1.35, slat_w * 0.42)+ return cq.Workplane("XY").box(slat_chord, slat_w, box_t)++ section_points = [+ (-0.50 * slat_w, 0.00),+ (-0.22 * slat_w, 0.56 * slat_t),+ (0.08 * slat_w, 0.64 * slat_t),+ (0.44 * slat_w, 0.10 * slat_t),+ (0.34 * slat_w, -0.22 * slat_t),+ (-0.10 * slat_w, -0.38 * slat_t),+ (-0.44 * slat_w, -0.14 * slat_t),+ ]+ return (+ cq.Workplane("YZ")+ .workplane(offset=-slat_chord * 0.5)+ .polyline(section_points)+ .close()+ .extrude(slat_chord)+ )++ for row_y in slat_rows:+ slat = _make_slat()+ slat = slat.rotate((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), slat_angle)+ slat = slat.translate((0.0, row_y, slat_z))+ shape = shape.union(slat)++ if divider_count > 0:+ divider_positions = _centered_axis_positions(+ opening_w * 0.5 - divider_width * 0.5,+ (opening_w - divider_width) / float(divider_count + 1),+ )+ divider_positions = (+ divider_positions[1:-1]+ if len(divider_positions) > divider_count+ else divider_positions+ )+ divider_depth = max(face_t * 0.90, slat_t * 1.10)+ divider_span_y = opening_h + min(frame * 0.35, 0.003)+ for x_pos in divider_positions[:divider_count]:+ divider = cq.Workplane("XY").box(divider_width, divider_span_y, divider_depth)+ divider = divider.translate((x_pos, 0.0, -face_t * 0.18))+ shape = shape.union(divider)++ if mount_style != "none":+ hole_depth = face_t + max(sleeve_depth, 0.0) + 0.01+ for hole_x, hole_y in mount_positions:+ hole = (+ cq.Workplane("XY")+ .circle(mount_hole_diameter * 0.5)+ .extrude(hole_depth * 0.5, both=True)+ .translate((hole_x, hole_y, -max(sleeve_depth, 0.0) * 0.5))+ )+ shape = shape.cut(hole)++ _panels_vents_finish(self, shape, center)diff --git a/toolkits/curation/articraft/shim_parts/primitives.py b/toolkits/curation/articraft/shim_parts/primitives.pynew file mode 100644index 00000000..e3ad0e04--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/primitives.py@@ -0,0 +1,124 @@+from articraft_cq import (cq, math, MeshGeometry, _to_shape, _revolve_profile,+ _extrude_profile, _loft_rings, ValidationError, Box, Cylinder, Sphere)++# ===========================================================================+# @section geom-primitives (Box/Cylinder/Cone/Sphere/Torus/Capsule/Dome)+#+# Clean-room CadQuery B-rep reimplementation of the Articraft sdk primitive+# MeshGeometry classes. Constructor signatures mirror the real sdk exactly;+# the produced shapes are smooth parametric solids (not triangle meshes).+# ===========================================================================+++def _coerce_positive_radii3(value, *, name):+ """Verbatim port of the sdk helper: accept a positive scalar or a positive+ 3-sequence and return an (rx, ry, rz) tuple."""+ if isinstance(value, (int, float)):+ radius = float(value)+ if radius <= 0.0:+ raise ValidationError("%s must be positive" % name)+ return (radius, radius, radius)+ if len(value) != 3:+ raise ValidationError("%s must be a positive float or a 3-sequence" % name)+ rx = float(value[0])+ ry = float(value[1])+ rz = float(value[2])+ if rx <= 0.0 or ry <= 0.0 or rz <= 0.0:+ raise ValidationError("%s values must be positive" % name)+ return (rx, ry, rz)+++class BoxGeometry(MeshGeometry):+ def __init__(self, size):+ super().__init__()+ x, y, z = (float(size[0]), float(size[1]), float(size[2]))+ solid = cq.Solid.makeBox(x, y, z, cq.Vector(-x / 2.0, -y / 2.0, -z / 2.0))+ self._set(solid)+++class CylinderGeometry(MeshGeometry):+ def __init__(self, radius, height, *, radial_segments=24, closed=True):+ super().__init__()+ radius = float(radius)+ height = float(height)+ solid = cq.Solid.makeCylinder(+ radius, height, cq.Vector(0, 0, -height / 2.0), cq.Vector(0, 0, 1)+ )+ self._set(solid)+++class ConeGeometry(MeshGeometry):+ def __init__(self, radius, height, *, radial_segments=24, closed=True):+ super().__init__()+ radius = float(radius)+ height = float(height)+ # apex at +height/2 (radius 0), base radius `radius` at -height/2+ solid = cq.Solid.makeCone(+ radius, 0.0, height, cq.Vector(0, 0, -height / 2.0), cq.Vector(0, 0, 1)+ )+ self._set(solid)+++class SphereGeometry(MeshGeometry):+ def __init__(self, radius, *, width_segments=24, height_segments=16):+ super().__init__()+ radius = float(radius)+ solid = cq.Solid.makeSphere(+ radius, angleDegrees1=-90, angleDegrees2=90, angleDegrees3=360+ )+ self._set(solid)+++class TorusGeometry(MeshGeometry):+ def __init__(self, radius, tube, *, radial_segments=16, tubular_segments=32):+ super().__init__()+ radius = float(radius)+ tube = float(tube)+ # ring radius `radius` in XY plane, tube radius `tube`, centered at origin+ solid = cq.Solid.makeTorus(radius, tube, cq.Vector(0, 0, 0), cq.Vector(0, 0, 1))+ self._set(solid)+++class CapsuleGeometry(MeshGeometry):+ def __init__(self, radius, length, *, radial_segments=24, height_segments=8):+ super().__init__()+ radius = float(radius)+ length = float(length)+ if radius <= 0.0:+ raise ValidationError("radius must be positive")+ if length < 0.0:+ raise ValidationError("length must be non-negative")++ sphere = cq.Solid.makeSphere(+ radius, angleDegrees1=-90, angleDegrees2=90, angleDegrees3=360+ )+ if length <= 1e-9:+ self._set(sphere)+ return++ half = length / 2.0+ # cylindrical body of height `length` centered, hemispherical caps at +-half+ body = cq.Solid.makeCylinder(+ radius, length, cq.Vector(0, 0, -half), cq.Vector(0, 0, 1)+ )+ top = sphere.translate(cq.Vector(0, 0, half))+ bottom = sphere.translate(cq.Vector(0, 0, -half))+ solid = body.fuse(top, bottom)+ try:+ solid = solid.clean()+ except Exception:+ pass+ self._set(solid)+++class DomeGeometry(MeshGeometry):+ def __init__(self, radius, *, radial_segments=24, height_segments=12, closed=True):+ super().__init__()+ rx, ry, rz = _coerce_positive_radii3(radius, name="radius")+ # upper hemisphere (z>=0) of a unit sphere, scaled to the half-ellipsoid+ hemi = cq.Solid.makeSphere(+ 1.0, angleDegrees1=0, angleDegrees2=90, angleDegrees3=360+ )+ m = cq.Matrix([[rx, 0, 0, 0], [0, ry, 0, 0], [0, 0, rz, 0]])+ solid = hemi.transformGeometry(m)+ self._set(solid)diff --git a/toolkits/curation/articraft/shim_parts/section_loft.py b/toolkits/curation/articraft/shim_parts/section_loft.pynew file mode 100644index 00000000..7c3c3318--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/section_loft.py@@ -0,0 +1,377 @@+from articraft_cq import (cq, math, MeshGeometry, _CQMesh, _to_shape,+ mesh_from_geometry, _loft_rings, _profile_points_3d, ValidationError)++from dataclasses import dataclass, field++# ===========================================================================+# @section geom-section-loft (section_loft / SectionLoftSpec / LoftSection /+# LoftTessellation / resample_side_sections / repair_loft)+#+# Clean-room CadQuery B-rep reimplementation of the Articraft sdk section-loft+# API. Public signatures mirror the real sdk exactly; instead of tessellating+# to a triangle mesh, the loft is returned as a parametric CadQuery B-rep+# wrapped in a MeshGeometry (so `mesh_from_geometry(...)` and the model+# epilogue see an exact shape). The real sdk already lofts with cadquery+# internally (cq.occ_impl.shapes.loft), so this mirrors that construction.+# ===========================================================================+++def _cq_shapes():+ """The low-level cadquery shape helpers the real sdk lofts through."""+ from cadquery.occ_impl import shapes as cq_shapes+ return cq_shapes+++# ---- validation / coercion helpers (verbatim from the Apache-2.0 sdk) ------+def _as_vec3(values, *, name):+ if len(values) != 3:+ raise ValidationError("%s must have 3 elements" % name)+ return (float(values[0]), float(values[1]), float(values[2]))+++def _normalize_section_points(points, *, name):+ raw = [_as_vec3(point, name=name) for point in points]+ if len(raw) < 3:+ raise ValidationError("%s must contain at least 3 points" % name)++ out = []+ for point in raw:+ if not out or point != out[-1]:+ out.append(point)++ if len(out) >= 2 and out[0] == out[-1]:+ out.pop()++ if len(out) < 3:+ raise ValidationError("%s must contain at least 3 distinct points" % name)+ return out+++# ===========================================================================+# Spec data types+# ===========================================================================+@dataclass(frozen=True)+class LoftTessellation:+ tolerance: float = 0.001+ angular_tolerance: float = 0.1++ def __post_init__(self):+ if float(self.tolerance) <= 0.0:+ raise ValidationError("tessellation.tolerance must be > 0")+ if float(self.angular_tolerance) <= 0.0:+ raise ValidationError("tessellation.angular_tolerance must be > 0")+ object.__setattr__(self, "tolerance", float(self.tolerance))+ object.__setattr__(self, "angular_tolerance", float(self.angular_tolerance))+++@dataclass(frozen=True)+class LoftSection:+ points: tuple++ def __post_init__(self):+ coerced = tuple(_normalize_section_points(self.points, name="section.points"))+ object.__setattr__(self, "points", coerced)+++@dataclass(frozen=True)+class SectionLoftSpec:+ sections: tuple+ path: tuple = None+ guide_curves: dict = None+ cap: bool = True+ solid: bool = True+ symmetry: str = None+ ruled: bool = False+ continuity: str = "C2"+ parametrization: str = "uniform"+ degree: int = 3+ compat: bool = True+ smoothing: bool = False+ weights: tuple = (1.0, 1.0, 1.0)+ repair: str = "auto"+ tessellation: LoftTessellation = field(default_factory=LoftTessellation)++ def __post_init__(self):+ sections = tuple(_coerce_loft_section(section) for section in self.sections)+ if len(sections) < 2:+ raise ValidationError("SectionLoftSpec.sections must contain at least two sections")+ object.__setattr__(self, "sections", sections)++ if self.path is not None:+ object.__setattr__(+ self, "path",+ tuple(_as_vec3(point, name="path[]") for point in self.path),+ )+ if self.guide_curves is not None:+ guide_curves = {+ str(name): tuple(_as_vec3(point, name="guide_curves[%r][]" % name) for point in pts)+ for name, pts in self.guide_curves.items()+ }+ object.__setattr__(self, "guide_curves", guide_curves)++ if self.symmetry is not None and self.symmetry != "mirror_yz":+ raise ValidationError("symmetry must be 'mirror_yz' or None")+ if self.degree < 1:+ raise ValidationError("degree must be >= 1")+ if len(self.weights) != 3:+ raise ValidationError("weights must contain exactly 3 values")+ object.__setattr__(self, "degree", int(self.degree))+ object.__setattr__(+ self, "weights",+ (float(self.weights[0]), float(self.weights[1]), float(self.weights[2])),+ )+++def _coerce_loft_section(value):+ if isinstance(value, LoftSection):+ return value+ return LoftSection(points=tuple(value))+++def _coerce_loft_spec(spec):+ if isinstance(spec, SectionLoftSpec):+ return spec+ return SectionLoftSpec(sections=tuple(_coerce_loft_section(section) for section in spec))+++# ===========================================================================+# CadQuery construction (mirrors the real sdk's cadquery internals)+# ===========================================================================+def _build_section_wires(spec):+ cq_shapes = _cq_shapes()+ wires = []+ for idx, section in enumerate(spec.sections):+ points = tuple(section.points)+ try:+ wire = cq_shapes.polygon(*points)+ except Exception as exc:+ raise ValidationError("Failed to build wire for section[%d]" % idx)+ wires.append(wire)+ return wires+++def _curve_from_points(points, *, name):+ cq_shapes = _cq_shapes()+ if len(points) < 2:+ raise ValidationError("%s must contain at least 2 points" % name)+ if len(points) == 2:+ return cq_shapes.segment(points[0], points[1])+ try:+ return cq_shapes.spline(*points)+ except Exception:+ try:+ return cq_shapes.polyline(*points)+ except Exception as exc:+ raise ValidationError("Failed to build %s" % name)+++def _resolve_path_points(spec):+ if spec.path is not None:+ return spec.path+ if spec.guide_curves is None:+ return None+ return spec.guide_curves.get("spine")+++def _resolve_aux_spine_points(spec):+ if spec.guide_curves is None:+ return None+ aux = spec.guide_curves.get("aux_spine")+ if aux is not None:+ return aux+ return spec.guide_curves.get("binormal")+++def _validate_guide_curve_names(spec):+ if spec.guide_curves is None:+ return+ supported = {"spine", "aux_spine", "binormal"}+ unsupported = sorted(set(spec.guide_curves) - supported)+ if unsupported:+ raise ValidationError(+ "Unsupported guide_curves keys: "+ + ", ".join(repr(name) for name in unsupported)+ + ". Supported keys are 'spine', 'aux_spine', and 'binormal'."+ )+++def _apply_symmetry_shape(shape, *, symmetry):+ if symmetry is None:+ return shape+ if symmetry != "mirror_yz":+ raise ValidationError("Unsupported symmetry mode: %r" % symmetry)+ try:+ mirrored = shape.mirror("YZ")+ if hasattr(shape, "fuse"):+ fused = shape.fuse(mirrored, glue=True)+ else:+ fused = shape+ if hasattr(fused, "fix"):+ fused = fused.fix()+ if hasattr(fused, "clean"):+ fused = fused.clean()+ return fused+ except Exception as exc:+ raise ValidationError("Failed to apply mirror_yz symmetry")+++def _heal_shape(shape, *, solid):+ cq_shapes = _cq_shapes()+ if hasattr(shape, "fix"):+ shape = shape.fix()+ if solid:+ try:+ shape = cq_shapes.solid(shape)+ except Exception:+ pass+ if hasattr(shape, "fix"):+ shape = shape.fix()+ return shape+++def section_loft(spec, /, **overrides):+ if overrides:+ spec = SectionLoftSpec(**({**_coerce_loft_spec(spec).__dict__, **overrides}))+ else:+ spec = _coerce_loft_spec(spec)++ cq_shapes = _cq_shapes()+ _validate_guide_curve_names(spec)+ wires = _build_section_wires(spec)+ path_points = _resolve_path_points(spec)+ aux_spine_points = _resolve_aux_spine_points(spec)++ if path_points is None:+ shape = cq_shapes.loft(+ wires,+ cap=bool(spec.cap and spec.solid),+ ruled=bool(spec.ruled),+ continuity=spec.continuity,+ parametrization=spec.parametrization,+ degree=int(spec.degree),+ compat=bool(spec.compat),+ smoothing=bool(spec.smoothing),+ weights=spec.weights,+ )+ else:+ path_curve = _curve_from_points(path_points, name="path")+ mode = None+ if aux_spine_points is not None:+ mode = _curve_from_points(aux_spine_points, name="guide_curves['aux_spine']")+ shape = cq_shapes.Solid.sweep_multi(+ wires,+ path_curve,+ makeSolid=bool(spec.solid and spec.cap),+ isFrenet=False,+ mode=mode,+ )++ shape = _apply_symmetry_shape(shape, symmetry=spec.symmetry)++ if spec.repair in {"auto", "kernel"}:+ shape = _heal_shape(shape, solid=bool(spec.solid and spec.cap))++ return MeshGeometry()._set(shape)+++def repair_loft(geometry_or_spec, /, *, repair="auto"):+ # B-rep results are already exact/manifold, so mesh-side repair is a no-op+ # copy; spec/raw inputs are rebuilt through section_loft (honoring `repair`).+ if isinstance(geometry_or_spec, MeshGeometry):+ return geometry_or_spec.copy()+ spec = _coerce_loft_spec(geometry_or_spec)+ if repair != "auto":+ spec = SectionLoftSpec(**{**spec.__dict__, "repair": repair})+ return section_loft(spec)+++# ===========================================================================+# resample_side_sections (pure-python helper, verbatim from sdk _mesh/common.py)+# ===========================================================================+def resample_side_sections(sections, *, samples_per_span=2, smooth_passes=0,+ min_height=1e-4, min_width=1e-4):+ """Densify and optionally smooth side-loft sections.++ Each section is ``(y, z_min, z_max, width)`` with the loft axis along +Y.+ Linearly resamples each span in Y, then applies a light 3-point smoothing+ kernel to ``z_min``, ``z_max`` and ``width`` while keeping ``y`` fixed and+ monotonic."""+ raw = [(float(y), float(z0), float(z1), float(w)) for (y, z0, z1, w) in sections]+ if len(raw) < 2:+ raise ValueError("resample_side_sections requires at least two sections")++ raw.sort(key=lambda row: row[0])++ # Merge duplicate y samples by averaging rails.+ merged = []+ for row in raw:+ if merged and abs(row[0] - merged[-1][0]) <= 1e-9:+ y0, a0, a1, aw = merged[-1]+ merged[-1] = (+ y0,+ 0.5 * (a0 + row[1]),+ 0.5 * (a1 + row[2]),+ 0.5 * (aw + row[3]),+ )+ else:+ merged.append(row)++ if len(merged) < 2:+ y0, z0, z1, w = merged[0]+ height = max(float(min_height), z1 - z0)+ return [(y0, z0, z0 + height, max(float(min_width), w))]++ spans = max(1, int(samples_per_span))++ def lerp(a, b, t):+ return a + (b - a) * t++ dense = []+ for idx in range(len(merged) - 1):+ a = merged[idx]+ b = merged[idx + 1]+ for j in range(spans):+ if idx > 0 and j == 0:+ continue+ t = float(j) / float(spans)+ dense.append(+ (+ lerp(a[0], b[0], t),+ lerp(a[1], b[1], t),+ lerp(a[2], b[2], t),+ lerp(a[3], b[3], t),+ )+ )+ dense.append(merged[-1])++ passes = max(0, int(smooth_passes))+ if passes > 0 and len(dense) >= 3:+ for _ in range(passes):+ smoothed = [dense[0]]+ for idx in range(1, len(dense) - 1):+ left = dense[idx - 1]+ cur = dense[idx]+ right = dense[idx + 1]+ smoothed.append(+ (+ cur[0],+ 0.25 * left[1] + 0.50 * cur[1] + 0.25 * right[1],+ 0.25 * left[2] + 0.50 * cur[2] + 0.25 * right[2],+ 0.25 * left[3] + 0.50 * cur[3] + 0.25 * right[3],+ )+ )+ smoothed.append(dense[-1])+ dense = smoothed++ min_h = max(1e-9, float(min_height))+ min_w = max(1e-9, float(min_width))+ out = []+ for y, z0, z1, w in dense:+ low = float(min(z0, z1))+ high = float(max(z0, z1))+ if high - low < min_h:+ center = 0.5 * (low + high)+ low = center - 0.5 * min_h+ high = center + 0.5 * min_h+ out.append((float(y), low, high, max(min_w, float(w))))+ return outdiff --git a/toolkits/curation/articraft/shim_parts/sweeps.py b/toolkits/curation/articraft/shim_parts/sweeps.pynew file mode 100644index 00000000..ffb3f121--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/sweeps.py@@ -0,0 +1,929 @@+from articraft_cq import (cq, math, MeshGeometry, _to_shape, superellipse_profile,+ _profile_points_2d, _ensure_ccw, ValidationError, _EPS)+from math import acos, atan2, cos, isfinite, pi, sin, sqrt, tan+from dataclasses import dataclass+from typing import Iterable, List, Optional, Sequence, Tuple, Union++# ===========================================================================+# @section geom-sweeps (tube/profile sweeps, wire paths, side-lofts)+#+# Clean-room CadQuery B-rep reimplementation of the Articraft sdk `sweeps`+# family. The pure-python spline samplers, wire-corner math and side-loft+# profile construction are copied verbatim from the Apache-2.0 sdk+# (_mesh/common.py, _mesh/sweeps.py) so sampling matches bit-for-bit; the+# only change is that swept/lofted cross-sections are stitched into a real+# parametric B-rep (ruled `cq.Solid.makeLoft`) instead of a triangle soup.+# ===========================================================================++Vec2 = Tuple[float, float]+Vec3 = Tuple[float, float, float]+++# ---------------------------------------------------------------------------+# vector helpers (verbatim from sdk _mesh/common.py)+# ---------------------------------------------------------------------------+def _v_add(a, b):+ return (a[0] + b[0], a[1] + b[1], a[2] + b[2])+++def _v_sub(a, b):+ return (a[0] - b[0], a[1] - b[1], a[2] - b[2])+++def _v_scale(v, s):+ return (v[0] * s, v[1] * s, v[2] * s)+++def _v_dot(a, b):+ return a[0] * b[0] + a[1] * b[1] + a[2] * b[2]+++def _v_cross(a, b):+ return (a[1] * b[2] - a[2] * b[1], a[2] * b[0] - a[0] * b[2], a[0] * b[1] - a[1] * b[0])+++def _v_norm(v):+ return sqrt(_v_dot(v, v))+++def _v_normalize(v):+ n = _v_norm(v)+ if n <= _EPS:+ raise ValueError("Cannot normalize near-zero vector")+ return (v[0] / n, v[1] / n, v[2] / n)+++def _v_rotate_rodrigues(v, axis, angle):+ k = _v_normalize(axis)+ c = cos(float(angle))+ s = sin(float(angle))+ term1 = _v_scale(v, c)+ term2 = _v_scale(_v_cross(k, v), s)+ term3 = _v_scale(k, _v_dot(k, v) * (1.0 - c))+ return _v_add(_v_add(term1, term2), term3)+++def _v_lerp(a, b, t):+ return (a[0] + (b[0] - a[0]) * t, a[1] + (b[1] - a[1]) * t, a[2] + (b[2] - a[2]) * t)+++def _sweeps_clamp(x, low, high):+ return max(low, min(high, x))+++# ---------------------------------------------------------------------------+# spline samplers (verbatim from sdk _mesh/common.py)+# ---------------------------------------------------------------------------+def sample_cubic_bezier_spline_2d(control_points, *, samples_per_segment: int = 12):+ """Sample a C1-continuous chained cubic Bezier spline in 2D.++ Control-point layout: ``[P0, P1, P2, P3, P4, P5, P6, ...]`` where each+ additional segment contributes three points and starts at the previous+ segment's end (so ``(n - 1) % 3 == 0``)."""+ pts = [(float(x), float(y)) for (x, y) in control_points]+ if len(pts) < 4:+ raise ValueError("Bezier spline requires at least 4 control points")+ if (len(pts) - 1) % 3 != 0:+ raise ValueError("Bezier spline control points must satisfy (n-1) % 3 == 0")++ seg_count = (len(pts) - 1) // 3+ steps = max(2, int(samples_per_segment))++ def eval_seg(p0, p1, p2, p3, t):+ u = 1.0 - t+ b0 = u * u * u+ b1 = 3.0 * u * u * t+ b2 = 3.0 * u * t * t+ b3 = t * t * t+ x = b0 * p0[0] + b1 * p1[0] + b2 * p2[0] + b3 * p3[0]+ y = b0 * p0[1] + b1 * p1[1] + b2 * p2[1] + b3 * p3[1]+ return (x, y)++ out = []+ for seg in range(seg_count):+ i = seg * 3+ p0, p1, p2, p3 = pts[i], pts[i + 1], pts[i + 2], pts[i + 3]+ for j in range(steps):+ if seg > 0 and j == 0:+ continue+ t = float(j) / float(steps)+ out.append(eval_seg(p0, p1, p2, p3, t))+ out.append(pts[-1])+ return out+++def sample_cubic_bezier_spline_3d(control_points, *, samples_per_segment: int = 12):+ """Sample a C1-continuous chained cubic Bezier spline in 3D."""+ pts = [(float(x), float(y), float(z)) for (x, y, z) in control_points]+ if len(pts) < 4:+ raise ValueError("Bezier spline requires at least 4 control points")+ if (len(pts) - 1) % 3 != 0:+ raise ValueError("Bezier spline control points must satisfy (n-1) % 3 == 0")++ seg_count = (len(pts) - 1) // 3+ steps = max(2, int(samples_per_segment))++ def eval_seg(p0, p1, p2, p3, t):+ u = 1.0 - t+ b0 = u * u * u+ b1 = 3.0 * u * u * t+ b2 = 3.0 * u * t * t+ b3 = t * t * t+ x = b0 * p0[0] + b1 * p1[0] + b2 * p2[0] + b3 * p3[0]+ y = b0 * p0[1] + b1 * p1[1] + b2 * p2[1] + b3 * p3[1]+ z = b0 * p0[2] + b1 * p1[2] + b2 * p2[2] + b3 * p3[2]+ return (x, y, z)++ out = []+ for seg in range(seg_count):+ i = seg * 3+ p0, p1, p2, p3 = pts[i], pts[i + 1], pts[i + 2], pts[i + 3]+ for j in range(steps):+ if seg > 0 and j == 0:+ continue+ t = float(j) / float(steps)+ out.append(eval_seg(p0, p1, p2, p3, t))+ out.append(pts[-1])+ return out+++def _tuple_lerp(a, b, t):+ return tuple(float(xa) + (float(xb) - float(xa)) * float(t) for xa, xb in zip(a, b))+++def _tuple_extrapolate(a, b):+ return tuple(2.0 * float(xa) - float(xb) for xa, xb in zip(a, b))+++def _sweeps_tuple_distance(a, b):+ return sqrt(sum((float(xa) - float(xb)) * (float(xa) - float(xb)) for xa, xb in zip(a, b)))+++def _catmull_rom_interp(a, b, ta, tb, t):+ if abs(tb - ta) <= _EPS:+ return tuple(float(v) for v in a)+ wa = (tb - t) / (tb - ta)+ wb = (t - ta) / (tb - ta)+ return tuple(wa * float(va) + wb * float(vb) for va, vb in zip(a, b))+++def _sample_catmull_rom_segment(p0, p1, p2, p3, *, steps, alpha):+ def tj(ti, pa, pb):+ dist = max(_sweeps_tuple_distance(pa, pb), _EPS)+ return ti + dist ** alpha++ t0 = 0.0+ t1 = tj(t0, p0, p1)+ t2 = tj(t1, p1, p2)+ t3 = tj(t2, p2, p3)++ out = []+ for j in range(steps):+ t = t1 + (t2 - t1) * (float(j) / float(steps))+ a1 = _catmull_rom_interp(p0, p1, t0, t1, t)+ a2 = _catmull_rom_interp(p1, p2, t1, t2, t)+ a3 = _catmull_rom_interp(p2, p3, t2, t3, t)+ b1 = _catmull_rom_interp(a1, a2, t0, t2, t)+ b2 = _catmull_rom_interp(a2, a3, t1, t3, t)+ out.append(_catmull_rom_interp(b1, b2, t1, t2, t))+ return out+++def _sample_catmull_rom_spline(points, *, samples_per_segment, closed, alpha):+ pts = [tuple(float(v) for v in point) for point in points]+ if len(pts) < 2:+ raise ValueError("Catmull-Rom spline requires at least two points")+ if not (0.0 <= float(alpha) <= 1.0):+ raise ValueError("Catmull-Rom alpha must be in [0, 1]")++ dedup = [pts[0]]+ for point in pts[1:]:+ if _sweeps_tuple_distance(point, dedup[-1]) > _EPS:+ dedup.append(point)++ if len(dedup) < 2:+ raise ValueError("Catmull-Rom spline requires at least two distinct points")++ steps = max(2, int(samples_per_segment))++ if closed:+ ring = list(dedup)+ if _sweeps_tuple_distance(ring[0], ring[-1]) <= _EPS:+ ring.pop()+ if len(ring) < 3:+ raise ValueError("Closed Catmull-Rom spline requires at least three distinct points")++ out = []+ n = len(ring)+ for i in range(n):+ seg = _sample_catmull_rom_segment(+ ring[(i - 1) % n], ring[i], ring[(i + 1) % n], ring[(i + 2) % n],+ steps=steps, alpha=float(alpha),+ )+ out.extend(seg)+ if not out:+ raise ValueError("Closed Catmull-Rom spline produced no samples")+ out.append(out[0])+ return out++ if len(dedup) == 2:+ out = [_tuple_lerp(dedup[0], dedup[1], float(j) / float(steps)) for j in range(steps)]+ out.append(dedup[-1])+ return out++ extended = [+ _tuple_extrapolate(dedup[0], dedup[1]),+ *dedup,+ _tuple_extrapolate(dedup[-1], dedup[-2]),+ ]+ out = []+ for i in range(len(dedup) - 1):+ seg = _sample_catmull_rom_segment(+ extended[i], extended[i + 1], extended[i + 2], extended[i + 3],+ steps=steps, alpha=float(alpha),+ )+ out.extend(seg)+ out.append(dedup[-1])+ return out+++def sample_catmull_rom_spline_2d(points, *, samples_per_segment: int = 12,+ closed: bool = False, alpha: float = 0.5):+ """Sample a centripetal Catmull-Rom spline through the input 2D points."""+ sampled = _sample_catmull_rom_spline(+ points, samples_per_segment=samples_per_segment, closed=bool(closed), alpha=float(alpha),+ )+ return [(float(x), float(y)) for (x, y) in sampled]+++def sample_catmull_rom_spline_3d(points, *, samples_per_segment: int = 12,+ closed: bool = False, alpha: float = 0.5):+ """Sample a centripetal Catmull-Rom spline through the input 3D points."""+ sampled = _sample_catmull_rom_spline(+ points, samples_per_segment=samples_per_segment, closed=bool(closed), alpha=float(alpha),+ )+ return [(float(x), float(y), float(z)) for (x, y, z) in sampled]+++def sample_arc_3d(*, start_point, center, normal, angle, segments: int = 16):+ """Sample a circular 3D arc from ``start_point`` around (``center``, ``normal``)."""+ start = (float(start_point[0]), float(start_point[1]), float(start_point[2]))+ ctr = (float(center[0]), float(center[1]), float(center[2]))+ n = _v_normalize((float(normal[0]), float(normal[1]), float(normal[2])))+ steps = max(2, int(segments))++ radius_vec = _v_sub(start, ctr)+ axial = abs(_v_dot(_v_normalize(radius_vec), n)) if _v_norm(radius_vec) > _EPS else 1.0+ if _v_norm(radius_vec) <= _EPS or axial > 1.0 - 1e-6:+ raise ValueError("start_point must be away from center and not collinear with normal")++ out = []+ for i in range(steps + 1):+ t = float(i) / float(steps)+ rv = _v_rotate_rodrigues(radius_vec, n, float(angle) * t)+ out.append(_v_add(ctr, rv))+ return out+++# ---------------------------------------------------------------------------+# WirePath builder (verbatim from sdk _mesh/common.py)+# ---------------------------------------------------------------------------+@dataclass+class WirePath:+ points: List[Vec3]++ def __init__(self, start: Vec3):+ s = (float(start[0]), float(start[1]), float(start[2]))+ self.points = [s]++ @classmethod+ def from_points(cls, points):+ pts = [(float(x), float(y), float(z)) for (x, y, z) in points]+ if len(pts) < 1:+ raise ValueError("WirePath.from_points requires at least one point")+ out = cls(pts[0])+ for p in pts[1:]:+ out.line_to(p)+ return out++ def _append_if_distinct(self, p):+ last = self.points[-1]+ if _v_norm(_v_sub(last, p)) > _EPS:+ self.points.append((float(p[0]), float(p[1]), float(p[2])))++ def line_to(self, point):+ p = (float(point[0]), float(point[1]), float(point[2]))+ self._append_if_distinct(p)+ return self++ def line_by(self, dx, dy, dz):+ cur = self.points[-1]+ self._append_if_distinct((cur[0] + float(dx), cur[1] + float(dy), cur[2] + float(dz)))+ return self++ def bezier_to(self, control1, control2, end, *, samples: int = 12):+ p0 = self.points[-1]+ pts = sample_cubic_bezier_spline_3d(+ [p0, control1, control2, end], samples_per_segment=max(2, int(samples)),+ )+ for p in pts[1:]:+ self._append_if_distinct(p)+ return self++ def arc(self, *, center, normal, angle, segments: int = 16):+ pts = sample_arc_3d(+ start_point=self.points[-1], center=center, normal=normal,+ angle=angle, segments=segments,+ )+ for p in pts[1:]:+ self._append_if_distinct(p)+ return self++ def extend(self, points):+ for p in points:+ self.line_to(p)+ return self++ def to_points(self):+ return list(self.points)+++# ---------------------------------------------------------------------------+# path / wire-centerline helpers (verbatim from sdk _mesh/sweeps.py)+# ---------------------------------------------------------------------------+def _dedupe_path_points(path):+ if not path:+ return []+ out = [path[0]]+ for p in path[1:]:+ if _v_norm(_v_sub(p, out[-1])) > _EPS:+ out.append(p)+ return out+++def _compute_path_tangents(path):+ n = len(path)+ if n < 2:+ raise ValueError("Pipe path requires at least two distinct points")+ tangents = []+ for i in range(n):+ if i == 0:+ d = _v_sub(path[1], path[0])+ elif i == n - 1:+ d = _v_sub(path[-1], path[-2])+ else:+ d = _v_sub(path[i + 1], path[i - 1])+ tangents.append(_v_normalize(d))+ return tangents+++def _initial_frame(tangent, up_hint):+ up = _v_normalize(up_hint)+ if abs(_v_dot(up, tangent)) > 0.95:+ up = (0.0, 1.0, 0.0) if abs(tangent[1]) < 0.95 else (1.0, 0.0, 0.0)+ n = _v_cross(up, tangent)+ if _v_norm(n) <= _EPS:+ n = _v_cross((0.0, 0.0, 1.0), tangent)+ n = _v_normalize(n)+ b = _v_normalize(_v_cross(tangent, n))+ return (n, b)+++@dataclass+class _WireCorner:+ incoming_end: Vec3+ outgoing_start: Vec3+ bridge: List[Vec3]+++def _append_distinct_path_point(points, point, *, tol=_EPS):+ p = (float(point[0]), float(point[1]), float(point[2]))+ if not points or _v_norm(_v_sub(points[-1], p)) > tol:+ points.append(p)+++def _preprocess_wire_polyline_points(points, *, min_segment_length, closed_path):+ raw = [(float(x), float(y), float(z)) for (x, y, z) in points]+ if len(raw) < 2:+ raise ValueError("WirePolylineGeometry requires at least two input points")++ tol = max(float(min_segment_length), _EPS)+ out = [raw[0]]+ for point in raw[1:]:+ if _v_norm(_v_sub(point, out[-1])) >= tol:+ out.append(point)++ if len(out) < 2:+ raise ValueError("Too few distinct points after dedupe")++ if closed_path:+ if _v_norm(_v_sub(out[0], out[-1])) <= tol:+ out[-1] = out[0]+ else:+ out.append(out[0])+ if len(out) < 4:+ raise ValueError("Closed path requires at least three distinct points")++ return out+++def _wire_corner_from_three_points(prev_point, corner_point, next_point, *,+ corner_mode, corner_radius, corner_segments):+ incoming = _v_sub(corner_point, prev_point)+ outgoing = _v_sub(next_point, corner_point)+ len_in = _v_norm(incoming)+ len_out = _v_norm(outgoing)++ if len_in <= _EPS or len_out <= _EPS:+ if corner_mode == "fillet" and corner_radius > 0.0:+ raise ValueError("Impossible fillet: corner has zero-length segment")+ return _WireCorner(corner_point, corner_point, [corner_point])++ u = _v_scale(incoming, 1.0 / len_in)+ v = _v_scale(outgoing, 1.0 / len_out)+ theta = acos(_sweeps_clamp(_v_dot(_v_scale(u, -1.0), v), -1.0, 1.0))++ if theta <= 1e-6 or abs(pi - theta) <= 1e-6:+ return _WireCorner(corner_point, corner_point, [corner_point])+ if corner_mode == "miter" or corner_radius <= 0.0:+ return _WireCorner(corner_point, corner_point, [corner_point])++ tan_half = tan(theta * 0.5)+ if abs(tan_half) <= _EPS:+ return _WireCorner(corner_point, corner_point, [corner_point])++ target_trim = corner_radius * tan_half+ if not isfinite(target_trim):+ target_trim = float("inf")+ trim = min(target_trim, 0.49 * min(len_in, len_out))+ if trim <= _EPS:+ return _WireCorner(corner_point, corner_point, [corner_point])++ a = _v_sub(corner_point, _v_scale(u, trim))+ b = _v_add(corner_point, _v_scale(v, trim))++ if corner_mode == "bevel":+ return _WireCorner(a, b, [a, b])++ radius_eff = trim / tan_half+ if radius_eff <= _EPS:+ return _WireCorner(corner_point, corner_point, [corner_point])++ bisector = _v_add(_v_scale(u, -1.0), v)+ if _v_norm(bisector) <= _EPS:+ return _WireCorner(corner_point, corner_point, [corner_point])+ bisector = _v_normalize(bisector)++ sin_half = sin(theta * 0.5)+ if abs(sin_half) <= _EPS:+ return _WireCorner(corner_point, corner_point, [corner_point])+ center_dist = radius_eff / sin_half+ center = _v_add(corner_point, _v_scale(bisector, center_dist))++ axis = _v_cross(u, v)+ if _v_norm(axis) <= _EPS:+ return _WireCorner(corner_point, corner_point, [corner_point])+ axis = _v_normalize(axis)++ ra = _v_sub(a, center)+ rb = _v_sub(b, center)+ if _v_norm(ra) <= _EPS or _v_norm(rb) <= _EPS:+ return _WireCorner(corner_point, corner_point, [corner_point])+ ra_u = _v_normalize(ra)+ rb_u = _v_normalize(rb)++ arc_angle = acos(_sweeps_clamp(_v_dot(ra_u, rb_u), -1.0, 1.0))+ if _v_dot(axis, _v_cross(ra_u, rb_u)) < 0.0:+ arc_angle = -arc_angle+ if abs(arc_angle) <= _EPS:+ return _WireCorner(corner_point, corner_point, [corner_point])++ arc_points = sample_arc_3d(+ start_point=a, center=center, normal=axis, angle=arc_angle,+ segments=max(2, int(corner_segments)),+ )+ arc_points[0] = a+ arc_points[-1] = b+ return _WireCorner(a, b, arc_points)+++def _build_wire_centerline(points, *, closed_path, corner_mode, corner_radius, corner_segments):+ if not closed_path:+ if len(points) < 2:+ raise ValueError("Wire path requires at least two points")+ if len(points) == 2:+ return list(points)++ out = []+ _append_distinct_path_point(out, points[0])+ for i in range(1, len(points) - 1):+ corner = _wire_corner_from_three_points(+ points[i - 1], points[i], points[i + 1],+ corner_mode=corner_mode, corner_radius=corner_radius, corner_segments=corner_segments,+ )+ _append_distinct_path_point(out, corner.incoming_end)+ for point in corner.bridge[1:]:+ _append_distinct_path_point(out, point)+ _append_distinct_path_point(out, points[-1])+ return out++ ring = points[:-1]+ if len(ring) < 3:+ raise ValueError("Closed path requires at least three distinct points")++ corners = []+ n = len(ring)+ for i in range(n):+ corners.append(+ _wire_corner_from_three_points(+ ring[(i - 1) % n], ring[i], ring[(i + 1) % n],+ corner_mode=corner_mode, corner_radius=corner_radius, corner_segments=corner_segments,+ )+ )++ out = []+ _append_distinct_path_point(out, corners[0].outgoing_start)+ for i in range(n):+ j = (i + 1) % n+ corner = corners[j]+ _append_distinct_path_point(out, corner.incoming_end)+ for point in corner.bridge[1:]:+ _append_distinct_path_point(out, point)+ if _v_norm(_v_sub(out[0], out[-1])) > _EPS:+ out.append(out[0])+ else:+ out[-1] = out[0]+ return out+++def _wire_circle_profile(radius, radial_segments):+ return [+ (float(radius) * cos(2.0 * pi * i / radial_segments),+ float(radius) * sin(2.0 * pi * i / radial_segments))+ for i in range(radial_segments)+ ]+++def _sample_supported_spline_path(points, *, spline, samples_per_segment, closed_spline,+ alpha, min_segment_length):+ spline_key = str(spline or "catmull_rom").strip().lower().replace("-", "_")+ if spline_key in {"catmullrom", "catmull_rom"}:+ sampled = sample_catmull_rom_spline_3d(+ list(points), samples_per_segment=int(samples_per_segment),+ closed=bool(closed_spline), alpha=float(alpha),+ )+ elif spline_key in {"bezier", "cubic_bezier", "cubicbezier"}:+ sampled = sample_cubic_bezier_spline_3d(+ list(points), samples_per_segment=int(samples_per_segment),+ )+ if bool(closed_spline) and _v_norm(_v_sub(sampled[0], sampled[-1])) > max(+ float(min_segment_length), _EPS+ ):+ raise ValueError(+ "closed_spline=True with spline='bezier' requires the sampled "+ "Bezier chain to end where it starts"+ )+ else:+ raise ValueError("spline must be one of: 'catmull_rom', 'bezier'")++ return _preprocess_wire_polyline_points(+ sampled, min_segment_length=float(min_segment_length), closed_path=bool(closed_spline),+ )+++# ---------------------------------------------------------------------------+# B-rep cross-section sweep core (replaces sdk PipeGeometry triangle soup)+# ---------------------------------------------------------------------------+def _pipe_rings(profile, path, *, up_hint, path_closed, closed=True):+ """Transport a closed 2D profile along a 3D polyline using the same+ parallel-transport frames as the sdk PipeGeometry; return the list of+ 3D cross-section rings (one closed loop of points per path station)."""+ raw_points = _profile_points_2d(profile)+ points = _ensure_ccw(raw_points) if closed else raw_points++ path_points = [(float(x), float(y), float(z)) for (x, y, z) in path]+ path_points = _dedupe_path_points(path_points)+ if len(path_points) < 2:+ raise ValueError("Pipe requires at least two distinct path points")+ if path_closed:+ if _v_norm(_v_sub(path_points[0], path_points[-1])) > _EPS:+ path_points.append(path_points[0])++ tangents = _compute_path_tangents(path_points)+ n0, b0 = _initial_frame(+ tangents[0], (float(up_hint[0]), float(up_hint[1]), float(up_hint[2]))+ )+ normals = [n0]+ binormals = [b0]+ for i in range(1, len(path_points)):+ t = tangents[i]+ prev_n = normals[-1]+ prev_b = binormals[-1]+ n = _v_sub(prev_n, _v_scale(t, _v_dot(prev_n, t)))+ if _v_norm(n) <= _EPS:+ n = _v_cross(prev_b, t)+ if _v_norm(n) <= _EPS:+ fallback = (0.0, 1.0, 0.0) if abs(t[1]) < 0.95 else (1.0, 0.0, 0.0)+ n = _v_cross(fallback, t)+ n = _v_normalize(n)+ b = _v_normalize(_v_cross(t, n))+ normals.append(n)+ binormals.append(b)+ if path_closed:+ normals[-1] = normals[0]+ binormals[-1] = binormals[0]++ rings = []+ for i, p in enumerate(path_points):+ n = normals[i]+ b = binormals[i]+ rings.append([+ (p[0] + x * n[0] + y * b[0], p[1] + x * n[1] + y * b[1], p[2] + x * n[2] + y * b[2])+ for (x, y) in points+ ])+ return rings+++def _loft_section_solid(rings, *, ruled=True):+ """Ruled-loft a list of section rings (each a closed loop of 3D points)+ into a single capped cq.Solid. Ruled stitching reproduces the sdk's+ straight quad strips between adjacent rings exactly."""+ if len(rings) < 2:+ raise ValueError("loft requires at least two section rings")+ wires = [cq.Wire.makePolygon([cq.Vector(*p) for p in ring], close=True) for ring in rings]+ return cq.Solid.makeLoft(wires, ruled=bool(ruled))+++# ---------------------------------------------------------------------------+# public assigned names+# ---------------------------------------------------------------------------+def wire_from_points(points, *, radius, radial_segments: int = 16, closed_path: bool = False,+ cap_ends: bool = False, corner_mode: str = "fillet", corner_radius: float = 0.0,+ corner_segments: int = 8, up_hint=(0.0, 0.0, 1.0),+ min_segment_length: float = 1e-6):+ """Build one circular tube/wire B-rep solid from a centerline path and radius.++ Corner handling (``fillet`` / ``miter`` / ``bevel``) matches the sdk+ WirePolylineGeometry centerline construction exactly; the swept circle is+ stitched into a parametric ruled loft instead of a triangle mesh."""+ radius = float(radius)+ radial_segments = int(radial_segments)+ closed_path = bool(closed_path)+ corner_mode = str(corner_mode).strip().lower()+ corner_radius = float(corner_radius)+ corner_segments = int(corner_segments)+ up_hint_v = (float(up_hint[0]), float(up_hint[1]), float(up_hint[2]))+ min_segment_length = float(min_segment_length)++ if radius <= 0.0:+ raise ValueError("radius must be > 0")+ if radial_segments < 6:+ raise ValueError("radial_segments must be >= 6")+ if corner_mode not in {"fillet", "miter", "bevel"}:+ raise ValueError("corner_mode must be one of: fillet, miter, bevel")+ if corner_radius < 0.0:+ raise ValueError("corner_radius must be >= 0")+ if corner_mode == "fillet" and corner_radius > 0.0 and corner_segments < 2:+ raise ValueError("corner_segments must be >= 2 when fillets are active")+ if _v_norm(up_hint_v) <= _EPS:+ raise ValueError("up_hint must be non-zero")+ if min_segment_length <= 0.0:+ raise ValueError("min_segment_length must be > 0")++ preprocessed = _preprocess_wire_polyline_points(+ points, min_segment_length=min_segment_length, closed_path=closed_path,+ )+ centerline = _build_wire_centerline(+ preprocessed, closed_path=closed_path, corner_mode=corner_mode,+ corner_radius=corner_radius, corner_segments=corner_segments,+ )+ centerline = _dedupe_path_points(centerline)+ if len(centerline) < 2:+ raise ValueError("Too few distinct points after corner preprocessing")++ profile = _wire_circle_profile(radius, radial_segments)+ rings = _pipe_rings(profile, centerline, up_hint=up_hint_v, path_closed=closed_path, closed=True)+ return MeshGeometry()._set(_loft_section_solid(rings, ruled=True))+++def tube_from_spline_points(points, *, radius, samples_per_segment: int = 12,+ closed_spline: bool = False, spline: str = "catmull_rom",+ alpha: float = 0.5, radial_segments: int = 16, cap_ends: bool = True,+ up_hint=(0.0, 0.0, 1.0), min_segment_length: float = 1e-6):+ """Fit a spline (``catmull_rom`` or chained ``bezier``) through the points,+ then build a circular tube B-rep along that path."""+ centerline = _sample_supported_spline_path(+ points, spline=spline, samples_per_segment=samples_per_segment,+ closed_spline=closed_spline, alpha=alpha, min_segment_length=min_segment_length,+ )+ return wire_from_points(+ centerline, radius=float(radius), radial_segments=int(radial_segments),+ closed_path=bool(closed_spline), cap_ends=bool(cap_ends), corner_mode="miter",+ up_hint=up_hint, min_segment_length=min_segment_length,+ )+++def sweep_profile_along_spline(points, *, profile, samples_per_segment: int = 12,+ closed_spline: bool = False, spline: str = "catmull_rom",+ alpha: float = 0.5, cap_profile: bool = True,+ up_hint=(0.0, 0.0, 1.0), min_segment_length: float = 1e-6):+ """Fit a spline through the points, then sweep a closed 2D profile along the+ sampled path with parallel-transported frames (B-rep ruled loft)."""+ centerline = _sample_supported_spline_path(+ points, spline=spline, samples_per_segment=samples_per_segment,+ closed_spline=closed_spline, alpha=alpha, min_segment_length=min_segment_length,+ )+ rings = _pipe_rings(profile, centerline, up_hint=up_hint, path_closed=bool(closed_spline),+ closed=True)+ return MeshGeometry()._set(_loft_section_solid(rings, ruled=True))+++def superellipse_side_loft(sections, *, exponents=2.8, segments: int = 56, cap: bool = True,+ closed: bool = True, min_height: float = 1e-4, min_width: float = 1e-4):+ """Build an organic casing from side-profile rails ``(y, z_min, z_max, width)``.++ The loft axis is +Y and each cross-section is a superellipse in XZ. Profile+ construction is verbatim from the sdk; the rings are stitched into a ruled+ B-rep loft (matching LoftGeometry's quad strips) and rotated into the +Y+ world frame."""+ if len(sections) < 2:+ raise ValueError("superellipse_side_loft requires at least two sections")++ if isinstance(exponents, (int, float)):+ expo_list = [float(exponents)] * len(sections)+ else:+ expo_list = [float(v) for v in exponents]+ if len(expo_list) != len(sections):+ raise ValueError("exponents must match section count")++ profiles = []+ segs = max(12, int(segments))+ min_h = max(1e-9, float(min_height))+ min_w = max(1e-9, float(min_width))++ for i, sec in enumerate(sections):+ y, z_min, z_max, width = sec+ y = float(y)+ z0 = float(z_min)+ z1 = float(z_max)+ if z1 < z0:+ z0, z1 = z1, z0++ height = max(min_h, z1 - z0)+ width = max(min_w, float(width))+ z_center = 0.5 * (z0 + z1)+ exponent = max(0.2, float(expo_list[i]))++ prof_xy = superellipse_profile(width, height, exponent=exponent, segments=segs)+ # Map final (x, y_section, z) into the loft frame: (x, -(z_local + z_center), y).+ prof_xyz = [(x, -(z_local + z_center), y) for (x, z_local) in prof_xy]+ profiles.append(prof_xyz)++ solid = _loft_section_solid(profiles, ruled=True)+ # rotate_x(-pi/2) to bring the loft axis to world +Y (matches sdk geom.rotate_x).+ geom = MeshGeometry()._set(solid)+ geom.rotate((1.0, 0.0, 0.0), -pi / 2.0)+ return geom+++# ===========================================================================+# @section geom-surface-frame (surface query frame; sdk placement.surface_frame)+# ===========================================================================+class SurfaceFrame:+ """Result of a surface query: a point on the surface, the outward unit+ normal there, and two orthonormal surface tangents."""+ __slots__ = ("point", "normal", "tangent_u", "tangent_v")++ def __init__(self, point, normal, tangent_u, tangent_v):+ self.point = tuple(float(c) for c in point)+ self.normal = tuple(float(c) for c in normal)+ self.tangent_u = tuple(float(c) for c in tangent_u)+ self.tangent_v = tuple(float(c) for c in tangent_v)++ def __repr__(self):+ return "SurfaceFrame(point=%r, normal=%r)" % (self.point, self.normal)+++def _sweeps_build_surface_tangents(normal, up_hint):+ """Verbatim from sdk placement._sweeps_build_surface_tangents."""+ up = _v_normalize(up_hint)+ projected = (+ up[0] - normal[0] * _v_dot(up, normal),+ up[1] - normal[1] * _v_dot(up, normal),+ up[2] - normal[2] * _v_dot(up, normal),+ )+ if _v_norm(projected) <= _EPS:+ fallback = (1.0, 0.0, 0.0)+ if abs(_v_dot(fallback, normal)) >= 0.95:+ fallback = (0.0, 1.0, 0.0)+ projected = _v_cross(fallback, normal)+ tangent_u = _v_normalize(projected)+ tangent_v = _v_normalize(_v_cross(normal, tangent_u))+ return tangent_u, tangent_v+++def _origin_shape(shape, origin):+ """Apply a shim Origin (xyz + rpy radians, URDF Rz@Ry@Rx) to a cq shape."""+ if origin is None:+ return shape+ xyz = tuple(float(v) for v in getattr(origin, "xyz", (0.0, 0.0, 0.0)))+ rpy = tuple(float(v) for v in getattr(origin, "rpy", (0.0, 0.0, 0.0)))+ s = shape+ if any(abs(a) > 0.0 for a in rpy):+ s = s.rotate(cq.Vector(0, 0, 0), cq.Vector(1, 0, 0), math.degrees(rpy[0]))+ s = s.rotate(cq.Vector(0, 0, 0), cq.Vector(0, 1, 0), math.degrees(rpy[1]))+ s = s.rotate(cq.Vector(0, 0, 0), cq.Vector(0, 0, 1), math.degrees(rpy[2]))+ if any(abs(a) > 0.0 for a in xyz):+ s = s.translate(cq.Vector(*xyz))+ return s+++def _subject_shape(target, *, prefer_collisions=False):+ """Resolve a Part / Visual / Geometry subject into a single cq.Shape."""+ visuals = getattr(target, "visuals", None)+ collisions = getattr(target, "collisions", None)+ if visuals is not None or collisions is not None:+ items = list(collisions) if (prefer_collisions and collisions) else list(visuals or [])+ if not items:+ items = list(collisions or [])+ shapes = []+ for item in items:+ geom = getattr(item, "geometry", None)+ if geom is None:+ continue+ shapes.append(_origin_shape(_to_shape(geom), getattr(item, "origin", None)))+ if not shapes:+ raise ValidationError("subject has no geometry to query")+ out = shapes[0]+ for s in shapes[1:]:+ out = out.fuse(s)+ return out+ if hasattr(target, "geometry") and hasattr(target, "origin"):+ return _origin_shape(_to_shape(target.geometry), target.origin)+ return _to_shape(target)+++def _surface_closest_point_normal(shape, query):+ """Closest point on a cq shape's surface to ``query`` plus the outward+ unit normal there (via OCC BRepExtrema + face geometry)."""+ from OCP.BRepExtrema import BRepExtrema_DistShapeShape, BRepExtrema_SupportType+ from OCP.BRepGProp import BRepGProp_Face+ from OCP.TopoDS import TopoDS+ from OCP.TopAbs import TopAbs_REVERSED+ from OCP.gp import gp_Pnt, gp_Vec++ vtx = cq.Vertex.makeVertex(float(query[0]), float(query[1]), float(query[2]))+ dss = BRepExtrema_DistShapeShape(shape.wrapped, vtx.wrapped)+ dss.Perform()+ if not dss.IsDone() or dss.NbSolution() < 1:+ raise ValidationError("surface_frame: distance query failed")++ p = dss.PointOnShape1(1)+ point = (p.X(), p.Y(), p.Z())++ normal = None+ if dss.SupportTypeShape1(1) == BRepExtrema_SupportType.BRepExtrema_IsInFace:+ face = TopoDS.Face_s(dss.SupportOnShape1(1))+ u, v = dss.ParOnFaceS1(1)+ bgf = BRepGProp_Face(face)+ pnt = gp_Pnt()+ nrm = gp_Vec()+ bgf.Normal(u, v, pnt, nrm)+ nl = sqrt(nrm.X() ** 2 + nrm.Y() ** 2 + nrm.Z() ** 2)+ if nl > _EPS:+ normal = (nrm.X() / nl, nrm.Y() / nl, nrm.Z() / nl)+ if face.Orientation() == TopAbs_REVERSED:+ normal = (-normal[0], -normal[1], -normal[2])++ if normal is None:+ d = _v_sub((float(query[0]), float(query[1]), float(query[2])), point)+ normal = _v_normalize(d) if _v_norm(d) > _EPS else (0.0, 0.0, 1.0)+ return point, normal+++def surface_frame(target, *, point_hint=None, direction=None, asset_root=None,+ prefer_collisions: bool = False, up_hint=(0.0, 0.0, 1.0)):+ """Query a point + tangent frame on the surface of ``target`` (a Part,+ Visual, or geometry). Provide exactly one of ``point_hint`` (nearest+ surface point) or ``direction`` (surface point along that direction)."""+ if (point_hint is None) == (direction is None):+ raise ValidationError("Exactly one of point_hint or direction must be provided")++ shape = _subject_shape(target, prefer_collisions=prefer_collisions)++ query = point_hint+ if direction is not None:+ d = _v_normalize((float(direction[0]), float(direction[1]), float(direction[2])))+ bb = shape.BoundingBox()+ center = ((bb.xmin + bb.xmax) * 0.5, (bb.ymin + bb.ymax) * 0.5, (bb.zmin + bb.zmax) * 0.5)+ size = (bb.xmax - bb.xmin, bb.ymax - bb.ymin, bb.zmax - bb.zmin)+ radius = max(_v_norm(size), 1e-3)+ query = (center[0] + d[0] * radius * 4.0,+ center[1] + d[1] * radius * 4.0,+ center[2] + d[2] * radius * 4.0)++ point, normal = _surface_closest_point_normal(shape, query)+ tangent_u, tangent_v = _sweeps_build_surface_tangents(+ normal, (float(up_hint[0]), float(up_hint[1]), float(up_hint[2]))+ )+ return SurfaceFrame(point=point, normal=normal, tangent_u=tangent_u, tangent_v=tangent_v)diff --git a/toolkits/curation/articraft/shim_parts/wheels_tires.py b/toolkits/curation/articraft/shim_parts/wheels_tires.pynew file mode 100644index 00000000..953f35e9--- /dev/null+++ b/toolkits/curation/articraft/shim_parts/wheels_tires.py@@ -0,0 +1,616 @@+from __future__ import annotations++from articraft_cq import (cq, math, MeshGeometry, _to_shape, mesh_from_geometry,+ boolean_difference, boolean_union, boolean_intersection,+ Box, Cylinder, Sphere, ValidationError)++from dataclasses import dataclass+from typing import Literal, Optional, Sequence++# ===========================================================================+# @section geom-wheels-tires+#+# Clean-room CadQuery B-rep reimplementation of the Articraft sdk+# wheels-and-tires geometry family (FAMILY = "wheels_tires").+#+# Wheels/tires are bodies of revolution aligned to local X: the width spans+# X, the radial profile lies in YZ, and revolves are taken about the X axis+# (matching the real sdk so the bbox/orientation are identical). Construction+# mirrors the real sdk exactly (sdk/_core/v0/_mesh/wheels.py) but keeps the+# parametric solid instead of tessellating to a triangle mesh.+# ===========================================================================++cos = math.cos+sin = math.sin+pi = math.pi+++# ---- copied helpers (verbatim ports of sdk cadquery_helpers/common) -------+def _wheels_tires_clamp(x, low, high):+ return max(low, min(high, x))+++def _cq_annulus_x(outer_radius, inner_radius, width):+ return (+ cq.Workplane("YZ")+ .circle(float(outer_radius))+ .circle(max(float(inner_radius), 1.0e-4))+ .extrude(float(width) * 0.5, both=True)+ )+++def _wheels_tires_centered_pattern_positions(count, spacing):+ if count <= 0:+ return []+ if count == 1:+ return [0.0]+ origin = -0.5 * float(spacing) * float(count - 1)+ return [origin + float(index) * float(spacing) for index in range(count)]+++def _wheels_tires_rounded_slot_profile(length, width, *, segments=10):+ radius = width * 0.5+ straight = max(length - width, 0.0)+ half = straight * 0.5+ points = []+ for index in range(segments + 1):+ theta = -pi * 0.5 + pi * (index / float(segments))+ points.append((half + radius * cos(theta), radius * sin(theta)))+ for index in range(segments + 1):+ theta = pi * 0.5 + pi * (index / float(segments))+ points.append((-half + radius * cos(theta), radius * sin(theta)))+ return points+++def _wheels_tires_loft_between_radii_z(radii_and_offsets):+ wp = None+ previous_offset = 0.0+ for radius, offset in radii_and_offsets:+ if wp is None:+ wp = cq.Workplane("XY").workplane(offset=offset).circle(radius)+ else:+ wp = wp.workplane(offset=offset - previous_offset).circle(radius)+ previous_offset = offset+ if wp is None:+ raise ValidationError("No loft sections were provided")+ return wp.loft(combine=True, ruled=False)+++def _wheels_tires_cut_with_pattern(shape, cutters):+ for cutter in cutters:+ shape = shape.cut(cutter)+ return shape+++def _shift_to_x0(shape):+ """Replicate sdk center=False: shift so the body's min X sits on x=0+ (inner mounting side on the origin plane)."""+ xmin = _to_shape(shape).BoundingBox().xmin+ return shape.translate((-xmin, 0.0, 0.0))+++# ---- sub-feature specs (frozen dataclasses; signatures match sdk exactly) --+@dataclass(frozen=True)+class WheelRim:+ outer_radius: Optional[float] = None+ inner_radius: Optional[float] = None+ flange_height: float = 0.0+ flange_thickness: float = 0.0+ bead_seat_depth: float = 0.0+++@dataclass(frozen=True)+class WheelHub:+ radius: float+ width: float+ cap_style: Literal["flat", "domed", "protruding", "recessed"] = "flat"+ bolt_pattern: Optional["BoltPattern"] = None+++@dataclass(frozen=True)+class WheelFace:+ dish_depth: float = 0.0+ front_inset: float = 0.0+ rear_inset: float = 0.0+ window_depth: float = 0.0+++@dataclass(frozen=True)+class WheelSpokes:+ style: Literal["none", "disc", "solid_slots", "straight", "split_y", "mesh"] = "none"+ count: Optional[int] = None+ thickness: float = 0.0+ window_radius: float = 0.0+ twist_deg: float = 0.0+++@dataclass(frozen=True)+class WheelBore:+ style: Literal["round", "keyed", "hex"] = "round"+ diameter: float = 0.0+ key_width: Optional[float] = None+++@dataclass(frozen=True)+class TireCarcass:+ crown_radius: Optional[float] = None+ belt_width_ratio: float = 0.34+ sidewall_bulge: float = 0.08+++@dataclass(frozen=True)+class TireTread:+ style: Literal["none", "circumferential", "block", "rib", "chevron", "lug"] = "none"+ depth: float = 0.0+ pitch: Optional[float] = None+ count: Optional[int] = None+ angle_deg: float = 0.0+ land_ratio: float = 0.5+++@dataclass(frozen=True)+class TireGroove:+ center_offset: float = 0.0+ width: float = 0.0+ depth: float = 0.0+++@dataclass(frozen=True)+class TireSidewall:+ style: Literal["flat", "bulged", "square", "rounded"] = "rounded"+ bulge: float = 0.08+ inset_ratio: float = 0.18+++@dataclass(frozen=True)+class TireShoulder:+ style: Literal["soft", "square"] = "soft"+ radius: float = 0.0+ width: float = 0.0+++# ---- geometry classes -----------------------------------------------------+class WheelGeometry(MeshGeometry):+ """Build a wheel/rim/hub visual aligned to local X."""++ def __init__(+ self,+ radius: float,+ width: float,+ *,+ rim: Optional[WheelRim] = None,+ hub: Optional[WheelHub] = None,+ face: Optional[WheelFace] = None,+ spokes: Optional[WheelSpokes] = None,+ bore: Optional[WheelBore] = None,+ flange: Optional["WheelFlange"] = None,+ center: bool = True,+ ):+ super().__init__()+ radius = float(radius)+ width = float(width)+ if radius <= 0.0 or width <= 0.0:+ raise ValidationError("radius and width must be positive")++ rim = rim or WheelRim()+ hub = hub or WheelHub(radius=radius * 0.18, width=width * 0.55)+ face = face or WheelFace()+ spokes = spokes or WheelSpokes(style="disc")+ bore = bore or WheelBore(style="round", diameter=max(radius * 0.18, 0.004))++ rim_outer = float(rim.outer_radius) if rim.outer_radius is not None else radius+ rim_inner = float(rim.inner_radius) if rim.inner_radius is not None else radius * 0.68+ if rim_inner <= 0.0 or rim_inner >= rim_outer:+ raise ValidationError("WheelRim.inner_radius must be positive and less than outer_radius")+ if hub.radius <= 0.0 or hub.width <= 0.0 or hub.radius >= rim_inner:+ raise ValidationError("WheelHub dimensions must be positive and fit inside the rim")+ if bore.diameter <= 0.0 or bore.diameter >= hub.radius * 2.0:+ raise ValidationError("WheelBore.diameter must fit inside the hub")++ shape = _cq_annulus_x(rim_outer, rim_inner, width)+ if rim.flange_height > 1e-6 and rim.flange_thickness > 1e-6:+ flange_inner = max(rim_outer - rim.flange_height, rim_inner + 1.0e-4)+ lip = _cq_annulus_x(rim_outer, flange_inner, rim.flange_thickness)+ shape = shape.union(lip.translate((width * 0.5 - rim.flange_thickness * 0.5, 0.0, 0.0)))+ shape = shape.union(+ lip.translate((-(width * 0.5 - rim.flange_thickness * 0.5), 0.0, 0.0))+ )+ if rim.bead_seat_depth > 1e-6:+ seat = _cq_annulus_x(+ rim_outer,+ max(rim_outer - rim.bead_seat_depth, rim_inner + 1.0e-4),+ max(width * 0.22, 0.004),+ )+ shape = shape.cut(seat)++ hub_cyl = cq.Workplane("YZ").circle(hub.radius).extrude(hub.width * 0.5, both=True)+ shape = shape.union(hub_cyl)+ if hub.cap_style == "domed":+ front_cap = (+ _wheels_tires_loft_between_radii_z(+ [+ (hub.radius, 0.0),+ (hub.radius * 0.82, hub.width * 0.12),+ (hub.radius * 0.30, hub.width * 0.28),+ ],+ )+ .rotate((0.0, 0.0, 0.0), (0.0, 1.0, 0.0), 90.0)+ .translate((hub.width * 0.5, 0.0, 0.0))+ )+ shape = shape.union(front_cap)+ elif hub.cap_style == "protruding":+ cap = cq.Workplane("YZ").circle(hub.radius * 0.82).extrude(max(width * 0.12, 0.003))+ shape = shape.union(cap.translate((hub.width * 0.5, 0.0, 0.0)))+ elif hub.cap_style == "recessed":+ pocket = cq.Workplane("YZ").circle(hub.radius * 0.55).extrude(max(width * 0.10, 0.002))+ shape = shape.cut(+ pocket.translate((hub.width * 0.5 - max(width * 0.10, 0.002), 0.0, 0.0))+ )++ disc_thickness = max(spokes.thickness, width * 0.08, 0.002)+ face_outer_radius = min(+ rim_outer - max(rim.flange_height * 0.25, radius * 0.02),+ rim_inner + (rim_outer - rim_inner) * 0.28 + face.window_depth,+ )+ face_outer_radius = max(face_outer_radius, hub.radius * 1.6)+ front_inset = _wheels_tires_clamp(+ face.front_inset + max(face.dish_depth, 0.0), 0.0, width * 0.5 - disc_thickness * 0.5+ )+ rear_inset = _wheels_tires_clamp(face.rear_inset, 0.0, width * 0.5 - disc_thickness * 0.5)+ disc_inner_radius = max(hub.radius * 0.78, hub.radius - max(disc_thickness * 0.35, 0.0012))+ front_disc = _cq_annulus_x(+ face_outer_radius, disc_inner_radius, disc_thickness+ ).translate((width * 0.5 - front_inset - disc_thickness * 0.5, 0.0, 0.0))+ rear_disc = _cq_annulus_x(+ face_outer_radius, disc_inner_radius, disc_thickness+ ).translate((-(width * 0.5 - rear_inset - disc_thickness * 0.5), 0.0, 0.0))+ shape = shape.union(front_disc).union(rear_disc)++ spoke_style = spokes.style+ if spoke_style not in {"none", "disc"}:+ spoke_count = spokes.count or (6 if spoke_style in {"straight", "solid_slots"} else 5)+ if spoke_count < 2:+ raise ValidationError("WheelSpokes.count must be at least 2")+ window_width = max(+ spokes.window_radius+ if spokes.window_radius > 0.0+ else (rim_outer - hub.radius) * 0.09,+ 0.003,+ )+ slot_length = max((face_outer_radius - hub.radius) * 0.75, 0.010)+ cut_width = width + 0.02+ cutters = []++ def radial_slot(angle_deg, center_radius, length, width_local):+ profile = _wheels_tires_rounded_slot_profile(length, width_local)+ slot = (+ cq.Workplane("YZ")+ .polyline([(point[1], point[0] + center_radius) for point in profile])+ .close()+ .extrude(cut_width * 0.5, both=True)+ .rotate((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), angle_deg)+ )+ return slot++ if spoke_style in {"straight", "solid_slots"}:+ for index in range(spoke_count):+ cutters.append(+ radial_slot(+ 360.0 * index / float(spoke_count),+ (hub.radius + face_outer_radius) * 0.5,+ slot_length,+ window_width * (1.5 if spoke_style == "solid_slots" else 1.0),+ )+ )+ elif spoke_style == "split_y":+ for index in range(spoke_count):+ base = 360.0 * index / float(spoke_count)+ cutters.append(+ radial_slot(+ base - 10.0,+ hub.radius + (face_outer_radius - hub.radius) * 0.44,+ slot_length * 0.55,+ window_width * 0.85,+ )+ )+ cutters.append(+ radial_slot(+ base + 10.0,+ hub.radius + (face_outer_radius - hub.radius) * 0.64,+ slot_length * 0.55,+ window_width * 0.85,+ )+ )+ else:+ rows = (+ hub.radius + (face_outer_radius - hub.radius) * 0.36,+ hub.radius + (face_outer_radius - hub.radius) * 0.68,+ )+ for row_index, center_radius in enumerate(rows):+ count = spoke_count * (2 if row_index == 1 else 1)+ for index in range(count):+ cutters.append(+ radial_slot(+ 360.0 * index / float(count)+ + (180.0 / float(count) if row_index == 1 else 0.0),+ center_radius,+ slot_length * (0.30 if row_index == 0 else 0.24),+ window_width * 0.72,+ )+ )+ shape = _wheels_tires_cut_with_pattern(shape, cutters)++ if hub.bolt_pattern is not None:+ pattern = hub.bolt_pattern+ if pattern.count < 2 or pattern.circle_diameter <= 0.0 or pattern.hole_diameter <= 0.0:+ raise ValidationError(+ "BoltPattern count, circle_diameter, and hole_diameter must be positive"+ )+ bolt_r = pattern.circle_diameter * 0.5+ if bolt_r + pattern.hole_diameter * 0.5 >= hub.radius:+ raise ValidationError("BoltPattern holes must fit inside the hub")+ for index in range(pattern.count):+ angle = 2.0 * pi * index / float(pattern.count)+ hole = (+ cq.Workplane("YZ")+ .circle(pattern.hole_diameter * 0.5)+ .extrude(width + 0.04, both=True)+ .translate((0.0, bolt_r * cos(angle), bolt_r * sin(angle)))+ )+ shape = shape.cut(hole)++ if bore.style == "round":+ bore_cut = (+ cq.Workplane("YZ").circle(bore.diameter * 0.5).extrude(width + 0.04, both=True)+ )+ elif bore.style == "hex":+ bore_cut = cq.Workplane("YZ").polygon(6, bore.diameter).extrude(width + 0.04, both=True)+ else:+ key_width = bore.key_width or bore.diameter * 0.32+ bore_cut = (+ cq.Workplane("YZ").circle(bore.diameter * 0.5).extrude(width + 0.04, both=True)+ )+ key = cq.Workplane("YZ").box(width + 0.04, key_width, bore.diameter * 0.5)+ bore_cut = bore_cut.union(key.translate((0.0, bore.diameter * 0.25, 0.0)))+ shape = shape.cut(bore_cut)++ if flange is not None:+ if flange.radius <= 0.0 or flange.thickness <= 0.0:+ raise ValidationError("WheelFlange radius and thickness must be positive")+ ring_inner = max(flange.radius - flange.thickness, hub.radius * 1.05)+ ring = _cq_annulus_x(flange.radius, ring_inner, flange.thickness * 0.6)+ bridge_len = max(flange.offset + flange.thickness * 0.6, flange.thickness * 0.8)+ bridge_radial = max(flange.radius - rim_outer + flange.thickness * 0.35, 0.006)+ bridge_thickness = max(flange.thickness * 0.55, 0.0015)+ strut = cq.Workplane("XY").box(bridge_len, bridge_radial, bridge_thickness)+ x_offsets = []+ if flange.side in {"front", "both"}:+ x_offsets.append(width * 0.5 + flange.offset)+ if flange.side in {"rear", "both"}:+ x_offsets.append(-(width * 0.5 + flange.offset))+ for x_offset in x_offsets:+ ring_shape = ring.translate((x_offset, 0.0, 0.0))+ shape = shape.union(ring_shape)+ axial_mid = (+ (width * 0.5 + x_offset) * 0.5+ if x_offset >= 0.0+ else (-(width * 0.5) + x_offset) * 0.5+ )+ radial_mid = (rim_outer + ring_inner) * 0.5+ for angle_deg in (0.0, 90.0, 180.0, 270.0):+ shape = shape.union(+ strut.translate((axial_mid, radial_mid, 0.0)).rotate(+ (0.0, 0.0, 0.0), (1.0, 0.0, 0.0), angle_deg+ )+ )++ if not center:+ shape = _shift_to_x0(shape)+ self._set(_to_shape(shape))+++class TireGeometry(MeshGeometry):+ """Build a tire aligned to local X."""++ def __init__(+ self,+ outer_radius: float,+ width: float,+ *,+ inner_radius: Optional[float] = None,+ carcass: Optional[TireCarcass] = None,+ tread: Optional[TireTread] = None,+ grooves: Sequence[TireGroove] = (),+ sidewall: Optional[TireSidewall] = None,+ shoulder: Optional[TireShoulder] = None,+ center: bool = True,+ ):+ super().__init__()+ outer_radius = float(outer_radius)+ width = float(width)+ inner_radius = float(inner_radius) if inner_radius is not None else outer_radius * 0.58+ carcass = carcass or TireCarcass()+ tread = tread or TireTread()+ sidewall = sidewall or TireSidewall()+ shoulder = shoulder or TireShoulder()++ if outer_radius <= 0.0 or width <= 0.0 or inner_radius <= 0.0:+ raise ValidationError("outer_radius, width, and inner_radius must be positive")+ if inner_radius >= outer_radius:+ raise ValidationError("inner_radius must be less than outer_radius")+ if tread.depth < 0.0:+ raise ValidationError("TireTread.depth must be non-negative")+ for groove in grooves:+ if groove.width < 0.0 or groove.depth < 0.0:+ raise ValidationError("TireGroove width/depth must be non-negative")++ half_w = width * 0.5+ belt_half = half_w * _wheels_tires_clamp(carcass.belt_width_ratio, 0.12, 0.96)+ shoulder_w = shoulder.width if shoulder.width > 1e-6 else width * 0.11+ side_bulge = max(sidewall.bulge, carcass.sidewall_bulge, 0.0)+ inner_crown = inner_radius * 0.82+ if sidewall.style == "square":+ outer_side = outer_radius+ shoulder_radius = max(+ outer_radius - max(shoulder.radius, tread.depth * 0.5, outer_radius * 0.01),+ inner_radius + 1.0e-4,+ )+ elif sidewall.style == "flat":+ outer_side = max(outer_radius - width * 0.08, inner_radius + 1.0e-4)+ shoulder_radius = outer_radius+ elif sidewall.style == "bulged":+ outer_side = min(outer_radius + side_bulge * width, outer_radius + width * 0.12)+ shoulder_radius = outer_radius+ else:+ outer_side = max(outer_radius - width * 0.04, inner_radius + 1.0e-4)+ shoulder_radius = outer_radius++ profile = [+ (-half_w, inner_radius),+ (-half_w * 0.92, inner_radius),+ (+ -half_w * 0.78,+ max(inner_radius + (outer_side - inner_radius) * 0.28, inner_radius + 1.0e-4),+ ),+ (-(belt_half + shoulder_w), outer_side),+ (-belt_half, shoulder_radius),+ (0.0, outer_radius),+ (belt_half, shoulder_radius),+ (belt_half + shoulder_w, outer_side),+ (+ half_w * 0.78,+ max(inner_radius + (outer_side - inner_radius) * 0.28, inner_radius + 1.0e-4),+ ),+ (half_w * 0.92, inner_radius),+ (half_w, inner_radius),+ (half_w * 0.55, inner_crown),+ (0.0, inner_crown),+ (-half_w * 0.55, inner_crown),+ (-half_w, inner_radius),+ ]+ shape = (+ cq.Workplane("XY")+ .polyline(profile)+ .close()+ .revolve(360.0, (0.0, 0.0, 0.0), (1.0, 0.0, 0.0))+ )++ for groove in grooves:+ if groove.width <= 1e-6 or groove.depth <= 1e-6:+ continue+ cutter = (+ cq.Workplane("XY")+ .moveTo(groove.center_offset - groove.width * 0.5, outer_radius - groove.depth)+ .lineTo(groove.center_offset + groove.width * 0.5, outer_radius - groove.depth)+ .lineTo(+ groove.center_offset + groove.width * 0.5, outer_radius + groove.depth * 1.6+ )+ .lineTo(+ groove.center_offset - groove.width * 0.5, outer_radius + groove.depth * 1.6+ )+ .close()+ .revolve(360.0, (0.0, 0.0, 0.0), (1.0, 0.0, 0.0))+ )+ shape = shape.cut(cutter)++ tread_style = tread.style+ tread_depth = tread.depth+ if tread_style == "circumferential" and tread_depth > 1e-6:+ groove_count = tread.count or 3+ band_positions = _wheels_tires_centered_pattern_positions(+ groove_count, max(tread.pitch or (width * 0.20), width * 0.14)+ )+ for pos in band_positions:+ cutter = (+ cq.Workplane("XY")+ .moveTo(pos - width * 0.03, outer_radius - tread_depth)+ .lineTo(pos + width * 0.03, outer_radius - tread_depth)+ .lineTo(pos + width * 0.03, outer_radius + tread_depth * 1.4)+ .lineTo(pos - width * 0.03, outer_radius + tread_depth * 1.4)+ .close()+ .revolve(360.0, (0.0, 0.0, 0.0), (1.0, 0.0, 0.0))+ )+ shape = shape.cut(cutter)+ elif tread_style == "rib" and tread_depth > 1e-6:+ rib_count = tread.count or 3+ rib_positions = _wheels_tires_centered_pattern_positions(+ rib_count, max(tread.pitch or (width * 0.20), width * 0.14)+ )+ for pos in rib_positions:+ rib = (+ cq.Workplane("XY")+ .moveTo(pos - width * 0.04, outer_radius)+ .lineTo(pos + width * 0.04, outer_radius)+ .lineTo(pos + width * 0.04, outer_radius + tread_depth)+ .lineTo(pos - width * 0.04, outer_radius + tread_depth)+ .close()+ .revolve(360.0, (0.0, 0.0, 0.0), (1.0, 0.0, 0.0))+ )+ shape = shape.union(rib)+ elif tread_style in {"block", "lug", "chevron"} and tread_depth > 1e-6:+ circ_count = tread.count or (+ 12+ if tread_style == "lug"+ else max(+ 14,+ int(+ (2.0 * pi * outer_radius) / max(tread.pitch or (width * 0.45), width * 0.24)+ ),+ )+ )+ axial_rows = (+ [-width * 0.22, width * 0.22]+ if tread_style == "lug"+ else (+ [-width * 0.22, 0.0, width * 0.22]+ if tread_style == "block"+ else [-width * 0.18, width * 0.18]+ )+ )+ tangential = (2.0 * pi * outer_radius / float(circ_count)) * _wheels_tires_clamp(+ tread.land_ratio, 0.18, 0.85+ )+ axial_width = width * (0.14 if tread_style == "lug" else 0.12)+ for row_index, row_x in enumerate(axial_rows):+ for index in range(circ_count):+ angle_deg = 360.0 * index / float(circ_count) + (+ 180.0 / float(circ_count) if row_index % 2 == 1 else 0.0+ )+ if tread_style == "chevron":+ for sign in (-1.0, 1.0):+ lug = (+ cq.Workplane("XY")+ .box(axial_width, tread_depth * 1.35, tangential * 0.48)+ .translate(+ (+ row_x + sign * axial_width * 0.38,+ outer_radius + tread_depth * 0.16,+ 0.0,+ )+ )+ .rotate(+ (0.0, 0.0, 0.0),+ (0.0, 1.0, 0.0),+ tread.angle_deg * sign+ if abs(tread.angle_deg) > 1e-6+ else 24.0 * sign,+ )+ .rotate((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), angle_deg)+ )+ shape = shape.union(lug)+ else:+ lug = (+ cq.Workplane("XY")+ .box(+ axial_width,+ tread_depth * 1.35,+ tangential * (0.55 if tread_style == "lug" else 0.42),+ )+ .translate((row_x, outer_radius + tread_depth * 0.16, 0.0))+ .rotate((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), angle_deg)+ )+ shape = shape.union(lug)++ if not center:+ shape = _shift_to_x0(shape)+ self._set(_to_shape(shape))diff --git a/toolkits/curation/articraft/upload_articraft.py b/toolkits/curation/articraft/upload_articraft.pynew file mode 100644index 00000000..56556cc3--- /dev/null+++ b/toolkits/curation/articraft/upload_articraft.py@@ -0,0 +1,142 @@+#!/usr/bin/env python3+"""Upload the finished folders to HF ilabai/3dcodeverse under articraft/.+ data/articraft_urdf_geo_only -> articraft/urdf_geo_only+ data/articraft_urdf_tex -> articraft/urdf_tex+ data/articraft_with_mesh -> articraft/with_mesh (verifier_pass samples only)+Chunked by sample name (multi-commit) so no single huge commit.+Resumable: huggingface_hub skips files already present.+Usage: python tools/upload_articraft.py [geo|tex|mesh|urdf|all]"""+import os, sys, math, time, json+# hf_transfer OFF: it has no 429 handling and HANGS when HF rate-limits us+# (uploading tens of thousands of small LFS render files blows the request quota).+# Plain huggingface_hub honors Retry-After and backs off on 429.+os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "0"+from huggingface_hub import HfApi, CommitOperationAdd++REPO = "ilabai/3dcodeverse"+_HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.environ.get("ARTICRAFT_ROOT", os.path.dirname(os.path.dirname(os.path.dirname(_HERE))))+TOK = open(os.path.expanduser("~/.hf_token")).read().strip()+api = HfApi(token=TOK)+CHUNKS = 6+which = sys.argv[1] if len(sys.argv) > 1 else "all"+++def upload_dir(local, prefix):+ names = sorted(os.listdir(local))+ per = math.ceil(len(names) / CHUNKS)+ print(f"=== {prefix}: {len(names)} samples in {CHUNKS} chunks of ~{per} ===", flush=True)+ for ci in range(0, len(names), per):+ batch = names[ci:ci + per]+ patterns = [f"{n}/**" for n in batch]+ idx = ci // per + 1+ t0 = time.time()+ for attempt in range(4):+ try:+ api.upload_folder(+ repo_id=REPO, repo_type="dataset", folder_path=local,+ path_in_repo=prefix, allow_patterns=patterns,+ commit_message=f"add {prefix} chunk {idx}/{CHUNKS} ({len(batch)} samples)",+ )+ print(f" chunk {idx}/{CHUNKS} OK ({len(batch)} samples) {time.time()-t0:.0f}s", flush=True)+ break+ except Exception as e:+ print(f" chunk {idx} attempt {attempt} FAIL: {str(e)[:120]}", flush=True)+ time.sleep(10 * (attempt + 1))+ else:+ print(f" chunk {idx} GAVE UP", flush=True)+++def _passed(d):+ try:+ return (json.load(open(f"{d}/meta.json")).get("conversion") or {}).get("verifier_pass") is True+ except Exception:+ return False+++def upload_filtered(local, prefix, names, chunks, pause=20):+ per = math.ceil(len(names) / chunks)+ print(f"=== {prefix}: {len(names)} samples in {chunks} chunks of ~{per} ===", flush=True)+ for ci in range(0, len(names), per):+ batch = names[ci:ci + per]+ patterns = [f"{n}/**" for n in batch]+ idx = ci // per + 1+ t0 = time.time()+ for attempt in range(8):+ try:+ api.upload_folder(+ repo_id=REPO, repo_type="dataset", folder_path=local,+ path_in_repo=prefix, allow_patterns=patterns,+ commit_message=f"add {prefix} chunk {idx}/{chunks} ({len(batch)} samples)",+ )+ print(f" chunk {idx}/{chunks} OK ({len(batch)} samples) {time.time()-t0:.0f}s", flush=True)+ break+ except Exception as e:+ msg = str(e)+ # 429 / rate-limit -> long backoff so we ride out the window+ wait = min(90 * (attempt + 1), 420) if any(s in msg for s in ("429", "rate limit", "Too Many")) else min(20 * (attempt + 1), 120)+ print(f" chunk {idx} attempt {attempt} FAIL ({msg[:90]}) -> wait {wait}s", flush=True)+ time.sleep(wait)+ else:+ print(f" chunk {idx} GAVE UP", flush=True)+ time.sleep(pause) # gentle gap between chunks to stay under the request quota+++def upload_lowcc(local, prefix, names, chunks, num_threads=5, pause=15):+ """Gentle upload via create_commit with a LOW LFS thread count (default 5) so+ we never burst past HF's request rate limit. Resumable: LFS objects already on+ the hub are skipped (hash check); a fully-present chunk is detected + skipped."""+ per = math.ceil(len(names) / chunks)+ print(f"=== {prefix}: {len(names)} samples in {chunks} chunks of ~{per} | num_threads={num_threads} ===", flush=True)+ for ci in range(0, len(names), per):+ batch = names[ci:ci + per]+ idx = ci // per + 1+ ops = []+ for s in batch:+ sd = f"{local}/{s}"+ for root, _, files in os.walk(sd):+ for f in files:+ full = os.path.join(root, f)+ rel = os.path.relpath(full, local) # "<sample>/..."+ ops.append(CommitOperationAdd(path_in_repo=f"{prefix}/{rel}", path_or_fileobj=full))+ t0 = time.time()+ for attempt in range(8):+ try:+ api.create_commit(repo_id=REPO, repo_type="dataset", operations=ops,+ commit_message=f"add {prefix} chunk {idx}/{chunks} ({len(batch)} samples)",+ num_threads=num_threads)+ print(f" chunk {idx}/{chunks} OK ({len(batch)} samples, {len(ops)} files) {time.time()-t0:.0f}s", flush=True)+ break+ except Exception as e:+ msg = str(e)+ if "No files" in msg or "no files" in msg or "nothing to commit" in msg.lower():+ print(f" chunk {idx}/{chunks} already present, skip", flush=True)+ break+ wait = min(90 * (attempt + 1), 420) if any(s in msg for s in ("429", "rate limit", "Too Many")) else min(20 * (attempt + 1), 120)+ print(f" chunk {idx} attempt {attempt} FAIL ({msg[:90]}) -> wait {wait}s", flush=True)+ time.sleep(wait)+ else:+ print(f" chunk {idx} GAVE UP", flush=True)+ time.sleep(pause)+++if which in ("all", "urdf", "geo"):+ upload_dir(f"{ROOT}/data/articraft_urdf_geo_only", "articraft/urdf_geo_only")+if which in ("all", "urdf", "tex"):+ upload_dir(f"{ROOT}/data/articraft_urdf_tex", "articraft/urdf_tex")+if which in ("all", "mesh"):+ DD = f"{ROOT}/data/articraft_with_mesh"+ names = sorted(n for n in os.listdir(DD)+ if _passed(f"{DD}/{n}") and os.path.exists(f"{DD}/{n}/renders/view_07.png"))+ NT = int(os.environ.get("UPLOAD_THREADS", "5"))+ upload_lowcc(DD, "articraft/cadquery_single_tex", names, 16, num_threads=NT)+if which == "meshnew": # only the debug-recovered samples (carry a geom_approx flag)+ DD = f"{ROOT}/data/articraft_with_mesh"+ def _new(d):+ c = json.load(open(f"{d}/meta.json")).get("conversion") or {}+ return c.get("verifier_pass") is True and "geom_approx" in c+ names = sorted(n for n in os.listdir(DD)+ if _new(f"{DD}/{n}") and os.path.exists(f"{DD}/{n}/renders/view_07.png"))+ print(f"meshnew: {len(names)} debug-recovered samples", flush=True)+ upload_lowcc(DD, "articraft/cadquery_single_tex", names, 2, num_threads=5)+print("UPLOAD_DONE", flush=True)diff --git a/toolkits/curation/articraft/verify_cadquery.py b/toolkits/curation/articraft/verify_cadquery.pynew file mode 100644index 00000000..e8a43936--- /dev/null+++ b/toolkits/curation/articraft/verify_cadquery.py@@ -0,0 +1,474 @@+#!/usr/bin/env python3+"""Deterministic verification harness for Articraft CadQuery reproductions.++Usage:+ python verify_cadquery.py <sample_dir> <code_py_path> [--render out.png]++It compares a self-contained CadQuery script (which builds the model in REST+pose) against the baked ground-truth meshes that the original Articraft record+produced (HF dataset camvsl/Articraft-10K).++Pipeline+--------+1. Read <sample_dir>/meta.json -> hf_name. Download+ https://huggingface.co/datasets/camvsl/Articraft-10K/resolve/main/<hf_name>.tar.gz+ (Bearer token), extract model.urdf + assets/meshes/*.obj, and combine the+ meshes/primitives placed per the URDF kinematic tree (rest pose) into ONE+ trimesh. Cache to /tmp/articraft_gt/<hf_name>.{stl,dir}.+2. Exec <code_py_path> in a fresh namespace; obtain the model object (see+ MODEL EXTRACTION CONVENTION below). Tessellate to ONE trimesh.+3. Normalize BOTH meshes (center to bbox origin, scale so max bbox dim = 1).+ Compute symmetric chamfer distance (~30k points each way), per-axis+ bbox-dim ratio (mine/GT), convex-hull volume ratio.+4. Print a single JSON line with chamfer / bbox_ratio / volume_ratio / pass /+ error. pass = (error is None) AND chamfer < 0.02 AND every bbox_ratio in+ [0.9, 1.1].+5. --render: save a side-by-side PNG (GT vs candidate).++Always exits 0 with a parseable JSON line (errors -> pass=false, error set).++MODEL EXTRACTION CONVENTION+---------------------------+The generated code.py SHOULD define a module-level variable `result` holding+the whole model (cq.Assembly preferred, or cq.Workplane / cq.Shape) in rest+pose. The extractor looks, in order:+ 1. module-level `result`+ 2. module-level `object_model`+ 3. the last module-level value that is a cq.Assembly / cq.Workplane /+ cq.Shape (Compound, Solid, ...)+ 4. fallback for hand-written PoCs that build under `if __name__ ==+ "__main__"`: call a zero-arg module-level builder function whose name+ matches a known allowlist (build_assembly, build_compound,+ build_model, build_object_model, build, make_model, ...) and return its+ cq object.+The module is exec'd with __name__ != "__main__" so any bottom `if __name__ ==+"__main__"` block (file writes / sys.argv) does NOT run.+"""+from __future__ import annotations++import argparse+import json+import os+import sys+import tarfile+import tempfile+import urllib.request+import xml.etree.ElementTree as ET++import numpy as np++GT_CACHE = "/tmp/articraft_gt"+HF_URL = "https://huggingface.co/datasets/camvsl/Articraft-10K/resolve/main/{}.tar.gz"+TOKEN_PATH = os.path.expanduser("~/.hf_token")++CHAMFER_PASS = 0.02+BBOX_LO, BBOX_HI = 0.9, 1.1+N_SAMPLES = 30000+TESS_TOL = 1e-4+TESS_ANG = 0.1+++# --------------------------------------------------------------------------- #+# Ground-truth construction from the URDF+# --------------------------------------------------------------------------- #+def _rpy_to_R(roll, pitch, yaw):+ cr, sr = np.cos(roll), np.sin(roll)+ cp, sp = np.cos(pitch), np.sin(pitch)+ cy, sy = np.cos(yaw), np.sin(yaw)+ Rx = np.array([[1, 0, 0], [0, cr, -sr], [0, sr, cr]])+ Ry = np.array([[cp, 0, sp], [0, 1, 0], [-sp, 0, cp]])+ Rz = np.array([[cy, -sy, 0], [sy, cy, 0], [0, 0, 1]])+ return Rz @ Ry @ Rx+++def _origin_T(elem):+ T = np.eye(4)+ if elem is None:+ return T+ o = elem.find("origin")+ if o is None:+ return T+ xyz = [float(v) for v in o.get("xyz", "0 0 0").split()]+ rpy = [float(v) for v in o.get("rpy", "0 0 0").split()]+ T[:3, :3] = _rpy_to_R(*rpy)+ T[:3, 3] = xyz+ return T+++def _read_token():+ for p in (TOKEN_PATH, os.environ.get("HF_TOKEN_FILE", "")):+ if p and os.path.exists(p):+ with open(p) as f:+ return f.read().strip()+ tok = os.environ.get("HF_TOKEN", "").strip()+ if tok:+ return tok+ raise RuntimeError("HF token not found at %s" % TOKEN_PATH)+++def _download_and_extract(hf_name, dest_dir):+ """Download <hf_name>.tar.gz and extract into dest_dir. Returns the root+ directory that contains model.urdf."""+ os.makedirs(dest_dir, exist_ok=True)+ tar_path = os.path.join(GT_CACHE, hf_name + ".tar.gz")+ if not os.path.exists(tar_path) or os.path.getsize(tar_path) == 0:+ url = HF_URL.format(hf_name)+ req = urllib.request.Request(url, headers={"Authorization": "Bearer " + _read_token()})+ with urllib.request.urlopen(req, timeout=300) as resp, open(tar_path, "wb") as out:+ out.write(resp.read())+ with tarfile.open(tar_path, "r:gz") as tf:+ tf.extractall(dest_dir)+ # locate model.urdf+ for root, _dirs, files in os.walk(dest_dir):+ if "model.urdf" in files:+ return root+ raise RuntimeError("model.urdf not found in %s" % tar_path)+++def _coerce_trimesh(obj):+ import trimesh++ if isinstance(obj, trimesh.Trimesh):+ return obj+ if isinstance(obj, trimesh.Scene):+ geoms = list(obj.dump())+ if not geoms:+ raise RuntimeError("empty scene")+ return trimesh.util.concatenate(geoms)+ # PointCloud / Path / other+ raise RuntimeError("unsupported geometry type %r" % type(obj))+++def _build_gt_mesh(urdf_root):+ import trimesh++ tree = ET.parse(os.path.join(urdf_root, "model.urdf"))+ root = tree.getroot()+ links = {l.get("name"): l for l in root.findall("link")}+ joints = root.findall("joint")++ parent_of, joint_T, children = {}, {}, set()+ for j in joints:+ child = j.find("child").get("link")+ parent = j.find("parent").get("link")+ parent_of[child] = parent+ joint_T[child] = _origin_T(j) # rest pose: joint variable = 0+ children.add(child)++ def world_T(name):+ T = np.eye(4)+ chain = []+ cur = name+ while cur in parent_of:+ chain.append(cur)+ cur = parent_of[cur]+ for n in reversed(chain):+ T = T @ joint_T[n]+ return T++ meshes = []+ for name, link in links.items():+ Tlink = world_T(name)+ for vis in link.findall("visual"):+ Tvis = Tlink @ _origin_T(vis)+ geom = vis.find("geometry")+ if geom is None:+ continue+ mesh_e = geom.find("mesh")+ box_e = geom.find("box")+ cyl_e = geom.find("cylinder")+ sph_e = geom.find("sphere")+ m = None+ if mesh_e is not None:+ fn = os.path.join(urdf_root, mesh_e.get("filename"))+ m = _coerce_trimesh(trimesh.load(fn, process=False))+ sc = mesh_e.get("scale")+ if sc:+ s = [float(v) for v in sc.split()]+ m = m.copy()+ m.apply_scale(s if len(s) == 3 else s[0])+ elif box_e is not None:+ sx, sy, sz = [float(v) for v in box_e.get("size").split()]+ m = trimesh.creation.box(extents=(sx, sy, sz))+ elif cyl_e is not None:+ r = float(cyl_e.get("radius"))+ h = float(cyl_e.get("length"))+ m = trimesh.creation.cylinder(radius=r, height=h, sections=48)+ elif sph_e is not None:+ m = trimesh.creation.icosphere(radius=float(sph_e.get("radius")))+ if m is not None:+ m = m.copy()+ m.apply_transform(Tvis)+ meshes.append(m)+ if not meshes:+ raise RuntimeError("no visual geometry in URDF")+ return trimesh.util.concatenate(meshes)+++def get_gt_mesh(hf_name):+ """Build & cache the ground-truth combined mesh. Returns a trimesh.Trimesh."""+ import trimesh++ os.makedirs(GT_CACHE, exist_ok=True)+ stl_cache = os.path.join(GT_CACHE, hf_name + ".stl")+ if os.path.exists(stl_cache) and os.path.getsize(stl_cache) > 0:+ return _coerce_trimesh(trimesh.load(stl_cache, process=False))+ urdf_root = _download_and_extract(hf_name, os.path.join(GT_CACHE, hf_name + "_src"))+ gt = _build_gt_mesh(urdf_root)+ try:+ gt.export(stl_cache)+ except Exception:+ pass+ return gt+++# --------------------------------------------------------------------------- #+# Candidate model extraction + tessellation+# --------------------------------------------------------------------------- #+BUILDER_NAMES = [+ "build_assembly",+ "build_compound",+ "build_model",+ "build_object_model",+ "build_object",+ "build_result",+ "make_model",+ "build",+]+++def _is_cq_model(obj, cq):+ return isinstance(obj, (cq.Assembly, cq.Workplane, cq.Shape))+++def extract_model(code_py_path):+ import cadquery as cq++ import types++ src = open(code_py_path).read()+ code_dir = os.path.dirname(os.path.abspath(code_py_path))+ # Register a real module in sys.modules so @dataclass (with string+ # annotations from `from __future__ import annotations`) can resolve+ # cls.__module__ during KW_ONLY detection. name != "__main__" so any+ # bottom `if __name__ == "__main__"` block won't run.+ modname = "verify_candidate"+ mod = types.ModuleType(modname)+ mod.__file__ = os.path.abspath(code_py_path)+ mod.__dict__["__builtins__"] = __builtins__+ sys.modules[modname] = mod+ sys.path.insert(0, code_dir)+ try:+ code = compile(src, code_py_path, "exec")+ exec(code, mod.__dict__)+ finally:+ sys.modules.pop(modname, None)+ if code_dir in sys.path:+ sys.path.remove(code_dir)+ ns = mod.__dict__++ # 1 & 2: preferred named variables+ for key in ("result", "object_model"):+ if key in ns and _is_cq_model(ns[key], cq):+ return ns[key]+ # 3: last module-level cq model object+ last = None+ for v in ns.values():+ if _is_cq_model(v, cq):+ last = v+ if last is not None:+ return last+ # 4: fallback -- call a known zero-arg builder function+ for fname in BUILDER_NAMES:+ fn = ns.get(fname)+ if callable(fn):+ try:+ obj = fn()+ except Exception:+ continue+ if _is_cq_model(obj, cq):+ return obj+ raise RuntimeError(+ "no model object found (expected module-level `result`, `object_model`, "+ "a module-level cq.Assembly/Workplane/Shape, or a known builder function)"+ )+++def model_to_trimesh(obj):+ import cadquery as cq+ import trimesh++ if isinstance(obj, cq.Assembly):+ shape = obj.toCompound()+ elif isinstance(obj, cq.Workplane):+ vals = [v for v in obj.vals() if isinstance(v, cq.Shape)]+ if not vals:+ raise RuntimeError("Workplane has no solid/shape values")+ shape = cq.Compound.makeCompound(vals) if len(vals) > 1 else vals[0]+ elif isinstance(obj, cq.Shape):+ shape = obj+ else:+ raise RuntimeError("unsupported model type %r" % type(obj))++ fd, tmp = tempfile.mkstemp(suffix=".stl")+ os.close(fd)+ try:+ cq.exporters.export(shape, tmp, tolerance=TESS_TOL, angularTolerance=TESS_ANG)+ mesh = _coerce_trimesh(trimesh.load(tmp, process=False))+ finally:+ if os.path.exists(tmp):+ os.remove(tmp)+ if mesh.vertices.shape[0] == 0:+ raise RuntimeError("tessellation produced empty mesh")+ return mesh+++# --------------------------------------------------------------------------- #+# Metrics+# --------------------------------------------------------------------------- #+def _normalize(mesh):+ """Center to bbox origin, scale so max bbox dim = 1. Returns (mesh, extents)."""+ m = mesh.copy()+ center = m.bounds.mean(axis=0)+ m.apply_translation(-center)+ ext = m.extents+ scale = ext.max()+ if scale <= 0:+ raise RuntimeError("degenerate mesh (zero extent)")+ m.apply_scale(1.0 / scale)+ return m, ext+++def compute_metrics(gt, mine, n=N_SAMPLES, seed=0):+ from scipy.spatial import cKDTree++ rng = np.random.RandomState(seed)++ gt_n, gt_ext = _normalize(gt)+ mn_n, mn_ext = _normalize(mine)++ bbox_ratio = (mn_ext / gt_ext).tolist()++ try:+ vg = gt.convex_hull.volume+ vm = mine.convex_hull.volume+ volume_ratio = float(vm / vg) if vg > 0 else None+ except Exception:+ volume_ratio = None++ pg = trimesh.sample.sample_surface(gt_n, n, seed=seed)[0]+ pm = trimesh.sample.sample_surface(mn_n, n, seed=seed)[0]++ tg = cKDTree(pg)+ tm = cKDTree(pm)+ d_m2g, _ = tg.query(pm)+ d_g2m, _ = tm.query(pg)+ chamfer = float((d_m2g.mean() + d_g2m.mean()) / 2.0)++ return {+ "chamfer": chamfer,+ "bbox_ratio": [float(r) for r in bbox_ratio],+ "volume_ratio": volume_ratio,+ "_norm": (gt_n, mn_n, pg, pm),+ }+++# --------------------------------------------------------------------------- #+# Render+# --------------------------------------------------------------------------- #+def render_compare(pg, pm, out_path):+ import matplotlib++ matplotlib.use("Agg")+ import matplotlib.pyplot as plt++ fig = plt.figure(figsize=(16, 8))+ views = [(20, -60), (90, -90)]+ cols = [("Ground truth (URDF)", pg, "tab:blue"), ("Candidate (CadQuery)", pm, "tab:orange")]+ for ci, (title, pts, color) in enumerate(cols):+ for ri, (el, az) in enumerate(views):+ ax = fig.add_subplot(2, 2, ri * 2 + ci + 1, projection="3d")+ k = min(15000, len(pts))+ idx = np.random.RandomState(0).choice(len(pts), k, replace=False)+ ax.scatter(pts[idx, 0], pts[idx, 1], pts[idx, 2], s=0.4, c=color)+ ax.view_init(elev=el, azim=az)+ ax.set_box_aspect((1, 1, 1))+ ax.set_xlim(-0.55, 0.55)+ ax.set_ylim(-0.55, 0.55)+ ax.set_zlim(-0.55, 0.55)+ ax.set_title("%s view(%d,%d)" % (title, el, az), fontsize=9)+ ax.set_xticks([])+ ax.set_yticks([])+ ax.set_zticks([])+ plt.tight_layout()+ plt.savefig(out_path, dpi=110)+ plt.close(fig)+++# --------------------------------------------------------------------------- #+# Main+# --------------------------------------------------------------------------- #+def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("sample_dir")+ ap.add_argument("code_py_path")+ ap.add_argument("--render", default=None)+ ap.add_argument("--glb", default=None, help="also export the exec'd model to this GLB (reuses the exec)")+ args = ap.parse_args()++ out = {"chamfer": None, "bbox_ratio": None, "volume_ratio": None, "pass": False, "error": None}++ try:+ global trimesh+ import trimesh # noqa: F401++ with open(os.path.join(args.sample_dir, "meta.json")) as f:+ meta = json.load(f)+ hf_name = meta["hf_name"]++ gt = get_gt_mesh(hf_name)+ model = extract_model(args.code_py_path)++ if args.glb: # reuse the exec'd model: export GLB for rendering+ try:+ import cadquery as cq+ import warnings as _w+ _w.filterwarnings("ignore")+ os.makedirs(os.path.dirname(args.glb), exist_ok=True)+ if isinstance(model, cq.Assembly):+ model.save(args.glb) # keeps per-part material colors+ else:+ shp = model.toCompound() if isinstance(model, cq.Workplane) else model+ cq.exporters.export(shp, args.glb)+ except Exception as e:+ out["glb_error"] = str(e)[:80]++ mine = model_to_trimesh(model)++ res = compute_metrics(gt, mine)+ gt_n, mn_n, pg, pm = res.pop("_norm")+ out["chamfer"] = res["chamfer"]+ out["bbox_ratio"] = res["bbox_ratio"]+ out["volume_ratio"] = res["volume_ratio"]++ if args.render:+ try:+ render_compare(pg, pm, args.render)+ except Exception as e: # rendering must never fail the run+ out["error"] = "render_failed: %s" % e++ ok_bbox = all(BBOX_LO <= r <= BBOX_HI for r in out["bbox_ratio"])+ out["pass"] = (out["error"] is None) and (out["chamfer"] < CHAMFER_PASS) and ok_bbox+ except Exception as e:+ import traceback++ out["error"] = "%s: %s" % (type(e).__name__, e)+ out["pass"] = False+ sys.stderr.write(traceback.format_exc())++ print(json.dumps(out))+ sys.exit(0)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/README.md b/toolkits/curation/deepcad/README.mdnew file mode 100644index 00000000..1862d1a4--- /dev/null+++ b/toolkits/curation/deepcad/README.md@@ -0,0 +1,136 @@+# DeepCAD → CadQuery curation tools++Tools used to curate the **DeepCAD** subset of [`ilabai/3dcodeverse`](https://huggingface.co/datasets/ilabai/3dcodeverse) (under `deepcad/v1/`).++[DeepCAD](https://github.com/ChrisWu1997/DeepCAD) (MIT) is a dataset of ~178k CAD+construction sequences (sketch + extrude, stored as quantized command vectors). We+convert each sequence into a **self-contained, executable CadQuery (B-rep) program**+— one `code.py` that only needs `import cadquery` to run — then validate, score+geometric fidelity, render, attach text, dedup, and upload.++**Result: 174,981 / 178,238 converted (98.2%); 174,177 produce a valid solid+(97.7%); 128,201 unique after dedup. Conversion fidelity vs DeepCAD's own+geometry: mean IoU 0.985, median 1.0, 96.9% > 0.9.**++## Approach++1. **Convert** the vectorized sequence → CadQuery, reusing the *validated*+ numeric logic from [GenCAD-Code](https://github.com/anniedoris/GenCAD-Code)+ (sketch plane from `CoordSystem`, two-sided/symmetric extrudes, arc geometry,+ the DeepCAD normalization scale). `convert_vec_to_cq.py` + `batch_convert.py`.+2. **Validate** by execution: run each `code.py`, compute exact volume ++ topological validity natively via OpenCASCADE (`GProp` + `BRepCheck_Analyzer`),+ export STEP. Batched worker imports cadquery once (`validate_fast.py` ++ `cq_check_batch.py`) — ~7× faster than per-model subprocess.+3. **Geometric fidelity (IoU)**: generate DeepCAD's *official* ground-truth mesh+ from the same vec via `cadlib.visualize.vec2CADsolid` (pythonocc-core, separate+ env), then voxel-IoU + Chamfer vs our CadQuery geometry. `gen_gt_stl.py` ++ `iou_compute.py`. This proves the conversion is geometrically faithful, not+ merely compilable.+4. **Render** 4 views (iso/front/right/top) + `object.glb` with headless pyrender+ (OSMesa). `render_all.py`.+5. **Text**: map [Text2CAD](https://huggingface.co/datasets/SadilKhan/Text2CAD)+ (CC-BY-NC-SA) 4-level descriptions by uid. `attach_text.py`. (Kept in+ `curation.json`; the official caption is filled by the maintainer.)+6. **Dedup** on the *source vec hash* (deterministic conversion ⇒ identical+ code+geometry). `dedup_vec.py`. (`dedup_fast.py` does a geometry-fingerprint+ variant — kept for reference; it over-merges, see Gotchas.)+7. **Assemble + finalize**: merge into a JSONL, mark dedup + leakage-free splits ++ IoU fidelity tiers. `assemble_corpus.py` → `mark_dedup.py` → `finalize.py`.+8. **Format + pack**: emit the official per-sample dir layout (`to_3dcodeverse.py`),+ then pack into tar shards + `metadata.parquet` 1:1 with `shadertoy/3X`+ (`pack_tar_parquet.py`) — turns 1.15M small files into ~30, sidestepping HF's+ request-rate limit and enabling HTTP-Range single-sample reads for the viewer.+9. **Upload** the ~30 packed files (`upload_to_hf.py` / `HfApi.upload_folder`).++## Layout++```+convert:+ convert_vec_to_cq.py vec(.h5) -> standalone CadQuery code (GenCAD-Code logic)+ batch_convert.py parallel conversion over the official split++validate:+ cq_check.py single-model worker: exec -> OCC volume+validity -> STEP+ cq_check_batch.py batched worker (import cadquery once per chunk)+ validate_fast.py fast driver (chunked subprocess, crash-isolating retry)+ validate_all.py simple per-model driver (slower; kept for reference)++fidelity (IoU):+ gen_gt_stl.py DeepCAD official vec2CADsolid -> ground-truth STL (pythonocc env)+ iou_compute.py voxel IoU + Chamfer: our STEP vs GT STL++render:+ render_all.py STEP -> tessellate -> pyrender 4 views (OSMesa)++text:+ attach_text.py Text2CAD CSV -> {uid: {abstract,beginner,intermediate,expert}}++assemble / dedup:+ assemble_corpus.py merge code+step+renders+text+iou -> JSONL+ mark_dedup.py annotate is_canonical/duplicate_of + cross-split leakage+ finalize.py fidelity tiers (IoU) + leakage-free split_dedup+ dedup_vec.py TRUE dedup on source vec hash+ dedup_fast.py geometry-fingerprint dedup (reference; over-merges)++format / pack / upload:+ to_3dcodeverse.py official per-sample dir layout (meta.json + curation.json + renders/)+ pack_tar_parquet.py pack -> <sf>-NNNNN.tar shards + metadata.parquet (3X layout)+ upload_to_hf.py HF upload (upload_large_folder / folder)++viz:+ viz_compare.py side-by-side ours-vs-ground-truth render (IoU-labeled)+```++## Configuration++Runs inside a Docker container (CadQuery needs conda-forge; pip OCP is fragile):+- main env: `cadquery=2.7`, `h5py`, `trimesh`, `scipy`, `pyrender`, `PyOpenGL==3.1.7`+- `gt` env (ground-truth): `pythonocc-core=7.7`, `matplotlib-base`, `numpy<1.24`+- `GENCAD_SCRIPTS` → cloned GenCAD-Code `scripts/` (pure geom helpers)+- `~/.hf_token` / token file → HF token (read for download, write for upload)+- `PYOPENGL_PLATFORM=osmesa` for headless rendering++## Typical pipeline++```bash+python batch_convert.py --vec_dir cad_vec --split split.json --out_dir converted_cq+python validate_fast.py --cq_dir converted_cq --step_dir step_files --out val.json+# ground-truth IoU (in the pythonocc 'gt' env, then back in cadquery env):+python gen_gt_stl.py --vec_dir cad_vec --ids ids.txt --out_dir gt_stl --deepcad DeepCAD+python iou_compute.py --step_dir step_files --gt_dir gt_stl --ids ids.txt --out iou.json+python render_all.py --step_dir step_files --out_dir renders+python attach_text.py --csv text2cad.csv --out text.json+python assemble_corpus.py ... | python mark_dedup.py ... | python finalize.py ...+python dedup_vec.py --vec_dir cad_vec --ids ids.txt --out dups.json+python to_3dcodeverse.py --jsonl final.jsonl --canonical_only --out_root corpus+python pack_tar_parquet.py --corpus corpus/deepcad --out packed --subfolder v1+python upload_to_hf.py --folder packed --repo ilabai/3dcodeverse --token_file ~/.hf_token+```++## Gotchas learned (baked into the tools)++- **vec→CadQuery, not JSON→CadQuery**: a first JSON→CQ attempt reconstructed the+ sketch plane via Euler `ZYX` angles — wrong. The vec path reuses GenCAD-Code's+ validated `CoordSystem` (origin/x_axis/normal directly) and handles two-sided+ extrudes + arc sweep + the `NORM_FACTOR=0.75` scale correctly.+- **trimesh can't load STEP** (needs `cascadio`). Validate with OCC natively+ (`GProp` volume + `BRepCheck_Analyzer`) instead of meshing.+- **Validation speed**: per-model subprocess pays the ~1.5s cadquery import every+ time. Batching one import per chunk is ~7× faster (2.7h → ~8min for 175k).+- **Dedup must key on vec/code, not geometry**: a volume/area/inertia fingerprint+ over-merges badly (one "dup group" had 1396 distinct vecs that merely share a+ bounding box). `dedup_vec.py` (source-vec hash) is correct: 128,201 unique.+- **DeepCAD's official split leaks**: 3,230 vec-duplicate groups span train/test+ (32,737 models). `finalize.py` emits a leakage-free `split_dedup`.+- **IoU is harsh on thin/tiny geometry**: voxelizing a hair-thin part at 48³ can+ read 0 overlap even when the shapes match. The <0.5-IoU tail (0.7%) is a mix of+ this metric artifact, pose diffs, and a few real local diffs (holes). Gate on+ IoU directly for fidelity-sensitive use; don't trust the quality flag alone.+- **pyrender + OSMesa**: needs `PyOpenGL==3.1.7` (3.1.0 lacks+ `OSMesaCreateContextAttribs`).+- **HF rate limit + many small files**: 1.15M-file `upload_large_folder` trips+ HF's 2500-req/5min limit and stalls on commits (8h → 14/100 buckets, ~890k 429s).+ Pack into tar shards + `metadata.parquet` (per-sample contiguous byte range,+ `<key>/...` naming) → ~30 files, uploads in minutes; viewer reads a single+ sample via HTTP Range on the shard.diff --git a/toolkits/curation/deepcad/assemble_corpus.py b/toolkits/curation/deepcad/assemble_corpus.pynew file mode 100644index 00000000..87a9c7e0--- /dev/null+++ b/toolkits/curation/deepcad/assemble_corpus.py@@ -0,0 +1,118 @@+"""+assemble_corpus.py+Merge CadQuery code + STEP + renders + Text2CAD text into a 3DCodeVerse JSONL.++Quality policy (from failure analysis):+ - OK -> quality="ok" (valid solid)+ - INVALID -> quality="geometry_warning" (exports STEP/renders but fails strict+ topological validity; usually sub-mm thin geometry from DeepCAD+ quantization). Kept, downstream can filter.+ - others (DEGENERATE/EXEC_FAIL/CRASH/CONVERT_FAIL) -> excluded.++Code source: fixed_cq/ if present (agent-fixed), else converted_cq/.+"""+import json+import argparse+from pathlib import Path+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--validation", required=True)+ ap.add_argument("--text", required=True)+ ap.add_argument("--split", required=True)+ ap.add_argument("--cq_dir", required=True)+ ap.add_argument("--fixed_dir", default="")+ ap.add_argument("--step_dir", required=True)+ ap.add_argument("--render_dir", default="")+ ap.add_argument("--iou", default="")+ ap.add_argument("--out", required=True)+ ap.add_argument("--include_invalid", action="store_true", default=True)+ args = ap.parse_args()++ val = json.load(open(args.validation))+ text_map = json.load(open(args.text))+ iou_map = json.load(open(args.iou)) if args.iou else {}+ split = json.load(open(args.split))+ id2split = {}+ for s in ("train", "validation", "test"):+ for mid in split.get(s, []):+ id2split[mid] = s++ keep = {mid["id"]: "ok" for mid in val.get("OK", [])}+ if args.include_invalid:+ for mid in val.get("INVALID", []):+ keep[mid["id"]] = "geometry_warning"++ cq_dir = Path(args.cq_dir)+ fixed_dir = Path(args.fixed_dir) if args.fixed_dir else None+ step_dir = Path(args.step_dir)+ render_dir = Path(args.render_dir) if args.render_dir else None+ views = ["iso", "front", "right", "top"]++ n_with_text = 0+ n_with_render = 0+ with open(args.out, "w") as out:+ for model_id in sorted(keep):+ sub, name = model_id.split("/")+ quality = keep[model_id]++ py = None+ fixed = False+ if fixed_dir and (fixed_dir / sub / (name + ".py")).exists():+ py = fixed_dir / sub / (name + ".py")+ fixed = True+ else:+ py = cq_dir / sub / (name + ".py")+ if not py.exists():+ continue++ step_path = step_dir / sub / (name + ".step")+ renders = {}+ if render_dir:+ for v in views:+ rp = render_dir / sub / f"{name}_{v}.png"+ if rp.exists():+ renders[v] = str(rp)+ if renders:+ n_with_render += 1++ text = text_map.get(model_id, {})+ if text:+ n_with_text += 1++ iou_rec = iou_map.get(model_id, {})+ iou = iou_rec.get("iou")+ chamfer = iou_rec.get("chamfer")++ entry = {+ "id": f"deepcad_MIT_{sub}_{name}",+ "source": "DeepCAD",+ "license": "MIT",+ "code_language": "cadquery_python",+ "code": py.read_text(),+ "step_path": str(step_path),+ "renders": renders,+ "text": text, # {abstract, beginner, intermediate, expert} (Text2CAD, CC BY-NC-SA)+ "metadata": {+ "quality": quality,+ "iou": iou, # geometric fidelity vs DeepCAD ground-truth+ "chamfer": chamfer,+ "conversion": "agent_fixed" if fixed else "auto",+ "provenance": "ABC/Onshape -> DeepCAD (ICCV 2021) -> CadQuery",+ "deepcad_id": model_id,+ "split": id2split.get(model_id, "unknown"),+ "text_license": "CC-BY-NC-SA-4.0" if text else None,+ },+ }+ out.write(json.dumps(entry) + "\n")++ n = sum(1 for _ in open(args.out))+ print(f"Wrote {n} entries to {args.out}")+ print(f" with text: {n_with_text} with renders: {n_with_render}")+ print(f" quality: ok={sum(1 for v in keep.values() if v=='ok')}, "+ f"geometry_warning={sum(1 for v in keep.values() if v=='geometry_warning')}")+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/attach_text.py b/toolkits/curation/deepcad/attach_text.pynew file mode 100644index 00000000..f34d5461--- /dev/null+++ b/toolkits/curation/deepcad/attach_text.py@@ -0,0 +1,51 @@+"""+attach_text.py+Parse the Text2CAD CSV into a {uid -> {level: description}} JSON map.++Text2CAD uid format ("0035/00359148") matches our DeepCAD model_id exactly.+We keep the 4 human-readable levels (abstract/beginner/intermediate/expert)+and drop the training-oriented columns (all_level_data, nli_data).+"""+import csv+import json+import argparse++csv.field_size_limit(10 * 1024 * 1024) # expert descriptions can be long++LEVELS = ["abstract", "beginner", "intermediate", "expert"]+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--csv", required=True)+ ap.add_argument("--out", required=True)+ args = ap.parse_args()++ text_map = {}+ n = 0+ with open(args.csv, newline="") as f:+ reader = csv.DictReader(f)+ for row in reader:+ uid = (row.get("uid") or "").strip()+ if not uid:+ continue+ entry = {}+ for lvl in LEVELS:+ v = (row.get(lvl) or "").strip()+ if v:+ entry[lvl] = v+ if entry:+ text_map[uid] = entry+ n += 1+ if n % 20000 == 0:+ print(f" parsed {n} rows, {len(text_map)} with text")++ json.dump(text_map, open(args.out, "w"))+ print(f"Done: {len(text_map)} uids with descriptions -> {args.out}")+ # quick coverage sample+ sample = next(iter(text_map.items()))+ print("sample uid:", sample[0], "levels:", list(sample[1].keys()))+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/batch_convert.py b/toolkits/curation/deepcad/batch_convert.pynew file mode 100644index 00000000..48abb2b2--- /dev/null+++ b/toolkits/curation/deepcad/batch_convert.py@@ -0,0 +1,71 @@+"""+batch_convert.py+Parallel DeepCAD vec(.h5) -> CadQuery .py conversion over the official split.++Usage:+ python batch_convert.py --vec_dir /data/.../cad_vec \+ --split /data/.../train_val_test_split.json \+ --out_dir /data/.../converted_cq --workers 16 [--limit 1000]+"""+import os+import json+import argparse+from pathlib import Path+from joblib import Parallel, delayed+from convert_vec_to_cq import vec_to_cadquery_code, extract_h5+++def process(model_id, vec_dir, out_dir):+ # model_id like "0000/00000007"+ sub, name = model_id.split("/")+ src = Path(vec_dir) / sub / (name + ".h5")+ dest_dir = Path(out_dir) / sub+ if not src.exists():+ return model_id, "MISSING_H5"+ try:+ vec = extract_h5(str(src))+ code = vec_to_cadquery_code(vec)+ dest_dir.mkdir(parents=True, exist_ok=True)+ (dest_dir / (name + ".py")).write_text(code)+ return model_id, "OK"+ except Exception as e:+ return model_id, f"CONVERT_FAIL: {type(e).__name__}: {e}"+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--vec_dir", required=True)+ ap.add_argument("--split", required=True)+ ap.add_argument("--out_dir", required=True)+ ap.add_argument("--workers", type=int, default=16)+ ap.add_argument("--limit", type=int, default=0, help="0 = all")+ ap.add_argument("--report", default="conversion_report.json")+ args = ap.parse_args()++ split = json.load(open(args.split))+ all_ids = split["train"] + split["validation"] + split["test"] \+ if "validation" in split else split["train"] + split["val"] + split["test"]+ if args.limit:+ all_ids = all_ids[: args.limit]++ print(f"Converting {len(all_ids)} models with {args.workers} workers...")+ results = Parallel(n_jobs=args.workers, verbose=5)(+ delayed(process)(mid, args.vec_dir, args.out_dir) for mid in all_ids+ )++ ok = [r for r in results if r[1] == "OK"]+ fails = [r for r in results if r[1] != "OK"]+ print(f"\nConverted: {len(ok)}/{len(all_ids)}, Failed: {len(fails)}")+ # bucket failure reasons+ buckets = {}+ for _, status in fails:+ key = status.split(":")[0]+ buckets[key] = buckets.get(key, 0) + 1+ print("Failure buckets:", buckets)+ json.dump({"ok": [r[0] for r in ok], "fail": [list(r) for r in fails],+ "buckets": buckets},+ open(args.report, "w"), indent=2)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/convert_vec_to_cq.py b/toolkits/curation/deepcad/convert_vec_to_cq.pynew file mode 100644index 00000000..035a6777--- /dev/null+++ b/toolkits/curation/deepcad/convert_vec_to_cq.py@@ -0,0 +1,289 @@+"""+convert_vec_to_cq.py+--------------------+Convert a DeepCAD vectorized command sequence (the .h5 `cad_vec` form) into a+standalone, executable CadQuery Python script.++This reuses the *validated* numeric conversion logic from GenCAD-Code+(scripts/h5tocadquery.py + geom_utils.py + deepcad_constants.py), which correctly+handles two-sided / symmetric extrudes, arc sweep geometry, and the DeepCAD+normalization scale. We only re-wrap it so that:+ * it returns the generated code as a string (no global side effects),+ * the heavy matplotlib/pandas imports of the original __main__ are avoided,+ * the produced script exports a STEP file to sys.argv[1].++The pure geometry helpers (CoordSystem, get_arc, polar_*) are imported unchanged+from the cloned GenCAD-Code repo.+"""+import os+import sys+import numpy as np++# --- locate the cloned GenCAD-Code scripts (pure helpers only) ---+GENCAD_SCRIPTS = os.environ.get(+ "GENCAD_SCRIPTS", "/data/ziyao/deepcad2cq/GenCAD-Code/scripts"+)+if GENCAD_SCRIPTS not in sys.path:+ sys.path.insert(0, GENCAD_SCRIPTS)++# These two modules are pure (numpy/math only) -- safe to import directly.+from deepcad_constants import ( # noqa: E402+ LINE, ARC, CIRCLE, EOS, SOL, EXTRUDE,+ EXTRUDE_OPERATIONS, EXTENT_TYPE, NORM_FACTOR,+)+from geom_utils import CoordSystem, get_arc # noqa: E402++import h5py # noqa: E402++UNQUANTIZE = True+++# ---------------------------------------------------------------------------+# The conversion logic below is adapted faithfully from GenCAD-Code's+# convert_h5_to_cadquery(); it builds the body of the CadQuery program as a+# string instead of writing it to disk. Variable `solid` holds the final result.+# ---------------------------------------------------------------------------+def vec_to_cadquery_code(vecs, truncate=6, use_fixed_decimal=True):+ """Return a CadQuery python program (string) for the given vec array.++ Raises TypeError / NotImplementedError on sequences DeepCAD/GenCAD cannot+ represent (zero extrude, malformed two-sided, unsupported curve combos).+ """++ def split_by_sketches(arr):+ split_indices = np.where(arr[:, 0] == EXTRUDE)[0] + 1+ split_arrays = np.split(arr, split_indices)[:-1]+ sketches = [a[:-1] for a in split_arrays]+ extrudes = [a[-1:] for a in split_arrays]+ return sketches, extrudes++ def split_by_loops(arr):+ split_indices = np.where(arr[:, 0] == SOL)[0]+ split_arrays = np.split(arr, split_indices)[1:]+ loops = [a[1:] for a in split_arrays]+ return loops++ def extract_extrude_params(extrusion_command, unquantize):+ c = extrusion_command[0]+ theta, phi, gamma = c[6], c[7], c[8]+ px, py, pz = c[9], c[10], c[11]+ scale = c[12]+ dir1, dir2 = c[13], c[14]+ op = int(c[15])+ typ = int(c[16])+ if unquantize:+ dir1 = dir1 / 256 * 2 - 1.0+ dir2 = dir2 / 256 * 2 - 1.0+ scale = scale / 256 * 2+ return theta, phi, gamma, px, py, pz, scale, dir1, dir2, op, typ++ def fmt(v):+ return f"{v:.{truncate}f}" if use_fixed_decimal else f"{v}"++ def cadquery_workplane(sp, sketch_num):+ cmt = f"# workplane for sketch {sketch_num}\n"+ cmd = (+ f"wp_sketch{sketch_num} = cq.Workplane(cq.Plane("+ f"cq.Vector({fmt(sp.origin[0])}, {fmt(sp.origin[1])}, {fmt(sp.origin[2])}), "+ f"cq.Vector({fmt(sp.x_axis[0])}, {fmt(sp.x_axis[1])}, {fmt(sp.x_axis[2])}), "+ f"cq.Vector({fmt(sp.normal[0])}, {fmt(sp.normal[1])}, {fmt(sp.normal[2])})))\n"+ )+ return cmt + cmd++ def sk_scale(extrude_scale):+ return extrude_scale / (256 / 2 * NORM_FACTOR - 1)++ def cadquery_line(x, y, cx, cy, loop_list, scale):+ if (x == cx) and (y == cy):+ return ""+ s = sk_scale(scale)+ t = -(256 / 2)+ x = (x + t) * s+ y = (y + t) * s+ if len(loop_list) == 0:+ return f".moveTo({fmt(x)}, {fmt(y)})"+ return f".lineTo({fmt(x)}, {fmt(y)})"++ def cadquery_arc(x, y, cx, cy, sweep, dir_flag, loop_list, scale):+ arc_out = get_arc(x, y, cx, cy, sweep, dir_flag, is_numerical=True)+ if arc_out is None:+ return ""+ start_point, mid_point, end_point = arc_out+ s = sk_scale(scale)+ t = -(256 / 2)+ mx = (mid_point[0] + t) * s+ my = (mid_point[1] + t) * s+ ex = (end_point[0] + t) * s+ ey = (end_point[1] + t) * s+ ccx = (cx + t) * s+ ccy = (cy + t) * s+ if len(loop_list) == 0:+ return (f".moveTo({fmt(ccx)}, {fmt(ccy)})"+ f".threePointArc(({fmt(mx)}, {fmt(my)}), ({fmt(ex)}, {fmt(ey)}))")+ return f".threePointArc(({fmt(mx)}, {fmt(my)}), ({fmt(ex)}, {fmt(ey)}))"++ def cadquery_circle(x, y, r, loop_list, scale):+ s = sk_scale(scale)+ t = -(256 / 2)+ x = (x + t) * s+ y = (y + t) * s+ r = r * s+ if len(loop_list) == 0:+ return f".moveTo({fmt(x)}, {fmt(y)}).circle({fmt(r)})"+ raise NotImplementedError("Circle with other things in loop")++ def cadquery_close_loop(loop_ops, loop_num, sketch_num):+ if len(loop_ops) > 1:+ return f"loop{loop_num}=wp_sketch{sketch_num}" + "".join(loop_ops) + ".close()\n"+ if "circle" in loop_ops[0]:+ return f"loop{loop_num}=wp_sketch{sketch_num}" + loop_ops[0] + "\n"+ if "Arc" in loop_ops[0]:+ return f"loop{loop_num}=wp_sketch{sketch_num}" + "".join(loop_ops) + ".close()\n"+ if len(loop_ops) == 1 and loop_ops[0] == "":+ raise TypeError("Empty Loop")+ raise NotImplementedError(f"1 loop something else:{loop_ops}")++ def join_loops(loops):+ return "".join(f".add(loop{l})" for l in loops)++ def cadquery_extrude(loops_joined, op, typ, dir1, dir2, sketch_num,+ repeat_sketch, repeat_loops):+ OS = EXTENT_TYPE.index("OneSideFeatureExtentType")+ SYM = EXTENT_TYPE.index("SymmetricFeatureExtentType")+ NEW = EXTRUDE_OPERATIONS.index("NewBodyFeatureOperation")+ JOIN = EXTRUDE_OPERATIONS.index("JoinFeatureOperation")+ CUT = EXTRUDE_OPERATIONS.index("CutFeatureOperation")++ def two_sided_dirs():+ if dir2 == 0:+ return None+ return dir1, -dir2++ if sketch_num == 0:+ if op in (NEW, JOIN, CUT, EXTRUDE_OPERATIONS.index("IntersectFeatureOperation")):+ if typ == OS:+ return (f"solid{sketch_num}=wp_sketch{sketch_num}{loops_joined}"+ f".extrude({fmt(dir1)})\nsolid=solid{sketch_num}\n")+ if typ == SYM:+ return (f"solid{sketch_num}=wp_sketch{sketch_num}{loops_joined}"+ f".extrude({fmt(dir1)}, both=True)\nsolid=solid{sketch_num}\n")+ td = two_sided_dirs()+ if td is None:+ return cadquery_extrude(loops_joined, op, OS, dir1, dir2,+ sketch_num, repeat_sketch, repeat_loops)+ dwn, dan = td+ return (+ f"solid{sketch_num}=wp_sketch{sketch_num}{loops_joined}.extrude({fmt(dwn)})\n"+ f"solid=solid{sketch_num}\n"+ + repeat_sketch.split("\n")[1] + "\n" + repeat_loops+ + f"solid{sketch_num}=wp_sketch{sketch_num}{loops_joined}.extrude({fmt(dan)})\n"+ f"solid=solid.union(solid{sketch_num})\n"+ )+ raise NotImplementedError("first body not newbody or join")++ # non-first bodies+ if op in (NEW, JOIN):+ combiner = "solid=solid.union(solid{n})\n"+ elif op == CUT:+ combiner = "solid=solid.cut(solid{n})\n"+ else:+ combiner = "solid=solid.intersect(solid{n})\n"++ if typ == OS:+ return (f"solid{sketch_num}=wp_sketch{sketch_num}{loops_joined}.extrude({fmt(dir1)})\n"+ + combiner.format(n=sketch_num))+ if typ == SYM:+ return (f"solid{sketch_num}=wp_sketch{sketch_num}{loops_joined}.extrude({fmt(dir1)}, both=True)\n"+ + combiner.format(n=sketch_num))+ td = two_sided_dirs()+ if td is None:+ return cadquery_extrude(loops_joined, op, OS, dir1, dir2,+ sketch_num, repeat_sketch, repeat_loops)+ dwn, dan = td+ join_or = "union" if op in (NEW, JOIN) else ("cut" if op == CUT else "intersect")+ return (+ f"solid{sketch_num}=wp_sketch{sketch_num}{loops_joined}.extrude({fmt(dwn)})\n"+ f"solid_temp=solid{sketch_num}\n"+ + repeat_sketch.split("\n")[1] + "\n" + repeat_loops+ + f"solid{sketch_num}=wp_sketch{sketch_num}{loops_joined}.extrude({fmt(dan)})\n"+ f"solid_temp=solid_temp.union(solid{sketch_num})\n"+ f"solid=solid.{join_or}(solid_temp)\n"+ )++ # ---- main assembly ----+ code = "import cadquery as cq\n"+ sketches, extrudes = split_by_sketches(vecs)++ total_loop_count = 0+ curr_x, curr_y = 128, 128++ for i, sketch in enumerate(sketches):+ extrude = extrudes[i]+ theta, phi, gamma, px, py, pz, scale, dir1, dir2, op, typ = \+ extract_extrude_params(extrude, UNQUANTIZE)++ if dir1 == 0:+ raise TypeError("Extrude is zero for dir1")+ if dir2 == 0 and typ == EXTENT_TYPE.index("TwoSidesFeatureExtentType"):+ raise TypeError("Two sided extrude with no second extrude value")++ sketch_plane = CoordSystem(np.array([px, py, pz]), theta, phi, gamma)+ sketch_plane.denumericalize(256)++ wp_code = cadquery_workplane(sketch_plane, i)+ code += wp_code++ loops = split_by_loops(sketch)+ loops_in_sketch = []+ loop_chunk = ""++ for loop in loops:+ loop_ops = []+ for sketch_op in loop:+ if sketch_op[0] == LINE:+ px2, py2 = sketch_op[1], sketch_op[2]+ loop_ops.append(cadquery_line(px2, py2, curr_x, curr_y, loop_ops, scale))+ curr_x, curr_y = px2, py2+ elif sketch_op[0] == ARC:+ ex, ey = sketch_op[1], sketch_op[2]+ loop_ops.append(cadquery_arc(ex, ey, curr_x, curr_y,+ sketch_op[3], sketch_op[4], loop_ops, scale))+ curr_x, curr_y = ex, ey+ elif sketch_op[0] == CIRCLE:+ cxx, cyy, r = sketch_op[1], sketch_op[2], sketch_op[5]+ loop_ops.append(cadquery_circle(cxx, cyy, r, loop_ops, scale))+ else:+ raise NotImplementedError("Command not implemented")++ close_cmd = cadquery_close_loop(loop_ops, total_loop_count, i)+ code += close_cmd+ loop_chunk += close_cmd+ loops_in_sketch.append(total_loop_count)+ total_loop_count += 1++ loops_joined = join_loops(loops_in_sketch)+ code += cadquery_extrude(loops_joined, op, typ, dir1, dir2, i, wp_code, loop_chunk)++ code += (+ "\nif __name__ == '__main__':\n"+ " import sys\n"+ " out = sys.argv[1] if len(sys.argv) > 1 else '/tmp/out.step'\n"+ " cq.exporters.export(solid, out)\n"+ )+ return code+++def extract_h5(path):+ with h5py.File(path, "r") as f:+ key = list(f.keys())[0]+ return f[key][()]+++if __name__ == "__main__":+ src = sys.argv[1] # path to .h5+ dest = sys.argv[2] # path to output .py+ vec = extract_h5(src)+ code = vec_to_cadquery_code(vec)+ with open(dest, "w") as f:+ f.write(code)+ print(f"wrote {dest}")diff --git a/toolkits/curation/deepcad/cq_check.py b/toolkits/curation/deepcad/cq_check.pynew file mode 100644index 00000000..87624eba--- /dev/null+++ b/toolkits/curation/deepcad/cq_check.py@@ -0,0 +1,66 @@+"""+cq_check.py -- per-model validation worker (run in a subprocess).++Executes a generated CadQuery script (without its __main__ export block),+computes exact volume + topological validity via OpenCASCADE, and exports the+STEP file for the corpus. Prints a single JSON line with the result.++Statuses: OK / INVALID / DEGENERATE / EXEC_FAIL / GEOM_FAIL / EXPORT_FAIL+"""+import sys+import os+import json+import runpy+++def main():+ py_path, step_out = sys.argv[1], sys.argv[2]++ # 1. Execute script body (run_name != __main__ skips its export block)+ try:+ ns = runpy.run_path(py_path, run_name="_cqcheck_")+ except Exception as e:+ print(json.dumps({"status": "EXEC_FAIL",+ "detail": f"{type(e).__name__}: {e}"[:300]}))+ return++ solid = ns.get("solid")+ if solid is None:+ print(json.dumps({"status": "EXEC_FAIL", "detail": "no `solid` variable"}))+ return++ # 2. OCC-native volume + validity+ try:+ shape = solid.val()+ from OCP.GProp import GProp_GProps+ from OCP.BRepGProp import BRepGProp+ from OCP.BRepCheck import BRepCheck_Analyzer+ props = GProp_GProps()+ BRepGProp.VolumeProperties_s(shape.wrapped, props)+ vol = abs(props.Mass())+ valid = bool(BRepCheck_Analyzer(shape.wrapped).IsValid())+ except Exception as e:+ print(json.dumps({"status": "GEOM_FAIL",+ "detail": f"{type(e).__name__}: {e}"[:300]}))+ return++ if vol < 1e-9:+ print(json.dumps({"status": "DEGENERATE", "detail": f"volume={vol:.3e}"}))+ return++ # 3. Export STEP for the corpus+ try:+ import cadquery as cq+ os.makedirs(os.path.dirname(step_out), exist_ok=True)+ cq.exporters.export(solid, step_out)+ except Exception as e:+ print(json.dumps({"status": "EXPORT_FAIL",+ "detail": str(e)[:200], "volume": vol}))+ return++ print(json.dumps({"status": "OK" if valid else "INVALID",+ "detail": f"volume={vol:.4f}", "volume": vol}))+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/cq_check_batch.py b/toolkits/curation/deepcad/cq_check_batch.pynew file mode 100644index 00000000..3e50030b--- /dev/null+++ b/toolkits/curation/deepcad/cq_check_batch.py@@ -0,0 +1,77 @@+"""+cq_check_batch.py -- batch validation worker.+Imports cadquery ONCE, then processes many models from a manifest file+(one "py_path<TAB>step_out" per line), printing one JSON line per model.++This amortizes the ~1.5s cadquery import over the whole batch instead of+paying it per model. Per-model SIGALRM timeout guards against hangs.+A hard crash (OCC segfault) kills the process; the driver detects the+missing tail of output and retries those models in isolation.+"""+import sys+import os+import json+import signal+import runpy+++class _Timeout(Exception):+ pass+++def _alarm(signum, frame):+ raise _Timeout()+++signal.signal(signal.SIGALRM, _alarm)++# Import heavy deps once.+import cadquery as cq # noqa: E402+from OCP.GProp import GProp_GProps # noqa: E402+from OCP.BRepGProp import BRepGProp # noqa: E402+from OCP.BRepCheck import BRepCheck_Analyzer # noqa: E402+++def check_one(py_path, step_out, timeout=25):+ try:+ signal.alarm(timeout)+ ns = runpy.run_path(py_path, run_name="_cqcheck_")+ solid = ns.get("solid")+ if solid is None:+ signal.alarm(0)+ return "EXEC_FAIL", "no `solid` variable"+ shape = solid.val()+ props = GProp_GProps()+ BRepGProp.VolumeProperties_s(shape.wrapped, props)+ vol = abs(props.Mass())+ if vol < 1e-9:+ signal.alarm(0)+ return "DEGENERATE", f"volume={vol:.3e}"+ valid = bool(BRepCheck_Analyzer(shape.wrapped).IsValid())+ os.makedirs(os.path.dirname(step_out), exist_ok=True)+ cq.exporters.export(solid, step_out)+ signal.alarm(0)+ return ("OK" if valid else "INVALID"), f"volume={vol:.4f}"+ except _Timeout:+ return "TIMEOUT", ""+ except Exception as e:+ signal.alarm(0)+ return "EXEC_FAIL", f"{type(e).__name__}: {e}"[:200]+++def main():+ manifest = sys.argv[1]+ for line in open(manifest):+ line = line.rstrip("\n")+ if not line:+ continue+ py_path, step_out = line.split("\t")+ sub = os.path.basename(os.path.dirname(py_path))+ name = os.path.splitext(os.path.basename(py_path))[0]+ status, detail = check_one(py_path, step_out)+ print(json.dumps({"id": f"{sub}/{name}", "status": status, "detail": detail}),+ flush=True)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/dedup_check.py b/toolkits/curation/deepcad/dedup_check.pynew file mode 100644index 00000000..adea7671--- /dev/null+++ b/toolkits/curation/deepcad/dedup_check.py@@ -0,0 +1,109 @@+"""+dedup_check.py+Compute a geometry fingerprint (volume, surface area, inertia-trace) for each+STEP and flag near-duplicates. Batched like validate_fast (import cadquery once+per worker). Writes duplicates.json: {kept_id: [dup_id, ...]}.++Fingerprint is rounded so numerically-identical solids collide; this catches+exact/near duplicates within the batch (and can be diffed against an existing+corpus's fingerprints later).+"""+import os+import sys+import json+import argparse+import tempfile+import subprocess+from pathlib import Path+from collections import defaultdict+from joblib import Parallel, delayed+from tqdm import tqdm++WORKER_SRC = '''+import sys, json, os+import cadquery as cq+from OCP.GProp import GProp_GProps+from OCP.BRepGProp import BRepGProp+for line in open(sys.argv[1]):+ step = line.strip()+ if not step: continue+ sub = os.path.basename(os.path.dirname(step))+ name = os.path.splitext(os.path.basename(step))[0]+ try:+ shape = cq.importers.importStep(step).val()+ vp = GProp_GProps(); BRepGProp.VolumeProperties_s(shape.wrapped, vp)+ sp = GProp_GProps(); BRepGProp.SurfaceProperties_s(shape.wrapped, sp)+ mat = vp.MatrixOfInertia()+ itr = mat.Value(1,1)+mat.Value(2,2)+mat.Value(3,3)+ fp = [round(abs(vp.Mass()),4), round(sp.Mass(),4), round(itr,4)]+ print(json.dumps({"id": f"{sub}/{name}", "fp": fp}), flush=True)+ except Exception as e:+ print(json.dumps({"id": f"{sub}/{name}", "fp": None}), flush=True)+'''+++def run_chunk(steps, py_exe, worker_path):+ with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False) as mf:+ mf.write("\n".join(str(s) for s in steps))+ manifest = mf.name+ out = []+ try:+ pr = subprocess.run([py_exe, worker_path, manifest],+ capture_output=True, text=True,+ timeout=30 * len(steps) + 60)+ for line in pr.stdout.splitlines():+ line = line.strip()+ if line:+ try:+ r = json.loads(line)+ out.append((r["id"], tuple(r["fp"]) if r["fp"] else None))+ except Exception:+ pass+ except subprocess.TimeoutExpired:+ pass+ finally:+ os.unlink(manifest)+ return out+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--step_dir", required=True)+ ap.add_argument("--workers", type=int, default=40)+ ap.add_argument("--chunk", type=int, default=64)+ ap.add_argument("--python", default=sys.executable)+ ap.add_argument("--out", default="duplicates.json")+ args = ap.parse_args()++ worker_path = str(Path(tempfile.gettempdir()) / "_dedup_worker.py")+ Path(worker_path).write_text(WORKER_SRC)++ steps = sorted(Path(args.step_dir).rglob("*.step"))+ chunks = [steps[i:i + args.chunk] for i in range(0, len(steps), args.chunk)]+ print(f"Fingerprinting {len(steps)} STEP files in {len(chunks)} chunks...")++ chunk_results = Parallel(n_jobs=args.workers)(+ delayed(run_chunk)(c, args.python, worker_path) for c in tqdm(chunks)+ )++ seen = {}+ dups = defaultdict(list)+ n_bad = 0+ for chunk in chunk_results:+ for mid, fp in chunk:+ if fp is None:+ n_bad += 1+ continue+ if fp in seen:+ dups[seen[fp]].append(mid)+ else:+ seen[fp] = mid++ total_dups = sum(len(v) for v in dups.values())+ print(f"Unique fingerprints: {len(seen)}, duplicate models: {total_dups}, "+ f"unreadable: {n_bad}")+ json.dump(dups, open(args.out, "w"), indent=2)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/dedup_fast.py b/toolkits/curation/deepcad/dedup_fast.pynew file mode 100644index 00000000..2fdcdb37--- /dev/null+++ b/toolkits/curation/deepcad/dedup_fast.py@@ -0,0 +1,101 @@+"""+dedup_fast.py+Geometry-fingerprint dedup using the already-generated ground-truth STLs+(trimesh reads STL directly -- far faster than re-importing STEP).++Fingerprint = (volume, area, inertia-trace), rounded. Exact-match collisions+are flagged as duplicates. Writes duplicates.json: {kept_id: [dup_id,...]}.+"""+import os+import sys+import json+import argparse+import tempfile+import subprocess+from pathlib import Path+from collections import defaultdict+from joblib import Parallel, delayed+from tqdm import tqdm++WORKER_SRC = r'''+import sys, os, json+import numpy as np+import trimesh+for line in open(sys.argv[1]):+ stl = line.strip()+ if not stl: continue+ sub=os.path.basename(os.path.dirname(stl)); name=os.path.splitext(os.path.basename(stl))[0]+ mid=f"{sub}/{name}"+ try:+ m = trimesh.load(stl, force="mesh")+ vol=round(abs(float(m.volume)),4); area=round(float(m.area),4)+ itr=round(float(np.trace(m.moment_inertia)),4)+ print(json.dumps({"id":mid,"fp":[vol,area,itr]}), flush=True)+ except Exception as e:+ print(json.dumps({"id":mid,"fp":None}), flush=True)+'''+++def run_chunk(ids, stl_dir, py_exe, worker_path):+ lines = [str(Path(stl_dir) / mid.split("/")[0] / (mid.split("/")[1] + ".stl")) for mid in ids]+ with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False) as mf:+ mf.write("\n".join(lines))+ manifest = mf.name+ res = []+ try:+ pr = subprocess.run([py_exe, worker_path, manifest], capture_output=True,+ text=True, timeout=10 * len(ids) + 60)+ for line in pr.stdout.splitlines():+ line = line.strip()+ if line:+ try:+ r = json.loads(line)+ res.append((r["id"], tuple(r["fp"]) if r["fp"] else None))+ except Exception:+ pass+ except subprocess.TimeoutExpired:+ pass+ finally:+ os.unlink(manifest)+ return res+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--stl_dir", required=True)+ ap.add_argument("--ids", required=True)+ ap.add_argument("--workers", type=int, default=48)+ ap.add_argument("--chunk", type=int, default=64)+ ap.add_argument("--python", default=sys.executable)+ ap.add_argument("--out", default="duplicates.json")+ args = ap.parse_args()++ ids = [l.strip() for l in open(args.ids) if l.strip()]+ worker_path = str(Path(tempfile.gettempdir()) / "_dedupfast_worker.py")+ Path(worker_path).write_text(WORKER_SRC)+ chunks = [ids[i:i + args.chunk] for i in range(0, len(ids), args.chunk)]+ print(f"Fingerprinting {len(ids)} models in {len(chunks)} chunks...")++ results = Parallel(n_jobs=args.workers)(+ delayed(run_chunk)(c, args.stl_dir, args.python, worker_path) for c in tqdm(chunks)+ )+ seen = {}+ dups = defaultdict(list)+ n_bad = 0+ for chunk in results:+ for mid, fp in chunk:+ if fp is None:+ n_bad += 1+ continue+ if fp in seen:+ dups[seen[fp]].append(mid)+ else:+ seen[fp] = mid+ total = sum(len(v) for v in dups.values())+ print(f"Unique: {len(seen)}, duplicate models: {total}, unreadable: {n_bad}")+ print(f"Dup groups (>=1 dup): {len(dups)}")+ json.dump(dict(dups), open(args.out, "w"))+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/dedup_vec.py b/toolkits/curation/deepcad/dedup_vec.pynew file mode 100644index 00000000..c71ea3f4--- /dev/null+++ b/toolkits/curation/deepcad/dedup_vec.py@@ -0,0 +1,101 @@+"""+dedup_vec.py+Correct dedup: two models are TRUE duplicates iff their source DeepCAD vec is+identical (deterministic conversion => identical code & geometry). Geometry-only+fingerprints over-merge distinct designs that happen to share volume/area, so we+key on the exact quantized command sequence instead.++Writes duplicates_vec.json: {kept_id: [dup_id, ...]}.+"""+import os+import sys+import json+import hashlib+import argparse+import tempfile+import subprocess+from pathlib import Path+from collections import defaultdict+from joblib import Parallel, delayed+from tqdm import tqdm++WORKER_SRC = r'''+import sys, os, json, hashlib, h5py+for line in open(sys.argv[1]):+ h5 = line.strip()+ if not h5: continue+ sub=os.path.basename(os.path.dirname(h5)); name=os.path.splitext(os.path.basename(h5))[0]+ mid=f"{sub}/{name}"+ try:+ with h5py.File(h5,"r") as f:+ arr = f[list(f.keys())[0]][()]+ print(json.dumps({"id":mid,"h":hashlib.md5(arr.tobytes()).hexdigest()}), flush=True)+ except Exception:+ print(json.dumps({"id":mid,"h":None}), flush=True)+'''+++def run_chunk(ids, vec_dir, py_exe, worker_path):+ lines = [str(Path(vec_dir) / mid.split("/")[0] / (mid.split("/")[1] + ".h5")) for mid in ids]+ with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False) as mf:+ mf.write("\n".join(lines))+ manifest = mf.name+ res = []+ try:+ pr = subprocess.run([py_exe, worker_path, manifest], capture_output=True,+ text=True, timeout=10 * len(ids) + 60)+ for line in pr.stdout.splitlines():+ line = line.strip()+ if line:+ try:+ r = json.loads(line)+ res.append((r["id"], r["h"]))+ except Exception:+ pass+ except subprocess.TimeoutExpired:+ pass+ finally:+ os.unlink(manifest)+ return res+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--vec_dir", required=True)+ ap.add_argument("--ids", required=True)+ ap.add_argument("--workers", type=int, default=48)+ ap.add_argument("--chunk", type=int, default=128)+ ap.add_argument("--python", default=sys.executable)+ ap.add_argument("--out", default="duplicates_vec.json")+ args = ap.parse_args()++ ids = [l.strip() for l in open(args.ids) if l.strip()]+ worker_path = str(Path(tempfile.gettempdir()) / "_dedupvec_worker.py")+ Path(worker_path).write_text(WORKER_SRC)+ chunks = [ids[i:i + args.chunk] for i in range(0, len(ids), args.chunk)]+ print(f"Hashing {len(ids)} vec sequences in {len(chunks)} chunks...")++ results = Parallel(n_jobs=args.workers)(+ delayed(run_chunk)(c, args.vec_dir, args.python, worker_path) for c in tqdm(chunks)+ )+ seen = {}+ dups = defaultdict(list)+ n_bad = 0+ for chunk in results:+ for mid, h in chunk:+ if h is None:+ n_bad += 1+ continue+ if h in seen:+ dups[seen[h]].append(mid)+ else:+ seen[h] = mid+ total = sum(len(v) for v in dups.values())+ print(f"Unique vec sequences: {len(seen)}, TRUE duplicate models: {total}, "+ f"unreadable: {n_bad}")+ print(f"Duplicate groups: {len(dups)}")+ json.dump(dict(dups), open(args.out, "w"))+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/finalize.py b/toolkits/curation/deepcad/finalize.pynew file mode 100644index 00000000..0bd24443--- /dev/null+++ b/toolkits/curation/deepcad/finalize.py@@ -0,0 +1,85 @@+"""+finalize.py+Two non-destructive post-processing passes on the corpus:++1. Fidelity tier : metadata.fidelity_tier from IoU+ high iou >= 0.9+ medium 0.5 <= iou < 0.9+ low iou < 0.5+ unverified iou is null (no ground-truth mesh)+ plus metadata.low_fidelity (bool) for quick filtering.++2. Leakage-free split : metadata.split_dedup -- every member of a vec-duplicate+ group is reassigned to the canonical member's split, so no vec spans >1 split.+ Original DeepCAD split is preserved in metadata.split.++Writes a new JSONL and reports the resulting split sizes + a zero-leakage check.+"""+import json+import argparse+from collections import defaultdict+++def tier(iou):+ if iou is None:+ return "unverified"+ if iou < 0.5:+ return "low"+ if iou < 0.9:+ return "medium"+ return "high"+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--jsonl", required=True)+ ap.add_argument("--dups", required=True)+ ap.add_argument("--out", required=True)+ args = ap.parse_args()++ dups = json.load(open(args.dups)) # {canonical: [dup,...]}+ canon_of = {} # member -> canonical+ for canon, members in dups.items():+ canon_of[canon] = canon+ for m in members:+ canon_of[m] = canon++ # pass 1: original split per id+ split_of = {}+ with open(args.jsonl) as f:+ for line in f:+ e = json.loads(line)+ split_of[e["metadata"]["deepcad_id"]] = e["metadata"]["split"]++ def group_split(did):+ c = canon_of.get(did, did)+ return split_of.get(c, split_of[did])++ counts = defaultdict(int)+ tiers = defaultdict(int)+ # group-level split for leakage verification+ group_splits = defaultdict(set)++ with open(args.jsonl) as f, open(args.out, "w") as out:+ for line in f:+ e = json.loads(line)+ md = e["metadata"]+ did = md["deepcad_id"]+ new_split = group_split(did)+ md["split_dedup"] = new_split+ t = tier(md.get("iou"))+ md["fidelity_tier"] = t+ md["low_fidelity"] = (t in ("low", "unverified"))+ counts[new_split] += 1+ tiers[t] += 1+ group_splits[canon_of.get(did, did)].add(new_split)+ out.write(json.dumps(e) + "\n")++ leak = sum(1 for s in group_splits.values() if len(s) > 1)+ print("Split_dedup sizes:", dict(counts))+ print("Fidelity tiers:", dict(tiers))+ print(f"Cross-split groups AFTER re-split: {leak} (should be 0)")+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/gen_gt_stl.py b/toolkits/curation/deepcad/gen_gt_stl.pynew file mode 100644index 00000000..1a951649--- /dev/null+++ b/toolkits/curation/deepcad/gen_gt_stl.py@@ -0,0 +1,124 @@+"""+gen_gt_stl.py -- run in the `gt` (pythonocc-core) env.++Generate DeepCAD's OFFICIAL ground-truth mesh for each model, straight from the+same cad_vec .h5 we converted, using cadlib.visualize.vec2CADsolid (OCC.Core).+This is the authoritative reference geometry to compare our CadQuery output+against (IoU). Batched: one worker imports OCC once and processes a chunk.++Usage:+ python gen_gt_stl.py --vec_dir /data/.../cad_vec --ids ids.txt \+ --out_dir /data/.../gt_stl --workers 40 --chunk 32 --deepcad /data/.../DeepCAD+"""+import os+import sys+import json+import argparse+import tempfile+import subprocess+from pathlib import Path+from joblib import Parallel, delayed+from tqdm import tqdm++WORKER_SRC = r'''+import sys, os, json, h5py, numpy as np+sys.path.insert(0, os.environ["DEEPCAD_ROOT"])+# numpy compat shim: DeepCAD's cadlib references np.int in numericalize (unused+# on this path), but guard anyway for newer numpy.+if not hasattr(np, "int"):+ np.int = int+from cadlib.visualize import vec2CADsolid+from OCC.Extend.DataExchange import write_stl_file++for line in open(sys.argv[1]):+ line = line.rstrip("\n")+ if not line:+ continue+ h5path, out = line.split("\t")+ sub = os.path.basename(os.path.dirname(h5path))+ name = os.path.splitext(os.path.basename(h5path))[0]+ mid = f"{sub}/{name}"+ try:+ with h5py.File(h5path, "r") as f:+ vec = f[list(f.keys())[0]][()]+ shape = vec2CADsolid(vec)+ os.makedirs(os.path.dirname(out), exist_ok=True)+ write_stl_file(shape, out, linear_deflection=0.01, angular_deflection=0.3)+ ok = os.path.exists(out) and os.path.getsize(out) > 0+ print(json.dumps({"id": mid, "status": "OK" if ok else "EMPTY"}), flush=True)+ except Exception as e:+ print(json.dumps({"id": mid, "status": "FAIL", "detail": str(e)[:150]}), flush=True)+'''+++def run_chunk(ids, vec_dir, out_dir, py_exe, worker_path):+ lines = []+ for mid in ids:+ sub, name = mid.split("/")+ h5 = str(Path(vec_dir) / sub / (name + ".h5"))+ out = str(Path(out_dir) / sub / (name + ".stl"))+ lines.append(f"{h5}\t{out}")+ with tempfile.NamedTemporaryFile("w", suffix=".tsv", delete=False) as mf:+ mf.write("\n".join(lines))+ manifest = mf.name+ res = []+ try:+ pr = subprocess.run([py_exe, worker_path, manifest], capture_output=True,+ text=True, timeout=40 * len(ids) + 60)+ for line in pr.stdout.splitlines():+ line = line.strip()+ if line:+ try:+ r = json.loads(line)+ res.append((r["id"], r["status"]))+ except Exception:+ pass+ except subprocess.TimeoutExpired:+ pass+ finally:+ os.unlink(manifest)+ done = {r[0] for r in res}+ for mid in ids:+ if mid not in done:+ res.append((mid, "CRASH"))+ return res+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--vec_dir", required=True)+ ap.add_argument("--ids", required=True, help="text file, one model_id per line")+ ap.add_argument("--out_dir", required=True)+ ap.add_argument("--deepcad", required=True, help="DeepCAD repo root")+ ap.add_argument("--workers", type=int, default=40)+ ap.add_argument("--chunk", type=int, default=32)+ ap.add_argument("--python", default=sys.executable)+ ap.add_argument("--out", default="gt_gen_report.json")+ ap.add_argument("--limit", type=int, default=0)+ args = ap.parse_args()++ os.environ["DEEPCAD_ROOT"] = args.deepcad+ ids = [l.strip() for l in open(args.ids) if l.strip()]+ if args.limit:+ ids = ids[: args.limit]+ worker_path = str(Path(tempfile.gettempdir()) / "_gtgen_worker.py")+ Path(worker_path).write_text(WORKER_SRC)++ chunks = [ids[i:i + args.chunk] for i in range(0, len(ids), args.chunk)]+ print(f"Generating GT STL for {len(ids)} models in {len(chunks)} chunks...")+ results = Parallel(n_jobs=args.workers)(+ delayed(run_chunk)(c, args.vec_dir, args.out_dir, args.python, worker_path)+ for c in tqdm(chunks)+ )+ flat = [r for chunk in results for r in chunk]+ buckets = {}+ for _, st in flat:+ buckets[st] = buckets.get(st, 0) + 1+ print("GT generation:", buckets)+ json.dump({"buckets": buckets,+ "fail": [r[0] for r in flat if r[1] != "OK"]},+ open(args.out, "w"), indent=2)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/iou_compute.py b/toolkits/curation/deepcad/iou_compute.pynew file mode 100644index 00000000..040eadc4--- /dev/null+++ b/toolkits/curation/deepcad/iou_compute.py@@ -0,0 +1,147 @@+"""+iou_compute.py -- run in the cadquery env.++For each model, compare OUR converted geometry against DeepCAD's official+ground-truth mesh:+ - ours: tessellate our STEP (cadquery/OCP)+ - gt: load DeepCAD's STL (generated by gen_gt_stl.py)+ - metrics: voxel IoU (occupancy, aligned to a shared grid) + Chamfer distance+ (surface point samples, scale-normalized).++High IoU => our CadQuery conversion faithfully reproduces DeepCAD's geometry.+Writes iou_results.json: {id: {iou, chamfer, ours_vol, gt_vol}}.+"""+import os+import sys+import json+import argparse+import tempfile+import subprocess+from pathlib import Path+from joblib import Parallel, delayed+from tqdm import tqdm++WORKER_SRC = r'''+import sys, os, json+import numpy as np+import trimesh+import cadquery as cq++def step_mesh(step):+ shape = cq.importers.importStep(step).val()+ v, f = shape.tessellate(0.05)+ return trimesh.Trimesh(np.array([[p.x,p.y,p.z] for p in v]), np.array(f), process=False)++def voxel_iou(m1, m2, res=48):+ lo = np.minimum(m1.bounds[0], m2.bounds[0])+ hi = np.maximum(m1.bounds[1], m2.bounds[1])+ pitch = max(float((hi-lo).max())/res, 1e-6)+ def occ(m):+ try:+ v = m.voxelized(pitch).fill()+ except Exception:+ v = m.voxelized(pitch)+ idx = np.round((v.points - lo)/pitch).astype(int)+ return set(map(tuple, idx.tolist()))+ s1, s2 = occ(m1), occ(m2)+ u = len(s1 | s2)+ return (len(s1 & s2)/u) if u else 0.0++def chamfer(m1, m2, n=2000):+ p1 = m1.sample(n); p2 = m2.sample(n)+ # scale by gt bbox diagonal so it's comparable across models+ scale = float(np.linalg.norm(m2.bounds[1]-m2.bounds[0])) or 1.0+ from scipy.spatial import cKDTree+ d12 = cKDTree(p2).query(p1)[0]; d21 = cKDTree(p1).query(p2)[0]+ return float((d12.mean()+d21.mean())/2.0/scale)++for line in open(sys.argv[1]):+ line=line.rstrip("\n")+ if not line: continue+ step, gt = line.split("\t")+ sub=os.path.basename(os.path.dirname(step)); name=os.path.splitext(os.path.basename(step))[0]+ mid=f"{sub}/{name}"+ try:+ mo = step_mesh(step)+ mg = trimesh.load(gt, force="mesh")+ iou = voxel_iou(mo, mg)+ cd = chamfer(mo, mg)+ out = {"id": mid, "iou": round(iou,4), "chamfer": round(cd,5),+ "ours_vol": round(float(mo.volume),4), "gt_vol": round(float(mg.volume),4)}+ print(json.dumps(out), flush=True)+ except Exception as e:+ print(json.dumps({"id": mid, "iou": None, "detail": str(e)[:150]}), flush=True)+'''+++def run_chunk(ids, step_dir, gt_dir, py_exe, worker_path):+ lines = []+ for mid in ids:+ sub, name = mid.split("/")+ step = str(Path(step_dir) / sub / (name + ".step"))+ gt = str(Path(gt_dir) / sub / (name + ".stl"))+ lines.append(f"{step}\t{gt}")+ with tempfile.NamedTemporaryFile("w", suffix=".tsv", delete=False) as mf:+ mf.write("\n".join(lines))+ manifest = mf.name+ res = []+ try:+ pr = subprocess.run([py_exe, worker_path, manifest], capture_output=True,+ text=True, timeout=30 * len(ids) + 60)+ for line in pr.stdout.splitlines():+ line = line.strip()+ if line:+ try:+ res.append(json.loads(line))+ except Exception:+ pass+ except subprocess.TimeoutExpired:+ pass+ finally:+ os.unlink(manifest)+ done = {r["id"] for r in res}+ for mid in ids:+ if mid not in done:+ res.append({"id": mid, "iou": None, "detail": "crash"})+ return res+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--step_dir", required=True)+ ap.add_argument("--gt_dir", required=True)+ ap.add_argument("--ids", required=True)+ ap.add_argument("--workers", type=int, default=40)+ ap.add_argument("--chunk", type=int, default=32)+ ap.add_argument("--python", default=sys.executable)+ ap.add_argument("--out", default="iou_results.json")+ ap.add_argument("--limit", type=int, default=0)+ args = ap.parse_args()++ ids = [l.strip() for l in open(args.ids) if l.strip()]+ if args.limit:+ ids = ids[: args.limit]+ worker_path = str(Path(tempfile.gettempdir()) / "_iou_worker.py")+ Path(worker_path).write_text(WORKER_SRC)++ chunks = [ids[i:i + args.chunk] for i in range(0, len(ids), args.chunk)]+ print(f"Computing IoU for {len(ids)} models in {len(chunks)} chunks...")+ results = Parallel(n_jobs=args.workers)(+ delayed(run_chunk)(c, args.step_dir, args.gt_dir, args.python, worker_path)+ for c in tqdm(chunks)+ )+ flat = {r["id"]: r for chunk in results for r in chunk}+ ious = [r["iou"] for r in flat.values() if r.get("iou") is not None]+ import statistics+ if ious:+ ious_sorted = sorted(ious)+ print(f"IoU: n={len(ious)} mean={statistics.mean(ious):.3f} "+ f"median={statistics.median(ious):.3f} "+ f">0.9: {100*sum(i>0.9 for i in ious)/len(ious):.1f}% "+ f">0.7: {100*sum(i>0.7 for i in ious)/len(ious):.1f}% "+ f"<0.5: {100*sum(i<0.5 for i in ious)/len(ious):.1f}%")+ json.dump(flat, open(args.out, "w"))+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/mark_dedup.py b/toolkits/curation/deepcad/mark_dedup.pynew file mode 100644index 00000000..8415bc35--- /dev/null+++ b/toolkits/curation/deepcad/mark_dedup.py@@ -0,0 +1,70 @@+"""+mark_dedup.py+Annotate the JSONL with dedup info (non-destructive) and report cross-split+leakage (the same vec appearing in both train and test/val is a real ML hazard).++Adds to metadata:+ is_canonical : bool (representative of its vec group)+ duplicate_of : str (deepcad_id of the canonical, if this is a duplicate)+"""+import json+import argparse+from collections import defaultdict+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--jsonl", required=True)+ ap.add_argument("--dups", required=True)+ ap.add_argument("--out", required=True)+ args = ap.parse_args()++ dups = json.load(open(args.dups)) # {canonical: [dup,...]}+ dup_to_canon = {}+ group_of = {}+ for canon, members in dups.items():+ group_of.setdefault(canon, canon)+ for m in members:+ dup_to_canon[m] = canon+ group_of[m] = canon++ # First pass: collect split per deepcad_id for leakage analysis+ split_of = {}+ with open(args.jsonl) as f:+ for line in f:+ e = json.loads(line)+ split_of[e["metadata"]["deepcad_id"]] = e["metadata"]["split"]++ # leakage: groups whose members span >1 split+ group_splits = defaultdict(set)+ all_ids = set(split_of)+ for did in all_ids:+ canon = group_of.get(did, did)+ group_splits[canon].add(split_of[did])+ leak_groups = {g: s for g, s in group_splits.items() if len(s) > 1}+ leaked_models = sum(1 for did in all_ids+ if len(group_splits[group_of.get(did, did)]) > 1)++ n_canon = 0+ n_dup = 0+ with open(args.jsonl) as f, open(args.out, "w") as out:+ for line in f:+ e = json.loads(line)+ did = e["metadata"]["deepcad_id"]+ is_canon = did not in dup_to_canon+ e["metadata"]["is_canonical"] = is_canon+ e["metadata"]["duplicate_of"] = dup_to_canon.get(did)+ if is_canon:+ n_canon += 1+ else:+ n_dup += 1+ out.write(json.dumps(e) + "\n")++ print(f"Annotated: {n_canon} canonical, {n_dup} duplicates")+ print(f"Cross-split leakage: {len(leak_groups)} vec-groups span multiple "+ f"splits, affecting {leaked_models} models")+ print(f" -> dedup-to-canonical leaves {n_canon} unique models")+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/pack_tar_parquet.py b/toolkits/curation/deepcad/pack_tar_parquet.pynew file mode 100644index 00000000..08acd5cf--- /dev/null+++ b/toolkits/curation/deepcad/pack_tar_parquet.py@@ -0,0 +1,177 @@+"""+pack_tar_parquet.py (v2 — spec-conformant, matches ilabai/3dcodeverse Storage Rules)++Emits the official chunked tar+parquet layout:++ deepcad/v1/ metadata.parquet (<20MB) + samples-000.tar (<=2.5GB) [+ 001...]+ deepcad/v1_1/ ...+ deepcad/v1_N/ ...++Rules honored:+- FLAT chunk folders; split a too-big subset into sibling folders v1, v1_1, v1_2...+ when a chunk's metadata approaches the size cap.+- metadata.parquet columns EXACTLY: id, key, name, captions{detailed,instruction,factory},+ meta_json, code, tar, byte_start, byte_len, n_files (NO curation_json).+- Subset extras (Text2CAD text, IoU fidelity, dedup/splits) are folded INTO meta_json+ (§7 "subsets add their own keys"), so they're visible in the standard meta_json column.+- `id` = "deepcad/v1/<key>" (logical subset, no chunk suffix).+- `tar` = repo-root-relative path "deepcad/<folder>/samples-NNN.tar".+- captions left empty (maintainer captions).+- plain uncompressed .tar; a sample's files are consecutive; byte_start/len is a sub-tar.++Reads the canonical dir tree produced by to_3dcodeverse.py (meta.json + curation.json++ code.py + renders/).+"""+import io+import os+import json+import tarfile+import argparse+from pathlib import Path++import pyarrow as pa+import pyarrow.parquet as pq++PARQUET_SCHEMA = pa.schema([+ ("id", pa.string()), ("key", pa.string()), ("name", pa.string()),+ ("captions", pa.struct([("detailed", pa.string()),+ ("instruction", pa.string()),+ ("factory", pa.string())])),+ ("meta_json", pa.string()), ("code", pa.string()), ("tar", pa.string()),+ ("byte_start", pa.int64()), ("byte_len", pa.int64()), ("n_files", pa.int32()),+])+EMPTY_CAP = {"detailed": "", "instruction": "", "factory": ""}+RENDER_ORDER = ["view_00.png", "view_01.png", "view_02.png", "view_03.png",+ "object.glb", "model.step"]+++def add_bytes(tar, arcname, data):+ ti = tarfile.TarInfo(arcname)+ ti.size = len(data)+ tar.addfile(ti, io.BytesIO(data))+++def build_meta_json(d, key, subset):+ """meta.json with subset extras (text2cad/fidelity/splits) folded in."""+ meta = json.loads((d / "meta.json").read_text())+ meta["id"] = f"deepcad/{subset}/{key}" # logical subset, no chunk suffix+ meta["source_url"] = "http://www.cs.columbia.edu/cg/deepcad/data.tar"+ cur = json.loads((d / "curation.json").read_text()) if (d / "curation.json").exists() else {}+ meta["text2cad"] = cur.get("text_text2cad", {}) # <- Text2CAD now here (visible)+ meta["text_license"] = cur.get("text_license")+ meta["fidelity"] = cur.get("fidelity", {}) # iou / chamfer / tier / quality+ meta["splits"] = cur.get("splits", {}) # split_original/dedup/is_canonical/dup_of+ meta["provenance"] = cur.get("provenance")+ return meta+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--corpus", required=True, help=".../corpus_3dcv/deepcad")+ ap.add_argument("--out", required=True)+ ap.add_argument("--subset", default="v1")+ ap.add_argument("--meta_raw_mb", type=int, default=38,+ help="raw code+meta bytes per chunk before split (~2.5x -> parquet MB)")+ ap.add_argument("--tar_gb", type=float, default=2.0, help="max bytes per tar within a chunk")+ ap.add_argument("--limit", type=int, default=0)+ args = ap.parse_args()++ corpus = Path(args.corpus)+ out = Path(args.out)+ out.mkdir(parents=True, exist_ok=True)+ RAW_LIMIT = args.meta_raw_mb * 1024 * 1024+ TAR_LIMIT = int(args.tar_gb * 1024 * 1024 * 1024)+ subset = args.subset++ samples = []+ for bucket in sorted(corpus.iterdir()):+ if bucket.is_dir():+ for d in sorted(bucket.iterdir()):+ if d.is_dir():+ samples.append(d)+ if args.limit:+ samples = samples[: args.limit]+ print(f"Packing {len(samples)} samples -> chunks (meta cap {args.meta_raw_mb}MB raw, "+ f"tar cap {args.tar_gb}GB)")++ chunk_idx = 0 # 0 -> v1, 1 -> v1_1, ...+ rows = []+ chunk_raw = 0+ tar_idx = 0+ tar = None+ chunk_dir = None+ n_chunks_written = 0++ def folder_name(ci):+ return subset if ci == 0 else f"{subset}_{ci}"++ def open_chunk(ci):+ nonlocal tar, tar_idx, chunk_dir+ chunk_dir = out / folder_name(ci)+ chunk_dir.mkdir(parents=True, exist_ok=True)+ tar_idx = 0+ tar = tarfile.open(chunk_dir / f"samples-{tar_idx:03d}.tar", "w")++ def close_chunk(ci, chunk_rows):+ nonlocal tar, n_chunks_written+ if tar is not None:+ tar.close()+ pq.write_table(pa.Table.from_pylist(chunk_rows, schema=PARQUET_SCHEMA),+ chunk_dir / "metadata.parquet", compression="zstd")+ n_chunks_written += 1+ sz = (chunk_dir / "metadata.parquet").stat().st_size / 1e6+ print(f" {folder_name(ci)}: {len(chunk_rows)} rows, metadata {sz:.1f} MB")++ open_chunk(chunk_idx)+ for d in samples:+ key = d.name+ meta = build_meta_json(d, key, subset)+ meta_json = json.dumps(meta, ensure_ascii=False)+ code = (d / "code.py").read_text()+ raw = len(meta_json) + len(code)++ # start a new CHUNK when metadata would get too big+ if rows and chunk_raw + raw > RAW_LIMIT:+ close_chunk(chunk_idx, rows)+ rows = []+ chunk_raw = 0+ chunk_idx += 1+ open_chunk(chunk_idx)++ # start a new TAR within the chunk when it gets too big+ if tar.offset > 0 and tar.offset >= TAR_LIMIT:+ tar.close()+ tar_idx += 1+ tar = tarfile.open(chunk_dir / f"samples-{tar_idx:03d}.tar", "w")++ start = tar.offset+ add_bytes(tar, f"{key}/captions.json", json.dumps(EMPTY_CAP).encode()); n = 1+ add_bytes(tar, f"{key}/code.py", code.encode()); n += 1+ add_bytes(tar, f"{key}/meta.json", meta_json.encode()); n += 1+ for rname in RENDER_ORDER:+ rp = d / "renders" / rname+ if rp.exists():+ tar.add(rp, arcname=f"{key}/renders/{rname}"); n += 1+ byte_len = tar.offset - start++ rows.append({+ "id": f"deepcad/{subset}/{key}",+ "key": key,+ "name": meta.get("name", f"deepcad_{key}"),+ "captions": dict(EMPTY_CAP),+ "meta_json": meta_json,+ "code": code,+ "tar": f"deepcad/{folder_name(chunk_idx)}/samples-{tar_idx:03d}.tar",+ "byte_start": start,+ "byte_len": byte_len,+ "n_files": n,+ })+ chunk_raw += raw++ if rows:+ close_chunk(chunk_idx, rows)+ print(f"Done: {n_chunks_written} chunks.")+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/render_all.py b/toolkits/curation/deepcad/render_all.pynew file mode 100644index 00000000..a3a1045d--- /dev/null+++ b/toolkits/curation/deepcad/render_all.py@@ -0,0 +1,129 @@+"""+render_all.py+Render multi-view (iso / front / right / top) PNG previews of each STEP file+using cadquery tessellation + pyrender (headless OSMesa).++Usage:+ python render_all.py --step_dir /data/.../step_files --out_dir /data/.../renders \+ --workers 24 [--size 512] [--limit N]+ python render_all.py --one /path/to.step --out_prefix /tmp/foo # single-model test+"""+import os+os.environ.setdefault("PYOPENGL_PLATFORM", "osmesa")+import sys+import argparse+from pathlib import Path++import numpy as np+import trimesh+import pyrender+import cadquery as cq+from PIL import Image++# view direction (camera sits along +dir looking at origin) and up vector+VIEWS = {+ "iso": (np.array([1.0, -1.0, 1.0]), np.array([0.0, 0.0, 1.0])),+ "front": (np.array([0.0, -1.0, 0.0]), np.array([0.0, 0.0, 1.0])),+ "right": (np.array([1.0, 0.0, 0.0]), np.array([0.0, 0.0, 1.0])),+ "top": (np.array([0.0, 0.0, 1.0]), np.array([0.0, 1.0, 0.0])),+}+++def look_at(eye, target, up):+ """4x4 camera pose (OpenGL convention: camera looks down -Z)."""+ f = (target - eye); f = f / np.linalg.norm(f)+ s = np.cross(f, up)+ if np.linalg.norm(s) < 1e-8:+ up = np.array([1.0, 0.0, 0.0])+ s = np.cross(f, up)+ s = s / np.linalg.norm(s)+ u = np.cross(s, f)+ m = np.eye(4)+ m[:3, 0] = s+ m[:3, 1] = u+ m[:3, 2] = -f+ m[:3, 3] = eye+ return m+++def step_to_mesh(step_path, tol=0.05):+ shape = cq.importers.importStep(str(step_path)).val()+ verts, faces = shape.tessellate(tol)+ v = np.array([[p.x, p.y, p.z] for p in verts], dtype=float)+ f = np.array(faces, dtype=np.int64)+ return trimesh.Trimesh(vertices=v, faces=f, process=False)+++def render_mesh(mesh, out_prefix, size=512):+ mesh = mesh.copy()+ mesh.apply_translation(-mesh.bounding_box.centroid)+ radius = float(np.linalg.norm(mesh.bounding_box.extents)) / 2.0 or 1.0+ dist = radius * 3.0++ pmesh = pyrender.Mesh.from_trimesh(mesh, smooth=False)+ r = pyrender.OffscreenRenderer(size, size)+ for name, (d, up) in VIEWS.items():+ d = d / np.linalg.norm(d)+ eye = d * dist+ pose = look_at(eye, np.zeros(3), up)+ scene = pyrender.Scene(bg_color=[1.0, 1.0, 1.0, 1.0],+ ambient_light=[0.3, 0.3, 0.3])+ scene.add(pmesh)+ cam = pyrender.PerspectiveCamera(yfov=np.pi / 4.0, aspectRatio=1.0)+ scene.add(cam, pose=pose)+ scene.add(pyrender.DirectionalLight(color=[1, 1, 1], intensity=4.0), pose=pose)+ color, _ = r.render(scene)+ Image.fromarray(color).save(f"{out_prefix}_{name}.png")+ r.delete()+++def render_one(step_path, out_prefix, size=512):+ mesh = step_to_mesh(step_path)+ render_mesh(mesh, out_prefix, size)+++def _worker(step_path, out_dir, size):+ sub = step_path.parent.name+ name = step_path.stem+ od = Path(out_dir) / sub+ od.mkdir(parents=True, exist_ok=True)+ try:+ render_one(step_path, str(od / name), size)+ return f"{sub}/{name}", "OK"+ except Exception as e:+ return f"{sub}/{name}", f"FAIL: {type(e).__name__}: {e}"[:200]+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--step_dir")+ ap.add_argument("--out_dir")+ ap.add_argument("--workers", type=int, default=16)+ ap.add_argument("--size", type=int, default=512)+ ap.add_argument("--limit", type=int, default=0)+ ap.add_argument("--one")+ ap.add_argument("--out_prefix")+ args = ap.parse_args()++ if args.one:+ render_one(Path(args.one), args.out_prefix or "/tmp/render", args.size)+ print("rendered:", args.out_prefix)+ return++ from joblib import Parallel, delayed+ from tqdm import tqdm+ steps = sorted(Path(args.step_dir).rglob("*.step"))+ if args.limit:+ steps = steps[: args.limit]+ print(f"Rendering {len(steps)} models x4 views, {args.workers} workers...")+ results = Parallel(n_jobs=args.workers)(+ delayed(_worker)(s, args.out_dir, args.size) for s in tqdm(steps)+ )+ fails = [r for r in results if r[1] != "OK"]+ print(f"Rendered OK: {len(results)-len(fails)}/{len(results)}, failed: {len(fails)}")+ import json+ json.dump({"fail": fails}, open(Path(args.out_dir) / "render_report.json", "w"), indent=2)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/to_3dcodeverse.py b/toolkits/curation/deepcad/to_3dcodeverse.pynew file mode 100644index 00000000..c20c790a--- /dev/null+++ b/toolkits/curation/deepcad/to_3dcodeverse.py@@ -0,0 +1,145 @@+"""+to_3dcodeverse.py+Emit the official 3DCodeVerse directory layout, aligned to the REAL reference+dataset ilabai/3dcodeverse (3dcodebench/factories_geo/<sample>/):++ deepcad/<sub>/<name>/+ code.py+ meta.json # FLAT official fields (matches reference exactly)+ curation.json # OUR extras (iou/quality/split/dedup/Text2CAD) -- kept+ # out of meta.json so the toolkit reads a clean card+ renders/+ view_00.png view_01.png view_02.png view_03.png # iso/front/right/top+ object.glb # mesh artifact+ model.step # CAD BREP (extra)++captions.json is intentionally NOT written -- the 3DCodeVerse maintainer captions+the data. Our Text2CAD text (CC-BY-NC-SA) is preserved in curation.json (clearly+licensed) for reference, not as the official caption.++meta.json reference (factories_geo sample):+ {id,name,type,language,entry,multi_file,environment,renders[],source,license,+ curator,status}+"""+import os+import json+import shutil+import argparse+from pathlib import Path+from joblib import Parallel, delayed+from tqdm import tqdm++VIEW_MAP = [("iso", "view_00"), ("front", "view_01"),+ ("right", "view_02"), ("top", "view_03")]+++def emit_one(entry, step_dir, render_dir, gt_dir, out_root, curator, want_glb):+ md = entry["metadata"]+ did = md["deepcad_id"] # "0001/00013172"+ sub, name = did.split("/")+ d = Path(out_root) / "deepcad" / sub / name+ rdir = d / "renders"+ rdir.mkdir(parents=True, exist_ok=True)++ (d / "code.py").write_text(entry["code"])++ renders_arr = []+ for view, vname in VIEW_MAP:+ src = Path(render_dir) / sub / f"{name}_{view}.png"+ if src.exists():+ shutil.copy(src, rdir / f"{vname}.png")+ renders_arr.append(f"renders/{vname}.png")+ if want_glb:+ gt = Path(gt_dir) / sub / (name + ".stl")+ if gt.exists():+ try:+ import trimesh+ trimesh.load(str(gt), force="mesh").export(str(rdir / "object.glb"))+ renders_arr.append("renders/object.glb")+ except Exception:+ pass+ step_src = Path(step_dir) / sub / (name + ".step")+ if step_src.exists():+ shutil.copy(step_src, rdir / "model.step")+ renders_arr.append("renders/model.step")++ # status: curated for clean high-fidelity, revise for flagged/low-fidelity+ status = "curated" if (md.get("quality") == "ok" and not md.get("low_fidelity")) else "revise"++ meta = {+ "id": f"deepcad/{sub}/{name}",+ "name": f"deepcad_{name}",+ "type": "3D Objects",+ "language": "CadQuery Python",+ "entry": "code.py",+ "multi_file": False,+ "environment": "CadQuery 2.7 / Python 3.10",+ "renders": renders_arr,+ "source": "DeepCAD",+ "license": "MIT",+ "curator": curator,+ "status": status,+ }+ (d / "meta.json").write_text(json.dumps(meta, ensure_ascii=False, indent=2))++ curation = {+ "deepcad_id": did,+ "provenance": md.get("provenance"),+ "fidelity": {+ "iou": md.get("iou"),+ "chamfer": md.get("chamfer"),+ "fidelity_tier": md.get("fidelity_tier"),+ "low_fidelity": md.get("low_fidelity"),+ "quality": md.get("quality"),+ },+ "splits": {+ "split_original": md.get("split"),+ "split_dedup": md.get("split_dedup"),+ "is_canonical": md.get("is_canonical"),+ "duplicate_of": md.get("duplicate_of"),+ },+ "text_text2cad": entry.get("text") or {},+ "text_license": md.get("text_license"),+ "note": "text_text2cad is from Text2CAD (CC-BY-NC-SA-4.0); NOT the official caption.",+ }+ (d / "curation.json").write_text(json.dumps(curation, ensure_ascii=False, indent=2))+ return did, "OK"+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--jsonl", required=True)+ ap.add_argument("--step_dir", required=True)+ ap.add_argument("--render_dir", required=True)+ ap.add_argument("--gt_dir", required=True)+ ap.add_argument("--out_root", required=True)+ ap.add_argument("--curator", default="ziyao")+ ap.add_argument("--workers", type=int, default=24)+ ap.add_argument("--limit", type=int, default=0)+ ap.add_argument("--canonical_only", action="store_true",+ help="emit only is_canonical==true samples (dedup)")+ ap.add_argument("--no_glb", action="store_true")+ args = ap.parse_args()++ entries = []+ with open(args.jsonl) as f:+ for line in f:+ e = json.loads(line)+ if args.canonical_only and not e["metadata"].get("is_canonical"):+ continue+ entries.append(e)+ if args.limit and len(entries) >= args.limit:+ break++ print(f"Emitting {len(entries)} samples to {args.out_root}/deepcad/ ...")+ results = Parallel(n_jobs=args.workers)(+ delayed(emit_one)(e, args.step_dir, args.render_dir, args.gt_dir,+ args.out_root, args.curator, not args.no_glb)+ for e in tqdm(entries)+ )+ ok = sum(1 for _, s in results if s == "OK")+ print(f"Emitted {ok}/{len(entries)} sample dirs.")+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/upload_to_hf.py b/toolkits/curation/deepcad/upload_to_hf.pynew file mode 100644index 00000000..016929ed--- /dev/null+++ b/toolkits/curation/deepcad/upload_to_hf.py@@ -0,0 +1,48 @@+"""+upload_to_hf.py+Upload the 3DCodeVerse-format corpus tree to a HuggingFace dataset repo.+Uses upload_large_folder (resumable, multi-threaded) -- required for the+~1.5M small files. NEEDS A WRITE TOKEN with access to the target repo.++Usage:+ python upload_to_hf.py --folder /data/.../corpus_3dcv \+ --repo ilabai/3dcodeverse --token_file /data/.../.hf_token_write+"""+import argparse+from pathlib import Path+from huggingface_hub import HfApi+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--folder", required=True, help="local folder whose contents map to repo root (contains deepcad/)")+ ap.add_argument("--repo", required=True, help="e.g. ilabai/3dcodeverse or adonis126/deepcad-3dcodeverse")+ ap.add_argument("--token_file", required=True)+ ap.add_argument("--repo_type", default="dataset")+ ap.add_argument("--create", action="store_true", help="create the repo if it doesn't exist")+ args = ap.parse_args()++ token = Path(args.token_file).read_text().strip()+ api = HfApi(token=token)++ me = api.whoami()+ role = me.get("auth", {}).get("accessToken", {}).get("role", "?")+ print(f"Authenticated as {me.get('name')} (token role: {role})")+ if role != "write":+ print("WARNING: token role is not 'write' -- upload will fail. Create a WRITE token.")++ if args.create:+ api.create_repo(args.repo, repo_type=args.repo_type, exist_ok=True)+ print(f"Ensured repo {args.repo} exists.")++ print(f"Uploading {args.folder} -> {args.repo} ({args.repo_type}) ...")+ api.upload_large_folder(+ repo_id=args.repo,+ repo_type=args.repo_type,+ folder_path=args.folder,+ )+ print("Upload complete.")+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/validate_all.py b/toolkits/curation/deepcad/validate_all.pynew file mode 100644index 00000000..e0a027be--- /dev/null+++ b/toolkits/curation/deepcad/validate_all.py@@ -0,0 +1,76 @@+"""+validate_all.py+Run each generated CadQuery .py through cq_check.py (subprocess: OCC volume ++validity + STEP export). Writes validation_results.json.++Statuses: OK / INVALID / DEGENERATE / EXEC_FAIL / GEOM_FAIL / EXPORT_FAIL / TIMEOUT / NO_OUTPUT+"""+import os+import sys+import json+import argparse+import subprocess+from pathlib import Path+from joblib import Parallel, delayed+from tqdm import tqdm++CHECKER = str(Path(__file__).with_name("cq_check.py"))+++def validate_one(py_path, step_root, py_exe, timeout):+ sub = py_path.parent.name+ name = py_path.stem+ model_id = f"{sub}/{name}"+ step_path = Path(step_root) / sub / (name + ".step")++ try:+ r = subprocess.run([py_exe, CHECKER, str(py_path), str(step_path)],+ timeout=timeout, capture_output=True, text=True)+ except subprocess.TimeoutExpired:+ return model_id, "TIMEOUT", ""++ out = r.stdout.strip().split("\n")[-1] if r.stdout.strip() else ""+ if not out:+ return model_id, "EXEC_FAIL", (r.stderr.strip().split("\n")[-1][:200] if r.stderr else "no stdout")+ try:+ res = json.loads(out)+ except Exception:+ return model_id, "NO_OUTPUT", out[:200]+ return model_id, res.get("status", "NO_OUTPUT"), res.get("detail", "")+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--cq_dir", required=True)+ ap.add_argument("--step_dir", required=True)+ ap.add_argument("--workers", type=int, default=32)+ ap.add_argument("--timeout", type=int, default=30)+ ap.add_argument("--python", default=sys.executable)+ ap.add_argument("--out", default="validation_results.json")+ ap.add_argument("--limit", type=int, default=0)+ args = ap.parse_args()++ py_files = sorted(Path(args.cq_dir).rglob("*.py"))+ if args.limit:+ py_files = py_files[: args.limit]+ print(f"Validating {len(py_files)} scripts with {args.workers} workers...")++ results = Parallel(n_jobs=args.workers)(+ delayed(validate_one)(p, args.step_dir, args.python, args.timeout)+ for p in tqdm(py_files)+ )++ by_status = {}+ for mid, status, detail in results:+ by_status.setdefault(status, []).append({"id": mid, "detail": detail})++ summary = {k: len(v) for k, v in by_status.items()}+ total = sum(summary.values())+ ok = summary.get("OK", 0)+ print("Summary:", summary)+ print(f"OK rate: {ok}/{total} = {100*ok/max(total,1):.1f}%")+ json.dump(by_status, open(args.out, "w"), indent=2)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/validate_fast.py b/toolkits/curation/deepcad/validate_fast.pynew file mode 100644index 00000000..e2eabe2b--- /dev/null+++ b/toolkits/curation/deepcad/validate_fast.py@@ -0,0 +1,126 @@+"""+validate_fast.py -- fast batched validation driver.++Splits the .py scripts into chunks; each chunk runs in ONE subprocess+(cq_check_batch.py) that imports cadquery once and emits a JSON line per model.+If a chunk's subprocess crashes (OCC segfault) and emits only a partial tail,+the unprocessed models are retried in a tiny isolated chunk so one bad model+can't drop its neighbours.++~50x fewer interpreter starts than the per-model subprocess approach.+"""+import os+import sys+import json+import argparse+import tempfile+import subprocess+from pathlib import Path+from joblib import Parallel, delayed+from tqdm import tqdm++WORKER = str(Path(__file__).with_name("cq_check_batch.py"))+++def run_chunk(py_paths, step_root, py_exe, per_model_timeout):+ """Run one chunk; return list of (id, status, detail). Handles partial crash."""+ manifest_lines = []+ id_order = []+ for p in py_paths:+ sub = p.parent.name+ name = p.stem+ step_out = str(Path(step_root) / sub / (name + ".step"))+ manifest_lines.append(f"{p}\t{step_out}")+ id_order.append(f"{sub}/{name}")++ with tempfile.NamedTemporaryFile("w", suffix=".tsv", delete=False) as mf:+ mf.write("\n".join(manifest_lines))+ manifest = mf.name++ results = {}+ try:+ proc = subprocess.run([py_exe, WORKER, manifest],+ capture_output=True, text=True,+ timeout=per_model_timeout * len(py_paths) + 60)+ for line in proc.stdout.splitlines():+ line = line.strip()+ if not line:+ continue+ try:+ r = json.loads(line)+ results[r["id"]] = (r["status"], r.get("detail", ""))+ except Exception:+ pass+ except subprocess.TimeoutExpired:+ pass+ finally:+ os.unlink(manifest)++ # Models with no output -> the worker crashed/timed out before reaching them.+ missing = [pid for pid in id_order if pid not in results]+ out = [(pid, results[pid][0], results[pid][1]) for pid in id_order if pid in results]++ # Retry missing ones individually (full isolation). If a single model still+ # produces nothing, it crashed the interpreter -> mark CRASH.+ for pid in missing:+ sub, name = pid.split("/")+ p = next(pp for pp in py_paths if pp.parent.name == sub and pp.stem == name)+ step_out = str(Path(step_root) / sub / (name + ".step"))+ with tempfile.NamedTemporaryFile("w", suffix=".tsv", delete=False) as mf:+ mf.write(f"{p}\t{step_out}")+ m1 = mf.name+ try:+ pr = subprocess.run([py_exe, WORKER, m1], capture_output=True,+ text=True, timeout=per_model_timeout + 60)+ line = pr.stdout.strip().split("\n")[-1] if pr.stdout.strip() else ""+ if line:+ r = json.loads(line)+ out.append((pid, r["status"], r.get("detail", "")))+ else:+ out.append((pid, "CRASH", "interpreter died"))+ except Exception:+ out.append((pid, "CRASH", "timeout/exception"))+ finally:+ os.unlink(m1)+ return out+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--cq_dir", required=True)+ ap.add_argument("--step_dir", required=True)+ ap.add_argument("--workers", type=int, default=40)+ ap.add_argument("--chunk", type=int, default=64)+ ap.add_argument("--timeout", type=int, default=25)+ ap.add_argument("--python", default=sys.executable)+ ap.add_argument("--out", default="validation_results.json")+ ap.add_argument("--limit", type=int, default=0)+ args = ap.parse_args()++ py_files = sorted(Path(args.cq_dir).rglob("*.py"))+ if args.limit:+ py_files = py_files[: args.limit]+ chunks = [py_files[i:i + args.chunk] for i in range(0, len(py_files), args.chunk)]+ print(f"Validating {len(py_files)} scripts in {len(chunks)} chunks "+ f"x {args.workers} workers (chunk={args.chunk})...")++ chunk_results = Parallel(n_jobs=args.workers)(+ delayed(run_chunk)(c, args.step_dir, args.python, args.timeout)+ for c in tqdm(chunks)+ )++ by_status = {}+ for chunk in chunk_results:+ for mid, status, detail in chunk:+ by_status.setdefault(status, []).append({"id": mid, "detail": detail})++ summary = {k: len(v) for k, v in by_status.items()}+ total = sum(summary.values())+ ok = summary.get("OK", 0)+ print("Summary:", summary)+ print(f"OK rate: {ok}/{total} = {100*ok/max(total,1):.1f}%")+ json.dump(by_status, open(args.out, "w"), indent=2)+++if __name__ == "__main__":+ main()diff --git a/toolkits/curation/deepcad/viz_compare.py b/toolkits/curation/deepcad/viz_compare.pynew file mode 100644index 00000000..cdc76be3--- /dev/null+++ b/toolkits/curation/deepcad/viz_compare.py@@ -0,0 +1,91 @@+"""+viz_compare.py+Side-by-side visual diff: OUR converted geometry (from STEP) vs DeepCAD's+official ground-truth (from gt STL), same iso view, labeled with IoU.+Stacks several models into one comparison grid PNG.+"""+import os+os.environ.setdefault("PYOPENGL_PLATFORM", "osmesa")+import sys+import json+import argparse+import numpy as np+import trimesh+import pyrender+import cadquery as cq+from PIL import Image, ImageDraw++W = "/data/deepcad2cq"+++def look_at(eye, target, up):+ f = target - eye; f = f / np.linalg.norm(f)+ s = np.cross(f, up)+ if np.linalg.norm(s) < 1e-8:+ up = np.array([1., 0, 0]); s = np.cross(f, up)+ s = s / np.linalg.norm(s); u = np.cross(s, f)+ m = np.eye(4); m[:3, 0] = s; m[:3, 1] = u; m[:3, 2] = -f; m[:3, 3] = eye+ return m+++def render(m, size=420, color=(150, 170, 200)):+ m = m.copy(); m.apply_translation(-m.bounding_box.centroid)+ rad = float(np.linalg.norm(m.bounding_box.extents)) / 2 or 1.0+ dist = rad * 3.0+ d = np.array([1., -1., 1.]) / np.sqrt(3); eye = d * dist+ pose = look_at(eye, np.zeros(3), np.array([0, 0, 1.]))+ mat = pyrender.MetallicRoughnessMaterial(baseColorFactor=[c/255 for c in color] + [1.0],+ metallicFactor=0.1, roughnessFactor=0.7)+ pm = pyrender.Mesh.from_trimesh(m, smooth=False, material=mat)+ sc = pyrender.Scene(bg_color=[1, 1, 1, 1], ambient_light=[0.35, 0.35, 0.35])+ sc.add(pm)+ cam = pyrender.PerspectiveCamera(yfov=np.pi/4, aspectRatio=1)+ sc.add(cam, pose=pose)+ sc.add(pyrender.DirectionalLight(color=[1, 1, 1], intensity=4.0), pose=pose)+ r = pyrender.OffscreenRenderer(size, size)+ col, _ = r.render(sc); r.delete()+ return Image.fromarray(col)+++def ours_mesh(mid):+ sub, name = mid.split("/")+ shape = cq.importers.importStep(f"{W}/step_files/{sub}/{name}.step").val()+ v, f = shape.tessellate(0.02)+ return trimesh.Trimesh(np.array([[p.x, p.y, p.z] for p in v]), np.array(f), process=False)+++def gt_mesh(mid):+ sub, name = mid.split("/")+ return trimesh.load(f"{W}/gt_stl/{sub}/{name}.stl", force="mesh")+++def main():+ ids = sys.argv[1].split(",")+ iou = json.load(open(f"{W}/iou_results_full.json"))+ rows = []+ sz = 420+ for mid in ids:+ try:+ o = render(ours_mesh(mid), sz, (120, 160, 210))+ g = render(gt_mesh(mid), sz, (160, 200, 160))+ rec = iou.get(mid, {})+ label = f"{mid} IoU={rec.get('iou')}"+ row = Image.new("RGB", (sz*2 + 30, sz + 28), (255, 255, 255))+ row.paste(o, (0, 28)); row.paste(g, (sz + 30, 28))+ dr = ImageDraw.Draw(row)+ dr.text((6, 8), f"OURS (CadQuery) | GROUND-TRUTH (DeepCAD) {label}", fill=(0, 0, 0))+ rows.append(row)+ except Exception as e:+ print(f"skip {mid}: {e}")+ if rows:+ H = sum(r.height for r in rows) + 10*len(rows)+ grid = Image.new("RGB", (rows[0].width, H), (255, 255, 255))+ y = 0+ for r in rows:+ grid.paste(r, (0, y)); y += r.height + 10+ grid.save(f"{W}/compare_grid.png")+ print(f"saved {W}/compare_grid.png ({len(rows)} models)")+++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/README.md b/toolkits/llamafactory/README.mdnew file mode 100644index 00000000..a86d5be5--- /dev/null+++ b/toolkits/llamafactory/README.md@@ -0,0 +1,38 @@+# llamafactory/ — 3DCodeVerse → LLaMA-Factory datasets++Scripts that turn the corpus on the Hub into datasets a training run can consume directly, and that verify the+data with the same runtimes the evaluation uses. See [`../README.md`](../README.md) for the table of every script+and the two rules they encode; the sections below are the details you need to run them.++## Output layout++```+<subdir>_llamafactory/ one per corpus subdirectory, next to the source+ train.parquet ShareGPT: system + conversations[human,gpt], plus id / caption_type / n_tokens+ dataset_info.json drop-in registry entry -> dataset: 3dcv_<subdir>+ qc.json per-subdir counters (drops, syntax, token stats)+ README.md what it is and what was dropped, with reasons+<subdir>_llamafactory_flat/ flattened multi-file variants (Shadertoy passes with Common merged)+ train.parquet + compiles (bool) and repair (original / repaired:<cat> / unfixable)+```++Multimodal datasets (`build_mm_dataset.py`) add an `images` column; the number of `<image>` placeholders in the+user turn **must** equal `len(images)` — LLaMA-Factory's `mm_plugin._validate_input` raises otherwise. Multimodal+rows also cannot be packed: set `packing: false` and `neat_packing: false`.++## Dialects covered++| dialect | runtime used for verification | notes |+|---|---|---|+| Blender-Python | Blender 5.0.1 headless (needs `scipy` in its bundled Python) | 15 of the 212 3DCodeBench reference scripts import scipy |+| CadQuery | CadQuery 2.8 | the runner captures `show_object`/exports and prefers `result`/`solid` variables |+| OpenSCAD | OpenSCAD AppImage (`--export-format binstl`) | decode with sampling: greedy collapses into the Customizer parameter block |+| GLSL | glslangValidator **and** headless WebGL2 | try ES 3.00 and 3.10 × the channel-type assignments a shader actually uses |+| three.js / web | headless Chromium (Playwright) | judge blankness from the composited screenshot, not `gl.readPixels` |+| URDF | XML parse + link/joint counts | code recovered from the source tars |++## Requirements++Python 3.12 with `pandas`, `pyarrow`, `transformers` (tokenizer only), `playwright` (+ `playwright install chromium`),+and `huggingface_hub` for the upload paths; `glslangValidator` on PATH (conda-forge `glslang`); Blender / OpenSCAD+as system binaries. Paths to the binaries are constants at the top of each script — change them there.diff --git a/toolkits/llamafactory/analyze_3dcodeverse.py b/toolkits/llamafactory/analyze_3dcodeverse.pynew file mode 100644index 00000000..699914fd--- /dev/null+++ b/toolkits/llamafactory/analyze_3dcodeverse.py@@ -0,0 +1,54 @@+"""Survey every metadata.parquet in 3DCodeVerse: counts, code length, caption availability, language, license, multi_file.+Writes data/analysis/3dcodeverse_survey.md (+ .json). Token counts are estimated with the Qwen3.5 tokenizer on a sample."""+import glob, json, os, random, sys+import numpy as np, pandas as pd+ROOT = "/wekafs/ict/hx_624/data/3dcodeverse_meta"+OUT = "/wekafs/ict/hx_624/llm-ft/data/analysis"; os.makedirs(OUT, exist_ok=True)+try:+ from transformers import AutoTokenizer+ tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+except Exception as e:+ tok = None; print("no tokenizer:", e)+random.seed(0)+rows = []+files = sorted(glob.glob(f"{ROOT}/**/metadata.parquet", recursive=True))+print(len(files), "metadata.parquet files")+agg = {}+for f in files:+ rel = os.path.relpath(os.path.dirname(f), ROOT)+ src = rel.split("/")[0]+ df = pd.read_parquet(f)+ n = len(df)+ code = df["code"].fillna("")+ clen = code.str.len().values+ meta0 = json.loads(df["meta_json"].iloc[0]) if "meta_json" in df and n else {}+ lang = df["language"].iloc[0] if "language" in df else meta0.get("language")+ caps = df["captions"].iloc[0] if "captions" in df else None+ cap_keys = list(caps.keys()) if isinstance(caps, dict) else []+ # caption availability+ def has(k): return float(np.mean([isinstance(c, dict) and bool(c.get(k)) for c in df["captions"]])) if "captions" in df else 0.0+ cap_avail = {k: round(has(k), 3) for k in ["detailed", "instruction", "factory", "brief", "synthetic_prompt"]}+ # token estimate on up to 200 random samples+ tok_p50 = tok_p90 = tok_mean = None+ if tok is not None and n:+ idx = random.sample(range(n), min(200, n))+ tl = np.array([len(tok(code.iloc[i]).input_ids) for i in idx])+ tok_p50, tok_p90, tok_mean = int(np.percentile(tl, 50)), int(np.percentile(tl, 90)), float(tl.mean())+ uniq = code.nunique()+ rows.append(dict(subset=rel, source=src, n=n, language=lang, license=meta0.get("license"), multi_file=meta0.get("multi_file"),+ entry=meta0.get("entry"), chars_p50=int(np.percentile(clen, 50)) if n else 0, chars_p90=int(np.percentile(clen, 90)) if n else 0,+ tok_p50=tok_p50, tok_p90=tok_p90, tok_mean=tok_mean, est_total_tokens=(tok_mean * n if tok_mean else None),+ unique_code_frac=round(uniq / max(n, 1), 3), caption_keys=cap_keys, cap_avail=cap_avail, columns=list(df.columns)))+ a = agg.setdefault(src, dict(n=0, est_tokens=0.0, langs=set()))+ a["n"] += n; a["est_tokens"] += (tok_mean * n if tok_mean else 0); a["langs"].add(str(lang))+ print(f"{rel:45s} n={n:6d} lang={str(lang):16s} tok_p50={tok_p50} tok_p90={tok_p90} uniq={uniq/max(n,1):.2f} caps={cap_avail}")+json.dump({"subsets": rows, "by_source": {k: dict(n=v["n"], est_tokens=v["est_tokens"], langs=sorted(v["langs"])) for k, v in agg.items()}},+ open(f"{OUT}/3dcodeverse_survey.json", "w"), indent=1, default=str)+with open(f"{OUT}/3dcodeverse_survey.md", "w") as f:+ f.write("| source | #samples | languages | est. tokens (code only) |\n|---|---|---|---|\n")+ for k, v in sorted(agg.items(), key=lambda kv: -kv[1]["n"]):+ f.write(f"| {k} | {v['n']} | {', '.join(sorted(v['langs']))} | {v['est_tokens']/1e6:.1f}M |\n")+ f.write("\n| subset | n | lang | tok p50 | tok p90 | unique code | captions (detailed/instruction/factory) | license |\n|---|---|---|---|---|---|---|---|\n")+ for r in rows:+ f.write(f"| {r['subset']} | {r['n']} | {r['language']} | {r['tok_p50']} | {r['tok_p90']} | {r['unique_code_frac']} | {r['cap_avail']['detailed']}/{r['cap_avail']['instruction']}/{r['cap_avail']['factory']} | {r['license']} |\n")+print("written", OUT)diff --git a/toolkits/llamafactory/build_hf_dataset.py b/toolkits/llamafactory/build_hf_dataset.pynew file mode 100644index 00000000..f5a08d6f--- /dev/null+++ b/toolkits/llamafactory/build_hf_dataset.py@@ -0,0 +1,231 @@+"""Package the finetuning data as a HuggingFace dataset in LLaMA-Factory (ShareGPT) format.++Design: publish the *corpus* + the *recipes* + the *derived* sets, not the 15 near-duplicate training mixes+(they are all samples of the same corpus and would multiply the size for nothing — `scripts/sample_mix.py`+regenerates any of them from a one-line budget spec).++ sft/<dialect>/train-*.parquet the md_max corpus (768k prompt→code pairs), sharded by dialect+ sft/blender_bootstrapped/ 15.7k execution-verified Blender samples generated by the flywheel+ dpo/<name>.parquet execution-feedback and geometry-feedback preference pairs+ test/<dialect>.parquet the held-out evaluation sets (never trained on)+ dataset_info.json drop-in LLaMA-Factory registry for everything above+ mixes.json the exact budget spec of every mix used in the report+ README.md dataset card++usage: python scripts/build_hf_dataset.py [--out /wekafs/ict/hx_624/hf_ds] [--push ilabai/3dcodeverse-llamafactory] [--private]+"""+import argparse+import collections+import glob+import json+import os++import pandas as pd++ROOT = "/wekafs/ict/hx_624/llm-ft"+SHARD_ROWS = 50000++MIXES = {+ "md_xl": {"desc": "9B best SFT mix (3DCodeBench 90.1%)", "budget": "blender=all,openscad=all(x4),threejs=all,cadquery=all,glsl=all", "captions_per_sample": 1,+ "extra": "+15.7k bootstrapped blender", "pairs": 270173, "tokens": "293M"},+ "md_big": {"desc": "half-size predecessor of md_xl (84.9%)", "budget": "cadquery=40k rows,glsl=40k rows,openscad=all(x4),blender=all", "captions_per_sample": 1,+ "extra": "+15.7k bootstrapped blender", "pairs": 101851, "tokens": "107M"},+ "md_27b_v2s": {"desc": "27B best mix (3DCodeBench 93.4%) — blender-heavy", "budget": "blender=all(x2)+bootstrapped, cadquery=25M tok, glsl=25M tok, openscad=all(x4), threejs=all, web=all",+ "captions_per_sample": 2, "extra": "filtered to <=3400 tokens", "pairs": 103770, "tokens": "75M"},+ "md_27b_v3": {"desc": "27B rebalanced (cadquery restored)", "budget": "cadquery=45M tok, glsl=20M tok, blender=all+bootstrapped, openscad/threejs/web=all",+ "captions_per_sample": 1, "extra": "filtered to <=3400 tokens", "pairs": 107366, "tokens": "73M"},+ "md_max_9b": {"desc": "caption-augmentation ablation — NEGATIVE result (3DCodeBench 34.4%)", "budget": "cadquery=140M tok, glsl=140M tok, rest=all",+ "captions_per_sample": 3, "extra": "no bootstrapped data", "pairs": 363708, "tokens": "304M"},+ "md_v3": {"desc": "control for the above (86.8%)", "budget": "cadquery=140M tok, glsl=140M tok, rest=all", "captions_per_sample": 1,+ "extra": "+15.7k bootstrapped blender", "pairs": 272802, "tokens": "236M"},+}++DPO_SETS = {+ "exec_feedback_blender_r1": ("data/lf/dpo_exec_pairs.json", "2,590 (runs, fails) pairs from LoRA-v1 samples executed in Blender → 75%→93.9% on 3DCodeBench"),+ "exec_feedback_blender_r2": ("data/lf/dpo_exec_pairs_r2.json", "second round on the round-1 model → 96.2%"),+ "exec_feedback_multidialect_md_mixed": ("data/lf/dpo_md_pairs.json", "581 pairs across 4 dialects (md_mixed policy)"),+ "exec_feedback_multidialect_md_big": ("data/lf/dpo_md_big_pairs.json", "1,052 pairs (md_big policy)"),+ "exec_feedback_multidialect_md_xl": ("data/lf/dpo_md_xl_pairs.json", "909 pairs (md_xl policy)"),+ "exec_feedback_27b": ("data/lf/dpo_27b_pairs.json", "756 pairs from the tuned Qwen3.8-27B (three.js 92.5%→100%)"),+ "geometry_feedback": ("data/lf/dpo_geo_pairs.json", "2,357 pairs ranked by [email protected] against the reference mesh — the only signal that moved geometric fidelity"),+}+++def write_parquet(rows, path_prefix, cols=None):+ os.makedirs(os.path.dirname(path_prefix), exist_ok=True)+ n = 0+ for i in range(0, len(rows), SHARD_ROWS):+ chunk = rows[i: i + SHARD_ROWS]+ df = pd.DataFrame(chunk, columns=cols) if cols else pd.DataFrame(chunk)+ suffix = f"-{i // SHARD_ROWS:05d}" if len(rows) > SHARD_ROWS else ""+ df.to_parquet(f"{path_prefix}{suffix}.parquet", index=False, compression="zstd")+ n += 1+ return n+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--out", default="/wekafs/ict/hx_624/hf_ds")+ ap.add_argument("--push", default=None, help="repo id, e.g. ilabai/3dcodeverse-llamafactory")+ ap.add_argument("--private", action="store_true")+ a = ap.parse_args()+ os.makedirs(a.out, exist_ok=True)+ info, stats = {}, {}++ # ---- 1. the corpus, sharded by dialect (ShareGPT: conversations + system) ----+ by = collections.defaultdict(list)+ for split in ("train", "val"):+ for l in open(f"{ROOT}/data/md_max/{split}.jsonl"):+ r = json.loads(l)+ msgs = r["messages"]+ by[r["dialect"]].append({+ "id": r["id"], "subset": r["subset"], "caption_type": r["caption"], "split": split,+ "system": msgs[0]["content"],+ "conversations": [{"from": "human", "value": msgs[1]["content"]},+ {"from": "gpt", "value": msgs[2]["content"]}],+ })+ for dia, rows in by.items():+ nsh = write_parquet(rows, f"{a.out}/sft/{dia}/train")+ info[f"3dcv_{dia}"] = {"file_name": f"sft/{dia}", "formatting": "sharegpt", "columns": {"messages": "conversations", "system": "system"}}+ stats[f"sft/{dia}"] = {"rows": len(rows), "shards": nsh}+ print(f"[hf] sft/{dia}: {len(rows)} rows, {nsh} shard(s)")++ # ---- 2. the bootstrapped Blender set (execution-verified, generated by our own flywheel) ----+ boot = []+ for l in open(f"{ROOT}/data/sft_v13_boot/train.jsonl"):+ if '"source": "boot_' not in l:+ continue+ m = json.loads(l)["messages"]+ boot.append({"system": m[0]["content"],+ "conversations": [{"from": "human", "value": m[1]["content"]}, {"from": "gpt", "value": m[2]["content"]}]})+ write_parquet(boot, f"{a.out}/sft/blender_bootstrapped/train")+ info["3dcv_blender_bootstrapped"] = {"file_name": "sft/blender_bootstrapped", "formatting": "sharegpt", "columns": {"messages": "conversations", "system": "system"}}+ stats["sft/blender_bootstrapped"] = {"rows": len(boot), "shards": 1}+ print(f"[hf] sft/blender_bootstrapped: {len(boot)} rows")++ # ---- 3. preference sets ----+ for name, (path, desc) in DPO_SETS.items():+ p = os.path.join(ROOT, path)+ if not os.path.exists(p):+ print(f"[hf] skip {name} (missing)")+ continue+ rows = json.load(open(p))+ write_parquet(rows, f"{a.out}/dpo/{name}")+ info[f"3dcv_dpo_{name}"] = {"file_name": f"dpo/{name}.parquet", "ranking": True, "formatting": "sharegpt",+ "columns": {"messages": "conversations", "system": "system", "chosen": "chosen", "rejected": "rejected"}}+ stats[f"dpo/{name}"] = {"rows": len(rows), "desc": desc}+ print(f"[hf] dpo/{name}: {len(rows)} pairs")++ # ---- 4. held-out test sets ----+ for f in sorted(glob.glob(f"{ROOT}/data/multidialect/*/test.jsonl")):+ dia = f.split("/")[-2]+ rows = []+ for l in open(f):+ r = json.loads(l)+ m = r["messages"]+ rows.append({"id": r["id"], "system": m[0]["content"], "prompt": m[1]["content"], "reference_code": m[2]["content"]})+ write_parquet(rows, f"{a.out}/test/{dia}")+ stats[f"test/{dia}"] = {"rows": len(rows)}+ print(f"[hf] test/{dia}: {len(rows)} rows")++ json.dump(info, open(f"{a.out}/dataset_info.json", "w"), indent=2, ensure_ascii=False)+ json.dump({"mixes": MIXES, "contents": stats}, open(f"{a.out}/mixes.json", "w"), indent=2, ensure_ascii=False)++ total = sum(v.get("rows", 0) for k, v in stats.items() if k.startswith("sft/"))+ card = f"""---+license: cc-by-nc-sa-4.0+task_categories:+ - text-generation+language:+ - en+tags:+ - 3d+ - code-generation+ - blender+ - cadquery+ - openscad+ - glsl+ - threejs+ - llama-factory+ - sft+ - dpo+size_categories:+ - 100K<n<1M+---++# 3DCodeVerse — LLaMA-Factory edition++Text→3D-code training data in **LLaMA-Factory ShareGPT format**, derived from+[`ilabai/3dcodeverse`](https://huggingface.co/datasets/ilabai/3dcodeverse). Same license and non-commercial+research terms as the parent dataset; the per-source licenses (Infinigen/BSD-3, Bioinspired3D/MIT, Shadertoy/CC BY-NC-SA,+Thingiverse, DeepCAD/MIT) still apply.++Training and evaluation code: <https://github.com/gaoypeng/3dcodeverse/tree/main/finetune>.+Full experiment log: [`llm_finetune_exps.md`](https://github.com/gaoypeng/3dcodeverse/blob/main/finetune/llm_finetune_exps.md).++## Contents++| path | what | rows |+|---|---|---|+| `sft/blender/` | Blender-Python (bioinspired3d, 3dcodebench factories, blender_distill) | {stats.get('sft/blender', {}).get('rows', 0):,} |+| `sft/cadquery/` | CadQuery (deepcad, articraft) | {stats.get('sft/cadquery', {}).get('rows', 0):,} |+| `sft/glsl/` | GLSL fragment shaders (shadertoy) | {stats.get('sft/glsl', {}).get('rows', 0):,} |+| `sft/openscad/` | OpenSCAD (thingiverse) | {stats.get('sft/openscad', {}).get('rows', 0):,} |+| `sft/threejs/` | single-file three.js pages | {stats.get('sft/threejs', {}).get('rows', 0):,} |+| `sft/web/` | animated web pages (animation2code) | {stats.get('sft/web', {}).get('rows', 0):,} |+| `sft/blender_bootstrapped/` | **execution-verified** Blender samples generated by a finetuned model on novel prompts and kept only if they ran | {stats.get('sft/blender_bootstrapped', {}).get('rows', 0):,} |+| `dpo/*.parquet` | execution-feedback and geometry-feedback preference pairs (see below) | 8k pairs total |+| `test/*.parquet` | held-out evaluation sets, never trained on | 593 |++Total SFT pairs: **{total:,}** (one row per prompt variant; each source sample has up to 3 caption styles —+`caption_type` is `instruction` / `detailed` / `factory`).++## How to use with LLaMA-Factory++```bash+huggingface-cli download ilabai/3dcodeverse-llamafactory --repo-type dataset --local-dir data/3dcv+cp data/3dcv/dataset_info.json data/3dcv/ # already there; point dataset_dir at data/3dcv+llamafactory-cli train your_config.yaml # with: dataset_dir: data/3dcv, dataset: 3dcv_blender,3dcv_cadquery,...+```++`dataset_info.json` registers every subset (`3dcv_<dialect>`, `3dcv_blender_bootstrapped`, `3dcv_dpo_<name>`).++## Reproducing the mixes from the report++The report's training mixes are *samples* of this corpus, so they are described by a budget rather than shipped+as duplicated files (`mixes.json` has the exact specs; `scripts/sample_mix.py` in the repo applies them):++| mix | pairs | tokens | captions/sample | result |+|---|---|---|---|---|+{chr(10).join(f"| `{k}` | {v['pairs']:,} | {v['tokens']} | {v['captions_per_sample']} | {v['desc']} |" for k, v in MIXES.items())}++## Two findings that matter when using this data++1. **Do not use all three caption variants of the same code sample.** At a fixed token budget, 3 captions per sample+ scored 34.4% on 3DCodeBench versus 90.1% for one caption per sample plus the bootstrapped set — what generalises+ is the number of *distinct runnable programs*, not the number of prompts.+2. **`sft/blender_bootstrapped/` is the highest-value subset per token.** Adding it (and raising the Blender share to+ ~40%) took a 27B model from 57.6% to 93.4% on 3DCodeBench.++## Evaluation++Execution-based: every generated program is run (Blender 5.0.1 headless / CadQuery / OpenSCAD / glslangValidator /+headless Chromium) and the resulting mesh compared to the reference with Chamfer distance and F-score. Scripts:+`finetune/eval/` in the GitHub repo. Note: Blender's bundled Python needs `scipy` — 15 of the 212 3DCodeBench+reference scripts import it.+"""+ open(f"{a.out}/README.md", "w").write(card)+ print(f"[hf] wrote {a.out} ({sum(os.path.getsize(p) for p in glob.glob(a.out + '/**/*', recursive=True) if os.path.isfile(p))/1e9:.2f} GB)")++ if a.push:+ from huggingface_hub import HfApi+ tok = os.environ.get("HF_TOKEN")+ api = HfApi(token=tok)+ api.create_repo(a.push, repo_type="dataset", private=a.private, exist_ok=True)+ api.upload_folder(folder_path=a.out, repo_id=a.push, repo_type="dataset",+ commit_message="3DCodeVerse in LLaMA-Factory format: SFT corpus by dialect, bootstrapped Blender set, DPO preference sets, held-out tests")+ print(f"[hf] pushed to https://huggingface.co/datasets/{a.push}")+++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/build_image_dataset.py b/toolkits/llamafactory/build_image_dataset.pynew file mode 100644index 00000000..f92b014b--- /dev/null+++ b/toolkits/llamafactory/build_image_dataset.py@@ -0,0 +1,163 @@+"""Build an image-conditioned (render → code) dataset in LLaMA-Factory multimodal format.++The `qwen3_5` template already registers a multimodal plugin (`qwen3_vl`, image token `<|image_pad|>`), so an+`images` column plus an `<image>` placeholder in the user turn is all that is needed — the only change to a normal+run is `freeze_vision_tower: false` if you want the tower to adapt.++Why this matters for this project: the report shows execution rate responds to more data, but *geometric fidelity*+(F-score / Chamfer) barely moved across every text-only SFT variant. A reference render carries exactly the signal+the caption lacks.++Sources (all already on disk):+ bioinspired3d tars 4 views + code per sample, Blender-Python — the dialect 3DCodeBench measures+ blender_distill 4 views per sample+ threejs_distill 4 views per sample+ animation2code 4 views per sample++usage: python scripts/build_image_dataset.py [--out data/img_code] [--views 1] [--max_per_source 20000]+"""+import argparse+import collections+import glob+import json+import os+import tarfile++IMG_ROOT = "/wekafs/ict/hx_624/data/images"+SYS = {+ "blender": "You are an expert in procedural 3D modeling with Blender Python (bpy). Given reference renders of an object, write a complete, standalone Blender 5 Python script that reproduces it (clear the default scene first). Output ONLY the code in one ```python block.",+ "threejs": "You are an expert three.js developer. Given reference renders, write a complete, standalone HTML page that reproduces the scene with three.js. Output ONLY the code in one ```html block.",+ "web": "You are an expert creative web developer. Given reference frames of an animation, write a complete, standalone HTML page (inline CSS/JS) that reproduces it. Output ONLY the code in one ```html block.",+}+ASK = {+ "blender": "Reproduce this 3D object with Blender Python code.",+ "threejs": "Reproduce this scene as a single-file three.js page.",+ "web": "Reproduce this animation as a single-file animated web page.",+}+FENCE = {"blender": "python", "threejs": "html", "web": "html"}+++def from_tars(pattern, dialect, out_rows, views, cap, caption_too=True):+ n = 0+ for t in sorted(glob.glob(pattern)):+ sub = os.path.basename(os.path.dirname(t))+ try:+ tf = tarfile.open(t)+ except Exception:+ continue+ members = collections.defaultdict(dict)+ for m in tf.getmembers():+ if not m.isfile():+ continue+ parts = m.name.split("/")+ if len(parts) < 2:+ continue+ key = parts[0]+ if m.name.endswith("code.py"):+ members[key]["code"] = m+ elif m.name.endswith("captions.json"):+ members[key]["caps"] = m+ elif "/renders/view_" in m.name and m.name.endswith(".png"):+ members[key].setdefault("imgs", []).append(m)+ for key, d in members.items():+ if "code" not in d or "imgs" not in d:+ continue+ try:+ code = tf.extractfile(d["code"]).read().decode("utf-8", "replace")+ except Exception:+ continue+ if len(code.strip()) < 40:+ continue+ caps = {}+ if caption_too and "caps" in d:+ try:+ caps = json.load(tf.extractfile(d["caps"]))+ except Exception:+ caps = {}+ imgs = sorted(d["imgs"], key=lambda m: m.name)[:views]+ paths = []+ for i, m in enumerate(imgs):+ dst = os.path.join(IMG_ROOT, sub, key, os.path.basename(m.name))+ if not os.path.exists(dst):+ os.makedirs(os.path.dirname(dst), exist_ok=True)+ with open(dst, "wb") as f:+ f.write(tf.extractfile(m).read())+ paths.append(dst)+ if not paths:+ continue+ hint = (caps.get("instruction") or caps.get("detailed") or "").strip()+ ask = ASK[dialect] + (f"\nHint: {hint}" if hint and len(hint) < 400 else "")+ out_rows.append({"id": f"{sub}/{key}", "dialect": dialect, "images": paths,+ "system": SYS[dialect],+ "conversations": [{"from": "human", "value": "<image>" * len(paths) + "\n" + ask},+ {"from": "gpt", "value": f"```{FENCE[dialect]}\n{code.strip()}\n```"}]})+ n += 1+ if n >= cap:+ tf.close()+ return n+ tf.close()+ return n+++def from_tree(root, dialect, out_rows, views, cap, code_name):+ n = 0+ for cp in sorted(glob.glob(f"/wekafs/ict/hx_624/data/3dcodeverse_files/{root}/*/*/{code_name}")):+ d = os.path.dirname(cp)+ imgs = sorted(glob.glob(os.path.join(d, "renders", "*.png")))[:views]+ if not imgs:+ continue+ code = open(cp, encoding="utf-8", errors="replace").read()+ if len(code.strip()) < 40:+ continue+ caps = {}+ cj = os.path.join(d, "captions.json")+ if os.path.exists(cj):+ try:+ caps = json.load(open(cj))+ except Exception:+ pass+ hint = (caps.get("user_instruction") or caps.get("instruction") or caps.get("brief") or "").strip()+ ask = ASK[dialect] + (f"\nHint: {hint}" if hint and len(hint) < 400 else "")+ out_rows.append({"id": os.path.relpath(d, "/wekafs/ict/hx_624/data/3dcodeverse_files"),+ "dialect": dialect, "images": imgs, "system": SYS[dialect],+ "conversations": [{"from": "human", "value": "<image>" * len(imgs) + "\n" + ask},+ {"from": "gpt", "value": f"```{FENCE[dialect]}\n{code.strip()}\n```"}]})+ n += 1+ if n >= cap:+ return n+ return n+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--out", default="/wekafs/ict/hx_624/llm-ft/data/img_code")+ ap.add_argument("--views", type=int, default=1)+ ap.add_argument("--max_per_source", type=int, default=20000)+ a = ap.parse_args()+ os.makedirs(IMG_ROOT, exist_ok=True)+ rows = []+ got = {}+ got["bioinspired3d"] = from_tars("/wekafs/ict/hx_624/data/3dcodeverse_tars/bioinspired3d/*/*.tar",+ "blender", rows, a.views, a.max_per_source)+ got["blender_distill"] = from_tree("blender_distill", "blender", rows, a.views, a.max_per_source, "code.py")+ got["threejs_distill"] = from_tree("threejs_distill", "threejs", rows, a.views, a.max_per_source, "code.html")+ got["animation2code"] = from_tree("animation2code", "web", rows, a.views, a.max_per_source, "index.html")+ os.makedirs(a.out, exist_ok=True)+ import random+ random.seed(3)+ random.shuffle(rows)+ val, train = rows[:200], rows[200:]+ for name, part in (("train", train), ("val", val)):+ with open(os.path.join(a.out, f"{name}.json"), "w") as f:+ json.dump(part, f, ensure_ascii=False)+ json.dump({"img_code_train": {"file_name": "train.json", "formatting": "sharegpt",+ "columns": {"messages": "conversations", "system": "system", "images": "images"}},+ "img_code_val": {"file_name": "val.json", "formatting": "sharegpt",+ "columns": {"messages": "conversations", "system": "system", "images": "images"}}},+ open(os.path.join(a.out, "dataset_info.json"), "w"), indent=2)+ print(f"[img] {len(rows)} samples ({dict(got)}), {a.views} view(s) each -> {a.out}")+ print("[img] dialects:", dict(collections.Counter(r["dialect"] for r in rows)))+++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/build_md_max.py b/toolkits/llamafactory/build_md_max.pynew file mode 100644index 00000000..409192f2--- /dev/null+++ b/toolkits/llamafactory/build_md_max.py@@ -0,0 +1,246 @@+"""md_max: use EVERY usable 3DCodeVerse sample, with multi-caption prompt expansion (~0.5M prompt→code pairs).++Sources (all parquet metadata + the distilled file trees):+ blender bioinspired3d geo+tex, 3dcodebench factories (test factories removed), blender_distill (*.py)+ cadquery deepcad v1..v1_4, articraft cadquery_single_tex*+ openscad thingiverse/openscad+ glsl shadertoy (112 shards)+ threejs threejs_distill (*.html)+ web animation2code (*.html) — new: animated web/3D pages++Deliberately excluded and why:+ 3dcodebench/instances_* (3904) 100% of them are re-seeded instances of the 212 benchmark factories → test leakage+ articraft/urdf_* (6146) `code` column is empty in the metadata (URDF lives inside the 51 GB tars) and we have no URDF executor+ threejs_repos multi-file projects, not single-file prompt→code+ every id in data/multidialect/*/test.jsonl and the 212 3DCodeBench task names++Each sample yields up to 3 training pairs (captions: instruction / detailed / factory), deduped by (prompt, code).+Token length is estimated as tokens(code) + tokens(prompt) + template overhead, tokenizing each unique code/prompt once.++usage: python scripts/build_md_max.py [--max_tokens 8192] [--captions 3] [--out data/md_max]+"""+import argparse+import glob+import hashlib+import json+import os+import random+import re+import sys+import unicodedata+from concurrent.futures import ProcessPoolExecutor++import pandas as pd++sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/scripts")+META = "/wekafs/ict/hx_624/data/3dcodeverse_meta"+FILES = "/wekafs/ict/hx_624/data/3dcodeverse_files"+MD = "/wekafs/ict/hx_624/llm-ft/data/multidialect"+BENCH_DIR = "/wekafs/ict/hx_624/data/3dcodebench/data"++SYS = {+ "blender": "You are an expert in procedural 3D modeling with Blender Python (bpy). Given a description of an object, write a complete, standalone Blender 5 Python script that builds it from scratch (clear the default scene first). Output ONLY the code in one ```python block.",+ "cadquery": "You are an expert CAD programmer. Given a description of a part, write a complete, standalone CadQuery (Python) script that builds it and leaves the final solid in a variable named `result`. Output ONLY the code in one ```python block.",+ "openscad": "You are an expert OpenSCAD programmer. Given a request, write a complete, standalone OpenSCAD (.scad) program that produces the described model. Output ONLY the code in one ```openscad block.",+ "glsl": "You are an expert shader programmer. Given a description, write a complete Shadertoy-style GLSL ES 3.00 fragment shader defining `void mainImage(out vec4 fragColor, in vec2 fragCoord)`. Output ONLY the code in one ```glsl block.",+ "threejs": "You are an expert three.js developer. Given a description, write a complete, standalone HTML page that renders the described 3D scene with three.js (import three from a CDN-style module URL, create a renderer/camera/lights, and animate). Output ONLY the code in one ```html block.",+ "web": "You are an expert creative web developer. Given a description of an animation, write a complete, standalone HTML page (inline CSS/JS, no local assets) that reproduces it. Output ONLY the code in one ```html block.",+}+FENCE = {"blender": "python", "cadquery": "python", "openscad": "openscad", "glsl": "glsl", "threejs": "html", "web": "html"}+LEAD = { # how to phrase a bare description as a request, per dialect+ "blender": "Create the 3D object described below using Python Blender code: ",+ "cadquery": "Write a CadQuery (Python) script that builds the following CAD model: ",+ "openscad": "Write an OpenSCAD program for the following: ",+ "glsl": "Write a GLSL fragment shader for Shadertoy that renders: ",+ "threejs": "Write a standalone three.js page that renders: ",+ "web": "Write a standalone animated web page that reproduces: ",+}+IMPERATIVE = ("write", "create", "implement", "make", "build", "generate", "design", "produce")+++def bad_chars(s):+ return sum(1 for ch in s if unicodedata.category(ch) == "Cc" and ch not in "\n\t\r") + s.count("�")+++def phrase(dialect, kind, text):+ t = (text or "").strip()+ if not t:+ return None+ if kind == "instruction" and t.lower().startswith(IMPERATIVE):+ return t+ return LEAD[dialect] + t+++def load_parquet_rows(dialect, sub, drop_names=frozenset()):+ p = f"{META}/{sub}/metadata.parquet"+ if not os.path.exists(p):+ return []+ df = pd.read_parquet(p)+ out = []+ for r in df.itertuples():+ name = str(getattr(r, "name", "") or "")+ norm = name.replace("Factory", "").replace("_geo", "").replace("_tex", "")+ if name in drop_names or name.rsplit("_", 1)[0] in drop_names or norm in drop_names:+ continue+ code = getattr(r, "code", None) or ""+ if not isinstance(code, str) or len(code.strip()) < 40:+ continue+ caps = r.captions if isinstance(r.captions, dict) else (json.loads(r.captions) if isinstance(r.captions, str) else {})+ out.append({"dialect": dialect, "subset": sub, "id": str(r.id), "name": name, "code": code, "caps": dict(caps)})+ return out+++def load_tree_rows(dialect, root, code_glob):+ """distill / animation2code trees: <sample>/{code.py|code.html|index.html} + captions.json"""+ rows = []+ for cp in glob.glob(f"{FILES}/{root}/{code_glob}", recursive=True):+ d = os.path.dirname(cp)+ capp = os.path.join(d, "captions.json")+ try:+ code = open(cp, encoding="utf-8", errors="replace").read()+ caps = json.load(open(capp)) if os.path.exists(capp) else {}+ except Exception:+ continue+ if len(code.strip()) < 40:+ continue+ rid = f"{root}/{os.path.relpath(d, f'{FILES}/{root}')}"+ caps = {k: v for k, v in caps.items() if isinstance(v, str) and v.strip()}+ # tree sources use their own caption keys -> map onto instruction/detailed+ norm = {}+ if caps.get("user_instruction"): norm["instruction"] = caps["user_instruction"] # imperative request+ if caps.get("instruction"): norm["detailed"] = caps["instruction"] # description+ if caps.get("brief"): # animation2code+ extra = caps.get("function", "")+ norm["instruction"] = (caps["brief"] + ((" — " + extra) if extra and extra not in caps["brief"] else "")).strip()+ if caps.get("source_title") and "detailed" not in norm: norm["detailed"] = caps["source_title"]+ if caps.get("style") and norm.get("detailed"): norm["detailed"] += f" (style: {caps['style']})"+ rows.append({"dialect": dialect, "subset": root, "id": rid, "name": os.path.basename(d), "code": code, "caps": norm or caps})+ return rows+++_TOK = None+++def _init_tok():+ global _TOK+ from transformers import AutoTokenizer+ _TOK = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+++def _ntok(s):+ return len(_TOK(s).input_ids)+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--max_tokens", type=int, default=8192)+ ap.add_argument("--captions", type=int, default=3, help="max prompt variants per sample")+ ap.add_argument("--out", default="/wekafs/ict/hx_624/llm-ft/data/md_max")+ ap.add_argument("--workers", type=int, default=16)+ a = ap.parse_args()+ random.seed(17)++ bench_factories = {os.path.basename(p).replace("_seed0", "") for p in glob.glob(f"{BENCH_DIR}/*_seed0")}+ held_out = set()+ for f in glob.glob(f"{MD}/*/test.jsonl"):+ for l in open(f):+ held_out.add(json.loads(l)["id"])+ print(f"[md_max] excluding {len(bench_factories)} benchmark factories and {len(held_out)} held-out test ids", flush=True)++ samples = []+ for sub in ["bioinspired3d/bpy_structures_geo", "bioinspired3d/bpy_structures_tex", "3dcodebench/factories_geo", "3dcodebench/factories_tex"]:+ samples += load_parquet_rows("blender", sub, bench_factories)+ for sub in sorted(glob.glob(f"{META}/deepcad/*/metadata.parquet")) + sorted(glob.glob(f"{META}/articraft/cadquery_single_tex*/metadata.parquet")):+ samples += load_parquet_rows("cadquery", "/".join(sub.split("/")[-3:-1]))+ samples += load_parquet_rows("openscad", "thingiverse/openscad")+ for sub in sorted(glob.glob(f"{META}/shadertoy/*/metadata.parquet")):+ samples += load_parquet_rows("glsl", "/".join(sub.split("/")[-3:-1]))+ samples += load_tree_rows("blender", "blender_distill", "*/*/code.py")+ samples += load_tree_rows("threejs", "threejs_distill", "*/*/code.html")+ samples += load_tree_rows("web", "animation2code", "*/*/index.html")+ samples = [s for s in samples if s["id"] not in held_out]+ print(f"[md_max] samples after source load: {len(samples)}", flush=True)++ # build (prompt, code) pairs from up to N caption variants+ pairs, seen = [], set()+ for s in samples:+ code = s["code"]+ if bad_chars(code) > 5:+ continue+ used = 0+ for kind in ("instruction", "detailed", "factory"):+ if used >= a.captions:+ break+ p = phrase(s["dialect"], kind, s["caps"].get(kind))+ if not p or len(p) < 20:+ continue+ h = hashlib.md5((p[:400] + "|" + code[:400]).encode()).hexdigest()+ if h in seen:+ continue+ seen.add(h)+ pairs.append({"dialect": s["dialect"], "subset": s["subset"], "id": s["id"], "caption": kind, "prompt": p, "code": code})+ used += 1+ print(f"[md_max] prompt-code pairs before length filter: {len(pairs)}", flush=True)++ # length: tokenize each unique code and prompt once+ ucode = {}+ uprompt = {}+ for p in pairs:+ ucode.setdefault(p["code"], None)+ uprompt.setdefault(p["prompt"], None)+ cache_path = "/wekafs/ict/hx_624/llm-ft/data/md_max/.ntok_cache.json"+ cache = {}+ if os.path.exists(cache_path):+ try: cache = json.load(open(cache_path))+ except Exception: cache = {}+ def _key(x): return hashlib.md5(x.encode()).hexdigest()+ for store in (ucode, uprompt):+ for k in list(store):+ v = cache.get(_key(k))+ if v is not None: store[k] = v+ codes = [k for k, v in ucode.items() if v is None]; prompts = [k for k, v in uprompt.items() if v is None]+ print(f"[md_max] tokenizing {len(codes)} unique codes + {len(prompts)} unique prompts on {a.workers} procs", flush=True)+ with ProcessPoolExecutor(a.workers, initializer=_init_tok) as ex:+ for k, n in zip(codes, ex.map(_ntok, codes, chunksize=64)):+ ucode[k] = n+ for k, n in zip(prompts, ex.map(_ntok, prompts, chunksize=256)):+ uprompt[k] = n+ os.makedirs(os.path.dirname(cache_path), exist_ok=True)+ json.dump({_key(k): v for store in (ucode, uprompt) for k, v in store.items()}, open(cache_path, "w"))++ OVERHEAD = 60 # chat template + system prompt + fence+ keep, stats = [], {}+ for p in pairs:+ n = ucode[p["code"]] + uprompt[p["prompt"]] + OVERHEAD+ if n > a.max_tokens:+ continue+ p["ntok"] = n+ keep.append(p)+ d = stats.setdefault(p["dialect"], {"n": 0, "tok": 0, "by_caption": {}})+ d["n"] += 1+ d["tok"] += n+ d["by_caption"][p["caption"]] = d["by_caption"].get(p["caption"], 0) + 1+ def _san(t): return t.replace("<image>", "<image >").replace("<video>", "<video >").replace("<audio>", "<audio >")+ for p_ in keep:+ p_["code"] = _san(p_["code"]); p_["prompt"] = _san(p_["prompt"])+ random.shuffle(keep)++ os.makedirs(a.out, exist_ok=True)+ val = keep[:600]+ train = keep[600:]+ for split, rows in (("train", train), ("val", val)):+ with open(f"{a.out}/{split}.jsonl", "w") as f:+ for p in rows:+ msgs = [{"role": "system", "content": SYS[p["dialect"]]},+ {"role": "user", "content": p["prompt"]},+ {"role": "assistant", "content": f"```{FENCE[p['dialect']]}\n{p['code'].strip()}\n```"}]+ f.write(json.dumps({"id": p["id"], "dialect": p["dialect"], "subset": p["subset"], "caption": p["caption"], "messages": msgs}, ensure_ascii=False) + "\n")+ tot_tok = sum(v["tok"] for v in stats.values())+ report = {"pairs_kept": len(keep), "tokens": tot_tok, "by_dialect": {k: {"n": v["n"], "Mtok": round(v["tok"] / 1e6, 1), "by_caption": v["by_caption"]} for k, v in sorted(stats.items(), key=lambda x: -x[1]["n"])}}+ json.dump(report, open(f"{a.out}/report.json", "w"), indent=2)+ print(json.dumps(report["by_dialect"], indent=1))+ print(f"[md_max] total samples {len(keep)} | tokens/epoch {tot_tok/1e6:.1f}M | train {len(train)} val {len(val)}", flush=True)+++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/build_mm_dataset.py b/toolkits/llamafactory/build_mm_dataset.pynew file mode 100644index 00000000..806cf266--- /dev/null+++ b/toolkits/llamafactory/build_mm_dataset.py@@ -0,0 +1,205 @@+"""Build the multimodal (render + instruction → code) datasets in standard LLaMA-Factory ShareGPT format.++Three variants are emitted from the SAME samples, registered as separate datasets so they can be mixed or ablated+against each other without rebuilding anything:++ 3dcv_mm_img image(s) only "<image>… Write the Blender Python code that builds this object."+ 3dcv_mm_img_text image(s) + instruction the realistic setting: a reference render AND what the user asked for+ 3dcv_mm_text instruction only the control, identical samples and split, no images++Format notes (checked against LLaMA-Factory's loader):+ * `images` is a list of absolute paths; the number of `<image>` placeholders in the user turn must equal+ len(images) — `mm_plugin._validate_input` raises otherwise.+ * `conversations` is [human, gpt]; `system` is a plain string column.+ * multimodal rows cannot be packed (`packing: false`), because image tokens must stay aligned with their sample.++Sources: bioinspired3d (tars, 4 views), 3dcodebench factories (tars, 4 views, benchmark factories excluded),+blender_distill / threejs_distill / animation2code (file trees, 4 views).++usage: python scripts/build_mm_dataset.py [--views 4] [--out data/mm]+"""+import argparse+import collections+import glob+import json+import os+import random+import tarfile++IMG_ROOT = "/wekafs/ict/hx_624/data/images"+BENCH = {os.path.basename(p).replace("_seed0", "") for p in glob.glob("/wekafs/ict/hx_624/data/3dcodebench/data/*_seed0")}++SYS = {+ "blender": "You are an expert in procedural 3D modeling with Blender Python (bpy). Write a complete, standalone Blender 5 Python script that builds the requested object from scratch (clear the default scene first). Output ONLY the code in one ```python block.",+ "threejs": "You are an expert three.js developer. Write a complete, standalone HTML page that renders the requested scene with three.js. Output ONLY the code in one ```html block.",+ "web": "You are an expert creative web developer. Write a complete, standalone HTML page (inline CSS/JS, no local assets) that reproduces the requested animation. Output ONLY the code in one ```html block.",+}+ASK_IMG = {+ "blender": "These are reference renders of one object. Write the Blender Python code that builds it.",+ "threejs": "These are reference renders of one scene. Write the single-file three.js page that renders it.",+ "web": "These are frames of one animation. Write the single-file animated web page that reproduces it.",+}+ASK_TEXT = {+ "blender": "Write the Blender Python code that builds this object: ",+ "threejs": "Write a single-file three.js page that renders: ",+ "web": "Write a single-file animated web page that reproduces: ",+}+ASK_BOTH = {+ "blender": "These are reference renders of one object. Write the Blender Python code that builds it.\n\nThe object: ",+ "threejs": "These are reference renders of one scene. Write the single-file three.js page that renders it.\n\nThe scene: ",+ "web": "These are frames of one animation. Write the single-file animated web page that reproduces it.\n\nThe animation: ",+}+FENCE = {"blender": "python", "threejs": "html", "web": "html"}+IMPERATIVE = ("write", "create", "implement", "make", "build", "generate", "design", "produce")+++def caption_of(caps):+ """the best available human description, normalised to a noun phrase (the ASK_* strings supply the verb)"""+ for k in ("user_instruction", "instruction", "detailed", "brief", "factory"):+ v = (caps.get(k) or "").strip()+ if not v or len(v) < 15:+ continue+ low = v.lower()+ if low.startswith(IMPERATIVE): # "Write a Blender script that builds a X" -> "a X"+ for sep in (" that builds ", " that renders ", " that reproduces ", " of ", " for "):+ if sep in low:+ v = v[low.index(sep) + len(sep):].strip()+ break+ else:+ continue+ return v[:1200]+ return None+++def add(rows, sample_id, dialect, imgs, caption, code):+ base = {"id": sample_id, "dialect": dialect, "system": SYS[dialect]}+ gpt = {"from": "gpt", "value": f"```{FENCE[dialect]}\n{code.strip()}\n```"}+ ph = "<image>" * len(imgs)+ out = {}+ out["img"] = dict(base, images=list(imgs),+ conversations=[{"from": "human", "value": ph + "\n" + ASK_IMG[dialect]}, gpt])+ if caption:+ out["img_text"] = dict(base, images=list(imgs),+ conversations=[{"from": "human", "value": ph + "\n" + ASK_BOTH[dialect] + caption}, gpt])+ out["text"] = dict(base, conversations=[{"from": "human", "value": ASK_TEXT[dialect] + caption}, gpt])+ for k, v in out.items():+ rows[k].append(v)+++def from_tars(pattern, dialect, rows, views, drop_bench=False):+ n = 0+ for t in sorted(glob.glob(pattern)):+ sub = os.path.basename(os.path.dirname(t))+ try:+ tf = tarfile.open(t)+ except Exception:+ continue+ members = collections.defaultdict(dict)+ for m in tf.getmembers():+ if not m.isfile():+ continue+ key = m.name.split("/")[0]+ if m.name.endswith("code.py"):+ members[key]["code"] = m+ elif m.name.endswith("captions.json"):+ members[key]["caps"] = m+ elif "/renders/view_" in m.name and m.name.endswith(".png"):+ members[key].setdefault("imgs", []).append(m)+ for key, d in members.items():+ if "code" not in d or "imgs" not in d:+ continue+ if drop_bench and key.replace("Factory", "").replace("_geo", "").replace("_tex", "") in BENCH:+ continue+ try:+ code = tf.extractfile(d["code"]).read().decode("utf-8", "replace")+ except Exception:+ continue+ if len(code.strip()) < 40:+ continue+ caps = {}+ if "caps" in d:+ try:+ caps = json.load(tf.extractfile(d["caps"]))+ except Exception:+ pass+ paths = []+ for m in sorted(d["imgs"], key=lambda m: m.name)[:views]:+ dst = os.path.join(IMG_ROOT, sub, key, os.path.basename(m.name))+ if not os.path.exists(dst):+ os.makedirs(os.path.dirname(dst), exist_ok=True)+ with open(dst, "wb") as f:+ f.write(tf.extractfile(m).read())+ paths.append(dst)+ if paths:+ add(rows, f"{sub}/{key}", dialect, paths, caption_of(caps), code)+ n += 1+ tf.close()+ return n+++def from_tree(root, dialect, rows, views, code_name):+ n = 0+ for cp in sorted(glob.glob(f"/wekafs/ict/hx_624/data/3dcodeverse_files/{root}/*/*/{code_name}")):+ d = os.path.dirname(cp)+ imgs = sorted(glob.glob(os.path.join(d, "renders", "*.png")))[:views]+ if not imgs:+ continue+ code = open(cp, encoding="utf-8", errors="replace").read()+ if len(code.strip()) < 40:+ continue+ caps = {}+ cj = os.path.join(d, "captions.json")+ if os.path.exists(cj):+ try:+ caps = json.load(open(cj))+ except Exception:+ pass+ add(rows, os.path.relpath(d, "/wekafs/ict/hx_624/data/3dcodeverse_files"), dialect, imgs, caption_of(caps), code)+ n += 1+ return n+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--out", default="/wekafs/ict/hx_624/llm-ft/data/mm")+ ap.add_argument("--views", type=int, default=4)+ a = ap.parse_args()+ os.makedirs(IMG_ROOT, exist_ok=True)+ rows = collections.defaultdict(list)+ got = {+ "bioinspired3d": from_tars("/wekafs/ict/hx_624/data/3dcodeverse_tars/bioinspired3d/*/*.tar", "blender", rows, a.views),+ "3dcodebench_factories": from_tars("/wekafs/ict/hx_624/data/3dcodeverse_tars/3dcodebench/factories_*/*.tar", "blender", rows, a.views, drop_bench=True),+ "blender_distill": from_tree("blender_distill", "blender", rows, a.views, "code.py"),+ "threejs_distill": from_tree("threejs_distill", "threejs", rows, a.views, "code.html"),+ "animation2code": from_tree("animation2code", "web", rows, a.views, "index.html"),+ }+ # one split shared by all three variants, so a comparison is never confounded by different val sets+ ids = sorted({r["id"] for r in rows["img"]})+ random.seed(7)+ random.shuffle(ids)+ val_ids = set(ids[:200])+ os.makedirs(a.out, exist_ok=True)+ info = {}+ for variant in ("img", "img_text", "text"):+ tr = [r for r in rows[variant] if r["id"] not in val_ids]+ va = [r for r in rows[variant] if r["id"] in val_ids]+ for split, part in (("train", tr), ("val", va)):+ name = f"mm_{variant}_{split}"+ json.dump(part, open(os.path.join(a.out, f"{name}.json"), "w"), ensure_ascii=False)+ cols = {"messages": "conversations", "system": "system"}+ if variant != "text":+ cols["images"] = "images"+ info[f"3dcv_{name}"] = {"file_name": f"{name}.json", "formatting": "sharegpt", "columns": cols}+ print(f"[mm] {variant:9s} train={len(tr):6,} val={len(va):4,}")+ json.dump(info, open(os.path.join(a.out, "dataset_info.json"), "w"), indent=2)+ # a sanity check on the invariant LLaMA-Factory enforces+ bad = [r for v in ("img", "img_text") for r in rows[v]+ if r["conversations"][0]["value"].count("<image>") != len(r.get("images", []))]+ print(f"[mm] sources: {got}")+ print(f"[mm] placeholder/image-count mismatches: {len(bad)} (must be 0)")+ print(f"[mm] dialects: {dict(collections.Counter(r['dialect'] for r in rows['img']))}")+ print(f"[mm] views per sample: {a.views} | shared val split: {len(val_ids)} ids -> {a.out}")+++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/cache_ntok.py b/toolkits/llamafactory/cache_ntok.pynew file mode 100644index 00000000..f944764f--- /dev/null+++ b/toolkits/llamafactory/cache_ntok.py@@ -0,0 +1,20 @@+"""One-off: cache per-sample token counts (Qwen3.5 chat template) for every multidialect train/test file and the bootstrapped set.+Writes <file>.ntok (one int per line, same order). Build scripts read this instead of re-tokenizing."""+import json, os, sys, glob+from multiprocessing import Pool+FILES = sorted(glob.glob("/wekafs/ict/hx_624/llm-ft/data/multidialect/*/train.jsonl") + glob.glob("/wekafs/ict/hx_624/llm-ft/data/multidialect/*/test.jsonl")) + ["/wekafs/ict/hx_624/llm-ft/data/sft_v13_boot/train.jsonl"]+tok = None+def init():+ global tok+ from transformers import AutoTokenizer+ tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+def count(line):+ r = json.loads(line); return len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)+if __name__ == "__main__":+ with Pool(12, initializer=init) as pool:+ for f in FILES:+ out = f + ".ntok"+ if os.path.exists(out) and sum(1 for _ in open(out)) == sum(1 for _ in open(f)): print("cached", f); continue+ lines = open(f).read().splitlines()+ ns = pool.map(count, lines, chunksize=256)+ open(out, "w").write("\n".join(map(str, ns)) + "\n"); print(f, len(ns), "done", flush=True)diff --git a/toolkits/llamafactory/convert_subdirs.py b/toolkits/llamafactory/convert_subdirs.pynew file mode 100644index 00000000..739f04e8--- /dev/null+++ b/toolkits/llamafactory/convert_subdirs.py@@ -0,0 +1,294 @@+"""Convert EVERY 3DCodeVerse subdirectory into its own LLaMA-Factory dataset + a data-quality report.++One output folder per source subdir, named `<subdir>_llamafactory/`, so it can sit next to the source on the+Hub (e.g. `3dcodebench/factories_geo/` → `3dcodebench/factories_geo_llamafactory/`):++ <subdir>_llamafactory/+ train.json ShareGPT: [{system, conversations:[human, gpt]}] — one row per caption variant+ dataset_info.json drop-in LLaMA-Factory registry entry for this folder+ qc.json per-subdir quality counters + token stats+ README.md what it is, how many rows, what was dropped and why++Quality checks applied per sample (a sample can fail several):+ empty_code / short_code (<40 chars) / bad_chars (control chars or U+FFFD) / no_caption /+ duplicate_code (same code hash inside the subdir) / syntax_bad (AST for Python dialects, brace balance for+ scad-glsl, tag check for html) / too_long (>8192 tokens, still exported but flagged) / media_placeholder+ (literal <image>/<video>/<audio> — breaks the qwen3_5 template unless escaped; we escape it and flag it)++usage: python scripts/convert_subdirs.py [--out DIR] [--workers 16] [--limit_subdirs N]+"""+import argparse+import ast+import collections+import glob+import hashlib+import json+import os+import re+import sys+import unicodedata+from concurrent.futures import ProcessPoolExecutor++import pandas as pd++sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/scripts")+META = "/wekafs/ict/hx_624/data/3dcodeverse_meta"+FILES = "/wekafs/ict/hx_624/data/3dcodeverse_files"+BENCH_DIR = "/wekafs/ict/hx_624/data/3dcodebench/data"++# subdir prefix -> dialect+DIALECT_OF = {+ "3dcodebench": "blender", "bioinspired3d": "blender", "blender_distill": "blender",+ "deepcad": "cadquery", "articraft": "cadquery", "shadertoy": "glsl",+ "thingiverse": "openscad", "threejs_distill": "threejs", "animation2code": "web",+}+SYS = {+ "blender": "You are an expert in procedural 3D modeling with Blender Python (bpy). Given a description of an object, write a complete, standalone Blender 5 Python script that builds it from scratch (clear the default scene first). Output ONLY the code in one ```python block.",+ "cadquery": "You are an expert CAD programmer. Given a description of a part, write a complete, standalone CadQuery (Python) script that builds it and leaves the final solid in a variable named `result`. Output ONLY the code in one ```python block.",+ "openscad": "You are an expert OpenSCAD programmer. Given a request, write a complete, standalone OpenSCAD (.scad) program that produces the described model. Output ONLY the code in one ```openscad block.",+ "glsl": "You are an expert shader programmer. Given a description, write a complete Shadertoy-style GLSL ES 3.00 fragment shader defining `void mainImage(out vec4 fragColor, in vec2 fragCoord)`. Output ONLY the code in one ```glsl block.",+ "threejs": "You are an expert three.js developer. Given a description, write a complete, standalone HTML page that renders the described 3D scene with three.js. Output ONLY the code in one ```html block.",+ "web": "You are an expert creative web developer. Given a description of an animation, write a complete, standalone HTML page (inline CSS/JS, no local assets) that reproduces it. Output ONLY the code in one ```html block.",+ "urdf": "You are an expert in articulated 3D assets. Given a description, write a complete, self-contained URDF file for it. Output ONLY the code in one ```xml block.",+}+FENCE = {"blender": "python", "cadquery": "python", "openscad": "openscad", "glsl": "glsl",+ "threejs": "html", "web": "html", "urdf": "xml"}+LEAD = {+ "blender": "Create the 3D object described below using Python Blender code: ",+ "cadquery": "Write a CadQuery (Python) script that builds the following CAD model: ",+ "openscad": "Write an OpenSCAD program for the following: ",+ "glsl": "Write a GLSL fragment shader for Shadertoy that renders: ",+ "threejs": "Write a standalone three.js page that renders: ",+ "web": "Write a standalone animated web page that reproduces: ",+ "urdf": "Write a URDF description of: ",+}+IMPERATIVE = ("write", "create", "implement", "make", "build", "generate", "design", "produce")+MEDIA = re.compile(r"<(image|video|audio)>")+++def bad_char_count(s):+ return sum(1 for ch in s if unicodedata.category(ch) == "Cc" and ch not in "\n\t\r") + s.count("�")+++def syntax_ok(code, dialect):+ if dialect in ("blender", "cadquery"):+ try:+ ast.parse(code)+ return True+ except Exception:+ return False+ if dialect == "openscad":+ return code.count("{") == code.count("}") and code.count("(") == code.count(")")+ if dialect == "glsl":+ return code.count("{") == code.count("}") and "void" in code+ if dialect in ("threejs", "web"):+ return "<" in code and ">" in code+ if dialect == "urdf":+ return "<robot" in code or "<link" in code+ return True+++def phrase(dialect, kind, text):+ t = (text or "").strip()+ if not t:+ return None+ if kind == "instruction" and t.lower().startswith(IMPERATIVE):+ return t+ return LEAD[dialect] + t+++def normalize_caps(caps):+ """tree sources use their own caption keys"""+ caps = {k: v for k, v in caps.items() if isinstance(v, str) and v.strip()}+ if any(k in caps for k in ("instruction", "detailed", "factory")) and "user_instruction" not in caps and "brief" not in caps:+ return caps+ norm = {}+ if caps.get("user_instruction"):+ norm["instruction"] = caps["user_instruction"]+ if caps.get("instruction"):+ norm["detailed"] = caps["instruction"]+ if caps.get("brief"):+ extra = caps.get("function", "")+ norm["instruction"] = (caps["brief"] + ((" — " + extra) if extra and extra not in caps["brief"] else "")).strip()+ if caps.get("source_title") and "detailed" not in norm:+ norm["detailed"] = caps["source_title"]+ if caps.get("style") and norm.get("detailed"):+ norm["detailed"] += f" (style: {caps['style']})"+ return norm or caps+++def load_subdir(sub):+ """-> (dialect, [ {id, name, code, caps, n_images} ]) for a parquet subdir or a file tree"""+ top = sub.split("/")[0]+ dialect = DIALECT_OF.get(top, "blender")+ if top == "articraft" and "urdf" in sub:+ dialect = "urdf"+ rows = []+ p = f"{META}/{sub}/metadata.parquet"+ if os.path.exists(p):+ df = pd.read_parquet(p)+ for r in df.itertuples():+ caps = r.captions if isinstance(r.captions, dict) else (json.loads(r.captions) if isinstance(r.captions, str) else {})+ meta = {}+ try:+ meta = json.loads(r.meta_json) if isinstance(r.meta_json, str) else (r.meta_json or {})+ except Exception:+ pass+ rows.append({"id": str(r.id), "name": str(getattr(r, "name", "") or ""),+ "code": getattr(r, "code", None) or "", "caps": dict(caps),+ "n_images": len(meta.get("renders", []) or [])})+ return dialect, rows+ # file tree+ for cp in sorted(glob.glob(f"{FILES}/{sub}/*/code.py") + glob.glob(f"{FILES}/{sub}/*/code.html")+ + glob.glob(f"{FILES}/{sub}/*/index.html") + glob.glob(f"{FILES}/{sub}/*/*/code.py")+ + glob.glob(f"{FILES}/{sub}/*/*/code.html") + glob.glob(f"{FILES}/{sub}/*/*/index.html")):+ d = os.path.dirname(cp)+ try:+ code = open(cp, encoding="utf-8", errors="replace").read()+ caps = json.load(open(os.path.join(d, "captions.json"))) if os.path.exists(os.path.join(d, "captions.json")) else {}+ except Exception:+ continue+ rows.append({"id": os.path.relpath(d, FILES), "name": os.path.basename(d), "code": code,+ "caps": caps, "n_images": len(glob.glob(os.path.join(d, "renders", "*.png")))})+ return dialect, rows+++_TOK = None+++def _init():+ global _TOK+ from transformers import AutoTokenizer+ _TOK = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+++def convert_one(args):+ sub, out_root, bench_factories = args+ dialect, rows = load_subdir(sub)+ if not rows:+ return {"subdir": sub, "error": "no rows"}+ qc = collections.Counter()+ seen_code, out, tok_lens, img_counts = {}, [], [], []+ for r in rows:+ code = r["code"] if isinstance(r["code"], str) else ""+ img_counts.append(r.get("n_images", 0))+ if not code.strip():+ qc["empty_code"] += 1+ continue+ if len(code.strip()) < 40:+ qc["short_code"] += 1+ continue+ if bad_char_count(code) > 5:+ qc["bad_chars"] += 1+ continue+ h = hashlib.md5(code.encode()).hexdigest()+ if h in seen_code:+ qc["duplicate_code"] += 1+ continue+ seen_code[h] = 1+ if not syntax_ok(code, dialect):+ qc["syntax_bad"] += 1 # exported anyway, but counted+ if MEDIA.search(code) or any(MEDIA.search(v) for v in r["caps"].values() if isinstance(v, str)):+ qc["media_placeholder"] += 1+ code = MEDIA.sub(lambda m: f"<{m.group(1)} >", code)+ norm_name = r["name"].replace("Factory", "").replace("_geo", "").replace("_tex", "")+ if r["name"] in bench_factories or r["name"].rsplit("_", 1)[0] in bench_factories or norm_name in bench_factories:+ qc["benchmark_overlap"] += 1 # flagged, and excluded below+ continue+ caps = normalize_caps(r["caps"])+ n_before = len(out)+ for kind in ("instruction", "detailed", "factory"):+ pr = phrase(dialect, kind, caps.get(kind))+ if not pr or len(pr) < 20:+ continue+ pr = MEDIA.sub(lambda m: f"<{m.group(1)} >", pr)+ n = len(_TOK(pr).input_ids) + len(_TOK(code).input_ids) + 60+ tok_lens.append(n)+ if n > 8192:+ qc["too_long_gt8192"] += 1+ out.append({"id": r["id"], "caption_type": kind, "n_tokens": n,+ "system": SYS[dialect],+ "conversations": [{"from": "human", "value": pr},+ {"from": "gpt", "value": f"```{FENCE[dialect]}\n{code.strip()}\n```"}]})+ if len(out) == n_before:+ qc["no_caption"] += 1+ name = sub.replace("/", "__")+ d = os.path.join(out_root, sub + "_llamafactory")+ os.makedirs(d, exist_ok=True)+ json.dump(out, open(os.path.join(d, "train.json"), "w"), ensure_ascii=False)+ key = f"3dcv_{name}"+ json.dump({key: {"file_name": "train.json", "formatting": "sharegpt",+ "columns": {"messages": "conversations", "system": "system"}}},+ open(os.path.join(d, "dataset_info.json"), "w"), indent=2)+ tok_lens.sort()+ stat = {"subdir": sub, "dialect": dialect, "source_samples": len(rows), "exported_pairs": len(out),+ "unique_samples_exported": len(seen_code) - qc["benchmark_overlap"],+ "images_per_sample_median": (sorted(img_counts)[len(img_counts) // 2] if img_counts else 0),+ "tokens_p50": tok_lens[len(tok_lens) // 2] if tok_lens else 0,+ "tokens_p90": tok_lens[int(len(tok_lens) * 0.9)] if tok_lens else 0,+ "tokens_max": tok_lens[-1] if tok_lens else 0,+ **{k: v for k, v in qc.items()}}+ json.dump(stat, open(os.path.join(d, "qc.json"), "w"), indent=2)+ issues = "\n".join(f"- **{k}**: {v}" for k, v in sorted(qc.items(), key=lambda x: -x[1])) or "- none"+ open(os.path.join(d, "README.md"), "w").write(f"""# {sub} — LLaMA-Factory format++Generated by `finetune/scripts/convert_subdirs.py` from `{sub}/metadata.parquet` (or its file tree).++| | |+|---|---|+| dialect | `{dialect}` |+| source samples | {len(rows):,} |+| exported pairs | {len(out):,} (one row per caption variant: instruction / detailed / factory) |+| unique code samples exported | {stat['unique_samples_exported']:,} |+| token length p50 / p90 / max | {stat['tokens_p50']} / {stat['tokens_p90']} / {stat['tokens_max']} |+| renders per sample (median) | {stat['images_per_sample_median']} |++## Data-quality findings++{issues}++`syntax_bad` rows are still exported (a shader can be valid without balanced braces in our crude check); every+other counter means the sample was dropped. `benchmark_overlap` = the sample belongs to one of the 212+3DCodeBench evaluation factories and is excluded to avoid test leakage.++## Use++```yaml+dataset_dir: <this folder>+dataset: {key}+```+""")+ return stat+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--out", default="/wekafs/ict/hx_624/hf_subdirs")+ ap.add_argument("--workers", type=int, default=16)+ ap.add_argument("--limit_subdirs", type=int, default=None)+ a = ap.parse_args()+ bench = frozenset(os.path.basename(p).replace("_seed0", "") for p in glob.glob(f"{BENCH_DIR}/*_seed0"))+ subs = ["/".join(p.split("/")[-3:-1]) for p in sorted(glob.glob(f"{META}/*/*/metadata.parquet"))]+ subs += ["blender_distill", "threejs_distill", "animation2code"]+ if a.limit_subdirs:+ subs = subs[: a.limit_subdirs]+ print(f"[conv] {len(subs)} subdirs -> {a.out}", flush=True)+ os.makedirs(a.out, exist_ok=True)+ stats = []+ with ProcessPoolExecutor(a.workers, initializer=_init) as ex:+ for st in ex.map(convert_one, [(s, a.out, bench) for s in subs]):+ stats.append(st)+ print(f"[conv] {st.get('subdir'):45s} pairs={st.get('exported_pairs', 0):7,} "+ f"issues={ {k: v for k, v in st.items() if k in ('empty_code','short_code','bad_chars','duplicate_code','syntax_bad','no_caption','too_long_gt8192','media_placeholder','benchmark_overlap')} }", flush=True)+ df = pd.DataFrame(stats)+ df.to_csv(os.path.join(a.out, "quality_report.csv"), index=False)+ try:+ df.to_excel(os.path.join(a.out, "quality_report.xlsx"), index=False)+ except Exception as e:+ print("[conv] xlsx skipped:", e)+ json.dump(stats, open(os.path.join(a.out, "quality_report.json"), "w"), indent=2)+ print(f"[conv] wrote quality_report.{{csv,xlsx,json}} | total exported pairs: {int(df['exported_pairs'].sum()):,}")+++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/extract.py b/toolkits/llamafactory/extract.pynew file mode 100644index 00000000..55eaa783--- /dev/null+++ b/toolkits/llamafactory/extract.py@@ -0,0 +1,220 @@+"""Dialect-aware code extraction from free-form model output.++Why this exists: instruct/reasoning models do not answer with bare code. A single response may contain+a <think> block (often with *draft* code), prose, several fenced blocks (setup shell commands, a JSON+snippet, the real program, then a "here's a variant" block), or no fence at all. Taking "the longest+fenced block" (the old rule) picks the draft or the variant often enough to matter.++extract(text, dialect) -> (code, meta)+ * strips reasoning blocks (<think>…</think>, <thinking>…</thinking>, <reasoning>…</reasoning>);+ if the model never closed the block the answer is considered truncated-in-thought and we fall back+ to mining the thought itself (meta["from_think"]=True) so we can still score something.+ * collects candidates: every fenced block (closed or trailing-unterminated), plus the whole visible+ text as a last resort.+ * scores each candidate by: dialect markers (import bpy / cq. / mainImage / THREE. / module …),+ language tag on the fence, syntax validity (ast.parse for python dialects, structural checks+ otherwise), and penalises obvious non-answers (shell commands, pip installs, json, error logs).+ * ties are broken toward the LAST high-scoring block (models put the final program last), then length.++meta keys: n_blocks, chosen, score, lang, had_think, think_chars, from_think, no_fence, valid, markers.+"""+import ast+import json+import re++THINK_RE = re.compile(r"<(think|thinking|reasoning)>(.*?)</\1>", re.S | re.I)+THINK_OPEN_RE = re.compile(r"<(think|thinking|reasoning)>(.*)$", re.S | re.I)+FENCE_RE = re.compile(r"```([A-Za-z0-9_+.#-]*)[ \t]*\r?\n(.*?)```", re.S)+FENCE_OPEN_RE = re.compile(r"```([A-Za-z0-9_+.#-]*)[ \t]*\r?\n(.*)$", re.S)++# per-dialect: fence language tags, regexes that prove the block is that dialect, and how to syntax-check+DIALECTS = {+ "blender": {+ "langs": {"python", "py", "python3", "blender", "bpy"},+ "markers": [r"^\s*import\s+bpy\b", r"\bbpy\.(ops|data|context)\b"],+ "check": "python",+ },+ "cadquery": {+ "langs": {"python", "py", "python3", "cadquery", "cq"},+ "markers": [r"^\s*import\s+cadquery\b", r"^\s*from\s+cadquery\b", r"\bcq\.Workplane\b", r"\bcadquery\.Workplane\b"],+ "check": "python",+ },+ "openscad": {+ "langs": {"openscad", "scad"},+ "markers": [r"^\s*module\s+\w+\s*\(", r"\b(cube|cylinder|sphere|polyhedron|linear_extrude|rotate_extrude|hull|minkowski)\s*\("],+ "check": "scad",+ },+ "glsl": {+ "langs": {"glsl", "frag", "fragment", "fs", "c", "cpp"},+ "markers": [r"void\s+mainImage\s*\(", r"\bgl_FragColor\b", r"^\s*#version\b", r"\bfragColor\b"],+ "check": "glsl",+ },+ "threejs": {+ "langs": {"html", "htm", "javascript", "js", "jsx"},+ "markers": [r"<!DOCTYPE\s+html", r"<html", r"\bnew\s+THREE\.", r"\bTHREE\.[A-Z]\w+"],+ "check": "html",+ },+ # generic python (used by "auto" when nothing dialect-specific matches)+ "python": {"langs": {"python", "py", "python3"}, "markers": [r"^\s*(import|from|def|class)\s+\w"], "check": "python"},+}++# blocks that are never the answer+NEGATIVE = [+ (re.compile(r"^\s*(pip|conda|apt|sudo|blender|python)\s+(install|-m|-b)\b", re.M), 4),+ (re.compile(r"^\s*[$#>]\s+\w"), 3), # shell prompt+ (re.compile(r"^\s*\{[\s\S]*\}\s*$"), 2), # bare JSON+ (re.compile(r"^\s*(Traceback|Error:|error:)", re.M), 3), # pasted error log+]+++def _syntax_ok(code, kind):+ if kind == "python":+ try:+ ast.parse(code)+ return True+ except SyntaxError:+ return False+ except Exception:+ return False+ if kind == "scad":+ return code.count("{") == code.count("}") and code.count("(") == code.count(")") and bool(re.search(r"[;{]", code))+ if kind == "glsl":+ return code.count("{") == code.count("}") and "void" in code+ if kind == "html":+ return ("<" in code and ">" in code) or "THREE" in code+ return True+++def _candidates(text):+ """[(lang, code, start_index)] for every fenced block, plus a trailing unterminated one."""+ out, last_end = [], 0+ for m in FENCE_RE.finditer(text):+ out.append((m.group(1).lower(), m.group(2), m.start()))+ last_end = m.end()+ tail = text[last_end:]+ m = FENCE_OPEN_RE.search(tail)+ if m and m.group(2).strip():+ out.append((m.group(1).lower(), m.group(2), last_end + m.start()))+ return out+++def _first_code_line(text):+ """No fence anywhere: drop the prose preamble, keep from the first code-looking line on."""+ pat = re.compile(r"^\s*(import\s+\w|from\s+\w|def\s+\w|class\s+\w|#\s*!|#version|module\s+\w|void\s+\w|<!DOCTYPE|<html|const\s+\w|let\s+\w|var\s+\w|function\s+\w|\$fn\s*=)")+ lines = text.splitlines()+ for i, ln in enumerate(lines):+ if pat.match(ln):+ return "\n".join(lines[i:])+ return text+++def _score(lang, code, spec):+ s, markers = 0.0, 0+ for pat in spec["markers"]:+ if re.search(pat, code, re.M):+ markers += 1+ s += 3.0 * min(markers, 2)+ if lang in spec["langs"]:+ s += 2.0+ elif lang == "":+ s += 0.5+ valid = _syntax_ok(code, spec["check"])+ if valid:+ s += 2.0+ for pat, pen in NEGATIVE:+ if pat.search(code):+ s -= pen+ if len(code.strip()) < 40:+ s -= 2.0+ return s, valid, markers+++def extract(text, dialect="auto"):+ meta = {"n_blocks": 0, "chosen": None, "score": None, "lang": None, "had_think": False,+ "think_chars": 0, "from_think": False, "no_fence": False, "valid": False, "markers": 0,+ "dialect": dialect}+ if not text:+ return "", meta++ # 1) reasoning blocks+ thinks = THINK_RE.findall(text)+ visible = THINK_RE.sub("", text)+ if thinks:+ meta["had_think"] = True+ meta["think_chars"] = sum(len(t[1]) for t in thinks)+ m = THINK_OPEN_RE.search(visible)+ if m: # unterminated <think>: everything after it is thought+ meta["had_think"] = True+ meta["think_chars"] += len(m.group(2))+ visible = visible[: m.start()]++ def _pick(src):+ cands = _candidates(src)+ if not cands:+ return None, None+ specs = [DIALECTS[dialect]] if dialect in DIALECTS else list(DIALECTS.values())+ best = None+ for i, (lang, code, _pos) in enumerate(cands):+ sc = max((_score(lang, code, sp) for sp in specs), key=lambda x: x[0])+ key = (round(sc[0], 3), i, len(code)) # score, then LAST block, then length+ if best is None or key > best[0]:+ best = (key, i, lang, code, sc)+ return cands, best++ cands, best = _pick(visible)+ if best is None and visible.strip(): # no fence in the visible answer+ code = _first_code_line(visible)+ specs = [DIALECTS[dialect]] if dialect in DIALECTS else list(DIALECTS.values())+ sc = max((_score("", code, sp) for sp in specs), key=lambda x: x[0])+ meta.update(n_blocks=0, no_fence=True, chosen=-1, score=round(sc[0], 2), lang="", valid=sc[1], markers=sc[2])+ return code.strip("\n"), meta+ if best is None: # nothing visible: mine the thought+ cands, best = _pick(text)+ meta["from_think"] = True+ if best is None:+ return "", meta++ meta.update(n_blocks=len(cands), chosen=best[1], score=round(best[4][0], 2), lang=best[2],+ valid=best[4][1], markers=best[4][2])+ return best[3].strip("\n"), meta+++def extract_code(text, dialect="auto"):+ """Backwards-compatible single-value API (old callers import this name from generate.py)."""+ return extract(text, dialect)[0]+++def summarize(metas):+ """Aggregate extraction metadata over a run -> dict for extract_stats.json."""+ n = max(1, len(metas))+ return {+ "n": len(metas),+ "with_think": sum(m.get("had_think", False) for m in metas),+ "mean_think_chars": round(sum(m.get("think_chars", 0) for m in metas) / n, 1),+ "no_fence": sum(m.get("no_fence", False) for m in metas),+ "from_think_fallback": sum(m.get("from_think", False) for m in metas),+ "multi_block": sum((m.get("n_blocks") or 0) > 1 for m in metas),+ "mean_blocks": round(sum(m.get("n_blocks") or 0 for m in metas) / n, 2),+ "chosen_not_last": sum(m.get("chosen") is not None and m.get("n_blocks") and m["chosen"] != m["n_blocks"] - 1 for m in metas),+ "syntax_valid": sum(m.get("valid", False) for m in metas),+ "no_dialect_marker": sum((m.get("markers") or 0) == 0 for m in metas),+ "empty": sum((m.get("score") is None) for m in metas),+ }+++if __name__ == "__main__": # tiny self-test: python eval/extract.py+ T = [+ ("<think>let me draft\n```python\nimport bpy\nbpy.ops.mesh.primitive_cube_add() # draft\n```\nno, better</think>\n"+ "Here is the script:\n```python\nimport bpy\nbpy.ops.mesh.primitive_uv_sphere_add(radius=2)\n```\nThat creates a sphere.",+ "blender", "uv_sphere"),+ ("First install:\n```bash\npip install cadquery\n```\nThen:\n```python\nimport cadquery as cq\nresult = cq.Workplane('XY').box(1,2,3)\n```",+ "cadquery", "Workplane"),+ ("Sure!\nimport bpy\nbpy.ops.mesh.primitive_cone_add()\n", "blender", "cone"),+ ("```\n$fn = 64;\nmodule part(){ cube([10,10,2]); }\npart();\n```", "openscad", "module part"),+ ("```glsl\nvoid mainImage(out vec4 O, in vec2 U){ O = vec4(1.0); }\n```", "glsl", "mainImage"),+ ("<think>thinking forever without closing\n```python\nimport bpy\nbpy.ops.mesh.primitive_torus_add()\n```",+ "blender", "torus"),+ ]+ for text, d, needle in T:+ code, meta = extract(text, d)+ print(("OK " if needle in code else "FAIL"), d, json.dumps({k: meta[k] for k in ("n_blocks", "chosen", "score", "had_think", "from_think", "no_fence", "valid")}))diff --git a/toolkits/llamafactory/fetch_hf_files.py b/toolkits/llamafactory/fetch_hf_files.pynew file mode 100644index 00000000..a370f5b3--- /dev/null+++ b/toolkits/llamafactory/fetch_hf_files.py@@ -0,0 +1,30 @@+"""Download only files matching a regex from an HF dataset repo (handles tree pagination, no file locks).+usage: python fetch_hf_files.py REPO OUTDIR REGEX [--subdir SUB] [--workers 32]"""+import argparse, os, re, sys, requests+from concurrent.futures import ThreadPoolExecutor+from huggingface_hub import HfApi+ap = argparse.ArgumentParser(); ap.add_argument("repo"); ap.add_argument("out"); ap.add_argument("regex")+ap.add_argument("--subdir", default=None); ap.add_argument("--workers", type=int, default=32)+a = ap.parse_args()+tok = os.environ["HF_TOKEN"]; api = HfApi(token=tok); pat = re.compile(a.regex)+files = [f.path for f in api.list_repo_tree(a.repo, repo_type="dataset", path_in_repo=a.subdir, recursive=True) if getattr(f, "size", None) is not None and pat.search(f.path)]+print(f"[fetch] {len(files)} files match", flush=True)+sess = requests.Session(); sess.headers["Authorization"] = f"Bearer {tok}"+def get(p):+ dst = os.path.join(a.out, p)+ if os.path.exists(dst) and os.path.getsize(dst) > 0: return 0+ os.makedirs(os.path.dirname(dst), exist_ok=True)+ for attempt in range(3):+ try:+ r = sess.get(f"https://huggingface.co/datasets/{a.repo}/resolve/main/{p}", timeout=60)+ if r.status_code == 200:+ open(dst, "wb").write(r.content); return 1+ except Exception as e:+ pass+ print("FAIL", p, flush=True); return -1+n = 0+with ThreadPoolExecutor(a.workers) as ex:+ for i, r in enumerate(ex.map(get, files)):+ n += (r == 1)+ if (i + 1) % 500 == 0: print(f"[fetch] {i+1}/{len(files)}", flush=True)+print(f"[fetch] done: {n} downloaded, {len(files)} total", flush=True)diff --git a/toolkits/llamafactory/finalize_glsl.py b/toolkits/llamafactory/finalize_glsl.pynew file mode 100644index 00000000..bbc883c1--- /dev/null+++ b/toolkits/llamafactory/finalize_glsl.py@@ -0,0 +1,85 @@+"""Final consolidation for the flattened Shadertoy corpus.++Order of preference, so the data stays as close to the author's source as possible:+ 1. the ORIGINAL merged program, if the (now correct) validator compiles it -> untouched+ 2. the agent-repaired program from glsl_repair/<cat>_fixed/, if it compiles -> repaired+ 3. otherwise -> kept, marked compiles=False+Writes the `compiles` / `repair` columns back into every shadertoy *_llamafactory_flat parquet.+"""+import collections, glob, hashlib, json, os, re, sys+from concurrent.futures import ProcessPoolExecutor+import pandas as pd+sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/scripts")+from qc_repair import code_of, glsl_compile++REPAIR = "/wekafs/ict/hx_624/glsl_repair"++def load_fixed():+ out = {}+ for cat in ("overload", "undeclared", "const_init", "syntax", "redefinition", "type", "other"):+ d = f"{REPAIR}/{cat}_fixed"+ idx = f"{REPAIR}/{cat}/index.json"+ if not (os.path.isdir(d) and os.path.exists(idx)):+ continue+ meta = json.load(open(idx))+ for f in glob.glob(f"{d}/*.glsl"):+ stem = os.path.basename(f)[:-5]+ h = (meta.get(stem) or {}).get("hash")+ if h:+ out[h] = (cat, open(f).read())+ return out++def job(args):+ h, body, fixed = args+ ok, _ = glsl_compile(body)+ if ok:+ return h, True, "original", None+ if fixed:+ cat, code = fixed+ ok2, _ = glsl_compile(code)+ if ok2:+ return h, True, f"repaired:{cat}", code+ return h, False, "unfixable", None++def main():+ fixed = load_fixed()+ print(f"[fin] agent repairs available: {len(fixed)}", flush=True)+ dirs = sorted(glob.glob("/wekafs/ict/hx_624/hf_subdirs/shadertoy/*_llamafactory_flat"))+ uniq, where = {}, collections.defaultdict(list)+ for d in dirs:+ df = pd.read_parquet(os.path.join(d, "train.parquet"))+ for i, row in enumerate(df.itertuples()):+ b = code_of(row.conversations[1]["value"]); h = hashlib.md5(b.encode()).hexdigest()+ uniq.setdefault(h, b); where[h].append(d)+ items = [(h, b, fixed.get(h)) for h, b in uniq.items()]+ print(f"[fin] validating {len(items)} unique programs", flush=True)+ res = {}+ with ProcessPoolExecutor(28) as ex:+ for n, (h, ok, how, code) in enumerate(ex.map(job, items, chunksize=8)):+ res[h] = (ok, how, code)+ if (n+1) % 10000 == 0:+ g = sum(1 for v in res.values() if v[0])+ print(f"[fin] {n+1}/{len(items)} ok={g} ({100*g/(n+1):.1f}%)", flush=True)+ stats = collections.Counter(v[1] for v in res.values())+ for d in dirs:+ p = os.path.join(d, "train.parquet"); df = pd.read_parquet(p)+ conv = df["conversations"].tolist(); okc, howc = [], []+ for c in conv:+ b = code_of(c[1]["value"]); h = hashlib.md5(b.encode()).hexdigest()+ ok, how, code = res.get(h, (False, "unknown", None))+ if code:+ c[1]["value"] = "```glsl\n" + code.strip() + "\n```"+ okc.append(ok); howc.append(how)+ df["conversations"] = conv; df["compiles"] = okc; df["repair"] = howc+ df.to_parquet(p, index=False, compression="zstd")+ json.dump({"rows": len(df), "compiles": int(sum(okc)),+ "repaired": int(sum(1 for x in howc if x.startswith("repaired")))},+ open(os.path.join(d, "qc.json"), "w"), indent=2)+ good = sum(1 for v in res.values() if v[0])+ json.dump({"unique_programs": len(items), "compile_ok": good,+ "compile_rate": round(good/len(items), 4), "how": dict(stats)},+ open("/wekafs/ict/hx_624/hf_subdirs/glsl_final_report.json", "w"), indent=2)+ print(f"[fin] FINAL {good}/{len(items)} = {100*good/len(items):.1f}% | {dict(stats)}", flush=True)++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/flatten_multifile.py b/toolkits/llamafactory/flatten_multifile.pynew file mode 100644index 00000000..23e9af06--- /dev/null+++ b/toolkits/llamafactory/flatten_multifile.py@@ -0,0 +1,181 @@+"""Flatten every multi-file sample into ONE self-contained file, so the whole corpus is single-file SFT data.++Two sources are multi-file in the original dataset and both can be flattened exactly:++ shadertoy `shader_json.renderpass` holds the source of every tab. Shadertoy itself compiles+ `Common + <pass>`, so we emit that concatenation verbatim:+ * the Image pass with Common prepended → the sample's main program+ * each Buffer pass with Common prepended → an extra sample (it is a complete shader too)+ A shader whose passes read `iChannel` as a *buffer* is still emitted, with a one-line comment+ stating which channels are buffers, because the code itself is complete.+ articraft `code` is empty in the metadata; the real `code.urdf` lives inside the subdir tars, which are+ downloaded separately and extracted by `--urdf_tars`.++Everything is then re-emitted in LLaMA-Factory ShareGPT format next to the other converted subdirs.++usage:+ python scripts/flatten_multifile.py --shadertoy [--out /wekafs/ict/hx_624/hf_subdirs]+ python scripts/flatten_multifile.py --urdf_tars /wekafs/ict/hx_624/data/3dcodeverse_tars+"""+import argparse+import collections+import glob+import io+import json+import os+import re+import tarfile++import pandas as pd++META = "/wekafs/ict/hx_624/data/3dcodeverse_meta"+SYS_GLSL = ("You are an expert shader programmer. Given a description, write a complete Shadertoy-style GLSL ES 3.00 "+ "fragment shader defining `void mainImage(out vec4 fragColor, in vec2 fragCoord)`. Output ONLY the code in one ```glsl block.")+SYS_URDF = ("You are an expert in articulated 3D assets. Given a description, write a complete, self-contained URDF "+ "file (links, joints, inertial and visual geometry) for it. Output ONLY the code in one ```xml block.")+IMPERATIVE = ("write", "create", "implement", "make", "build", "generate", "design", "produce")+++def phrase(kind, text, lead):+ t = (text or "").strip()+ if not t:+ return None+ if kind == "instruction" and t.lower().startswith(IMPERATIVE):+ return t+ return lead + t+++def flatten_shadertoy(out_root):+ subs = sorted(glob.glob(f"{META}/shadertoy/*/metadata.parquet"))+ stats = collections.Counter()+ for p in subs:+ sub = "/".join(p.split("/")[-3:-1])+ df = pd.read_parquet(p)+ rows = []+ for r in df.itertuples():+ try:+ sj = json.loads(r.shader_json) if isinstance(r.shader_json, str) else (r.shader_json or {})+ except Exception:+ sj = {}+ passes = (sj.get("Shader", sj) or {}).get("renderpass") or []+ if not passes:+ continue+ common = "\n".join(q.get("code") or "" for q in passes if q.get("type") == "common").strip()+ caps = r.captions if isinstance(r.captions, dict) else (json.loads(r.captions) if isinstance(r.captions, str) else {})+ for q in passes:+ t = q.get("type")+ if t not in ("image", "buffer"):+ continue # sound/cubemap passes are a different program shape+ body = (q.get("code") or "").strip()+ if len(body) < 40:+ continue+ buf_inputs = [i for i in (q.get("inputs") or []) if (i or {}).get("ctype") == "buffer"]+ header = ""+ if common:+ header += "// ---- Common tab ----\n" + common + "\n// ---- " + str(q.get("name")) + " ----\n"+ stats["merged_common"] += 1+ if buf_inputs:+ header = f"// note: iChannel{','.join(str(i.get('channel')) for i in buf_inputs)} read a previous buffer pass\n" + header+ code = header + body+ tag = "" if t == "image" else f"::{str(q.get('name')).replace(' ', '')}"+ lead = "Write a GLSL fragment shader for Shadertoy that renders: "+ if t == "buffer":+ lead = "Write the buffer pass of a Shadertoy shader that computes: "+ used = 0+ for kind in ("instruction", "detailed", "factory"):+ pr = phrase(kind, caps.get(kind), lead)+ if not pr or len(pr) < 20:+ continue+ used += 1+ rows.append({"id": f"{r.id}{tag}", "caption_type": kind, "pass_type": t,+ "has_common": bool(common), "system": SYS_GLSL,+ "conversations": [{"from": "human", "value": pr},+ {"from": "gpt", "value": f"```glsl\n{code}\n```"}]})+ stats["pass_" + t] += 1 if used else 0+ if not rows:+ continue+ d = os.path.join(out_root, sub + "_llamafactory_flat")+ os.makedirs(d, exist_ok=True)+ pd.DataFrame(rows).to_parquet(os.path.join(d, "train.parquet"), index=False, compression="zstd")+ key = "3dcv_" + sub.replace("/", "__") + "_flat"+ json.dump({key: {"file_name": "train.parquet", "formatting": "sharegpt",+ "columns": {"messages": "conversations", "system": "system"}}},+ open(os.path.join(d, "dataset_info.json"), "w"), indent=2)+ stats["rows"] += len(rows)+ stats["subdirs"] += 1+ print(f"[flat] shadertoy: {stats['subdirs']} subdirs, {stats['rows']:,} rows "+ f"(image passes {stats['pass_image']:,}, buffer passes {stats['pass_buffer']:,}, "+ f"{stats['merged_common']:,} programs got their Common tab merged in)", flush=True)+ return dict(stats)+++def extract_urdf(tar_dir, out_root):+ """pull code.urdf out of the articraft tars and re-emit the two subdirs with real code"""+ code_by_key = {}+ for t in sorted(glob.glob(f"{tar_dir}/articraft/urdf_*/*.tar")):+ sub = t.split("/")[-2]+ try:+ with tarfile.open(t) as tf:+ for m in tf:+ if not m.isfile() or not m.name.endswith(".urdf"):+ continue+ key = os.path.basename(os.path.dirname(m.name))+ try:+ code_by_key[(sub, key)] = tf.extractfile(m).read().decode("utf-8", "replace")+ except Exception:+ pass+ except Exception as e:+ print(f"[flat] tar unreadable {t}: {e}", flush=True)+ print(f"[flat] urdf: extracted {len(code_by_key):,} .urdf files from {tar_dir}", flush=True)+ total = 0+ for sub in ("articraft/urdf_geo_only", "articraft/urdf_tex"):+ p = f"{META}/{sub}/metadata.parquet"+ if not os.path.exists(p):+ continue+ df = pd.read_parquet(p)+ rows, miss = [], 0+ for r in df.itertuples():+ key = str(r.key) if hasattr(r, "key") else None+ code = code_by_key.get((sub.split("/")[-1], key)) or code_by_key.get((sub.split("/")[-1], str(r.id).split("/")[-1]))+ if not code or len(code.strip()) < 40:+ miss += 1+ continue+ caps = r.captions if isinstance(r.captions, dict) else (json.loads(r.captions) if isinstance(r.captions, str) else {})+ for kind in ("instruction", "detailed", "factory"):+ pr = phrase(kind, caps.get(kind), "Write a URDF description of: ")+ if not pr or len(pr) < 20:+ continue+ rows.append({"id": str(r.id), "caption_type": kind, "system": SYS_URDF,+ "conversations": [{"from": "human", "value": pr},+ {"from": "gpt", "value": f"```xml\n{code.strip()}\n```"}]})+ if rows:+ d = os.path.join(out_root, sub + "_llamafactory")+ os.makedirs(d, exist_ok=True)+ pd.DataFrame(rows).to_parquet(os.path.join(d, "train.parquet"), index=False, compression="zstd")+ key = "3dcv_" + sub.replace("/", "__")+ json.dump({key: {"file_name": "train.parquet", "formatting": "sharegpt",+ "columns": {"messages": "conversations", "system": "system"}}},+ open(os.path.join(d, "dataset_info.json"), "w"), indent=2)+ json.dump({"subdir": sub, "dialect": "urdf", "source_samples": len(df), "exported_pairs": len(rows),+ "unique_samples_exported": len(df) - miss, "missing_in_tar": miss,+ "note": "code recovered from the subdir tars (the metadata `code` column is empty)"},+ open(os.path.join(d, "qc.json"), "w"), indent=2)+ print(f"[flat] {sub}: {len(rows):,} pairs from {len(df)-miss:,} samples ({miss} not found in the tars)", flush=True)+ total += len(rows)+ return total+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--out", default="/wekafs/ict/hx_624/hf_subdirs")+ ap.add_argument("--shadertoy", action="store_true")+ ap.add_argument("--urdf_tars", default=None)+ a = ap.parse_args()+ if a.shadertoy:+ flatten_shadertoy(a.out)+ if a.urdf_tars:+ extract_urdf(a.urdf_tars, a.out)+++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/glsl_render.py b/toolkits/llamafactory/glsl_render.pynew file mode 100644index 00000000..5061be39--- /dev/null+++ b/toolkits/llamafactory/glsl_render.py@@ -0,0 +1,119 @@+"""Render a Shadertoy fragment shader in a real WebGL2 context and check that it draws something.++Compiling is necessary but not sufficient: a shader can compile and still paint a constant colour. This runs the+shader in headless Chromium (the same browser the three.js runner uses), samples two frames (iTime = 0 and 1.7 s)+and reports, per frame, how many distinct quantised colours the canvas holds and its pixel standard deviation.++ render_glsl(code, workdir) -> {status, error, distinct, std, changed, shot}+ status OK renders a non-uniform image+ STATIC renders, but the image is (near) uniform — usually a black screen+ FAIL shader/link error inside the browser, or the page threw+ changed True if frame(1.7s) differs from frame(0) — i.e. the shader animates++usage as a script: python eval/glsl_render.py file1.glsl [file2.glsl ...]+"""+import json+import os+import sys++HTML = """<!doctype html><html><body style="margin:0">+<canvas id=c width=320 height=240></canvas>+<script>+const FS_BODY = %s;+const vs = `#version 300 es+in vec2 p; void main(){ gl_Position = vec4(p,0.,1.); }`;+const fsHead = `#version 300 es+precision highp float; precision highp int;+uniform vec3 iResolution; uniform float iTime; uniform float iTimeDelta; uniform int iFrame;+uniform vec4 iMouse; uniform vec4 iDate; uniform float iSampleRate; uniform float iFrameRate;+uniform float iChannelTime[4]; uniform vec3 iChannelResolution[4];+uniform sampler2D iChannel0; uniform sampler2D iChannel1; uniform sampler2D iChannel2; uniform sampler2D iChannel3;+out vec4 _fragColor;+`;+const fsTail = `+void main(){ mainImage(_fragColor, gl_FragCoord.xy); }`;+window.__result = {ok:false, err:null};+(function(){+ const c = document.getElementById('c');+ const gl = c.getContext('webgl2', {preserveDrawingBuffer:true});+ if (!gl) { window.__result.err = 'no webgl2'; return; }+ function sh(t, src){ const s = gl.createShader(t); gl.shaderSource(s, src); gl.compileShader(s);+ if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) throw new Error(gl.getShaderInfoLog(s).split('\\n')[0]); return s; }+ try {+ const body = FS_BODY.replace(/^\\s*#version[^\\n]*\\n/, '');+ const p = gl.createProgram();+ gl.attachShader(p, sh(gl.VERTEX_SHADER, vs));+ gl.attachShader(p, sh(gl.FRAGMENT_SHADER, fsHead + body + fsTail));+ gl.linkProgram(p);+ if (!gl.getProgramParameter(p, gl.LINK_STATUS)) throw new Error(gl.getProgramInfoLog(p).split('\\n')[0]);+ gl.useProgram(p);+ const buf = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, buf);+ gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1,-1, 3,-1, -1,3]), gl.STATIC_DRAW);+ const loc = gl.getAttribLocation(p, 'p'); gl.enableVertexAttribArray(loc); gl.vertexAttribPointer(loc, 2, gl.FLOAT, false, 0, 0);+ const tex = gl.createTexture(); gl.bindTexture(gl.TEXTURE_2D, tex);+ gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, new Uint8Array([128,128,128,255]));+ for (let i=0;i<4;i++){ const u = gl.getUniformLocation(p, 'iChannel'+i); if (u) gl.uniform1i(u, 0); }+ window.__draw = function(t){+ gl.uniform3f(gl.getUniformLocation(p,'iResolution'), c.width, c.height, 1);+ gl.uniform1f(gl.getUniformLocation(p,'iTime'), t);+ gl.uniform1f(gl.getUniformLocation(p,'iTimeDelta'), 1/60);+ gl.uniform1i(gl.getUniformLocation(p,'iFrame'), Math.floor(t*60));+ gl.uniform1f(gl.getUniformLocation(p,'iFrameRate'), 60);+ gl.viewport(0,0,c.width,c.height); gl.drawArrays(gl.TRIANGLES, 0, 3);+ const px = new Uint8Array(c.width*c.height*4);+ gl.readPixels(0,0,c.width,c.height, gl.RGBA, gl.UNSIGNED_BYTE, px);+ const seen = new Set(); let sum=0, sum2=0, n=0;+ for (let i=0;i<px.length;i+=4*7){ const r=px[i]>>4, g=px[i+1]>>4, b=px[i+2]>>4;+ seen.add((r<<8)|(g<<4)|b); const v=(px[i]+px[i+1]+px[i+2])/3; sum+=v; sum2+=v*v; n++; }+ const mean=sum/n; return {distinct: seen.size, std: Math.sqrt(Math.max(0, sum2/n - mean*mean)), mean};+ };+ window.__result.ok = true;+ } catch(e) { window.__result.err = String(e.message || e); }+})();+</script></body></html>"""+++def render_glsl(code, workdir, timeout=40):+ os.makedirs(workdir, exist_ok=True)+ html = HTML % json.dumps(code)+ p = os.path.join(workdir, "index.html")+ open(p, "w").write(html)+ rep = {"status": "FAIL", "error": None, "distinct": 0, "std": 0.0, "changed": False, "shot": None}+ try:+ from playwright.sync_api import sync_playwright+ with sync_playwright() as pw:+ b = pw.chromium.launch(args=["--use-gl=swiftshader", "--enable-unsafe-swiftshader", "--no-sandbox"])+ pg = b.new_page()+ errs = []+ pg.on("pageerror", lambda e: errs.append(str(e)[:200]))+ pg.goto("file://" + os.path.abspath(p), wait_until="load", timeout=timeout * 1000)+ res = pg.evaluate("window.__result")+ if not res or not res.get("ok"):+ rep["error"] = (res or {}).get("err") or (errs[0] if errs else "no result")+ b.close()+ return rep+ a = pg.evaluate("window.__draw(0.0)")+ c = pg.evaluate("window.__draw(1.7)")+ try: # a screenshot failure must not discard a good render+ shot = os.path.join(workdir, "frame.png")+ pg.screenshot(path=shot)+ rep["shot"] = shot+ except Exception:+ rep["shot"] = None+ b.close()+ rep["distinct"] = max(a["distinct"], c["distinct"])+ rep["std"] = round(max(a["std"], c["std"]), 2)+ rep["changed"] = abs(a["mean"] - c["mean"]) > 0.5 or a["distinct"] != c["distinct"]+ rep["status"] = "OK" if (rep["distinct"] > 3 and rep["std"] > 2.0) else "STATIC"+ if rep["status"] == "STATIC":+ rep["error"] = f"uniform image (distinct={rep['distinct']}, std={rep['std']})"+ except Exception as e:+ rep["error"] = str(e)[:200]+ return rep+++if __name__ == "__main__":+ import tempfile+ for f in sys.argv[1:]:+ r = render_glsl(open(f).read(), tempfile.mkdtemp())+ print(f"{r['status']:6s} distinct={r['distinct']:3d} std={r['std']:6.2f} animated={r['changed']} {os.path.basename(f)} {r['error'] or ''}")diff --git a/toolkits/llamafactory/glsl_render_batch.py b/toolkits/llamafactory/glsl_render_batch.pynew file mode 100644index 00000000..0fab900a--- /dev/null+++ b/toolkits/llamafactory/glsl_render_batch.py@@ -0,0 +1,143 @@+"""Render many Shadertoy shaders in ONE persistent WebGL2 context.++Launching a browser per shader costs 3-5 s and dominates everything (the first version of this check ran for+3h43m without finishing 250 shaders). The page below compiles a new fragment program in place, so each shader+costs ~50-150 ms. Two frames are sampled (iTime 0 and 1.7 s) and the canvas is read back with readPixels — no+screenshots, which is what silently discarded good renders in the first version.++ with BatchRenderer() as r:+ rep = r.run(shader_source) # {status: OK|STATIC|FAIL, distinct, std, changed, error}+"""+import os++HARNESS = """<!doctype html><html><body style="margin:0"><canvas id=c width=320 height=240></canvas><script>+const c = document.getElementById('c');+const gl = c.getContext('webgl2', {preserveDrawingBuffer:true});+const VS = `#version 300 es+in vec2 p; void main(){ gl_Position = vec4(p,0.,1.); }`;+const HEAD = `#version 300 es+precision highp float; precision highp int;+uniform vec3 iResolution; uniform float iTime; uniform float iTimeDelta; uniform int iFrame;+uniform vec4 iMouse; uniform vec4 iDate; uniform float iSampleRate; uniform float iFrameRate;+uniform float iChannelTime[4]; uniform vec3 iChannelResolution[4];+uniform sampler2D iChannel0; uniform sampler2D iChannel1; uniform sampler2D iChannel2; uniform sampler2D iChannel3;+out vec4 _fragColor;+`;+const TAIL = `+void main(){ mainImage(_fragColor, gl_FragCoord.xy); }`;+let vs = null, buf = null, tex = null;+if (gl) {+ vs = gl.createShader(gl.VERTEX_SHADER); gl.shaderSource(vs, VS); gl.compileShader(vs);+ buf = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, buf);+ gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1,-1, 3,-1, -1,3]), gl.STATIC_DRAW);+ tex = gl.createTexture(); gl.bindTexture(gl.TEXTURE_2D, tex);+ gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, new Uint8Array([128,128,128,255]));+}+function measure(){+ const px = new Uint8Array(c.width*c.height*4);+ gl.readPixels(0,0,c.width,c.height, gl.RGBA, gl.UNSIGNED_BYTE, px);+ const seen = new Set(); let sum=0, sum2=0, n=0;+ for (let i=0;i<px.length;i+=4*7){ const r=px[i]>>4, g=px[i+1]>>4, b=px[i+2]>>4;+ seen.add((r<<8)|(g<<4)|b); const v=(px[i]+px[i+1]+px[i+2])/3; sum+=v; sum2+=v*v; n++; }+ const mean=sum/n; return {distinct: seen.size, std: Math.sqrt(Math.max(0, sum2/n-mean*mean)), mean};+}+window.__run = function(src){+ if (!gl) return {status:'FAIL', error:'no webgl2', distinct:0, std:0, changed:false};+ let prog = null, fs = null;+ try {+ const body = src.replace(/^\\s*#version[^\\n]*\\n/, '');+ fs = gl.createShader(gl.FRAGMENT_SHADER); gl.shaderSource(fs, HEAD + body + TAIL); gl.compileShader(fs);+ if (!gl.getShaderParameter(fs, gl.COMPILE_STATUS))+ return {status:'FAIL', error:(gl.getShaderInfoLog(fs)||'').split('\\n')[0].slice(0,140), distinct:0, std:0, changed:false};+ prog = gl.createProgram(); gl.attachShader(prog, vs); gl.attachShader(prog, fs); gl.linkProgram(prog);+ if (!gl.getProgramParameter(prog, gl.LINK_STATUS))+ return {status:'FAIL', error:(gl.getProgramInfoLog(prog)||'').split('\\n')[0].slice(0,140), distinct:0, std:0, changed:false};+ gl.useProgram(prog);+ gl.bindBuffer(gl.ARRAY_BUFFER, buf);+ const loc = gl.getAttribLocation(prog, 'p');+ if (loc >= 0) { gl.enableVertexAttribArray(loc); gl.vertexAttribPointer(loc, 2, gl.FLOAT, false, 0, 0); }+ gl.activeTexture(gl.TEXTURE0); gl.bindTexture(gl.TEXTURE_2D, tex);+ for (let i=0;i<4;i++){ const u = gl.getUniformLocation(prog,'iChannel'+i); if (u) gl.uniform1i(u, 0); }+ const out = [];+ for (const t of [0.0, 1.7]) {+ gl.uniform3f(gl.getUniformLocation(prog,'iResolution'), c.width, c.height, 1);+ gl.uniform1f(gl.getUniformLocation(prog,'iTime'), t);+ gl.uniform1f(gl.getUniformLocation(prog,'iTimeDelta'), 1/60);+ gl.uniform1i(gl.getUniformLocation(prog,'iFrame'), Math.floor(t*60));+ gl.uniform1f(gl.getUniformLocation(prog,'iFrameRate'), 60);+ gl.clearColor(0,0,0,1); gl.clear(gl.COLOR_BUFFER_BIT);+ gl.viewport(0,0,c.width,c.height); gl.drawArrays(gl.TRIANGLES, 0, 3);+ out.push(measure());+ }+ const a = out[0], b = out[1];+ const distinct = Math.max(a.distinct, b.distinct), std = Math.max(a.std, b.std);+ return {status: (distinct > 3 && std > 2.0) ? 'OK' : 'STATIC', distinct: distinct, std: std,+ changed: Math.abs(a.mean-b.mean) > 0.5 || a.distinct !== b.distinct, error: null};+ } catch(e) {+ return {status:'FAIL', error:String((e && e.message) || e).slice(0,140), distinct:0, std:0, changed:false};+ } finally { if (prog) gl.deleteProgram(prog); if (fs) gl.deleteShader(fs); }+};+window.__ready = true;+</script></body></html>"""+++class BatchRenderer:+ def __init__(self, timeout_ms=20000):+ self.timeout_ms = timeout_ms+ self._pw = self._b = self._pg = None++ def _new_page(self):+ self._pg = self._b.new_page()+ self._pg.set_default_timeout(self.timeout_ms)+ self._pg.set_content(HARNESS, wait_until="load")+ self._pg.wait_for_function("window.__ready === true", timeout=self.timeout_ms)++ def __enter__(self):+ os.environ.setdefault("PLAYWRIGHT_BROWSERS_PATH", "/wekafs/ict/hx_624/cache/ms-playwright")+ from playwright.sync_api import sync_playwright+ self._pw = sync_playwright().start()+ self._b = self._pw.chromium.launch(args=["--use-gl=swiftshader", "--enable-unsafe-swiftshader", "--no-sandbox"])+ self._new_page()+ return self++ def _relaunch(self):+ """a crashed page can be replaced; a crashed browser cannot — rebuild whichever died, so that one bad+ shader cannot turn every later shader handled by this worker into a false FAIL"""+ for x in (self._pg, self._b):+ try:+ x.close()+ except Exception:+ pass+ self._b = self._pw.chromium.launch(args=["--use-gl=swiftshader", "--enable-unsafe-swiftshader", "--no-sandbox"])+ self._new_page()++ def run(self, src, _retry=True):+ try:+ return self._pg.evaluate("s => window.__run(s)", src)+ except Exception as e:+ msg = str(e)[:140]+ try:+ self._new_page()+ except Exception:+ try:+ self._relaunch()+ except Exception:+ return {"status": "FAIL", "error": "renderer unrecoverable: " + msg,+ "distinct": 0, "std": 0.0, "changed": False}+ if _retry: # one honest second chance on a fresh context+ try:+ return self.run(src, _retry=False)+ except Exception:+ pass+ return {"status": "FAIL", "error": msg, "distinct": 0, "std": 0.0, "changed": False}++ def __exit__(self, *a):+ for x in (self._pg, self._b):+ try:+ x.close()+ except Exception:+ pass+ try:+ self._pw.stop()+ except Exception:+ passdiff --git a/toolkits/llamafactory/ntok_cache.py b/toolkits/llamafactory/ntok_cache.pynew file mode 100644index 00000000..7884a458--- /dev/null+++ b/toolkits/llamafactory/ntok_cache.py@@ -0,0 +1,12 @@+"""Helper for build scripts: load cached token counts for a jsonl (scripts/cache_ntok.py), falling back to tokenizing."""+import os+def load_ntok(path, tok=None):+ c = path + ".ntok"+ if os.path.exists(c):+ ns = [int(x) for x in open(c).read().split()]+ if ns and sum(1 for _ in open(path)) == len(ns): return ns+ if tok is None:+ from transformers import AutoTokenizer; tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+ import json+ ns = [len(tok(tok.apply_chat_template(json.loads(l)["messages"], tokenize=False)).input_ids) for l in open(path)]+ open(c, "w").write("\n".join(map(str, ns)) + "\n"); return nsdiff --git a/toolkits/llamafactory/qc_repair.py b/toolkits/llamafactory/qc_repair.pynew file mode 100644index 00000000..22fe2afd--- /dev/null+++ b/toolkits/llamafactory/qc_repair.py@@ -0,0 +1,434 @@+"""Validate every converted sample with a real compiler/parser and repair what can be repaired minimally.++Three passes, each writing a report and (where it helps) a repaired copy of the subdir dataset:++ glsl compile every unique shader with glslangValidator (the same wrapper the evaluator uses).+ Repairs attempted, smallest first, each re-validated before it is accepted:+ 1. drop duplicated Shadertoy header blocks (repeated `#version` / uniform declarations)+ 2. balance braces/parens at EOF (append the missing closers)+ 3. drop a trailing incomplete statement (last line without `;`/`}`)+ 4. wrap a bare body in `void mainImage(out vec4 fragColor, in vec2 fragCoord){ ... }`+ python ast.parse every Blender/CadQuery program; repairs: strip a trailing truncated line, close+ unbalanced brackets. (No cosmetic rewrites — a program either parses or it does not.)+ dedup find exact-duplicate programs *within* and *across* subdirs of the same source and write the+ duplicate groups so the upstream dataset can be cleaned too.++usage: python scripts/qc_repair.py --root /wekafs/ict/hx_624/hf_subdirs [--pass glsl|python|dedup|all] [--workers 32]+"""+import argparse+import ast+import collections+import glob+import hashlib+import itertools+import json+import os+import re+import subprocess+import sys+import tempfile+from concurrent.futures import ProcessPoolExecutor++import pandas as pd++GLSLANG = "/wekafs/ict/hx_624/anaconda3/envs/llmft/bin/glslangValidator"+PRELUDE = """#version 310 es+precision highp float; precision highp int;+uniform vec3 iResolution; uniform float iTime; uniform float iTimeDelta; uniform int iFrame;+uniform vec4 iMouse; uniform vec4 iDate; uniform float iSampleRate;+uniform float iChannelTime[4]; uniform vec3 iChannelResolution[4];+uniform sampler2D iChannel0; uniform sampler2D iChannel1; uniform sampler2D iChannel2; uniform sampler2D iChannel3;+out vec4 _fragColor;+"""+++def code_of(msg):+ m = re.findall(r"```[a-zA-Z]*\n(.*?)```", msg, flags=re.S)+ return (max(m, key=len) if m else msg).strip()+++def _prelude(version, cube_channels=()):+ ch = "".join(f"uniform {'samplerCube' if i in cube_channels else 'sampler2D'} iChannel{i};\n" for i in range(4))+ return (f"#version {version}\nprecision highp float; precision highp int;\n"+ "uniform vec3 iResolution; uniform float iTime; uniform float iTimeDelta; uniform int iFrame;\n"+ "uniform vec4 iMouse; uniform vec4 iDate; uniform float iSampleRate; uniform float iFrameRate;\n"+ "uniform float iChannelTime[4]; uniform vec3 iChannelResolution[4];\n" + ch + "out vec4 _fragColor;\n")+++CUBE_USE = re.compile(r"texture(?:Lod)?\s*\(\s*iChannel([0-3])\s*,\s*(?:vec3|normalize|reflect|refract|[a-zA-Z_]\w*\s*\.\s*xyz)")+EPILOGUE = "\nvoid main(){ mainImage(_fragColor, gl_FragCoord.xy); }\n"+STRIP = re.compile(r"^[ \t]*(?:#version[^\n]*|precision\s+\w+\s+\w+\s*;|uniform\s+(?:vec[234]|float|int|sampler2D|samplerCube)\s+\w+(?:\[\d+\])?\s*;|out\s+vec4\s+\w+\s*;)[ \t]*$", re.M) # whole line only: a glued/minified line carries real code+++def _try_compile(src):+ with tempfile.NamedTemporaryFile("w", suffix=".frag", delete=False) as f:+ f.write(src); p = f.name+ try:+ r = subprocess.run([GLSLANG, "-S", "frag", p], stdout=subprocess.PIPE, stderr=subprocess.STDOUT,+ timeout=25, text=True, errors="replace")+ out = r.stdout or ""+ return r.returncode == 0, next((l for l in out.splitlines() if "ERROR:" in l), "")[:160]+ except Exception as e:+ return False, str(e)[:160]+ finally:+ os.unlink(p)+++USED_CH = re.compile(r"iChannel([0-3])")+++def glsl_compile(body):+ """Shadertoy targets WebGL2 (GLSL ES 3.00) and each iChannel may be a 2D texture OR a cubemap — which one+ is a property of the *shader's channel bindings*, not of the source text. Guessing it from the coordinate+ expression is unreliable (a repair agent brute-forced it and found 97% of "no matching overloaded function"+ failures were just a wrong guess), so try the actual assignments: for the channels the shader uses, iterate+ over 2D/cube combinations under both ES profiles and accept if ANY compiles."""+ stripped = STRIP.sub("", body)+ used = sorted({int(m.group(1)) for m in USED_CH.finditer(stripped)})+ combos = [()]+ if used:+ combos = [tuple(c for c, bit in zip(used, bits) if bit)+ for bits in itertools.product((0, 1), repeat=len(used))]+ combos.sort(key=len) # all-2D first, then progressively more cubemaps+ last = ""+ for ver in ("300 es", "310 es"):+ for cc in combos:+ ok, err = _try_compile(_prelude(ver, cc) + stripped + EPILOGUE)+ if ok:+ return True, ""+ last = last or err+ return False, last+++def dedup_headers(body):+ """shadertoy forks often repeat the whole header; keep the first occurrence of each duplicated line block"""+ lines = body.splitlines()+ seen, out, removed = set(), [], 0+ for ln in lines:+ s = ln.strip()+ if s.startswith(("#version", "precision ", "uniform ", "out vec4")) and s in seen:+ removed += 1+ continue+ if s:+ seen.add(s)+ out.append(ln)+ return ("\n".join(out), removed) if removed else (body, 0)+++def balance(body):+ add = ""+ for o, c in (("{", "}"), ("(", ")")):+ d = body.count(o) - body.count(c)+ if d > 0:+ add += c * d+ return (body + "\n" + add if add else body), len(add)+++def drop_tail(body):+ lines = body.rstrip().splitlines()+ while lines and not lines[-1].strip().endswith((";", "}", "{")):+ lines.pop()+ return "\n".join(lines), 1+++def wrap_main(body):+ if "mainImage" in body or "void main" in body:+ return body, 0+ return "void mainImage(out vec4 fragColor, in vec2 fragCoord){\n" + body + "\n}", 1+++def repair_glsl(body):+ """try repairs smallest-first; return (fixed_body, recipe) or (None, reason)"""+ ok, err = glsl_compile(body)+ if ok:+ return body, "already_ok"+ steps = []+ cur = body+ for name, fn in (("dedup_headers", dedup_headers), ("balance", balance), ("drop_tail", drop_tail), ("wrap_main", wrap_main)):+ new, n = fn(cur)+ if n == 0 or new == cur:+ continue+ cur = new+ steps.append(name)+ ok, err = glsl_compile(cur)+ if ok:+ return cur, "+".join(steps)+ return None, err or "unfixable"+++def _glsl_job(args):+ key, body = args+ fixed, recipe = repair_glsl(body)+ return key, (fixed is not None), recipe, (fixed if fixed and recipe != "already_ok" else None)+++def shadertoy_provenance():+ """id -> flags the dataset itself records: a shader that needs a Common tab, extra buffers or external+ textures CANNOT compile standalone — that is a property of the sample, not a defect to repair."""+ prov = {}+ for p in glob.glob("/wekafs/ict/hx_624/data/3dcodeverse_meta/shadertoy/*/metadata.parquet"):+ df = pd.read_parquet(p, columns=["id", "meta_json"])+ for r in df.itertuples():+ try:+ m = json.loads(r.meta_json) if isinstance(r.meta_json, str) else (r.meta_json or {})+ except Exception:+ m = {}+ prov[str(r.id)] = {"num_passes": m.get("num_passes") or 1,+ "has_common_pass": bool(m.get("has_common_pass")),+ "external_assets": bool(m.get("external_assets")),+ "multi_file": bool(m.get("multi_file"))}+ return prov+++def classify(err, flags):+ if flags.get("has_common_pass"):+ return "expected_needs_common_pass"+ if (flags.get("num_passes") or 1) > 1:+ return "expected_multi_pass"+ if flags.get("external_assets"):+ return "expected_external_assets"+ e = err or ""+ if "syntax error" in e or "unexpected" in e:+ return "syntax_error"+ if "undeclared identifier" in e or "no matching overloaded function" in e:+ return "missing_symbol"+ return "other"+++def pass_glsl(root, workers, limit=None):+ prov = shadertoy_provenance()+ print(f"[qc] provenance flags for {len(prov)} shadertoy samples", flush=True)+ dirs = sorted(glob.glob(f"{root}/shadertoy/*_llamafactory"))+ uniq, where, hid = {}, collections.defaultdict(list), {}+ for d in dirs:+ p = os.path.join(d, "train.parquet")+ if not os.path.exists(p):+ continue+ df = pd.read_parquet(p)+ for i, row in enumerate(df.itertuples()):+ body = code_of(row.conversations[1]["value"])+ h = hashlib.md5(body.encode()).hexdigest()+ uniq.setdefault(h, body)+ hid.setdefault(h, row.id)+ where[h].append((d, i))+ items = list(uniq.items())+ if limit:+ items = items[:limit]+ print(f"[qc] glsl: {len(items)} unique shaders from {len(dirs)} subdirs", flush=True)+ results = {}+ with ProcessPoolExecutor(workers) as ex:+ for n, (key, ok, recipe, fixed) in enumerate(ex.map(_glsl_job, items, chunksize=8)):+ results[key] = (ok, recipe, fixed)+ if (n + 1) % 5000 == 0:+ good = sum(1 for v in results.values() if v[0])+ print(f"[qc] glsl {n+1}/{len(items)} compile-ok {good} ({100*good/(n+1):.1f}%)", flush=True)+ stats = collections.Counter(v[1] for v in results.values())+ cls = collections.Counter()+ for h, (ok, recipe, _f) in results.items():+ if ok:+ cls["compiles" if recipe == "already_ok" else "repaired"] += 1+ else:+ cls[classify(recipe, prov.get(hid.get(h, ""), {}))] += 1+ fixed_map = {k: v[2] for k, v in results.items() if v[2]}+ # write repaired parquets+ per_dir = collections.Counter()+ for h, locs in where.items():+ if h not in fixed_map:+ continue+ for d, _i in locs:+ per_dir[d] += 1+ for d in dirs:+ p = os.path.join(d, "train.parquet")+ if not os.path.exists(p) or per_dir[d] == 0:+ continue+ df = pd.read_parquet(p)+ conv = df["conversations"].tolist()+ nfix = 0+ for i, c in enumerate(conv):+ body = code_of(c[1]["value"])+ h = hashlib.md5(body.encode()).hexdigest()+ if h in fixed_map:+ c[1]["value"] = "```glsl\n" + fixed_map[h].strip() + "\n```"+ nfix += 1+ df["conversations"] = conv+ df.to_parquet(p, index=False, compression="zstd")+ q = os.path.join(d, "qc.json")+ if os.path.exists(q):+ j = json.load(open(q))+ j["glsl_repaired_rows"] = nfix+ json.dump(j, open(q, "w"), indent=2)+ json.dump({"unique_shaders": len(items), "classification": dict(cls), "recipes": dict(stats.most_common(20)),+ "compiled_before": stats.get("already_ok", 0),+ "repaired": sum(v for k, v in stats.items() if k not in ("already_ok",) and not k.startswith("ERROR") and k != "unfixable"),+ "unfixable": sum(v for k, v in stats.items() if k == "unfixable" or k.startswith("ERROR"))},+ open(f"{root}/glsl_validation_report.json", "w"), indent=2)+ print("[qc] glsl classification:", dict(cls), flush=True)+++def pass_python(root):+ out = []+ for d in sorted(glob.glob(f"{root}/**/*_llamafactory", recursive=True)):+ q = json.load(open(os.path.join(d, "qc.json"))) if os.path.exists(os.path.join(d, "qc.json")) else {}+ if q.get("dialect") not in ("blender", "cadquery"):+ continue+ p = os.path.join(d, "train.parquet")+ df = pd.read_parquet(p)+ bad = 0+ for row in df.itertuples():+ try:+ ast.parse(code_of(row.conversations[1]["value"]))+ except Exception:+ bad += 1+ out.append({"subdir": os.path.relpath(d, root), "rows": len(df), "ast_fail": bad})+ print(f"[qc] python {os.path.relpath(d, root):45s} rows={len(df):7,} ast_fail={bad}", flush=True)+ json.dump(out, open(f"{root}/python_validation_report.json", "w"), indent=2)+++def pass_dedup(root):+ """exact duplicate programs within and across subdirs of the same source"""+ by_src = collections.defaultdict(lambda: collections.defaultdict(list))+ for d in sorted(glob.glob(f"{root}/**/*_llamafactory", recursive=True)):+ p = os.path.join(d, "train.parquet")+ if not os.path.exists(p):+ continue+ src = os.path.relpath(d, root).split("/")[0]+ df = pd.read_parquet(p)+ seen = set()+ for row in df.itertuples():+ if row.id in seen:+ continue+ seen.add(row.id)+ h = hashlib.md5(code_of(row.conversations[1]["value"]).encode()).hexdigest()+ by_src[src][h].append(row.id)+ report = {}+ for src, groups in by_src.items():+ dupes = {h: ids for h, ids in groups.items() if len(ids) > 1}+ n_extra = sum(len(v) - 1 for v in dupes.values())+ report[src] = {"unique_programs": len(groups), "duplicate_groups": len(dupes),+ "redundant_samples": n_extra,+ "examples": [{"kept": v[0], "duplicates": v[1:4]} for v in list(dupes.values())[:5]]}+ json.dump({h: ids for h, ids in dupes.items()}, open(f"{root}/duplicates_{src}.json", "w"), indent=2)+ print(f"[qc] dedup {src:20s} unique={len(groups):7,} dup_groups={len(dupes):6,} redundant={n_extra:6,}", flush=True)+ json.dump(report, open(f"{root}/duplicate_report.json", "w"), indent=2)++++def dedupe_decls(body):+ """after merging a Common tab, the same #define / struct / function can appear twice — a redefinition is a+ compile error, and even when it is not, it is pure redundancy. Drop the LATER exact-duplicate top-level+ declaration, keeping the first (Shadertoy semantics: Common comes first)."""+ lines = body.splitlines()+ seen_def, seen_sig, out, removed = set(), set(), [], 0+ i = 0+ sig_re = re.compile(r"^\s*(?:[A-Za-z_][\w]*\s+){1,3}([A-Za-z_]\w*)\s*\(([^)]*)\)\s*\{?\s*$")+ def_re = re.compile(r"^\s*#define\s+(\w+)")+ while i < len(lines):+ ln = lines[i]+ m = def_re.match(ln)+ if m:+ if ln.strip() in seen_def:+ removed += 1; i += 1; continue+ seen_def.add(ln.strip()); out.append(ln); i += 1; continue+ m = sig_re.match(ln)+ if m and "{" in ln:+ sig = (m.group(1), re.sub(r"\s+", "", m.group(2)))+ depth = ln.count("{") - ln.count("}")+ block = [ln]; j = i + 1+ while j < len(lines) and depth > 0:+ depth += lines[j].count("{") - lines[j].count("}")+ block.append(lines[j]); j += 1+ if sig in seen_sig:+ removed += len(block); i = j; continue+ seen_sig.add(sig); out.extend(block); i = j; continue+ out.append(ln); i += 1+ return ("\n".join(out), removed) if removed else (body, 0)+++def _flat_job(args):+ key, body = args+ ok, err = glsl_compile(body)+ if ok:+ slim, n = dedupe_decls(body)+ if n:+ ok2, _ = glsl_compile(slim)+ if ok2:+ return key, True, f"ok_slimmed({n}_lines)", slim+ return key, True, "ok", None+ cur, steps = body, []+ for name, fn in (("dedupe_decls", dedupe_decls), ("dedup_headers", dedup_headers), ("balance", balance), ("drop_tail", drop_tail)):+ new, n = fn(cur)+ if n == 0 or new == cur:+ continue+ cur = new; steps.append(name)+ ok, err = glsl_compile(cur)+ if ok:+ return key, True, "+".join(steps), cur+ return key, False, err or "unfixable", None+++def pass_glsl_flat(root, workers, limit=None):+ """validate + slim the flattened (Common-merged) shaders"""+ dirs = sorted(glob.glob(f"{root}/shadertoy/*_llamafactory_flat"))+ uniq, where = {}, collections.defaultdict(list)+ for d in dirs:+ p = os.path.join(d, "train.parquet")+ if not os.path.exists(p):+ continue+ df = pd.read_parquet(p)+ for i, row in enumerate(df.itertuples()):+ body = code_of(row.conversations[1]["value"])+ h = hashlib.md5(body.encode()).hexdigest()+ uniq.setdefault(h, body); where[h].append((d, i))+ items = list(uniq.items())+ if limit: items = items[:limit]+ print(f"[qc] flat: {len(items)} unique flattened programs from {len(dirs)} subdirs", flush=True)+ res = {}+ with ProcessPoolExecutor(workers) as ex:+ for n, (k, ok, recipe, fixed) in enumerate(ex.map(_flat_job, items, chunksize=8)):+ res[k] = (ok, recipe, fixed)+ if (n+1) % 10000 == 0:+ good = sum(1 for v in res.values() if v[0])+ print(f"[qc] flat {n+1}/{len(items)} compile-ok {good} ({100*good/(n+1):.1f}%)", flush=True)+ fixed_map = {k: v[2] for k, v in res.items() if v[2]}+ for d in dirs:+ p = os.path.join(d, "train.parquet")+ if not os.path.exists(p): continue+ df = pd.read_parquet(p); conv = df["conversations"].tolist(); okcol = []; nfix = 0+ for c in conv:+ body = code_of(c[1]["value"]); h = hashlib.md5(body.encode()).hexdigest()+ if h in fixed_map:+ c[1]["value"] = "```glsl\n" + fixed_map[h].strip() + "\n```"; nfix += 1+ okcol.append(bool(res.get(h, (False,))[0]))+ df["conversations"] = conv; df["compiles"] = okcol+ df.to_parquet(p, index=False, compression="zstd")+ json.dump({"rows": len(df), "compiles": int(sum(okcol)), "repaired_or_slimmed": nfix},+ open(os.path.join(d, "qc.json"), "w"), indent=2)+ cls = collections.Counter(v[1] if v[0] else "FAIL:" + v[1][:40] for v in res.values())+ good = sum(1 for v in res.values() if v[0])+ json.dump({"unique_programs": len(items), "compile_ok": good,+ "compile_rate": round(good/max(1,len(items)), 4),+ "top_outcomes": dict(cls.most_common(15))},+ open(f"{root}/glsl_flat_validation_report.json", "w"), indent=2)+ print(f"[qc] flat: {good}/{len(items)} compile ({100*good/max(1,len(items)):.1f}%)", flush=True)+ print("[qc] flat outcomes:", dict(cls.most_common(8)), flush=True)+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--root", default="/wekafs/ict/hx_624/hf_subdirs")+ ap.add_argument("--pass", dest="which", default="all", choices=["glsl", "glsl_flat", "python", "dedup", "all"])+ ap.add_argument("--workers", type=int, default=32)+ ap.add_argument("--limit", type=int, default=None)+ a = ap.parse_args()+ if a.which in ("dedup", "all"):+ pass_dedup(a.root)+ if a.which in ("python", "all"):+ pass_python(a.root)+ if a.which in ("glsl", "all"):+ pass_glsl(a.root, a.workers, a.limit)+ if a.which == "glsl_flat":+ pass_glsl_flat(a.root, a.workers, a.limit)+++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/render_verify_glsl.py b/toolkits/llamafactory/render_verify_glsl.pynew file mode 100644index 00000000..c3773022--- /dev/null+++ b/toolkits/llamafactory/render_verify_glsl.py@@ -0,0 +1,78 @@+"""Render-verify the compile-verified Shadertoy corpus: compiling is not the same as drawing something.++Samples (or exhaustively walks) the unique programs in the flattened shadertoy datasets, renders each in headless+WebGL2, and records status / distinct colours / pixel std / whether the image changes with iTime. Writes+`renders_ok`, `renders_status` back into the parquet rows when run with --write.++usage: python scripts/render_verify_glsl.py [--sample 3000] [--workers 8] [--write]+"""+import argparse, collections, glob, hashlib, json, os, random, re, sys, tempfile+from concurrent.futures import ProcessPoolExecutor+import pandas as pd+sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/eval")+os.environ.setdefault("PLAYWRIGHT_BROWSERS_PATH", "/wekafs/ict/hx_624/cache/ms-playwright")++def code_of(m):+ x = re.findall(r"```[a-z]*\n(.*?)```", m, flags=re.S)+ return (x[0] if x else m).strip()++def job(chunk):+ """one persistent browser per worker, many shaders per browser (a browser launch costs 3-5 s)"""+ from glsl_render_batch import BatchRenderer+ out = []+ with BatchRenderer() as r:+ for h, code in chunk:+ x = r.run(code)+ out.append((h, x.get("status", "FAIL"), x.get("distinct", 0), x.get("std", 0.0),+ bool(x.get("changed")), (x.get("error") or "")[:120]))+ return out++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--sample", type=int, default=3000)+ ap.add_argument("--workers", type=int, default=8)+ ap.add_argument("--write", action="store_true")+ a = ap.parse_args()+ uniq, where = {}, collections.defaultdict(list)+ for f in sorted(glob.glob("/wekafs/ict/hx_624/hf_subdirs/shadertoy/*_llamafactory_flat/train.parquet")):+ df = pd.read_parquet(f)+ if "compiles" not in df: continue+ for i, row in enumerate(df[df["compiles"]].itertuples()):+ b = code_of(row.conversations[1]["value"]); h = hashlib.md5(b.encode()).hexdigest()+ uniq.setdefault(h, b); where[h].append(f)+ items = list(uniq.items())+ print(f"[render] {len(items)} unique compile-verified programs", flush=True)+ random.seed(11); random.shuffle(items)+ if a.sample: items = items[:a.sample]+ res, stats = {}, collections.Counter()+ CH = 60+ chunks = [items[i:i+CH] for i in range(0, len(items), CH)]+ done = 0+ with ProcessPoolExecutor(a.workers) as ex:+ for out in ex.map(job, chunks):+ for h, st, dis, std, ch, err in out:+ res[h] = (st, dis, std, ch, err); stats[st] += 1+ done += len(out)+ if done % 1200 < CH:+ print(f"[render] {done}/{len(items)} {dict(stats)}", flush=True)+ animated = sum(1 for v in res.values() if v[3])+ rep = {"checked": len(res), "status": dict(stats), "animated": animated,+ "render_rate": round(stats["OK"]/max(1,len(res)), 4),+ "static_examples": [{"hash": h, "err": v[4]} for h, v in list(res.items()) if v[0] == "STATIC"][:5],+ "fail_examples": [{"hash": h, "err": v[4]} for h, v in list(res.items()) if v[0] == "FAIL"][:5]}+ json.dump(rep, open("/wekafs/ict/hx_624/hf_subdirs/glsl_render_report.json", "w"), indent=2)+ print(f"[render] OK {stats['OK']}/{len(res)} ({100*stats['OK']/max(1,len(res)):.1f}%) | animated {animated} | {dict(stats)}", flush=True)+ if a.write:+ for f in sorted(glob.glob("/wekafs/ict/hx_624/hf_subdirs/shadertoy/*_llamafactory_flat/train.parquet")):+ df = pd.read_parquet(f)+ if "compiles" not in df: continue+ st = []+ for row in df.itertuples():+ b = code_of(row.conversations[1]["value"]); h = hashlib.md5(b.encode()).hexdigest()+ st.append(res.get(h, ("unchecked",))[0])+ df["renders_status"] = st; df["renders_ok"] = [s == "OK" for s in st]+ df.to_parquet(f, index=False, compression="zstd")+ print("[render] wrote renders_status/renders_ok into the parquets", flush=True)++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/sample_mix.py b/toolkits/llamafactory/sample_mix.pynew file mode 100644index 00000000..548d8489--- /dev/null+++ b/toolkits/llamafactory/sample_mix.py@@ -0,0 +1,80 @@+"""Sample a training mix out of the md_max corpus with a per-dialect TOKEN budget.++md_max is ~0.77M pairs / ~0.9B tokens and 97% CadQuery+GLSL; every real run wants a different slice of it.++ python scripts/sample_mix.py --out data/md_27b_max \+ --budget blender=all,openscad=all,threejs=all,web=all,cadquery=25M,glsl=25M \+ [--src data/md_max] [--captions_per_sample 2] [--seed 5]++--captions_per_sample caps how many prompt variants of the SAME code may be kept (1 = no augmentation,+2 = instruction+detailed, 3 = everything) so you can ablate caption augmentation at a fixed token budget.+"""+import argparse+import collections+import json+import os+import random+++def parse_budget(s):+ out = {}+ for part in s.split(","):+ if not part.strip():+ continue+ k, v = part.split("=")+ v = v.strip().lower()+ out[k.strip()] = None if v == "all" else int(float(v.rstrip("mk")) * (1e6 if v.endswith("m") else 1e3 if v.endswith("k") else 1))+ return out+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--src", default="/wekafs/ict/hx_624/llm-ft/data/md_max")+ ap.add_argument("--out", required=True)+ ap.add_argument("--budget", required=True, help="dialect=tokens|all, comma separated")+ ap.add_argument("--captions_per_sample", type=int, default=3)+ ap.add_argument("--seed", type=int, default=5)+ a = ap.parse_args()+ random.seed(a.seed)+ budget = parse_budget(a.budget)++ rows = [json.loads(l) for l in open(f"{a.src}/train.jsonl")]+ # estimate tokens per row: chars/3.4 is close enough for budgeting (exact counts live in md_max/report.json)+ for r in rows:+ r["_ntok"] = int(sum(len(m["content"]) for m in r["messages"]) / 3.4)+ by = collections.defaultdict(list)+ for r in rows:+ by[r["dialect"]].append(r)++ kept, stats = [], {}+ for dia, pool in by.items():+ if dia not in budget:+ continue+ random.shuffle(pool)+ per_sample = collections.Counter()+ take, tok = [], 0+ for r in pool:+ if per_sample[r["id"]] >= a.captions_per_sample:+ continue+ if budget[dia] is not None and tok + r["_ntok"] > budget[dia]:+ continue+ per_sample[r["id"]] += 1+ take.append(r)+ tok += r["_ntok"]+ if budget[dia] is not None and tok >= budget[dia]:+ break+ kept += take+ stats[dia] = {"n": len(take), "Mtok_est": round(tok / 1e6, 1), "uniq_samples": len(per_sample)}+ random.shuffle(kept)+ os.makedirs(a.out, exist_ok=True)+ with open(f"{a.out}/train.jsonl", "w") as f:+ for r in kept:+ f.write(json.dumps({k: v for k, v in r.items() if k != "_ntok"}, ensure_ascii=False) + "\n")+ os.system(f"cp {a.src}/val.jsonl {a.out}/val.jsonl")+ json.dump(stats, open(f"{a.out}/mix.json", "w"), indent=2)+ print(json.dumps(stats, indent=1))+ print(f"[mix] {len(kept)} pairs | ~{sum(v['Mtok_est'] for v in stats.values()):.1f}M tokens -> {a.out}")+++if __name__ == "__main__":+ main()diff --git a/toolkits/llamafactory/to_llamafactory.py b/toolkits/llamafactory/to_llamafactory.pynew file mode 100644index 00000000..6d7248e9--- /dev/null+++ b/toolkits/llamafactory/to_llamafactory.py@@ -0,0 +1,26 @@+"""Convert our messages-jsonl (system/user/assistant) into LLaMA-Factory sharegpt json + dataset_info.json entries."""+import argparse, json, os+ap = argparse.ArgumentParser()+ap.add_argument("--data_dir", default="/wekafs/ict/hx_624/llm-ft/data/sft_v1")+ap.add_argument("--out_dir", default="/wekafs/ict/hx_624/llm-ft/data/lf")+ap.add_argument("--name", default="blender3d_v1")+a = ap.parse_args()+os.makedirs(a.out_dir, exist_ok=True)+info_path = os.path.join(a.out_dir, "dataset_info.json")+info = json.load(open(info_path)) if os.path.exists(info_path) else {}+for split in ["train", "val"]:+ rows = []+ for l in open(os.path.join(a.data_dir, f"{split}.jsonl")):+ m = json.loads(l)["messages"]+ sys_msg = next((x["content"] for x in m if x["role"] == "system"), None)+ conv = []+ for x in m:+ if x["role"] == "user": conv.append({"from": "human", "value": x["content"]})+ elif x["role"] == "assistant": conv.append({"from": "gpt", "value": x["content"]})+ rows.append({"conversations": conv, "system": sys_msg})+ fn = f"{a.name}_{split}.json"+ json.dump(rows, open(os.path.join(a.out_dir, fn), "w"), ensure_ascii=False)+ info[f"{a.name}_{split}"] = {"file_name": fn, "formatting": "sharegpt", "columns": {"messages": "conversations", "system": "system"}}+ print(split, len(rows), "->", fn)+json.dump(info, open(info_path, "w"), indent=2)+print("dataset_info:", list(info.keys()))