Skip to content

Official Codebase of 3DCodeVerse

Python

finetune: LLM finetuning recipes + execution-based eval for 3DCodeVerse (Qwen3.5-9B LoRA/full SFT, execution- & geometry-feedback DPO, multi-dialect scale-up; unpatched LLaMA-Factory configs, vLLM eval stack, report)

gaoypeng0ee8a4b9497d
diff --git a/README.md b/README.md
index 588b310b..d6813a99 100644
--- a/README.md
+++ b/README.md
@@ -8,5 +8,11 @@ The 3dcodeverse project. Top-level layout:
plan → generate → gate → render → judge → refine loop, and emits
data-flywheel records. See `harness/README.md`.
+- **`finetune/`** — LLM finetuning recipes (LoRA / full-parameter SFT, execution-
+ and geometry-feedback DPO via unpatched LLaMA-Factory) and the execution-based
+ evaluation stack (3DCodeBench + per-dialect executors) used to train open
+ models on 3DCodeVerse data; results and conclusions in `finetune/docs/REPORT.md`.
+ See `finetune/README.md`.
+
Other large components (datasets, papers, web) live in their own top-level
folders.
diff --git a/finetune/CLAUDE.md b/finetune/CLAUDE.md
new file mode 100644
index 00000000..49c099e3
--- /dev/null
+++ b/finetune/CLAUDE.md
@@ -0,0 +1,6 @@
+# finetune/ (component notes for agents)
+
+LLM finetuning + execution-based eval for 3DCodeVerse. Read `README.md` here and `docs/REPORT.md` (conclusions + pitfalls) first.
+- Training is **unpatched LLaMA-Factory** driven by `configs/lf/*.yaml`; do not fork the framework — add a YAML + a dataset entry.
+- Evaluate by *executing* the code (`eval/`): 3DCodeBench (Blender → Chamfer/F vs GT), per-dialect held-out sets, pass@N. Greedy decoding collapses on boilerplate-heavy dialects (OpenSCAD/GLSL) — use sampling there.
+- Absolute paths point at the original machine; adapt `env.sh` + YAML path keys + tool paths in `eval/*runner*.py` before running.
diff --git a/finetune/README.md b/finetune/README.md
new file mode 100644
index 00000000..00182c50
--- /dev/null
+++ b/finetune/README.md
@@ -0,0 +1,36 @@
+# finetune/ — LLM finetuning recipes + execution-based evaluation for 3DCodeVerse
+
+Everything needed to reproduce the Qwen3.5-9B finetuning study on 3DCodeVerse
+(text → Blender-Python / CadQuery / OpenSCAD / GLSL / three.js code), and the
+conclusions in `docs/REPORT.md`. The harness (`../harness`) generates data with
+API models; this folder trains and evaluates open models on that data.
+
+**Nothing in LLaMA-Factory is patched** — all training is plain
+`llamafactory-cli` YAML (`configs/lf/`) + ShareGPT JSON registered in
+`data_specs/dataset_info.json`. Pinned environments are in `env/`.
+
+## Layout
+
+| path | what |
+|---|---|
+| `scripts/` | data builders (`prep_data.py`, `build_multidialect*.py`, `build_md_big.py`, `build_md_xl.py`, `build_md_bal.py`, `build_boot_data.py`, `cache_ntok.py`), thin launchers around LLaMA-Factory (`lf_train.sh`, `lf_export.sh`, `make_lora_cfg.py`), experiment pipelines train→merge→eval (`run_exp.sh`, `run_md_exp.sh`, `run_full_ft*.sh`, `*_ckpt_sweep.sh`), execution-feedback / geometry-feedback DPO rounds (`run_dpo_exec.sh`, `dpo_round2.sh`, `md_dpo_round.sh`, `md_big_dpo_round.sh`, `md_xl_dpo_round.sh`, `geo_dpo_build.sh`), bootstrapping (`self_train_round*.sh`), report tooling (`build_report_html.py`, `contact_sheet_3dcodebench.py`) |
+| `configs/lf/` | one LLaMA-Factory YAML per experiment: LoRA SFT (`lora_*.yaml`), full-parameter ZeRO-3 SFT (`qwen35_9b_full_sft*.yaml`, `full_md_bal.yaml`, `ds_z3_offload.json`), DPO (`dpo_*.yaml`), LoRA merge/export (`export_*.yaml`) |
+| `eval/` | vLLM generation (`generate_vllm.py`), 3DCodeBench runner (`run_bench.py`, `blender_runner.py`), geometry metrics Chamfer / F-score vs GT (`metrics.py`), pass@N / best-of-N (`best_of_n.sh`, `pass_at_n.py`), per-dialect executors (`dialect_runners.py`: CadQuery, OpenSCAD, glslangValidator; `threejs_runner.py`: Playwright headless Chromium), held-out dialect evals (`dialect_eval.py`, `blender_dialect_eval.py`, `threejs_eval.py`, `run_dialect_eval.sh`), renders for qualitative checks (`render_glb.py`) |
+| `data_specs/dataset_info.json` | LLaMA-Factory dataset registry for every SFT/DPO set used (ShareGPT format; the JSON files themselves are rebuilt by `scripts/`) |
+| `env/requirements-{llmft,lf,vllm}.txt` | pip freezes of the three conda envs (data/eval, LLaMA-Factory training, vLLM inference) |
+| `env.sh` | shell env (caches, secrets file) used by all scripts |
+| `docs/REPORT.md` | full experiment report (setup, data audit, ablations, DPO, multi-dialect, scale-up, training-amount curves, qualitative checks); `docs/PROJECT_README.md` is the original working README; `docs/assets/` renders |
+
+## Reproducing (short version)
+
+1. Envs (see `docs/REPORT.md` §1 for the pitfalls): `lf` = LLaMA-Factory (editable, upstream, unpatched) + torch 2.8.0+cu128 + flash-attn 2.8.3 + flash-linear-attention 0.5.2 + **triton 3.7.1** (Qwen3.5 GDN kernels on Hopper) + deepspeed; `vllm` = vLLM 0.27; `llmft` = data/eval (trimesh, cadquery, playwright, …). Blender 5.0.1 headless and an OpenSCAD AppImage are called by path.
+2. Data: only `metadata.parquet` of each 3DCodeVerse subset is needed. `scripts/build_multidialect.py` → `data/multidialect/<dialect>/{train,test}.jsonl`; `scripts/build_md_*.py` build the mixes (token-length filter ≤8192, cached by `scripts/cache_ntok.py`); `scripts/to_llamafactory.py` writes ShareGPT JSON + `dataset_info.json`.
+3. Train: `GPUS=0,1,2,3 scripts/lf_train.sh configs/lf/lora_md_xl.yaml gradient_accumulation_steps=2` (LoRA r64, cutoff 8192, packing, FA2) → `scripts/lf_export.sh configs/lf/export_md_xl.yaml` to merge.
+4. Evaluate: `GPUS=0 eval/run_eval_vllm.sh <merged> eval/out/<name> --no_think --max_new_tokens 6144` (3DCodeBench: vLLM gen → Blender exec → Chamfer/F vs GT) and `GPUS=0 eval/run_dialect_eval.sh <merged> <name>` (held-out CadQuery/OpenSCAD/GLSL/Blender). Decode OpenSCAD/GLSL with sampling (T=0.7), not greedy.
+5. Execution-feedback DPO on top of any SFT model: `scripts/md_xl_dpo_round.sh` (sample → run in each dialect's executor → (OK, FAIL) pairs → `stage: dpo`).
+
+Paths are absolute for the machine the study ran on (`/wekafs/ict/hx_624/...`); adapt `env.sh`, the `*_dir`/`model_name_or_path` keys in `configs/lf/*.yaml`, and the tool paths at the top of `eval/*runner*.py` / `scripts/*.sh` before running elsewhere.
+
+## Headline results (3DCodeBench, 212 tasks, execution rate / [email protected]; details in `docs/REPORT.md`)
+
+Qwen3.5-9B zero-shot 0% → LoRA on 5k Blender samples 75–82% → + execution-feedback DPO 93.9–96.2% → 4-dialect mixes: 25.6k 67%, 102k 84.9%, 270k (all 3DCodeVerse) 90.1%; 4 H100s: LoRA ≈17k tok/s, 270k samples / 293M tokens in 4.8 h.
diff --git a/finetune/configs/ds_zero2.json b/finetune/configs/ds_zero2.json
new file mode 100644
index 00000000..30356c5b
--- /dev/null
+++ b/finetune/configs/ds_zero2.json
@@ -0,0 +1,10 @@
+{
+ "bf16": {"enabled": true},
+ "zero_optimization": {"stage": 2, "overlap_comm": true, "contiguous_gradients": true, "reduce_bucket_size": 5e8, "allgather_bucket_size": 5e8},
+ "gradient_accumulation_steps": "auto",
+ "gradient_clipping": 1.0,
+ "train_batch_size": "auto",
+ "train_micro_batch_size_per_gpu": "auto",
+ "steps_per_print": 1e9,
+ "wall_clock_breakdown": false
+}
diff --git a/finetune/configs/ds_zero3.json b/finetune/configs/ds_zero3.json
new file mode 100644
index 00000000..97ca1c29
--- /dev/null
+++ b/finetune/configs/ds_zero3.json
@@ -0,0 +1,20 @@
+{
+ "bf16": {"enabled": true},
+ "zero_optimization": {
+ "stage": 3,
+ "overlap_comm": true,
+ "contiguous_gradients": true,
+ "reduce_bucket_size": "auto",
+ "stage3_prefetch_bucket_size": "auto",
+ "stage3_param_persistence_threshold": "auto",
+ "stage3_max_live_parameters": 1e9,
+ "stage3_max_reuse_distance": 1e9,
+ "stage3_gather_16bit_weights_on_model_save": true
+ },
+ "gradient_accumulation_steps": "auto",
+ "gradient_clipping": 1.0,
+ "train_batch_size": "auto",
+ "train_micro_batch_size_per_gpu": "auto",
+ "steps_per_print": 1e9,
+ "wall_clock_breakdown": false
+}
diff --git a/finetune/configs/lf/dpo_exec_v1.yaml b/finetune/configs/lf/dpo_exec_v1.yaml
new file mode 100644
index 00000000..ade129d3
--- /dev/null
+++ b/finetune/configs/lf/dpo_exec_v1.yaml
@@ -0,0 +1,42 @@
+### DPO with execution-feedback preference pairs, on top of LoRA-v1 (merged)
+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged
+trust_remote_code: true
+flash_attn: fa2
+stage: dpo
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+pref_beta: 0.1
+pref_loss: sigmoid
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: dpo_exec_pairs
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 2
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1
+logging_steps: 5
+save_steps: 1000
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 5.0e-6
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.1
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+val_size: 0.02
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
diff --git a/finetune/configs/lf/dpo_exec_v2.yaml b/finetune/configs/lf/dpo_exec_v2.yaml
new file mode 100644
index 00000000..6bd40832
--- /dev/null
+++ b/finetune/configs/lf/dpo_exec_v2.yaml
@@ -0,0 +1,42 @@
+### DPO with execution-feedback preference pairs, on top of LoRA-v1 (merged)
+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1/merged_r2
+trust_remote_code: true
+flash_attn: fa2
+stage: dpo
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+pref_beta: 0.1
+pref_loss: sigmoid
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: dpo_exec_pairs_r2
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 2
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2
+logging_steps: 5
+save_steps: 1000
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 5.0e-6
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.1
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+val_size: 0.02
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
diff --git a/finetune/configs/lf/dpo_md_big.yaml b/finetune/configs/lf/dpo_md_big.yaml
new file mode 100644
index 00000000..e1f83b0e
--- /dev/null
+++ b/finetune/configs/lf/dpo_md_big.yaml
@@ -0,0 +1,42 @@
+### DPO with execution-feedback preference pairs, on top of LoRA-v1 (merged)
+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/merged
+trust_remote_code: true
+flash_attn: fa2
+stage: dpo
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+pref_beta: 0.1
+pref_loss: sigmoid
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: dpo_md_big_pairs
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 2
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big
+logging_steps: 5
+save_steps: 1000
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 5.0e-6
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.1
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+val_size: 0.02
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
diff --git a/finetune/configs/lf/dpo_md_mixed.yaml b/finetune/configs/lf/dpo_md_mixed.yaml
new file mode 100644
index 00000000..d05a50c1
--- /dev/null
+++ b/finetune/configs/lf/dpo_md_mixed.yaml
@@ -0,0 +1,42 @@
+### DPO with execution-feedback preference pairs, on top of LoRA-v1 (merged)
+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed/merged
+trust_remote_code: true
+flash_attn: fa2
+stage: dpo
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+pref_beta: 0.1
+pref_loss: sigmoid
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: dpo_md_pairs
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 2
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed
+logging_steps: 5
+save_steps: 1000
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 5.0e-6
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.1
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+val_size: 0.02
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
diff --git a/finetune/configs/lf/ds_z3_offload.json b/finetune/configs/lf/ds_z3_offload.json
new file mode 100644
index 00000000..9ab30b0b
--- /dev/null
+++ b/finetune/configs/lf/ds_z3_offload.json
@@ -0,0 +1,22 @@
+{
+ "train_batch_size": "auto",
+ "train_micro_batch_size_per_gpu": "auto",
+ "gradient_accumulation_steps": "auto",
+ "gradient_clipping": "auto",
+ "zero_allow_untested_optimizer": true,
+ "fp16": {"enabled": "auto", "loss_scale": 0, "loss_scale_window": 1000, "initial_scale_power": 16, "hysteresis": 2, "min_loss_scale": 1},
+ "bf16": {"enabled": "auto"},
+ "zero_optimization": {
+ "stage": 3,
+ "offload_optimizer": {"device": "cpu", "pin_memory": true},
+ "overlap_comm": false,
+ "contiguous_gradients": true,
+ "sub_group_size": 1e9,
+ "reduce_bucket_size": "auto",
+ "stage3_prefetch_bucket_size": "auto",
+ "stage3_param_persistence_threshold": "auto",
+ "stage3_max_live_parameters": 1e9,
+ "stage3_max_reuse_distance": 1e9,
+ "stage3_gather_16bit_weights_on_model_save": true
+ }
+}
diff --git a/finetune/configs/lf/export_dpo_exec_v1.yaml b/finetune/configs/lf/export_dpo_exec_v1.yaml
new file mode 100644
index 00000000..19dd568a
--- /dev/null
+++ b/finetune/configs/lf/export_dpo_exec_v1.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_dpo_exec_v1_r2.yaml b/finetune/configs/lf/export_dpo_exec_v1_r2.yaml
new file mode 100644
index 00000000..86c82577
--- /dev/null
+++ b/finetune/configs/lf/export_dpo_exec_v1_r2.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1/merged_r2
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_dpo_exec_v2.yaml b/finetune/configs/lf/export_dpo_exec_v2.yaml
new file mode 100644
index 00000000..be216069
--- /dev/null
+++ b/finetune/configs/lf/export_dpo_exec_v2.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1/merged_r2
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_dpo_md_big.yaml b/finetune/configs/lf/export_dpo_md_big.yaml
new file mode 100644
index 00000000..d3a5d00d
--- /dev/null
+++ b/finetune/configs/lf/export_dpo_md_big.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/merged
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_dpo_md_mixed.yaml b/finetune/configs/lf/export_dpo_md_mixed.yaml
new file mode 100644
index 00000000..e55a44c4
--- /dev/null
+++ b/finetune/configs/lf/export_dpo_md_mixed.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed/merged
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_lora_template.yaml b/finetune/configs/lf/export_lora_template.yaml
new file mode 100644
index 00000000..cb180b71
--- /dev/null
+++ b/finetune/configs/lf/export_lora_template.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_big.yaml b/finetune/configs/lf/export_md_big.yaml
new file mode 100644
index 00000000..d2760a99
--- /dev/null
+++ b/finetune/configs/lf/export_md_big.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_big_ck2200.yaml b/finetune/configs/lf/export_md_big_ck2200.yaml
new file mode 100644
index 00000000..8b05ac97
--- /dev/null
+++ b/finetune/configs/lf/export_md_big_ck2200.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2200
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2200/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_big_ck2420.yaml b/finetune/configs/lf/export_md_big_ck2420.yaml
new file mode 100644
index 00000000..b94ab337
--- /dev/null
+++ b/finetune/configs/lf/export_md_big_ck2420.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2420
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2420/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_big_ck2640.yaml b/finetune/configs/lf/export_md_big_ck2640.yaml
new file mode 100644
index 00000000..d5c70c4c
--- /dev/null
+++ b/finetune/configs/lf/export_md_big_ck2640.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2640
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2640/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_big_ck2860.yaml b/finetune/configs/lf/export_md_big_ck2860.yaml
new file mode 100644
index 00000000..38ddb9d7
--- /dev/null
+++ b/finetune/configs/lf/export_md_big_ck2860.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2860
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2860/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_big_ck3080.yaml b/finetune/configs/lf/export_md_big_ck3080.yaml
new file mode 100644
index 00000000..ec45a8ce
--- /dev/null
+++ b/finetune/configs/lf/export_md_big_ck3080.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-3080
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-3080/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_big_ck3292.yaml b/finetune/configs/lf/export_md_big_ck3292.yaml
new file mode 100644
index 00000000..1ba58e15
--- /dev/null
+++ b/finetune/configs/lf/export_md_big_ck3292.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-3292
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-3292/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_blender.yaml b/finetune/configs/lf/export_md_blender.yaml
new file mode 100644
index 00000000..6adc1487
--- /dev/null
+++ b/finetune/configs/lf/export_md_blender.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_blender
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_blender/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_cadquery.yaml b/finetune/configs/lf/export_md_cadquery.yaml
new file mode 100644
index 00000000..73477fb5
--- /dev/null
+++ b/finetune/configs/lf/export_md_cadquery.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_cadquery
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_cadquery/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_glsl.yaml b/finetune/configs/lf/export_md_glsl.yaml
new file mode 100644
index 00000000..6bca1ca4
--- /dev/null
+++ b/finetune/configs/lf/export_md_glsl.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_glsl
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_glsl/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_mixed.yaml b/finetune/configs/lf/export_md_mixed.yaml
new file mode 100644
index 00000000..4040c6dd
--- /dev/null
+++ b/finetune/configs/lf/export_md_mixed.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_openscad.yaml b/finetune/configs/lf/export_md_openscad.yaml
new file mode 100644
index 00000000..26fd21c5
--- /dev/null
+++ b/finetune/configs/lf/export_md_openscad.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_openscad
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_openscad/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_threejs.yaml b/finetune/configs/lf/export_md_threejs.yaml
new file mode 100644
index 00000000..be0d2128
--- /dev/null
+++ b/finetune/configs/lf/export_md_threejs.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_threejs
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_threejs/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_xl.yaml b/finetune/configs/lf/export_md_xl.yaml
new file mode 100644
index 00000000..469e0218
--- /dev/null
+++ b/finetune/configs/lf/export_md_xl.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_xl_ck450.yaml b/finetune/configs/lf/export_md_xl_ck450.yaml
new file mode 100644
index 00000000..3dc49d14
--- /dev/null
+++ b/finetune/configs/lf/export_md_xl_ck450.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl/checkpoint-450
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl/checkpoint-450/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_md_xl_geo.yaml b/finetune/configs/lf/export_md_xl_geo.yaml
new file mode 100644
index 00000000..72d06ece
--- /dev/null
+++ b/finetune/configs/lf/export_md_xl_geo.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl/merged_geo
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v1.yaml b/finetune/configs/lf/export_v1.yaml
new file mode 100644
index 00000000..cb180b71
--- /dev/null
+++ b/finetune/configs/lf/export_v1.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v10_4b.yaml b/finetune/configs/lf/export_v10_4b.yaml
new file mode 100644
index 00000000..47055b84
--- /dev/null
+++ b/finetune/configs/lf/export_v10_4b.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-4B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v10_4b
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v10_4b/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v11_selftrain.yaml b/finetune/configs/lf/export_v11_selftrain.yaml
new file mode 100644
index 00000000..0d2ecd19
--- /dev/null
+++ b/finetune/configs/lf/export_v11_selftrain.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11_selftrain
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11_selftrain/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v11b_selfprompts.yaml b/finetune/configs/lf/export_v11b_selfprompts.yaml
new file mode 100644
index 00000000..fde0c71a
--- /dev/null
+++ b/finetune/configs/lf/export_v11b_selfprompts.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11b_selfprompts
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11b_selfprompts/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v12_qwen3_8b.yaml b/finetune/configs/lf/export_v12_qwen3_8b.yaml
new file mode 100644
index 00000000..3fa9e2cb
--- /dev/null
+++ b/finetune/configs/lf/export_v12_qwen3_8b.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3-8B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v12_qwen3_8b
+template: qwen3
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v12_qwen3_8b/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v13_boot.yaml b/finetune/configs/lf/export_v13_boot.yaml
new file mode 100644
index 00000000..7b29279f
--- /dev/null
+++ b/finetune/configs/lf/export_v13_boot.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v13_boot
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v13_boot/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v1_ckpt100.yaml b/finetune/configs/lf/export_v1_ckpt100.yaml
new file mode 100644
index 00000000..8e0cf7e0
--- /dev/null
+++ b/finetune/configs/lf/export_v1_ckpt100.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/checkpoint-100
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/checkpoint-100/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v1_rep.yaml b/finetune/configs/lf/export_v1_rep.yaml
new file mode 100644
index 00000000..0572fc33
--- /dev/null
+++ b/finetune/configs/lf/export_v1_rep.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1_rep
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1_rep/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v2_indist.yaml b/finetune/configs/lf/export_v2_indist.yaml
new file mode 100644
index 00000000..f8501a47
--- /dev/null
+++ b/finetune/configs/lf/export_v2_indist.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2_indist
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2_indist/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v2b_indist16k.yaml b/finetune/configs/lf/export_v2b_indist16k.yaml
new file mode 100644
index 00000000..600cd46f
--- /dev/null
+++ b/finetune/configs/lf/export_v2b_indist16k.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2b_indist16k
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2b_indist16k/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v3_bio.yaml b/finetune/configs/lf/export_v3_bio.yaml
new file mode 100644
index 00000000..ca2ceee7
--- /dev/null
+++ b/finetune/configs/lf/export_v3_bio.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v3_bio_ckpt100.yaml b/finetune/configs/lf/export_v3_bio_ckpt100.yaml
new file mode 100644
index 00000000..58510df2
--- /dev/null
+++ b/finetune/configs/lf/export_v3_bio_ckpt100.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/checkpoint-100
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/checkpoint-100/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v3_bio_ckpt50.yaml b/finetune/configs/lf/export_v3_bio_ckpt50.yaml
new file mode 100644
index 00000000..18e2b383
--- /dev/null
+++ b/finetune/configs/lf/export_v3_bio_ckpt50.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/checkpoint-50
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/checkpoint-50/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v4_cq.yaml b/finetune/configs/lf/export_v4_cq.yaml
new file mode 100644
index 00000000..51fa42e0
--- /dev/null
+++ b/finetune/configs/lf/export_v4_cq.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v4_cq
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v4_cq/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v5_instr.yaml b/finetune/configs/lf/export_v5_instr.yaml
new file mode 100644
index 00000000..bb75a95a
--- /dev/null
+++ b/finetune/configs/lf/export_v5_instr.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v5_instr
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v5_instr/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v6_detail.yaml b/finetune/configs/lf/export_v6_detail.yaml
new file mode 100644
index 00000000..2cdca3eb
--- /dev/null
+++ b/finetune/configs/lf/export_v6_detail.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v6_detail
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v6_detail/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v7_ep4.yaml b/finetune/configs/lf/export_v7_ep4.yaml
new file mode 100644
index 00000000..62243783
--- /dev/null
+++ b/finetune/configs/lf/export_v7_ep4.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v7_ep4
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v7_ep4/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v8_r16.yaml b/finetune/configs/lf/export_v8_r16.yaml
new file mode 100644
index 00000000..c96edf7d
--- /dev/null
+++ b/finetune/configs/lf/export_v8_r16.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v8_r16
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v8_r16/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/export_v9_lr2e4.yaml b/finetune/configs/lf/export_v9_lr2e4.yaml
new file mode 100644
index 00000000..9b5a3de2
--- /dev/null
+++ b/finetune/configs/lf/export_v9_lr2e4.yaml
@@ -0,0 +1,12 @@
+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v9_lr2e4
+template: qwen3_5
+enable_thinking: false
+trust_remote_code: true
+finetuning_type: lora
+
+export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v9_lr2e4/merged
+export_size: 5
+export_device: cpu
+export_legacy_format: false
diff --git a/finetune/configs/lf/full_md_bal.yaml b/finetune/configs/lf/full_md_bal.yaml
new file mode 100644
index 00000000..a4cb4bf0
--- /dev/null
+++ b/finetune/configs/lf/full_md_bal.yaml
@@ -0,0 +1,41 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: full
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+deepspeed: /wekafs/ict/hx_624/tools/LLaMA-Factory/examples/deepspeed/ds_z3_config.json
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_bal_train
+eval_dataset: md_bal_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_bal_full_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_md_bal
+logging_steps: 5
+save_steps: 200
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 2
+learning_rate: 1.0e-05
+num_train_epochs: 1.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 200
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_md_bal.yaml b/finetune/configs/lf/lora_md_bal.yaml
new file mode 100644
index 00000000..4d8ed44c
--- /dev/null
+++ b/finetune/configs/lf/lora_md_bal.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_bal_train
+eval_dataset: md_bal_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_bal_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_bal
+logging_steps: 5
+save_steps: 200
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 1.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 200
+save_total_limit: 12
diff --git a/finetune/configs/lf/lora_md_big.yaml b/finetune/configs/lf/lora_md_big.yaml
new file mode 100644
index 00000000..83cc865d
--- /dev/null
+++ b/finetune/configs/lf/lora_md_big.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_big_train
+eval_dataset: md_big_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_big_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big
+logging_steps: 5
+save_steps: 220
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 220
+save_total_limit: 6
diff --git a/finetune/configs/lf/lora_md_blender.yaml b/finetune/configs/lf/lora_md_blender.yaml
new file mode 100644
index 00000000..69b0fd2c
--- /dev/null
+++ b/finetune/configs/lf/lora_md_blender.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_blender_train
+eval_dataset: md_blender_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_blender_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_blender
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_md_cadquery.yaml b/finetune/configs/lf/lora_md_cadquery.yaml
new file mode 100644
index 00000000..c4a555b0
--- /dev/null
+++ b/finetune/configs/lf/lora_md_cadquery.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_cadquery_train
+eval_dataset: md_cadquery_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_cadquery_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_cadquery
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_md_glsl.yaml b/finetune/configs/lf/lora_md_glsl.yaml
new file mode 100644
index 00000000..b0039aa3
--- /dev/null
+++ b/finetune/configs/lf/lora_md_glsl.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_glsl_train
+eval_dataset: md_glsl_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_glsl_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_glsl
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_md_mixed.yaml b/finetune/configs/lf/lora_md_mixed.yaml
new file mode 100644
index 00000000..d3ba9ff8
--- /dev/null
+++ b/finetune/configs/lf/lora_md_mixed.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_mixed_train
+eval_dataset: md_mixed_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_mixed_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_md_openscad.yaml b/finetune/configs/lf/lora_md_openscad.yaml
new file mode 100644
index 00000000..89250f53
--- /dev/null
+++ b/finetune/configs/lf/lora_md_openscad.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_openscad_train
+eval_dataset: md_openscad_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_openscad_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_openscad
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_md_threejs.yaml b/finetune/configs/lf/lora_md_threejs.yaml
new file mode 100644
index 00000000..e6ad9357
--- /dev/null
+++ b/finetune/configs/lf/lora_md_threejs.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_threejs_train
+eval_dataset: md_threejs_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 16384
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_threejs_len16384_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_threejs
+logging_steps: 5
+save_steps: 1000
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 3.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 1000
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_md_xl.yaml b/finetune/configs/lf/lora_md_xl.yaml
new file mode 100644
index 00000000..b298f06a
--- /dev/null
+++ b/finetune/configs/lf/lora_md_xl.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: md_xl_train
+eval_dataset: md_xl_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_xl_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl
+logging_steps: 5
+save_steps: 450
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 1.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 450
+save_total_limit: 14
diff --git a/finetune/configs/lf/lora_v1.yaml b/finetune/configs/lf/lora_v1.yaml
new file mode 100644
index 00000000..40607d24
--- /dev/null
+++ b/finetune/configs/lf/lora_v1.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v10_4b.yaml b/finetune/configs/lf/lora_v10_4b.yaml
new file mode 100644
index 00000000..7f660d31
--- /dev/null
+++ b/finetune/configs/lf/lora_v10_4b.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-4B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v10_4b
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v11_selftrain.yaml b/finetune/configs/lf/lora_v11_selftrain.yaml
new file mode 100644
index 00000000..eaf67c23
--- /dev/null
+++ b/finetune/configs/lf/lora_v11_selftrain.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: v11_selftrain_train
+eval_dataset: v11_selftrain_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v11_selftrain_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11_selftrain
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v11b_selfprompts.yaml b/finetune/configs/lf/lora_v11b_selfprompts.yaml
new file mode 100644
index 00000000..2acb9306
--- /dev/null
+++ b/finetune/configs/lf/lora_v11b_selfprompts.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: v11b_selfprompts_train
+eval_dataset: v11b_selfprompts_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v11b_selfprompts_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11b_selfprompts
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v12_qwen3_8b.yaml b/finetune/configs/lf/lora_v12_qwen3_8b.yaml
new file mode 100644
index 00000000..5e614d0c
--- /dev/null
+++ b/finetune/configs/lf/lora_v12_qwen3_8b.yaml
@@ -0,0 +1,42 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3-8B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_qwen3_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v12_qwen3_8b
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v13_boot.yaml b/finetune/configs/lf/lora_v13_boot.yaml
new file mode 100644
index 00000000..033cb840
--- /dev/null
+++ b/finetune/configs/lf/lora_v13_boot.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: v13_boot_train
+eval_dataset: v13_boot_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v13_boot_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v13_boot
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v1_rep.yaml b/finetune/configs/lf/lora_v1_rep.yaml
new file mode 100644
index 00000000..919978e9
--- /dev/null
+++ b/finetune/configs/lf/lora_v1_rep.yaml
@@ -0,0 +1,45 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1_rep
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
+seed: 1
diff --git a/finetune/configs/lf/lora_v2_indist.yaml b/finetune/configs/lf/lora_v2_indist.yaml
new file mode 100644
index 00000000..a3a34010
--- /dev/null
+++ b/finetune/configs/lf/lora_v2_indist.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: v2_indist_train
+eval_dataset: v2_indist_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v2_indist_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2_indist
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v2b_indist16k.yaml b/finetune/configs/lf/lora_v2b_indist16k.yaml
new file mode 100644
index 00000000..0efc29b8
--- /dev/null
+++ b/finetune/configs/lf/lora_v2b_indist16k.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: v2b_indist16k_train
+eval_dataset: v2b_indist16k_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 16384
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v2b_indist16k_len16384_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2b_indist16k
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v3_bio.yaml b/finetune/configs/lf/lora_v3_bio.yaml
new file mode 100644
index 00000000..0eafe410
--- /dev/null
+++ b/finetune/configs/lf/lora_v3_bio.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: v3_bio_train
+eval_dataset: v3_bio_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v3_bio_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v4_cq.yaml b/finetune/configs/lf/lora_v4_cq.yaml
new file mode 100644
index 00000000..33a62b4d
--- /dev/null
+++ b/finetune/configs/lf/lora_v4_cq.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: v4_cq_train
+eval_dataset: v4_cq_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v4_cq_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v4_cq
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v5_instr.yaml b/finetune/configs/lf/lora_v5_instr.yaml
new file mode 100644
index 00000000..a5eaaa1c
--- /dev/null
+++ b/finetune/configs/lf/lora_v5_instr.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: v5_instr_train
+eval_dataset: v5_instr_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v5_instr_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v5_instr
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v6_detail.yaml b/finetune/configs/lf/lora_v6_detail.yaml
new file mode 100644
index 00000000..f374b184
--- /dev/null
+++ b/finetune/configs/lf/lora_v6_detail.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: v6_detail_train
+eval_dataset: v6_detail_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v6_detail_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v6_detail
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v7_ep4.yaml b/finetune/configs/lf/lora_v7_ep4.yaml
new file mode 100644
index 00000000..afa1ec53
--- /dev/null
+++ b/finetune/configs/lf/lora_v7_ep4.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v7_ep4
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 4.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v8_r16.yaml b/finetune/configs/lf/lora_v8_r16.yaml
new file mode 100644
index 00000000..35275900
--- /dev/null
+++ b/finetune/configs/lf/lora_v8_r16.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v8_r16
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/lora_v9_lr2e4.yaml b/finetune/configs/lf/lora_v9_lr2e4.yaml
new file mode 100644
index 00000000..7d30553d
--- /dev/null
+++ b/finetune/configs/lf/lora_v9_lr2e4.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v9_lr2e4
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0002
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/qwen35_9b_full_sft.yaml b/finetune/configs/lf/qwen35_9b_full_sft.yaml
new file mode 100644
index 00000000..caf070dc
--- /dev/null
+++ b/finetune/configs/lf/qwen35_9b_full_sft.yaml
@@ -0,0 +1,41 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: full
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+deepspeed: /wekafs/ict/hx_624/tools/LLaMA-Factory/examples/deepspeed/ds_z3_config.json
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1
+logging_steps: 5
+save_steps: 60
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 3
+learning_rate: 1.0e-05
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 60
+save_total_limit: 1
diff --git a/finetune/configs/lf/qwen35_9b_full_sft_3gpu.yaml b/finetune/configs/lf/qwen35_9b_full_sft_3gpu.yaml
new file mode 100644
index 00000000..7b00d840
--- /dev/null
+++ b/finetune/configs/lf/qwen35_9b_full_sft_3gpu.yaml
@@ -0,0 +1,41 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: full
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+deepspeed: /wekafs/ict/hx_624/tools/LLaMA-Factory/examples/deepspeed/ds_z3_config.json
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v2_3gpu
+logging_steps: 5
+save_steps: 1000
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 3
+learning_rate: 1.0e-05
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 50
+save_total_limit: 1
diff --git a/finetune/configs/lf/qwen35_9b_full_sft_4gpu.yaml b/finetune/configs/lf/qwen35_9b_full_sft_4gpu.yaml
new file mode 100644
index 00000000..a461fa6d
--- /dev/null
+++ b/finetune/configs/lf/qwen35_9b_full_sft_4gpu.yaml
@@ -0,0 +1,41 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: full
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+deepspeed: /wekafs/ict/hx_624/tools/LLaMA-Factory/examples/deepspeed/ds_z3_config.json
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v3_4gpu
+logging_steps: 5
+save_steps: 1000
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 2
+learning_rate: 1.0e-05
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 1000
+save_total_limit: 1
diff --git a/finetune/configs/lf/qwen35_9b_lora_sft.yaml b/finetune/configs/lf/qwen35_9b_lora_sft.yaml
new file mode 100644
index 00000000..599e6dff
--- /dev/null
+++ b/finetune/configs/lf/qwen35_9b_lora_sft.yaml
@@ -0,0 +1,44 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+flash_attn: fa2
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_rank: 64
+lora_alpha: 128
+lora_dropout: 0.05
+lora_target: all
+freeze_vision_tower: true
+freeze_multi_modal_projector: true
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+dataloader_num_workers: 4
+neat_packing: true
+packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1
+logging_steps: 5
+save_steps: 50
+save_only_model: true
+plot_loss: true
+overwrite_output_dir: true
+report_to: tensorboard
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 8
+learning_rate: 0.0001
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+gradient_checkpointing: true
+ddp_timeout: 180000000
+resume_from_checkpoint: null
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 100
+save_total_limit: 2
diff --git a/finetune/configs/lf/qwen35_9b_preprocess.yaml b/finetune/configs/lf/qwen35_9b_preprocess.yaml
new file mode 100644
index 00000000..b3683907
--- /dev/null
+++ b/finetune/configs/lf/qwen35_9b_preprocess.yaml
@@ -0,0 +1,17 @@
+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B
+trust_remote_code: true
+stage: sft
+do_train: true
+finetuning_type: lora
+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf
+dataset: blender3d_v1_train
+eval_dataset: blender3d_v1_val
+template: qwen3_5
+enable_thinking: false
+cutoff_len: 8192
+preprocessing_num_workers: 16
+packing: true
+neat_packing: true
+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed
+output_dir: /wekafs/ict/hx_624/llm-ft/runs/_preprocess_tmp
+overwrite_output_dir: true
diff --git a/finetune/data_specs/dataset_info.json b/finetune/data_specs/dataset_info.json
new file mode 100644
index 00000000..d4cb7d10
--- /dev/null
+++ b/finetune/data_specs/dataset_info.json
@@ -0,0 +1,350 @@
+{
+ "blender3d_v1_train": {
+ "file_name": "blender3d_v1_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "blender3d_v1_val": {
+ "file_name": "blender3d_v1_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v2_indist_train": {
+ "file_name": "v2_indist_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v2_indist_val": {
+ "file_name": "v2_indist_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v3_bio_train": {
+ "file_name": "v3_bio_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v3_bio_val": {
+ "file_name": "v3_bio_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v4_cq_train": {
+ "file_name": "v4_cq_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v4_cq_val": {
+ "file_name": "v4_cq_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v5_instr_train": {
+ "file_name": "v5_instr_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v5_instr_val": {
+ "file_name": "v5_instr_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v6_detail_train": {
+ "file_name": "v6_detail_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v6_detail_val": {
+ "file_name": "v6_detail_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v2b_indist16k_train": {
+ "file_name": "v2b_indist16k_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v2b_indist16k_val": {
+ "file_name": "v2b_indist16k_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v11_selftrain_train": {
+ "file_name": "v11_selftrain_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v11_selftrain_val": {
+ "file_name": "v11_selftrain_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v13_boot_train": {
+ "file_name": "v13_boot_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v13_boot_val": {
+ "file_name": "v13_boot_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v11b_selfprompts_train": {
+ "file_name": "v11b_selfprompts_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "v11b_selfprompts_val": {
+ "file_name": "v11b_selfprompts_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "dpo_exec_pairs": {
+ "file_name": "dpo_exec_pairs.json",
+ "ranking": true,
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system",
+ "chosen": "chosen",
+ "rejected": "rejected"
+ }
+ },
+ "md_blender_train": {
+ "file_name": "md_blender_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_blender_val": {
+ "file_name": "md_blender_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_cadquery_train": {
+ "file_name": "md_cadquery_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_cadquery_val": {
+ "file_name": "md_cadquery_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_openscad_train": {
+ "file_name": "md_openscad_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_openscad_val": {
+ "file_name": "md_openscad_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_glsl_train": {
+ "file_name": "md_glsl_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_glsl_val": {
+ "file_name": "md_glsl_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_mixed_train": {
+ "file_name": "md_mixed_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_mixed_val": {
+ "file_name": "md_mixed_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "dpo_exec_pairs_r2": {
+ "file_name": "dpo_exec_pairs_r2.json",
+ "ranking": true,
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system",
+ "chosen": "chosen",
+ "rejected": "rejected"
+ }
+ },
+ "dpo_md_pairs": {
+ "file_name": "dpo_md_pairs.json",
+ "ranking": true,
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system",
+ "chosen": "chosen",
+ "rejected": "rejected"
+ }
+ },
+ "md_big_train": {
+ "file_name": "md_big_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_big_val": {
+ "file_name": "md_big_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_threejs_train": {
+ "file_name": "md_threejs_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_threejs_val": {
+ "file_name": "md_threejs_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "dpo_md_big_pairs": {
+ "file_name": "dpo_md_big_pairs.json",
+ "ranking": true,
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system",
+ "chosen": "chosen",
+ "rejected": "rejected"
+ }
+ },
+ "md_xl_train": {
+ "file_name": "md_xl_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_xl_val": {
+ "file_name": "md_xl_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_bal_train": {
+ "file_name": "md_bal_train.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ },
+ "md_bal_val": {
+ "file_name": "md_bal_val.json",
+ "formatting": "sharegpt",
+ "columns": {
+ "messages": "conversations",
+ "system": "system"
+ }
+ }
+}
\ No newline at end of file
diff --git a/finetune/docs/PROJECT_README.md b/finetune/docs/PROJECT_README.md
new file mode 100644
index 00000000..f667ec29
--- /dev/null
+++ b/finetune/docs/PROJECT_README.md
@@ -0,0 +1,52 @@
+# llm-ft — fine-tuning code LLMs for text → Blender-Python (3D) on dgx03
+
+Everything lives under `/wekafs/ict/hx_624` (home is full). GPUs allowed: **0–3 only** (2/3 are often busy with another user's jobs → check `nvidia-smi`).
+
+## Environments
+| env | purpose | how |
+|---|---|---|
+| `llmft` (conda) | data prep, HF generation eval, metrics | `source llm-ft/env.sh` |
+| `lf` (conda) | **LLaMA-Factory** training (`/wekafs/ict/hx_624/tools/LLaMA-Factory`) | `scripts/lf_train.sh` activates it |
+| Blender 5.0.1 | executes generated scripts headlessly | `/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender` |
+
+Anaconda 2026.07 is at `/wekafs/ict/hx_624/anaconda3` (already `conda init`-ed in `~/.bashrc`).
+
+## Data
+* Raw: `/wekafs/ict/hx_624/data/` — `3dcodebench/` (212 test tasks, GT GLB), `3dcodeverse/*.parquet` (bioinspired3d, factories, …), `thingiverse_5_0/`, `3dcodeverse/blender_distill/`.
+* `scripts/prep_data.py` → `data/sft_v1/{train,val}.jsonl` (+ `bench_prompts.jsonl`, `stats.json`). The 212 benchmark factories are **excluded** from training (no contamination); only the 31 non-overlapping factories are used.
+* `scripts/to_llamafactory.py` → `data/lf/*.json` + `dataset_info.json` (sharegpt format for LLaMA-Factory).
+
+## Train (LLaMA-Factory)
+```bash
+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_lora_sft.yaml # LoRA
+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml # full FT, DeepSpeed ZeRO-3
+```
+(`scripts/train_sft.py` + `scripts/launch_sft.sh` are an older plain-TRL alternative, kept for reference.)
+
+## Evaluate on 3DCodeBench (212 tasks)
+```bash
+GPUS=0,1 eval/run_eval.sh <model_dir> eval/out/<name> [--no_think] [--batch_size 16]
+```
+= `eval/generate.py` (HF batched greedy generation, sharded over GPUs) → `eval/run_bench.py` (runs every `code.py` in headless Blender via `eval/blender_runner.py`, exports GLB) → `eval/metrics.py` (Chamfer / F-score vs GT GLB, best of 4 yaw rotations). Results: `eval/out/<name>/summary.json`.
+Sanity: re-running the *reference* scripts through the harness gives Chamfer≈0.016, [email protected]≈1.0 (`eval/ref_oracle`).
+
+## Experiment tooling (added 2026-08-21)
+* `scripts/make_lora_cfg.py NAME DATANAME [--epochs --lr --rank --cutoff]` → `configs/lf/lora_NAME.yaml`
+* `scripts/run_exp.sh <GPU[,GPU]> NAME` → train LoRA → merge (`lf_export.sh`) → vLLM generation → Blender exec + metrics (background) → `eval/out/qwen35_9b_lora_NAME/summary.json`
+* `scripts/queue.sh <GPU> NAME1 NAME2 …` → run experiments sequentially on one GPU
+* `scripts/ckpt_sweep.sh <GPU> <run_dir> NAME step1 step2 …` → evaluate intermediate adapter checkpoints
+* `scripts/run_full_ft*.sh` → full-parameter ZeRO-3 SFT orchestration (3 GPUs, or 2 GPUs + CPU offload via `configs/lf/ds_z3_offload.json`)
+* `eval/run_eval_vllm.sh <model> <out>` → vLLM-based eval (≈3k tok/s, 212 prompts in ~1.5 min); `eval/compare.py`, `eval/analyze_runs.py` → tables/analysis
+* Results + conclusions: `REPORT.md`
+
+## Multi-dialect tooling (added 2026-08-21 evening)
+* `scripts/build_multidialect.py` → `data/multidialect/<dialect>/{train,test}.jsonl` + `report.json` (QC of every 3DCodeVerse subset; dialects: blender, cadquery, openscad, glsl)
+* `scripts/build_multidialect_train.py` → token-filtered subsets `data/md_<dialect>` + `data/md_mixed` (LF datasets `md_*`)
+* Executors: `eval/dialect_runners.py` (CadQuery via `cadquery` 2.8 in env `llmft`; OpenSCAD AppImage at `tools/openscad/squashfs-root/AppRun`; GLSL compile via conda-forge `glslangValidator`); Blender via `eval/blender_runner.py`
+* `eval/run_dialect_eval.sh <model> <name> [dialects]` → vLLM generation + execution + reference-based F-score per dialect → `eval/out/md_<name>_<dialect>/summary.json`
+* `scripts/run_md_exp.sh <GPU> md_<dialect>` → LoRA train → 3DCodeBench + 4-dialect eval
+
+## 数据复用(2026-08-22)
+- 各语言清洗结果固定在 `data/multidialect/<dialect>/{train,test}.jsonl`,所有 md_* 配比脚本(`scripts/build_md_big.py` / `build_md_xl.py`)只从这里取样;
+- 每条样本的 token 数缓存在同名 `.ntok` 文件(`scripts/cache_ntok.py` 一次生成,`scripts/ntok_cache.py::load_ntok` 读取),改配比/上限不再重新 tokenize;
+- LLaMA-Factory 的 `tokenized_path`(`data/lf/tokenized_<name>_len8192_packed`)缓存打包后的数据集,同一数据+cutoff 的后续训练直接复用。
diff --git a/finetune/docs/assets/md_xl_3dcodebench_contact_sheet.png b/finetune/docs/assets/md_xl_3dcodebench_contact_sheet.png
new file mode 100644
index 00000000..ac2b91dc
Binary files /dev/null and b/finetune/docs/assets/md_xl_3dcodebench_contact_sheet.png differ
diff --git a/finetune/env.sh b/finetune/env.sh
new file mode 100644
index 00000000..01249a0a
--- /dev/null
+++ b/finetune/env.sh
@@ -0,0 +1,11 @@
+# source this before running anything in the project
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh
+conda activate llmft
+source /wekafs/ict/hx_624/.secrets/hf.env
+export WORKSPACE=/wekafs/ict/hx_624
+export HF_HOME=$WORKSPACE/cache/huggingface HF_XET_HIGH_PERFORMANCE=1
+export TRITON_CACHE_DIR=$WORKSPACE/cache/triton TORCH_EXTENSIONS_DIR=$WORKSPACE/cache/torch_extensions
+export PIP_CACHE_DIR=$WORKSPACE/cache/pip CUDA_HOME=/usr/local/cuda-12.8
+export TOKENIZERS_PARALLELISM=false
+mkdir -p $TRITON_CACHE_DIR $TORCH_EXTENSIONS_DIR
+export VLLM_CACHE_ROOT=$WORKSPACE/cache/vllm
diff --git a/finetune/env/requirements-lf.txt b/finetune/env/requirements-lf.txt
new file mode 100644
index 00000000..65ec0d10
--- /dev/null
+++ b/finetune/env/requirements-lf.txt
@@ -0,0 +1,143 @@
+absl-py==2.5.0
+accelerate==1.11.0
+aiofiles==24.1.0
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.3
+aiosignal==1.4.0
+annotated-doc==0.0.5
+annotated-types==0.8.0
+antlr4-python3-runtime==4.9.3
+anyio==4.14.2
+attrs==26.1.0
+av==16.0.0
+brotli==1.2.0
+causal-conv1d==1.7.0
+certifi==2026.7.22
+charset-normalizer==3.5.1
+click==8.4.2
+contourpy==1.3.3
+cycler==0.12.1
+datasets==4.0.0
+deepspeed==0.19.5
+dill==0.3.8
+docstring_parser==0.18.0
+einops==0.8.2
+fastapi==0.141.1
+ffmpy==1.0.0
+filelock==3.32.3
+fire==0.7.1
+fla-core==0.5.2
+flash-linear-attention==0.5.2
+flash_attn @ https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3/flash_attn-2.8.3+cu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl#sha256=f25da18657a87fc83dc1bfb8b7751b82246e9db355510226b674fd437c34b5fb
+fonttools==4.63.0
+frozenlist==1.8.0
+fsspec==2025.3.0
+gradio==5.50.0
+gradio_client==1.14.0
+groovy==0.1.2
+grpcio==1.83.0
+h11==0.16.0
+hf-xet==1.6.0
+hf_transfer==0.1.9
+hjson==3.1.0
+httpcore==1.0.9
+httpx==0.28.1
+huggingface_hub==1.28.0
+idna==3.19
+Jinja2==3.1.6
+jsonlines==4.0.0
+kiwisolver==1.5.0
+liger_kernel==0.8.2
+-e git+https://github.com/hiyouga/LLaMA-Factory.git@c4e09c7cbe18844816af9e18a97fe465515edbcd#egg=llamafactory
+Markdown==3.10.3
+markdown-it-py==4.2.0
+MarkupSafe==3.0.3
+matplotlib==3.11.1
+mdurl==0.1.2
+modelscope==1.39.1
+modelscope-hub==0.2.0
+mpmath==1.3.0
+msgpack==1.2.1
+multidict==6.7.1
+multiprocess==0.70.16
+networkx==3.6.1
+ninja==1.13.0
+numpy==2.5.2
+nvidia-cublas-cu12==12.8.4.1
+nvidia-cuda-cupti-cu12==12.8.90
+nvidia-cuda-nvrtc-cu12==12.8.93
+nvidia-cuda-runtime-cu12==12.8.90
+nvidia-cudnn-cu12==9.10.2.21
+nvidia-cufft-cu12==11.3.3.83
+nvidia-cufile-cu12==1.13.1.3
+nvidia-curand-cu12==10.3.9.90
+nvidia-cusolver-cu12==11.7.3.90
+nvidia-cusparse-cu12==12.5.8.93
+nvidia-cusparselt-cu12==0.7.1
+nvidia-nccl-cu12==2.27.3
+nvidia-nvjitlink-cu12==12.8.93
+nvidia-nvshmem-cu12==3.3.20
+nvidia-nvtx-cu12==12.8.90
+omegaconf==2.3.1
+orjson==3.12.0
+packaging==26.3
+pandas==2.3.3
+peft==0.18.1
+pillow==11.3.0
+propcache==0.5.2
+protobuf==7.36.0
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pyarrow==25.0.1
+pydantic==2.12.3
+pydantic_core==2.41.4
+pydub==0.25.1
+Pygments==2.21.0
+pyparsing==3.3.2
+python-dateutil==2.9.0.post0
+python-multipart==0.0.32
+pytz==2026.3.post1
+PyYAML==6.0.3
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+ruff==0.16.4
+safehttpx==0.1.7
+safetensors==0.8.0
+scipy==1.18.0
+semantic-version==2.10.0
+sentencepiece==0.2.2
+setuptools==84.0.0
+shellingham==1.5.4
+shtab==1.11.0
+six==1.17.0
+sse-starlette==3.4.8
+starlette==0.52.1
+sympy==1.14.0
+tensorboard==2.21.0
+tensorboard-data-server==0.7.2
+termcolor==3.3.0
+tiktoken==0.14.0
+tokenizers==0.22.2
+tomlkit==0.13.3
+torch==2.8.0+cu128
+torchaudio==2.8.0+cu128
+torchdata==0.11.0
+torchvision==0.23.0+cu128
+tqdm==4.70.0
+transformers==5.8.0
+trimesh==5.0.0
+triton==3.7.1
+trl==0.24.0
+typer==0.27.1
+typing-inspection==0.4.4
+typing_extensions==4.16.0
+tyro==0.8.14
+tzdata==2026.3
+urllib3==2.7.0
+uvicorn==0.52.4
+websockets==15.0.1
+Werkzeug==3.1.8
+wheel==0.48.0
+xxhash==4.0.1
+yarl==1.24.5
diff --git a/finetune/env/requirements-llmft.txt b/finetune/env/requirements-llmft.txt
new file mode 100644
index 00000000..191f5e6a
--- /dev/null
+++ b/finetune/env/requirements-llmft.txt
@@ -0,0 +1,167 @@
+absl-py==2.5.0
+accelerate==1.14.0
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.3
+aiosignal==1.4.0
+annotated-doc==0.0.5
+annotated-types==0.8.0
+anyio==4.14.2
+attrs==26.1.0
+bitsandbytes==0.50.1
+cadquery==2.8.0
+cadquery-ocp==7.9.3.1.1
+cadquery-ocp-proxy==7.9.3.1.1
+casadi==3.7.2
+certifi==2026.7.22
+cffi==2.1.1
+charset-normalizer==3.5.1
+click==8.4.2
+contourpy==1.3.3
+cryptography==50.0.0
+cycler==0.12.1
+dataclasses-json==0.6.7
+datasets==5.0.1
+deepspeed==0.19.5
+Deprecated==1.3.1
+dill==0.4.1
+dnspython==2.8.0
+einops==0.8.2
+email-validator==2.3.0
+ezdxf==1.4.4
+filelock==3.32.3
+fla-core==0.5.2
+flash-linear-attention==0.5.2
+flash_attn @ https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3/flash_attn-2.8.3+cu12torch2.9cxx11abiTRUE-cp312-cp312-linux_x86_64.whl#sha256=4e2f9e39313266b1544b68138b15b91ee6221eccf14f7902b7c6620351340810
+fonttools==4.63.0
+frozenlist==1.8.0
+fsspec==2026.6.0
+greenlet==3.5.5
+grpcio==1.83.0
+h11==0.16.0
+hf-xet==1.6.0
+hf_transfer==0.1.9
+hjson==3.1.0
+httpcore==1.0.9
+httpx==0.28.1
+huggingface_hub==1.28.0
+id==1.6.1
+idna==3.19
+Jinja2==3.1.6
+joblib==1.5.3
+jsonlines==4.0.0
+kernels==0.16.0
+kernels-data==0.16.0
+kiwisolver==1.5.0
+liger_kernel==0.8.2
+llvmlite==0.49.0
+Markdown==3.10.3
+markdown-it-py==4.2.0
+MarkupSafe==3.0.3
+marshmallow==3.26.2
+matplotlib==3.11.1
+mdurl==0.1.2
+more-itertools==11.1.0
+mpmath==1.3.0
+msgpack==1.2.1
+multidict==6.7.1
+multimethod==1.12
+multiprocess==0.70.19
+mypy_extensions==1.1.0
+narwhals==2.25.0
+networkx==3.6.1
+ninja==1.13.0
+nlopt==2.11.0
+numba==0.67.0
+numpy==2.5.2
+nvidia-cublas-cu12==12.8.4.1
+nvidia-cuda-cupti-cu12==12.8.90
+nvidia-cuda-nvrtc-cu12==12.8.93
+nvidia-cuda-runtime-cu12==12.8.90
+nvidia-cudnn-cu12==9.10.2.21
+nvidia-cufft-cu12==11.3.3.83
+nvidia-cufile-cu12==1.13.1.3
+nvidia-curand-cu12==10.3.9.90
+nvidia-cusolver-cu12==11.7.3.90
+nvidia-cusparse-cu12==12.5.8.93
+nvidia-cusparselt-cu12==0.7.1
+nvidia-nccl-cu12==2.27.5
+nvidia-nvjitlink-cu12==12.8.93
+nvidia-nvshmem-cu12==3.3.20
+nvidia-nvtx-cu12==12.8.90
+opentelemetry-api==1.44.0
+packaging==26.3
+pandas==3.0.5
+peft==0.20.0
+pillow==12.3.0
+platformdirs==4.11.3
+playwright==1.62.0
+propcache==0.5.2
+protobuf==7.36.0
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pyarrow==25.0.1
+pyasn1==0.6.4
+pycparser==3.0
+pydantic==2.13.4
+pydantic_core==2.46.4
+pyee==13.0.1
+pygltflib==1.16.5
+Pygments==2.21.0
+PyJWT==2.13.0
+pyOpenSSL==26.4.0
+pyparsing==3.3.2
+python-dateutil==2.9.0.post0
+PyYAML==6.0.3
+regex==2026.7.19
+requests==2.34.2
+rfc3161-client==1.0.8
+rfc8785==0.1.4
+rich==15.0.0
+runtype==0.5.3
+safetensors==0.8.0
+scikit-learn==1.9.0
+scipy==1.18.0
+securesystemslib==1.4.0
+sentencepiece==0.2.2
+sentry-sdk==2.68.0
+setuptools==84.0.0
+shellingham==1.5.4
+sigstore==4.5.0
+sigstore-models==0.0.6
+sigstore-rekor-types==0.0.18
+six==1.17.0
+sympy==1.14.0
+tensorboard==2.21.0
+tensorboard-data-server==0.7.2
+threadpoolctl==3.6.0
+tiktoken==0.14.0
+tokenizers==0.22.2
+tomlkit==0.15.1
+torch==2.9.1+cu128
+torchvision==0.24.1+cu128
+tqdm==4.70.0
+trame==3.13.2
+trame-client==3.13.6
+trame-common==1.2.7
+trame-components==2.5.0
+trame-server==3.14.0
+trame-vtk==2.11.16
+trame-vuetify==3.2.5
+transformers==5.15.1
+trimesh==5.0.0
+triton==3.5.1
+trl==1.10.0
+tuf==7.0.0
+typer==0.27.1
+typing-inspect==0.9.0
+typing-inspection==0.4.4
+typing_extensions==4.16.0
+urllib3==2.7.0
+vtk==9.6.2
+wandb==0.28.2
+Werkzeug==3.1.8
+wheel==0.48.0
+wrapt==2.3.0
+wslink==2.5.7
+xxhash==4.0.1
+yarl==1.24.5
diff --git a/finetune/env/requirements-vllm.txt b/finetune/env/requirements-vllm.txt
new file mode 100644
index 00000000..0e1bba95
--- /dev/null
+++ b/finetune/env/requirements-vllm.txt
@@ -0,0 +1,200 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.3
+aiosignal==1.4.0
+annotated-doc==0.0.5
+annotated-types==0.8.0
+anthropic==1.0.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.11
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.7
+cbor2==6.1.4
+certifi==2026.7.22
+cffi==2.1.1
+charset-normalizer==3.5.1
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.17.0
+cryptography==50.0.0
+cuda-bindings==12.9.7
+cuda-core==1.1.1
+cuda-pathfinder==1.6.1
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+dnspython==2.8.0
+docstring_parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.136.3
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.23.0
+fastar==0.12.0
+fastsafetensors==0.3.3
+filelock==3.32.3
+fla-core==0.5.2
+flash-linear-attention==0.5.2
+flashinfer-python==0.6.16.post3
+frozenlist==1.8.0
+fsspec==2026.7.0
+googleapis-common-protos==1.75.1
+grpcio==1.83.0
+h11==0.16.0
+hf-xet==1.6.0
+httpcore==1.0.9
+httpcore2==2.12.0
+httptools==0.8.0
+httpx==0.28.1
+httpx2==2.12.0
+huggingface_hub==1.28.0
+humming-kernels==0.1.10
+idna==3.19
+ijson==3.5.1
+interegular==0.3.3
+Jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.7.6
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+MarkupSafe==3.0.3
+mcp==2.0.0
+mcp-types==2.0.0
+mdurl==0.1.2
+mistral_common==1.11.7
+ml_dtypes==0.6.0
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+nccl4py==0.4.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cccl-cu12==12.9.27
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvcc-cu12==12.9.86
+nvidia-cuda-nvdisasm==13.3.73
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.20.0.48
+nvidia-cudnn-frontend==1.27.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.8.1
+nvidia-cutlass-dsl==4.6.0
+nvidia-cutlass-dsl-libs-base==4.6.0
+nvidia-cutlass-dsl-libs-core==4.6.0
+nvidia-cutlass-dsl-libs-cu12==4.6.0
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.29.7
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+nvtx==0.2.15
+openai==3.3.1
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines_core==0.2.14
+packaging==26.2
+pandas==3.0.5
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-fastapi-instrumentator==8.1.0
+prometheus_client==0.26.0
+propcache==0.5.2
+protobuf==6.33.6
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pyarrow==25.0.1
+pybase64==1.5.0
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.15.0
+pydantic_core==2.46.4
+pyelftools==0.33
+Pygments==2.21.0
+PyJWT==2.13.0
+pynvvideocodec==2.0.4
+python-dateutil==2.9.0.post0
+python-dotenv==1.2.3
+python-json-logger==4.2.0
+python-multipart==0.0.32
+PyYAML==6.0.3
+pyzmq==27.2.0
+quack-kernels==0.6.1
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.1
+rpds-py==2026.6.3
+safetensors==0.8.0
+scipy==1.18.0
+sentencepiece==0.2.2
+sentry-sdk==2.68.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.8
+starlette==1.6.0
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.14.0
+tilelang==0.1.12
+tokenizers==0.22.2
+tokenspeed-mla==0.1.8
+tokenspeed-triton==3.8.10.post20260721
+torch==2.13.0+cu129
+torch_c_dlpack_ext==0.1.5
+torchaudio==2.11.0+cu129
+torchcodec==0.16.0+cu129
+torchvision==0.28.0+cu129
+tqdm==4.70.0
+transformers==5.15.1
+trimesh==5.0.0
+triton==3.7.1
+truststore==0.10.4
+typer==0.27.1
+typing-inspection==0.4.4
+typing_extensions==4.16.0
+urllib3==2.7.0
+uvicorn==0.52.4
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl#sha256=bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b
+watchfiles==1.2.0
+websockets==17.0.1
+wheel==0.47.0
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0
diff --git a/finetune/eval/.vllm_ok b/finetune/eval/.vllm_ok
new file mode 100644
index 00000000..e69de29b
diff --git a/finetune/eval/all_results.json b/finetune/eval/all_results.json
new file mode 100644
index 00000000..5e0e6ce2
--- /dev/null
+++ b/finetune/eval/all_results.json
@@ -0,0 +1,262 @@
+[
+ {
+ "name": "qwen25coder7b_base",
+ "exec_ok": 18,
+ "exec_rate": 0.0849,
+ "cd": 0.312,
+ "f05": 0.2312,
+ "f10": 0.3936,
+ "f05_all": 0.0142,
+ "f10_all": 0.0241,
+ "avg_tok": 1040,
+ "train_loss": null,
+ "train_min": 0.0
+ },
+ {
+ "name": "qwen35_9b_base_nothink",
+ "exec_ok": 0,
+ "exec_rate": 0.0,
+ "cd": null,
+ "f05": null,
+ "f10": null,
+ "f05_all": 0.0,
+ "f10_all": 0.0,
+ "avg_tok": 3520,
+ "train_loss": null,
+ "train_min": 0.0
+ },
+ {
+ "name": "qwen35_9b_base_nothink_lenient",
+ "exec_ok": 4,
+ "exec_rate": 0.0189,
+ "cd": 0.2648,
+ "f05": 0.2389,
+ "f10": 0.4858,
+ "f05_all": 0.0045,
+ "f10_all": 0.0092,
+ "avg_tok": 0,
+ "train_loss": null,
+ "train_min": 0.0
+ },
+ {
+ "name": "qwen35_9b_full_v1_ckpt60",
+ "exec_ok": 151,
+ "exec_rate": 0.7123,
+ "cd": 0.1996,
+ "f05": 0.3806,
+ "f10": 0.6036,
+ "f05_all": 0.2693,
+ "f10_all": 0.427,
+ "avg_tok": 874,
+ "train_loss": null,
+ "train_min": 0.0
+ },
+ {
+ "name": "qwen35_9b_lora_v1",
+ "exec_ok": 159,
+ "exec_rate": 0.75,
+ "cd": 0.2012,
+ "f05": 0.3823,
+ "f10": 0.5979,
+ "f05_all": 0.2849,
+ "f10_all": 0.4456,
+ "avg_tok": 774,
+ "train_loss": 0.26390162311376086,
+ "train_min": 30.171525
+ },
+ {
+ "name": "qwen35_9b_lora_v1_T07_s1",
+ "exec_ok": 162,
+ "exec_rate": 0.7642,
+ "cd": 0.2008,
+ "f05": 0.3716,
+ "f10": 0.5925,
+ "f05_all": 0.2787,
+ "f10_all": 0.4444,
+ "avg_tok": 562,
+ "train_loss": null,
+ "train_min": 0.0
+ },
+ {
+ "name": "qwen35_9b_lora_v1_T07_s2",
+ "exec_ok": 141,
+ "exec_rate": 0.6651,
+ "cd": 0.1974,
+ "f05": 0.3822,
+ "f10": 0.6028,
+ "f05_all": 0.2524,
+ "f10_all": 0.3981,
+ "avg_tok": 643,
+ "train_loss": null,
+ "train_min": 0.0
+ },
+ {
+ "name": "qwen35_9b_lora_v1_T07_s3",
+ "exec_ok": 137,
+ "exec_rate": 0.6462,
+ "cd": 0.195,
+ "f05": 0.4005,
+ "f10": 0.6189,
+ "f05_all": 0.2588,
+ "f10_all": 0.3999,
+ "avg_tok": 969,
+ "train_loss": null,
+ "train_min": 0.0
+ },
+ {
+ "name": "qwen35_9b_lora_v1_T07_s4",
+ "exec_ok": 159,
+ "exec_rate": 0.75,
+ "cd": 0.1994,
+ "f05": 0.3862,
+ "f10": 0.6118,
+ "f05_all": 0.286,
+ "f10_all": 0.4531,
+ "avg_tok": 666,
+ "train_loss": null,
+ "train_min": 0.0
+ },
+ {
+ "name": "qwen35_9b_lora_v1_ckpt100",
+ "exec_ok": 168,
+ "exec_rate": 0.7925,
+ "cd": 0.198,
+ "f05": 0.3848,
+ "f10": 0.5995,
+ "f05_all": 0.3013,
+ "f10_all": 0.4694,
+ "avg_tok": 705,
+ "train_loss": 0.26390162311376086,
+ "train_min": 30.171525
+ },
+ {
+ "name": "qwen35_9b_lora_v2_indist",
+ "exec_ok": 131,
+ "exec_rate": 0.6179,
+ "cd": 0.1864,
+ "f05": 0.4112,
+ "f10": 0.6422,
+ "f05_all": 0.2521,
+ "f10_all": 0.3938,
+ "avg_tok": 1360,
+ "train_loss": 0.29489868678725684,
+ "train_min": 37.09996666666667
+ },
+ {
+ "name": "qwen35_9b_lora_v2b_indist16k",
+ "exec_ok": 114,
+ "exec_rate": 0.5377,
+ "cd": 0.1969,
+ "f05": 0.4144,
+ "f10": 0.622,
+ "f05_all": 0.2229,
+ "f10_all": 0.3345,
+ "avg_tok": 2579,
+ "train_loss": 0.34209747883406555,
+ "train_min": 58.625461666666666
+ },
+ {
+ "name": "qwen35_9b_lora_v3_bio",
+ "exec_ok": 156,
+ "exec_rate": 0.7358,
+ "cd": 0.205,
+ "f05": 0.3804,
+ "f10": 0.5968,
+ "f05_all": 0.2781,
+ "f10_all": 0.4363,
+ "avg_tok": 539,
+ "train_loss": 0.202895405292511,
+ "train_min": 24.804971666666667
+ },
+ {
+ "name": "qwen35_9b_lora_v3_bio_ckpt100",
+ "exec_ok": 154,
+ "exec_rate": 0.7264,
+ "cd": 0.2149,
+ "f05": 0.3683,
+ "f10": 0.588,
+ "f05_all": 0.264,
+ "f10_all": 0.4216,
+ "avg_tok": 563,
+ "train_loss": 0.202895405292511,
+ "train_min": 24.804971666666667
+ },
+ {
+ "name": "qwen35_9b_lora_v3_bio_ckpt50",
+ "exec_ok": 152,
+ "exec_rate": 0.717,
+ "cd": 0.211,
+ "f05": 0.3753,
+ "f10": 0.5863,
+ "f05_all": 0.2673,
+ "f10_all": 0.4176,
+ "avg_tok": 523,
+ "train_loss": 0.202895405292511,
+ "train_min": 24.804971666666667
+ },
+ {
+ "name": "qwen35_9b_lora_v4_cq",
+ "exec_ok": 157,
+ "exec_rate": 0.7406,
+ "cd": 0.2008,
+ "f05": 0.3751,
+ "f10": 0.5956,
+ "f05_all": 0.2742,
+ "f10_all": 0.4355,
+ "avg_tok": 694,
+ "train_loss": 0.18613976782018488,
+ "train_min": 67.80998000000001
+ },
+ {
+ "name": "qwen35_9b_lora_v5_instr",
+ "exec_ok": 143,
+ "exec_rate": 0.6745,
+ "cd": 0.1988,
+ "f05": 0.4025,
+ "f10": 0.6094,
+ "f05_all": 0.2696,
+ "f10_all": 0.4082,
+ "avg_tok": 876,
+ "train_loss": 0.26363916967123274,
+ "train_min": 31.889743333333335
+ },
+ {
+ "name": "qwen35_9b_lora_v6_detail",
+ "exec_ok": 139,
+ "exec_rate": 0.6557,
+ "cd": 0.2077,
+ "f05": 0.3688,
+ "f10": 0.5878,
+ "f05_all": 0.2401,
+ "f10_all": 0.3826,
+ "avg_tok": 1128,
+ "train_loss": 0.2620609815304096,
+ "train_min": 33.901199999999996
+ },
+ {
+ "name": "qwen35_9b_lora_v7_ep4",
+ "exec_ok": 147,
+ "exec_rate": 0.6934,
+ "cd": 0.2052,
+ "f05": 0.3799,
+ "f10": 0.6022,
+ "f05_all": 0.2616,
+ "f10_all": 0.4147,
+ "avg_tok": 857,
+ "train_loss": 0.20162874489786134,
+ "train_min": 61.887908333333336
+ },
+ {
+ "name": "qwen35_9b_lora_v8_r16",
+ "exec_ok": 162,
+ "exec_rate": 0.7642,
+ "cd": 0.2096,
+ "f05": 0.3717,
+ "f10": 0.5902,
+ "f05_all": 0.284,
+ "f10_all": 0.451,
+ "avg_tok": 667,
+ "train_loss": 0.31294202780912794,
+ "train_min": 31.807015
+ }
+]
\ No newline at end of file
diff --git a/finetune/eval/analyze_runs.py b/finetune/eval/analyze_runs.py
new file mode 100644
index 00000000..8d6d2edc
--- /dev/null
+++ b/finetune/eval/analyze_runs.py
@@ -0,0 +1,32 @@
+"""Cross-run analysis: per-task success overlap, code length vs success, per-category success. Usage: analyze_runs.py run1 run2 ..."""
+import json, os, sys, collections, numpy as np
+OUT = "/wekafs/ict/hx_624/llm-ft/eval/out"
+runs = sys.argv[1:]
+data = {}
+for r in runs:
+ ex = {json.loads(l)["task"]: json.loads(l) for l in open(f"{OUT}/{r}/exec_results.jsonl")}
+ gens = {json.loads(l)["task"]: json.loads(l) for f in os.listdir(f"{OUT}/{r}") if f.startswith("gens_shard") for l in open(f"{OUT}/{r}/{f}")}
+ met = {json.loads(l)["task"]: json.loads(l) for l in open(f"{OUT}/{r}/metrics.jsonl")} if os.path.exists(f"{OUT}/{r}/metrics.jsonl") else {}
+ data[r] = (ex, gens, met)
+tasks = sorted(next(iter(data.values()))[0].keys())
+print("=== success by output length bucket (tokens) ===")
+for r in runs:
+ ex, gens, met = data[r]; b = collections.defaultdict(lambda: [0, 0])
+ for t in tasks:
+ n = gens.get(t, {}).get("n_new_tokens", 0); k = "<500" if n < 500 else "<1000" if n < 1000 else "<2000" if n < 2000 else "<4000" if n < 4000 else ">=4000"
+ b[k][1] += 1; b[k][0] += ex[t]["status"] == "OK"
+ print(f"{r:32s}", {k: f"{v[0]}/{v[1]}" for k, v in sorted(b.items())})
+print("\n=== pairwise overlap of OK tasks ===")
+oks = {r: {t for t in tasks if data[r][0][t]["status"] == "OK"} for r in runs}
+for i, a in enumerate(runs):
+ for b_ in runs[i+1:]:
+ print(f"{a} ∩ {b_}: {len(oks[a] & oks[b_])} | only {a}: {len(oks[a]-oks[b_])} | only {b_}: {len(oks[b_]-oks[a])}")
+print("\n=== per-task category (first word of task name) success counts for each run ===")
+cats = collections.defaultdict(list)
+for t in tasks: cats[t[:1]].append(t)
+print("(by first letter, just to eyeball)")
+for r in runs:
+ ex = data[r][0]; print(f"{r:32s}", " ".join(f"{c}:{sum(ex[t]['status']=='OK' for t in ts)}/{len(ts)}" for c, ts in sorted(cats.items())))
+print("\n=== best/worst tasks ([email protected]) in", runs[-1], "===")
+met = data[runs[-1]][2]; sc = sorted([(m.get("[email protected]", -1), t) for t, m in met.items() if "[email protected]" in m], reverse=True)
+print("best:", sc[:8]); print("worst:", sc[-8:])
diff --git a/finetune/eval/best_of_n.sh b/finetune/eval/best_of_n.sh
new file mode 100755
index 00000000..f244e922
--- /dev/null
+++ b/finetune/eval/best_of_n.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+# usage: GPUS=0 eval/best_of_n.sh <model_dir> <out_prefix> <N> [temperature]
+# N sampled generations (different seeds) -> exec each -> pass@N (union) via eval/pass_at_n.py
+set -uo pipefail
+MODEL=$(readlink -f "$1"); PFX=$2; N=$3; T=${4:-0.7}
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=${GPUS:-0}
+cd /wekafs/ict/hx_624/llm-ft
+for i in $(seq 1 $N); do
+ OUT=eval/out/${PFX}_s$i; mkdir -p $OUT
+ python eval/generate_vllm.py --model $MODEL --prompts data/sft_v1/bench_prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 6144 --temperature $T --seed $i 2>&1 | grep -E "gen-vllm|Error" | tail -2
+ cp $OUT/gens_shard0.jsonl $OUT/gens.jsonl
+done
+conda activate llmft
+for i in $(seq 1 $N); do OUT=eval/out/${PFX}_s$i; python eval/run_bench.py --gen_dir $OUT --workers 32 --timeout 300 > $OUT/exec.log 2>&1; python eval/metrics.py --gen_dir $OUT > $OUT/metrics.log 2>&1; grep "status counts" $OUT/exec.log; done
+python eval/pass_at_n.py $PFX $N
+echo BEST_OF_N_DONE
diff --git a/finetune/eval/blender_dialect_eval.py b/finetune/eval/blender_dialect_eval.py
new file mode 100644
index 00000000..0b2a257c
--- /dev/null
+++ b/finetune/eval/blender_dialect_eval.py
@@ -0,0 +1,44 @@
+"""Held-out Blender test (bioinspired3d-style): execute generated & reference scripts in Blender, compare meshes."""
+import argparse, json, os, re, subprocess, sys, numpy as np
+from concurrent.futures import ThreadPoolExecutor
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from metrics import load_points, chamfer_f, rot_y
+BLENDER = "/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; RUNNER = os.path.join(os.path.dirname(os.path.abspath(__file__)), "blender_runner.py")
+ap = argparse.ArgumentParser(); ap.add_argument("--test", required=True); ap.add_argument("--gen_dir", required=True); ap.add_argument("--workers", type=int, default=32); a = ap.parse_args()
+tests = [json.loads(l) for l in open(a.test)]; ref_cache = os.path.join(os.path.dirname(a.test), "ref_exec_blender")
+def code_of(msg):
+ m = re.findall(r"```[a-zA-Z]*\n(.*?)```", msg, flags=re.S); return (max(m, key=len) if m else msg).strip()
+def run_blender(code, d):
+ d = os.path.abspath(d); os.makedirs(d, exist_ok=True); sp = os.path.join(d, "code.py"); glb = os.path.join(d, "out.glb"); rp = os.path.join(d, "exec.json")
+ if not os.path.exists(sp): open(sp, "w").write(code)
+ try:
+ subprocess.run([BLENDER, "-b", "--factory-startup", "-noaudio", "--python", RUNNER, "--", "--script", sp, "--out", glb, "--report", rp], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=180, env={**os.environ, "HOME": "/tmp"})
+ r = json.load(open(rp)) if os.path.exists(rp) else {"status": "CRASH"}
+ except subprocess.TimeoutExpired: r = {"status": "TIMEOUT"}
+ r["mesh"] = glb if r.get("status") == "OK" else None; return r
+def ref_job(t):
+ tid = t["id"].replace("/", "__"); d = os.path.join(ref_cache, tid)
+ if os.path.exists(os.path.join(d, "rep.json")): return tid, json.load(open(os.path.join(d, "rep.json")))
+ rep = run_blender(code_of(t["messages"][-1]["content"]), d); json.dump(rep, open(os.path.join(d, "rep.json"), "w")); return tid, rep
+def gen_job(t):
+ tid = t["id"].replace("/", "__"); d = os.path.join(a.gen_dir, tid)
+ if not os.path.exists(os.path.join(d, "code.py")): return tid, {"status": "MISSING", "mesh": None}
+ return tid, run_blender(None, d)
+with ThreadPoolExecutor(a.workers) as ex: refs = dict(ex.map(ref_job, tests)); gens = dict(ex.map(gen_job, tests))
+rows = []
+for t in tests:
+ tid = t["id"].replace("/", "__"); r = {"id": tid, "ref": refs[tid].get("status"), "gen": gens[tid].get("status"), "gen_error": (gens[tid].get("error") or "")[:200]}
+ if r["ref"] == "OK" and r["gen"] == "OK":
+ try:
+ pr = load_points(refs[tid]["mesh"], 5000); pg = load_points(gens[tid]["mesh"], 5000)
+ if pr is not None and pg is not None:
+ best = min((chamfer_f(rot_y(pg, k), pr) for k in range(4)), key=lambda x: x[0]); r["chamfer"] = best[0]; r.update(best[1])
+ except Exception as e: r["metric_error"] = str(e)[:100]
+ rows.append(r)
+n = len(rows); sc = [r for r in rows if "[email protected]" in r]
+summary = {"dialect": "blender_heldout", "n": n, "ref_exec_ok": sum(r["ref"] == "OK" for r in rows), "gen_exec_ok": sum(r["gen"] == "OK" for r in rows), "gen_exec_rate": round(sum(r["gen"] == "OK" for r in rows) / n, 4),
+ "n_scored": len(sc), "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None, "[email protected]_mean_all": round(float(sum(r.get("[email protected]", 0) for r in rows) / n), 4), "chamfer_mean_scored": round(float(np.mean([r["chamfer"] for r in sc])), 4) if sc else None}
+os.makedirs(a.gen_dir, exist_ok=True)
+with open(os.path.join(a.gen_dir, "dialect_metrics.jsonl"), "w") as f:
+ for r in rows: f.write(json.dumps(r) + "\n")
+json.dump(summary, open(os.path.join(a.gen_dir, "summary.json"), "w"), indent=2); print(json.dumps(summary))
diff --git a/finetune/eval/blender_runner.py b/finetune/eval/blender_runner.py
new file mode 100644
index 00000000..1da13185
--- /dev/null
+++ b/finetune/eval/blender_runner.py
@@ -0,0 +1,72 @@
+"""Run ONE generated Blender-Python script headlessly inside Blender and export a GLB + JSON report.
+
+Usage (inside blender):
+ blender -b --factory-startup --python blender_runner.py -- --script gen.py --out out.glb --report out.json
+"""
+import sys, json, time, traceback, argparse, runpy, os, io, contextlib
+
+def parse():
+ argv = sys.argv[sys.argv.index("--") + 1:] if "--" in sys.argv else []
+ ap = argparse.ArgumentParser()
+ ap.add_argument("--script", required=True)
+ ap.add_argument("--out", required=True)
+ ap.add_argument("--report", required=True)
+ return ap.parse_args(argv)
+
+def main():
+ args = parse()
+ import bpy
+ t0 = time.time()
+ rep = {"script": args.script, "out_glb": args.out, "status": "FAIL", "error": None,
+ "n_meshes": 0, "n_verts": 0, "n_faces": 0, "size_kb": 0.0, "latency_s": 0.0}
+ # factory startup scene (keeps the default 'Collection' in the view layer, like the official export),
+ # but remove the default Cube/Light/Camera so an empty generation cannot pass as "OK"
+ # NOTE: do not call read_factory_settings() here (leaves bpy.context.view_layer stale); rely on --factory-startup
+ for ob in list(bpy.data.objects):
+ bpy.data.objects.remove(ob, do_unlink=True)
+ for coll_name in ("meshes", "materials", "lights", "cameras", "curves"):
+ coll = getattr(bpy.data, coll_name)
+ for blk in list(coll):
+ if blk.users == 0: coll.remove(blk)
+ bpy.context.view_layer.update()
+ log = io.StringIO()
+ try:
+ # Make the script dir importable & chdir so relative asset paths resolve
+ sdir = os.path.dirname(os.path.abspath(args.script))
+ sys.path.insert(0, sdir); os.chdir(sdir)
+ with contextlib.redirect_stdout(log):
+ runpy.run_path(args.script, run_name="__main__")
+ # collect meshes (evaluate modifiers)
+ bpy.context.view_layer.update()
+ deps = bpy.context.evaluated_depsgraph_get()
+ vl_objs = list(bpy.context.view_layer.objects) # only objects actually in the scene/view layer count
+ nm = nv = nf = 0
+ for ob in vl_objs:
+ if ob.type == "MESH":
+ ev = ob.evaluated_get(deps)
+ me = ev.to_mesh()
+ if me is not None:
+ nm += 1; nv += len(me.vertices); nf += len(me.polygons)
+ ev.to_mesh_clear()
+ rep.update(n_meshes=nm, n_verts=nv, n_faces=nf)
+ if nm == 0 or nv == 0:
+ rep["status"] = "EMPTY"; rep["error"] = "no mesh geometry produced"
+ else:
+ for ob in vl_objs:
+ try: ob.select_set(ob.type == "MESH" and not ob.hide_viewport and not ob.hide_render)
+ except Exception: pass
+ bpy.ops.export_scene.gltf(filepath=args.out, export_format="GLB", use_selection=True,
+ export_apply=True, export_materials="EXPORT", export_yup=True)
+ rep["size_kb"] = round(os.path.getsize(args.out) / 1024, 1)
+ rep["status"] = "OK"
+ except BaseException as e: # noqa (catch SystemExit too)
+ tb = traceback.format_exc()
+ rep["error"] = (type(e).__name__ + ": " + str(e))[:2000]
+ rep["traceback"] = tb[-3000:]
+ rep["latency_s"] = round(time.time() - t0, 2)
+ rep["stdout_tail"] = log.getvalue()[-1500:]
+ with open(args.report, "w") as f:
+ json.dump(rep, f, indent=2)
+ print("RUNNER_STATUS", rep["status"], rep["error"] or "")
+
+main()
diff --git a/finetune/eval/compare.py b/finetune/eval/compare.py
new file mode 100644
index 00000000..4971b0e3
--- /dev/null
+++ b/finetune/eval/compare.py
@@ -0,0 +1,15 @@
+"""Tabulate eval/out/*/summary.json (+ generation stats) into one table."""
+import glob, json, os, sys
+rows = []
+for d in sorted(glob.glob("/wekafs/ict/hx_624/llm-ft/eval/out/*/")):
+ sp = os.path.join(d, "summary.json")
+ if not os.path.exists(sp): continue
+ s = json.load(open(sp))
+ gens = [json.loads(l) for f in glob.glob(os.path.join(d, "gens_shard*.jsonl")) for l in open(f)]
+ ntok = sum(g["n_new_tokens"] for g in gens) / max(len(gens), 1)
+ fin = sum(g["finished"] for g in gens) / max(len(gens), 1)
+ rows.append((os.path.basename(d.rstrip("/")), s["exec_ok"], s["n_tasks"], s["exec_ok_rate"], s["chamfer_mean_scored"], s["chamfer_median_scored"], s["[email protected]_mean_scored"], s["[email protected]_mean_scored"], s["[email protected]_mean_all(fail=0)"], s["[email protected]_mean_all(fail=0)"], ntok, fin))
+hdr = ("run", "exec_ok", "n", "ok_rate", "CD_mean", "CD_med", "[email protected](ok)", "[email protected](ok)", "[email protected](all)", "[email protected](all)", "avg_tok", "finished")
+print("| " + " | ".join(hdr) + " |"); print("|" + "---|" * len(hdr))
+for r in rows:
+ print("| " + " | ".join(f"{x:.3f}" if isinstance(x, float) else str(x) for x in r) + " |")
diff --git a/finetune/eval/dialect_eval.py b/finetune/eval/dialect_eval.py
new file mode 100644
index 00000000..92f87a02
--- /dev/null
+++ b/finetune/eval/dialect_eval.py
@@ -0,0 +1,43 @@
+"""Evaluate a model on a non-Blender dialect test set: generate (vLLM, done separately) -> execute generated & reference -> geometry metrics vs reference mesh.
+usage: dialect_eval.py --dialect cadquery --test data/multidialect/cadquery/test.jsonl --gen_dir <dir with <id>/code.py> [--workers 32]"""
+import argparse, json, os, re, sys, numpy as np
+from concurrent.futures import ThreadPoolExecutor
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from dialect_runners import RUNNERS
+from metrics import load_points, chamfer_f, rot_y
+ap = argparse.ArgumentParser(); ap.add_argument("--dialect", required=True); ap.add_argument("--test", required=True); ap.add_argument("--gen_dir", required=True); ap.add_argument("--workers", type=int, default=32); ap.add_argument("--ref_cache", default=None)
+a = ap.parse_args()
+run = RUNNERS[a.dialect]
+tests = [json.loads(l) for l in open(a.test)]
+ref_cache = a.ref_cache or os.path.join(os.path.dirname(a.test), f"ref_exec_{a.dialect}")
+def code_of(msg):
+ m = re.findall(r"```[a-zA-Z]*\n(.*?)```", msg, flags=re.S); return (max(m, key=len) if m else msg).strip()
+def ref_job(t):
+ tid = t["id"].replace("/", "__"); d = os.path.join(ref_cache, tid)
+ if os.path.exists(os.path.join(d, "rep.json")): return tid, json.load(open(os.path.join(d, "rep.json")))
+ rep = run(code_of(t["messages"][-1]["content"]), d); json.dump(rep, open(os.path.join(d, "rep.json"), "w")); return tid, rep
+def gen_job(t):
+ tid = t["id"].replace("/", "__"); d = os.path.join(a.gen_dir, tid); cp = os.path.join(d, "code.py")
+ if not os.path.exists(cp): return tid, {"status": "MISSING", "error": "no generation", "mesh": None}
+ rep = run(open(cp).read(), os.path.join(d, "exec")); return tid, rep
+with ThreadPoolExecutor(a.workers) as ex:
+ refs = dict(ex.map(ref_job, tests)); gens = dict(ex.map(gen_job, tests))
+rows = []; np.random.seed(0)
+for t in tests:
+ tid = t["id"].replace("/", "__"); r = {"id": tid, "ref": refs[tid]["status"], "gen": gens[tid]["status"], "gen_error": (gens[tid].get("error") or "")[:200]}
+ if a.dialect != "glsl" and refs[tid]["status"] == "OK" and gens[tid]["status"] == "OK":
+ try:
+ pr = load_points(refs[tid]["mesh"], 5000); pg = load_points(gens[tid]["mesh"], 5000)
+ if pr is not None and pg is not None:
+ best = min((chamfer_f(rot_y(pg, k), pr) for k in range(4)), key=lambda x: x[0]); r["chamfer"] = best[0]; r.update(best[1])
+ except Exception as e: r["metric_error"] = str(e)[:100]
+ rows.append(r)
+n = len(rows); ref_ok = sum(r["ref"] == "OK" for r in rows); gen_ok = sum(r["gen"] == "OK" for r in rows); sc = [r for r in rows if "[email protected]" in r]
+summary = {"dialect": a.dialect, "n": n, "ref_exec_ok": ref_ok, "gen_exec_ok": gen_ok, "gen_exec_rate": round(gen_ok / n, 4), "n_scored": len(sc),
+ "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None, "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None,
+ "[email protected]_mean_all": round(float(sum(r.get("[email protected]", 0) for r in rows) / n), 4), "chamfer_mean_scored": round(float(np.mean([r["chamfer"] for r in sc])), 4) if sc else None}
+os.makedirs(a.gen_dir, exist_ok=True)
+with open(os.path.join(a.gen_dir, "dialect_metrics.jsonl"), "w") as f:
+ for r in rows: f.write(json.dumps(r) + "\n")
+json.dump(summary, open(os.path.join(a.gen_dir, "summary.json"), "w"), indent=2); print(json.dumps(summary))
+from collections import Counter; print("gen failure modes:", Counter(re.sub(r"'[^']*'", "'<x>'", (r["gen_error"] or r["gen"]).split("\n")[0][:70]) for r in rows if r["gen"] != "OK").most_common(8))
diff --git a/finetune/eval/dialect_runners.py b/finetune/eval/dialect_runners.py
new file mode 100644
index 00000000..d58dfec0
--- /dev/null
+++ b/finetune/eval/dialect_runners.py
@@ -0,0 +1,101 @@
+"""Executors for non-Blender dialects. Each run_* returns dict(status, error, mesh_path or None, latency_s)."""
+import os, subprocess, sys, time, json, tempfile, re
+OPENSCAD = "/wekafs/ict/hx_624/tools/openscad/squashfs-root/AppRun"
+GLSLANG = "/wekafs/ict/hx_624/anaconda3/envs/llmft/bin/glslangValidator"
+PY = "/wekafs/ict/hx_624/anaconda3/envs/llmft/bin/python"
+CQ_RUNNER = r'''
+import sys, json, runpy, traceback, os
+os.environ.setdefault("OMP_NUM_THREADS","1")
+script, out_stl = sys.argv[1], sys.argv[2]
+rep = {"status":"FAIL","error":None}
+try:
+ import cadquery as cq
+ sys.argv = [script, out_stl]
+ _orig_export = cq.exporters.export
+ captured = []
+ def _capture(o, *a, **k): captured.append(o)
+ # neutralise export calls (the scripts export STEP/STL to arbitrary paths) but remember the exported object
+ for modname in ("cadquery.occ_impl.exporters", "cadquery.exporters"):
+ try:
+ import importlib; _m = importlib.import_module(modname); _m.export = _capture
+ except Exception: pass
+ try: cq.exporters.export = _capture
+ except Exception: pass
+ g = runpy.run_path(script, run_name="__main__", init_globals={"show_object": _capture, "debug": _capture})
+ obj = None
+ for c in reversed(captured):
+ if isinstance(c, (cq.Workplane, cq.Shape, cq.Assembly)): obj = c; break
+ if obj is None:
+ for name in ("result", "final", "model", "part", "solid", "shape", "body", "assembly", "assy", "obj", "res"):
+ if isinstance(g.get(name), (cq.Workplane, cq.Shape, cq.Assembly)): obj = g[name]; break
+ if obj is None:
+ cands = [v for v in g.values() if isinstance(v, (cq.Workplane, cq.Shape, cq.Assembly))]
+ obj = cands[-1] if cands else None
+ if obj is None: raise RuntimeError("no CadQuery result object found (expected variable `result`)")
+ if isinstance(obj, cq.Assembly): obj = obj.toCompound()
+ if isinstance(obj, cq.Workplane):
+ vals = obj.vals()
+ if not vals: raise RuntimeError("empty workplane")
+ obj = vals[0] if len(vals)==1 else cq.Compound.makeCompound([v for v in vals if isinstance(v, cq.Shape)])
+ if not isinstance(obj, cq.Shape): raise RuntimeError(f"unsupported object {type(obj)}")
+ if obj.Volume() <= 1e-9 and not obj.Faces(): raise RuntimeError("empty shape")
+ _orig_export(obj, out_stl, tolerance=0.01, angularTolerance=0.1)
+ rep["status"]="OK"
+except BaseException as e:
+ rep["error"]=(type(e).__name__+": "+str(e))[:500]; rep["traceback"]=traceback.format_exc()[-1500:]
+print("CQ_REPORT "+json.dumps(rep))
+'''
+def run_cadquery(code, workdir, timeout=90):
+ workdir = os.path.abspath(workdir); os.makedirs(workdir, exist_ok=True); sp = os.path.join(workdir, "script.py"); stl = os.path.join(workdir, "out.stl"); rp = os.path.join(workdir, "runner.py")
+ open(sp, "w").write(code); open(rp, "w").write(CQ_RUNNER)
+ t0 = time.time()
+ try:
+ p = subprocess.run([PY, rp, sp, stl], cwd=workdir, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=timeout, text=True, errors="replace")
+ m = re.search(r"CQ_REPORT (\{.*\})", p.stdout)
+ rep = json.loads(m.group(1)) if m else {"status": "CRASH", "error": p.stdout[-800:]}
+ except subprocess.TimeoutExpired:
+ rep = {"status": "TIMEOUT", "error": f"timeout {timeout}s"}
+ rep["latency_s"] = round(time.time() - t0, 1); rep["mesh"] = stl if rep["status"] == "OK" and os.path.exists(stl) else None
+ if rep["status"] == "OK" and rep["mesh"] is None: rep["status"] = "EMPTY"
+ return rep
+def run_openscad(code, workdir, timeout=120):
+ workdir = os.path.abspath(workdir); os.makedirs(workdir, exist_ok=True); sc = os.path.join(workdir, "model.scad"); stl = os.path.join(workdir, "out.stl")
+ open(sc, "w").write(code); t0 = time.time()
+ try:
+ p = subprocess.run([OPENSCAD, "-o", stl, "--export-format", "binstl", sc], cwd=workdir, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=timeout, text=True, errors="replace",
+ env={**os.environ, "HOME": workdir, "QT_QPA_PLATFORM": "offscreen"})
+ ok = p.returncode == 0 and os.path.exists(stl) and os.path.getsize(stl) > 100
+ rep = {"status": "OK" if ok else ("EMPTY" if p.returncode == 0 else "FAIL"), "error": None if ok else p.stdout[-800:]}
+ except subprocess.TimeoutExpired:
+ rep = {"status": "TIMEOUT", "error": f"timeout {timeout}s"}
+ rep["latency_s"] = round(time.time() - t0, 1); rep["mesh"] = stl if rep["status"] == "OK" else None
+ return rep
+GLSL_PRELUDE = """#version 300 es
+precision highp float; precision highp int;
+uniform vec3 iResolution; uniform float iTime; uniform float iTimeDelta; uniform int iFrame; uniform vec4 iMouse; uniform vec4 iDate; uniform float iSampleRate;
+uniform sampler2D iChannel0; uniform sampler2D iChannel1; uniform sampler2D iChannel2; uniform sampler2D iChannel3;
+uniform vec3 iChannelResolution[4]; uniform float iChannelTime[4];
+out vec4 _outColor;
+"""
+GLSL_MAIN = "\nvoid main(){ vec4 c = vec4(0.0); mainImage(c, gl_FragCoord.xy); _outColor = c; }\n"
+def run_glsl(code, workdir, timeout=30):
+ workdir = os.path.abspath(workdir); os.makedirs(workdir, exist_ok=True); fp = os.path.join(workdir, "shader.frag")
+ # normalise: drop #version / precision lines and re-declarations of Shadertoy uniforms / the out variable (the prelude provides them)
+ import re as _re
+ lines = []
+ for ln in code.splitlines():
+ if _re.match(r"\s*#\s*version\b", ln) or _re.match(r"\s*precision\s+(low|medium|high)p\s+", ln): continue
+ if _re.match(r"\s*uniform\s+.*\b(iResolution|iTime|iTimeDelta|iFrame|iMouse|iDate|iSampleRate|iChannel[0-3]|iChannelResolution|iChannelTime)\b", ln): continue
+ if _re.match(r"\s*out\s+vec4\s+\w+\s*;", ln): continue
+ lines.append(ln)
+ src = "\n".join(lines)
+ if "mainImage" not in src: return {"status": "FAIL", "error": "no mainImage()", "latency_s": 0, "mesh": None}
+ open(fp, "w").write(GLSL_PRELUDE + src + GLSL_MAIN); t0 = time.time()
+ try:
+ p = subprocess.run([GLSLANG, "-S", "frag", fp], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=timeout, text=True, errors="replace")
+ rep = {"status": "OK" if p.returncode == 0 else "FAIL", "error": None if p.returncode == 0 else p.stdout[-600:]}
+ except subprocess.TimeoutExpired:
+ rep = {"status": "TIMEOUT", "error": "timeout"}
+ rep["latency_s"] = round(time.time() - t0, 2); rep["mesh"] = None
+ return rep
+RUNNERS = {"cadquery": run_cadquery, "openscad": run_openscad, "glsl": run_glsl}
diff --git a/finetune/eval/generate.py b/finetune/eval/generate.py
new file mode 100644
index 00000000..cabaf670
--- /dev/null
+++ b/finetune/eval/generate.py
@@ -0,0 +1,81 @@
+"""Batched generation for the 3DCodeBench prompts with a HF causal LM. Writes <out>/<task>/code.py + gens.jsonl"""
+import argparse, json, os, re, time, torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+def extract_code(text):
+ m = re.findall(r"```[a-zA-Z0-9_+.-]*[ \t]*\n(.*?)```", text, flags=re.S) # any language fence (python/glsl/openscad/...)
+ if m:
+ return max(m, key=len).strip("\n")
+ m = re.search(r"```[a-zA-Z0-9_+.-]*[ \t]*\n(.*)$", text, flags=re.S) # unterminated block
+ if m: return m.group(1).strip("\n")
+ return text.strip()
+
+def main():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("--model", required=True)
+ ap.add_argument("--prompts", required=True)
+ ap.add_argument("--out", required=True)
+ ap.add_argument("--batch_size", type=int, default=16)
+ ap.add_argument("--max_new_tokens", type=int, default=6144)
+ ap.add_argument("--temperature", type=float, default=0.0)
+ ap.add_argument("--top_p", type=float, default=0.95)
+ ap.add_argument("--shard", type=int, default=0); ap.add_argument("--num_shards", type=int, default=1)
+ ap.add_argument("--attn", default="auto")
+ ap.add_argument("--limit", type=int, default=None)
+ ap.add_argument("--no_think", action="store_true", help="pass enable_thinking=False to the chat template (Qwen3/3.5)")
+ a = ap.parse_args()
+ os.makedirs(a.out, exist_ok=True)
+ attn = a.attn
+ if attn == "auto":
+ try: import flash_attn; attn = "flash_attention_2" # noqa
+ except Exception: attn = "sdpa"
+ tok = AutoTokenizer.from_pretrained(a.model, padding_side="left")
+ if tok.pad_token is None: tok.pad_token = tok.eos_token
+ try:
+ model = AutoModelForCausalLM.from_pretrained(a.model, dtype=torch.bfloat16, attn_implementation=attn, device_map="cuda")
+ except Exception as e: # multimodal wrappers (e.g. Qwen3.5 = Qwen3_5ForConditionalGeneration)
+ print("[gen] AutoModelForCausalLM failed (%s); falling back to AutoModelForImageTextToText" % str(e)[:200], flush=True)
+ from transformers import AutoModelForImageTextToText
+ model = AutoModelForImageTextToText.from_pretrained(a.model, dtype=torch.bfloat16, attn_implementation=attn, device_map="cuda")
+ model.eval()
+ rows = [json.loads(l) for l in open(a.prompts)]
+ rows = rows[a.shard::a.num_shards]
+ if a.limit: rows = rows[:a.limit]
+ # skip already-done
+ done = set()
+ outf = f"{a.out}/gens_shard{a.shard}.jsonl"
+ if os.path.exists(outf):
+ done = {json.loads(l)["task"] for l in open(outf)}
+ rows = [r for r in rows if r["task"] not in done]
+ print(f"[gen] {len(rows)} prompts to do (shard {a.shard}/{a.num_shards}) attn={attn}", flush=True)
+ # sort by prompt length for efficient batching
+ tmpl_kwargs = {"enable_thinking": False} if a.no_think else {}
+ texts = [tok.apply_chat_template(r["messages"], tokenize=False, add_generation_prompt=True, **tmpl_kwargs) for r in rows]
+ if texts: print("[gen] prompt tail example:", repr(texts[0][-200:]), flush=True)
+ order = sorted(range(len(rows)), key=lambda i: len(texts[i]))
+ gen_kwargs = dict(max_new_tokens=a.max_new_tokens, pad_token_id=tok.pad_token_id)
+ if a.temperature > 0: gen_kwargs.update(do_sample=True, temperature=a.temperature, top_p=a.top_p)
+ else: gen_kwargs.update(do_sample=False)
+ t0 = time.time(); ntok = 0
+ with open(outf, "a") as f:
+ for b in range(0, len(order), a.batch_size):
+ idx = order[b:b + a.batch_size]
+ enc = tok([texts[i] for i in idx], return_tensors="pt", padding=True).to("cuda")
+ with torch.no_grad():
+ out = model.generate(**enc, **gen_kwargs)
+ gen = out[:, enc["input_ids"].shape[1]:]
+ for j, i in enumerate(idx):
+ ids = gen[j]
+ n = int((ids != tok.pad_token_id).sum())
+ text = tok.decode(ids, skip_special_tokens=True)
+ code = extract_code(text)
+ finished = bool((ids == tok.eos_token_id).any()) or any((ids == t).any() for t in tok.all_special_ids if t != tok.pad_token_id) if n < a.max_new_tokens else False
+ task = rows[i]["task"]; d = f"{a.out}/{task}"; os.makedirs(d, exist_ok=True)
+ open(f"{d}/code.py", "w").write(code + "\n"); open(f"{d}/raw.txt", "w").write(text)
+ f.write(json.dumps({"task": task, "n_new_tokens": n, "finished": finished, "has_code_block": "```" in text}) + "\n"); f.flush()
+ ntok += n
+ el = time.time() - t0
+ print(f"[gen] {min(b + a.batch_size, len(order))}/{len(order)} done | {ntok} tok | {ntok/el:.0f} tok/s | {el/60:.1f} min", flush=True)
+
+if __name__ == "__main__":
+ main()
diff --git a/finetune/eval/generate_vllm.py b/finetune/eval/generate_vllm.py
new file mode 100644
index 00000000..28c4b867
--- /dev/null
+++ b/finetune/eval/generate_vllm.py
@@ -0,0 +1,33 @@
+"""vLLM version of generate.py: same outputs (<out>/<task>/code.py + raw.txt, <out>/gens_shard0.jsonl)."""
+import argparse, json, os, re, time, sys
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from generate import extract_code
+
+def main():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("--model", required=True); ap.add_argument("--prompts", required=True); ap.add_argument("--out", required=True)
+ ap.add_argument("--max_new_tokens", type=int, default=6144); ap.add_argument("--temperature", type=float, default=0.0)
+ ap.add_argument("--max_model_len", type=int, default=12288); ap.add_argument("--no_think", action="store_true")
+ ap.add_argument("--tp", type=int, default=1); ap.add_argument("--gpu_mem", type=float, default=0.88); ap.add_argument("--limit", type=int, default=None); ap.add_argument("--seed", type=int, default=0)
+ a = ap.parse_args()
+ from vllm import LLM, SamplingParams
+ os.makedirs(a.out, exist_ok=True)
+ rows = [json.loads(l) for l in open(a.prompts)]
+ if a.limit: rows = rows[:a.limit]
+ llm = LLM(model=a.model, dtype="bfloat16", tensor_parallel_size=a.tp, max_model_len=a.max_model_len, gpu_memory_utilization=a.gpu_mem,
+ enable_prefix_caching=True, limit_mm_per_prompt={"image": 0, "video": 0}, trust_remote_code=True)
+ sp = SamplingParams(temperature=a.temperature, top_p=0.95 if a.temperature > 0 else 1.0, max_tokens=a.max_new_tokens, seed=a.seed)
+ kw = {"chat_template_kwargs": {"enable_thinking": False}} if a.no_think else {}
+ t0 = time.time()
+ outs = llm.chat([r["messages"] for r in rows], sp, use_tqdm=True, **kw)
+ el = time.time() - t0; ntok = 0
+ with open(f"{a.out}/gens_shard0.jsonl", "w") as f:
+ for r, o in zip(rows, outs):
+ text = o.outputs[0].text; n = len(o.outputs[0].token_ids); ntok += n
+ d = f"{a.out}/{r['task']}"; os.makedirs(d, exist_ok=True)
+ open(f"{d}/code.py", "w").write(extract_code(text) + "\n"); open(f"{d}/raw.txt", "w").write(text)
+ f.write(json.dumps({"task": r["task"], "n_new_tokens": n, "finished": o.outputs[0].finish_reason == "stop", "has_code_block": "```" in text}) + "\n")
+ print(f"[gen-vllm] {len(rows)} prompts | {ntok} tok | {ntok/el:.0f} tok/s | {el/60:.1f} min", flush=True)
+
+if __name__ == "__main__":
+ main()
diff --git a/finetune/eval/metrics.py b/finetune/eval/metrics.py
new file mode 100644
index 00000000..b2229a24
--- /dev/null
+++ b/finetune/eval/metrics.py
@@ -0,0 +1,77 @@
+"""Geometry metrics between generated GLB and the 3DCodeBench ground-truth GLB.
+Normalisation: center at bbox center, scale so max bbox extent = 1. Metrics: symmetric Chamfer-L2 distance (mean NN distance),
+F-score @0.05 / @0.1, best over 4 yaw rotations (objects may face any direction). Writes <gen_dir>/metrics.jsonl + summary.json"""
+import argparse, json, os, numpy as np
+import trimesh
+from scipy.spatial import cKDTree
+
+BENCH = "/wekafs/ict/hx_624/data/3dcodebench/data"
+
+def load_points(path, n):
+ m = trimesh.load(path, force="mesh")
+ if isinstance(m, trimesh.Scene): m = trimesh.util.concatenate(list(m.geometry.values()))
+ if m.is_empty or len(m.faces) == 0: return None
+ v = m.vertices
+ c = (v.max(0) + v.min(0)) / 2; s = (v.max(0) - v.min(0)).max()
+ if s <= 0: return None
+ m.apply_translation(-c); m.apply_scale(1.0 / s)
+ pts, _ = trimesh.sample.sample_surface(m, n)
+ return np.asarray(pts)
+
+def chamfer_f(a, b, taus=(0.05, 0.1)):
+ ta, tb = cKDTree(a), cKDTree(b)
+ da, _ = tb.query(a); db, _ = ta.query(b)
+ cd = float(da.mean() + db.mean())
+ fs = {}
+ for t in taus:
+ p = float((da < t).mean()); r = float((db < t).mean())
+ fs[f"f@{t}"] = 2 * p * r / (p + r) if p + r > 0 else 0.0
+ return cd, fs
+
+def rot_y(p, k):
+ th = np.pi / 2 * k; c, s = np.cos(th), np.sin(th)
+ R = np.array([[c, 0, s], [0, 1, 0], [-s, 0, c]])
+ return p @ R.T
+
+def main():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("--gen_dir", required=True)
+ ap.add_argument("--n_points", type=int, default=10000)
+ a = ap.parse_args()
+ np.random.seed(0)
+ ex = {json.loads(l)["task"]: json.loads(l) for l in open(os.path.join(a.gen_dir, "exec_results.jsonl"))}
+ tasks = sorted(os.listdir(BENCH))
+ rows = []
+ for t in tasks:
+ r = {"task": t, "exec": ex.get(t, {}).get("status", "MISSING")}
+ gen_glb = os.path.join(a.gen_dir, t, "out.glb"); gt_glb = os.path.join(BENCH, t, "glb", f"{t}.glb")
+ if r["exec"] == "OK" and os.path.exists(gen_glb) and os.path.exists(gt_glb):
+ try:
+ pg = load_points(gen_glb, a.n_points); pt = load_points(gt_glb, a.n_points)
+ if pg is not None and pt is not None:
+ best = None
+ for k in range(4):
+ cd, fs = chamfer_f(rot_y(pg, k), pt)
+ if best is None or cd < best[0]: best = (cd, fs)
+ r.update(chamfer=best[0], **best[1])
+ except Exception as e:
+ r["metric_error"] = str(e)[:200]
+ rows.append(r)
+ with open(os.path.join(a.gen_dir, "metrics.jsonl"), "w") as f:
+ for r in rows: f.write(json.dumps(r) + "\n")
+ n = len(rows); ok = [r for r in rows if r["exec"] == "OK"]; sc = [r for r in rows if "chamfer" in r]
+ # Failed / empty generations get the worst score for the aggregate (chamfer: mean of worst-10 for penalty; f: 0)
+ summary = {
+ "n_tasks": n, "exec_ok": len(ok), "exec_ok_rate": round(len(ok) / n, 4), "n_scored": len(sc),
+ "chamfer_mean_scored": round(float(np.mean([r["chamfer"] for r in sc])), 4) if sc else None,
+ "chamfer_median_scored": round(float(np.median([r["chamfer"] for r in sc])), 4) if sc else None,
+ "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None,
+ "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None,
+ "[email protected]_mean_all(fail=0)": round(float(sum(r.get("[email protected]", 0) for r in rows) / n), 4),
+ "[email protected]_mean_all(fail=0)": round(float(sum(r.get("[email protected]", 0) for r in rows) / n), 4),
+ }
+ json.dump(summary, open(os.path.join(a.gen_dir, "summary.json"), "w"), indent=2)
+ print(json.dumps(summary, indent=2))
+
+if __name__ == "__main__":
+ main()
diff --git a/finetune/eval/pass_at_n.py b/finetune/eval/pass_at_n.py
new file mode 100644
index 00000000..fde16843
--- /dev/null
+++ b/finetune/eval/pass_at_n.py
@@ -0,0 +1,10 @@
+import json, sys
+pfx, n = sys.argv[1], int(sys.argv[2])
+ok = {}; best = {}
+for i in range(1, n + 1):
+ for l in open(f"/wekafs/ict/hx_624/llm-ft/eval/out/{pfx}_s{i}/metrics.jsonl"):
+ r = json.loads(l); t = r["task"]
+ ok.setdefault(t, 0); ok[t] += r["exec"] == "OK"
+ if "[email protected]" in r: best[t] = max(best.get(t, 0), r["[email protected]"])
+T = len(ok)
+print(f"pass@1 (avg over samples): {sum(ok.values())/(n*T):.3f} | pass@{n} (any sample OK): {sum(v>0 for v in ok.values())/T:.3f} | best-of-{n} [email protected] (all, fail=0): {sum(best.values())/T:.3f}")
diff --git a/finetune/eval/reextract.py b/finetune/eval/reextract.py
new file mode 100644
index 00000000..1fdb3789
--- /dev/null
+++ b/finetune/eval/reextract.py
@@ -0,0 +1,9 @@
+"""Re-extract code.py from raw.txt in a generation dir (after extractor fixes)."""
+import sys, os, glob
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from generate import extract_code
+for d in sys.argv[1:]:
+ n=0
+ for raw in glob.glob(os.path.join(d, "*", "raw.txt")):
+ code = extract_code(open(raw).read()); open(os.path.join(os.path.dirname(raw), "code.py"), "w").write(code + "\n"); n+=1
+ print(d, "re-extracted", n)
diff --git a/finetune/eval/render_glb.py b/finetune/eval/render_glb.py
new file mode 100644
index 00000000..21fc6e21
--- /dev/null
+++ b/finetune/eval/render_glb.py
@@ -0,0 +1,29 @@
+"""Blender headless: render a GLB to PNG with an auto-framed 3/4 camera (workbench, solid shading).
+usage: blender -b --factory-startup -noaudio --python eval/render_glb.py -- --glb in.glb --out out.png [--size 320]"""
+import bpy, sys, math, argparse
+from mathutils import Vector
+argv = sys.argv[sys.argv.index("--")+1:] if "--" in sys.argv else []
+ap = argparse.ArgumentParser(); ap.add_argument("--glb"); ap.add_argument("--out"); ap.add_argument("--size", type=int, default=320); a = ap.parse_args(argv)
+bpy.ops.wm.read_factory_settings(use_empty=True)
+bpy.ops.import_scene.gltf(filepath=a.glb)
+meshes = [o for o in bpy.context.scene.objects if o.type == "MESH"]
+if not meshes: raise SystemExit("no mesh")
+# bounds
+pts = [o.matrix_world @ Vector(c) for o in meshes for c in o.bound_box]
+mn = Vector((min(p.x for p in pts), min(p.y for p in pts), min(p.z for p in pts))); mx = Vector((max(p.x for p in pts), max(p.y for p in pts), max(p.z for p in pts)))
+center = (mn + mx) / 2; radius = max((mx - mn).length / 2, 1e-3)
+cam_data = bpy.data.cameras.new("cam"); cam = bpy.data.objects.new("cam", cam_data); bpy.context.scene.collection.objects.link(cam)
+direction = Vector((1.0, -1.2, 0.8)).normalized(); cam.location = center + direction * radius * 2.6
+cam.rotation_euler = (center - cam.location).to_track_quat("-Z", "Y").to_euler(); cam_data.lens = 40; cam_data.clip_end = radius * 100
+bpy.context.scene.camera = cam
+sc = bpy.context.scene; sc.render.engine = "CYCLES"; sc.cycles.device = "CPU"; sc.cycles.samples = 32; sc.cycles.use_denoising = False
+sc.render.resolution_x = a.size; sc.render.resolution_y = a.size; sc.render.film_transparent = False
+w = bpy.data.worlds.new("w"); w.use_nodes = True; bg = w.node_tree.nodes["Background"]; bg.inputs[0].default_value = (0.92, 0.92, 0.92, 1); bg.inputs[1].default_value = 1.0; sc.world = w
+sun_d = bpy.data.lights.new("sun", "SUN"); sun_d.energy = 3.0; sun = bpy.data.objects.new("sun", sun_d); sc.collection.objects.link(sun)
+sun.rotation_euler = (math.radians(45), math.radians(20), math.radians(30))
+import random; random.seed(0)
+for o in meshes:
+ mat = bpy.data.materials.new(f"m_{o.name}"); mat.use_nodes = True; bsdf = mat.node_tree.nodes.get("Principled BSDF")
+ if bsdf: bsdf.inputs["Base Color"].default_value = (random.uniform(0.25, 0.85), random.uniform(0.25, 0.85), random.uniform(0.25, 0.85), 1); bsdf.inputs["Roughness"].default_value = 0.6
+ o.data.materials.clear(); o.data.materials.append(mat)
+sc.render.filepath = a.out; bpy.ops.render.render(write_still=True)
diff --git a/finetune/eval/results_table.py b/finetune/eval/results_table.py
new file mode 100644
index 00000000..db0d0176
--- /dev/null
+++ b/finetune/eval/results_table.py
@@ -0,0 +1,20 @@
+"""Collect all eval/out/*/summary.json + training stats into a single JSON (for the final report)."""
+import glob, json, os
+OUT = "/wekafs/ict/hx_624/llm-ft/eval/out"; RUNS = "/wekafs/ict/hx_624/llm-ft/runs"
+rows = []
+for d in sorted(glob.glob(f"{OUT}/*/")):
+ name = os.path.basename(d.rstrip("/"))
+ if name.startswith("_"): continue
+ sp = os.path.join(d, "summary.json")
+ if not os.path.exists(sp): continue
+ s = json.load(open(sp))
+ gens = [json.loads(l) for f in glob.glob(os.path.join(d, "gens_shard*.jsonl")) for l in open(f)]
+ avg_tok = sum(g["n_new_tokens"] for g in gens) / max(len(gens), 1)
+ run = name.replace("qwen35_9b_lora_", "lf_qwen35_9b_lora_").replace("qwen35_9b_full_v1", "lf_qwen35_9b_full_v1")
+ tr = {}
+ trp = os.path.join(RUNS, run.split("_ckpt")[0], "train_results.json")
+ if os.path.exists(trp): tr = json.load(open(trp))
+ rows.append(dict(name=name, exec_ok=s["exec_ok"], exec_rate=s["exec_ok_rate"], cd=s["chamfer_mean_scored"], f05=s["[email protected]_mean_scored"], f10=s["[email protected]_mean_scored"],
+ f05_all=s["[email protected]_mean_all(fail=0)"], f10_all=s["[email protected]_mean_all(fail=0)"], avg_tok=round(avg_tok), train_loss=tr.get("train_loss"), train_min=(tr.get("train_runtime") or 0) / 60))
+json.dump(rows, open("/wekafs/ict/hx_624/llm-ft/eval/all_results.json", "w"), indent=1)
+for r in rows: print(f"{r['name']:36s} exec={r['exec_ok']:3d} ({r['exec_rate']*100:4.1f}%) CD={r['cd'] if r['cd'] is None else round(r['cd'],3)} F05={r['f05'] if r['f05'] is None else round(r['f05'],3)} F05all={round(r['f05_all'],3)} tok={r['avg_tok']} loss={r['train_loss'] if r['train_loss'] is None else round(r['train_loss'],3)} {r['train_min']:.0f}min")
diff --git a/finetune/eval/run_bench.py b/finetune/eval/run_bench.py
new file mode 100644
index 00000000..ad02c77a
--- /dev/null
+++ b/finetune/eval/run_bench.py
@@ -0,0 +1,48 @@
+"""Execute every <gen_dir>/<task>/code.py in headless Blender (parallel), export GLB, collect reports -> <gen_dir>/exec_results.jsonl"""
+import argparse, json, os, subprocess, sys, time
+from concurrent.futures import ThreadPoolExecutor, as_completed
+BLENDER = "/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"
+RUNNER = os.path.join(os.path.dirname(os.path.abspath(__file__)), "blender_runner.py")
+
+def run_one(task_dir, timeout):
+ task = os.path.basename(task_dir)
+ script = os.path.join(task_dir, "code.py"); glb = os.path.join(task_dir, "out.glb"); rep = os.path.join(task_dir, "exec.json")
+ if os.path.exists(glb): os.remove(glb)
+ cmd = [BLENDER, "-b", "--factory-startup", "-noaudio", "--python", RUNNER, "--", "--script", script, "--out", glb, "--report", rep]
+ t0 = time.time()
+ try:
+ p = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=timeout, text=True, errors="replace",
+ env={**os.environ, "HOME": "/tmp", "XDG_CONFIG_HOME": "/tmp/.bcfg"})
+ if os.path.exists(rep):
+ r = json.load(open(rep))
+ else:
+ r = {"status": "CRASH", "error": p.stdout[-1500:], "latency_s": round(time.time() - t0, 1)}
+ except subprocess.TimeoutExpired:
+ r = {"status": "TIMEOUT", "error": f"timeout {timeout}s", "latency_s": timeout}
+ r["task"] = task
+ return r
+
+def main():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("--gen_dir", required=True)
+ ap.add_argument("--workers", type=int, default=16)
+ ap.add_argument("--timeout", type=int, default=300)
+ a = ap.parse_args()
+ a.gen_dir = os.path.abspath(a.gen_dir) # the Blender runner chdir()s into the script dir -> all paths must be absolute
+ tasks = sorted(d for d in os.listdir(a.gen_dir) if os.path.isfile(os.path.join(a.gen_dir, d, "code.py")))
+ print(f"[exec] {len(tasks)} scripts, {a.workers} workers", flush=True)
+ res = []
+ with ThreadPoolExecutor(a.workers) as ex:
+ futs = {ex.submit(run_one, os.path.join(a.gen_dir, t), a.timeout): t for t in tasks}
+ for i, fu in enumerate(as_completed(futs)):
+ r = fu.result(); res.append(r)
+ if (i + 1) % 20 == 0: print(f"[exec] {i+1}/{len(tasks)}", flush=True)
+ res.sort(key=lambda r: r["task"])
+ with open(os.path.join(a.gen_dir, "exec_results.jsonl"), "w") as f:
+ for r in res: f.write(json.dumps(r) + "\n")
+ from collections import Counter
+ c = Counter(r["status"] for r in res)
+ print("[exec] status counts:", dict(c), f"| OK rate = {c.get('OK',0)}/{len(res)} = {100*c.get('OK',0)/max(len(res),1):.1f}%")
+
+if __name__ == "__main__":
+ main()
diff --git a/finetune/eval/run_dialect_eval.sh b/finetune/eval/run_dialect_eval.sh
new file mode 100755
index 00000000..99df059e
--- /dev/null
+++ b/finetune/eval/run_dialect_eval.sh
@@ -0,0 +1,27 @@
+#!/bin/bash
+# usage: GPUS=2 eval/run_dialect_eval.sh <model_dir> <name> [dialects...] -> eval/out/md_<name>_<dialect>/summary.json
+set -uo pipefail; MODEL=$(readlink -f "$1"); NAME=$2; shift 2; DIALECTS=${@:-cadquery openscad glsl blender}
+cd /wekafs/ict/hx_624/llm-ft; source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=${GPUS:-0}
+for D in $DIALECTS; do
+ OUT=eval/out/md_${NAME}_$D; mkdir -p $OUT
+ python - "$D" "$OUT" <<PY
+import json,sys
+D,OUT=sys.argv[1],sys.argv[2]
+with open(f"{OUT}/prompts.jsonl","w") as f:
+ for l in open(f"data/multidialect/{D}/test.jsonl"):
+ r=json.loads(l); f.write(json.dumps({"task": r["id"].replace("/","__"), "messages": r["messages"][:2]})+"\n")
+PY
+ python eval/generate_vllm.py --model $MODEL --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 2>&1 | grep -E "gen-vllm|Error" | tail -2
+done
+conda activate llmft
+for D in $DIALECTS; do
+ OUT=eval/out/md_${NAME}_$D
+ if [ "$D" = "blender" ]; then
+ # blender held-out test: run in Blender and compare to reference via the same exec harness (reference meshes cached)
+ python eval/blender_dialect_eval.py --test data/multidialect/blender/test.jsonl --gen_dir $OUT --workers 32 2>&1 | tail -2
+ else
+ python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir $OUT --workers 32 2>&1 | tail -2
+ fi
+done
+echo "[md-eval $NAME] DONE"
diff --git a/finetune/eval/run_eval.sh b/finetune/eval/run_eval.sh
new file mode 100755
index 00000000..2834a350
--- /dev/null
+++ b/finetune/eval/run_eval.sh
@@ -0,0 +1,22 @@
+#!/bin/bash
+# End-to-end 3DCodeBench eval: generate (sharded over GPUS) -> execute in Blender -> geometry metrics
+# usage: GPUS=0,1 ./run_eval.sh <model_path> <out_dir> [extra generate.py args]
+set -euo pipefail
+source /wekafs/ict/hx_624/llm-ft/env.sh
+MODEL=$(readlink -f "$1"); OUT=$(readlink -m "$2"); shift 2
+GPUS=${GPUS:-0}
+PROMPTS=/wekafs/ict/hx_624/llm-ft/data/sft_v1/bench_prompts.jsonl
+[ -f $PROMPTS ] || PROMPTS=/wekafs/ict/hx_624/llm-ft/data/sft_v0/bench_prompts.jsonl
+mkdir -p $OUT
+IFS=',' read -ra G <<< "$GPUS"; N=${#G[@]}
+echo "[eval] model=$MODEL out=$OUT gpus=$GPUS shards=$N"
+pids=()
+for i in "${!G[@]}"; do
+ CUDA_VISIBLE_DEVICES=${G[$i]} python /wekafs/ict/hx_624/llm-ft/eval/generate.py --model $MODEL --prompts $PROMPTS --out $OUT --shard $i --num_shards $N "$@" > $OUT/gen_shard$i.log 2>&1 &
+ pids+=($!)
+done
+for p in "${pids[@]}"; do wait $p; done
+cat $OUT/gens_shard*.jsonl > $OUT/gens.jsonl
+echo "[eval] generation done: $(wc -l < $OUT/gens.jsonl) samples"
+python /wekafs/ict/hx_624/llm-ft/eval/run_bench.py --gen_dir $OUT --workers ${WORKERS:-32} --timeout ${TIMEOUT:-300} 2>&1 | tee $OUT/exec.log
+python /wekafs/ict/hx_624/llm-ft/eval/metrics.py --gen_dir $OUT 2>&1 | tee $OUT/metrics.log
diff --git a/finetune/eval/run_eval_vllm.sh b/finetune/eval/run_eval_vllm.sh
new file mode 100755
index 00000000..524ddc90
--- /dev/null
+++ b/finetune/eval/run_eval_vllm.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+# usage: GPUS=0 ./run_eval_vllm.sh <model_path> <out_dir> [extra generate_vllm.py args] (TP = number of GPUs listed)
+set -euo pipefail
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+source /wekafs/ict/hx_624/.secrets/hf.env; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING TOKENIZERS_PARALLELISM=false
+MODEL=$(readlink -f "$1"); OUT=$(readlink -m "$2"); shift 2
+export CUDA_VISIBLE_DEVICES=${GPUS:-0}; TP=$(echo $CUDA_VISIBLE_DEVICES | tr ',' '\n' | wc -l)
+PROMPTS=${PROMPTS:-/wekafs/ict/hx_624/llm-ft/data/sft_v1/bench_prompts.jsonl}
+mkdir -p $OUT; echo "[eval-vllm] model=$MODEL out=$OUT gpus=$CUDA_VISIBLE_DEVICES tp=$TP"
+python /wekafs/ict/hx_624/llm-ft/eval/generate_vllm.py --model $MODEL --prompts $PROMPTS --out $OUT --tp $TP "$@" 2>&1 | grep -vE "^\s*$" | tail -n 40
+cp $OUT/gens_shard0.jsonl $OUT/gens.jsonl
+conda activate llmft
+python /wekafs/ict/hx_624/llm-ft/eval/run_bench.py --gen_dir $OUT --workers ${WORKERS:-32} --timeout ${TIMEOUT:-300} 2>&1 | tee $OUT/exec.log
+python /wekafs/ict/hx_624/llm-ft/eval/metrics.py --gen_dir $OUT 2>&1 | tee $OUT/metrics.log
diff --git a/finetune/eval/threejs_eval.py b/finetune/eval/threejs_eval.py
new file mode 100644
index 00000000..ab969ad3
--- /dev/null
+++ b/finetune/eval/threejs_eval.py
@@ -0,0 +1,17 @@
+"""Run every <gen_dir>/<task>/code.py (HTML) through the headless-browser runner (parallel) and summarise."""
+import argparse, json, os, sys
+from concurrent.futures import ThreadPoolExecutor
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))); from threejs_runner import run_threejs
+ap = argparse.ArgumentParser(); ap.add_argument("--gen_dir", required=True); ap.add_argument("--workers", type=int, default=8); a = ap.parse_args()
+tasks = sorted(d for d in os.listdir(a.gen_dir) if os.path.isfile(os.path.join(a.gen_dir, d, "code.py")))
+def job(t):
+ html = open(os.path.join(a.gen_dir, t, "code.py")).read()
+ r = run_threejs(html, os.path.join(a.gen_dir, t, "exec")); r["task"] = t; r.pop("shot", None); return r
+with ThreadPoolExecutor(a.workers) as ex: res = list(ex.map(job, tasks))
+from collections import Counter
+c = Counter(r["status"] for r in res)
+with open(os.path.join(a.gen_dir, "threejs_results.jsonl"), "w") as f:
+ for r in res: f.write(json.dumps(r) + "\n")
+summ = {"dialect": "threejs", "n": len(res), "exec_ok": c.get("OK", 0), "exec_rate": round(c.get("OK", 0) / max(1, len(res)), 4), "counts": dict(c)}
+json.dump(summ, open(os.path.join(a.gen_dir, "summary.json"), "w"), indent=2); print(json.dumps(summ))
+print("failure modes:", Counter((r["error"] or r["status"])[:80] for r in res if r["status"] != "OK").most_common(6))
diff --git a/finetune/eval/threejs_runner.py b/finetune/eval/threejs_runner.py
new file mode 100644
index 00000000..c44f6a36
--- /dev/null
+++ b/finetune/eval/threejs_runner.py
@@ -0,0 +1,56 @@
+"""Headless-Chromium executor for single-file three.js HTML (Playwright). Returns dict(status, error, console_errors, canvas_nonblank, latency_s, shot).
+status OK = page loaded, no uncaught JS error / failed module import, a <canvas> exists and its rendered pixels are not (nearly) uniform."""
+import os, sys, json, time, base64, io
+def run_threejs(html, workdir, timeout=40, settle_ms=4000):
+ os.environ.setdefault("PLAYWRIGHT_BROWSERS_PATH", "/wekafs/ict/hx_624/cache/ms-playwright")
+ workdir = os.path.abspath(workdir); os.makedirs(workdir, exist_ok=True)
+ hp = os.path.join(workdir, "index.html"); open(hp, "w").write(html)
+ rep = {"status": "FAIL", "error": None, "console_errors": [], "canvas_nonblank": None, "latency_s": 0, "shot": None}
+ t0 = time.time()
+ try:
+ from playwright.sync_api import sync_playwright
+ with sync_playwright() as p:
+ browser = p.chromium.launch(headless=True, args=["--use-gl=angle", "--use-angle=swiftshader", "--enable-unsafe-swiftshader", "--ignore-gpu-blocklist", "--no-sandbox", "--disable-dev-shm-usage"])
+ page = browser.new_page(viewport={"width": 640, "height": 480})
+ errs = []
+ page.on("pageerror", lambda e: errs.append("pageerror: " + str(e)[:300]))
+ page.on("console", lambda m: errs.append("console.error: " + m.text[:300]) if m.type == "error" else None)
+ page.on("requestfailed", lambda r: errs.append("requestfailed: " + r.url[:200]))
+ page.goto("file://" + hp, wait_until="load", timeout=timeout * 1000)
+ page.wait_for_timeout(settle_ms)
+ # canvas check: count distinct pixels over a downsampled grid
+ info = page.evaluate("""() => {
+ const cs = document.querySelectorAll('canvas'); if (!cs.length) return {n:0};
+ const c = cs[0]; const w=c.width, h=c.height; if (!w||!h) return {n:cs.length, w, h, distinct:0};
+ let gl = c.getContext('webgl2') || c.getContext('webgl');
+ let pix = null;
+ if (gl) { pix = new Uint8Array(w*h*4); try { gl.readPixels(0,0,w,h,gl.RGBA,gl.UNSIGNED_BYTE,pix); } catch(e) { pix=null; } }
+ if (!pix) { try { const t=document.createElement('canvas'); t.width=w; t.height=h; const ctx=t.getContext('2d'); ctx.drawImage(c,0,0); pix=ctx.getImageData(0,0,w,h).data; } catch(e) { return {n:cs.length, w, h, distinct:-1, err:String(e)}; } }
+ const seen = new Set(); const step = Math.max(1, Math.floor((w*h)/4000));
+ for (let i=0;i<w*h;i+=step){ const o=i*4; seen.add((pix[o]>>3)+','+(pix[o+1]>>3)+','+(pix[o+2]>>3)); if (seen.size>64) break; }
+ return {n:cs.length, w, h, distinct:seen.size};
+ }""")
+ shot = os.path.join(workdir, "shot.png"); page.screenshot(path=shot); rep["shot"] = shot
+ browser.close()
+ rep["console_errors"] = errs[:10]
+ fatal = [e for e in errs if e.startswith("pageerror") or "Failed to resolve module" in e or "Uncaught" in e or e.startswith("requestfailed")]
+ # blank check from the composited screenshot (the WebGL drawing buffer is cleared after compositing, so in-page readPixels is unreliable)
+ try:
+ from PIL import Image; import numpy as np
+ im = np.array(Image.open(shot).convert("RGB")); q = (im[::6, ::6].reshape(-1, 3) // 16)
+ distinct = len(set(map(tuple, q))); rep["shot_distinct"] = distinct; rep["shot_std"] = float(im.std())
+ nonblank = distinct > 6 and float(im.std()) > 4.0
+ except Exception as e:
+ nonblank = info.get("distinct", 0) > 3
+ rep["canvas_nonblank"] = bool(info.get("n", 0)) and nonblank
+ if fatal: rep["status"] = "FAIL"; rep["error"] = fatal[0]
+ elif not info.get("n"): rep["status"] = "EMPTY"; rep["error"] = "no <canvas>"
+ elif not rep["canvas_nonblank"]: rep["status"] = "EMPTY"; rep["error"] = f"blank canvas (shot distinct={rep.get('shot_distinct')})"
+ else: rep["status"] = "OK"
+ except Exception as e:
+ rep["error"] = (type(e).__name__ + ": " + str(e))[:400]
+ if "Timeout" in rep["error"]: rep["status"] = "TIMEOUT"
+ rep["latency_s"] = round(time.time() - t0, 1)
+ return rep
+if __name__ == "__main__":
+ print(json.dumps(run_threejs(open(sys.argv[1]).read(), sys.argv[2]), indent=1))
diff --git a/finetune/scripts/after_lora.sh b/finetune/scripts/after_lora.sh
new file mode 100755
index 00000000..e5bf6f3b
--- /dev/null
+++ b/finetune/scripts/after_lora.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+# wait for LoRA run to finish -> merge adapter -> eval merged model on 3DCodeBench
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1
+cd /wekafs/ict/hx_624/llm-ft
+until [ -f $RUN/train_results.json ] || ! pgrep -f "configs/lf/qwen35_9b_lora_sft.yaml" >/dev/null; do sleep 20; done
+[ -f $RUN/train_results.json ] || { echo "LORA_TRAIN_DID_NOT_FINISH"; exit 1; }
+echo "[after_lora] train finished: $(cat $RUN/train_results.json | tr -d '\n')"
+scripts/lf_export.sh configs/lf/export_lora_template.yaml > logs/export_lora_v1.log 2>&1 && echo "[after_lora] export done" || { echo "EXPORT_FAILED"; exit 1; }
+ls $RUN/merged | head
+GPUS=${EVAL_GPUS:-1} eval/run_eval.sh $RUN/merged eval/out/qwen35_9b_lora_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_qwen35_9b_lora_v1.log 2>&1
+echo "[after_lora] eval done"; cat eval/out/qwen35_9b_lora_v1/summary.json; echo AFTER_LORA_DONE
diff --git a/finetune/scripts/after_lora_v2.sh b/finetune/scripts/after_lora_v2.sh
new file mode 100755
index 00000000..c490d139
--- /dev/null
+++ b/finetune/scripts/after_lora_v2.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+# wait LoRA-v1 -> merge -> test vLLM on GPU 0 -> eval merged model with vLLM (GPU 0); fallback: HF sharded eval on GPUs 0,2
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1; cd /wekafs/ict/hx_624/llm-ft
+until [ -f $RUN/train_results.json ] || ! pgrep -f "configs/lf/qwen35_9b_lora_sft.yaml" >/dev/null; do sleep 20; done
+[ -f $RUN/train_results.json ] || { echo "LORA_TRAIN_DID_NOT_FINISH"; exit 1; }
+echo "[after_lora] train finished: $(tr -d '\n' < $RUN/train_results.json)"
+scripts/lf_export.sh configs/lf/export_lora_template.yaml > logs/export_lora_v1.log 2>&1 && echo "[after_lora] export done" || { echo "EXPORT_FAILED"; exit 1; }
+echo "[after_lora] vllm smoke test on GPU 0"
+if GPUS=0 eval/run_eval_vllm.sh /wekafs/ict/hx_624/models/Qwen3.5-9B eval/out/_vllm_smoke --no_think --limit 8 --max_new_tokens 1024 > logs/vllm_smoke.log 2>&1; then
+ echo "[after_lora] VLLM_OK -> eval merged model with vLLM on GPU 0"
+ GPUS=0 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_lora_v1 --no_think --max_new_tokens 6144 > logs/eval_qwen35_9b_lora_v1.log 2>&1
+else
+ echo "[after_lora] VLLM_FAILED -> HF eval on GPUs 0,2"; tail -20 logs/vllm_smoke.log
+ GPUS=0,2 eval/run_eval.sh $RUN/merged eval/out/qwen35_9b_lora_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_qwen35_9b_lora_v1.log 2>&1
+fi
+echo "[after_lora] eval done"; cat eval/out/qwen35_9b_lora_v1/summary.json; echo AFTER_LORA_DONE
diff --git a/finetune/scripts/analyze_3dcodeverse.py b/finetune/scripts/analyze_3dcodeverse.py
new file mode 100644
index 00000000..699914fd
--- /dev/null
+++ b/finetune/scripts/analyze_3dcodeverse.py
@@ -0,0 +1,54 @@
+"""Survey every metadata.parquet in 3DCodeVerse: counts, code length, caption availability, language, license, multi_file.
+Writes data/analysis/3dcodeverse_survey.md (+ .json). Token counts are estimated with the Qwen3.5 tokenizer on a sample."""
+import glob, json, os, random, sys
+import numpy as np, pandas as pd
+ROOT = "/wekafs/ict/hx_624/data/3dcodeverse_meta"
+OUT = "/wekafs/ict/hx_624/llm-ft/data/analysis"; os.makedirs(OUT, exist_ok=True)
+try:
+ from transformers import AutoTokenizer
+ tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")
+except Exception as e:
+ tok = None; print("no tokenizer:", e)
+random.seed(0)
+rows = []
+files = sorted(glob.glob(f"{ROOT}/**/metadata.parquet", recursive=True))
+print(len(files), "metadata.parquet files")
+agg = {}
+for f in files:
+ rel = os.path.relpath(os.path.dirname(f), ROOT)
+ src = rel.split("/")[0]
+ df = pd.read_parquet(f)
+ n = len(df)
+ code = df["code"].fillna("")
+ clen = code.str.len().values
+ meta0 = json.loads(df["meta_json"].iloc[0]) if "meta_json" in df and n else {}
+ lang = df["language"].iloc[0] if "language" in df else meta0.get("language")
+ caps = df["captions"].iloc[0] if "captions" in df else None
+ cap_keys = list(caps.keys()) if isinstance(caps, dict) else []
+ # caption availability
+ def has(k): return float(np.mean([isinstance(c, dict) and bool(c.get(k)) for c in df["captions"]])) if "captions" in df else 0.0
+ cap_avail = {k: round(has(k), 3) for k in ["detailed", "instruction", "factory", "brief", "synthetic_prompt"]}
+ # token estimate on up to 200 random samples
+ tok_p50 = tok_p90 = tok_mean = None
+ if tok is not None and n:
+ idx = random.sample(range(n), min(200, n))
+ tl = np.array([len(tok(code.iloc[i]).input_ids) for i in idx])
+ tok_p50, tok_p90, tok_mean = int(np.percentile(tl, 50)), int(np.percentile(tl, 90)), float(tl.mean())
+ uniq = code.nunique()
+ rows.append(dict(subset=rel, source=src, n=n, language=lang, license=meta0.get("license"), multi_file=meta0.get("multi_file"),
+ entry=meta0.get("entry"), chars_p50=int(np.percentile(clen, 50)) if n else 0, chars_p90=int(np.percentile(clen, 90)) if n else 0,
+ tok_p50=tok_p50, tok_p90=tok_p90, tok_mean=tok_mean, est_total_tokens=(tok_mean * n if tok_mean else None),
+ unique_code_frac=round(uniq / max(n, 1), 3), caption_keys=cap_keys, cap_avail=cap_avail, columns=list(df.columns)))
+ a = agg.setdefault(src, dict(n=0, est_tokens=0.0, langs=set()))
+ a["n"] += n; a["est_tokens"] += (tok_mean * n if tok_mean else 0); a["langs"].add(str(lang))
+ print(f"{rel:45s} n={n:6d} lang={str(lang):16s} tok_p50={tok_p50} tok_p90={tok_p90} uniq={uniq/max(n,1):.2f} caps={cap_avail}")
+json.dump({"subsets": rows, "by_source": {k: dict(n=v["n"], est_tokens=v["est_tokens"], langs=sorted(v["langs"])) for k, v in agg.items()}},
+ open(f"{OUT}/3dcodeverse_survey.json", "w"), indent=1, default=str)
+with open(f"{OUT}/3dcodeverse_survey.md", "w") as f:
+ f.write("| source | #samples | languages | est. tokens (code only) |\n|---|---|---|---|\n")
+ for k, v in sorted(agg.items(), key=lambda kv: -kv[1]["n"]):
+ f.write(f"| {k} | {v['n']} | {', '.join(sorted(v['langs']))} | {v['est_tokens']/1e6:.1f}M |\n")
+ f.write("\n| subset | n | lang | tok p50 | tok p90 | unique code | captions (detailed/instruction/factory) | license |\n|---|---|---|---|---|---|---|---|\n")
+ for r in rows:
+ f.write(f"| {r['subset']} | {r['n']} | {r['language']} | {r['tok_p50']} | {r['tok_p90']} | {r['unique_code_frac']} | {r['cap_avail']['detailed']}/{r['cap_avail']['instruction']}/{r['cap_avail']['factory']} | {r['license']} |\n")
+print("written", OUT)
diff --git a/finetune/scripts/bon8_v1.sh b/finetune/scripts/bon8_v1.sh
new file mode 100755
index 00000000..2d58f997
--- /dev/null
+++ b/finetune/scripts/bon8_v1.sh
@@ -0,0 +1,18 @@
+#!/bin/bash
+# best-of-8 for LoRA-v1: 8 sampled generations (T=0.7) on 4 GPUs now; Blender exec deferred until round-2 exec finishes (CPU contention); then pass@n
+cd /wekafs/ict/hx_624/llm-ft; PFX=qwen35_9b_lora_v1_T07
+scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_bon8.log 2>&1 || { echo "[bon8] EXPORT FAILED"; exit 1; }
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING
+for round in 0 1; do
+ for g in 0 1 2 3; do i=$((round*4+g+1)); [ $i -le 4 ] && continue # samples 1-4 already exist
+ OUT=eval/out/${PFX}_s$i; mkdir -p $OUT
+ CUDA_VISIBLE_DEVICES=$g python eval/generate_vllm.py --model runs/lf_qwen35_9b_lora_v1/merged --prompts data/sft_v1/bench_prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 6144 --temperature 0.7 --seed $i > $OUT/gen.log 2>&1 &
+ done; wait
+done
+for i in 5 6 7 8; do cp eval/out/${PFX}_s$i/gens_shard0.jsonl eval/out/${PFX}_s$i/gens.jsonl; grep gen-vllm eval/out/${PFX}_s$i/gen.log | tail -1; done
+rm -rf runs/lf_qwen35_9b_lora_v1/merged; echo "[bon8] $(date) gens done (s5-s8)"
+until grep -q "\[r2\] .*exec done" logs/self_train_round2.log 2>/dev/null; do sleep 60; done
+conda activate llmft
+for i in 5 6 7 8; do OUT=eval/out/${PFX}_s$i; python eval/run_bench.py --gen_dir $OUT --workers 32 --timeout 300 > $OUT/exec.log 2>&1; python eval/metrics.py --gen_dir $OUT > $OUT/metrics.log 2>&1; grep "status counts" $OUT/exec.log; done
+python eval/pass_at_n.py $PFX 8; python eval/pass_at_n.py $PFX 4; echo "[bon8] DONE"
diff --git a/finetune/scripts/build_boot_data.py b/finetune/scripts/build_boot_data.py
new file mode 100644
index 00000000..75e81a34
--- /dev/null
+++ b/finetune/scripts/build_boot_data.py
@@ -0,0 +1,37 @@
+"""Build bootstrapped SFT sets from execution-verified generations (proper python file; the earlier bash-heredoc version mangled the ```python fence via backtick substitution).
+usage: build_boot_data.py <round_dir> <out_name> [--base data/sft_v1]"""
+import argparse, glob, hashlib, json, os, random, shutil
+ap = argparse.ArgumentParser(); ap.add_argument("round_dir"); ap.add_argument("out_name"); ap.add_argument("--base", default="data/sft_v1"); ap.add_argument("--max_per_prompt", type=int, default=2)
+a = ap.parse_args(); random.seed(0)
+W = a.round_dir
+prompts = {}
+for f in glob.glob(f"{W}/prompts*.jsonl"):
+ for l in open(f):
+ r = json.loads(l); prompts[r["task"]] = r
+base = [json.loads(l) for l in open(f"{a.base}/train.jsonl")]
+def h_of(code): return hashlib.sha1(("```python\n" + code + "\n```").encode()).hexdigest()
+base_hashes = {hashlib.sha1(r["messages"][-1]["content"].encode()).hexdigest() for r in base}
+seen = set(base_hashes); new = []; n_ok = 0; dup_base = 0; dup_self = 0; per_prompt = {}
+for d in sorted(glob.glob(f"{W}/gen_s*/")) + sorted(glob.glob(f"{W}/gen_s*_g*/")):
+ if not os.path.exists(os.path.join(d, "exec_results.jsonl")): continue
+ for l in open(os.path.join(d, "exec_results.jsonl")):
+ r = json.loads(l)
+ if r["status"] != "OK": continue
+ n_ok += 1
+ code = open(os.path.join(d, r["task"], "code.py")).read().strip()
+ if code.count("\n") < 4: continue
+ h = h_of(code)
+ if h in base_hashes: dup_base += 1; continue
+ if h in seen: dup_self += 1; continue
+ if per_prompt.get(r["task"], 0) >= a.max_per_prompt: continue
+ seen.add(h); per_prompt[r["task"]] = per_prompt.get(r["task"], 0) + 1
+ p = prompts[r["task"]]
+ new.append({"messages": p["messages"][:2] + [{"role": "assistant", "content": "```python\n" + code + "\n```"}],
+ "source": "boot_" + str(p.get("src", p.get("source", "train"))), "id": r["task"], "prompt_kind": "boot"})
+out = f"data/{a.out_name}"; os.makedirs(out, exist_ok=True)
+allrows = base + new; random.shuffle(allrows)
+with open(f"{out}/train.jsonl", "w") as f:
+ for r in allrows: f.write(json.dumps(r, ensure_ascii=False) + "\n")
+shutil.copy(f"{a.base}/val.jsonl", f"{out}/val.jsonl")
+stats = {"n_train": len(allrows), "n_base": len(base), "n_new": len(new), "n_ok_generations": n_ok, "dup_of_training_code": dup_base, "dup_within_generations": dup_self, "prompts_with_new": len(per_prompt)}
+json.dump(stats, open(f"{out}/stats.json", "w"), indent=1); print(json.dumps(stats))
diff --git a/finetune/scripts/build_md_bal.py b/finetune/scripts/build_md_bal.py
new file mode 100644
index 00000000..3153d04d
--- /dev/null
+++ b/finetune/scripts/build_md_bal.py
@@ -0,0 +1,34 @@
+"""md_bal: re-balanced mix (~110M tok like md_big) with a much larger Blender share: bio x4, boot x2, CadQuery 35k, GLSL 25k, OpenSCAD x4, three.js."""
+import json, random, os, sys
+sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/scripts"); from ntok_cache import load_ntok
+from transformers import AutoTokenizer
+random.seed(13); tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")
+MD = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; CAP = {"cadquery": 35000, "glsl": 25000, "openscad": 10**9, "blender": 10**9, "threejs": 10**9}; REP = {"openscad": 4, "blender": 4}; BOOT_REP = 2
+def ntok(r): return len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)
+out_rows = []; stats = {}
+for d, cap in CAP.items():
+ rows = [json.loads(l) for l in open(f"{MD}/{d}/train.jsonl")]; NT = load_ntok(f"{MD}/{d}/train.jsonl", tok)
+ for r, n_ in zip(rows, NT): r["_ntok"] = n_
+ random.shuffle(rows); keep = []; toks = 0
+ for r in rows:
+ if len(keep) >= cap: break
+ n = r.pop("_ntok")
+ if n <= 8192: keep.append(r); toks += n
+ rep = REP.get(d, 1); stats[d] = (len(keep) * rep, toks * rep); out_rows += keep * rep
+BOOT = "/wekafs/ict/hx_624/llm-ft/data/sft_v13_boot/train.jsonl"; bl = open(BOOT).read().splitlines(); bn = load_ntok(BOOT, tok)
+boot = [(json.loads(l), n) for l, n in zip(bl, bn) if '"source": "boot_' in l]
+bt = 0; kb = []
+for r, n in boot:
+ if n <= 8192: kb.append({"messages": r["messages"], "dialect": "blender", "subset": "boot"}); bt += n
+stats["blender_boot"] = (len(kb) * BOOT_REP, bt * BOOT_REP); out_rows += kb * BOOT_REP
+# LLaMA-Factory's qwen3_5 (multimodal) template treats literal <image>/<video>/<audio> in text as media placeholders -> escape them
+def _san(t): return t.replace("<image>", "<image >").replace("<video>", "<video >").replace("<audio>", "<audio >")
+for r in out_rows:
+ for m in r["messages"]: m["content"] = _san(m["content"])
+random.shuffle(out_rows)
+out = "/wekafs/ict/hx_624/llm-ft/data/md_bal"; os.makedirs(out, exist_ok=True)
+with open(f"{out}/train.jsonl", "w") as f:
+ for r in out_rows: f.write(json.dumps(r, ensure_ascii=False) + "\n")
+os.system(f"cp /wekafs/ict/hx_624/llm-ft/data/md_mixed/val.jsonl {out}/val.jsonl")
+tot = sum(v[1] for v in stats.values())
+print({k: (v[0], round(v[1]/1e6, 1)) for k, v in stats.items()}, "| total samples", len(out_rows), "| tokens/epoch %.1fM" % (tot/1e6))
diff --git a/finetune/scripts/build_md_big.py b/finetune/scripts/build_md_big.py
new file mode 100644
index 00000000..57821a86
--- /dev/null
+++ b/finetune/scripts/build_md_big.py
@@ -0,0 +1,31 @@
+"""md_big: larger multi-dialect SFT set (token-filtered <=8192): blender (multidialect train + v13 bootstrapped), cadquery 40k, glsl 40k, openscad all."""
+import json, random, os, sys
+sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/scripts"); from ntok_cache import load_ntok
+from transformers import AutoTokenizer
+random.seed(11); tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")
+MD = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; CAP = {"cadquery": 40000, "glsl": 40000, "openscad": 5000, "blender": 10000}
+def ntok(r): return len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)
+out_rows = []; stats = {}
+for d, cap in CAP.items():
+ rows = [json.loads(l) for l in open(f"{MD}/{d}/train.jsonl")]; NT = load_ntok(f"{MD}/{d}/train.jsonl", tok)
+ for r, n_ in zip(rows, NT): r["_ntok"] = n_
+ random.shuffle(rows); keep = []; toks = 0
+ for r in rows:
+ if len(keep) >= cap: break
+ n = r.pop("_ntok")
+ if n <= 8192: keep.append(r); toks += n
+ stats[d] = (len(keep), toks); out_rows += keep
+# bootstrapped blender samples (execution-verified, novel prompts)
+BOOT = "/wekafs/ict/hx_624/llm-ft/data/sft_v13_boot/train.jsonl"; bl = open(BOOT).read().splitlines(); bn = load_ntok(BOOT, tok)
+boot = [(json.loads(l), n) for l, n in zip(bl, bn) if '"source": "boot_' in l]
+bt = 0; kb = []
+for r, n in boot:
+ if n <= 8192: kb.append({"messages": r["messages"], "dialect": "blender", "subset": "boot"}); bt += n
+stats["blender_boot"] = (len(kb), bt); out_rows += kb
+random.shuffle(out_rows)
+out = "/wekafs/ict/hx_624/llm-ft/data/md_big"; os.makedirs(out, exist_ok=True)
+with open(f"{out}/train.jsonl", "w") as f:
+ for r in out_rows: f.write(json.dumps(r, ensure_ascii=False) + "\n")
+os.system(f"cp /wekafs/ict/hx_624/llm-ft/data/md_mixed/val.jsonl {out}/val.jsonl")
+tot = sum(v[1] for v in stats.values())
+print({k: (v[0], round(v[1]/1e6, 1)) for k, v in stats.items()}, "| total samples", len(out_rows), "| tokens/epoch %.1fM" % (tot/1e6))
diff --git a/finetune/scripts/build_md_xl.py b/finetune/scripts/build_md_xl.py
new file mode 100644
index 00000000..02d15af2
--- /dev/null
+++ b/finetune/scripts/build_md_xl.py
@@ -0,0 +1,34 @@
+"""md_xl: scale-up of md_big — ALL cadquery (deepcad+articraft) + ALL glsl + all blender (+15.7k bootstrapped) + openscad x4 upsampled + threejs, token-filtered <=8192."""
+import json, random, os, sys
+sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/scripts"); from ntok_cache import load_ntok
+from transformers import AutoTokenizer
+random.seed(12); tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")
+MD = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; CAP = {"cadquery": 10**9, "glsl": 10**9, "openscad": 10**9, "blender": 10**9, "threejs": 10**9}; REP = {"openscad": 4}
+def ntok(r): return len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)
+out_rows = []; stats = {}
+for d, cap in CAP.items():
+ rows = [json.loads(l) for l in open(f"{MD}/{d}/train.jsonl")]; NT = load_ntok(f"{MD}/{d}/train.jsonl", tok)
+ for r, n_ in zip(rows, NT): r["_ntok"] = n_
+ random.shuffle(rows); keep = []; toks = 0
+ for r in rows:
+ if len(keep) >= cap: break
+ n = r.pop("_ntok")
+ if n <= 8192: keep.append(r); toks += n
+ rep = REP.get(d, 1); stats[d] = (len(keep) * rep, toks * rep); out_rows += keep * rep
+BOOT = "/wekafs/ict/hx_624/llm-ft/data/sft_v13_boot/train.jsonl"; bl = open(BOOT).read().splitlines(); bn = load_ntok(BOOT, tok)
+boot = [(json.loads(l), n) for l, n in zip(bl, bn) if '"source": "boot_' in l]
+bt = 0; kb = []
+for r, n in boot:
+ if n <= 8192: kb.append({"messages": r["messages"], "dialect": "blender", "subset": "boot"}); bt += n
+stats["blender_boot"] = (len(kb), bt); out_rows += kb
+# LLaMA-Factory's qwen3_5 (multimodal) template treats literal <image>/<video>/<audio> in text as media placeholders -> escape them
+def _san(t): return t.replace("<image>", "<image >").replace("<video>", "<video >").replace("<audio>", "<audio >")
+for r in out_rows:
+ for m in r["messages"]: m["content"] = _san(m["content"])
+random.shuffle(out_rows)
+out = "/wekafs/ict/hx_624/llm-ft/data/md_xl"; os.makedirs(out, exist_ok=True)
+with open(f"{out}/train.jsonl", "w") as f:
+ for r in out_rows: f.write(json.dumps(r, ensure_ascii=False) + "\n")
+os.system(f"cp /wekafs/ict/hx_624/llm-ft/data/md_mixed/val.jsonl {out}/val.jsonl")
+tot = sum(v[1] for v in stats.values())
+print({k: (v[0], round(v[1]/1e6, 1)) for k, v in stats.items()}, "| total samples", len(out_rows), "| tokens/epoch %.1fM" % (tot/1e6))
diff --git a/finetune/scripts/build_multidialect.py b/finetune/scripts/build_multidialect.py
new file mode 100644
index 00000000..4cdb033f
--- /dev/null
+++ b/finetune/scripts/build_multidialect.py
@@ -0,0 +1,77 @@
+"""Build per-dialect SFT sets from ALL 3DCodeVerse metadata parquets + data-quality report.
+Dialects: blender (bioinspired3d+factories non-test), cadquery_deepcad, cadquery_articraft, openscad (thingiverse), glsl (shadertoy).
+Outputs data/multidialect/<dialect>/{train,test}.jsonl (+ all.jsonl) and data/multidialect/report.json"""
+import glob, hashlib, json, os, random, re, sys, unicodedata
+import pandas as pd, numpy as np
+random.seed(7)
+META = "/wekafs/ict/hx_624/data/3dcodeverse_meta"; OUT = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; os.makedirs(OUT, exist_ok=True)
+SYS = {
+ "blender": "You are an expert in procedural 3D modeling with Blender Python (bpy). Given a description of an object, write a complete, standalone Blender 5 Python script that builds it from scratch (clear the default scene first). Output ONLY the code in one ```python block.",
+ "cadquery": "You are an expert CAD programmer. Given a description of a part, write a complete, standalone CadQuery (Python) script that builds it and leaves the final solid in a variable named `result`. Output ONLY the code in one ```python block.",
+ "openscad": "You are an expert OpenSCAD programmer. Given a request, write a complete, standalone OpenSCAD (.scad) program that produces the described model. Output ONLY the code in one ```openscad block.",
+ "glsl": "You are an expert shader programmer. Given a description, write a complete Shadertoy-style GLSL ES 3.00 fragment shader defining `void mainImage(out vec4 fragColor, in vec2 fragCoord)`. Output ONLY the code in one ```glsl block.",
+}
+FENCE = {"blender": "python", "cadquery": "python", "openscad": "openscad", "glsl": "glsl"}
+def bad_chars(s):
+ return sum(1 for ch in s if unicodedata.category(ch) in ("Cc",) and ch not in "\n\t\r") + s.count("�")
+def wrap(dialect, caps, name):
+ ins = (caps or {}).get("instruction") or ""; det = (caps or {}).get("detailed") or ""
+ if dialect == "cadquery":
+ return "Write a CadQuery (Python) script that builds the following CAD model: " + (ins or det)
+ if dialect == "glsl":
+ return ins if ins.lower().startswith(("write", "create", "implement", "make")) else "Write a GLSL fragment shader for Shadertoy that renders: " + (ins or det)
+ if dialect == "openscad":
+ return det or ins # thingiverse captions are already long OpenSCAD requests
+ return ins or ("Create the 3D object described below using Python Blender code: " + det)
+sets = {}
+def add(dialect, sub, df, code_col="code"):
+ rows = sets.setdefault(dialect, [])
+ for r in df.itertuples():
+ code = getattr(r, code_col) or ""; caps = dict(r.captions) if isinstance(r.captions, dict) else {}
+ rows.append(dict(dialect=dialect, subset=sub, id=r.id, code=code, prompt=wrap(dialect, caps, getattr(r, "name", None)), caps=caps))
+# --- sources
+test_names = {d.replace("_seed0", "") for d in os.listdir("/wekafs/ict/hx_624/data/3dcodebench/data")}
+for f in glob.glob(f"{META}/bioinspired3d/*/metadata.parquet"): add("blender", "bioinspired3d", pd.read_parquet(f))
+for f in glob.glob(f"{META}/3dcodebench/*/metadata.parquet"):
+ df = pd.read_parquet(f); df = df[~df["name"].str.replace("Factory", "", regex=False).isin(test_names)]; add("blender", "factories_nontest", df)
+for f in glob.glob(f"{META}/deepcad/*/metadata.parquet"): add("cadquery", "deepcad", pd.read_parquet(f))
+for f in glob.glob(f"{META}/articraft/cadquery_single_tex*/metadata.parquet"): add("cadquery", "articraft", pd.read_parquet(f))
+add("openscad", "thingiverse", pd.read_parquet(f"{META}/thingiverse/openscad/metadata.parquet"))
+for f in glob.glob(f"{META}/shadertoy/*/metadata.parquet"): add("glsl", "shadertoy", pd.read_parquet(f))
+# --- QC + build
+from transformers import AutoTokenizer
+tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")
+report = {}
+for dialect, rows in sets.items():
+ n0 = len(rows); issues = dict(empty_code=0, short_code=0, bad_chars=0, no_prompt=0, dup_code=0, too_long=0)
+ seen = set(); keep = []
+ for s in rows:
+ code = s["code"].strip()
+ if not code: issues["empty_code"] += 1; continue
+ if code.count("\n") < 3: issues["short_code"] += 1; continue
+ if bad_chars(code) > 0: issues["bad_chars"] += 1; continue
+ if len(s["prompt"]) < 20: issues["no_prompt"] += 1; continue
+ h = hashlib.sha1(code.encode()).hexdigest()
+ if h in seen: issues["dup_code"] += 1; continue
+ seen.add(h); s["code"] = code; keep.append(s)
+ # token lengths on a sample
+ samp = random.sample(keep, min(500, len(keep)))
+ lens = [len(tok(s["prompt"] + "\n" + s["code"]).input_ids) for s in samp]
+ p50, p90, p99 = np.percentile(lens, [50, 90, 99]); frac_gt8k = float(np.mean([l > 8192 for l in lens])); frac_gt16k = float(np.mean([l > 16384 for l in lens]))
+ random.shuffle(keep)
+ n_test = min(200, max(50, len(keep) // 50))
+ test, train = keep[:n_test], keep[n_test:]
+ d = f"{OUT}/{dialect}"; os.makedirs(d, exist_ok=True)
+ for split, rs in [("train", train), ("test", test)]:
+ with open(f"{d}/{split}.jsonl", "w") as f:
+ for s in rs:
+ f.write(json.dumps({"messages": [{"role": "system", "content": SYS[dialect]}, {"role": "user", "content": s["prompt"]},
+ {"role": "assistant", "content": f"```{FENCE[dialect]}\n{s['code']}\n```"}],
+ "dialect": dialect, "subset": s["subset"], "id": s["id"]}, ensure_ascii=False) + "\n")
+ by_sub = {}
+ for s in keep: by_sub[s["subset"]] = by_sub.get(s["subset"], 0) + 1
+ report[dialect] = dict(raw=n0, kept=len(keep), train=len(train), test=len(test), issues=issues, by_subset=by_sub,
+ tok_p50=int(p50), tok_p90=int(p90), tok_p99=int(p99), frac_gt8k=round(frac_gt8k, 3), frac_gt16k=round(frac_gt16k, 3),
+ est_train_tokens_M=round(float(np.mean(lens)) * len(train) / 1e6, 1))
+ print(dialect, json.dumps(report[dialect]))
+json.dump(report, open(f"{OUT}/report.json", "w"), indent=1)
diff --git a/finetune/scripts/build_multidialect_train.py b/finetune/scripts/build_multidialect_train.py
new file mode 100644
index 00000000..cf6d2e9a
--- /dev/null
+++ b/finetune/scripts/build_multidialect_train.py
@@ -0,0 +1,27 @@
+"""Subsample per-dialect train sets (token-filtered ≤ 8192), build LF datasets md_<dialect> and md_mixed."""
+import json, random, os, subprocess
+from transformers import AutoTokenizer
+random.seed(3); tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")
+MD = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; CAP = {"blender": 6000, "cadquery": 10000, "openscad": 2000, "glsl": 10000}
+mixed = []
+for d, cap in CAP.items():
+ rows = [json.loads(l) for l in open(f"{MD}/{d}/train.jsonl")]; random.shuffle(rows)
+ keep = []
+ for r in rows:
+ if len(keep) >= cap: break
+ n = len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)
+ if n <= 8192: keep.append(r)
+ out = f"/wekafs/ict/hx_624/llm-ft/data/md_{d}"; os.makedirs(out, exist_ok=True)
+ nv = max(20, len(keep) // 50); val, train = keep[:nv], keep[nv:]
+ for split, rs in [("train", train), ("val", val)]:
+ with open(f"{out}/{split}.jsonl", "w") as f:
+ for r in rs: f.write(json.dumps(r, ensure_ascii=False) + "\n")
+ print(d, "train", len(train), "val", len(val)); mixed += train
+random.shuffle(mixed); out = "/wekafs/ict/hx_624/llm-ft/data/md_mixed"; os.makedirs(out, exist_ok=True)
+with open(f"{out}/train.jsonl", "w") as f:
+ for r in mixed: f.write(json.dumps(r, ensure_ascii=False) + "\n")
+# mixed val = concat of per-dialect vals
+with open(f"{out}/val.jsonl", "w") as f:
+ for d in CAP:
+ for l in open(f"/wekafs/ict/hx_624/llm-ft/data/md_{d}/val.jsonl"): f.write(l)
+print("mixed train", len(mixed))
diff --git a/finetune/scripts/build_report_html.py b/finetune/scripts/build_report_html.py
new file mode 100644
index 00000000..514d441d
--- /dev/null
+++ b/finetune/scripts/build_report_html.py
@@ -0,0 +1,62 @@
+"""Render REPORT.md into a self-contained, themed HTML page (for publishing as an Artifact)."""
+import markdown, re, datetime, sys
+SRC = "/wekafs/ict/hx_624/llm-ft/REPORT.md"; DST = "/wekafs/ict/hx_624/llm-ft/REPORT.html"
+md = open(SRC).read()
+body = markdown.markdown(md, extensions=["tables", "fenced_code", "toc"], extension_configs={"toc": {"toc_depth": "2-3"}})
+toc = re.search(r'<div class="toc">(.*?)</div>', body, re.S)
+toc_html = toc.group(1) if toc else ""
+body = body.replace(toc.group(0), "") if toc else body
+css = """
+:root{--bg:#f6f4ef;--paper:#fffdf8;--ink:#1f2a2e;--muted:#5b6a70;--line:#d9d4c7;--accent:#0e7c7b;--accent-2:#b5541c;--code:#eef2f1;--head:#13343b}
+@media (prefers-color-scheme: dark){:root:not([data-theme="light"]){--bg:#15191b;--paper:#1c2124;--ink:#e6e2d8;--muted:#a3ada9;--line:#2f373a;--accent:#5ec8c5;--accent-2:#e59a62;--code:#242b2e;--head:#dfe9e7}}
+:root[data-theme="dark"]{--bg:#15191b;--paper:#1c2124;--ink:#e6e2d8;--muted:#a3ada9;--line:#2f373a;--accent:#5ec8c5;--accent-2:#e59a62;--code:#242b2e;--head:#dfe9e7}
+*{box-sizing:border-box}
+body{margin:0;background:var(--bg);color:var(--ink);font-family:"IBM Plex Sans","Noto Sans SC","PingFang SC","Microsoft YaHei",system-ui,sans-serif;font-size:16px;line-height:1.7}
+.wrap{display:grid;grid-template-columns:minmax(0,1fr) 260px;gap:40px;max-width:1180px;margin:0 auto;padding:40px 28px 80px}
+@media (max-width:900px){.wrap{grid-template-columns:1fr}.toc{display:none}}
+main{min-width:0;background:var(--paper);border:1px solid var(--line);border-radius:6px;padding:40px 44px}
+@media (max-width:600px){main{padding:22px 18px}}
+header.hero{margin-bottom:28px;border-bottom:2px solid var(--accent);padding-bottom:18px}
+header.hero .eyebrow{font-family:"IBM Plex Mono",ui-monospace,monospace;font-size:12px;letter-spacing:.12em;text-transform:uppercase;color:var(--accent-2)}
+h1,h2,h3,h4{color:var(--head);text-wrap:balance;line-height:1.25;font-family:"IBM Plex Sans","Noto Sans SC","PingFang SC",sans-serif}
+h1{font-size:30px;margin:6px 0 8px;font-weight:600}
+h2{font-size:22px;margin:44px 0 12px;padding-top:14px;border-top:1px solid var(--line);font-weight:600}
+h3{font-size:17px;margin:28px 0 8px;font-weight:600}
+p,li{max-width:78ch}
+blockquote{margin:14px 0;padding:10px 16px;border-left:3px solid var(--accent);background:var(--code);color:var(--muted);border-radius:0 4px 4px 0}
+code,pre{font-family:"IBM Plex Mono",ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.88em}
+code{background:var(--code);padding:1px 5px;border-radius:3px}
+pre{background:var(--code);padding:12px 14px;border-radius:4px;overflow-x:auto}
+pre code{background:none;padding:0}
+.tbl{overflow-x:auto;margin:14px 0}
+table{border-collapse:collapse;width:100%;font-size:14px;font-variant-numeric:tabular-nums}
+th,td{border-bottom:1px solid var(--line);padding:7px 10px;text-align:left;vertical-align:top}
+th{color:var(--muted);font-weight:600;font-size:12.5px;letter-spacing:.04em;text-transform:uppercase;background:var(--code)}
+tr:hover td{background:color-mix(in srgb,var(--accent) 6%,transparent)}
+strong{color:var(--head)}
+a{color:var(--accent)}
+.toc{position:sticky;top:24px;align-self:start;font-size:13.5px;line-height:1.5}
+.toc ul{list-style:none;padding-left:0;margin:0}
+.toc ul ul{padding-left:12px;margin-top:2px}
+.toc li{margin:4px 0}
+.toc a{color:var(--muted);text-decoration:none}
+.toc a:hover{color:var(--accent)}
+.toc .label{font-family:"IBM Plex Mono",monospace;font-size:11px;letter-spacing:.12em;text-transform:uppercase;color:var(--accent-2);margin-bottom:8px}
+footer{margin-top:40px;color:var(--muted);font-size:13px}
+"""
+body = re.sub(r"<table>", '<div class="tbl"><table>', body); body = re.sub(r"</table>", "</table></div>", body)
+body = re.sub(r"<h1[^>]*>.*?</h1>", "", body, count=1)
+html = f"""<title>3D-code LLM Finetune Lab Notes</title>
+<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=IBM+Plex+Sans:wght@400;500;600&family=IBM+Plex+Mono:wght@400;500&family=Noto+Sans+SC:wght@400;500;700&display=swap">
+<style>{css}</style>
+<div class="wrap">
+<main>
+<header class="hero"><div class="eyebrow">dgx03 · 4×H100 · {datetime.date.today().isoformat()}</div>
+<h1>3D-code LLM finetune 调研报告</h1>
+<p style="color:var(--muted);margin:0">Qwen3.5-9B × LLaMA-Factory × 3DCodeVerse → 3DCodeBench(Blender 5.0 执行评测)。工程目录 <code>/wekafs/ict/hx_624/llm-ft</code>。</p></header>
+{body}
+<footer>Generated from <code>llm-ft/REPORT.md</code> · all numbers reproducible via <code>eval/compare.py</code> / <code>eval/results_table.py</code>.</footer>
+</main>
+<nav class="toc"><div class="label">Contents</div>{toc_html}</nav>
+</div>"""
+open(DST, "w").write(html); print("wrote", DST, len(html), "bytes")
diff --git a/finetune/scripts/build_variants.sh b/finetune/scripts/build_variants.sh
new file mode 100755
index 00000000..bcebf979
--- /dev/null
+++ b/finetune/scripts/build_variants.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+set -x
+source /wekafs/ict/hx_624/llm-ft/env.sh; cd /wekafs/ict/hx_624/llm-ft
+python scripts/prep_data.py --out data/sft_v2_indist --include_test_factories > logs/prep_v2.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v2_indist --name v2_indist
+python scripts/prep_data.py --out data/sft_v3_bio --sources bio3d > logs/prep_v3.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v3_bio --name v3_bio
+python scripts/prep_data.py --out data/sft_v4_cq --sources bio3d,fac,distill,thingi50,deepcad --deepcad_n 6000 > logs/prep_v4.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v4_cq --name v4_cq
+python scripts/prep_data.py --out data/sft_v5_instr --prompt_fields instruction:1 > logs/prep_v5.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v5_instr --name v5_instr
+python scripts/prep_data.py --out data/sft_v6_detail --prompt_fields detailed:1 > logs/prep_v6.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v6_detail --name v6_detail
+for v in v2_indist v3_bio v4_cq v5_instr v6_detail; do python scripts/make_lora_cfg.py $v $v; done
+for v in v2_indist v3_bio v4_cq v5_instr v6_detail; do echo "== $v"; grep -E '"n_train"|"n_val"' data/sft_$v/stats.json | tr -d '\n'; echo; done
+echo BUILD_VARIANTS_DONE
diff --git a/finetune/scripts/cache_ntok.py b/finetune/scripts/cache_ntok.py
new file mode 100644
index 00000000..f944764f
--- /dev/null
+++ b/finetune/scripts/cache_ntok.py
@@ -0,0 +1,20 @@
+"""One-off: cache per-sample token counts (Qwen3.5 chat template) for every multidialect train/test file and the bootstrapped set.
+Writes <file>.ntok (one int per line, same order). Build scripts read this instead of re-tokenizing."""
+import json, os, sys, glob
+from multiprocessing import Pool
+FILES = sorted(glob.glob("/wekafs/ict/hx_624/llm-ft/data/multidialect/*/train.jsonl") + glob.glob("/wekafs/ict/hx_624/llm-ft/data/multidialect/*/test.jsonl")) + ["/wekafs/ict/hx_624/llm-ft/data/sft_v13_boot/train.jsonl"]
+tok = None
+def init():
+ global tok
+ from transformers import AutoTokenizer
+ tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")
+def count(line):
+ r = json.loads(line); return len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)
+if __name__ == "__main__":
+ with Pool(12, initializer=init) as pool:
+ for f in FILES:
+ out = f + ".ntok"
+ if os.path.exists(out) and sum(1 for _ in open(out)) == sum(1 for _ in open(f)): print("cached", f); continue
+ lines = open(f).read().splitlines()
+ ns = pool.map(count, lines, chunksize=256)
+ open(out, "w").write("\n".join(map(str, ns)) + "\n"); print(f, len(ns), "done", flush=True)
diff --git a/finetune/scripts/ckpt_sweep.sh b/finetune/scripts/ckpt_sweep.sh
new file mode 100755
index 00000000..55a3cb6f
--- /dev/null
+++ b/finetune/scripts/ckpt_sweep.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+# ckpt_sweep.sh <GPU> <RUN_DIR> <NAME> <ckpt_step> [<ckpt_step> ...]: merge each intermediate adapter checkpoint, vLLM-generate, exec in background
+GPU=$1; RUN=$(readlink -f "$2"); NAME=$3; shift 3
+cd /wekafs/ict/hx_624/llm-ft
+for STEP in "$@"; do
+ CK=$RUN/checkpoint-$STEP; [ -d $CK ] || { echo "[ckpt $NAME-$STEP] missing"; continue; }
+ OUT=eval/out/qwen35_9b_lora_${NAME}_ckpt$STEP
+ sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $CK|" -e "s|export_dir:.*|export_dir: $CK/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_${NAME}_ckpt$STEP.yaml
+ scripts/lf_export.sh configs/lf/export_${NAME}_ckpt$STEP.yaml > logs/export_${NAME}_ckpt$STEP.log 2>&1 || { echo "[ckpt $NAME-$STEP] EXPORT FAILED"; continue; }
+ ( source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU
+ mkdir -p $OUT; python eval/generate_vllm.py --model $CK/merged --prompts data/sft_v1/bench_prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 6144 2>&1 | grep -E "gen-vllm|Error|Traceback" | tail -3 ) > logs/eval_${NAME}_ckpt$STEP.log 2>&1
+ cp $OUT/gens_shard0.jsonl $OUT/gens.jsonl; rm -rf $CK/merged
+ echo "[ckpt $NAME-$STEP] gen done: $(grep gen-vllm logs/eval_${NAME}_ckpt$STEP.log | tail -1)"
+ nohup bash -c "source /wekafs/ict/hx_624/llm-ft/env.sh; cd /wekafs/ict/hx_624/llm-ft; python eval/run_bench.py --gen_dir $OUT --workers 24 --timeout 300 > $OUT/exec.log 2>&1; python eval/metrics.py --gen_dir $OUT > $OUT/metrics.log 2>&1; echo \"[ckpt $NAME-$STEP] eval done\"; cat $OUT/summary.json" >> logs/ckpt_sweep_$NAME.log 2>&1 &
+done
+echo "CKPT_SWEEP_DONE $NAME"
diff --git a/finetune/scripts/contact_sheet_3dcodebench.py b/finetune/scripts/contact_sheet_3dcodebench.py
new file mode 100644
index 00000000..4886f710
--- /dev/null
+++ b/finetune/scripts/contact_sheet_3dcodebench.py
@@ -0,0 +1,33 @@
+"""Render generated vs GT for a spread of 3DCodeBench tasks and compose a labelled contact sheet.
+usage: python scripts/contact_sheet_3dcodebench.py <eval_out_dir> <out_png> [n_per_bucket]"""
+import json, os, sys, subprocess, random
+from concurrent.futures import ThreadPoolExecutor
+from PIL import Image, ImageDraw, ImageFont
+BL = "/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; R = os.path.abspath("eval/render_glb.py"); GT = "/wekafs/ict/hx_624/data/3dcodebench/data"
+gen_dir, out_png = sys.argv[1], sys.argv[2]; k = int(sys.argv[3]) if len(sys.argv) > 3 else 4
+m = [json.loads(l) for l in open(f"{gen_dir}/metrics.jsonl")]; ok = sorted([r for r in m if r["exec"] == "OK" and r.get("[email protected]") is not None], key=lambda r: -r["[email protected]"])
+random.seed(3); n = len(ok)
+sel = [("best", r) for r in ok[:k]] + [("median", r) for r in ok[n//2 - k//2: n//2 - k//2 + k]] + [("worst", r) for r in ok[-k:]] + [("random", r) for r in random.sample(ok[k:-k], k)]
+work = os.path.abspath(f"{gen_dir}/_renders"); os.makedirs(work, exist_ok=True)
+def render(glb, png):
+ if os.path.exists(png): return
+ subprocess.run([BL, "-b", "--factory-startup", "-noaudio", "--python", R, "--", "--glb", glb, "--out", png], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=300, env={**os.environ, "HOME": "/tmp"})
+jobs = []
+for tag, r in sel:
+ t = r["task"]; jobs.append((os.path.abspath(f"{gen_dir}/{t}/out.glb"), f"{work}/{t}_gen.png")); jobs.append((f"{GT}/{t}/glb/{t}.glb", f"{work}/{t}_gt.png"))
+with ThreadPoolExecutor(8) as ex: list(ex.map(lambda j: render(*j), jobs))
+S = 200; cols = k; rows = 4
+sheet = Image.new("RGB", (cols * (2 * S + 16) + 40, rows * (S + 52) + 10), "white"); d = ImageDraw.Draw(sheet)
+try: font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", 13); fb = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf", 13)
+except Exception: font = fb = ImageFont.load_default()
+for i, (tag, r) in enumerate(sel):
+ row, col = i // cols, i % cols; x0 = 40 + col * (2 * S + 16); y0 = 10 + row * (S + 52); t = r["task"]
+ for j, kind in enumerate(("gen", "gt")):
+ p = f"{work}/{t}_{kind}.png"
+ try: im = Image.open(p).convert("RGB").resize((S, S))
+ except Exception: im = Image.new("RGB", (S, S), (230, 230, 230))
+ sheet.paste(im, (x0 + j * S, y0 + 20))
+ d.text((x0, y0 + 2), f"{t.replace('_seed0','')} [email protected]={r['[email protected]']:.2f}", fill="black", font=fb)
+ d.text((x0, y0 + S + 24), "generated (md_xl)", fill=(60, 60, 60), font=font); d.text((x0 + S, y0 + S + 24), "ground truth", fill=(60, 60, 60), font=font)
+ if col == 0: d.text((4, y0 + S // 2), tag, fill=(120, 0, 0), font=fb)
+sheet.save(out_png); print("wrote", out_png, sheet.size, "| tasks:", [r["task"] for _, r in sel])
diff --git a/finetune/scripts/desc_prompt_eval.sh b/finetune/scripts/desc_prompt_eval.sh
new file mode 100755
index 00000000..873be1f1
--- /dev/null
+++ b/finetune/scripts/desc_prompt_eval.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+# Cross prompt-style eval: 3DCodeBench prompt_description.txt (appearance descriptions) instead of prompt_instruction.txt, for base / v1 / v6_detail
+set -uo pipefail; cd /wekafs/ict/hx_624/llm-ft; export PROMPTS=/wekafs/ict/hx_624/llm-ft/data/sft_v1/bench_prompts_desc.jsonl
+until grep -q "\[dpo\] DONE\|FAILED" logs/exp_dpo_exec_v1.log 2>/dev/null; do sleep 30; done
+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 2,3 | sort -n | tail -1)" -lt 10000 ]; do sleep 20; done
+echo "[desc] $(date) start"
+[ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_desc.log 2>&1
+GPUS=2 eval/run_eval_vllm.sh runs/lf_qwen35_9b_lora_v1/merged eval/out/desc_qwen35_9b_lora_v1 --no_think --max_new_tokens 6144 > logs/eval_desc_v1.log 2>&1 &
+GPUS=3 eval/run_eval_vllm.sh /wekafs/ict/hx_624/models/Qwen3.5-9B eval/out/desc_qwen35_9b_base --no_think --max_new_tokens 6144 > logs/eval_desc_base.log 2>&1 &
+wait; echo "[desc] v1/base done"; cat eval/out/desc_qwen35_9b_lora_v1/summary.json | tr -d '\n'; echo; cat eval/out/desc_qwen35_9b_base/summary.json | tr -d '\n'; echo
+scripts/lf_export.sh configs/lf/export_v6_detail.yaml > logs/export_v6_desc.log 2>&1 && GPUS=2 eval/run_eval_vllm.sh runs/lf_qwen35_9b_lora_v6_detail/merged eval/out/desc_qwen35_9b_lora_v6_detail --no_think --max_new_tokens 6144 > logs/eval_desc_v6.log 2>&1; echo "[desc] v6 done"; cat eval/out/desc_qwen35_9b_lora_v6_detail/summary.json | tr -d '\n'; echo
+rm -rf runs/lf_qwen35_9b_lora_v6_detail/merged runs/lf_qwen35_9b_lora_v1/merged; echo "[desc] DONE"
diff --git a/finetune/scripts/dpo_round2.sh b/finetune/scripts/dpo_round2.sh
new file mode 100755
index 00000000..356ef164
--- /dev/null
+++ b/finetune/scripts/dpo_round2.sh
@@ -0,0 +1,43 @@
+#!/bin/bash
+# Iterative DPO round 2: DPO_v1 model samples K=2 on the 9.9k novel prompts -> Blender filter -> new (OK, FAIL) pairs -> DPO on top of DPO_v1 -> eval
+set -uo pipefail; GPU=${GPUS:-1}; cd /wekafs/ict/hx_624/llm-ft; W=data/dpo_round2; mkdir -p $W
+echo "[dpo2] $(date) start"
+[ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_dpo2.log 2>&1
+BASE=runs/lf_qwen35_9b_dpo_exec_v1/merged_r2
+sed -e "s|export_dir:.*|export_dir: $(readlink -f runs/lf_qwen35_9b_dpo_exec_v1)/merged_r2|" configs/lf/export_dpo_exec_v1.yaml > configs/lf/export_dpo_exec_v1_r2.yaml
+[ -d $BASE ] || scripts/lf_export.sh configs/lf/export_dpo_exec_v1_r2.yaml > logs/export_dpo_r2.log 2>&1 || { echo "[dpo2] EXPORT FAILED"; exit 1; }
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU
+for i in 1 2; do python eval/generate_vllm.py --model $BASE --prompts data/selftrain_round2/prompts.jsonl --out $W/gen_s$i --tp 1 --no_think --max_new_tokens 4096 --temperature 0.8 --seed $((300+i)) 2>&1 | grep -E "gen-vllm|Error" | tail -1; done
+echo "[dpo2] $(date) sampling done"
+conda activate llmft
+for i in 1 2; do python eval/run_bench.py --gen_dir $W/gen_s$i --workers 56 --timeout 120 > $W/gen_s$i/exec.log 2>&1; grep "status counts" $W/gen_s$i/exec.log; done
+python - <<'PY'
+import json, glob, os, random
+random.seed(1); W="data/dpo_round2"
+prompts={}
+for l in open("data/selftrain_round2/prompts.jsonl"): r=json.loads(l); prompts[r["task"]]=r
+by={}
+for d in sorted(glob.glob(f"{W}/gen_s*/")):
+ for l in open(d+"exec_results.jsonl"):
+ r=json.loads(l); code=open(f"{d}/{r['task']}/code.py").read().strip()
+ if code.count("\n")<3: continue
+ by.setdefault(r["task"],{"ok":[],"bad":[]})["ok" if r["status"]=="OK" else "bad"].append(code)
+pairs=[]
+for t,v in by.items():
+ if v["ok"] and v["bad"]:
+ ok=random.choice(v["ok"]); bad=random.choice(v["bad"])
+ if ok==bad: continue
+ p=prompts[t]; pairs.append({"conversations":[{"from":"human","value":p["messages"][1]["content"]}],"system":p["messages"][0]["content"],"chosen":{"from":"gpt","value":"```python\n"+ok+"\n```"},"rejected":{"from":"gpt","value":"```python\n"+bad+"\n```"}})
+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_exec_pairs_r2.json","w"), ensure_ascii=False)
+info=json.load(open("data/lf/dataset_info.json")); info["dpo_exec_pairs_r2"]={"file_name":"dpo_exec_pairs_r2.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}
+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2); print("round2 pairs:", len(pairs), "prompts with any OK:", sum(1 for v in by.values() if v["ok"]), "/", len(by))
+PY
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_exec_pairs_r2|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_exec_v2.yaml
+echo "[dpo2] $(date) train"; GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_exec_v2.yaml gradient_accumulation_steps=16 > logs/train_dpo_exec_v2.log 2>&1
+RUN=runs/lf_qwen35_9b_dpo_exec_v2; [ -f $RUN/train_results.json ] || { echo "[dpo2] TRAIN FAILED"; exit 1; }
+echo "[dpo2] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_exec_v2.yaml
+scripts/lf_export.sh configs/lf/export_dpo_exec_v2.yaml > logs/export_dpo_exec_v2.log 2>&1 || { echo "[dpo2] EXPORT2 FAILED"; exit 1; }
+GPUS=$GPU eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_exec_v2 --no_think --max_new_tokens 6144 > logs/eval_dpo_exec_v2.log 2>&1
+echo "[dpo2] $(date) eval done"; cat eval/out/qwen35_9b_dpo_exec_v2/summary.json; rm -rf $RUN/merged $BASE; echo "[dpo2] DONE"
diff --git a/finetune/scripts/explore_sched.sh b/finetune/scripts/explore_sched.sh
new file mode 100755
index 00000000..c758f847
--- /dev/null
+++ b/finetune/scripts/explore_sched.sh
@@ -0,0 +1,38 @@
+#!/bin/bash
+# 8h exploration scheduler: (1) after md_xl sweep + DPO finish -> md_bal LoRA on 4 GPUs (train+evals)
+# (2) right after md_bal 'train done' -> full FT on the same md_bal data on GPUs 1,2,3 (ZeRO-3) (3) after md_bal evals free GPU 0 -> geometry-feedback DPO train+eval on GPU 0
+set -uo pipefail; cd /wekafs/ict/hx_624/llm-ft
+until grep -q "\[bigsweep\] DONE" logs/md_xl_ckpt_sweep.log 2>/dev/null; do sleep 60; done
+until grep -qE "\[xldpo\] (DONE|TRAIN FAILED|EXPORT.* FAILED)" logs/md_xl_dpo_round.log 2>/dev/null; do sleep 60; done
+echo "[sched] $(date) md_bal LoRA on GPUs 0,1,2,3"
+scripts/run_md_exp.sh 0,1,2,3 md_bal > logs/exp_md_bal.log 2>&1 &
+until grep -qE "\[md md_bal\] .*train done|TRAIN FAILED" logs/exp_md_bal.log 2>/dev/null; do sleep 60; done
+sleep 60
+echo "[sched] $(date) full FT md_bal on GPUs 1,2,3 (ZeRO-3, accum 3)"
+( GPUS=1,2,3 scripts/lf_train.sh configs/lf/full_md_bal.yaml gradient_accumulation_steps=3 > logs/train_full_md_bal.log 2>&1
+ RUN=runs/lf_qwen35_9b_full_md_bal; [ -f $RUN/train_results.json ] || { echo "[sched] FULL TRAIN FAILED"; exit 1; }
+ echo "[sched] $(date) full md_bal train done: $(tr -d '\n' < $RUN/train_results.json | cut -c1-200)"
+ until grep -q "\[md md_bal\] DONE" logs/exp_md_bal.log 2>/dev/null; do sleep 60; done # wait until GPU 1 is surely free of md_bal evals? (evals run on GPU 0) -> use GPU 1
+ GPUS=1 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_md_bal --no_think --max_new_tokens 6144 > logs/eval_full_md_bal.log 2>&1
+ echo "[sched] full md_bal 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_full_md_bal/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"
+ GPUS=1 eval/run_dialect_eval.sh $RUN full_md_bal > logs/mdeval_full_md_bal.log 2>&1
+ for D in cadquery openscad glsl blender; do echo "[sched] full md_bal $D: $(cat eval/out/md_full_md_bal_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done
+ echo "[sched] FULL_DONE" ) &
+until grep -q "\[md md_bal\] DONE" logs/exp_md_bal.log 2>/dev/null; do sleep 60; done
+until grep -q "GEO_PAIRS_READY" logs/geo_dpo_build.log 2>/dev/null; do sleep 60; done
+echo "[sched] $(date) geometry-feedback DPO on GPU 0"
+GPUS=0 scripts/lf_train.sh configs/lf/dpo_geo_md_xl.yaml gradient_accumulation_steps=8 cutoff_len=4096 > logs/train_dpo_geo_md_xl.log 2>&1
+RUN=runs/lf_qwen35_9b_dpo_geo_md_xl; [ -f $RUN/train_results.json ] || { echo "[sched] GEO DPO TRAIN FAILED"; grep -E "Error" logs/train_dpo_geo_md_xl.log | grep -v errors | tail -2 | cut -c1-200; }
+if [ -f $RUN/train_results.json ]; then
+ echo "[sched] $(date) geo dpo train done: $(tr -d '\n' < $RUN/train_results.json | cut -c1-200)"
+ BASE=runs/lf_qwen35_9b_lora_md_xl/merged_geo
+ sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_geo_md_xl.yaml
+ scripts/lf_export.sh configs/lf/export_dpo_geo_md_xl.yaml > logs/export_dpo_geo_md_xl.log 2>&1
+ GPUS=0 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_geo_md_xl --no_think --max_new_tokens 6144 > logs/eval_dpo_geo_md_xl.log 2>&1
+ echo "[sched] geo dpo 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_geo_md_xl/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3), 'scored', round(s['[email protected]_mean_scored'],3))")"
+ GPUS=0 eval/run_dialect_eval.sh $RUN/merged dpo_geo_md_xl blender cadquery > logs/mdeval_dpo_geo_md_xl.log 2>&1
+ for D in blender cadquery; do echo "[sched] geo dpo $D: $(cat eval/out/md_dpo_geo_md_xl_$D/summary.json | tr -d '\n ' | cut -c1-200)"; done
+ GPUS=0 eval/best_of_n.sh $RUN/merged dpo_geo_md_xl_T07 4 0.7 2>&1 | grep -E "pass@" | tail -1 | sed 's/^/[sched] geo dpo /'
+fi
+echo "[sched] GEO_DONE"
+wait; echo "[sched] ALL DONE"
diff --git a/finetune/scripts/fetch_hf_files.py b/finetune/scripts/fetch_hf_files.py
new file mode 100644
index 00000000..a370f5b3
--- /dev/null
+++ b/finetune/scripts/fetch_hf_files.py
@@ -0,0 +1,30 @@
+"""Download only files matching a regex from an HF dataset repo (handles tree pagination, no file locks).
+usage: python fetch_hf_files.py REPO OUTDIR REGEX [--subdir SUB] [--workers 32]"""
+import argparse, os, re, sys, requests
+from concurrent.futures import ThreadPoolExecutor
+from huggingface_hub import HfApi
+ap = argparse.ArgumentParser(); ap.add_argument("repo"); ap.add_argument("out"); ap.add_argument("regex")
+ap.add_argument("--subdir", default=None); ap.add_argument("--workers", type=int, default=32)
+a = ap.parse_args()
+tok = os.environ["HF_TOKEN"]; api = HfApi(token=tok); pat = re.compile(a.regex)
+files = [f.path for f in api.list_repo_tree(a.repo, repo_type="dataset", path_in_repo=a.subdir, recursive=True) if getattr(f, "size", None) is not None and pat.search(f.path)]
+print(f"[fetch] {len(files)} files match", flush=True)
+sess = requests.Session(); sess.headers["Authorization"] = f"Bearer {tok}"
+def get(p):
+ dst = os.path.join(a.out, p)
+ if os.path.exists(dst) and os.path.getsize(dst) > 0: return 0
+ os.makedirs(os.path.dirname(dst), exist_ok=True)
+ for attempt in range(3):
+ try:
+ r = sess.get(f"https://huggingface.co/datasets/{a.repo}/resolve/main/{p}", timeout=60)
+ if r.status_code == 200:
+ open(dst, "wb").write(r.content); return 1
+ except Exception as e:
+ pass
+ print("FAIL", p, flush=True); return -1
+n = 0
+with ThreadPoolExecutor(a.workers) as ex:
+ for i, r in enumerate(ex.map(get, files)):
+ n += (r == 1)
+ if (i + 1) % 500 == 0: print(f"[fetch] {i+1}/{len(files)}", flush=True)
+print(f"[fetch] done: {n} downloaded, {len(files)} total", flush=True)
diff --git a/finetune/scripts/fetch_textfiles.sh b/finetune/scripts/fetch_textfiles.sh
new file mode 100755
index 00000000..e5caf550
--- /dev/null
+++ b/finetune/scripts/fetch_textfiles.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+# Fetch only small text files (code/captions/meta/reports) of per-sample HF dataset folders with parallel curl.
+# usage: fetch_textfiles.sh <repo_id> <subpath-or-empty> <outdir> <regex-of-files-to-keep>
+REPO=$1; SUB=$2; OUT=$3; PAT=$4
+mkdir -p "$OUT"; cd "$OUT" || exit 1
+curl -s -H "Authorization: Bearer $HF_TOKEN" "https://huggingface.co/api/datasets/$REPO/tree/main/$SUB?recursive=true" \
+ | python3 -c "import sys,json,re; pat=re.compile(sys.argv[1]); [print(x['path']) for x in json.load(sys.stdin) if x['type']=='file' and pat.search(x['path'])]" "$PAT" > .filelist
+echo "files to fetch: $(wc -l < .filelist)"
+cat .filelist | xargs -P 32 -I{} bash -c 'f="{}"; mkdir -p "$(dirname "$f")"; [ -s "$f" ] || curl -sS -L -H "Authorization: Bearer $HF_TOKEN" -o "$f" "https://huggingface.co/datasets/'$REPO'/resolve/main/$f" || echo "FAIL $f"'
+echo "FETCH_DONE $(find . -type f ! -name .filelist | wc -l) files"
diff --git a/finetune/scripts/geo_dpo_build.sh b/finetune/scripts/geo_dpo_build.sh
new file mode 100755
index 00000000..12b071b5
--- /dev/null
+++ b/finetune/scripts/geo_dpo_build.sh
@@ -0,0 +1,50 @@
+#!/bin/bash
+# Geometry-feedback DPO data: sample md_xl (merged_geo) K times on train prompts that have reference code -> execute -> [email protected] vs reference mesh -> (high-F, low-F/FAIL) pairs
+set -uo pipefail; cd /wekafs/ict/hx_624/llm-ft; G=${GPUS:-0}; W=data/geo_dpo; M=runs/lf_qwen35_9b_lora_md_xl/merged_geo
+until [ -f $M/config.json ] && [ -n "$(ls $M/*.safetensors 2>/dev/null)" ] && [ $(( $(date +%s) - $(stat -c %Y logs/export_md_xl_geo.log) )) -gt 90 ]; do sleep 30; done
+until grep -q "\[xlextra\] DONE" logs/md_xl_extra_eval.log 2>/dev/null; do sleep 30; done
+echo "[geo] $(date) sampling on GPU $G"
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$G
+for i in 1 2 3 4; do python eval/generate_vllm.py --model $M --prompts $W/blender_vllm_prompts.jsonl --out $W/gen_blender_s$i --tp 1 --no_think --max_new_tokens 4096 --temperature 0.8 --seed $((500+i)) 2>&1 | grep -E "gen-vllm|Error" | tail -1; done
+for i in 1 2; do python eval/generate_vllm.py --model $M --prompts $W/cadquery_vllm_prompts.jsonl --out $W/gen_cadquery_s$i --tp 1 --no_think --max_new_tokens 4096 --temperature 0.8 --seed $((600+i)) 2>&1 | grep -E "gen-vllm|Error" | tail -1; done
+echo "[geo] $(date) sampling done; executing + scoring"
+conda activate llmft
+for i in 1 2 3 4; do python eval/blender_dialect_eval.py --test $W/blender_prompts.jsonl --gen_dir $W/gen_blender_s$i --workers 40 2>&1 | tail -1 | cut -c1-200; done
+for i in 1 2; do python eval/dialect_eval.py --dialect cadquery --test $W/cadquery_prompts.jsonl --gen_dir $W/gen_cadquery_s$i --workers 32 2>&1 | tail -1 | cut -c1-200; done
+python - <<'PY'
+import json, os, re, random, glob
+from collections import Counter, defaultdict
+W="data/geo_dpo"; random.seed(7)
+def code_of(d, tid):
+ p=f"{d}/{tid}/code.py"; return open(p).read().strip() if os.path.exists(p) else None
+prompts={}
+for dia in ["blender","cadquery"]:
+ for l in open(f"{W}/{dia}_prompts.jsonl"):
+ r=json.loads(l); prompts[r["id"].replace("/","__")]=(dia, r["messages"][0]["content"], r["messages"][1]["content"])
+by=defaultdict(list) # tid -> [(F or None, status, code)]
+for d in sorted(glob.glob(f"{W}/gen_*_s*")):
+ mp=f"{d}/dialect_metrics.jsonl"
+ if not os.path.exists(mp): continue
+ for l in open(mp):
+ r=json.loads(l); tid=r["id"]; code=code_of(d, tid)
+ if not code or code.count("\n")<3: continue
+ by[tid].append((r.get("[email protected]"), r.get("gen"), code))
+pairs=[]; kinds=Counter(); per=Counter()
+for tid, lst in by.items():
+ dia, sys_, user=prompts[tid]; ok=[x for x in lst if x[1]=="OK" and x[0] is not None]; bad=[x for x in lst if x[1]!="OK"]
+ if not ok: continue
+ best=max(ok, key=lambda x:x[0]); worst=min(ok, key=lambda x:x[0])
+ def mk(ch, rj, kind):
+ if ch==rj: return
+ pairs.append({"conversations":[{"from":"human","value":user}],"system":sys_,"chosen":{"from":"gpt","value":ch if ch.startswith("```") else "```\n"+ch+"\n```"},"rejected":{"from":"gpt","value":rj if rj.startswith("```") else "```\n"+rj+"\n```"}}); kinds[kind]+=1; per[dia]+=1
+ if best[0]>=0.25 and len(ok)>=2 and best[0]-worst[0]>=0.15: mk(best[2], worst[2], "geo")
+ if best[0]>=0.25 and bad: mk(best[2], random.choice(bad)[2], "exec")
+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_geo_pairs.json","w"), ensure_ascii=False)
+info=json.load(open("data/lf/dataset_info.json")); info["dpo_geo_pairs"]={"file_name":"dpo_geo_pairs.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}
+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2)
+okF=[x[0] for l in by.values() for x in l if x[1]=="OK" and x[0] is not None]
+print("geo pairs:", len(pairs), dict(kinds), dict(per), "| prompts with samples:", len(by), "| mean F of OK samples %.3f" % (sum(okF)/max(1,len(okF))))
+PY
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $M)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_geo_pairs|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_geo_md_xl|" -e "s|overwrite_output_dir:.*|overwrite_output_dir: true|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_geo_md_xl.yaml
+echo "GEO_PAIRS_READY"
diff --git a/finetune/scripts/launch_sft.sh b/finetune/scripts/launch_sft.sh
new file mode 100755
index 00000000..3b37a0c1
--- /dev/null
+++ b/finetune/scripts/launch_sft.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+# usage: GPUS=0,1 ./launch_sft.sh <run_name> [extra train_sft.py args...]
+# Runs torchrun on the given GPUs (must be a subset of 0-3 per the user's rule).
+set -euo pipefail
+source /wekafs/ict/hx_624/llm-ft/env.sh
+export CUDA_VISIBLE_DEVICES=${GPUS:-0,1}
+RUN=$1; shift
+NGPU=$(echo $CUDA_VISIBLE_DEVICES | tr ',' '\n' | wc -l)
+OUT=/wekafs/ict/hx_624/llm-ft/runs/$RUN
+mkdir -p $OUT
+PORT=$((29500 + RANDOM % 500))
+echo "[launch] run=$RUN gpus=$CUDA_VISIBLE_DEVICES ngpu=$NGPU port=$PORT out=$OUT"
+cd /wekafs/ict/hx_624/llm-ft
+torchrun --nproc_per_node=$NGPU --master_port=$PORT scripts/train_sft.py --output_dir $OUT "$@" 2>&1 | tee -a $OUT/train.log
diff --git a/finetune/scripts/lf_export.sh b/finetune/scripts/lf_export.sh
new file mode 100755
index 00000000..421e3acd
--- /dev/null
+++ b/finetune/scripts/lf_export.sh
@@ -0,0 +1,6 @@
+#!/bin/bash
+# usage: ./lf_export.sh <export_yaml>
+set -euo pipefail
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate lf
+export HF_HOME=/wekafs/ict/hx_624/cache/huggingface DISABLE_VERSION_CHECK=1 CUDA_VISIBLE_DEVICES=""
+CFG=$(readlink -f "$1"); cd /wekafs/ict/hx_624/tools/LLaMA-Factory && llamafactory-cli export "$CFG"
diff --git a/finetune/scripts/lf_train.sh b/finetune/scripts/lf_train.sh
new file mode 100755
index 00000000..b604b4b8
--- /dev/null
+++ b/finetune/scripts/lf_train.sh
@@ -0,0 +1,19 @@
+#!/bin/bash
+# usage: GPUS=0,1,2 ./lf_train.sh configs/lf/<cfg>.yaml [extra llamafactory-cli overrides: key=value ...]
+set -euo pipefail
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate lf
+source /wekafs/ict/hx_624/.secrets/hf.env
+export WORKSPACE=/wekafs/ict/hx_624 HF_HOME=/wekafs/ict/hx_624/cache/huggingface HF_XET_HIGH_PERFORMANCE=1
+export TORCH_EXTENSIONS_DIR=$WORKSPACE/cache/torch_extensions CUDA_HOME=/usr/local/cuda-12.8
+# per-run Triton cache on LOCAL disk (shared cache on wekafs races between concurrent runs); seed it from the shared cache to avoid recompiling
+TC=/tmp/hx_624_triton/$(basename "$(python -c "import yaml,sys; print(yaml.safe_load(open(sys.argv[1]))['output_dir'])" "$(readlink -f "$1")")")
+[ -d "$TC" ] || { mkdir -p /tmp/hx_624_triton && cp -r $WORKSPACE/cache/triton "$TC" 2>/dev/null || mkdir -p "$TC"; }
+export TRITON_CACHE_DIR=$TC
+export TOKENIZERS_PARALLELISM=false DISABLE_VERSION_CHECK=1
+export CUDA_VISIBLE_DEVICES=${GPUS:-0,1}
+CFG=$(readlink -f "$1"); shift
+RUN=$(python -c "import yaml,sys; print(yaml.safe_load(open('$CFG'))['output_dir'])")
+mkdir -p $RUN
+echo "[lf_train] cfg=$CFG gpus=$CUDA_VISIBLE_DEVICES out=$RUN"
+cd /wekafs/ict/hx_624/tools/LLaMA-Factory
+FORCE_TORCHRUN=1 MASTER_PORT=$((29600 + RANDOM % 300)) llamafactory-cli train $CFG "$@" 2>&1 | tee -a $RUN/train.log
diff --git a/finetune/scripts/make_lora_cfg.py b/finetune/scripts/make_lora_cfg.py
new file mode 100644
index 00000000..ed02ec62
--- /dev/null
+++ b/finetune/scripts/make_lora_cfg.py
@@ -0,0 +1,12 @@
+"""make_lora_cfg.py NAME DATANAME [--epochs 2] [--lr 1e-4] -> configs/lf/lora_<NAME>.yaml (from the v1 LoRA template)"""
+import sys, argparse, yaml
+ap = argparse.ArgumentParser(); ap.add_argument("name"); ap.add_argument("dataname"); ap.add_argument("--epochs", type=float, default=2.0); ap.add_argument("--lr", type=float, default=1e-4)
+ap.add_argument("--rank", type=int, default=64); ap.add_argument("--cutoff", type=int, default=8192)
+a = ap.parse_args()
+cfg = yaml.safe_load(open("/wekafs/ict/hx_624/llm-ft/configs/lf/qwen35_9b_lora_sft.yaml"))
+cfg.update(dataset=f"{a.dataname}_train", eval_dataset=f"{a.dataname}_val",
+ tokenized_path=f"/wekafs/ict/hx_624/llm-ft/data/lf/tokenized_{a.dataname}_len{a.cutoff}_packed",
+ output_dir=f"/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_{a.name}",
+ num_train_epochs=a.epochs, learning_rate=a.lr, lora_rank=a.rank, lora_alpha=2 * a.rank, cutoff_len=a.cutoff)
+out = f"/wekafs/ict/hx_624/llm-ft/configs/lf/lora_{a.name}.yaml"
+yaml.safe_dump(cfg, open(out, "w"), sort_keys=False); print(out)
diff --git a/finetune/scripts/md_big_ckpt_sweep.sh b/finetune/scripts/md_big_ckpt_sweep.sh
new file mode 100755
index 00000000..a012b5df
--- /dev/null
+++ b/finetune/scripts/md_big_ckpt_sweep.sh
@@ -0,0 +1,20 @@
+#!/bin/bash
+# After md_big training: evaluate intermediate checkpoints (training-amount curve) on 3DCodeBench (greedy) + dialect tests (greedy blender; T=0.7 cq/scad/glsl)
+set -uo pipefail; GPU=${GPUS:-1}; cd /wekafs/ict/hx_624/llm-ft; RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big
+until grep -q "\[md md_big\] .*train done" logs/exp_md_big.log 2>/dev/null; do sleep 60; done
+CKS=$(ls -d $RUN/checkpoint-* 2>/dev/null | sed 's/.*checkpoint-//' | sort -n | tr '\n' ' '); echo "[bigsweep] checkpoints: $CKS"
+for STEP in $CKS; do
+ CK=$RUN/checkpoint-$STEP; NAME=md_big_ck$STEP
+ sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $CK|" -e "s|export_dir:.*|export_dir: $CK/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml
+ scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[bigsweep] EXPORT FAILED $STEP"; continue; }
+ GPUS=$GPU eval/run_eval_vllm.sh $CK/merged eval/out/qwen35_9b_lora_$NAME --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1
+ echo "[bigsweep] ck$STEP 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_lora_$NAME/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"
+ GPUS=$GPU eval/run_dialect_eval.sh $CK/merged $NAME blender > logs/mdeval_$NAME.log 2>&1
+ source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU
+ for D in glsl openscad cadquery; do OUT=eval/out/md_${NAME}_T07_$D; mkdir -p $OUT; cp eval/out/md_md_mixed_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $CK/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done
+ conda activate llmft
+ for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_${NAME}_T07_$D --workers 32 2>&1 | grep -E "dialect"; done
+ echo "[bigsweep] ck$STEP blender: $(cat eval/out/md_${NAME}_blender/summary.json | tr -d '\n ' | cut -c1-140)"
+ rm -rf $CK/merged
+done
+echo "[bigsweep] DONE"
diff --git a/finetune/scripts/md_big_dpo_round.sh b/finetune/scripts/md_big_dpo_round.sh
new file mode 100755
index 00000000..042f8ed2
--- /dev/null
+++ b/finetune/scripts/md_big_dpo_round.sh
@@ -0,0 +1,90 @@
+#!/bin/bash
+# Multi-dialect execution-feedback DPO on md_big: sample K=2 (T=0.8) on 1500 train prompts per dialect -> execute with each dialect's executor -> (OK, FAIL) pairs -> DPO -> eval (sampling for cq/scad/glsl, greedy for blender & 3DCodeBench)
+set -uo pipefail; GPU=${GPUS:-2,3}; G1=$(echo $GPU | cut -d, -f1); G2=$(echo $GPU | cut -d, -f2); cd /wekafs/ict/hx_624/llm-ft; W=data/md_big_dpo_round; mkdir -p $W
+echo "[bigdpo] $(date) start"
+source /wekafs/ict/hx_624/llm-ft/env.sh
+python - <<'PY'
+import json, random; random.seed(5)
+rows=[]
+for d in ["cadquery","openscad","glsl","blender"]:
+ rs=[json.loads(l) for l in open(f"data/md_{d}/train.jsonl")]; random.shuffle(rs)
+ for i,r in enumerate(rs[:1500]): rows.append({"task": f"{d}_{i:05d}", "dialect": d, "messages": r["messages"][:2]})
+random.shuffle(rows)
+for g in range(2):
+ with open(f"data/md_big_dpo_round/prompts_g{g}.jsonl","w") as f:
+ for r in rows[g::2]: f.write(json.dumps(r)+"\n")
+print("prompts", len(rows))
+PY
+BASE=runs/lf_qwen35_9b_lora_md_big/merged
+[ -d $BASE ] || scripts/lf_export.sh configs/lf/export_md_big.yaml > logs/export_md_big_dpo.log 2>&1 || { echo "[bigdpo] EXPORT FAILED"; exit 1; }
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING
+for i in 1 2; do
+ CUDA_VISIBLE_DEVICES=$G1 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g0.jsonl --out $W/gen_s${i}_g0 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((400+i)) > $W/gen_s${i}_g0.log 2>&1 &
+ CUDA_VISIBLE_DEVICES=$G2 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g1.jsonl --out $W/gen_s${i}_g1 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((410+i)) > $W/gen_s${i}_g1.log 2>&1 &
+ wait; grep -h gen-vllm $W/gen_s${i}_g0.log $W/gen_s${i}_g1.log
+done
+echo "[bigdpo] $(date) sampling done"
+conda activate llmft
+python - <<'PY'
+# execute every generation with its dialect's executor (parallel), then build pairs
+import json, glob, os, sys, random
+from concurrent.futures import ThreadPoolExecutor
+sys.path.insert(0, "eval"); from dialect_runners import RUNNERS
+import subprocess
+BL="/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; RUNNER=os.path.abspath("eval/blender_runner.py")
+def run_blender(code, d):
+ d=os.path.abspath(d); os.makedirs(d, exist_ok=True); sp=os.path.join(d,"code.py"); open(sp,"w").write(code)
+ try:
+ subprocess.run([BL,"-b","--factory-startup","-noaudio","--python",RUNNER,"--","--script",sp,"--out",os.path.join(d,"out.glb"),"--report",os.path.join(d,"exec.json")],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=120,env={**os.environ,"HOME":"/tmp"})
+ r=json.load(open(os.path.join(d,"exec.json"))) if os.path.exists(os.path.join(d,"exec.json")) else {"status":"CRASH"}
+ except subprocess.TimeoutExpired: r={"status":"TIMEOUT"}
+ return r
+W="data/md_big_dpo_round"; prompts={}
+for f in glob.glob(f"{W}/prompts_g*.jsonl"):
+ for l in open(f): r=json.loads(l); prompts[r["task"]]=r
+jobs=[]
+for d in sorted(glob.glob(f"{W}/gen_s*_g*/")):
+ for t in os.listdir(d):
+ cp=os.path.join(d,t,"code.py")
+ if os.path.isfile(cp): jobs.append((d,t))
+def job(x):
+ d,t=x; code=open(os.path.join(d,t,"code.py")).read(); dia=prompts[t]["dialect"]; wd=os.path.join(d,t,"exec")
+ r = run_blender(code, wd) if dia=="blender" else RUNNERS[dia](code, wd)
+ return (d,t,dia,r.get("status"))
+with ThreadPoolExecutor(48) as ex: res=list(ex.map(job, jobs))
+by={}
+for d,t,dia,st in res:
+ code=open(os.path.join(d,t,"code.py")).read().strip()
+ if code.count("\n")<3: continue
+ by.setdefault(t,{"dia":dia,"ok":[],"bad":[]})["ok" if st=="OK" else "bad"].append(code)
+from collections import Counter
+print("per-dialect pass rate:", {dia: round(sum(1 for d_,t_,di,st in res if di==dia and st=="OK")/max(1,sum(1 for d_,t_,di,st in res if di==dia)),3) for dia in ["cadquery","openscad","glsl","blender"]})
+random.seed(2); pairs=[]; per=Counter()
+for t,v in by.items():
+ if v["ok"] and v["bad"]:
+ ok=random.choice(v["ok"]); bad=random.choice(v["bad"])
+ if ok==bad: continue
+ p=prompts[t]; per[v["dia"]]+=1
+ pairs.append({"conversations":[{"from":"human","value":p["messages"][1]["content"]}],"system":p["messages"][0]["content"],"chosen":{"from":"gpt","value":ok if ok.startswith("```") else "```\n"+ok+"\n```"},"rejected":{"from":"gpt","value":bad if bad.startswith("```") else "```\n"+bad+"\n```"}})
+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_md_big_pairs.json","w"), ensure_ascii=False)
+info=json.load(open("data/lf/dataset_info.json")); info["dpo_md_big_pairs"]={"file_name":"dpo_md_big_pairs.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}
+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2); print("md pairs:", len(pairs), dict(per))
+PY
+echo "[bigdpo] $(date) pairs built"
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_md_big_pairs|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_md_big.yaml
+GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_md_big.yaml gradient_accumulation_steps=8 cutoff_len=4096 > logs/train_dpo_md_big.log 2>&1
+RUN=runs/lf_qwen35_9b_dpo_md_big; [ -f $RUN/train_results.json ] || { echo "[bigdpo] TRAIN FAILED"; exit 1; }
+echo "[bigdpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_md_big.yaml
+scripts/lf_export.sh configs/lf/export_dpo_md_big.yaml > logs/export_dpo_md_big.log 2>&1 || { echo "[bigdpo] EXPORT2 FAILED"; exit 1; }
+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_md_big --no_think --max_new_tokens 6144 > logs/eval_dpo_md_big.log 2>&1
+echo "[bigdpo] 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_md_big/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"
+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged dpo_md_big > logs/mdeval_dpo_md_big.log 2>&1
+# sampling eval for cq/scad/glsl
+conda activate vllm; export CUDA_VISIBLE_DEVICES=$G1
+for D in glsl openscad cadquery; do OUT=eval/out/md_dpo_md_big_T07_$D; mkdir -p $OUT; cp eval/out/md_dpo_md_big_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $RUN/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done
+conda activate llmft
+for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_dpo_md_big_T07_$D --workers 32 2>&1 | grep -E "dialect"; done
+for D in cadquery openscad glsl blender; do echo "[bigdpo] greedy $D: $(cat eval/out/md_dpo_md_big_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done
+rm -rf $RUN/merged $BASE; echo "[bigdpo] DONE"
diff --git a/finetune/scripts/md_dpo_resume.sh b/finetune/scripts/md_dpo_resume.sh
new file mode 100755
index 00000000..80bb068e
--- /dev/null
+++ b/finetune/scripts/md_dpo_resume.sh
@@ -0,0 +1,19 @@
+#!/bin/bash
+# resume multi-dialect DPO after OOM: train with cutoff 4096 (long GLSL/OpenSCAD pairs), then export + evals
+set -uo pipefail; GPU=${GPUS:-2,3}; G1=$(echo $GPU | cut -d, -f1); cd /wekafs/ict/hx_624/llm-ft
+BASE=runs/lf_qwen35_9b_lora_md_mixed/merged; RUN=runs/lf_qwen35_9b_dpo_md_mixed
+echo "[mddpo] $(date) retrain with cutoff 4096"
+GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_md_mixed.yaml gradient_accumulation_steps=8 cutoff_len=4096 > logs/train_dpo_md_mixed.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[mddpo] TRAIN FAILED"; grep -E "Error" logs/train_dpo_md_mixed.log | grep -v errors | tail -2 | cut -c1-200; exit 1; }
+echo "[mddpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_md_mixed.yaml
+scripts/lf_export.sh configs/lf/export_dpo_md_mixed.yaml > logs/export_dpo_md_mixed.log 2>&1 || { echo "[mddpo] EXPORT2 FAILED"; exit 1; }
+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_md_mixed --no_think --max_new_tokens 6144 > logs/eval_dpo_md_mixed.log 2>&1
+echo "[mddpo] 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_md_mixed/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"
+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged dpo_md_mixed > logs/mdeval_dpo_md_mixed.log 2>&1
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$G1
+for D in glsl openscad cadquery; do OUT=eval/out/md_dpo_md_mixed_T07_$D; mkdir -p $OUT; cp eval/out/md_dpo_md_mixed_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $RUN/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done
+conda activate llmft
+for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_dpo_md_mixed_T07_$D --workers 32 2>&1 | grep -E "dialect"; done
+for D in cadquery openscad glsl blender; do echo "[mddpo] greedy $D: $(cat eval/out/md_dpo_md_mixed_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done
+rm -rf $RUN/merged $BASE; echo "[mddpo] DONE"
diff --git a/finetune/scripts/md_dpo_round.sh b/finetune/scripts/md_dpo_round.sh
new file mode 100755
index 00000000..38e47570
--- /dev/null
+++ b/finetune/scripts/md_dpo_round.sh
@@ -0,0 +1,90 @@
+#!/bin/bash
+# Multi-dialect execution-feedback DPO on md_mixed: sample K=2 (T=0.8) on 1500 train prompts per dialect -> execute with each dialect's executor -> (OK, FAIL) pairs -> DPO -> eval (sampling for cq/scad/glsl, greedy for blender & 3DCodeBench)
+set -uo pipefail; GPU=${GPUS:-2,3}; G1=$(echo $GPU | cut -d, -f1); G2=$(echo $GPU | cut -d, -f2); cd /wekafs/ict/hx_624/llm-ft; W=data/md_dpo_round; mkdir -p $W
+echo "[mddpo] $(date) start"
+source /wekafs/ict/hx_624/llm-ft/env.sh
+python - <<'PY'
+import json, random; random.seed(5)
+rows=[]
+for d in ["cadquery","openscad","glsl","blender"]:
+ rs=[json.loads(l) for l in open(f"data/md_{d}/train.jsonl")]; random.shuffle(rs)
+ for i,r in enumerate(rs[:1500]): rows.append({"task": f"{d}_{i:05d}", "dialect": d, "messages": r["messages"][:2]})
+random.shuffle(rows)
+for g in range(2):
+ with open(f"data/md_dpo_round/prompts_g{g}.jsonl","w") as f:
+ for r in rows[g::2]: f.write(json.dumps(r)+"\n")
+print("prompts", len(rows))
+PY
+BASE=runs/lf_qwen35_9b_lora_md_mixed/merged
+[ -d $BASE ] || scripts/lf_export.sh configs/lf/export_md_mixed.yaml > logs/export_md_mixed_dpo.log 2>&1 || { echo "[mddpo] EXPORT FAILED"; exit 1; }
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING
+for i in 1 2; do
+ CUDA_VISIBLE_DEVICES=$G1 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g0.jsonl --out $W/gen_s${i}_g0 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((400+i)) > $W/gen_s${i}_g0.log 2>&1 &
+ CUDA_VISIBLE_DEVICES=$G2 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g1.jsonl --out $W/gen_s${i}_g1 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((410+i)) > $W/gen_s${i}_g1.log 2>&1 &
+ wait; grep -h gen-vllm $W/gen_s${i}_g0.log $W/gen_s${i}_g1.log
+done
+echo "[mddpo] $(date) sampling done"
+conda activate llmft
+python - <<'PY'
+# execute every generation with its dialect's executor (parallel), then build pairs
+import json, glob, os, sys, random
+from concurrent.futures import ThreadPoolExecutor
+sys.path.insert(0, "eval"); from dialect_runners import RUNNERS
+import subprocess
+BL="/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; RUNNER=os.path.abspath("eval/blender_runner.py")
+def run_blender(code, d):
+ d=os.path.abspath(d); os.makedirs(d, exist_ok=True); sp=os.path.join(d,"code.py"); open(sp,"w").write(code)
+ try:
+ subprocess.run([BL,"-b","--factory-startup","-noaudio","--python",RUNNER,"--","--script",sp,"--out",os.path.join(d,"out.glb"),"--report",os.path.join(d,"exec.json")],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=120,env={**os.environ,"HOME":"/tmp"})
+ r=json.load(open(os.path.join(d,"exec.json"))) if os.path.exists(os.path.join(d,"exec.json")) else {"status":"CRASH"}
+ except subprocess.TimeoutExpired: r={"status":"TIMEOUT"}
+ return r
+W="data/md_dpo_round"; prompts={}
+for f in glob.glob(f"{W}/prompts_g*.jsonl"):
+ for l in open(f): r=json.loads(l); prompts[r["task"]]=r
+jobs=[]
+for d in sorted(glob.glob(f"{W}/gen_s*_g*/")):
+ for t in os.listdir(d):
+ cp=os.path.join(d,t,"code.py")
+ if os.path.isfile(cp): jobs.append((d,t))
+def job(x):
+ d,t=x; code=open(os.path.join(d,t,"code.py")).read(); dia=prompts[t]["dialect"]; wd=os.path.join(d,t,"exec")
+ r = run_blender(code, wd) if dia=="blender" else RUNNERS[dia](code, wd)
+ return (d,t,dia,r.get("status"))
+with ThreadPoolExecutor(48) as ex: res=list(ex.map(job, jobs))
+by={}
+for d,t,dia,st in res:
+ code=open(os.path.join(d,t,"code.py")).read().strip()
+ if code.count("\n")<3: continue
+ by.setdefault(t,{"dia":dia,"ok":[],"bad":[]})["ok" if st=="OK" else "bad"].append(code)
+from collections import Counter
+print("per-dialect pass rate:", {dia: round(sum(1 for d_,t_,di,st in res if di==dia and st=="OK")/max(1,sum(1 for d_,t_,di,st in res if di==dia)),3) for dia in ["cadquery","openscad","glsl","blender"]})
+random.seed(2); pairs=[]; per=Counter()
+for t,v in by.items():
+ if v["ok"] and v["bad"]:
+ ok=random.choice(v["ok"]); bad=random.choice(v["bad"])
+ if ok==bad: continue
+ p=prompts[t]; per[v["dia"]]+=1
+ pairs.append({"conversations":[{"from":"human","value":p["messages"][1]["content"]}],"system":p["messages"][0]["content"],"chosen":{"from":"gpt","value":ok if ok.startswith("```") else "```\n"+ok+"\n```"},"rejected":{"from":"gpt","value":bad if bad.startswith("```") else "```\n"+bad+"\n```"}})
+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_md_pairs.json","w"), ensure_ascii=False)
+info=json.load(open("data/lf/dataset_info.json")); info["dpo_md_pairs"]={"file_name":"dpo_md_pairs.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}
+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2); print("md pairs:", len(pairs), dict(per))
+PY
+echo "[mddpo] $(date) pairs built"
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_md_pairs|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_md_mixed.yaml
+GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_md_mixed.yaml gradient_accumulation_steps=8 > logs/train_dpo_md_mixed.log 2>&1
+RUN=runs/lf_qwen35_9b_dpo_md_mixed; [ -f $RUN/train_results.json ] || { echo "[mddpo] TRAIN FAILED"; exit 1; }
+echo "[mddpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_md_mixed.yaml
+scripts/lf_export.sh configs/lf/export_dpo_md_mixed.yaml > logs/export_dpo_md_mixed.log 2>&1 || { echo "[mddpo] EXPORT2 FAILED"; exit 1; }
+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_md_mixed --no_think --max_new_tokens 6144 > logs/eval_dpo_md_mixed.log 2>&1
+echo "[mddpo] 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_md_mixed/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"
+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged dpo_md_mixed > logs/mdeval_dpo_md_mixed.log 2>&1
+# sampling eval for cq/scad/glsl
+conda activate vllm; export CUDA_VISIBLE_DEVICES=$G1
+for D in glsl openscad cadquery; do OUT=eval/out/md_dpo_md_mixed_T07_$D; mkdir -p $OUT; cp eval/out/md_dpo_md_mixed_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $RUN/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done
+conda activate llmft
+for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_dpo_md_mixed_T07_$D --workers 32 2>&1 | grep -E "dialect"; done
+for D in cadquery openscad glsl blender; do echo "[mddpo] greedy $D: $(cat eval/out/md_dpo_md_mixed_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done
+rm -rf $RUN/merged $BASE; echo "[mddpo] DONE"
diff --git a/finetune/scripts/md_xl_ckpt_sweep.sh b/finetune/scripts/md_xl_ckpt_sweep.sh
new file mode 100755
index 00000000..093b5037
--- /dev/null
+++ b/finetune/scripts/md_xl_ckpt_sweep.sh
@@ -0,0 +1,20 @@
+#!/bin/bash
+# After md_xl training: evaluate intermediate checkpoints (training-amount curve) on 3DCodeBench (greedy) + dialect tests (greedy blender; T=0.7 cq/scad/glsl)
+set -uo pipefail; GPU=${GPUS:-1}; cd /wekafs/ict/hx_624/llm-ft; RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl
+until grep -q "\[md md_xl\] .*train done" logs/exp_md_xl.log 2>/dev/null; do sleep 60; done
+CKS=$(ls -d $RUN/checkpoint-* 2>/dev/null | sed 's/.*checkpoint-//' | sort -n | tr '\n' ' '); echo "[bigsweep] checkpoints: $CKS"
+for STEP in $CKS; do
+ CK=$RUN/checkpoint-$STEP; NAME=md_xl_ck$STEP
+ sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $CK|" -e "s|export_dir:.*|export_dir: $CK/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml
+ scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[bigsweep] EXPORT FAILED $STEP"; continue; }
+ GPUS=$GPU eval/run_eval_vllm.sh $CK/merged eval/out/qwen35_9b_lora_$NAME --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1
+ echo "[bigsweep] ck$STEP 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_lora_$NAME/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"
+ GPUS=$GPU eval/run_dialect_eval.sh $CK/merged $NAME blender > logs/mdeval_$NAME.log 2>&1
+ source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU
+ for D in glsl openscad cadquery; do OUT=eval/out/md_${NAME}_T07_$D; mkdir -p $OUT; cp eval/out/md_md_mixed_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $CK/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done
+ conda activate llmft
+ for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_${NAME}_T07_$D --workers 32 2>&1 | grep -E "dialect"; done
+ echo "[bigsweep] ck$STEP blender: $(cat eval/out/md_${NAME}_blender/summary.json | tr -d '\n ' | cut -c1-140)"
+ rm -rf $CK/merged
+done
+echo "[bigsweep] DONE"
diff --git a/finetune/scripts/md_xl_dpo_round.sh b/finetune/scripts/md_xl_dpo_round.sh
new file mode 100755
index 00000000..2314030c
--- /dev/null
+++ b/finetune/scripts/md_xl_dpo_round.sh
@@ -0,0 +1,90 @@
+#!/bin/bash
+# Multi-dialect execution-feedback DPO on md_xl: sample K=2 (T=0.8) on 1500 train prompts per dialect -> execute with each dialect's executor -> (OK, FAIL) pairs -> DPO -> eval (sampling for cq/scad/glsl, greedy for blender & 3DCodeBench)
+set -uo pipefail; GPU=${GPUS:-2,3}; G1=$(echo $GPU | cut -d, -f1); G2=$(echo $GPU | cut -d, -f2); cd /wekafs/ict/hx_624/llm-ft; W=data/md_xl_dpo_round; mkdir -p $W
+echo "[xldpo] $(date) start"
+source /wekafs/ict/hx_624/llm-ft/env.sh
+python - <<'PY'
+import json, random; random.seed(5)
+rows=[]
+for d in ["cadquery","openscad","glsl","blender"]:
+ rs=[json.loads(l) for l in open(f"data/md_{d}/train.jsonl")]; random.shuffle(rs)
+ for i,r in enumerate(rs[:1500]): rows.append({"task": f"{d}_{i:05d}", "dialect": d, "messages": r["messages"][:2]})
+random.shuffle(rows)
+for g in range(2):
+ with open(f"data/md_xl_dpo_round/prompts_g{g}.jsonl","w") as f:
+ for r in rows[g::2]: f.write(json.dumps(r)+"\n")
+print("prompts", len(rows))
+PY
+BASE=runs/lf_qwen35_9b_lora_md_xl/merged
+[ -d $BASE ] || scripts/lf_export.sh configs/lf/export_md_xl.yaml > logs/export_md_xl_dpo.log 2>&1 || { echo "[xldpo] EXPORT FAILED"; exit 1; }
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING
+for i in 1 2; do
+ CUDA_VISIBLE_DEVICES=$G1 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g0.jsonl --out $W/gen_s${i}_g0 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((400+i)) > $W/gen_s${i}_g0.log 2>&1 &
+ CUDA_VISIBLE_DEVICES=$G2 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g1.jsonl --out $W/gen_s${i}_g1 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((410+i)) > $W/gen_s${i}_g1.log 2>&1 &
+ wait; grep -h gen-vllm $W/gen_s${i}_g0.log $W/gen_s${i}_g1.log
+done
+echo "[xldpo] $(date) sampling done"
+conda activate llmft
+python - <<'PY'
+# execute every generation with its dialect's executor (parallel), then build pairs
+import json, glob, os, sys, random
+from concurrent.futures import ThreadPoolExecutor
+sys.path.insert(0, "eval"); from dialect_runners import RUNNERS
+import subprocess
+BL="/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; RUNNER=os.path.abspath("eval/blender_runner.py")
+def run_blender(code, d):
+ d=os.path.abspath(d); os.makedirs(d, exist_ok=True); sp=os.path.join(d,"code.py"); open(sp,"w").write(code)
+ try:
+ subprocess.run([BL,"-b","--factory-startup","-noaudio","--python",RUNNER,"--","--script",sp,"--out",os.path.join(d,"out.glb"),"--report",os.path.join(d,"exec.json")],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=120,env={**os.environ,"HOME":"/tmp"})
+ r=json.load(open(os.path.join(d,"exec.json"))) if os.path.exists(os.path.join(d,"exec.json")) else {"status":"CRASH"}
+ except subprocess.TimeoutExpired: r={"status":"TIMEOUT"}
+ return r
+W="data/md_xl_dpo_round"; prompts={}
+for f in glob.glob(f"{W}/prompts_g*.jsonl"):
+ for l in open(f): r=json.loads(l); prompts[r["task"]]=r
+jobs=[]
+for d in sorted(glob.glob(f"{W}/gen_s*_g*/")):
+ for t in os.listdir(d):
+ cp=os.path.join(d,t,"code.py")
+ if os.path.isfile(cp): jobs.append((d,t))
+def job(x):
+ d,t=x; code=open(os.path.join(d,t,"code.py")).read(); dia=prompts[t]["dialect"]; wd=os.path.join(d,t,"exec")
+ r = run_blender(code, wd) if dia=="blender" else RUNNERS[dia](code, wd)
+ return (d,t,dia,r.get("status"))
+with ThreadPoolExecutor(48) as ex: res=list(ex.map(job, jobs))
+by={}
+for d,t,dia,st in res:
+ code=open(os.path.join(d,t,"code.py")).read().strip()
+ if code.count("\n")<3: continue
+ by.setdefault(t,{"dia":dia,"ok":[],"bad":[]})["ok" if st=="OK" else "bad"].append(code)
+from collections import Counter
+print("per-dialect pass rate:", {dia: round(sum(1 for d_,t_,di,st in res if di==dia and st=="OK")/max(1,sum(1 for d_,t_,di,st in res if di==dia)),3) for dia in ["cadquery","openscad","glsl","blender"]})
+random.seed(2); pairs=[]; per=Counter()
+for t,v in by.items():
+ if v["ok"] and v["bad"]:
+ ok=random.choice(v["ok"]); bad=random.choice(v["bad"])
+ if ok==bad: continue
+ p=prompts[t]; per[v["dia"]]+=1
+ pairs.append({"conversations":[{"from":"human","value":p["messages"][1]["content"]}],"system":p["messages"][0]["content"],"chosen":{"from":"gpt","value":ok if ok.startswith("```") else "```\n"+ok+"\n```"},"rejected":{"from":"gpt","value":bad if bad.startswith("```") else "```\n"+bad+"\n```"}})
+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_md_xl_pairs.json","w"), ensure_ascii=False)
+info=json.load(open("data/lf/dataset_info.json")); info["dpo_md_xl_pairs"]={"file_name":"dpo_md_xl_pairs.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}
+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2); print("md pairs:", len(pairs), dict(per))
+PY
+echo "[xldpo] $(date) pairs built"
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_md_xl_pairs|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_xl|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_md_xl.yaml
+GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_md_xl.yaml gradient_accumulation_steps=8 cutoff_len=4096 > logs/train_dpo_md_xl.log 2>&1
+RUN=runs/lf_qwen35_9b_dpo_md_xl; [ -f $RUN/train_results.json ] || { echo "[xldpo] TRAIN FAILED"; exit 1; }
+echo "[xldpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_md_xl.yaml
+scripts/lf_export.sh configs/lf/export_dpo_md_xl.yaml > logs/export_dpo_md_xl.log 2>&1 || { echo "[xldpo] EXPORT2 FAILED"; exit 1; }
+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_md_xl --no_think --max_new_tokens 6144 > logs/eval_dpo_md_xl.log 2>&1
+echo "[xldpo] 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_md_xl/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"
+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged dpo_md_xl > logs/mdeval_dpo_md_xl.log 2>&1
+# sampling eval for cq/scad/glsl
+conda activate vllm; export CUDA_VISIBLE_DEVICES=$G1
+for D in glsl openscad cadquery; do OUT=eval/out/md_dpo_md_xl_T07_$D; mkdir -p $OUT; cp eval/out/md_dpo_md_xl_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $RUN/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done
+conda activate llmft
+for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_dpo_md_xl_T07_$D --workers 32 2>&1 | grep -E "dialect"; done
+for D in cadquery openscad glsl blender; do echo "[xldpo] greedy $D: $(cat eval/out/md_dpo_md_xl_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done
+rm -rf $RUN/merged $BASE; echo "[xldpo] DONE"
diff --git a/finetune/scripts/md_xl_extra_eval.sh b/finetune/scripts/md_xl_extra_eval.sh
new file mode 100755
index 00000000..a4f71b76
--- /dev/null
+++ b/finetune/scripts/md_xl_extra_eval.sh
@@ -0,0 +1,15 @@
+#!/bin/bash
+# GPU 0 while DPO (GPUs 2,3) and sweep (GPU 1) run: three.js eval (greedy + T=0.7) and 3DCodeBench best-of-4 for md_xl (uses the merged model re-exported by the DPO round)
+cd /wekafs/ict/hx_624/llm-ft; M=runs/lf_qwen35_9b_lora_md_xl/merged
+until [ -f data/md_xl_dpo_round/gen_s1_g0.log ] && [ -f $M/config.json ]; do sleep 30; done; sleep 60
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=0 PLAYWRIGHT_BROWSERS_PATH=/wekafs/ict/hx_624/cache/ms-playwright
+echo "[xlextra] $(date) three.js gens"
+mkdir -p eval/out/md_md_xl_threejs eval/out/md_md_xl_T07_threejs
+python eval/generate_vllm.py --model $M --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_md_xl_threejs --tp 1 --no_think --max_new_tokens 12288 2>&1 | grep -E "gen-vllm|Error" | tail -1
+python eval/generate_vllm.py --model $M --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_md_xl_T07_threejs --tp 1 --no_think --max_new_tokens 12288 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm|Error" | tail -1
+echo "[xlextra] $(date) best-of-4"
+GPUS=0 eval/best_of_n.sh $M md_xl_T07 4 0.7 2>&1 | grep -E "pass@|OK rate|BEST"
+conda activate llmft
+for N in md_md_xl_threejs md_md_xl_T07_threejs; do echo "== $N"; python eval/threejs_eval.py --gen_dir eval/out/$N 2>&1 | tail -2; done
+echo "[xlextra] DONE"
diff --git a/finetune/scripts/md_xl_launch.sh b/finetune/scripts/md_xl_launch.sh
new file mode 100755
index 00000000..2ea3bb05
--- /dev/null
+++ b/finetune/scripts/md_xl_launch.sh
@@ -0,0 +1,20 @@
+#!/bin/bash
+cd /wekafs/ict/hx_624/llm-ft; source env.sh
+echo "[md_xl] $(date) build data"
+python scripts/build_md_xl.py 2>&1 | grep -v Warning | tail -1 | tee logs/build_md_xl.stats
+python scripts/to_llamafactory.py --data_dir /wekafs/ict/hx_624/llm-ft/data/md_xl --name md_xl 2>&1 | tail -3 | cut -c1-200
+# ~10 checkpoints over the run: steps = tokens / (4 GPUs x 2 accum x 8192)
+python3 - <<'PY'
+import re, yaml
+s=open("logs/build_md_xl.stats").read(); tok=float(re.search(r"tokens/epoch ([0-9.]+)M", s).group(1))*1e6
+steps=tok/(8*8192); save=max(50, int(round(steps/10/50))*50)
+p="configs/lf/lora_md_xl.yaml"; c=yaml.safe_load(open(p)); c["save_steps"]=save; c["eval_steps"]=save; c["save_total_limit"]=14
+yaml.safe_dump(c, open(p,"w"), sort_keys=False); print(f"[md_xl] est. steps {steps:.0f}, save/eval every {save} steps")
+PY
+echo "BUILD_MD_XL_DONE"
+G=0,1,2,3; echo "[md_xl] using GPUs $G"
+scripts/run_md_exp.sh $G md_xl 2>&1 | tee -a logs/exp_md_xl.log
+grep -q "\[md md_xl\] DONE" logs/exp_md_xl.log || { echo "[md_xl] pipeline failed, not sweeping"; exit 1; }
+echo "[md_xl] $(date) starting checkpoint sweep on GPU 1"
+GPUS=1 scripts/md_xl_ckpt_sweep.sh > logs/md_xl_ckpt_sweep.log 2>&1
+echo "[md_xl] ALL DONE"
diff --git a/finetune/scripts/ntok_cache.py b/finetune/scripts/ntok_cache.py
new file mode 100644
index 00000000..7884a458
--- /dev/null
+++ b/finetune/scripts/ntok_cache.py
@@ -0,0 +1,12 @@
+"""Helper for build scripts: load cached token counts for a jsonl (scripts/cache_ntok.py), falling back to tokenizing."""
+import os
+def load_ntok(path, tok=None):
+ c = path + ".ntok"
+ if os.path.exists(c):
+ ns = [int(x) for x in open(c).read().split()]
+ if ns and sum(1 for _ in open(path)) == len(ns): return ns
+ if tok is None:
+ from transformers import AutoTokenizer; tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")
+ import json
+ ns = [len(tok(tok.apply_chat_template(json.loads(l)["messages"], tokenize=False)).input_ids) for l in open(path)]
+ open(c, "w").write("\n".join(map(str, ns)) + "\n"); return ns
diff --git a/finetune/scripts/prep_data.py b/finetune/scripts/prep_data.py
new file mode 100644
index 00000000..87141c34
--- /dev/null
+++ b/finetune/scripts/prep_data.py
@@ -0,0 +1,184 @@
+"""Build SFT data (chat 'messages' jsonl) for text -> Blender-Python from the 3DCodeVerse-family sources.
+
+Sources (all Blender Python):
+ * bioinspired3d geo/tex metadata.parquet (ilabai/3dcodeverse) ~6.3k, short scripts
+ * 3dcodebench factories geo/tex parquet (ilabai/3dcodeverse) 243*2, Infinigen style, LONG scripts
+ -> by default ONLY the 31 factories that are NOT in the 3DCodeBench test set (avoid contamination)
+ * blender_distill (Claude-generated) (ilabai/3dcodeverse) ~60-100
+ * thingiverse-openscad-blender-5-0 (ilabai/...) ~1.2k, keep native-CSG only (no mesh fallback)
+Outputs: <out>/train.jsonl, <out>/val.jsonl, <out>/stats.json, <out>/bench_prompts.jsonl
+"""
+import argparse, glob, hashlib, json, os, random, re
+import pandas as pd
+
+SYSTEM = ("You are an expert in procedural 3D modeling with Blender Python (bpy). "
+ "Given a description of an object, write a complete, standalone Blender 5 Python script that builds it "
+ "from scratch (clear the default scene first, create geometry with bpy/bmesh, assign simple materials when "
+ "relevant, leave the finished mesh objects in the scene). Output ONLY the code in one ```python block.")
+
+def ans(code):
+ return "```python\n" + code.strip("\n") + "\n```"
+
+def wrap_desc(desc):
+ starters = ["Write a Blender Python script that builds the following object: ",
+ "Create this 3D object with Blender Python (bpy): ",
+ "Using Python Blender code, procedurally model: ",
+ "Generate a Blender Python (bpy) script for a 3D model matching this description: "]
+ return random.choice(starters) + desc.strip()
+
+PROMPT_W = {"instruction": 6, "detailed": 3, "name": 1}
+def pick_prompt(name, caps):
+ """caps: dict with keys detailed / instruction / factory (some may be missing)."""
+ ins = (caps or {}).get("instruction") or ""
+ det = (caps or {}).get("detailed") or ""
+ opts = []
+ if ins and len(ins) > 20: opts += [("instruction", ins)] * PROMPT_W.get("instruction", 0)
+ if det and len(det) > 20: opts += [("detailed", wrap_desc(det))] * PROMPT_W.get("detailed", 0)
+ if name and len(name) > 15 and not name.lower().startswith("deepcad"): opts += [("name", name)] * PROMPT_W.get("name", 0)
+ if not opts: return None, None
+ return random.choice(opts)
+
+def code_ok(code, min_lines=4):
+ if not code or code.count("\n") < min_lines: return False
+ if "import bpy" not in code and "bpy." not in code and "import cadquery" not in code: return False
+ if "reference_mesh.json" in code or "MESH_REFERENCE_ASSET" in code: return False # mesh fallback
+ return True
+
+def main():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("--root", default="/wekafs/ict/hx_624/data")
+ ap.add_argument("--out", default="/wekafs/ict/hx_624/llm-ft/data/sft_v1")
+ ap.add_argument("--val_frac", type=float, default=0.02)
+ ap.add_argument("--seed", type=int, default=42)
+ ap.add_argument("--include_test_factories", action="store_true", help="(contaminated) also include the 212 factories that overlap the benchmark")
+ ap.add_argument("--max_chars", type=int, default=60000, help="drop samples whose code is longer than this (~15k tokens)")
+ ap.add_argument("--tokenizer", default="/wekafs/ict/hx_624/models/Qwen2.5-Coder-7B-Instruct")
+ ap.add_argument("--max_tokens", type=int, default=8192, help="drop samples whose full chat exceeds this many tokens")
+ ap.add_argument("--sources", default="bio3d,fac,distill,thingi50", help="comma list of sources to include: bio3d,fac,distill,thingi50,deepcad")
+ ap.add_argument("--prompt_fields", default="instruction:6,detailed:3,name:1", help="weighted prompt field choice, e.g. 'instruction:1' for instruction-only")
+ ap.add_argument("--deepcad_n", type=int, default=20000, help="number of deepcad CadQuery samples when 'deepcad' in --sources")
+ args = ap.parse_args()
+ random.seed(args.seed)
+ os.makedirs(args.out, exist_ok=True)
+ SRC = set(args.sources.split(","))
+ PROMPT_W.clear(); PROMPT_W.update({k: int(v) for k, v in (x.split(":") for x in args.prompt_fields.split(","))})
+ R = args.root
+ samples = [] # dict(source, id, prompt_kind, messages)
+
+ # ---- test-set names (for contamination control)
+ bench_dir = f"{R}/3dcodebench/data"
+ test_names = {d.replace("_seed0", "") for d in os.listdir(bench_dir)}
+
+ # ---- bioinspired3d
+ for sub in (["bpy_structures_geo", "bpy_structures_tex"] if "bio3d" in SRC else []):
+ df = pd.read_parquet(f"{R}/3dcodeverse/bioinspired3d_{sub}_metadata.parquet")
+ for r in df.itertuples():
+ if not code_ok(r.code): continue
+ kind, p = pick_prompt(r.name, dict(r.captions))
+ if not p: continue
+ samples.append(dict(source=f"bio3d_{sub[-3:]}", id=r.id, prompt_kind=kind, code=r.code, prompt=p))
+
+ # ---- 3dcodebench factories (Infinigen style)
+ for sub in (["factories_geo", "factories_tex"] if "fac" in SRC else []):
+ df = pd.read_parquet(f"{R}/3dcodeverse/3dcodebench_{sub}_metadata.parquet")
+ for r in df.itertuples():
+ fac = r.name.replace("Factory", "")
+ if fac in test_names and not args.include_test_factories: continue
+ if not code_ok(r.code): continue
+ kind, p = pick_prompt(None, dict(r.captions))
+ if not p: continue
+ samples.append(dict(source=f"fac_{sub[-3:]}" + ("_TEST" if fac in test_names else ""), id=r.id, prompt_kind=kind, code=r.code, prompt=p))
+
+ # ---- blender_distill
+ for cp in (glob.glob(f"{R}/3dcodeverse/blender_distill/**/code.py", recursive=True) if "distill" in SRC else []):
+ d = os.path.dirname(cp); code = open(cp).read()
+ if not code_ok(code): continue
+ caps = json.load(open(os.path.join(d, "captions.json"))) if os.path.exists(os.path.join(d, "captions.json")) else {}
+ p = caps.get("user_instruction") or (caps.get("instruction") and wrap_desc(caps["instruction"]))
+ if not p: continue
+ # distill scripts define build() but may not call it -> make sure the script executes standalone
+ if re.search(r"^def build\(\):", code, re.M) and not re.search(r"^\s*build\(\)\s*$", code, re.M) and "__main__" not in code:
+ code = code.rstrip("\n") + "\n\n\nif __name__ == \"__main__\":\n build()\n"
+ samples.append(dict(source="distill", id=os.path.relpath(d, R), prompt_kind="user_instruction", code=code, prompt=p))
+
+ # ---- deepcad (CadQuery) -- cross-dialect data; the code is NOT Blender, system prompt is adapted
+ if "deepcad" in SRC:
+ dfs = [pd.read_parquet(f) for f in sorted(glob.glob(f"{R}/3dcodeverse_meta/deepcad/*/metadata.parquet"))]
+ dd = pd.concat(dfs, ignore_index=True).sample(n=min(args.deepcad_n, sum(len(d) for d in dfs)), random_state=args.seed)
+ for r in dd.itertuples():
+ code = r.code or ""
+ if "import cadquery" not in code or code.count("\n") < 3: continue
+ caps = dict(r.captions); ins = caps.get("instruction") or caps.get("detailed") or ""
+ if len(ins) < 20: continue
+ p = "Write a CadQuery (Python) script that builds the following CAD model: " + ins.strip()
+ samples.append(dict(source="deepcad", id=r.id, prompt_kind="instruction", code=code, prompt=p, dialect="cadquery"))
+ # ---- thingiverse-openscad-blender-5-0 (native CSG only)
+ for cp in (glob.glob(f"{R}/thingiverse_5_0/*/code.py") if "thingi50" in SRC else []):
+ d = os.path.dirname(cp); code = open(cp).read()
+ rep_p = os.path.join(d, "reports", "translation_report.json")
+ status = json.load(open(rep_p)).get("status", "") if os.path.exists(rep_p) else ""
+ if "mesh_fallback" in status or not code_ok(code): continue
+ caps = json.load(open(os.path.join(d, "captions.json"))) if os.path.exists(os.path.join(d, "captions.json")) else {}
+ desc = caps.get("synthetic_prompt") or caps.get("detailed") or ""
+ if len(desc) < 40: continue
+ p = ("The following request was originally written for OpenSCAD. Implement the same design in Blender Python (bpy) "
+ "instead, as a standalone script that builds the geometry with native Blender primitives/booleans.\n\n" + desc.strip())
+ samples.append(dict(source="thingi50", id=os.path.basename(d), prompt_kind="synthetic_prompt", code=code, prompt=p))
+
+ # ---- dedup by code hash, length filter
+ seen, uniq = set(), []
+ for s in samples:
+ h = hashlib.sha1(s["code"].strip().encode()).hexdigest()
+ if h in seen or len(s["code"]) > args.max_chars: continue
+ seen.add(h); uniq.append(s)
+ # ---- token length filter (drop too-long samples instead of truncating the completion)
+ tok = None
+ try:
+ from transformers import AutoTokenizer
+ tok = AutoTokenizer.from_pretrained(args.tokenizer)
+ except Exception as e:
+ print("tokenizer unavailable:", e)
+ if tok is not None:
+ before = len(uniq)
+ def ntok(s):
+ msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": s["prompt"]}, {"role": "assistant", "content": ans(s["code"])}]
+ return len(tok(tok.apply_chat_template(msgs, tokenize=False)).input_ids)
+ uniq = [s for s in uniq if ntok(s) <= args.max_tokens]
+ print(f"dropped {before - len(uniq)} samples longer than {args.max_tokens} tokens")
+ random.shuffle(uniq)
+ n_val = max(1, int(len(uniq) * args.val_frac))
+ val, train = uniq[:n_val], uniq[n_val:]
+
+ SYSTEM_CQ = ("You are an expert CAD programmer. Given a description of a part, write a complete, standalone CadQuery (Python) "
+ "script that builds it. Output ONLY the code in one ```python block.")
+ def to_msgs(s):
+ return {"messages": [{"role": "system", "content": SYSTEM_CQ if s.get("dialect") == "cadquery" else SYSTEM},
+ {"role": "user", "content": s["prompt"]},
+ {"role": "assistant", "content": ans(s["code"])}],
+ "source": s["source"], "id": s["id"], "prompt_kind": s["prompt_kind"]}
+
+ stats = {"n_train": len(train), "n_val": len(val), "by_source": {}, "token_len": {}}
+ for split, rows in [("train", train), ("val", val)]:
+ lens = []
+ with open(f"{args.out}/{split}.jsonl", "w") as f:
+ for s in rows:
+ m = to_msgs(s)
+ f.write(json.dumps(m, ensure_ascii=False) + "\n")
+ stats["by_source"].setdefault(split, {}); stats["by_source"][split][s["source"]] = stats["by_source"][split].get(s["source"], 0) + 1
+ if tok is not None:
+ lens.append(len(tok(tok.apply_chat_template(m["messages"], tokenize=False)).input_ids))
+ if lens:
+ import numpy as np
+ lens = np.array(lens)
+ stats["token_len"][split] = {"p50": int(np.percentile(lens, 50)), "p90": int(np.percentile(lens, 90)), "p99": int(np.percentile(lens, 99)), "max": int(lens.max()), "total": int(lens.sum()), "n_gt_8192": int((lens > 8192).sum()), "n_gt_16384": int((lens > 16384).sum())}
+
+ # ---- benchmark prompts
+ with open(f"{args.out}/bench_prompts.jsonl", "w") as f:
+ for d in sorted(os.listdir(bench_dir)):
+ p = open(f"{bench_dir}/{d}/prompt_instruction.txt").read().strip()
+ f.write(json.dumps({"task": d, "messages": [{"role": "system", "content": SYSTEM}, {"role": "user", "content": p}]}) + "\n")
+ json.dump(stats, open(f"{args.out}/stats.json", "w"), indent=2)
+ print(json.dumps(stats, indent=2))
+
+if __name__ == "__main__":
+ main()
diff --git a/finetune/scripts/queue.sh b/finetune/scripts/queue.sh
new file mode 100755
index 00000000..5bb5c90f
--- /dev/null
+++ b/finetune/scripts/queue.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+# queue.sh <GPU> <NAME1> [NAME2 ...]: run experiments sequentially on one GPU (waits until no training of ours is using that GPU)
+GPU=$1; shift
+cd /wekafs/ict/hx_624/llm-ft
+for NAME in "$@"; do
+ # wait until every GPU in the list has < 10 GB used (previous job finished)
+ until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i $GPU | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done
+ scripts/run_exp.sh $GPU $NAME 2>&1 | tee -a logs/exp_$NAME.log
+done
+echo "QUEUE_DONE gpu=$GPU"
diff --git a/finetune/scripts/run_dpo_exec.sh b/finetune/scripts/run_dpo_exec.sh
new file mode 100755
index 00000000..4b7f69cf
--- /dev/null
+++ b/finetune/scripts/run_dpo_exec.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+set -uo pipefail; cd /wekafs/ict/hx_624/llm-ft; GPU=${GPUS:-2,3}; NAME=dpo_exec_v1; RUN=runs/lf_qwen35_9b_$NAME
+echo "[dpo] $(date) export v1 merged"; [ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_dpo.log 2>&1 || { echo "[dpo] EXPORT FAILED"; exit 1; }
+echo "[dpo] $(date) train on GPUs $GPU"; GPUS=$GPU scripts/lf_train.sh configs/lf/$NAME.yaml > logs/train_$NAME.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[dpo] TRAIN FAILED"; grep -E "Error|error" logs/train_$NAME.log | tail -5 | cut -c1-200; exit 1; }
+echo "[dpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|model_name_or_path:.*|model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml
+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[dpo] EXPORT2 FAILED"; exit 1; }
+echo "[dpo] $(date) merged"; G1=$(echo $GPU | cut -d, -f1)
+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_$NAME --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1
+echo "[dpo] $(date) eval done"; cat eval/out/qwen35_9b_$NAME/summary.json; rm -rf $RUN/merged; echo "[dpo] DONE"
diff --git a/finetune/scripts/run_exp.sh b/finetune/scripts/run_exp.sh
new file mode 100755
index 00000000..042f5554
--- /dev/null
+++ b/finetune/scripts/run_exp.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+# run_exp.sh <GPU(s)> <NAME>: train LoRA (configs/lf/lora_<NAME>.yaml) -> merge -> vLLM generation (GPU) ; Blender exec + metrics run in the background (CPU)
+set -uo pipefail
+GPU=$1; NAME=$2
+NGPU=$(echo $GPU | tr ',' '\n' | wc -l); ACC=$((8 / NGPU))
+cd /wekafs/ict/hx_624/llm-ft
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_$NAME; OUT=/wekafs/ict/hx_624/llm-ft/eval/out/qwen35_9b_lora_$NAME
+echo "[exp $NAME] $(date) train on GPU(s) $GPU (grad_accum=$ACC)"
+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_$NAME.yaml gradient_accumulation_steps=$ACC > logs/train_$NAME.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[exp $NAME] TRAIN FAILED"; exit 1; }
+echo "[exp $NAME] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $RUN|" -e "s|export_dir:.*|export_dir: $RUN/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml
+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[exp $NAME] EXPORT FAILED"; exit 1; }
+echo "[exp $NAME] $(date) merged"
+# --- generation on GPU (vLLM), then free the GPU
+( source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING TOKENIZERS_PARALLELISM=false CUDA_VISIBLE_DEVICES=$GPU
+ mkdir -p $OUT; python eval/generate_vllm.py --model $RUN/merged --prompts data/sft_v1/bench_prompts.jsonl --out $OUT --tp $NGPU --no_think --max_new_tokens 6144 2>&1 | grep -E "gen-vllm|Error|Traceback" | tail -5 ) > logs/eval_$NAME.log 2>&1
+cp $OUT/gens_shard0.jsonl $OUT/gens.jsonl 2>/dev/null || { echo "[exp $NAME] GEN FAILED"; cat logs/eval_$NAME.log | tail -5; exit 1; }
+echo "[exp $NAME] $(date) gen done: $(grep gen-vllm logs/eval_$NAME.log | tail -1)"
+rm -rf $RUN/merged
+# --- Blender exec + metrics in background (CPU only) so the GPU queue can continue
+nohup bash -c "source /wekafs/ict/hx_624/llm-ft/env.sh; cd /wekafs/ict/hx_624/llm-ft; python eval/run_bench.py --gen_dir $OUT --workers 32 --timeout 300 > $OUT/exec.log 2>&1; python eval/metrics.py --gen_dir $OUT > $OUT/metrics.log 2>&1; echo \"[exp $NAME] \$(date) eval done\"; cat $OUT/summary.json; echo \"[exp $NAME] DONE\"" >> logs/exp_$NAME.log 2>&1 &
+echo "[exp $NAME] exec+metrics running in background"
diff --git a/finetune/scripts/run_exp_4b.sh b/finetune/scripts/run_exp_4b.sh
new file mode 100755
index 00000000..d925bd02
--- /dev/null
+++ b/finetune/scripts/run_exp_4b.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+# like run_exp.sh but merges onto Qwen3.5-4B
+set -uo pipefail; GPU=$1; NAME=$2; cd /wekafs/ict/hx_624/llm-ft
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_$NAME; OUT=/wekafs/ict/hx_624/llm-ft/eval/out/qwen35_9b_lora_$NAME
+echo "[exp $NAME] $(date) train on GPU(s) $GPU"
+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_$NAME.yaml > logs/train_$NAME.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[exp $NAME] TRAIN FAILED"; exit 1; }
+echo "[exp $NAME] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|model_name_or_path:.*|model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-4B|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $RUN|" -e "s|export_dir:.*|export_dir: $RUN/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml
+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[exp $NAME] EXPORT FAILED"; exit 1; }
+echo "[exp $NAME] $(date) merged"
+GPUS=$GPU eval/run_eval_vllm.sh $RUN/merged $OUT --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1
+echo "[exp $NAME] $(date) eval done"; cat $OUT/summary.json; rm -rf $RUN/merged; echo "[exp $NAME] DONE"
diff --git a/finetune/scripts/run_exp_qwen3.sh b/finetune/scripts/run_exp_qwen3.sh
new file mode 100755
index 00000000..7eaf8223
--- /dev/null
+++ b/finetune/scripts/run_exp_qwen3.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+# like run_exp.sh but merges onto Qwen3.5-4B
+set -uo pipefail; GPU=$1; NAME=$2; cd /wekafs/ict/hx_624/llm-ft
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_$NAME; OUT=/wekafs/ict/hx_624/llm-ft/eval/out/qwen35_9b_lora_$NAME
+echo "[exp $NAME] $(date) train on GPU(s) $GPU"
+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_$NAME.yaml > logs/train_$NAME.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[exp $NAME] TRAIN FAILED"; exit 1; }
+echo "[exp $NAME] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|model_name_or_path:.*|model_name_or_path: /wekafs/ict/hx_624/models/Qwen3-8B|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $RUN|" -e "s|export_dir:.*|export_dir: $RUN/merged|" configs/lf/export_lora_template.yaml | sed "s/^template: qwen3_5/template: qwen3/" > configs/lf/export_$NAME.yaml
+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[exp $NAME] EXPORT FAILED"; exit 1; }
+echo "[exp $NAME] $(date) merged"
+GPUS=$GPU eval/run_eval_vllm.sh $RUN/merged $OUT --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1
+echo "[exp $NAME] $(date) eval done"; cat $OUT/summary.json; rm -rf $RUN/merged; echo "[exp $NAME] DONE"
diff --git a/finetune/scripts/run_full_3gpu.sh b/finetune/scripts/run_full_3gpu.sh
new file mode 100755
index 00000000..dad87edd
--- /dev/null
+++ b/finetune/scripts/run_full_3gpu.sh
@@ -0,0 +1,15 @@
+#!/bin/bash
+# wait for self-train sampling to finish (GPU0 free), stop the self-train script (its training would collide), then full FT on GPUs 0,1,3 (ZeRO-3, no offload), then eval.
+cd /wekafs/ict/hx_624/llm-ft
+until grep -q "\[self\] .*sampling done" logs/self_train_round1.log 2>/dev/null; do sleep 20; done
+for p in $(pgrep -u hx_624 -f "scripts/self_train_round.s[h]"); do kill $p; done; echo "[full3] self-train script stopped after sampling (exec/build/train to be run manually)"
+sleep 20
+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1,3 | sort -n | tail -1)" -lt 10000 ]; do sleep 15; done
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v2_3gpu
+echo "[full3] $(date) start full FT on GPUs 0,1,3 (ZeRO-3 no offload, grad_accum=3)"
+GPUS=0,1,3 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft_3gpu.yaml > logs/train_full_v2_3gpu.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[full3] TRAIN FAILED"; grep -E "Error|error" logs/train_full_v2_3gpu.log | tail -5 | cut -c1-200; exit 1; }
+echo "[full3] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+for f in preprocessor_config.json video_preprocessor_config.json vocab.json merges.txt; do [ -f $RUN/$f ] || cp /wekafs/ict/hx_624/models/Qwen3.5-9B/$f $RUN/; done
+GPUS=1 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v2_3gpu --no_think --max_new_tokens 6144 > logs/eval_full_v2_3gpu.log 2>&1
+echo "[full3] $(date) eval done"; cat eval/out/qwen35_9b_full_v2_3gpu/summary.json; echo FULL3_DONE
diff --git a/finetune/scripts/run_full_ft.sh b/finetune/scripts/run_full_ft.sh
new file mode 100755
index 00000000..8bb43da6
--- /dev/null
+++ b/finetune/scripts/run_full_ft.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+# Full-parameter SFT (ZeRO-3) on GPUs 0,1,2 after all LoRA queues finish, then eval.
+cd /wekafs/ict/hx_624/llm-ft
+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "QUEUE_DONE gpu=2" logs/queue_gpu2.log 2>/dev/null; do sleep 60; done
+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1,2 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1
+echo "[full] $(date) start full FT on GPUs 0,1,2"
+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml > logs/train_full_v1.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; exit 1; }
+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+if [ -f eval/.vllm_ok ]; then GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1
+else GPUS=0,1,2 eval/run_eval.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1; fi
+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONE
diff --git a/finetune/scripts/run_full_ft2.sh b/finetune/scripts/run_full_ft2.sh
new file mode 100755
index 00000000..6ee08257
--- /dev/null
+++ b/finetune/scripts/run_full_ft2.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+# Full-parameter SFT (ZeRO-3) on GPUs 0,1,2 after all LoRA queues finish, then eval.
+cd /wekafs/ict/hx_624/llm-ft
+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "QUEUE_DONE gpu=2" logs/exp_v3_bio_retry.log 2>/dev/null; do sleep 60; done
+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1,2 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1
+echo "[full] $(date) start full FT on GPUs 0,1,2"
+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml > logs/train_full_v1.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; exit 1; }
+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+if [ -f eval/.vllm_ok ]; then GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1
+else GPUS=0,1,2 eval/run_eval.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1; fi
+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONE
diff --git a/finetune/scripts/run_full_ft3.sh b/finetune/scripts/run_full_ft3.sh
new file mode 100755
index 00000000..b9fc1a6d
--- /dev/null
+++ b/finetune/scripts/run_full_ft3.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+# Full-parameter SFT (ZeRO-3) on GPUs 0,1,2 after all LoRA queues finish, then eval.
+cd /wekafs/ict/hx_624/llm-ft
+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "QUEUE_DONE gpu=2" logs/exp_v3_bio_retry.log 2>/dev/null && grep -q "CKPT_SWEEP_DONE v7_ep4" logs/ckpt_sweep_main.log 2>/dev/null; do sleep 60; done
+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1,2 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1
+echo "[full] $(date) start full FT on GPUs 0,1,2"
+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml > logs/train_full_v1.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; exit 1; }
+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+if [ -f eval/.vllm_ok ]; then GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1
+else GPUS=0,1,2 eval/run_eval.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1; fi
+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONE
diff --git a/finetune/scripts/run_full_ft4.sh b/finetune/scripts/run_full_ft4.sh
new file mode 100755
index 00000000..f9c4bcec
--- /dev/null
+++ b/finetune/scripts/run_full_ft4.sh
@@ -0,0 +1,19 @@
+#!/bin/bash
+# Full-parameter SFT after LoRA queues + ckpt sweeps; uses GPUs 0,1,2 (ZeRO-3) if GPU 2 is free, else GPUs 0,1 with ZeRO-3 + CPU offload. Then eval, then the postponed v2b LoRA on GPU 0.
+cd /wekafs/ict/hx_624/llm-ft
+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "CKPT_SWEEP_DONE v7_ep4" logs/ckpt_sweep_main.log 2>/dev/null; do sleep 60; done
+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1
+if [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 2)" -lt 10000 ]; then
+ echo "[full] $(date) GPU 2 free -> 3 GPUs, ZeRO-3 (no offload), grad_accum=3"
+ GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml gradient_accumulation_steps=3 > logs/train_full_v1.log 2>&1
+else
+ echo "[full] $(date) GPU 2 busy (other user) -> 2 GPUs, ZeRO-3 + CPU optimizer offload, grad_accum=4"
+ GPUS=0,1 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml deepspeed=/wekafs/ict/hx_624/llm-ft/configs/lf/ds_z3_offload.json gradient_accumulation_steps=4 > logs/train_full_v1.log 2>&1
+fi
+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; tail -30 logs/train_full_v1.log | cut -c1-200; exit 1; }
+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1
+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONE
+# postponed ablation
+scripts/run_exp.sh 0 v2b_indist16k 2>&1 | tee -a logs/exp_v2b_indist16k.log
diff --git a/finetune/scripts/run_full_ft5.sh b/finetune/scripts/run_full_ft5.sh
new file mode 100755
index 00000000..e98fa64c
--- /dev/null
+++ b/finetune/scripts/run_full_ft5.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+# Full-parameter SFT after LoRA queues + ckpt sweeps; uses GPUs 0,1,2 (ZeRO-3) if GPU 2 is free, else GPUs 0,1 with ZeRO-3 + CPU offload. Then eval, then the postponed v2b LoRA on GPU 0.
+cd /wekafs/ict/hx_624/llm-ft
+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "CKPT_SWEEP_DONE v7_ep4" logs/ckpt_sweep_main.log 2>/dev/null && grep -q "CKPT_SWEEP_DONE v3_bio" logs/ckpt_sweep_v3.log 2>/dev/null; do sleep 60; done
+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1
+if [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 2)" -lt 10000 ]; then
+ echo "[full] $(date) GPU 2 free -> 3 GPUs, ZeRO-3 (no offload), grad_accum=3"
+ GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml gradient_accumulation_steps=3 > logs/train_full_v1.log 2>&1
+else
+ echo "[full] $(date) GPU 2 busy (other user) -> 2 GPUs, ZeRO-3 + CPU optimizer offload, grad_accum=4"
+ GPUS=0,1 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml deepspeed=/wekafs/ict/hx_624/llm-ft/configs/lf/ds_z3_offload.json gradient_accumulation_steps=4 > logs/train_full_v1.log 2>&1
+fi
+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; tail -30 logs/train_full_v1.log | cut -c1-200; exit 1; }
+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1
+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONE
diff --git a/finetune/scripts/run_full_ft6.sh b/finetune/scripts/run_full_ft6.sh
new file mode 100755
index 00000000..c02a36ea
--- /dev/null
+++ b/finetune/scripts/run_full_ft6.sh
@@ -0,0 +1,22 @@
+#!/bin/bash
+# Full-parameter SFT: start as soon as the GPU-0 queue is done; use whichever of GPUs 0,1,2 are free (3 -> ZeRO-3, 2 -> ZeRO-3 + CPU offload). Then eval.
+cd /wekafs/ict/hx_624/llm-ft
+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null; do sleep 60; done
+while true; do
+ FREE=$(for g in 0 1 2; do [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i $g)" -lt 10000 ] && echo -n "$g,"; done | sed 's/,$//')
+ N=$(echo $FREE | tr ',' '\n' | grep -c .)
+ [ "$N" -ge 2 ] && break; sleep 30
+done
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1
+if [ "$N" -ge 3 ]; then
+ echo "[full] $(date) GPUs $FREE -> ZeRO-3 (no offload), grad_accum=3"
+ GPUS=$FREE scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml gradient_accumulation_steps=3 > logs/train_full_v1.log 2>&1
+else
+ echo "[full] $(date) GPUs $FREE -> ZeRO-3 + CPU optimizer offload, grad_accum=4"
+ GPUS=$FREE scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml deepspeed=/wekafs/ict/hx_624/llm-ft/configs/lf/ds_z3_offload.json gradient_accumulation_steps=4 > logs/train_full_v1.log 2>&1
+fi
+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; grep -E "Error|error" logs/train_full_v1.log | tail -5 | cut -c1-200; exit 1; }
+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+G1=$(echo $FREE | cut -d, -f1)
+GPUS=$G1 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1
+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONE
diff --git a/finetune/scripts/run_full_ft7.sh b/finetune/scripts/run_full_ft7.sh
new file mode 100755
index 00000000..0e9ca0bf
--- /dev/null
+++ b/finetune/scripts/run_full_ft7.sh
@@ -0,0 +1,22 @@
+#!/bin/bash
+# Full-parameter SFT: start as soon as the GPU-0 queue is done; use whichever of GPUs 0,1,2 are free (3 -> ZeRO-3, 2 -> ZeRO-3 + CPU offload). Then eval.
+cd /wekafs/ict/hx_624/llm-ft
+until grep -q "\[exp v2b_indist16k\] .*gen done" logs/exp_v2b_indist16k.log 2>/dev/null && grep -q "\[exp v8_r16\] .*gen done" logs/exp_v8_r16.log 2>/dev/null; do sleep 30; done; sleep 30
+while true; do
+ FREE=$(for g in 0 1 2; do [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i $g)" -lt 10000 ] && echo -n "$g,"; done | sed 's/,$//')
+ N=$(echo $FREE | tr ',' '\n' | grep -c .)
+ [ "$N" -ge 2 ] && break; sleep 30
+done
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1
+if [ "$N" -ge 3 ]; then
+ echo "[full] $(date) GPUs $FREE -> ZeRO-3 (no offload), grad_accum=3"
+ GPUS=$FREE scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml gradient_accumulation_steps=3 > logs/train_full_v1.log 2>&1
+else
+ echo "[full] $(date) GPUs $FREE -> ZeRO-3 + CPU optimizer offload, grad_accum=4"
+ GPUS=$FREE scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml deepspeed=/wekafs/ict/hx_624/llm-ft/configs/lf/ds_z3_offload.json gradient_accumulation_steps=4 > logs/train_full_v1.log 2>&1
+fi
+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; grep -E "Error|error" logs/train_full_v1.log | tail -5 | cut -c1-200; exit 1; }
+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+G1=$(echo $FREE | cut -d, -f1)
+GPUS=$G1 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1
+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONE
diff --git a/finetune/scripts/run_md_exp.sh b/finetune/scripts/run_md_exp.sh
new file mode 100755
index 00000000..e0c8f8a8
--- /dev/null
+++ b/finetune/scripts/run_md_exp.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+# run_md_exp.sh <GPU(s)> <NAME>: LoRA train (configs/lf/lora_<NAME>.yaml) -> merge -> 3DCodeBench eval + 4-dialect eval -> rm merged
+set -uo pipefail; GPU=$1; NAME=$2; NGPU=$(echo $GPU | tr ',' '\n' | wc -l); ACC=$((8 / NGPU)); cd /wekafs/ict/hx_624/llm-ft
+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_$NAME
+echo "[md $NAME] $(date) train on GPU(s) $GPU"
+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_$NAME.yaml gradient_accumulation_steps=$ACC > logs/train_$NAME.log 2>&1
+[ -f $RUN/train_results.json ] || { echo "[md $NAME] TRAIN FAILED"; exit 1; }
+echo "[md $NAME] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $RUN|" -e "s|export_dir:.*|export_dir: $RUN/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml
+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[md $NAME] EXPORT FAILED"; exit 1; }
+G1=$(echo $GPU | cut -d, -f1)
+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_lora_$NAME --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1
+echo "[md $NAME] $(date) 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_lora_$NAME/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"
+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged $NAME > logs/mdeval_$NAME.log 2>&1
+for D in cadquery openscad glsl blender; do echo "[md $NAME] $D: $(cat eval/out/md_${NAME}_$D/summary.json 2>/dev/null | tr -d '\n ' | cut -c1-200)"; done
+rm -rf $RUN/merged; echo "[md $NAME] DONE"
diff --git a/finetune/scripts/self_train_finish.sh b/finetune/scripts/self_train_finish.sh
new file mode 100755
index 00000000..2d24b1cd
--- /dev/null
+++ b/finetune/scripts/self_train_finish.sh
@@ -0,0 +1,39 @@
+#!/bin/bash
+# Finish the self-training round after its script was stopped post-sampling: exec (CPU) -> build data -> (after 3-GPU full FT) train on GPUs 0,3 -> eval. Then re-run Qwen3-8B on GPU 1.
+cd /wekafs/ict/hx_624/llm-ft; W=data/selftrain_round1; K=2
+until grep -q "self-train script stopped" logs/exp_full_v2_3gpu.log 2>/dev/null; do sleep 30; done
+source /wekafs/ict/hx_624/llm-ft/env.sh
+for i in $(seq 1 $K); do [ -f $W/gen_s$i/exec_results.jsonl ] || python eval/run_bench.py --gen_dir $W/gen_s$i --workers 48 --timeout 120 > $W/gen_s$i/exec.log 2>&1; grep "status counts" $W/gen_s$i/exec.log; done
+echo "[self] $(date) exec done"
+python - <<PY
+import json, hashlib, os, random, shutil
+random.seed(0); W="$W"; K=$K; SRC="data/sft_v1"
+prompts={json.loads(l)["task"]: json.loads(l) for l in open(f"{W}/prompts.jsonl")}
+base=[json.loads(l) for l in open(f"{SRC}/train.jsonl")]
+seen={hashlib.sha1(r["messages"][-1]["content"].encode()).hexdigest() for r in base}
+new=[]; n_ok=0
+for i in range(1,K+1):
+ for l in open(f"{W}/gen_s{i}/exec_results.jsonl"):
+ r=json.loads(l)
+ if r["status"]!="OK": continue
+ n_ok+=1
+ code=open(f"{W}/gen_s{i}/{r['task']}/code.py").read().strip()
+ if code.count("\n")<4: continue
+ h=hashlib.sha1(("```python\n"+code+"\n```").encode()).hexdigest()
+ if h in seen: continue
+ seen.add(h); p=prompts[r["task"]]
+ new.append({"messages": p["messages"]+[{"role":"assistant","content":"```python\n"+code+"\n```"}], "source":"selftrain", "id":r["task"], "prompt_kind":"self"})
+print(f"verified OK generations: {n_ok}, new unique samples: {len(new)}")
+out="data/sft_v11_selftrain"; os.makedirs(out, exist_ok=True)
+allrows=base+new; random.shuffle(allrows)
+with open(f"{out}/train.jsonl","w") as f:
+ for r in allrows: f.write(json.dumps(r, ensure_ascii=False)+"\n")
+shutil.copy(f"{SRC}/val.jsonl", f"{out}/val.jsonl")
+json.dump({"n_train":len(allrows),"n_new":len(new),"n_ok_generations":n_ok}, open(f"{out}/stats.json","w"), indent=1)
+PY
+python scripts/to_llamafactory.py --data_dir data/sft_v11_selftrain --name v11_selftrain >/dev/null && python scripts/make_lora_cfg.py v11_selftrain v11_selftrain >/dev/null
+cat data/sft_v11_selftrain/stats.json; echo "[self] $(date) data built"
+until grep -q "FULL3_DONE\|TRAIN FAILED" logs/exp_full_v2_3gpu.log 2>/dev/null; do sleep 30; done
+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,3 | sort -n | tail -1)" -lt 10000 ]; do sleep 15; done
+scripts/run_exp.sh 0,3 v11_selftrain 2>&1 | tee -a logs/exp_v11_selftrain.log
+echo "[self] DONE"
diff --git a/finetune/scripts/self_train_round.sh b/finetune/scripts/self_train_round.sh
new file mode 100755
index 00000000..d8da9ff5
--- /dev/null
+++ b/finetune/scripts/self_train_round.sh
@@ -0,0 +1,68 @@
+#!/bin/bash
+# One round of execution-verified self-training:
+# LoRA-v1 merged model -> sample K completions (T) for every training prompt -> run all in Blender -> keep OK -> v1 data + verified samples -> LoRA -> eval
+# usage: GPUS=0 scripts/self_train_round.sh <K> <T> (default K=2, T=0.8)
+set -uo pipefail
+K=${1:-2}; T=${2:-0.8}; GPU=${GPUS:-0}
+cd /wekafs/ict/hx_624/llm-ft
+NAME=v11_selftrain; SRC=data/sft_v1; WORK=data/selftrain_round1; mkdir -p $WORK
+echo "[self] $(date) start (K=$K T=$T GPU=$GPU)"
+# 0) prompts file from train.jsonl (system+user only)
+source /wekafs/ict/hx_624/llm-ft/env.sh
+python - <<PY
+import json
+rows=[json.loads(l) for l in open("$SRC/train.jsonl")]
+with open("$WORK/prompts.jsonl","w") as f:
+ for i,r in enumerate(rows):
+ f.write(json.dumps({"task": f"t{i:05d}", "messages": r["messages"][:2], "source": r.get("source")})+"\n")
+print("prompts:", len(rows))
+PY
+# 1) merged LoRA-v1
+[ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_self.log 2>&1 || { echo "[self] EXPORT FAILED"; exit 1; }
+# 2) sample K completions
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU
+for i in $(seq 1 $K); do
+ python eval/generate_vllm.py --model runs/lf_qwen35_9b_lora_v1/merged --prompts $WORK/prompts.jsonl --out $WORK/gen_s$i --tp 1 --no_think --max_new_tokens 4096 --temperature $T --seed $((100+i)) 2>&1 | grep -E "gen-vllm|Error|Traceback" | tail -3
+done
+rm -rf runs/lf_qwen35_9b_lora_v1/merged
+echo "[self] $(date) sampling done"
+# 3) execute everything in Blender (CPU)
+conda activate llmft
+for i in $(seq 1 $K); do python eval/run_bench.py --gen_dir $WORK/gen_s$i --workers 48 --timeout 120 > $WORK/gen_s$i/exec.log 2>&1; grep "status counts" $WORK/gen_s$i/exec.log; done
+echo "[self] $(date) exec done"
+# 4) build v11 = v1 + verified samples (dedup by code hash); val = v1 val
+python - <<PY
+import json, hashlib, os, random
+random.seed(0)
+W="$WORK"; K=$K
+prompts={json.loads(l)["task"]: json.loads(l) for l in open(f"{W}/prompts.jsonl")}
+base=[json.loads(l) for l in open("$SRC/train.jsonl")]
+seen={hashlib.sha1(r["messages"][-1]["content"].encode()).hexdigest() for r in base}
+new=[]; n_ok=0
+for i in range(1,K+1):
+ for l in open(f"{W}/gen_s{i}/exec_results.jsonl"):
+ r=json.loads(l)
+ if r["status"]!="OK": continue
+ n_ok+=1
+ code=open(f"{W}/gen_s{i}/{r['task']}/code.py").read().strip()
+ if code.count("\n")<4: continue
+ h=hashlib.sha1(("```python\n"+code+"\n```").encode()).hexdigest()
+ if h in seen: continue
+ seen.add(h)
+ p=prompts[r["task"]]
+ new.append({"messages": p["messages"]+[{"role":"assistant","content":"```python\n"+code+"\n```"}], "source":"selftrain", "id":r["task"], "prompt_kind":"self"})
+print(f"verified OK generations: {n_ok}, new unique samples: {len(new)}")
+out="data/sft_v11_selftrain"; os.makedirs(out, exist_ok=True)
+allrows=base+new; random.shuffle(allrows)
+with open(f"{out}/train.jsonl","w") as f:
+ for r in allrows: f.write(json.dumps(r, ensure_ascii=False)+"\n")
+import shutil; shutil.copy("$SRC/val.jsonl", f"{out}/val.jsonl")
+json.dump({"n_train":len(allrows),"n_new":len(new),"n_ok_generations":n_ok}, open(f"{out}/stats.json","w"), indent=1)
+PY
+python scripts/to_llamafactory.py --data_dir data/sft_v11_selftrain --name v11_selftrain >/dev/null && python scripts/make_lora_cfg.py $NAME v11_selftrain >/dev/null
+cat data/sft_v11_selftrain/stats.json
+echo "[self] $(date) data built -> train"
+# 5) train + eval (run_exp)
+scripts/run_exp.sh $GPU $NAME 2>&1 | tee -a logs/exp_$NAME.log
+echo "[self] DONE"
diff --git a/finetune/scripts/self_train_round2.sh b/finetune/scripts/self_train_round2.sh
new file mode 100755
index 00000000..cfa58f2c
--- /dev/null
+++ b/finetune/scripts/self_train_round2.sh
@@ -0,0 +1,59 @@
+#!/bin/bash
+# Round 2 (novel prompts): LoRA-v1 samples K completions for 9.9k articraft/deepcad-derived Blender prompts on 4 GPUs -> Blender filter -> v13 = v1 + verified -> LoRA (4-GPU DDP) -> eval
+set -uo pipefail; K=${1:-2}; T=${2:-0.8}; W=data/selftrain_round2; NAME=v13_boot; cd /wekafs/ict/hx_624/llm-ft
+echo "[r2] $(date) start K=$K T=$T"
+[ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_r2.log 2>&1 || { echo "[r2] EXPORT FAILED"; exit 1; }
+source /wekafs/ict/hx_624/llm-ft/env.sh
+python - <<PY
+import json
+rows=[json.loads(l) for l in open("$W/prompts.jsonl")]
+for g in range(4):
+ with open(f"$W/prompts_g{g}.jsonl","w") as f:
+ for r in rows[g::4]: f.write(json.dumps(r)+"\n")
+print("sharded", len(rows))
+PY
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING
+for i in $(seq 1 $K); do
+ for g in 0 1 2 3; do
+ CUDA_VISIBLE_DEVICES=$g python eval/generate_vllm.py --model runs/lf_qwen35_9b_lora_v1/merged --prompts $W/prompts_g$g.jsonl --out $W/gen_s${i}_g$g --tp 1 --no_think --max_new_tokens 4096 --temperature $T --seed $((200+i*10+g)) > $W/gen_s${i}_g$g.log 2>&1 &
+ done; wait
+ for g in 0 1 2 3; do grep -E "gen-vllm|Error" $W/gen_s${i}_g$g.log | tail -1; done
+done
+rm -rf runs/lf_qwen35_9b_lora_v1/merged; echo "[r2] $(date) sampling done"
+conda activate llmft
+for d in $W/gen_s*_g*/; do python eval/run_bench.py --gen_dir $d --workers 56 --timeout 120 > $d/exec.log 2>&1; grep "status counts" $d/exec.log; done
+echo "[r2] $(date) exec done"
+python - <<PY
+import json, hashlib, os, random, shutil, glob
+random.seed(0); W="$W"; SRC="data/sft_v1"
+prompts={}
+for f in glob.glob(f"{W}/prompts_g*.jsonl"):
+ for l in open(f): r=json.loads(l); prompts[r["task"]]=r
+base=[json.loads(l) for l in open(f"{SRC}/train.jsonl")]
+seen={hashlib.sha1(r["messages"][-1]["content"].encode()).hexdigest() for r in base}
+new=[]; n_ok=0; per_src={}
+for d in sorted(glob.glob(f"{W}/gen_s*_g*/")):
+ if not os.path.exists(d+"exec_results.jsonl"): continue
+ for l in open(d+"exec_results.jsonl"):
+ r=json.loads(l)
+ if r["status"]!="OK": continue
+ n_ok+=1
+ code=open(f"{d}/{r['task']}/code.py").read().strip()
+ if code.count("\n")<4: continue
+ h=hashlib.sha1(("```python\n"+code+"\n```").encode()).hexdigest()
+ if h in seen: continue
+ seen.add(h); p=prompts[r["task"]]; per_src[p["src"]]=per_src.get(p["src"],0)+1
+ new.append({"messages": p["messages"]+[{"role":"assistant","content":"```python\n"+code+"\n```"}], "source":"boot_"+p["src"], "id":r["task"], "prompt_kind":"boot"})
+print(f"verified OK generations: {n_ok}, new unique samples: {len(new)}, by source: {per_src}")
+out="data/sft_v13_boot"; os.makedirs(out, exist_ok=True)
+allrows=base+new; random.shuffle(allrows)
+with open(f"{out}/train.jsonl","w") as f:
+ for r in allrows: f.write(json.dumps(r, ensure_ascii=False)+"\n")
+shutil.copy(f"{SRC}/val.jsonl", f"{out}/val.jsonl")
+json.dump({"n_train":len(allrows),"n_new":len(new),"n_ok_generations":n_ok,"by_source":per_src}, open(f"{out}/stats.json","w"), indent=1)
+PY
+python scripts/to_llamafactory.py --data_dir data/sft_v13_boot --name v13_boot >/dev/null && python scripts/make_lora_cfg.py $NAME v13_boot >/dev/null
+cat data/sft_v13_boot/stats.json; echo "[r2] $(date) data built -> train on 4 GPUs"
+scripts/run_exp.sh 0,1,2,3 $NAME 2>&1 | tee -a logs/exp_$NAME.log
+echo "[r2] DONE"
diff --git a/finetune/scripts/strip_thingi_fallback.py b/finetune/scripts/strip_thingi_fallback.py
new file mode 100644
index 00000000..c06155ee
--- /dev/null
+++ b/finetune/scripts/strip_thingi_fallback.py
@@ -0,0 +1,26 @@
+"""Strip the embedded reference-mesh fallback from thingiverse-openscad-blender-5-0 'native_csg_generated' scripts.
+Writes <out>/<sample>/code.py (+ copies captions.json) for samples whose stripped script still parses."""
+import ast, glob, json, os, re, shutil, sys
+SRC = "/wekafs/ict/hx_624/data/thingiverse_5_0"; OUT = "/wekafs/ict/hx_624/data/thingiverse_5_0_native"
+os.makedirs(OUT, exist_ok=True); kept = dropped = 0
+for cp in sorted(glob.glob(f"{SRC}/*/code.py")):
+ d = os.path.dirname(cp); rep = os.path.join(d, "reports/translation_report.json")
+ st = json.load(open(rep)).get("status", "") if os.path.exists(rep) else ""
+ if st != "native_csg_generated": continue
+ code = open(cp).read()
+ # 1) remove the fallback function definition
+ code2 = re.sub(r"\ndef load_mesh_fallback\(\):.*?(?=\n(?:def |if __name__|[A-Za-z_])|\Z)", "\n", code, flags=re.S)
+ # 2) rewrite 'try: <native> except ...: load_mesh_fallback()' -> just the native call (common pattern)
+ code2 = re.sub(r"try:\n(\s+)(.*?)\n\s*except [^\n]*:\n\s+(?:[^\n]*\n)*?\s*(?:obj\s*=\s*)?load_mesh_fallback\(\)[^\n]*\n", lambda m: m.group(1) + m.group(2) + "\n", code2, flags=re.S)
+ code2 = re.sub(r"^.*load_mesh_fallback\(\).*$", "", code2, flags=re.M)
+ code2 = re.sub(r"^MESH_REFERENCE_ASSET\s*=.*$", "", code2, flags=re.M)
+ if "load_mesh_fallback" in code2 or "reference_mesh" in code2:
+ dropped += 1; continue
+ try: ast.parse(code2)
+ except SyntaxError: dropped += 1; continue
+ od = os.path.join(OUT, os.path.basename(d)); os.makedirs(od, exist_ok=True)
+ open(os.path.join(od, "code.py"), "w").write(code2)
+ for f in ["captions.json", "meta.json"]:
+ if os.path.exists(os.path.join(d, f)): shutil.copy(os.path.join(d, f), od)
+ kept += 1
+print(f"kept {kept} dropped {dropped} -> {OUT}")
diff --git a/finetune/scripts/threejs_check.sh b/finetune/scripts/threejs_check.sh
new file mode 100755
index 00000000..ab20f345
--- /dev/null
+++ b/finetune/scripts/threejs_check.sh
@@ -0,0 +1,34 @@
+#!/bin/bash
+# three.js pipeline: wait Playwright -> reference runner check -> base eval (GPU) -> LoRA (300 samples, cutoff 16k) -> eval -> open the md_big gate
+set -uo pipefail; GPU=${GPUS:-2}; cd /wekafs/ict/hx_624/llm-ft; source /wekafs/ict/hx_624/llm-ft/env.sh; export PLAYWRIGHT_BROWSERS_PATH=/wekafs/ict/hx_624/cache/ms-playwright
+until grep -q PW_INSTALL_DONE /wekafs/ict/hx_624/tools/playwright_install.log 2>/dev/null; do sleep 15; done
+echo "[3js] $(date) runner check on 6 reference samples"
+python - <<'PY'
+import json, sys, re; sys.path.insert(0,"eval"); from threejs_runner import run_threejs
+rows=[json.loads(l) for l in open("data/multidialect/threejs/test.jsonl")][:6]
+ok=0
+for i,r in enumerate(rows):
+ code=re.findall(r"```html\n(.*?)```", r["messages"][-1]["content"], flags=re.S)[0]
+ rep=run_threejs(code, f"eval/out/_3js_refcheck/{i}"); ok+=rep["status"]=="OK"; print(i, rep["status"], (rep["error"] or "")[:120], rep["latency_s"], "s")
+print("REFCHECK_OK", ok, "/", len(rows))
+PY
+echo "[3js] $(date) base eval"
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU
+python - <<'PY'
+import json
+with open("eval/out/_3js_prompts.jsonl","w") as f:
+ for l in open("data/multidialect/threejs/test.jsonl"):
+ r=json.loads(l); f.write(json.dumps({"task": r["id"].replace("/","__"), "messages": r["messages"][:2]})+"\n")
+PY
+mkdir -p eval/out/md_base_threejs; python eval/generate_vllm.py --model /wekafs/ict/hx_624/models/Qwen3.5-9B --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_base_threejs --tp 1 --no_think --max_new_tokens 12288 2>&1 | grep -E "gen-vllm|Error" | tail -1
+conda activate llmft; python eval/threejs_eval.py --gen_dir eval/out/md_base_threejs 2>&1 | tail -2
+echo "[3js] $(date) LoRA train (300 samples, cutoff 16k, 3 epochs) on GPU $GPU"
+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_md_threejs.yaml > logs/train_md_threejs.log 2>&1
+RUN=runs/lf_qwen35_9b_lora_md_threejs; [ -f $RUN/train_results.json ] || { echo "[3js] TRAIN FAILED"; touch logs/.gate_threejs; exit 1; }
+echo "[3js] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"
+sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_md_threejs.yaml
+scripts/lf_export.sh configs/lf/export_md_threejs.yaml > logs/export_md_threejs.log 2>&1 || { echo "[3js] EXPORT FAILED"; touch logs/.gate_threejs; exit 1; }
+conda activate vllm; mkdir -p eval/out/md_md_threejs_threejs; python eval/generate_vllm.py --model $RUN/merged --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_md_threejs_threejs --tp 1 --no_think --max_new_tokens 12288 2>&1 | grep -E "gen-vllm|Error" | tail -1
+touch logs/.gate_threejs; echo "[3js] gate opened for md_big"
+conda activate llmft; python eval/threejs_eval.py --gen_dir eval/out/md_md_threejs_threejs 2>&1 | tail -2
+rm -rf $RUN/merged; echo "[3js] DONE"
diff --git a/finetune/scripts/threejs_regen32k.sh b/finetune/scripts/threejs_regen32k.sh
new file mode 100755
index 00000000..8845d6fc
--- /dev/null
+++ b/finetune/scripts/threejs_regen32k.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+# re-merge the three.js LoRA (merged dir was cleaned up), re-generate with a 32k budget on GPU 2, evaluate
+cd /wekafs/ict/hx_624/llm-ft
+RUN=runs/lf_qwen35_9b_lora_md_threejs
+[ -f $RUN/merged/config.json ] || scripts/lf_export.sh configs/lf/export_md_threejs.yaml > logs/export_md_threejs_2.log 2>&1 || { echo "[3js32k] EXPORT FAILED"; exit 1; }
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=2 PLAYWRIGHT_BROWSERS_PATH=/wekafs/ict/hx_624/cache/ms-playwright
+echo "[3js32k] $(date) gen LoRA 32k"
+mkdir -p eval/out/md_md_threejs_threejs_32k
+python eval/generate_vllm.py --model $RUN/merged --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_md_threejs_threejs_32k --tp 1 --no_think --max_new_tokens 30000 --max_model_len 32768 2>&1 | grep -E "gen-vllm|Error" | tail -1
+conda activate llmft
+python eval/threejs_eval.py --gen_dir eval/out/md_md_threejs_threejs_32k 2>&1 | tail -2
+echo "[3js32k] done2"
diff --git a/finetune/scripts/threejs_t07.sh b/finetune/scripts/threejs_t07.sh
new file mode 100755
index 00000000..5029b0c9
--- /dev/null
+++ b/finetune/scripts/threejs_t07.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+cd /wekafs/ict/hx_624/llm-ft; RUN=runs/lf_qwen35_9b_lora_md_threejs
+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm
+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=2 PLAYWRIGHT_BROWSERS_PATH=/wekafs/ict/hx_624/cache/ms-playwright
+for M in "$RUN/merged md_md_threejs_T07_threejs" "/wekafs/ict/hx_624/models/Qwen3.5-9B md_base_T07_threejs"; do set -- $M
+ mkdir -p eval/out/$2; python eval/generate_vllm.py --model $1 --prompts eval/out/_3js_prompts.jsonl --out eval/out/$2 --tp 1 --no_think --max_new_tokens 12288 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm|Error" | tail -1
+done
+conda activate llmft
+for N in md_md_threejs_T07_threejs md_base_T07_threejs; do echo "== $N"; python eval/threejs_eval.py --gen_dir eval/out/$N 2>&1 | tail -2; done
+echo "[3jsT07] done"
diff --git a/finetune/scripts/to_llamafactory.py b/finetune/scripts/to_llamafactory.py
new file mode 100644
index 00000000..6d7248e9
--- /dev/null
+++ b/finetune/scripts/to_llamafactory.py
@@ -0,0 +1,26 @@
+"""Convert our messages-jsonl (system/user/assistant) into LLaMA-Factory sharegpt json + dataset_info.json entries."""
+import argparse, json, os
+ap = argparse.ArgumentParser()
+ap.add_argument("--data_dir", default="/wekafs/ict/hx_624/llm-ft/data/sft_v1")
+ap.add_argument("--out_dir", default="/wekafs/ict/hx_624/llm-ft/data/lf")
+ap.add_argument("--name", default="blender3d_v1")
+a = ap.parse_args()
+os.makedirs(a.out_dir, exist_ok=True)
+info_path = os.path.join(a.out_dir, "dataset_info.json")
+info = json.load(open(info_path)) if os.path.exists(info_path) else {}
+for split in ["train", "val"]:
+ rows = []
+ for l in open(os.path.join(a.data_dir, f"{split}.jsonl")):
+ m = json.loads(l)["messages"]
+ sys_msg = next((x["content"] for x in m if x["role"] == "system"), None)
+ conv = []
+ for x in m:
+ if x["role"] == "user": conv.append({"from": "human", "value": x["content"]})
+ elif x["role"] == "assistant": conv.append({"from": "gpt", "value": x["content"]})
+ rows.append({"conversations": conv, "system": sys_msg})
+ fn = f"{a.name}_{split}.json"
+ json.dump(rows, open(os.path.join(a.out_dir, fn), "w"), ensure_ascii=False)
+ info[f"{a.name}_{split}"] = {"file_name": fn, "formatting": "sharegpt", "columns": {"messages": "conversations", "system": "system"}}
+ print(split, len(rows), "->", fn)
+json.dump(info, open(info_path, "w"), indent=2)
+print("dataset_info:", list(info.keys()))
diff --git a/finetune/scripts/train_sft.py b/finetune/scripts/train_sft.py
new file mode 100644
index 00000000..f13e3b32
--- /dev/null
+++ b/finetune/scripts/train_sft.py
@@ -0,0 +1,129 @@
+"""SFT (full or LoRA) of a causal LM on chat-format jsonl using TRL SFTTrainer.
+
+Data: jsonl rows with {"messages":[system,user,assistant]} -> converted to conversational prompt/completion
+so that the loss is computed on the assistant completion only.
+"""
+import argparse, json, os, sys, time
+import torch
+from datasets import load_dataset
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from trl import SFTConfig, SFTTrainer
+
+def parse():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("--model", required=True)
+ ap.add_argument("--data_dir", required=True)
+ ap.add_argument("--output_dir", required=True)
+ ap.add_argument("--max_length", type=int, default=8192)
+ ap.add_argument("--epochs", type=float, default=2.0)
+ ap.add_argument("--lr", type=float, default=1e-5)
+ ap.add_argument("--warmup_ratio", type=float, default=0.03)
+ ap.add_argument("--per_device_bs", type=int, default=1)
+ ap.add_argument("--grad_accum", type=int, default=8)
+ ap.add_argument("--eval_bs", type=int, default=2)
+ ap.add_argument("--lora", action="store_true")
+ ap.add_argument("--lora_r", type=int, default=64)
+ ap.add_argument("--lora_alpha", type=int, default=128)
+ ap.add_argument("--lora_dropout", type=float, default=0.05)
+ ap.add_argument("--packing", action="store_true")
+ ap.add_argument("--no_liger", action="store_true")
+ ap.add_argument("--attn", default="auto", help="auto|flash_attention_2|sdpa|eager")
+ ap.add_argument("--grad_ckpt", action="store_true", default=True)
+ ap.add_argument("--deepspeed", default=None)
+ ap.add_argument("--save_steps", type=int, default=200)
+ ap.add_argument("--eval_steps", type=int, default=100)
+ ap.add_argument("--logging_steps", type=int, default=5)
+ ap.add_argument("--seed", type=int, default=42)
+ ap.add_argument("--max_train_samples", type=int, default=None)
+ ap.add_argument("--report_to", default="tensorboard")
+ ap.add_argument("--run_name", default=None)
+ ap.add_argument("--weight_decay", type=float, default=0.0)
+ ap.add_argument("--neftune", type=float, default=None)
+ return ap.parse_args()
+
+def to_prompt_completion(ex):
+ msgs = ex["messages"]
+ return {"prompt": msgs[:-1], "completion": [msgs[-1]]}
+
+def main():
+ a = parse()
+ torch.manual_seed(a.seed)
+ attn = a.attn
+ if attn == "auto":
+ try:
+ import flash_attn # noqa
+ attn = "flash_attention_2"
+ except Exception:
+ attn = "sdpa"
+ tok = AutoTokenizer.from_pretrained(a.model)
+ if tok.pad_token is None:
+ tok.pad_token = tok.eos_token
+
+ ds = load_dataset("json", data_files={"train": f"{a.data_dir}/train.jsonl", "val": f"{a.data_dir}/val.jsonl"})
+ keep = ["messages"]
+ ds = ds.map(to_prompt_completion, remove_columns=[c for c in ds["train"].column_names if c not in keep])
+ ds = ds.remove_columns(["messages"])
+ if a.max_train_samples:
+ ds["train"] = ds["train"].select(range(min(a.max_train_samples, len(ds["train"]))))
+ print(f"[data] train={len(ds['train'])} val={len(ds['val'])} attn={attn}", flush=True)
+
+ model_kwargs = dict(dtype=torch.bfloat16, attn_implementation=attn)
+ model = AutoModelForCausalLM.from_pretrained(a.model, **model_kwargs)
+ model.config.use_cache = False
+
+ peft_config = None
+ if a.lora:
+ from peft import LoraConfig
+ peft_config = LoraConfig(r=a.lora_r, lora_alpha=a.lora_alpha, lora_dropout=a.lora_dropout, bias="none",
+ task_type="CAUSAL_LM",
+ target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"])
+
+ cfg_kwargs = dict(
+ output_dir=a.output_dir, run_name=a.run_name or os.path.basename(a.output_dir.rstrip("/")),
+ num_train_epochs=a.epochs, learning_rate=a.lr, lr_scheduler_type="cosine", warmup_ratio=a.warmup_ratio,
+ weight_decay=a.weight_decay,
+ per_device_train_batch_size=a.per_device_bs, per_device_eval_batch_size=a.eval_bs,
+ gradient_accumulation_steps=a.grad_accum,
+ bf16=True, tf32=True,
+ gradient_checkpointing=a.grad_ckpt, gradient_checkpointing_kwargs={"use_reentrant": False},
+ logging_steps=a.logging_steps, eval_strategy="steps", eval_steps=a.eval_steps,
+ save_strategy="steps", save_steps=a.save_steps, save_total_limit=2,
+ report_to=a.report_to, seed=a.seed,
+ max_length=a.max_length, packing=a.packing,
+ completion_only_loss=True,
+ use_liger_kernel=not a.no_liger,
+ dataloader_num_workers=2, remove_unused_columns=True,
+ deepspeed=a.deepspeed,
+ ddp_find_unused_parameters=False,
+ save_only_model=True,
+ )
+ if a.neftune: cfg_kwargs["neftune_noise_alpha"] = a.neftune
+ cfg = SFTConfig(**cfg_kwargs)
+
+ trainer = SFTTrainer(model=model, args=cfg, train_dataset=ds["train"], eval_dataset=ds["val"],
+ processing_class=tok, peft_config=peft_config)
+ if trainer.accelerator.is_main_process:
+ n_tr = sum(p.numel() for p in trainer.model.parameters() if p.requires_grad)
+ n_all = sum(p.numel() for p in trainer.model.parameters())
+ print(f"[model] trainable={n_tr/1e6:.1f}M / total={n_all/1e6:.1f}M ({100*n_tr/max(n_all,1):.2f}%)", flush=True)
+ # show one tokenized example length
+ ex = trainer.train_dataset[0]
+ print("[data] example keys:", list(ex.keys()), "| input_ids len:", len(ex.get("input_ids", [])), flush=True)
+ t0 = time.time()
+ trainer.train()
+ if trainer.accelerator.is_main_process:
+ print(f"[done] train time {(time.time()-t0)/60:.1f} min", flush=True)
+ trainer.save_model(os.path.join(a.output_dir, "final"))
+ tok.save_pretrained(os.path.join(a.output_dir, "final"))
+ if a.lora and trainer.accelerator.is_main_process:
+ # also save a merged full model for easy eval
+ try:
+ merged = trainer.model.merge_and_unload()
+ merged.save_pretrained(os.path.join(a.output_dir, "final_merged"), safe_serialization=True)
+ tok.save_pretrained(os.path.join(a.output_dir, "final_merged"))
+ print("[done] merged model saved", flush=True)
+ except Exception as e:
+ print("merge failed:", e)
+
+if __name__ == "__main__":
+ main()