Official Codebase of 3DCodeVerse
Python
finetune: LLM finetuning recipes + execution-based eval for 3DCodeVerse (Qwen3.5-9B LoRA/full SFT, execution- & geometry-feedback DPO, multi-dialect scale-up; unpatched LLaMA-Factory configs, vLLM eval stack, report)
gaoypeng0ee8a4b9497d
diff --git a/README.md b/README.mdindex 588b310b..d6813a99 100644--- a/README.md+++ b/README.md@@ -8,5 +8,11 @@ The 3dcodeverse project. Top-level layout:plan → generate → gate → render → judge → refine loop, and emitsdata-flywheel records. See `harness/README.md`.+- **`finetune/`** — LLM finetuning recipes (LoRA / full-parameter SFT, execution-+ and geometry-feedback DPO via unpatched LLaMA-Factory) and the execution-based+ evaluation stack (3DCodeBench + per-dialect executors) used to train open+ models on 3DCodeVerse data; results and conclusions in `finetune/docs/REPORT.md`.+ See `finetune/README.md`.+Other large components (datasets, papers, web) live in their own top-levelfolders.diff --git a/finetune/CLAUDE.md b/finetune/CLAUDE.mdnew file mode 100644index 00000000..49c099e3--- /dev/null+++ b/finetune/CLAUDE.md@@ -0,0 +1,6 @@+# finetune/ (component notes for agents)++LLM finetuning + execution-based eval for 3DCodeVerse. Read `README.md` here and `docs/REPORT.md` (conclusions + pitfalls) first.+- Training is **unpatched LLaMA-Factory** driven by `configs/lf/*.yaml`; do not fork the framework — add a YAML + a dataset entry.+- Evaluate by *executing* the code (`eval/`): 3DCodeBench (Blender → Chamfer/F vs GT), per-dialect held-out sets, pass@N. Greedy decoding collapses on boilerplate-heavy dialects (OpenSCAD/GLSL) — use sampling there.+- Absolute paths point at the original machine; adapt `env.sh` + YAML path keys + tool paths in `eval/*runner*.py` before running.diff --git a/finetune/README.md b/finetune/README.mdnew file mode 100644index 00000000..00182c50--- /dev/null+++ b/finetune/README.md@@ -0,0 +1,36 @@+# finetune/ — LLM finetuning recipes + execution-based evaluation for 3DCodeVerse++Everything needed to reproduce the Qwen3.5-9B finetuning study on 3DCodeVerse+(text → Blender-Python / CadQuery / OpenSCAD / GLSL / three.js code), and the+conclusions in `docs/REPORT.md`. The harness (`../harness`) generates data with+API models; this folder trains and evaluates open models on that data.++**Nothing in LLaMA-Factory is patched** — all training is plain+`llamafactory-cli` YAML (`configs/lf/`) + ShareGPT JSON registered in+`data_specs/dataset_info.json`. Pinned environments are in `env/`.++## Layout++| path | what |+|---|---|+| `scripts/` | data builders (`prep_data.py`, `build_multidialect*.py`, `build_md_big.py`, `build_md_xl.py`, `build_md_bal.py`, `build_boot_data.py`, `cache_ntok.py`), thin launchers around LLaMA-Factory (`lf_train.sh`, `lf_export.sh`, `make_lora_cfg.py`), experiment pipelines train→merge→eval (`run_exp.sh`, `run_md_exp.sh`, `run_full_ft*.sh`, `*_ckpt_sweep.sh`), execution-feedback / geometry-feedback DPO rounds (`run_dpo_exec.sh`, `dpo_round2.sh`, `md_dpo_round.sh`, `md_big_dpo_round.sh`, `md_xl_dpo_round.sh`, `geo_dpo_build.sh`), bootstrapping (`self_train_round*.sh`), report tooling (`build_report_html.py`, `contact_sheet_3dcodebench.py`) |+| `configs/lf/` | one LLaMA-Factory YAML per experiment: LoRA SFT (`lora_*.yaml`), full-parameter ZeRO-3 SFT (`qwen35_9b_full_sft*.yaml`, `full_md_bal.yaml`, `ds_z3_offload.json`), DPO (`dpo_*.yaml`), LoRA merge/export (`export_*.yaml`) |+| `eval/` | vLLM generation (`generate_vllm.py`), 3DCodeBench runner (`run_bench.py`, `blender_runner.py`), geometry metrics Chamfer / F-score vs GT (`metrics.py`), pass@N / best-of-N (`best_of_n.sh`, `pass_at_n.py`), per-dialect executors (`dialect_runners.py`: CadQuery, OpenSCAD, glslangValidator; `threejs_runner.py`: Playwright headless Chromium), held-out dialect evals (`dialect_eval.py`, `blender_dialect_eval.py`, `threejs_eval.py`, `run_dialect_eval.sh`), renders for qualitative checks (`render_glb.py`) |+| `data_specs/dataset_info.json` | LLaMA-Factory dataset registry for every SFT/DPO set used (ShareGPT format; the JSON files themselves are rebuilt by `scripts/`) |+| `env/requirements-{llmft,lf,vllm}.txt` | pip freezes of the three conda envs (data/eval, LLaMA-Factory training, vLLM inference) |+| `env.sh` | shell env (caches, secrets file) used by all scripts |+| `docs/REPORT.md` | full experiment report (setup, data audit, ablations, DPO, multi-dialect, scale-up, training-amount curves, qualitative checks); `docs/PROJECT_README.md` is the original working README; `docs/assets/` renders |++## Reproducing (short version)++1. Envs (see `docs/REPORT.md` §1 for the pitfalls): `lf` = LLaMA-Factory (editable, upstream, unpatched) + torch 2.8.0+cu128 + flash-attn 2.8.3 + flash-linear-attention 0.5.2 + **triton 3.7.1** (Qwen3.5 GDN kernels on Hopper) + deepspeed; `vllm` = vLLM 0.27; `llmft` = data/eval (trimesh, cadquery, playwright, …). Blender 5.0.1 headless and an OpenSCAD AppImage are called by path.+2. Data: only `metadata.parquet` of each 3DCodeVerse subset is needed. `scripts/build_multidialect.py` → `data/multidialect/<dialect>/{train,test}.jsonl`; `scripts/build_md_*.py` build the mixes (token-length filter ≤8192, cached by `scripts/cache_ntok.py`); `scripts/to_llamafactory.py` writes ShareGPT JSON + `dataset_info.json`.+3. Train: `GPUS=0,1,2,3 scripts/lf_train.sh configs/lf/lora_md_xl.yaml gradient_accumulation_steps=2` (LoRA r64, cutoff 8192, packing, FA2) → `scripts/lf_export.sh configs/lf/export_md_xl.yaml` to merge.+4. Evaluate: `GPUS=0 eval/run_eval_vllm.sh <merged> eval/out/<name> --no_think --max_new_tokens 6144` (3DCodeBench: vLLM gen → Blender exec → Chamfer/F vs GT) and `GPUS=0 eval/run_dialect_eval.sh <merged> <name>` (held-out CadQuery/OpenSCAD/GLSL/Blender). Decode OpenSCAD/GLSL with sampling (T=0.7), not greedy.+5. Execution-feedback DPO on top of any SFT model: `scripts/md_xl_dpo_round.sh` (sample → run in each dialect's executor → (OK, FAIL) pairs → `stage: dpo`).++Paths are absolute for the machine the study ran on (`/wekafs/ict/hx_624/...`); adapt `env.sh`, the `*_dir`/`model_name_or_path` keys in `configs/lf/*.yaml`, and the tool paths at the top of `eval/*runner*.py` / `scripts/*.sh` before running elsewhere.++## Headline results (3DCodeBench, 212 tasks, execution rate / [email protected]; details in `docs/REPORT.md`)++Qwen3.5-9B zero-shot 0% → LoRA on 5k Blender samples 75–82% → + execution-feedback DPO 93.9–96.2% → 4-dialect mixes: 25.6k 67%, 102k 84.9%, 270k (all 3DCodeVerse) 90.1%; 4 H100s: LoRA ≈17k tok/s, 270k samples / 293M tokens in 4.8 h.diff --git a/finetune/configs/ds_zero2.json b/finetune/configs/ds_zero2.jsonnew file mode 100644index 00000000..30356c5b--- /dev/null+++ b/finetune/configs/ds_zero2.json@@ -0,0 +1,10 @@+{+ "bf16": {"enabled": true},+ "zero_optimization": {"stage": 2, "overlap_comm": true, "contiguous_gradients": true, "reduce_bucket_size": 5e8, "allgather_bucket_size": 5e8},+ "gradient_accumulation_steps": "auto",+ "gradient_clipping": 1.0,+ "train_batch_size": "auto",+ "train_micro_batch_size_per_gpu": "auto",+ "steps_per_print": 1e9,+ "wall_clock_breakdown": false+}diff --git a/finetune/configs/ds_zero3.json b/finetune/configs/ds_zero3.jsonnew file mode 100644index 00000000..97ca1c29--- /dev/null+++ b/finetune/configs/ds_zero3.json@@ -0,0 +1,20 @@+{+ "bf16": {"enabled": true},+ "zero_optimization": {+ "stage": 3,+ "overlap_comm": true,+ "contiguous_gradients": true,+ "reduce_bucket_size": "auto",+ "stage3_prefetch_bucket_size": "auto",+ "stage3_param_persistence_threshold": "auto",+ "stage3_max_live_parameters": 1e9,+ "stage3_max_reuse_distance": 1e9,+ "stage3_gather_16bit_weights_on_model_save": true+ },+ "gradient_accumulation_steps": "auto",+ "gradient_clipping": 1.0,+ "train_batch_size": "auto",+ "train_micro_batch_size_per_gpu": "auto",+ "steps_per_print": 1e9,+ "wall_clock_breakdown": false+}diff --git a/finetune/configs/lf/dpo_exec_v1.yaml b/finetune/configs/lf/dpo_exec_v1.yamlnew file mode 100644index 00000000..ade129d3--- /dev/null+++ b/finetune/configs/lf/dpo_exec_v1.yaml@@ -0,0 +1,42 @@+### DPO with execution-feedback preference pairs, on top of LoRA-v1 (merged)+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged+trust_remote_code: true+flash_attn: fa2+stage: dpo+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+pref_beta: 0.1+pref_loss: sigmoid+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: dpo_exec_pairs+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 2+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1+logging_steps: 5+save_steps: 1000+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 5.0e-6+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.1+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+val_size: 0.02+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100diff --git a/finetune/configs/lf/dpo_exec_v2.yaml b/finetune/configs/lf/dpo_exec_v2.yamlnew file mode 100644index 00000000..6bd40832--- /dev/null+++ b/finetune/configs/lf/dpo_exec_v2.yaml@@ -0,0 +1,42 @@+### DPO with execution-feedback preference pairs, on top of LoRA-v1 (merged)+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1/merged_r2+trust_remote_code: true+flash_attn: fa2+stage: dpo+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+pref_beta: 0.1+pref_loss: sigmoid+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: dpo_exec_pairs_r2+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 2+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2+logging_steps: 5+save_steps: 1000+save_only_model: true+plot_loss: true+overwrite_output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 5.0e-6+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.1+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+val_size: 0.02+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100diff --git a/finetune/configs/lf/dpo_md_big.yaml b/finetune/configs/lf/dpo_md_big.yamlnew file mode 100644index 00000000..e1f83b0e--- /dev/null+++ b/finetune/configs/lf/dpo_md_big.yaml@@ -0,0 +1,42 @@+### DPO with execution-feedback preference pairs, on top of LoRA-v1 (merged)+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/merged+trust_remote_code: true+flash_attn: fa2+stage: dpo+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+pref_beta: 0.1+pref_loss: sigmoid+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: dpo_md_big_pairs+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 2+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big+logging_steps: 5+save_steps: 1000+save_only_model: true+plot_loss: true+overwrite_output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 5.0e-6+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.1+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+val_size: 0.02+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100diff --git a/finetune/configs/lf/dpo_md_mixed.yaml b/finetune/configs/lf/dpo_md_mixed.yamlnew file mode 100644index 00000000..d05a50c1--- /dev/null+++ b/finetune/configs/lf/dpo_md_mixed.yaml@@ -0,0 +1,42 @@+### DPO with execution-feedback preference pairs, on top of LoRA-v1 (merged)+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed/merged+trust_remote_code: true+flash_attn: fa2+stage: dpo+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+pref_beta: 0.1+pref_loss: sigmoid+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: dpo_md_pairs+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 2+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed+logging_steps: 5+save_steps: 1000+save_only_model: true+plot_loss: true+overwrite_output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 5.0e-6+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.1+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+val_size: 0.02+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100diff --git a/finetune/configs/lf/ds_z3_offload.json b/finetune/configs/lf/ds_z3_offload.jsonnew file mode 100644index 00000000..9ab30b0b--- /dev/null+++ b/finetune/configs/lf/ds_z3_offload.json@@ -0,0 +1,22 @@+{+ "train_batch_size": "auto",+ "train_micro_batch_size_per_gpu": "auto",+ "gradient_accumulation_steps": "auto",+ "gradient_clipping": "auto",+ "zero_allow_untested_optimizer": true,+ "fp16": {"enabled": "auto", "loss_scale": 0, "loss_scale_window": 1000, "initial_scale_power": 16, "hysteresis": 2, "min_loss_scale": 1},+ "bf16": {"enabled": "auto"},+ "zero_optimization": {+ "stage": 3,+ "offload_optimizer": {"device": "cpu", "pin_memory": true},+ "overlap_comm": false,+ "contiguous_gradients": true,+ "sub_group_size": 1e9,+ "reduce_bucket_size": "auto",+ "stage3_prefetch_bucket_size": "auto",+ "stage3_param_persistence_threshold": "auto",+ "stage3_max_live_parameters": 1e9,+ "stage3_max_reuse_distance": 1e9,+ "stage3_gather_16bit_weights_on_model_save": true+ }+}diff --git a/finetune/configs/lf/export_dpo_exec_v1.yaml b/finetune/configs/lf/export_dpo_exec_v1.yamlnew file mode 100644index 00000000..19dd568a--- /dev/null+++ b/finetune/configs/lf/export_dpo_exec_v1.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_dpo_exec_v1_r2.yaml b/finetune/configs/lf/export_dpo_exec_v1_r2.yamlnew file mode 100644index 00000000..86c82577--- /dev/null+++ b/finetune/configs/lf/export_dpo_exec_v1_r2.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1/merged_r2+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_dpo_exec_v2.yaml b/finetune/configs/lf/export_dpo_exec_v2.yamlnew file mode 100644index 00000000..be216069--- /dev/null+++ b/finetune/configs/lf/export_dpo_exec_v2.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v1/merged_r2+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_dpo_md_big.yaml b/finetune/configs/lf/export_dpo_md_big.yamlnew file mode 100644index 00000000..d3a5d00d--- /dev/null+++ b/finetune/configs/lf/export_dpo_md_big.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/merged+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_dpo_md_mixed.yaml b/finetune/configs/lf/export_dpo_md_mixed.yamlnew file mode 100644index 00000000..e55a44c4--- /dev/null+++ b/finetune/configs/lf/export_dpo_md_mixed.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed/merged+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_lora_template.yaml b/finetune/configs/lf/export_lora_template.yamlnew file mode 100644index 00000000..cb180b71--- /dev/null+++ b/finetune/configs/lf/export_lora_template.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_big.yaml b/finetune/configs/lf/export_md_big.yamlnew file mode 100644index 00000000..d2760a99--- /dev/null+++ b/finetune/configs/lf/export_md_big.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_big_ck2200.yaml b/finetune/configs/lf/export_md_big_ck2200.yamlnew file mode 100644index 00000000..8b05ac97--- /dev/null+++ b/finetune/configs/lf/export_md_big_ck2200.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2200+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2200/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_big_ck2420.yaml b/finetune/configs/lf/export_md_big_ck2420.yamlnew file mode 100644index 00000000..b94ab337--- /dev/null+++ b/finetune/configs/lf/export_md_big_ck2420.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2420+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2420/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_big_ck2640.yaml b/finetune/configs/lf/export_md_big_ck2640.yamlnew file mode 100644index 00000000..d5c70c4c--- /dev/null+++ b/finetune/configs/lf/export_md_big_ck2640.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2640+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2640/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_big_ck2860.yaml b/finetune/configs/lf/export_md_big_ck2860.yamlnew file mode 100644index 00000000..38ddb9d7--- /dev/null+++ b/finetune/configs/lf/export_md_big_ck2860.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2860+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-2860/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_big_ck3080.yaml b/finetune/configs/lf/export_md_big_ck3080.yamlnew file mode 100644index 00000000..ec45a8ce--- /dev/null+++ b/finetune/configs/lf/export_md_big_ck3080.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-3080+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-3080/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_big_ck3292.yaml b/finetune/configs/lf/export_md_big_ck3292.yamlnew file mode 100644index 00000000..1ba58e15--- /dev/null+++ b/finetune/configs/lf/export_md_big_ck3292.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-3292+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big/checkpoint-3292/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_blender.yaml b/finetune/configs/lf/export_md_blender.yamlnew file mode 100644index 00000000..6adc1487--- /dev/null+++ b/finetune/configs/lf/export_md_blender.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_blender+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_blender/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_cadquery.yaml b/finetune/configs/lf/export_md_cadquery.yamlnew file mode 100644index 00000000..73477fb5--- /dev/null+++ b/finetune/configs/lf/export_md_cadquery.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_cadquery+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_cadquery/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_glsl.yaml b/finetune/configs/lf/export_md_glsl.yamlnew file mode 100644index 00000000..6bca1ca4--- /dev/null+++ b/finetune/configs/lf/export_md_glsl.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_glsl+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_glsl/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_mixed.yaml b/finetune/configs/lf/export_md_mixed.yamlnew file mode 100644index 00000000..4040c6dd--- /dev/null+++ b/finetune/configs/lf/export_md_mixed.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_openscad.yaml b/finetune/configs/lf/export_md_openscad.yamlnew file mode 100644index 00000000..26fd21c5--- /dev/null+++ b/finetune/configs/lf/export_md_openscad.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_openscad+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_openscad/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_threejs.yaml b/finetune/configs/lf/export_md_threejs.yamlnew file mode 100644index 00000000..be0d2128--- /dev/null+++ b/finetune/configs/lf/export_md_threejs.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_threejs+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_threejs/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_xl.yaml b/finetune/configs/lf/export_md_xl.yamlnew file mode 100644index 00000000..469e0218--- /dev/null+++ b/finetune/configs/lf/export_md_xl.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_xl_ck450.yaml b/finetune/configs/lf/export_md_xl_ck450.yamlnew file mode 100644index 00000000..3dc49d14--- /dev/null+++ b/finetune/configs/lf/export_md_xl_ck450.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl/checkpoint-450+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl/checkpoint-450/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_md_xl_geo.yaml b/finetune/configs/lf/export_md_xl_geo.yamlnew file mode 100644index 00000000..72d06ece--- /dev/null+++ b/finetune/configs/lf/export_md_xl_geo.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl/merged_geo+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v1.yaml b/finetune/configs/lf/export_v1.yamlnew file mode 100644index 00000000..cb180b71--- /dev/null+++ b/finetune/configs/lf/export_v1.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v10_4b.yaml b/finetune/configs/lf/export_v10_4b.yamlnew file mode 100644index 00000000..47055b84--- /dev/null+++ b/finetune/configs/lf/export_v10_4b.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-4B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v10_4b+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v10_4b/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v11_selftrain.yaml b/finetune/configs/lf/export_v11_selftrain.yamlnew file mode 100644index 00000000..0d2ecd19--- /dev/null+++ b/finetune/configs/lf/export_v11_selftrain.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11_selftrain+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11_selftrain/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v11b_selfprompts.yaml b/finetune/configs/lf/export_v11b_selfprompts.yamlnew file mode 100644index 00000000..fde0c71a--- /dev/null+++ b/finetune/configs/lf/export_v11b_selfprompts.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11b_selfprompts+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11b_selfprompts/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v12_qwen3_8b.yaml b/finetune/configs/lf/export_v12_qwen3_8b.yamlnew file mode 100644index 00000000..3fa9e2cb--- /dev/null+++ b/finetune/configs/lf/export_v12_qwen3_8b.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3-8B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v12_qwen3_8b+template: qwen3+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v12_qwen3_8b/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v13_boot.yaml b/finetune/configs/lf/export_v13_boot.yamlnew file mode 100644index 00000000..7b29279f--- /dev/null+++ b/finetune/configs/lf/export_v13_boot.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v13_boot+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v13_boot/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v1_ckpt100.yaml b/finetune/configs/lf/export_v1_ckpt100.yamlnew file mode 100644index 00000000..8e0cf7e0--- /dev/null+++ b/finetune/configs/lf/export_v1_ckpt100.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/checkpoint-100+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/checkpoint-100/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v1_rep.yaml b/finetune/configs/lf/export_v1_rep.yamlnew file mode 100644index 00000000..0572fc33--- /dev/null+++ b/finetune/configs/lf/export_v1_rep.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1_rep+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1_rep/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v2_indist.yaml b/finetune/configs/lf/export_v2_indist.yamlnew file mode 100644index 00000000..f8501a47--- /dev/null+++ b/finetune/configs/lf/export_v2_indist.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2_indist+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2_indist/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v2b_indist16k.yaml b/finetune/configs/lf/export_v2b_indist16k.yamlnew file mode 100644index 00000000..600cd46f--- /dev/null+++ b/finetune/configs/lf/export_v2b_indist16k.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2b_indist16k+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2b_indist16k/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v3_bio.yaml b/finetune/configs/lf/export_v3_bio.yamlnew file mode 100644index 00000000..ca2ceee7--- /dev/null+++ b/finetune/configs/lf/export_v3_bio.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v3_bio_ckpt100.yaml b/finetune/configs/lf/export_v3_bio_ckpt100.yamlnew file mode 100644index 00000000..58510df2--- /dev/null+++ b/finetune/configs/lf/export_v3_bio_ckpt100.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/checkpoint-100+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/checkpoint-100/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v3_bio_ckpt50.yaml b/finetune/configs/lf/export_v3_bio_ckpt50.yamlnew file mode 100644index 00000000..18e2b383--- /dev/null+++ b/finetune/configs/lf/export_v3_bio_ckpt50.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/checkpoint-50+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio/checkpoint-50/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v4_cq.yaml b/finetune/configs/lf/export_v4_cq.yamlnew file mode 100644index 00000000..51fa42e0--- /dev/null+++ b/finetune/configs/lf/export_v4_cq.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v4_cq+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v4_cq/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v5_instr.yaml b/finetune/configs/lf/export_v5_instr.yamlnew file mode 100644index 00000000..bb75a95a--- /dev/null+++ b/finetune/configs/lf/export_v5_instr.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v5_instr+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v5_instr/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v6_detail.yaml b/finetune/configs/lf/export_v6_detail.yamlnew file mode 100644index 00000000..2cdca3eb--- /dev/null+++ b/finetune/configs/lf/export_v6_detail.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v6_detail+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v6_detail/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v7_ep4.yaml b/finetune/configs/lf/export_v7_ep4.yamlnew file mode 100644index 00000000..62243783--- /dev/null+++ b/finetune/configs/lf/export_v7_ep4.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v7_ep4+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v7_ep4/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v8_r16.yaml b/finetune/configs/lf/export_v8_r16.yamlnew file mode 100644index 00000000..c96edf7d--- /dev/null+++ b/finetune/configs/lf/export_v8_r16.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v8_r16+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v8_r16/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/export_v9_lr2e4.yaml b/finetune/configs/lf/export_v9_lr2e4.yamlnew file mode 100644index 00000000..9b5a3de2--- /dev/null+++ b/finetune/configs/lf/export_v9_lr2e4.yaml@@ -0,0 +1,12 @@+### Merge a LoRA adapter into the base model (LLaMA-Factory): llamafactory-cli export <this file>+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+adapter_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v9_lr2e4+template: qwen3_5+enable_thinking: false+trust_remote_code: true+finetuning_type: lora++export_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v9_lr2e4/merged+export_size: 5+export_device: cpu+export_legacy_format: falsediff --git a/finetune/configs/lf/full_md_bal.yaml b/finetune/configs/lf/full_md_bal.yamlnew file mode 100644index 00000000..a4cb4bf0--- /dev/null+++ b/finetune/configs/lf/full_md_bal.yaml@@ -0,0 +1,41 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: full+freeze_vision_tower: true+freeze_multi_modal_projector: true+deepspeed: /wekafs/ict/hx_624/tools/LLaMA-Factory/examples/deepspeed/ds_z3_config.json+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_bal_train+eval_dataset: md_bal_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_bal_full_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_md_bal+logging_steps: 5+save_steps: 200+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 2+learning_rate: 1.0e-05+num_train_epochs: 1.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 200+save_total_limit: 2diff --git a/finetune/configs/lf/lora_md_bal.yaml b/finetune/configs/lf/lora_md_bal.yamlnew file mode 100644index 00000000..4d8ed44c--- /dev/null+++ b/finetune/configs/lf/lora_md_bal.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_bal_train+eval_dataset: md_bal_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_bal_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_bal+logging_steps: 5+save_steps: 200+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 1.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 200+save_total_limit: 12diff --git a/finetune/configs/lf/lora_md_big.yaml b/finetune/configs/lf/lora_md_big.yamlnew file mode 100644index 00000000..83cc865d--- /dev/null+++ b/finetune/configs/lf/lora_md_big.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_big_train+eval_dataset: md_big_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_big_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big+logging_steps: 5+save_steps: 220+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 220+save_total_limit: 6diff --git a/finetune/configs/lf/lora_md_blender.yaml b/finetune/configs/lf/lora_md_blender.yamlnew file mode 100644index 00000000..69b0fd2c--- /dev/null+++ b/finetune/configs/lf/lora_md_blender.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_blender_train+eval_dataset: md_blender_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_blender_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_blender+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_md_cadquery.yaml b/finetune/configs/lf/lora_md_cadquery.yamlnew file mode 100644index 00000000..c4a555b0--- /dev/null+++ b/finetune/configs/lf/lora_md_cadquery.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_cadquery_train+eval_dataset: md_cadquery_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_cadquery_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_cadquery+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_md_glsl.yaml b/finetune/configs/lf/lora_md_glsl.yamlnew file mode 100644index 00000000..b0039aa3--- /dev/null+++ b/finetune/configs/lf/lora_md_glsl.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_glsl_train+eval_dataset: md_glsl_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_glsl_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_glsl+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_md_mixed.yaml b/finetune/configs/lf/lora_md_mixed.yamlnew file mode 100644index 00000000..d3ba9ff8--- /dev/null+++ b/finetune/configs/lf/lora_md_mixed.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_mixed_train+eval_dataset: md_mixed_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_mixed_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_mixed+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_md_openscad.yaml b/finetune/configs/lf/lora_md_openscad.yamlnew file mode 100644index 00000000..89250f53--- /dev/null+++ b/finetune/configs/lf/lora_md_openscad.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_openscad_train+eval_dataset: md_openscad_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_openscad_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_openscad+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_md_threejs.yaml b/finetune/configs/lf/lora_md_threejs.yamlnew file mode 100644index 00000000..e6ad9357--- /dev/null+++ b/finetune/configs/lf/lora_md_threejs.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_threejs_train+eval_dataset: md_threejs_val+template: qwen3_5+enable_thinking: false+cutoff_len: 16384+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_threejs_len16384_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_threejs+logging_steps: 5+save_steps: 1000+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 3.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 1000+save_total_limit: 2diff --git a/finetune/configs/lf/lora_md_xl.yaml b/finetune/configs/lf/lora_md_xl.yamlnew file mode 100644index 00000000..b298f06a--- /dev/null+++ b/finetune/configs/lf/lora_md_xl.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: md_xl_train+eval_dataset: md_xl_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_md_xl_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl+logging_steps: 5+save_steps: 450+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 1.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 450+save_total_limit: 14diff --git a/finetune/configs/lf/lora_v1.yaml b/finetune/configs/lf/lora_v1.yamlnew file mode 100644index 00000000..40607d24--- /dev/null+++ b/finetune/configs/lf/lora_v1.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v10_4b.yaml b/finetune/configs/lf/lora_v10_4b.yamlnew file mode 100644index 00000000..7f660d31--- /dev/null+++ b/finetune/configs/lf/lora_v10_4b.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-4B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v10_4b+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v11_selftrain.yaml b/finetune/configs/lf/lora_v11_selftrain.yamlnew file mode 100644index 00000000..eaf67c23--- /dev/null+++ b/finetune/configs/lf/lora_v11_selftrain.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: v11_selftrain_train+eval_dataset: v11_selftrain_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v11_selftrain_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11_selftrain+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v11b_selfprompts.yaml b/finetune/configs/lf/lora_v11b_selfprompts.yamlnew file mode 100644index 00000000..2acb9306--- /dev/null+++ b/finetune/configs/lf/lora_v11b_selfprompts.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: v11b_selfprompts_train+eval_dataset: v11b_selfprompts_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v11b_selfprompts_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v11b_selfprompts+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v12_qwen3_8b.yaml b/finetune/configs/lf/lora_v12_qwen3_8b.yamlnew file mode 100644index 00000000..5e614d0c--- /dev/null+++ b/finetune/configs/lf/lora_v12_qwen3_8b.yaml@@ -0,0 +1,42 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3-8B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_qwen3_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v12_qwen3_8b+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v13_boot.yaml b/finetune/configs/lf/lora_v13_boot.yamlnew file mode 100644index 00000000..033cb840--- /dev/null+++ b/finetune/configs/lf/lora_v13_boot.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: v13_boot_train+eval_dataset: v13_boot_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v13_boot_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v13_boot+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v1_rep.yaml b/finetune/configs/lf/lora_v1_rep.yamlnew file mode 100644index 00000000..919978e9--- /dev/null+++ b/finetune/configs/lf/lora_v1_rep.yaml@@ -0,0 +1,45 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1_rep+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2+seed: 1diff --git a/finetune/configs/lf/lora_v2_indist.yaml b/finetune/configs/lf/lora_v2_indist.yamlnew file mode 100644index 00000000..a3a34010--- /dev/null+++ b/finetune/configs/lf/lora_v2_indist.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: v2_indist_train+eval_dataset: v2_indist_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v2_indist_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2_indist+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v2b_indist16k.yaml b/finetune/configs/lf/lora_v2b_indist16k.yamlnew file mode 100644index 00000000..0efc29b8--- /dev/null+++ b/finetune/configs/lf/lora_v2b_indist16k.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: v2b_indist16k_train+eval_dataset: v2b_indist16k_val+template: qwen3_5+enable_thinking: false+cutoff_len: 16384+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v2b_indist16k_len16384_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v2b_indist16k+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v3_bio.yaml b/finetune/configs/lf/lora_v3_bio.yamlnew file mode 100644index 00000000..0eafe410--- /dev/null+++ b/finetune/configs/lf/lora_v3_bio.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: v3_bio_train+eval_dataset: v3_bio_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v3_bio_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v3_bio+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v4_cq.yaml b/finetune/configs/lf/lora_v4_cq.yamlnew file mode 100644index 00000000..33a62b4d--- /dev/null+++ b/finetune/configs/lf/lora_v4_cq.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: v4_cq_train+eval_dataset: v4_cq_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v4_cq_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v4_cq+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v5_instr.yaml b/finetune/configs/lf/lora_v5_instr.yamlnew file mode 100644index 00000000..a5eaaa1c--- /dev/null+++ b/finetune/configs/lf/lora_v5_instr.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: v5_instr_train+eval_dataset: v5_instr_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v5_instr_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v5_instr+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v6_detail.yaml b/finetune/configs/lf/lora_v6_detail.yamlnew file mode 100644index 00000000..f374b184--- /dev/null+++ b/finetune/configs/lf/lora_v6_detail.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: v6_detail_train+eval_dataset: v6_detail_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_v6_detail_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v6_detail+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v7_ep4.yaml b/finetune/configs/lf/lora_v7_ep4.yamlnew file mode 100644index 00000000..afa1ec53--- /dev/null+++ b/finetune/configs/lf/lora_v7_ep4.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v7_ep4+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 4.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v8_r16.yaml b/finetune/configs/lf/lora_v8_r16.yamlnew file mode 100644index 00000000..35275900--- /dev/null+++ b/finetune/configs/lf/lora_v8_r16.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 16+lora_alpha: 32+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v8_r16+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/lora_v9_lr2e4.yaml b/finetune/configs/lf/lora_v9_lr2e4.yamlnew file mode 100644index 00000000..7d30553d--- /dev/null+++ b/finetune/configs/lf/lora_v9_lr2e4.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_blender3d_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v9_lr2e4+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0002+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/qwen35_9b_full_sft.yaml b/finetune/configs/lf/qwen35_9b_full_sft.yamlnew file mode 100644index 00000000..caf070dc--- /dev/null+++ b/finetune/configs/lf/qwen35_9b_full_sft.yaml@@ -0,0 +1,41 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: full+freeze_vision_tower: true+freeze_multi_modal_projector: true+deepspeed: /wekafs/ict/hx_624/tools/LLaMA-Factory/examples/deepspeed/ds_z3_config.json+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1+logging_steps: 5+save_steps: 60+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 3+learning_rate: 1.0e-05+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 60+save_total_limit: 1diff --git a/finetune/configs/lf/qwen35_9b_full_sft_3gpu.yaml b/finetune/configs/lf/qwen35_9b_full_sft_3gpu.yamlnew file mode 100644index 00000000..7b00d840--- /dev/null+++ b/finetune/configs/lf/qwen35_9b_full_sft_3gpu.yaml@@ -0,0 +1,41 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: full+freeze_vision_tower: true+freeze_multi_modal_projector: true+deepspeed: /wekafs/ict/hx_624/tools/LLaMA-Factory/examples/deepspeed/ds_z3_config.json+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v2_3gpu+logging_steps: 5+save_steps: 1000+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 3+learning_rate: 1.0e-05+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 50+save_total_limit: 1diff --git a/finetune/configs/lf/qwen35_9b_full_sft_4gpu.yaml b/finetune/configs/lf/qwen35_9b_full_sft_4gpu.yamlnew file mode 100644index 00000000..a461fa6d--- /dev/null+++ b/finetune/configs/lf/qwen35_9b_full_sft_4gpu.yaml@@ -0,0 +1,41 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: full+freeze_vision_tower: true+freeze_multi_modal_projector: true+deepspeed: /wekafs/ict/hx_624/tools/LLaMA-Factory/examples/deepspeed/ds_z3_config.json+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v3_4gpu+logging_steps: 5+save_steps: 1000+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 2+learning_rate: 1.0e-05+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 1000+save_total_limit: 1diff --git a/finetune/configs/lf/qwen35_9b_lora_sft.yaml b/finetune/configs/lf/qwen35_9b_lora_sft.yamlnew file mode 100644index 00000000..599e6dff--- /dev/null+++ b/finetune/configs/lf/qwen35_9b_lora_sft.yaml@@ -0,0 +1,44 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+flash_attn: fa2+stage: sft+do_train: true+finetuning_type: lora+lora_rank: 64+lora_alpha: 128+lora_dropout: 0.05+lora_target: all+freeze_vision_tower: true+freeze_multi_modal_projector: true+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+dataloader_num_workers: 4+neat_packing: true+packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1+logging_steps: 5+save_steps: 50+save_only_model: true+plot_loss: true+overwrite_output_dir: true+report_to: tensorboard+per_device_train_batch_size: 1+gradient_accumulation_steps: 8+learning_rate: 0.0001+num_train_epochs: 2.0+lr_scheduler_type: cosine+warmup_ratio: 0.05+bf16: true+gradient_checkpointing: true+ddp_timeout: 180000000+resume_from_checkpoint: null+per_device_eval_batch_size: 1+eval_strategy: steps+eval_steps: 100+save_total_limit: 2diff --git a/finetune/configs/lf/qwen35_9b_preprocess.yaml b/finetune/configs/lf/qwen35_9b_preprocess.yamlnew file mode 100644index 00000000..b3683907--- /dev/null+++ b/finetune/configs/lf/qwen35_9b_preprocess.yaml@@ -0,0 +1,17 @@+model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-9B+trust_remote_code: true+stage: sft+do_train: true+finetuning_type: lora+dataset_dir: /wekafs/ict/hx_624/llm-ft/data/lf+dataset: blender3d_v1_train+eval_dataset: blender3d_v1_val+template: qwen3_5+enable_thinking: false+cutoff_len: 8192+preprocessing_num_workers: 16+packing: true+neat_packing: true+tokenized_path: /wekafs/ict/hx_624/llm-ft/data/lf/tokenized_qwen35_v1_len8192_packed+output_dir: /wekafs/ict/hx_624/llm-ft/runs/_preprocess_tmp+overwrite_output_dir: truediff --git a/finetune/data_specs/dataset_info.json b/finetune/data_specs/dataset_info.jsonnew file mode 100644index 00000000..d4cb7d10--- /dev/null+++ b/finetune/data_specs/dataset_info.json@@ -0,0 +1,350 @@+{+ "blender3d_v1_train": {+ "file_name": "blender3d_v1_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "blender3d_v1_val": {+ "file_name": "blender3d_v1_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v2_indist_train": {+ "file_name": "v2_indist_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v2_indist_val": {+ "file_name": "v2_indist_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v3_bio_train": {+ "file_name": "v3_bio_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v3_bio_val": {+ "file_name": "v3_bio_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v4_cq_train": {+ "file_name": "v4_cq_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v4_cq_val": {+ "file_name": "v4_cq_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v5_instr_train": {+ "file_name": "v5_instr_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v5_instr_val": {+ "file_name": "v5_instr_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v6_detail_train": {+ "file_name": "v6_detail_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v6_detail_val": {+ "file_name": "v6_detail_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v2b_indist16k_train": {+ "file_name": "v2b_indist16k_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v2b_indist16k_val": {+ "file_name": "v2b_indist16k_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v11_selftrain_train": {+ "file_name": "v11_selftrain_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v11_selftrain_val": {+ "file_name": "v11_selftrain_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v13_boot_train": {+ "file_name": "v13_boot_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v13_boot_val": {+ "file_name": "v13_boot_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v11b_selfprompts_train": {+ "file_name": "v11b_selfprompts_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "v11b_selfprompts_val": {+ "file_name": "v11b_selfprompts_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "dpo_exec_pairs": {+ "file_name": "dpo_exec_pairs.json",+ "ranking": true,+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system",+ "chosen": "chosen",+ "rejected": "rejected"+ }+ },+ "md_blender_train": {+ "file_name": "md_blender_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_blender_val": {+ "file_name": "md_blender_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_cadquery_train": {+ "file_name": "md_cadquery_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_cadquery_val": {+ "file_name": "md_cadquery_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_openscad_train": {+ "file_name": "md_openscad_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_openscad_val": {+ "file_name": "md_openscad_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_glsl_train": {+ "file_name": "md_glsl_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_glsl_val": {+ "file_name": "md_glsl_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_mixed_train": {+ "file_name": "md_mixed_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_mixed_val": {+ "file_name": "md_mixed_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "dpo_exec_pairs_r2": {+ "file_name": "dpo_exec_pairs_r2.json",+ "ranking": true,+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system",+ "chosen": "chosen",+ "rejected": "rejected"+ }+ },+ "dpo_md_pairs": {+ "file_name": "dpo_md_pairs.json",+ "ranking": true,+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system",+ "chosen": "chosen",+ "rejected": "rejected"+ }+ },+ "md_big_train": {+ "file_name": "md_big_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_big_val": {+ "file_name": "md_big_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_threejs_train": {+ "file_name": "md_threejs_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_threejs_val": {+ "file_name": "md_threejs_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "dpo_md_big_pairs": {+ "file_name": "dpo_md_big_pairs.json",+ "ranking": true,+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system",+ "chosen": "chosen",+ "rejected": "rejected"+ }+ },+ "md_xl_train": {+ "file_name": "md_xl_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_xl_val": {+ "file_name": "md_xl_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_bal_train": {+ "file_name": "md_bal_train.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ },+ "md_bal_val": {+ "file_name": "md_bal_val.json",+ "formatting": "sharegpt",+ "columns": {+ "messages": "conversations",+ "system": "system"+ }+ }+}\ No newline at end of filediff --git a/finetune/docs/PROJECT_README.md b/finetune/docs/PROJECT_README.mdnew file mode 100644index 00000000..f667ec29--- /dev/null+++ b/finetune/docs/PROJECT_README.md@@ -0,0 +1,52 @@+# llm-ft — fine-tuning code LLMs for text → Blender-Python (3D) on dgx03++Everything lives under `/wekafs/ict/hx_624` (home is full). GPUs allowed: **0–3 only** (2/3 are often busy with another user's jobs → check `nvidia-smi`).++## Environments+| env | purpose | how |+|---|---|---|+| `llmft` (conda) | data prep, HF generation eval, metrics | `source llm-ft/env.sh` |+| `lf` (conda) | **LLaMA-Factory** training (`/wekafs/ict/hx_624/tools/LLaMA-Factory`) | `scripts/lf_train.sh` activates it |+| Blender 5.0.1 | executes generated scripts headlessly | `/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender` |++Anaconda 2026.07 is at `/wekafs/ict/hx_624/anaconda3` (already `conda init`-ed in `~/.bashrc`).++## Data+* Raw: `/wekafs/ict/hx_624/data/` — `3dcodebench/` (212 test tasks, GT GLB), `3dcodeverse/*.parquet` (bioinspired3d, factories, …), `thingiverse_5_0/`, `3dcodeverse/blender_distill/`.+* `scripts/prep_data.py` → `data/sft_v1/{train,val}.jsonl` (+ `bench_prompts.jsonl`, `stats.json`). The 212 benchmark factories are **excluded** from training (no contamination); only the 31 non-overlapping factories are used.+* `scripts/to_llamafactory.py` → `data/lf/*.json` + `dataset_info.json` (sharegpt format for LLaMA-Factory).++## Train (LLaMA-Factory)+```bash+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_lora_sft.yaml # LoRA+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml # full FT, DeepSpeed ZeRO-3+```+(`scripts/train_sft.py` + `scripts/launch_sft.sh` are an older plain-TRL alternative, kept for reference.)++## Evaluate on 3DCodeBench (212 tasks)+```bash+GPUS=0,1 eval/run_eval.sh <model_dir> eval/out/<name> [--no_think] [--batch_size 16]+```+= `eval/generate.py` (HF batched greedy generation, sharded over GPUs) → `eval/run_bench.py` (runs every `code.py` in headless Blender via `eval/blender_runner.py`, exports GLB) → `eval/metrics.py` (Chamfer / F-score vs GT GLB, best of 4 yaw rotations). Results: `eval/out/<name>/summary.json`.+Sanity: re-running the *reference* scripts through the harness gives Chamfer≈0.016, [email protected]≈1.0 (`eval/ref_oracle`).++## Experiment tooling (added 2026-08-21)+* `scripts/make_lora_cfg.py NAME DATANAME [--epochs --lr --rank --cutoff]` → `configs/lf/lora_NAME.yaml`+* `scripts/run_exp.sh <GPU[,GPU]> NAME` → train LoRA → merge (`lf_export.sh`) → vLLM generation → Blender exec + metrics (background) → `eval/out/qwen35_9b_lora_NAME/summary.json`+* `scripts/queue.sh <GPU> NAME1 NAME2 …` → run experiments sequentially on one GPU+* `scripts/ckpt_sweep.sh <GPU> <run_dir> NAME step1 step2 …` → evaluate intermediate adapter checkpoints+* `scripts/run_full_ft*.sh` → full-parameter ZeRO-3 SFT orchestration (3 GPUs, or 2 GPUs + CPU offload via `configs/lf/ds_z3_offload.json`)+* `eval/run_eval_vllm.sh <model> <out>` → vLLM-based eval (≈3k tok/s, 212 prompts in ~1.5 min); `eval/compare.py`, `eval/analyze_runs.py` → tables/analysis+* Results + conclusions: `REPORT.md`++## Multi-dialect tooling (added 2026-08-21 evening)+* `scripts/build_multidialect.py` → `data/multidialect/<dialect>/{train,test}.jsonl` + `report.json` (QC of every 3DCodeVerse subset; dialects: blender, cadquery, openscad, glsl)+* `scripts/build_multidialect_train.py` → token-filtered subsets `data/md_<dialect>` + `data/md_mixed` (LF datasets `md_*`)+* Executors: `eval/dialect_runners.py` (CadQuery via `cadquery` 2.8 in env `llmft`; OpenSCAD AppImage at `tools/openscad/squashfs-root/AppRun`; GLSL compile via conda-forge `glslangValidator`); Blender via `eval/blender_runner.py`+* `eval/run_dialect_eval.sh <model> <name> [dialects]` → vLLM generation + execution + reference-based F-score per dialect → `eval/out/md_<name>_<dialect>/summary.json`+* `scripts/run_md_exp.sh <GPU> md_<dialect>` → LoRA train → 3DCodeBench + 4-dialect eval++## 数据复用(2026-08-22)+- 各语言清洗结果固定在 `data/multidialect/<dialect>/{train,test}.jsonl`,所有 md_* 配比脚本(`scripts/build_md_big.py` / `build_md_xl.py`)只从这里取样;+- 每条样本的 token 数缓存在同名 `.ntok` 文件(`scripts/cache_ntok.py` 一次生成,`scripts/ntok_cache.py::load_ntok` 读取),改配比/上限不再重新 tokenize;+- LLaMA-Factory 的 `tokenized_path`(`data/lf/tokenized_<name>_len8192_packed`)缓存打包后的数据集,同一数据+cutoff 的后续训练直接复用。diff --git a/finetune/docs/assets/md_xl_3dcodebench_contact_sheet.png b/finetune/docs/assets/md_xl_3dcodebench_contact_sheet.pngnew file mode 100644index 00000000..ac2b91dcBinary files /dev/null and b/finetune/docs/assets/md_xl_3dcodebench_contact_sheet.png differdiff --git a/finetune/env.sh b/finetune/env.shnew file mode 100644index 00000000..01249a0a--- /dev/null+++ b/finetune/env.sh@@ -0,0 +1,11 @@+# source this before running anything in the project+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh+conda activate llmft+source /wekafs/ict/hx_624/.secrets/hf.env+export WORKSPACE=/wekafs/ict/hx_624+export HF_HOME=$WORKSPACE/cache/huggingface HF_XET_HIGH_PERFORMANCE=1+export TRITON_CACHE_DIR=$WORKSPACE/cache/triton TORCH_EXTENSIONS_DIR=$WORKSPACE/cache/torch_extensions+export PIP_CACHE_DIR=$WORKSPACE/cache/pip CUDA_HOME=/usr/local/cuda-12.8+export TOKENIZERS_PARALLELISM=false+mkdir -p $TRITON_CACHE_DIR $TORCH_EXTENSIONS_DIR+export VLLM_CACHE_ROOT=$WORKSPACE/cache/vllmdiff --git a/finetune/env/requirements-lf.txt b/finetune/env/requirements-lf.txtnew file mode 100644index 00000000..65ec0d10--- /dev/null+++ b/finetune/env/requirements-lf.txt@@ -0,0 +1,143 @@+absl-py==2.5.0+accelerate==1.11.0+aiofiles==24.1.0+aiohappyeyeballs==2.7.1+aiohttp==3.14.3+aiosignal==1.4.0+annotated-doc==0.0.5+annotated-types==0.8.0+antlr4-python3-runtime==4.9.3+anyio==4.14.2+attrs==26.1.0+av==16.0.0+brotli==1.2.0+causal-conv1d==1.7.0+certifi==2026.7.22+charset-normalizer==3.5.1+click==8.4.2+contourpy==1.3.3+cycler==0.12.1+datasets==4.0.0+deepspeed==0.19.5+dill==0.3.8+docstring_parser==0.18.0+einops==0.8.2+fastapi==0.141.1+ffmpy==1.0.0+filelock==3.32.3+fire==0.7.1+fla-core==0.5.2+flash-linear-attention==0.5.2+flash_attn @ https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3/flash_attn-2.8.3+cu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl#sha256=f25da18657a87fc83dc1bfb8b7751b82246e9db355510226b674fd437c34b5fb+fonttools==4.63.0+frozenlist==1.8.0+fsspec==2025.3.0+gradio==5.50.0+gradio_client==1.14.0+groovy==0.1.2+grpcio==1.83.0+h11==0.16.0+hf-xet==1.6.0+hf_transfer==0.1.9+hjson==3.1.0+httpcore==1.0.9+httpx==0.28.1+huggingface_hub==1.28.0+idna==3.19+Jinja2==3.1.6+jsonlines==4.0.0+kiwisolver==1.5.0+liger_kernel==0.8.2+-e git+https://github.com/hiyouga/LLaMA-Factory.git@c4e09c7cbe18844816af9e18a97fe465515edbcd#egg=llamafactory+Markdown==3.10.3+markdown-it-py==4.2.0+MarkupSafe==3.0.3+matplotlib==3.11.1+mdurl==0.1.2+modelscope==1.39.1+modelscope-hub==0.2.0+mpmath==1.3.0+msgpack==1.2.1+multidict==6.7.1+multiprocess==0.70.16+networkx==3.6.1+ninja==1.13.0+numpy==2.5.2+nvidia-cublas-cu12==12.8.4.1+nvidia-cuda-cupti-cu12==12.8.90+nvidia-cuda-nvrtc-cu12==12.8.93+nvidia-cuda-runtime-cu12==12.8.90+nvidia-cudnn-cu12==9.10.2.21+nvidia-cufft-cu12==11.3.3.83+nvidia-cufile-cu12==1.13.1.3+nvidia-curand-cu12==10.3.9.90+nvidia-cusolver-cu12==11.7.3.90+nvidia-cusparse-cu12==12.5.8.93+nvidia-cusparselt-cu12==0.7.1+nvidia-nccl-cu12==2.27.3+nvidia-nvjitlink-cu12==12.8.93+nvidia-nvshmem-cu12==3.3.20+nvidia-nvtx-cu12==12.8.90+omegaconf==2.3.1+orjson==3.12.0+packaging==26.3+pandas==2.3.3+peft==0.18.1+pillow==11.3.0+propcache==0.5.2+protobuf==7.36.0+psutil==7.2.2+py-cpuinfo==9.0.0+pyarrow==25.0.1+pydantic==2.12.3+pydantic_core==2.41.4+pydub==0.25.1+Pygments==2.21.0+pyparsing==3.3.2+python-dateutil==2.9.0.post0+python-multipart==0.0.32+pytz==2026.3.post1+PyYAML==6.0.3+regex==2026.7.19+requests==2.34.2+rich==15.0.0+ruff==0.16.4+safehttpx==0.1.7+safetensors==0.8.0+scipy==1.18.0+semantic-version==2.10.0+sentencepiece==0.2.2+setuptools==84.0.0+shellingham==1.5.4+shtab==1.11.0+six==1.17.0+sse-starlette==3.4.8+starlette==0.52.1+sympy==1.14.0+tensorboard==2.21.0+tensorboard-data-server==0.7.2+termcolor==3.3.0+tiktoken==0.14.0+tokenizers==0.22.2+tomlkit==0.13.3+torch==2.8.0+cu128+torchaudio==2.8.0+cu128+torchdata==0.11.0+torchvision==0.23.0+cu128+tqdm==4.70.0+transformers==5.8.0+trimesh==5.0.0+triton==3.7.1+trl==0.24.0+typer==0.27.1+typing-inspection==0.4.4+typing_extensions==4.16.0+tyro==0.8.14+tzdata==2026.3+urllib3==2.7.0+uvicorn==0.52.4+websockets==15.0.1+Werkzeug==3.1.8+wheel==0.48.0+xxhash==4.0.1+yarl==1.24.5diff --git a/finetune/env/requirements-llmft.txt b/finetune/env/requirements-llmft.txtnew file mode 100644index 00000000..191f5e6a--- /dev/null+++ b/finetune/env/requirements-llmft.txt@@ -0,0 +1,167 @@+absl-py==2.5.0+accelerate==1.14.0+aiohappyeyeballs==2.7.1+aiohttp==3.14.3+aiosignal==1.4.0+annotated-doc==0.0.5+annotated-types==0.8.0+anyio==4.14.2+attrs==26.1.0+bitsandbytes==0.50.1+cadquery==2.8.0+cadquery-ocp==7.9.3.1.1+cadquery-ocp-proxy==7.9.3.1.1+casadi==3.7.2+certifi==2026.7.22+cffi==2.1.1+charset-normalizer==3.5.1+click==8.4.2+contourpy==1.3.3+cryptography==50.0.0+cycler==0.12.1+dataclasses-json==0.6.7+datasets==5.0.1+deepspeed==0.19.5+Deprecated==1.3.1+dill==0.4.1+dnspython==2.8.0+einops==0.8.2+email-validator==2.3.0+ezdxf==1.4.4+filelock==3.32.3+fla-core==0.5.2+flash-linear-attention==0.5.2+flash_attn @ https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3/flash_attn-2.8.3+cu12torch2.9cxx11abiTRUE-cp312-cp312-linux_x86_64.whl#sha256=4e2f9e39313266b1544b68138b15b91ee6221eccf14f7902b7c6620351340810+fonttools==4.63.0+frozenlist==1.8.0+fsspec==2026.6.0+greenlet==3.5.5+grpcio==1.83.0+h11==0.16.0+hf-xet==1.6.0+hf_transfer==0.1.9+hjson==3.1.0+httpcore==1.0.9+httpx==0.28.1+huggingface_hub==1.28.0+id==1.6.1+idna==3.19+Jinja2==3.1.6+joblib==1.5.3+jsonlines==4.0.0+kernels==0.16.0+kernels-data==0.16.0+kiwisolver==1.5.0+liger_kernel==0.8.2+llvmlite==0.49.0+Markdown==3.10.3+markdown-it-py==4.2.0+MarkupSafe==3.0.3+marshmallow==3.26.2+matplotlib==3.11.1+mdurl==0.1.2+more-itertools==11.1.0+mpmath==1.3.0+msgpack==1.2.1+multidict==6.7.1+multimethod==1.12+multiprocess==0.70.19+mypy_extensions==1.1.0+narwhals==2.25.0+networkx==3.6.1+ninja==1.13.0+nlopt==2.11.0+numba==0.67.0+numpy==2.5.2+nvidia-cublas-cu12==12.8.4.1+nvidia-cuda-cupti-cu12==12.8.90+nvidia-cuda-nvrtc-cu12==12.8.93+nvidia-cuda-runtime-cu12==12.8.90+nvidia-cudnn-cu12==9.10.2.21+nvidia-cufft-cu12==11.3.3.83+nvidia-cufile-cu12==1.13.1.3+nvidia-curand-cu12==10.3.9.90+nvidia-cusolver-cu12==11.7.3.90+nvidia-cusparse-cu12==12.5.8.93+nvidia-cusparselt-cu12==0.7.1+nvidia-nccl-cu12==2.27.5+nvidia-nvjitlink-cu12==12.8.93+nvidia-nvshmem-cu12==3.3.20+nvidia-nvtx-cu12==12.8.90+opentelemetry-api==1.44.0+packaging==26.3+pandas==3.0.5+peft==0.20.0+pillow==12.3.0+platformdirs==4.11.3+playwright==1.62.0+propcache==0.5.2+protobuf==7.36.0+psutil==7.2.2+py-cpuinfo==9.0.0+pyarrow==25.0.1+pyasn1==0.6.4+pycparser==3.0+pydantic==2.13.4+pydantic_core==2.46.4+pyee==13.0.1+pygltflib==1.16.5+Pygments==2.21.0+PyJWT==2.13.0+pyOpenSSL==26.4.0+pyparsing==3.3.2+python-dateutil==2.9.0.post0+PyYAML==6.0.3+regex==2026.7.19+requests==2.34.2+rfc3161-client==1.0.8+rfc8785==0.1.4+rich==15.0.0+runtype==0.5.3+safetensors==0.8.0+scikit-learn==1.9.0+scipy==1.18.0+securesystemslib==1.4.0+sentencepiece==0.2.2+sentry-sdk==2.68.0+setuptools==84.0.0+shellingham==1.5.4+sigstore==4.5.0+sigstore-models==0.0.6+sigstore-rekor-types==0.0.18+six==1.17.0+sympy==1.14.0+tensorboard==2.21.0+tensorboard-data-server==0.7.2+threadpoolctl==3.6.0+tiktoken==0.14.0+tokenizers==0.22.2+tomlkit==0.15.1+torch==2.9.1+cu128+torchvision==0.24.1+cu128+tqdm==4.70.0+trame==3.13.2+trame-client==3.13.6+trame-common==1.2.7+trame-components==2.5.0+trame-server==3.14.0+trame-vtk==2.11.16+trame-vuetify==3.2.5+transformers==5.15.1+trimesh==5.0.0+triton==3.5.1+trl==1.10.0+tuf==7.0.0+typer==0.27.1+typing-inspect==0.9.0+typing-inspection==0.4.4+typing_extensions==4.16.0+urllib3==2.7.0+vtk==9.6.2+wandb==0.28.2+Werkzeug==3.1.8+wheel==0.48.0+wrapt==2.3.0+wslink==2.5.7+xxhash==4.0.1+yarl==1.24.5diff --git a/finetune/env/requirements-vllm.txt b/finetune/env/requirements-vllm.txtnew file mode 100644index 00000000..0e1bba95--- /dev/null+++ b/finetune/env/requirements-vllm.txt@@ -0,0 +1,200 @@+aiohappyeyeballs==2.7.1+aiohttp==3.14.3+aiosignal==1.4.0+annotated-doc==0.0.5+annotated-types==0.8.0+anthropic==1.0.0+anyio==4.14.2+apache-tvm-ffi==0.1.11+astor==0.8.1+attrs==26.1.0+blake3==1.0.9+cachetools==7.1.7+cbor2==6.1.4+certifi==2026.7.22+cffi==2.1.1+charset-normalizer==3.5.1+click==8.4.2+cloudpickle==3.1.2+compressed-tensors==0.17.0+cryptography==50.0.0+cuda-bindings==12.9.7+cuda-core==1.1.1+cuda-pathfinder==1.6.1+cuda-python==12.9.7+cuda-tile==1.5.0+cuda-toolkit==12.9.1+depyf==0.20.0+detect-installer==0.1.0+dill==0.4.1+dnspython==2.8.0+docstring_parser==0.18.0+einops==0.8.2+email-validator==2.3.0+fastapi==0.136.3+fastapi-cli==0.0.32+fastapi-cloud-cli==0.23.0+fastar==0.12.0+fastsafetensors==0.3.3+filelock==3.32.3+fla-core==0.5.2+flash-linear-attention==0.5.2+flashinfer-python==0.6.16.post3+frozenlist==1.8.0+fsspec==2026.7.0+googleapis-common-protos==1.75.1+grpcio==1.83.0+h11==0.16.0+hf-xet==1.6.0+httpcore==1.0.9+httpcore2==2.12.0+httptools==0.8.0+httpx==0.28.1+httpx2==2.12.0+huggingface_hub==1.28.0+humming-kernels==0.1.10+idna==3.19+ijson==3.5.1+interegular==0.3.3+Jinja2==3.1.6+jiter==0.16.0+jmespath==1.1.0+jsonschema==4.26.0+jsonschema-specifications==2025.9.1+lark==1.2.2+llguidance==1.7.6+llvmlite==0.47.0+lm-format-enforcer==0.11.3+loguru==0.7.3+markdown-it-py==4.2.0+MarkupSafe==3.0.3+mcp==2.0.0+mcp-types==2.0.0+mdurl==0.1.2+mistral_common==1.11.7+ml_dtypes==0.6.0+model-hosting-container-standards==0.1.16+mpmath==1.3.0+msgspec==0.21.1+multidict==6.7.1+nccl4py==0.4.1+networkx==3.6.1+ninja==1.13.0+numba==0.65.0+numpy==2.3.5+nvidia-cublas-cu12==12.9.1.4+nvidia-cuda-cccl-cu12==12.9.27+nvidia-cuda-cupti-cu12==12.9.79+nvidia-cuda-nvcc-cu12==12.9.86+nvidia-cuda-nvdisasm==13.3.73+nvidia-cuda-nvrtc-cu12==12.9.86+nvidia-cuda-runtime-cu12==12.9.79+nvidia-cudnn-cu12==9.20.0.48+nvidia-cudnn-frontend==1.27.0+nvidia-cufft-cu12==11.4.1.4+nvidia-cufile-cu12==1.14.1.1+nvidia-curand-cu12==10.3.10.19+nvidia-cusolver-cu12==11.7.5.82+nvidia-cusparse-cu12==12.5.10.65+nvidia-cusparselt-cu12==0.8.1+nvidia-cutlass-dsl==4.6.0+nvidia-cutlass-dsl-libs-base==4.6.0+nvidia-cutlass-dsl-libs-core==4.6.0+nvidia-cutlass-dsl-libs-cu12==4.6.0+nvidia-ml-py==13.610.43+nvidia-nccl-cu12==2.29.7+nvidia-nvjitlink-cu12==12.9.86+nvidia-nvshmem-cu12==3.4.5+nvidia-nvtx-cu12==12.9.79+nvtx==0.2.15+openai==3.3.1+openai-harmony==0.0.8+opencv-python-headless==5.0.0.93+opentelemetry-api==1.44.0+opentelemetry-exporter-otlp==1.44.0+opentelemetry-exporter-otlp-proto-common==1.44.0+opentelemetry-exporter-otlp-proto-grpc==1.44.0+opentelemetry-exporter-otlp-proto-http==1.44.0+opentelemetry-proto==1.44.0+opentelemetry-sdk==1.44.0+opentelemetry-semantic-conventions==0.65b0+opentelemetry-semantic-conventions-ai==0.5.1+outlines_core==0.2.14+packaging==26.2+pandas==3.0.5+partial-json-parser==0.2.1.1.post7+pillow==12.3.0+prometheus-fastapi-instrumentator==8.1.0+prometheus_client==0.26.0+propcache==0.5.2+protobuf==6.33.6+psutil==7.2.2+py-cpuinfo==9.0.0+pyarrow==25.0.1+pybase64==1.5.0+pycountry==26.2.16+pycparser==3.0+pydantic==2.13.4+pydantic-extra-types==2.11.1+pydantic-settings==2.15.0+pydantic_core==2.46.4+pyelftools==0.33+Pygments==2.21.0+PyJWT==2.13.0+pynvvideocodec==2.0.4+python-dateutil==2.9.0.post0+python-dotenv==1.2.3+python-json-logger==4.2.0+python-multipart==0.0.32+PyYAML==6.0.3+pyzmq==27.2.0+quack-kernels==0.6.1+referencing==0.37.0+regex==2026.7.19+requests==2.34.2+rich==15.0.0+rich-toolkit==0.20.3+rignore==0.8.1+rpds-py==2026.6.3+safetensors==0.8.0+scipy==1.18.0+sentencepiece==0.2.2+sentry-sdk==2.68.0+setproctitle==1.3.7+setuptools==80.10.2+shellingham==1.5.4+six==1.17.0+sniffio==1.3.1+sse-starlette==3.4.8+starlette==1.6.0+supervisor==4.3.0+sympy==1.14.0+tabulate==0.10.0+tiktoken==0.14.0+tilelang==0.1.12+tokenizers==0.22.2+tokenspeed-mla==0.1.8+tokenspeed-triton==3.8.10.post20260721+torch==2.13.0+cu129+torch_c_dlpack_ext==0.1.5+torchaudio==2.11.0+cu129+torchcodec==0.16.0+cu129+torchvision==0.28.0+cu129+tqdm==4.70.0+transformers==5.15.1+trimesh==5.0.0+triton==3.7.1+truststore==0.10.4+typer==0.27.1+typing-inspection==0.4.4+typing_extensions==4.16.0+urllib3==2.7.0+uvicorn==0.52.4+uvloop==0.22.1+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl#sha256=bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b+watchfiles==1.2.0+websockets==17.0.1+wheel==0.47.0+xgrammar==0.2.3+yarl==1.24.5+z3-solver==4.15.4.0diff --git a/finetune/eval/.vllm_ok b/finetune/eval/.vllm_oknew file mode 100644index 00000000..e69de29bdiff --git a/finetune/eval/all_results.json b/finetune/eval/all_results.jsonnew file mode 100644index 00000000..5e0e6ce2--- /dev/null+++ b/finetune/eval/all_results.json@@ -0,0 +1,262 @@+[+ {+ "name": "qwen25coder7b_base",+ "exec_ok": 18,+ "exec_rate": 0.0849,+ "cd": 0.312,+ "f05": 0.2312,+ "f10": 0.3936,+ "f05_all": 0.0142,+ "f10_all": 0.0241,+ "avg_tok": 1040,+ "train_loss": null,+ "train_min": 0.0+ },+ {+ "name": "qwen35_9b_base_nothink",+ "exec_ok": 0,+ "exec_rate": 0.0,+ "cd": null,+ "f05": null,+ "f10": null,+ "f05_all": 0.0,+ "f10_all": 0.0,+ "avg_tok": 3520,+ "train_loss": null,+ "train_min": 0.0+ },+ {+ "name": "qwen35_9b_base_nothink_lenient",+ "exec_ok": 4,+ "exec_rate": 0.0189,+ "cd": 0.2648,+ "f05": 0.2389,+ "f10": 0.4858,+ "f05_all": 0.0045,+ "f10_all": 0.0092,+ "avg_tok": 0,+ "train_loss": null,+ "train_min": 0.0+ },+ {+ "name": "qwen35_9b_full_v1_ckpt60",+ "exec_ok": 151,+ "exec_rate": 0.7123,+ "cd": 0.1996,+ "f05": 0.3806,+ "f10": 0.6036,+ "f05_all": 0.2693,+ "f10_all": 0.427,+ "avg_tok": 874,+ "train_loss": null,+ "train_min": 0.0+ },+ {+ "name": "qwen35_9b_lora_v1",+ "exec_ok": 159,+ "exec_rate": 0.75,+ "cd": 0.2012,+ "f05": 0.3823,+ "f10": 0.5979,+ "f05_all": 0.2849,+ "f10_all": 0.4456,+ "avg_tok": 774,+ "train_loss": 0.26390162311376086,+ "train_min": 30.171525+ },+ {+ "name": "qwen35_9b_lora_v1_T07_s1",+ "exec_ok": 162,+ "exec_rate": 0.7642,+ "cd": 0.2008,+ "f05": 0.3716,+ "f10": 0.5925,+ "f05_all": 0.2787,+ "f10_all": 0.4444,+ "avg_tok": 562,+ "train_loss": null,+ "train_min": 0.0+ },+ {+ "name": "qwen35_9b_lora_v1_T07_s2",+ "exec_ok": 141,+ "exec_rate": 0.6651,+ "cd": 0.1974,+ "f05": 0.3822,+ "f10": 0.6028,+ "f05_all": 0.2524,+ "f10_all": 0.3981,+ "avg_tok": 643,+ "train_loss": null,+ "train_min": 0.0+ },+ {+ "name": "qwen35_9b_lora_v1_T07_s3",+ "exec_ok": 137,+ "exec_rate": 0.6462,+ "cd": 0.195,+ "f05": 0.4005,+ "f10": 0.6189,+ "f05_all": 0.2588,+ "f10_all": 0.3999,+ "avg_tok": 969,+ "train_loss": null,+ "train_min": 0.0+ },+ {+ "name": "qwen35_9b_lora_v1_T07_s4",+ "exec_ok": 159,+ "exec_rate": 0.75,+ "cd": 0.1994,+ "f05": 0.3862,+ "f10": 0.6118,+ "f05_all": 0.286,+ "f10_all": 0.4531,+ "avg_tok": 666,+ "train_loss": null,+ "train_min": 0.0+ },+ {+ "name": "qwen35_9b_lora_v1_ckpt100",+ "exec_ok": 168,+ "exec_rate": 0.7925,+ "cd": 0.198,+ "f05": 0.3848,+ "f10": 0.5995,+ "f05_all": 0.3013,+ "f10_all": 0.4694,+ "avg_tok": 705,+ "train_loss": 0.26390162311376086,+ "train_min": 30.171525+ },+ {+ "name": "qwen35_9b_lora_v2_indist",+ "exec_ok": 131,+ "exec_rate": 0.6179,+ "cd": 0.1864,+ "f05": 0.4112,+ "f10": 0.6422,+ "f05_all": 0.2521,+ "f10_all": 0.3938,+ "avg_tok": 1360,+ "train_loss": 0.29489868678725684,+ "train_min": 37.09996666666667+ },+ {+ "name": "qwen35_9b_lora_v2b_indist16k",+ "exec_ok": 114,+ "exec_rate": 0.5377,+ "cd": 0.1969,+ "f05": 0.4144,+ "f10": 0.622,+ "f05_all": 0.2229,+ "f10_all": 0.3345,+ "avg_tok": 2579,+ "train_loss": 0.34209747883406555,+ "train_min": 58.625461666666666+ },+ {+ "name": "qwen35_9b_lora_v3_bio",+ "exec_ok": 156,+ "exec_rate": 0.7358,+ "cd": 0.205,+ "f05": 0.3804,+ "f10": 0.5968,+ "f05_all": 0.2781,+ "f10_all": 0.4363,+ "avg_tok": 539,+ "train_loss": 0.202895405292511,+ "train_min": 24.804971666666667+ },+ {+ "name": "qwen35_9b_lora_v3_bio_ckpt100",+ "exec_ok": 154,+ "exec_rate": 0.7264,+ "cd": 0.2149,+ "f05": 0.3683,+ "f10": 0.588,+ "f05_all": 0.264,+ "f10_all": 0.4216,+ "avg_tok": 563,+ "train_loss": 0.202895405292511,+ "train_min": 24.804971666666667+ },+ {+ "name": "qwen35_9b_lora_v3_bio_ckpt50",+ "exec_ok": 152,+ "exec_rate": 0.717,+ "cd": 0.211,+ "f05": 0.3753,+ "f10": 0.5863,+ "f05_all": 0.2673,+ "f10_all": 0.4176,+ "avg_tok": 523,+ "train_loss": 0.202895405292511,+ "train_min": 24.804971666666667+ },+ {+ "name": "qwen35_9b_lora_v4_cq",+ "exec_ok": 157,+ "exec_rate": 0.7406,+ "cd": 0.2008,+ "f05": 0.3751,+ "f10": 0.5956,+ "f05_all": 0.2742,+ "f10_all": 0.4355,+ "avg_tok": 694,+ "train_loss": 0.18613976782018488,+ "train_min": 67.80998000000001+ },+ {+ "name": "qwen35_9b_lora_v5_instr",+ "exec_ok": 143,+ "exec_rate": 0.6745,+ "cd": 0.1988,+ "f05": 0.4025,+ "f10": 0.6094,+ "f05_all": 0.2696,+ "f10_all": 0.4082,+ "avg_tok": 876,+ "train_loss": 0.26363916967123274,+ "train_min": 31.889743333333335+ },+ {+ "name": "qwen35_9b_lora_v6_detail",+ "exec_ok": 139,+ "exec_rate": 0.6557,+ "cd": 0.2077,+ "f05": 0.3688,+ "f10": 0.5878,+ "f05_all": 0.2401,+ "f10_all": 0.3826,+ "avg_tok": 1128,+ "train_loss": 0.2620609815304096,+ "train_min": 33.901199999999996+ },+ {+ "name": "qwen35_9b_lora_v7_ep4",+ "exec_ok": 147,+ "exec_rate": 0.6934,+ "cd": 0.2052,+ "f05": 0.3799,+ "f10": 0.6022,+ "f05_all": 0.2616,+ "f10_all": 0.4147,+ "avg_tok": 857,+ "train_loss": 0.20162874489786134,+ "train_min": 61.887908333333336+ },+ {+ "name": "qwen35_9b_lora_v8_r16",+ "exec_ok": 162,+ "exec_rate": 0.7642,+ "cd": 0.2096,+ "f05": 0.3717,+ "f10": 0.5902,+ "f05_all": 0.284,+ "f10_all": 0.451,+ "avg_tok": 667,+ "train_loss": 0.31294202780912794,+ "train_min": 31.807015+ }+]\ No newline at end of filediff --git a/finetune/eval/analyze_runs.py b/finetune/eval/analyze_runs.pynew file mode 100644index 00000000..8d6d2edc--- /dev/null+++ b/finetune/eval/analyze_runs.py@@ -0,0 +1,32 @@+"""Cross-run analysis: per-task success overlap, code length vs success, per-category success. Usage: analyze_runs.py run1 run2 ..."""+import json, os, sys, collections, numpy as np+OUT = "/wekafs/ict/hx_624/llm-ft/eval/out"+runs = sys.argv[1:]+data = {}+for r in runs:+ ex = {json.loads(l)["task"]: json.loads(l) for l in open(f"{OUT}/{r}/exec_results.jsonl")}+ gens = {json.loads(l)["task"]: json.loads(l) for f in os.listdir(f"{OUT}/{r}") if f.startswith("gens_shard") for l in open(f"{OUT}/{r}/{f}")}+ met = {json.loads(l)["task"]: json.loads(l) for l in open(f"{OUT}/{r}/metrics.jsonl")} if os.path.exists(f"{OUT}/{r}/metrics.jsonl") else {}+ data[r] = (ex, gens, met)+tasks = sorted(next(iter(data.values()))[0].keys())+print("=== success by output length bucket (tokens) ===")+for r in runs:+ ex, gens, met = data[r]; b = collections.defaultdict(lambda: [0, 0])+ for t in tasks:+ n = gens.get(t, {}).get("n_new_tokens", 0); k = "<500" if n < 500 else "<1000" if n < 1000 else "<2000" if n < 2000 else "<4000" if n < 4000 else ">=4000"+ b[k][1] += 1; b[k][0] += ex[t]["status"] == "OK"+ print(f"{r:32s}", {k: f"{v[0]}/{v[1]}" for k, v in sorted(b.items())})+print("\n=== pairwise overlap of OK tasks ===")+oks = {r: {t for t in tasks if data[r][0][t]["status"] == "OK"} for r in runs}+for i, a in enumerate(runs):+ for b_ in runs[i+1:]:+ print(f"{a} ∩ {b_}: {len(oks[a] & oks[b_])} | only {a}: {len(oks[a]-oks[b_])} | only {b_}: {len(oks[b_]-oks[a])}")+print("\n=== per-task category (first word of task name) success counts for each run ===")+cats = collections.defaultdict(list)+for t in tasks: cats[t[:1]].append(t)+print("(by first letter, just to eyeball)")+for r in runs:+ ex = data[r][0]; print(f"{r:32s}", " ".join(f"{c}:{sum(ex[t]['status']=='OK' for t in ts)}/{len(ts)}" for c, ts in sorted(cats.items())))+print("\n=== best/worst tasks ([email protected]) in", runs[-1], "===")+met = data[runs[-1]][2]; sc = sorted([(m.get("[email protected]", -1), t) for t, m in met.items() if "[email protected]" in m], reverse=True)+print("best:", sc[:8]); print("worst:", sc[-8:])diff --git a/finetune/eval/best_of_n.sh b/finetune/eval/best_of_n.shnew file mode 100755index 00000000..f244e922--- /dev/null+++ b/finetune/eval/best_of_n.sh@@ -0,0 +1,17 @@+#!/bin/bash+# usage: GPUS=0 eval/best_of_n.sh <model_dir> <out_prefix> <N> [temperature]+# N sampled generations (different seeds) -> exec each -> pass@N (union) via eval/pass_at_n.py+set -uo pipefail+MODEL=$(readlink -f "$1"); PFX=$2; N=$3; T=${4:-0.7}+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=${GPUS:-0}+cd /wekafs/ict/hx_624/llm-ft+for i in $(seq 1 $N); do+ OUT=eval/out/${PFX}_s$i; mkdir -p $OUT+ python eval/generate_vllm.py --model $MODEL --prompts data/sft_v1/bench_prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 6144 --temperature $T --seed $i 2>&1 | grep -E "gen-vllm|Error" | tail -2+ cp $OUT/gens_shard0.jsonl $OUT/gens.jsonl+done+conda activate llmft+for i in $(seq 1 $N); do OUT=eval/out/${PFX}_s$i; python eval/run_bench.py --gen_dir $OUT --workers 32 --timeout 300 > $OUT/exec.log 2>&1; python eval/metrics.py --gen_dir $OUT > $OUT/metrics.log 2>&1; grep "status counts" $OUT/exec.log; done+python eval/pass_at_n.py $PFX $N+echo BEST_OF_N_DONEdiff --git a/finetune/eval/blender_dialect_eval.py b/finetune/eval/blender_dialect_eval.pynew file mode 100644index 00000000..0b2a257c--- /dev/null+++ b/finetune/eval/blender_dialect_eval.py@@ -0,0 +1,44 @@+"""Held-out Blender test (bioinspired3d-style): execute generated & reference scripts in Blender, compare meshes."""+import argparse, json, os, re, subprocess, sys, numpy as np+from concurrent.futures import ThreadPoolExecutor+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+from metrics import load_points, chamfer_f, rot_y+BLENDER = "/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; RUNNER = os.path.join(os.path.dirname(os.path.abspath(__file__)), "blender_runner.py")+ap = argparse.ArgumentParser(); ap.add_argument("--test", required=True); ap.add_argument("--gen_dir", required=True); ap.add_argument("--workers", type=int, default=32); a = ap.parse_args()+tests = [json.loads(l) for l in open(a.test)]; ref_cache = os.path.join(os.path.dirname(a.test), "ref_exec_blender")+def code_of(msg):+ m = re.findall(r"```[a-zA-Z]*\n(.*?)```", msg, flags=re.S); return (max(m, key=len) if m else msg).strip()+def run_blender(code, d):+ d = os.path.abspath(d); os.makedirs(d, exist_ok=True); sp = os.path.join(d, "code.py"); glb = os.path.join(d, "out.glb"); rp = os.path.join(d, "exec.json")+ if not os.path.exists(sp): open(sp, "w").write(code)+ try:+ subprocess.run([BLENDER, "-b", "--factory-startup", "-noaudio", "--python", RUNNER, "--", "--script", sp, "--out", glb, "--report", rp], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=180, env={**os.environ, "HOME": "/tmp"})+ r = json.load(open(rp)) if os.path.exists(rp) else {"status": "CRASH"}+ except subprocess.TimeoutExpired: r = {"status": "TIMEOUT"}+ r["mesh"] = glb if r.get("status") == "OK" else None; return r+def ref_job(t):+ tid = t["id"].replace("/", "__"); d = os.path.join(ref_cache, tid)+ if os.path.exists(os.path.join(d, "rep.json")): return tid, json.load(open(os.path.join(d, "rep.json")))+ rep = run_blender(code_of(t["messages"][-1]["content"]), d); json.dump(rep, open(os.path.join(d, "rep.json"), "w")); return tid, rep+def gen_job(t):+ tid = t["id"].replace("/", "__"); d = os.path.join(a.gen_dir, tid)+ if not os.path.exists(os.path.join(d, "code.py")): return tid, {"status": "MISSING", "mesh": None}+ return tid, run_blender(None, d)+with ThreadPoolExecutor(a.workers) as ex: refs = dict(ex.map(ref_job, tests)); gens = dict(ex.map(gen_job, tests))+rows = []+for t in tests:+ tid = t["id"].replace("/", "__"); r = {"id": tid, "ref": refs[tid].get("status"), "gen": gens[tid].get("status"), "gen_error": (gens[tid].get("error") or "")[:200]}+ if r["ref"] == "OK" and r["gen"] == "OK":+ try:+ pr = load_points(refs[tid]["mesh"], 5000); pg = load_points(gens[tid]["mesh"], 5000)+ if pr is not None and pg is not None:+ best = min((chamfer_f(rot_y(pg, k), pr) for k in range(4)), key=lambda x: x[0]); r["chamfer"] = best[0]; r.update(best[1])+ except Exception as e: r["metric_error"] = str(e)[:100]+ rows.append(r)+n = len(rows); sc = [r for r in rows if "[email protected]" in r]+summary = {"dialect": "blender_heldout", "n": n, "ref_exec_ok": sum(r["ref"] == "OK" for r in rows), "gen_exec_ok": sum(r["gen"] == "OK" for r in rows), "gen_exec_rate": round(sum(r["gen"] == "OK" for r in rows) / n, 4),+ "n_scored": len(sc), "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None, "[email protected]_mean_all": round(float(sum(r.get("[email protected]", 0) for r in rows) / n), 4), "chamfer_mean_scored": round(float(np.mean([r["chamfer"] for r in sc])), 4) if sc else None}+os.makedirs(a.gen_dir, exist_ok=True)+with open(os.path.join(a.gen_dir, "dialect_metrics.jsonl"), "w") as f:+ for r in rows: f.write(json.dumps(r) + "\n")+json.dump(summary, open(os.path.join(a.gen_dir, "summary.json"), "w"), indent=2); print(json.dumps(summary))diff --git a/finetune/eval/blender_runner.py b/finetune/eval/blender_runner.pynew file mode 100644index 00000000..1da13185--- /dev/null+++ b/finetune/eval/blender_runner.py@@ -0,0 +1,72 @@+"""Run ONE generated Blender-Python script headlessly inside Blender and export a GLB + JSON report.++Usage (inside blender):+ blender -b --factory-startup --python blender_runner.py -- --script gen.py --out out.glb --report out.json+"""+import sys, json, time, traceback, argparse, runpy, os, io, contextlib++def parse():+ argv = sys.argv[sys.argv.index("--") + 1:] if "--" in sys.argv else []+ ap = argparse.ArgumentParser()+ ap.add_argument("--script", required=True)+ ap.add_argument("--out", required=True)+ ap.add_argument("--report", required=True)+ return ap.parse_args(argv)++def main():+ args = parse()+ import bpy+ t0 = time.time()+ rep = {"script": args.script, "out_glb": args.out, "status": "FAIL", "error": None,+ "n_meshes": 0, "n_verts": 0, "n_faces": 0, "size_kb": 0.0, "latency_s": 0.0}+ # factory startup scene (keeps the default 'Collection' in the view layer, like the official export),+ # but remove the default Cube/Light/Camera so an empty generation cannot pass as "OK"+ # NOTE: do not call read_factory_settings() here (leaves bpy.context.view_layer stale); rely on --factory-startup+ for ob in list(bpy.data.objects):+ bpy.data.objects.remove(ob, do_unlink=True)+ for coll_name in ("meshes", "materials", "lights", "cameras", "curves"):+ coll = getattr(bpy.data, coll_name)+ for blk in list(coll):+ if blk.users == 0: coll.remove(blk)+ bpy.context.view_layer.update()+ log = io.StringIO()+ try:+ # Make the script dir importable & chdir so relative asset paths resolve+ sdir = os.path.dirname(os.path.abspath(args.script))+ sys.path.insert(0, sdir); os.chdir(sdir)+ with contextlib.redirect_stdout(log):+ runpy.run_path(args.script, run_name="__main__")+ # collect meshes (evaluate modifiers)+ bpy.context.view_layer.update()+ deps = bpy.context.evaluated_depsgraph_get()+ vl_objs = list(bpy.context.view_layer.objects) # only objects actually in the scene/view layer count+ nm = nv = nf = 0+ for ob in vl_objs:+ if ob.type == "MESH":+ ev = ob.evaluated_get(deps)+ me = ev.to_mesh()+ if me is not None:+ nm += 1; nv += len(me.vertices); nf += len(me.polygons)+ ev.to_mesh_clear()+ rep.update(n_meshes=nm, n_verts=nv, n_faces=nf)+ if nm == 0 or nv == 0:+ rep["status"] = "EMPTY"; rep["error"] = "no mesh geometry produced"+ else:+ for ob in vl_objs:+ try: ob.select_set(ob.type == "MESH" and not ob.hide_viewport and not ob.hide_render)+ except Exception: pass+ bpy.ops.export_scene.gltf(filepath=args.out, export_format="GLB", use_selection=True,+ export_apply=True, export_materials="EXPORT", export_yup=True)+ rep["size_kb"] = round(os.path.getsize(args.out) / 1024, 1)+ rep["status"] = "OK"+ except BaseException as e: # noqa (catch SystemExit too)+ tb = traceback.format_exc()+ rep["error"] = (type(e).__name__ + ": " + str(e))[:2000]+ rep["traceback"] = tb[-3000:]+ rep["latency_s"] = round(time.time() - t0, 2)+ rep["stdout_tail"] = log.getvalue()[-1500:]+ with open(args.report, "w") as f:+ json.dump(rep, f, indent=2)+ print("RUNNER_STATUS", rep["status"], rep["error"] or "")++main()diff --git a/finetune/eval/compare.py b/finetune/eval/compare.pynew file mode 100644index 00000000..4971b0e3--- /dev/null+++ b/finetune/eval/compare.py@@ -0,0 +1,15 @@+"""Tabulate eval/out/*/summary.json (+ generation stats) into one table."""+import glob, json, os, sys+rows = []+for d in sorted(glob.glob("/wekafs/ict/hx_624/llm-ft/eval/out/*/")):+ sp = os.path.join(d, "summary.json")+ if not os.path.exists(sp): continue+ s = json.load(open(sp))+ gens = [json.loads(l) for f in glob.glob(os.path.join(d, "gens_shard*.jsonl")) for l in open(f)]+ ntok = sum(g["n_new_tokens"] for g in gens) / max(len(gens), 1)+ fin = sum(g["finished"] for g in gens) / max(len(gens), 1)+ rows.append((os.path.basename(d.rstrip("/")), s["exec_ok"], s["n_tasks"], s["exec_ok_rate"], s["chamfer_mean_scored"], s["chamfer_median_scored"], s["[email protected]_mean_scored"], s["[email protected]_mean_scored"], s["[email protected]_mean_all(fail=0)"], s["[email protected]_mean_all(fail=0)"], ntok, fin))+hdr = ("run", "exec_ok", "n", "ok_rate", "CD_mean", "CD_med", "[email protected](ok)", "[email protected](ok)", "[email protected](all)", "[email protected](all)", "avg_tok", "finished")+print("| " + " | ".join(hdr) + " |"); print("|" + "---|" * len(hdr))+for r in rows:+ print("| " + " | ".join(f"{x:.3f}" if isinstance(x, float) else str(x) for x in r) + " |")diff --git a/finetune/eval/dialect_eval.py b/finetune/eval/dialect_eval.pynew file mode 100644index 00000000..92f87a02--- /dev/null+++ b/finetune/eval/dialect_eval.py@@ -0,0 +1,43 @@+"""Evaluate a model on a non-Blender dialect test set: generate (vLLM, done separately) -> execute generated & reference -> geometry metrics vs reference mesh.+usage: dialect_eval.py --dialect cadquery --test data/multidialect/cadquery/test.jsonl --gen_dir <dir with <id>/code.py> [--workers 32]"""+import argparse, json, os, re, sys, numpy as np+from concurrent.futures import ThreadPoolExecutor+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+from dialect_runners import RUNNERS+from metrics import load_points, chamfer_f, rot_y+ap = argparse.ArgumentParser(); ap.add_argument("--dialect", required=True); ap.add_argument("--test", required=True); ap.add_argument("--gen_dir", required=True); ap.add_argument("--workers", type=int, default=32); ap.add_argument("--ref_cache", default=None)+a = ap.parse_args()+run = RUNNERS[a.dialect]+tests = [json.loads(l) for l in open(a.test)]+ref_cache = a.ref_cache or os.path.join(os.path.dirname(a.test), f"ref_exec_{a.dialect}")+def code_of(msg):+ m = re.findall(r"```[a-zA-Z]*\n(.*?)```", msg, flags=re.S); return (max(m, key=len) if m else msg).strip()+def ref_job(t):+ tid = t["id"].replace("/", "__"); d = os.path.join(ref_cache, tid)+ if os.path.exists(os.path.join(d, "rep.json")): return tid, json.load(open(os.path.join(d, "rep.json")))+ rep = run(code_of(t["messages"][-1]["content"]), d); json.dump(rep, open(os.path.join(d, "rep.json"), "w")); return tid, rep+def gen_job(t):+ tid = t["id"].replace("/", "__"); d = os.path.join(a.gen_dir, tid); cp = os.path.join(d, "code.py")+ if not os.path.exists(cp): return tid, {"status": "MISSING", "error": "no generation", "mesh": None}+ rep = run(open(cp).read(), os.path.join(d, "exec")); return tid, rep+with ThreadPoolExecutor(a.workers) as ex:+ refs = dict(ex.map(ref_job, tests)); gens = dict(ex.map(gen_job, tests))+rows = []; np.random.seed(0)+for t in tests:+ tid = t["id"].replace("/", "__"); r = {"id": tid, "ref": refs[tid]["status"], "gen": gens[tid]["status"], "gen_error": (gens[tid].get("error") or "")[:200]}+ if a.dialect != "glsl" and refs[tid]["status"] == "OK" and gens[tid]["status"] == "OK":+ try:+ pr = load_points(refs[tid]["mesh"], 5000); pg = load_points(gens[tid]["mesh"], 5000)+ if pr is not None and pg is not None:+ best = min((chamfer_f(rot_y(pg, k), pr) for k in range(4)), key=lambda x: x[0]); r["chamfer"] = best[0]; r.update(best[1])+ except Exception as e: r["metric_error"] = str(e)[:100]+ rows.append(r)+n = len(rows); ref_ok = sum(r["ref"] == "OK" for r in rows); gen_ok = sum(r["gen"] == "OK" for r in rows); sc = [r for r in rows if "[email protected]" in r]+summary = {"dialect": a.dialect, "n": n, "ref_exec_ok": ref_ok, "gen_exec_ok": gen_ok, "gen_exec_rate": round(gen_ok / n, 4), "n_scored": len(sc),+ "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None, "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None,+ "[email protected]_mean_all": round(float(sum(r.get("[email protected]", 0) for r in rows) / n), 4), "chamfer_mean_scored": round(float(np.mean([r["chamfer"] for r in sc])), 4) if sc else None}+os.makedirs(a.gen_dir, exist_ok=True)+with open(os.path.join(a.gen_dir, "dialect_metrics.jsonl"), "w") as f:+ for r in rows: f.write(json.dumps(r) + "\n")+json.dump(summary, open(os.path.join(a.gen_dir, "summary.json"), "w"), indent=2); print(json.dumps(summary))+from collections import Counter; print("gen failure modes:", Counter(re.sub(r"'[^']*'", "'<x>'", (r["gen_error"] or r["gen"]).split("\n")[0][:70]) for r in rows if r["gen"] != "OK").most_common(8))diff --git a/finetune/eval/dialect_runners.py b/finetune/eval/dialect_runners.pynew file mode 100644index 00000000..d58dfec0--- /dev/null+++ b/finetune/eval/dialect_runners.py@@ -0,0 +1,101 @@+"""Executors for non-Blender dialects. Each run_* returns dict(status, error, mesh_path or None, latency_s)."""+import os, subprocess, sys, time, json, tempfile, re+OPENSCAD = "/wekafs/ict/hx_624/tools/openscad/squashfs-root/AppRun"+GLSLANG = "/wekafs/ict/hx_624/anaconda3/envs/llmft/bin/glslangValidator"+PY = "/wekafs/ict/hx_624/anaconda3/envs/llmft/bin/python"+CQ_RUNNER = r'''+import sys, json, runpy, traceback, os+os.environ.setdefault("OMP_NUM_THREADS","1")+script, out_stl = sys.argv[1], sys.argv[2]+rep = {"status":"FAIL","error":None}+try:+ import cadquery as cq+ sys.argv = [script, out_stl]+ _orig_export = cq.exporters.export+ captured = []+ def _capture(o, *a, **k): captured.append(o)+ # neutralise export calls (the scripts export STEP/STL to arbitrary paths) but remember the exported object+ for modname in ("cadquery.occ_impl.exporters", "cadquery.exporters"):+ try:+ import importlib; _m = importlib.import_module(modname); _m.export = _capture+ except Exception: pass+ try: cq.exporters.export = _capture+ except Exception: pass+ g = runpy.run_path(script, run_name="__main__", init_globals={"show_object": _capture, "debug": _capture})+ obj = None+ for c in reversed(captured):+ if isinstance(c, (cq.Workplane, cq.Shape, cq.Assembly)): obj = c; break+ if obj is None:+ for name in ("result", "final", "model", "part", "solid", "shape", "body", "assembly", "assy", "obj", "res"):+ if isinstance(g.get(name), (cq.Workplane, cq.Shape, cq.Assembly)): obj = g[name]; break+ if obj is None:+ cands = [v for v in g.values() if isinstance(v, (cq.Workplane, cq.Shape, cq.Assembly))]+ obj = cands[-1] if cands else None+ if obj is None: raise RuntimeError("no CadQuery result object found (expected variable `result`)")+ if isinstance(obj, cq.Assembly): obj = obj.toCompound()+ if isinstance(obj, cq.Workplane):+ vals = obj.vals()+ if not vals: raise RuntimeError("empty workplane")+ obj = vals[0] if len(vals)==1 else cq.Compound.makeCompound([v for v in vals if isinstance(v, cq.Shape)])+ if not isinstance(obj, cq.Shape): raise RuntimeError(f"unsupported object {type(obj)}")+ if obj.Volume() <= 1e-9 and not obj.Faces(): raise RuntimeError("empty shape")+ _orig_export(obj, out_stl, tolerance=0.01, angularTolerance=0.1)+ rep["status"]="OK"+except BaseException as e:+ rep["error"]=(type(e).__name__+": "+str(e))[:500]; rep["traceback"]=traceback.format_exc()[-1500:]+print("CQ_REPORT "+json.dumps(rep))+'''+def run_cadquery(code, workdir, timeout=90):+ workdir = os.path.abspath(workdir); os.makedirs(workdir, exist_ok=True); sp = os.path.join(workdir, "script.py"); stl = os.path.join(workdir, "out.stl"); rp = os.path.join(workdir, "runner.py")+ open(sp, "w").write(code); open(rp, "w").write(CQ_RUNNER)+ t0 = time.time()+ try:+ p = subprocess.run([PY, rp, sp, stl], cwd=workdir, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=timeout, text=True, errors="replace")+ m = re.search(r"CQ_REPORT (\{.*\})", p.stdout)+ rep = json.loads(m.group(1)) if m else {"status": "CRASH", "error": p.stdout[-800:]}+ except subprocess.TimeoutExpired:+ rep = {"status": "TIMEOUT", "error": f"timeout {timeout}s"}+ rep["latency_s"] = round(time.time() - t0, 1); rep["mesh"] = stl if rep["status"] == "OK" and os.path.exists(stl) else None+ if rep["status"] == "OK" and rep["mesh"] is None: rep["status"] = "EMPTY"+ return rep+def run_openscad(code, workdir, timeout=120):+ workdir = os.path.abspath(workdir); os.makedirs(workdir, exist_ok=True); sc = os.path.join(workdir, "model.scad"); stl = os.path.join(workdir, "out.stl")+ open(sc, "w").write(code); t0 = time.time()+ try:+ p = subprocess.run([OPENSCAD, "-o", stl, "--export-format", "binstl", sc], cwd=workdir, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=timeout, text=True, errors="replace",+ env={**os.environ, "HOME": workdir, "QT_QPA_PLATFORM": "offscreen"})+ ok = p.returncode == 0 and os.path.exists(stl) and os.path.getsize(stl) > 100+ rep = {"status": "OK" if ok else ("EMPTY" if p.returncode == 0 else "FAIL"), "error": None if ok else p.stdout[-800:]}+ except subprocess.TimeoutExpired:+ rep = {"status": "TIMEOUT", "error": f"timeout {timeout}s"}+ rep["latency_s"] = round(time.time() - t0, 1); rep["mesh"] = stl if rep["status"] == "OK" else None+ return rep+GLSL_PRELUDE = """#version 300 es+precision highp float; precision highp int;+uniform vec3 iResolution; uniform float iTime; uniform float iTimeDelta; uniform int iFrame; uniform vec4 iMouse; uniform vec4 iDate; uniform float iSampleRate;+uniform sampler2D iChannel0; uniform sampler2D iChannel1; uniform sampler2D iChannel2; uniform sampler2D iChannel3;+uniform vec3 iChannelResolution[4]; uniform float iChannelTime[4];+out vec4 _outColor;+"""+GLSL_MAIN = "\nvoid main(){ vec4 c = vec4(0.0); mainImage(c, gl_FragCoord.xy); _outColor = c; }\n"+def run_glsl(code, workdir, timeout=30):+ workdir = os.path.abspath(workdir); os.makedirs(workdir, exist_ok=True); fp = os.path.join(workdir, "shader.frag")+ # normalise: drop #version / precision lines and re-declarations of Shadertoy uniforms / the out variable (the prelude provides them)+ import re as _re+ lines = []+ for ln in code.splitlines():+ if _re.match(r"\s*#\s*version\b", ln) or _re.match(r"\s*precision\s+(low|medium|high)p\s+", ln): continue+ if _re.match(r"\s*uniform\s+.*\b(iResolution|iTime|iTimeDelta|iFrame|iMouse|iDate|iSampleRate|iChannel[0-3]|iChannelResolution|iChannelTime)\b", ln): continue+ if _re.match(r"\s*out\s+vec4\s+\w+\s*;", ln): continue+ lines.append(ln)+ src = "\n".join(lines)+ if "mainImage" not in src: return {"status": "FAIL", "error": "no mainImage()", "latency_s": 0, "mesh": None}+ open(fp, "w").write(GLSL_PRELUDE + src + GLSL_MAIN); t0 = time.time()+ try:+ p = subprocess.run([GLSLANG, "-S", "frag", fp], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=timeout, text=True, errors="replace")+ rep = {"status": "OK" if p.returncode == 0 else "FAIL", "error": None if p.returncode == 0 else p.stdout[-600:]}+ except subprocess.TimeoutExpired:+ rep = {"status": "TIMEOUT", "error": "timeout"}+ rep["latency_s"] = round(time.time() - t0, 2); rep["mesh"] = None+ return rep+RUNNERS = {"cadquery": run_cadquery, "openscad": run_openscad, "glsl": run_glsl}diff --git a/finetune/eval/generate.py b/finetune/eval/generate.pynew file mode 100644index 00000000..cabaf670--- /dev/null+++ b/finetune/eval/generate.py@@ -0,0 +1,81 @@+"""Batched generation for the 3DCodeBench prompts with a HF causal LM. Writes <out>/<task>/code.py + gens.jsonl"""+import argparse, json, os, re, time, torch+from transformers import AutoTokenizer, AutoModelForCausalLM++def extract_code(text):+ m = re.findall(r"```[a-zA-Z0-9_+.-]*[ \t]*\n(.*?)```", text, flags=re.S) # any language fence (python/glsl/openscad/...)+ if m:+ return max(m, key=len).strip("\n")+ m = re.search(r"```[a-zA-Z0-9_+.-]*[ \t]*\n(.*)$", text, flags=re.S) # unterminated block+ if m: return m.group(1).strip("\n")+ return text.strip()++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--model", required=True)+ ap.add_argument("--prompts", required=True)+ ap.add_argument("--out", required=True)+ ap.add_argument("--batch_size", type=int, default=16)+ ap.add_argument("--max_new_tokens", type=int, default=6144)+ ap.add_argument("--temperature", type=float, default=0.0)+ ap.add_argument("--top_p", type=float, default=0.95)+ ap.add_argument("--shard", type=int, default=0); ap.add_argument("--num_shards", type=int, default=1)+ ap.add_argument("--attn", default="auto")+ ap.add_argument("--limit", type=int, default=None)+ ap.add_argument("--no_think", action="store_true", help="pass enable_thinking=False to the chat template (Qwen3/3.5)")+ a = ap.parse_args()+ os.makedirs(a.out, exist_ok=True)+ attn = a.attn+ if attn == "auto":+ try: import flash_attn; attn = "flash_attention_2" # noqa+ except Exception: attn = "sdpa"+ tok = AutoTokenizer.from_pretrained(a.model, padding_side="left")+ if tok.pad_token is None: tok.pad_token = tok.eos_token+ try:+ model = AutoModelForCausalLM.from_pretrained(a.model, dtype=torch.bfloat16, attn_implementation=attn, device_map="cuda")+ except Exception as e: # multimodal wrappers (e.g. Qwen3.5 = Qwen3_5ForConditionalGeneration)+ print("[gen] AutoModelForCausalLM failed (%s); falling back to AutoModelForImageTextToText" % str(e)[:200], flush=True)+ from transformers import AutoModelForImageTextToText+ model = AutoModelForImageTextToText.from_pretrained(a.model, dtype=torch.bfloat16, attn_implementation=attn, device_map="cuda")+ model.eval()+ rows = [json.loads(l) for l in open(a.prompts)]+ rows = rows[a.shard::a.num_shards]+ if a.limit: rows = rows[:a.limit]+ # skip already-done+ done = set()+ outf = f"{a.out}/gens_shard{a.shard}.jsonl"+ if os.path.exists(outf):+ done = {json.loads(l)["task"] for l in open(outf)}+ rows = [r for r in rows if r["task"] not in done]+ print(f"[gen] {len(rows)} prompts to do (shard {a.shard}/{a.num_shards}) attn={attn}", flush=True)+ # sort by prompt length for efficient batching+ tmpl_kwargs = {"enable_thinking": False} if a.no_think else {}+ texts = [tok.apply_chat_template(r["messages"], tokenize=False, add_generation_prompt=True, **tmpl_kwargs) for r in rows]+ if texts: print("[gen] prompt tail example:", repr(texts[0][-200:]), flush=True)+ order = sorted(range(len(rows)), key=lambda i: len(texts[i]))+ gen_kwargs = dict(max_new_tokens=a.max_new_tokens, pad_token_id=tok.pad_token_id)+ if a.temperature > 0: gen_kwargs.update(do_sample=True, temperature=a.temperature, top_p=a.top_p)+ else: gen_kwargs.update(do_sample=False)+ t0 = time.time(); ntok = 0+ with open(outf, "a") as f:+ for b in range(0, len(order), a.batch_size):+ idx = order[b:b + a.batch_size]+ enc = tok([texts[i] for i in idx], return_tensors="pt", padding=True).to("cuda")+ with torch.no_grad():+ out = model.generate(**enc, **gen_kwargs)+ gen = out[:, enc["input_ids"].shape[1]:]+ for j, i in enumerate(idx):+ ids = gen[j]+ n = int((ids != tok.pad_token_id).sum())+ text = tok.decode(ids, skip_special_tokens=True)+ code = extract_code(text)+ finished = bool((ids == tok.eos_token_id).any()) or any((ids == t).any() for t in tok.all_special_ids if t != tok.pad_token_id) if n < a.max_new_tokens else False+ task = rows[i]["task"]; d = f"{a.out}/{task}"; os.makedirs(d, exist_ok=True)+ open(f"{d}/code.py", "w").write(code + "\n"); open(f"{d}/raw.txt", "w").write(text)+ f.write(json.dumps({"task": task, "n_new_tokens": n, "finished": finished, "has_code_block": "```" in text}) + "\n"); f.flush()+ ntok += n+ el = time.time() - t0+ print(f"[gen] {min(b + a.batch_size, len(order))}/{len(order)} done | {ntok} tok | {ntok/el:.0f} tok/s | {el/60:.1f} min", flush=True)++if __name__ == "__main__":+ main()diff --git a/finetune/eval/generate_vllm.py b/finetune/eval/generate_vllm.pynew file mode 100644index 00000000..28c4b867--- /dev/null+++ b/finetune/eval/generate_vllm.py@@ -0,0 +1,33 @@+"""vLLM version of generate.py: same outputs (<out>/<task>/code.py + raw.txt, <out>/gens_shard0.jsonl)."""+import argparse, json, os, re, time, sys+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+from generate import extract_code++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--model", required=True); ap.add_argument("--prompts", required=True); ap.add_argument("--out", required=True)+ ap.add_argument("--max_new_tokens", type=int, default=6144); ap.add_argument("--temperature", type=float, default=0.0)+ ap.add_argument("--max_model_len", type=int, default=12288); ap.add_argument("--no_think", action="store_true")+ ap.add_argument("--tp", type=int, default=1); ap.add_argument("--gpu_mem", type=float, default=0.88); ap.add_argument("--limit", type=int, default=None); ap.add_argument("--seed", type=int, default=0)+ a = ap.parse_args()+ from vllm import LLM, SamplingParams+ os.makedirs(a.out, exist_ok=True)+ rows = [json.loads(l) for l in open(a.prompts)]+ if a.limit: rows = rows[:a.limit]+ llm = LLM(model=a.model, dtype="bfloat16", tensor_parallel_size=a.tp, max_model_len=a.max_model_len, gpu_memory_utilization=a.gpu_mem,+ enable_prefix_caching=True, limit_mm_per_prompt={"image": 0, "video": 0}, trust_remote_code=True)+ sp = SamplingParams(temperature=a.temperature, top_p=0.95 if a.temperature > 0 else 1.0, max_tokens=a.max_new_tokens, seed=a.seed)+ kw = {"chat_template_kwargs": {"enable_thinking": False}} if a.no_think else {}+ t0 = time.time()+ outs = llm.chat([r["messages"] for r in rows], sp, use_tqdm=True, **kw)+ el = time.time() - t0; ntok = 0+ with open(f"{a.out}/gens_shard0.jsonl", "w") as f:+ for r, o in zip(rows, outs):+ text = o.outputs[0].text; n = len(o.outputs[0].token_ids); ntok += n+ d = f"{a.out}/{r['task']}"; os.makedirs(d, exist_ok=True)+ open(f"{d}/code.py", "w").write(extract_code(text) + "\n"); open(f"{d}/raw.txt", "w").write(text)+ f.write(json.dumps({"task": r["task"], "n_new_tokens": n, "finished": o.outputs[0].finish_reason == "stop", "has_code_block": "```" in text}) + "\n")+ print(f"[gen-vllm] {len(rows)} prompts | {ntok} tok | {ntok/el:.0f} tok/s | {el/60:.1f} min", flush=True)++if __name__ == "__main__":+ main()diff --git a/finetune/eval/metrics.py b/finetune/eval/metrics.pynew file mode 100644index 00000000..b2229a24--- /dev/null+++ b/finetune/eval/metrics.py@@ -0,0 +1,77 @@+"""Geometry metrics between generated GLB and the 3DCodeBench ground-truth GLB.+Normalisation: center at bbox center, scale so max bbox extent = 1. Metrics: symmetric Chamfer-L2 distance (mean NN distance),+F-score @0.05 / @0.1, best over 4 yaw rotations (objects may face any direction). Writes <gen_dir>/metrics.jsonl + summary.json"""+import argparse, json, os, numpy as np+import trimesh+from scipy.spatial import cKDTree++BENCH = "/wekafs/ict/hx_624/data/3dcodebench/data"++def load_points(path, n):+ m = trimesh.load(path, force="mesh")+ if isinstance(m, trimesh.Scene): m = trimesh.util.concatenate(list(m.geometry.values()))+ if m.is_empty or len(m.faces) == 0: return None+ v = m.vertices+ c = (v.max(0) + v.min(0)) / 2; s = (v.max(0) - v.min(0)).max()+ if s <= 0: return None+ m.apply_translation(-c); m.apply_scale(1.0 / s)+ pts, _ = trimesh.sample.sample_surface(m, n)+ return np.asarray(pts)++def chamfer_f(a, b, taus=(0.05, 0.1)):+ ta, tb = cKDTree(a), cKDTree(b)+ da, _ = tb.query(a); db, _ = ta.query(b)+ cd = float(da.mean() + db.mean())+ fs = {}+ for t in taus:+ p = float((da < t).mean()); r = float((db < t).mean())+ fs[f"f@{t}"] = 2 * p * r / (p + r) if p + r > 0 else 0.0+ return cd, fs++def rot_y(p, k):+ th = np.pi / 2 * k; c, s = np.cos(th), np.sin(th)+ R = np.array([[c, 0, s], [0, 1, 0], [-s, 0, c]])+ return p @ R.T++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--gen_dir", required=True)+ ap.add_argument("--n_points", type=int, default=10000)+ a = ap.parse_args()+ np.random.seed(0)+ ex = {json.loads(l)["task"]: json.loads(l) for l in open(os.path.join(a.gen_dir, "exec_results.jsonl"))}+ tasks = sorted(os.listdir(BENCH))+ rows = []+ for t in tasks:+ r = {"task": t, "exec": ex.get(t, {}).get("status", "MISSING")}+ gen_glb = os.path.join(a.gen_dir, t, "out.glb"); gt_glb = os.path.join(BENCH, t, "glb", f"{t}.glb")+ if r["exec"] == "OK" and os.path.exists(gen_glb) and os.path.exists(gt_glb):+ try:+ pg = load_points(gen_glb, a.n_points); pt = load_points(gt_glb, a.n_points)+ if pg is not None and pt is not None:+ best = None+ for k in range(4):+ cd, fs = chamfer_f(rot_y(pg, k), pt)+ if best is None or cd < best[0]: best = (cd, fs)+ r.update(chamfer=best[0], **best[1])+ except Exception as e:+ r["metric_error"] = str(e)[:200]+ rows.append(r)+ with open(os.path.join(a.gen_dir, "metrics.jsonl"), "w") as f:+ for r in rows: f.write(json.dumps(r) + "\n")+ n = len(rows); ok = [r for r in rows if r["exec"] == "OK"]; sc = [r for r in rows if "chamfer" in r]+ # Failed / empty generations get the worst score for the aggregate (chamfer: mean of worst-10 for penalty; f: 0)+ summary = {+ "n_tasks": n, "exec_ok": len(ok), "exec_ok_rate": round(len(ok) / n, 4), "n_scored": len(sc),+ "chamfer_mean_scored": round(float(np.mean([r["chamfer"] for r in sc])), 4) if sc else None,+ "chamfer_median_scored": round(float(np.median([r["chamfer"] for r in sc])), 4) if sc else None,+ "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None,+ "[email protected]_mean_scored": round(float(np.mean([r["[email protected]"] for r in sc])), 4) if sc else None,+ "[email protected]_mean_all(fail=0)": round(float(sum(r.get("[email protected]", 0) for r in rows) / n), 4),+ "[email protected]_mean_all(fail=0)": round(float(sum(r.get("[email protected]", 0) for r in rows) / n), 4),+ }+ json.dump(summary, open(os.path.join(a.gen_dir, "summary.json"), "w"), indent=2)+ print(json.dumps(summary, indent=2))++if __name__ == "__main__":+ main()diff --git a/finetune/eval/pass_at_n.py b/finetune/eval/pass_at_n.pynew file mode 100644index 00000000..fde16843--- /dev/null+++ b/finetune/eval/pass_at_n.py@@ -0,0 +1,10 @@+import json, sys+pfx, n = sys.argv[1], int(sys.argv[2])+ok = {}; best = {}+for i in range(1, n + 1):+ for l in open(f"/wekafs/ict/hx_624/llm-ft/eval/out/{pfx}_s{i}/metrics.jsonl"):+ r = json.loads(l); t = r["task"]+ ok.setdefault(t, 0); ok[t] += r["exec"] == "OK"+ if "[email protected]" in r: best[t] = max(best.get(t, 0), r["[email protected]"])+T = len(ok)+print(f"pass@1 (avg over samples): {sum(ok.values())/(n*T):.3f} | pass@{n} (any sample OK): {sum(v>0 for v in ok.values())/T:.3f} | best-of-{n} [email protected] (all, fail=0): {sum(best.values())/T:.3f}")diff --git a/finetune/eval/reextract.py b/finetune/eval/reextract.pynew file mode 100644index 00000000..1fdb3789--- /dev/null+++ b/finetune/eval/reextract.py@@ -0,0 +1,9 @@+"""Re-extract code.py from raw.txt in a generation dir (after extractor fixes)."""+import sys, os, glob+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))+from generate import extract_code+for d in sys.argv[1:]:+ n=0+ for raw in glob.glob(os.path.join(d, "*", "raw.txt")):+ code = extract_code(open(raw).read()); open(os.path.join(os.path.dirname(raw), "code.py"), "w").write(code + "\n"); n+=1+ print(d, "re-extracted", n)diff --git a/finetune/eval/render_glb.py b/finetune/eval/render_glb.pynew file mode 100644index 00000000..21fc6e21--- /dev/null+++ b/finetune/eval/render_glb.py@@ -0,0 +1,29 @@+"""Blender headless: render a GLB to PNG with an auto-framed 3/4 camera (workbench, solid shading).+usage: blender -b --factory-startup -noaudio --python eval/render_glb.py -- --glb in.glb --out out.png [--size 320]"""+import bpy, sys, math, argparse+from mathutils import Vector+argv = sys.argv[sys.argv.index("--")+1:] if "--" in sys.argv else []+ap = argparse.ArgumentParser(); ap.add_argument("--glb"); ap.add_argument("--out"); ap.add_argument("--size", type=int, default=320); a = ap.parse_args(argv)+bpy.ops.wm.read_factory_settings(use_empty=True)+bpy.ops.import_scene.gltf(filepath=a.glb)+meshes = [o for o in bpy.context.scene.objects if o.type == "MESH"]+if not meshes: raise SystemExit("no mesh")+# bounds+pts = [o.matrix_world @ Vector(c) for o in meshes for c in o.bound_box]+mn = Vector((min(p.x for p in pts), min(p.y for p in pts), min(p.z for p in pts))); mx = Vector((max(p.x for p in pts), max(p.y for p in pts), max(p.z for p in pts)))+center = (mn + mx) / 2; radius = max((mx - mn).length / 2, 1e-3)+cam_data = bpy.data.cameras.new("cam"); cam = bpy.data.objects.new("cam", cam_data); bpy.context.scene.collection.objects.link(cam)+direction = Vector((1.0, -1.2, 0.8)).normalized(); cam.location = center + direction * radius * 2.6+cam.rotation_euler = (center - cam.location).to_track_quat("-Z", "Y").to_euler(); cam_data.lens = 40; cam_data.clip_end = radius * 100+bpy.context.scene.camera = cam+sc = bpy.context.scene; sc.render.engine = "CYCLES"; sc.cycles.device = "CPU"; sc.cycles.samples = 32; sc.cycles.use_denoising = False+sc.render.resolution_x = a.size; sc.render.resolution_y = a.size; sc.render.film_transparent = False+w = bpy.data.worlds.new("w"); w.use_nodes = True; bg = w.node_tree.nodes["Background"]; bg.inputs[0].default_value = (0.92, 0.92, 0.92, 1); bg.inputs[1].default_value = 1.0; sc.world = w+sun_d = bpy.data.lights.new("sun", "SUN"); sun_d.energy = 3.0; sun = bpy.data.objects.new("sun", sun_d); sc.collection.objects.link(sun)+sun.rotation_euler = (math.radians(45), math.radians(20), math.radians(30))+import random; random.seed(0)+for o in meshes:+ mat = bpy.data.materials.new(f"m_{o.name}"); mat.use_nodes = True; bsdf = mat.node_tree.nodes.get("Principled BSDF")+ if bsdf: bsdf.inputs["Base Color"].default_value = (random.uniform(0.25, 0.85), random.uniform(0.25, 0.85), random.uniform(0.25, 0.85), 1); bsdf.inputs["Roughness"].default_value = 0.6+ o.data.materials.clear(); o.data.materials.append(mat)+sc.render.filepath = a.out; bpy.ops.render.render(write_still=True)diff --git a/finetune/eval/results_table.py b/finetune/eval/results_table.pynew file mode 100644index 00000000..db0d0176--- /dev/null+++ b/finetune/eval/results_table.py@@ -0,0 +1,20 @@+"""Collect all eval/out/*/summary.json + training stats into a single JSON (for the final report)."""+import glob, json, os+OUT = "/wekafs/ict/hx_624/llm-ft/eval/out"; RUNS = "/wekafs/ict/hx_624/llm-ft/runs"+rows = []+for d in sorted(glob.glob(f"{OUT}/*/")):+ name = os.path.basename(d.rstrip("/"))+ if name.startswith("_"): continue+ sp = os.path.join(d, "summary.json")+ if not os.path.exists(sp): continue+ s = json.load(open(sp))+ gens = [json.loads(l) for f in glob.glob(os.path.join(d, "gens_shard*.jsonl")) for l in open(f)]+ avg_tok = sum(g["n_new_tokens"] for g in gens) / max(len(gens), 1)+ run = name.replace("qwen35_9b_lora_", "lf_qwen35_9b_lora_").replace("qwen35_9b_full_v1", "lf_qwen35_9b_full_v1")+ tr = {}+ trp = os.path.join(RUNS, run.split("_ckpt")[0], "train_results.json")+ if os.path.exists(trp): tr = json.load(open(trp))+ rows.append(dict(name=name, exec_ok=s["exec_ok"], exec_rate=s["exec_ok_rate"], cd=s["chamfer_mean_scored"], f05=s["[email protected]_mean_scored"], f10=s["[email protected]_mean_scored"],+ f05_all=s["[email protected]_mean_all(fail=0)"], f10_all=s["[email protected]_mean_all(fail=0)"], avg_tok=round(avg_tok), train_loss=tr.get("train_loss"), train_min=(tr.get("train_runtime") or 0) / 60))+json.dump(rows, open("/wekafs/ict/hx_624/llm-ft/eval/all_results.json", "w"), indent=1)+for r in rows: print(f"{r['name']:36s} exec={r['exec_ok']:3d} ({r['exec_rate']*100:4.1f}%) CD={r['cd'] if r['cd'] is None else round(r['cd'],3)} F05={r['f05'] if r['f05'] is None else round(r['f05'],3)} F05all={round(r['f05_all'],3)} tok={r['avg_tok']} loss={r['train_loss'] if r['train_loss'] is None else round(r['train_loss'],3)} {r['train_min']:.0f}min")diff --git a/finetune/eval/run_bench.py b/finetune/eval/run_bench.pynew file mode 100644index 00000000..ad02c77a--- /dev/null+++ b/finetune/eval/run_bench.py@@ -0,0 +1,48 @@+"""Execute every <gen_dir>/<task>/code.py in headless Blender (parallel), export GLB, collect reports -> <gen_dir>/exec_results.jsonl"""+import argparse, json, os, subprocess, sys, time+from concurrent.futures import ThreadPoolExecutor, as_completed+BLENDER = "/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"+RUNNER = os.path.join(os.path.dirname(os.path.abspath(__file__)), "blender_runner.py")++def run_one(task_dir, timeout):+ task = os.path.basename(task_dir)+ script = os.path.join(task_dir, "code.py"); glb = os.path.join(task_dir, "out.glb"); rep = os.path.join(task_dir, "exec.json")+ if os.path.exists(glb): os.remove(glb)+ cmd = [BLENDER, "-b", "--factory-startup", "-noaudio", "--python", RUNNER, "--", "--script", script, "--out", glb, "--report", rep]+ t0 = time.time()+ try:+ p = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=timeout, text=True, errors="replace",+ env={**os.environ, "HOME": "/tmp", "XDG_CONFIG_HOME": "/tmp/.bcfg"})+ if os.path.exists(rep):+ r = json.load(open(rep))+ else:+ r = {"status": "CRASH", "error": p.stdout[-1500:], "latency_s": round(time.time() - t0, 1)}+ except subprocess.TimeoutExpired:+ r = {"status": "TIMEOUT", "error": f"timeout {timeout}s", "latency_s": timeout}+ r["task"] = task+ return r++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--gen_dir", required=True)+ ap.add_argument("--workers", type=int, default=16)+ ap.add_argument("--timeout", type=int, default=300)+ a = ap.parse_args()+ a.gen_dir = os.path.abspath(a.gen_dir) # the Blender runner chdir()s into the script dir -> all paths must be absolute+ tasks = sorted(d for d in os.listdir(a.gen_dir) if os.path.isfile(os.path.join(a.gen_dir, d, "code.py")))+ print(f"[exec] {len(tasks)} scripts, {a.workers} workers", flush=True)+ res = []+ with ThreadPoolExecutor(a.workers) as ex:+ futs = {ex.submit(run_one, os.path.join(a.gen_dir, t), a.timeout): t for t in tasks}+ for i, fu in enumerate(as_completed(futs)):+ r = fu.result(); res.append(r)+ if (i + 1) % 20 == 0: print(f"[exec] {i+1}/{len(tasks)}", flush=True)+ res.sort(key=lambda r: r["task"])+ with open(os.path.join(a.gen_dir, "exec_results.jsonl"), "w") as f:+ for r in res: f.write(json.dumps(r) + "\n")+ from collections import Counter+ c = Counter(r["status"] for r in res)+ print("[exec] status counts:", dict(c), f"| OK rate = {c.get('OK',0)}/{len(res)} = {100*c.get('OK',0)/max(len(res),1):.1f}%")++if __name__ == "__main__":+ main()diff --git a/finetune/eval/run_dialect_eval.sh b/finetune/eval/run_dialect_eval.shnew file mode 100755index 00000000..99df059e--- /dev/null+++ b/finetune/eval/run_dialect_eval.sh@@ -0,0 +1,27 @@+#!/bin/bash+# usage: GPUS=2 eval/run_dialect_eval.sh <model_dir> <name> [dialects...] -> eval/out/md_<name>_<dialect>/summary.json+set -uo pipefail; MODEL=$(readlink -f "$1"); NAME=$2; shift 2; DIALECTS=${@:-cadquery openscad glsl blender}+cd /wekafs/ict/hx_624/llm-ft; source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=${GPUS:-0}+for D in $DIALECTS; do+ OUT=eval/out/md_${NAME}_$D; mkdir -p $OUT+ python - "$D" "$OUT" <<PY+import json,sys+D,OUT=sys.argv[1],sys.argv[2]+with open(f"{OUT}/prompts.jsonl","w") as f:+ for l in open(f"data/multidialect/{D}/test.jsonl"):+ r=json.loads(l); f.write(json.dumps({"task": r["id"].replace("/","__"), "messages": r["messages"][:2]})+"\n")+PY+ python eval/generate_vllm.py --model $MODEL --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 2>&1 | grep -E "gen-vllm|Error" | tail -2+done+conda activate llmft+for D in $DIALECTS; do+ OUT=eval/out/md_${NAME}_$D+ if [ "$D" = "blender" ]; then+ # blender held-out test: run in Blender and compare to reference via the same exec harness (reference meshes cached)+ python eval/blender_dialect_eval.py --test data/multidialect/blender/test.jsonl --gen_dir $OUT --workers 32 2>&1 | tail -2+ else+ python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir $OUT --workers 32 2>&1 | tail -2+ fi+done+echo "[md-eval $NAME] DONE"diff --git a/finetune/eval/run_eval.sh b/finetune/eval/run_eval.shnew file mode 100755index 00000000..2834a350--- /dev/null+++ b/finetune/eval/run_eval.sh@@ -0,0 +1,22 @@+#!/bin/bash+# End-to-end 3DCodeBench eval: generate (sharded over GPUS) -> execute in Blender -> geometry metrics+# usage: GPUS=0,1 ./run_eval.sh <model_path> <out_dir> [extra generate.py args]+set -euo pipefail+source /wekafs/ict/hx_624/llm-ft/env.sh+MODEL=$(readlink -f "$1"); OUT=$(readlink -m "$2"); shift 2+GPUS=${GPUS:-0}+PROMPTS=/wekafs/ict/hx_624/llm-ft/data/sft_v1/bench_prompts.jsonl+[ -f $PROMPTS ] || PROMPTS=/wekafs/ict/hx_624/llm-ft/data/sft_v0/bench_prompts.jsonl+mkdir -p $OUT+IFS=',' read -ra G <<< "$GPUS"; N=${#G[@]}+echo "[eval] model=$MODEL out=$OUT gpus=$GPUS shards=$N"+pids=()+for i in "${!G[@]}"; do+ CUDA_VISIBLE_DEVICES=${G[$i]} python /wekafs/ict/hx_624/llm-ft/eval/generate.py --model $MODEL --prompts $PROMPTS --out $OUT --shard $i --num_shards $N "$@" > $OUT/gen_shard$i.log 2>&1 &+ pids+=($!)+done+for p in "${pids[@]}"; do wait $p; done+cat $OUT/gens_shard*.jsonl > $OUT/gens.jsonl+echo "[eval] generation done: $(wc -l < $OUT/gens.jsonl) samples"+python /wekafs/ict/hx_624/llm-ft/eval/run_bench.py --gen_dir $OUT --workers ${WORKERS:-32} --timeout ${TIMEOUT:-300} 2>&1 | tee $OUT/exec.log+python /wekafs/ict/hx_624/llm-ft/eval/metrics.py --gen_dir $OUT 2>&1 | tee $OUT/metrics.logdiff --git a/finetune/eval/run_eval_vllm.sh b/finetune/eval/run_eval_vllm.shnew file mode 100755index 00000000..524ddc90--- /dev/null+++ b/finetune/eval/run_eval_vllm.sh@@ -0,0 +1,14 @@+#!/bin/bash+# usage: GPUS=0 ./run_eval_vllm.sh <model_path> <out_dir> [extra generate_vllm.py args] (TP = number of GPUs listed)+set -euo pipefail+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+source /wekafs/ict/hx_624/.secrets/hf.env; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING TOKENIZERS_PARALLELISM=false+MODEL=$(readlink -f "$1"); OUT=$(readlink -m "$2"); shift 2+export CUDA_VISIBLE_DEVICES=${GPUS:-0}; TP=$(echo $CUDA_VISIBLE_DEVICES | tr ',' '\n' | wc -l)+PROMPTS=${PROMPTS:-/wekafs/ict/hx_624/llm-ft/data/sft_v1/bench_prompts.jsonl}+mkdir -p $OUT; echo "[eval-vllm] model=$MODEL out=$OUT gpus=$CUDA_VISIBLE_DEVICES tp=$TP"+python /wekafs/ict/hx_624/llm-ft/eval/generate_vllm.py --model $MODEL --prompts $PROMPTS --out $OUT --tp $TP "$@" 2>&1 | grep -vE "^\s*$" | tail -n 40+cp $OUT/gens_shard0.jsonl $OUT/gens.jsonl+conda activate llmft+python /wekafs/ict/hx_624/llm-ft/eval/run_bench.py --gen_dir $OUT --workers ${WORKERS:-32} --timeout ${TIMEOUT:-300} 2>&1 | tee $OUT/exec.log+python /wekafs/ict/hx_624/llm-ft/eval/metrics.py --gen_dir $OUT 2>&1 | tee $OUT/metrics.logdiff --git a/finetune/eval/threejs_eval.py b/finetune/eval/threejs_eval.pynew file mode 100644index 00000000..ab969ad3--- /dev/null+++ b/finetune/eval/threejs_eval.py@@ -0,0 +1,17 @@+"""Run every <gen_dir>/<task>/code.py (HTML) through the headless-browser runner (parallel) and summarise."""+import argparse, json, os, sys+from concurrent.futures import ThreadPoolExecutor+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))); from threejs_runner import run_threejs+ap = argparse.ArgumentParser(); ap.add_argument("--gen_dir", required=True); ap.add_argument("--workers", type=int, default=8); a = ap.parse_args()+tasks = sorted(d for d in os.listdir(a.gen_dir) if os.path.isfile(os.path.join(a.gen_dir, d, "code.py")))+def job(t):+ html = open(os.path.join(a.gen_dir, t, "code.py")).read()+ r = run_threejs(html, os.path.join(a.gen_dir, t, "exec")); r["task"] = t; r.pop("shot", None); return r+with ThreadPoolExecutor(a.workers) as ex: res = list(ex.map(job, tasks))+from collections import Counter+c = Counter(r["status"] for r in res)+with open(os.path.join(a.gen_dir, "threejs_results.jsonl"), "w") as f:+ for r in res: f.write(json.dumps(r) + "\n")+summ = {"dialect": "threejs", "n": len(res), "exec_ok": c.get("OK", 0), "exec_rate": round(c.get("OK", 0) / max(1, len(res)), 4), "counts": dict(c)}+json.dump(summ, open(os.path.join(a.gen_dir, "summary.json"), "w"), indent=2); print(json.dumps(summ))+print("failure modes:", Counter((r["error"] or r["status"])[:80] for r in res if r["status"] != "OK").most_common(6))diff --git a/finetune/eval/threejs_runner.py b/finetune/eval/threejs_runner.pynew file mode 100644index 00000000..c44f6a36--- /dev/null+++ b/finetune/eval/threejs_runner.py@@ -0,0 +1,56 @@+"""Headless-Chromium executor for single-file three.js HTML (Playwright). Returns dict(status, error, console_errors, canvas_nonblank, latency_s, shot).+status OK = page loaded, no uncaught JS error / failed module import, a <canvas> exists and its rendered pixels are not (nearly) uniform."""+import os, sys, json, time, base64, io+def run_threejs(html, workdir, timeout=40, settle_ms=4000):+ os.environ.setdefault("PLAYWRIGHT_BROWSERS_PATH", "/wekafs/ict/hx_624/cache/ms-playwright")+ workdir = os.path.abspath(workdir); os.makedirs(workdir, exist_ok=True)+ hp = os.path.join(workdir, "index.html"); open(hp, "w").write(html)+ rep = {"status": "FAIL", "error": None, "console_errors": [], "canvas_nonblank": None, "latency_s": 0, "shot": None}+ t0 = time.time()+ try:+ from playwright.sync_api import sync_playwright+ with sync_playwright() as p:+ browser = p.chromium.launch(headless=True, args=["--use-gl=angle", "--use-angle=swiftshader", "--enable-unsafe-swiftshader", "--ignore-gpu-blocklist", "--no-sandbox", "--disable-dev-shm-usage"])+ page = browser.new_page(viewport={"width": 640, "height": 480})+ errs = []+ page.on("pageerror", lambda e: errs.append("pageerror: " + str(e)[:300]))+ page.on("console", lambda m: errs.append("console.error: " + m.text[:300]) if m.type == "error" else None)+ page.on("requestfailed", lambda r: errs.append("requestfailed: " + r.url[:200]))+ page.goto("file://" + hp, wait_until="load", timeout=timeout * 1000)+ page.wait_for_timeout(settle_ms)+ # canvas check: count distinct pixels over a downsampled grid+ info = page.evaluate("""() => {+ const cs = document.querySelectorAll('canvas'); if (!cs.length) return {n:0};+ const c = cs[0]; const w=c.width, h=c.height; if (!w||!h) return {n:cs.length, w, h, distinct:0};+ let gl = c.getContext('webgl2') || c.getContext('webgl');+ let pix = null;+ if (gl) { pix = new Uint8Array(w*h*4); try { gl.readPixels(0,0,w,h,gl.RGBA,gl.UNSIGNED_BYTE,pix); } catch(e) { pix=null; } }+ if (!pix) { try { const t=document.createElement('canvas'); t.width=w; t.height=h; const ctx=t.getContext('2d'); ctx.drawImage(c,0,0); pix=ctx.getImageData(0,0,w,h).data; } catch(e) { return {n:cs.length, w, h, distinct:-1, err:String(e)}; } }+ const seen = new Set(); const step = Math.max(1, Math.floor((w*h)/4000));+ for (let i=0;i<w*h;i+=step){ const o=i*4; seen.add((pix[o]>>3)+','+(pix[o+1]>>3)+','+(pix[o+2]>>3)); if (seen.size>64) break; }+ return {n:cs.length, w, h, distinct:seen.size};+ }""")+ shot = os.path.join(workdir, "shot.png"); page.screenshot(path=shot); rep["shot"] = shot+ browser.close()+ rep["console_errors"] = errs[:10]+ fatal = [e for e in errs if e.startswith("pageerror") or "Failed to resolve module" in e or "Uncaught" in e or e.startswith("requestfailed")]+ # blank check from the composited screenshot (the WebGL drawing buffer is cleared after compositing, so in-page readPixels is unreliable)+ try:+ from PIL import Image; import numpy as np+ im = np.array(Image.open(shot).convert("RGB")); q = (im[::6, ::6].reshape(-1, 3) // 16)+ distinct = len(set(map(tuple, q))); rep["shot_distinct"] = distinct; rep["shot_std"] = float(im.std())+ nonblank = distinct > 6 and float(im.std()) > 4.0+ except Exception as e:+ nonblank = info.get("distinct", 0) > 3+ rep["canvas_nonblank"] = bool(info.get("n", 0)) and nonblank+ if fatal: rep["status"] = "FAIL"; rep["error"] = fatal[0]+ elif not info.get("n"): rep["status"] = "EMPTY"; rep["error"] = "no <canvas>"+ elif not rep["canvas_nonblank"]: rep["status"] = "EMPTY"; rep["error"] = f"blank canvas (shot distinct={rep.get('shot_distinct')})"+ else: rep["status"] = "OK"+ except Exception as e:+ rep["error"] = (type(e).__name__ + ": " + str(e))[:400]+ if "Timeout" in rep["error"]: rep["status"] = "TIMEOUT"+ rep["latency_s"] = round(time.time() - t0, 1)+ return rep+if __name__ == "__main__":+ print(json.dumps(run_threejs(open(sys.argv[1]).read(), sys.argv[2]), indent=1))diff --git a/finetune/scripts/after_lora.sh b/finetune/scripts/after_lora.shnew file mode 100755index 00000000..e5bf6f3b--- /dev/null+++ b/finetune/scripts/after_lora.sh@@ -0,0 +1,11 @@+#!/bin/bash+# wait for LoRA run to finish -> merge adapter -> eval merged model on 3DCodeBench+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1+cd /wekafs/ict/hx_624/llm-ft+until [ -f $RUN/train_results.json ] || ! pgrep -f "configs/lf/qwen35_9b_lora_sft.yaml" >/dev/null; do sleep 20; done+[ -f $RUN/train_results.json ] || { echo "LORA_TRAIN_DID_NOT_FINISH"; exit 1; }+echo "[after_lora] train finished: $(cat $RUN/train_results.json | tr -d '\n')"+scripts/lf_export.sh configs/lf/export_lora_template.yaml > logs/export_lora_v1.log 2>&1 && echo "[after_lora] export done" || { echo "EXPORT_FAILED"; exit 1; }+ls $RUN/merged | head+GPUS=${EVAL_GPUS:-1} eval/run_eval.sh $RUN/merged eval/out/qwen35_9b_lora_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_qwen35_9b_lora_v1.log 2>&1+echo "[after_lora] eval done"; cat eval/out/qwen35_9b_lora_v1/summary.json; echo AFTER_LORA_DONEdiff --git a/finetune/scripts/after_lora_v2.sh b/finetune/scripts/after_lora_v2.shnew file mode 100755index 00000000..c490d139--- /dev/null+++ b/finetune/scripts/after_lora_v2.sh@@ -0,0 +1,16 @@+#!/bin/bash+# wait LoRA-v1 -> merge -> test vLLM on GPU 0 -> eval merged model with vLLM (GPU 0); fallback: HF sharded eval on GPUs 0,2+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1; cd /wekafs/ict/hx_624/llm-ft+until [ -f $RUN/train_results.json ] || ! pgrep -f "configs/lf/qwen35_9b_lora_sft.yaml" >/dev/null; do sleep 20; done+[ -f $RUN/train_results.json ] || { echo "LORA_TRAIN_DID_NOT_FINISH"; exit 1; }+echo "[after_lora] train finished: $(tr -d '\n' < $RUN/train_results.json)"+scripts/lf_export.sh configs/lf/export_lora_template.yaml > logs/export_lora_v1.log 2>&1 && echo "[after_lora] export done" || { echo "EXPORT_FAILED"; exit 1; }+echo "[after_lora] vllm smoke test on GPU 0"+if GPUS=0 eval/run_eval_vllm.sh /wekafs/ict/hx_624/models/Qwen3.5-9B eval/out/_vllm_smoke --no_think --limit 8 --max_new_tokens 1024 > logs/vllm_smoke.log 2>&1; then+ echo "[after_lora] VLLM_OK -> eval merged model with vLLM on GPU 0"+ GPUS=0 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_lora_v1 --no_think --max_new_tokens 6144 > logs/eval_qwen35_9b_lora_v1.log 2>&1+else+ echo "[after_lora] VLLM_FAILED -> HF eval on GPUs 0,2"; tail -20 logs/vllm_smoke.log+ GPUS=0,2 eval/run_eval.sh $RUN/merged eval/out/qwen35_9b_lora_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_qwen35_9b_lora_v1.log 2>&1+fi+echo "[after_lora] eval done"; cat eval/out/qwen35_9b_lora_v1/summary.json; echo AFTER_LORA_DONEdiff --git a/finetune/scripts/analyze_3dcodeverse.py b/finetune/scripts/analyze_3dcodeverse.pynew file mode 100644index 00000000..699914fd--- /dev/null+++ b/finetune/scripts/analyze_3dcodeverse.py@@ -0,0 +1,54 @@+"""Survey every metadata.parquet in 3DCodeVerse: counts, code length, caption availability, language, license, multi_file.+Writes data/analysis/3dcodeverse_survey.md (+ .json). Token counts are estimated with the Qwen3.5 tokenizer on a sample."""+import glob, json, os, random, sys+import numpy as np, pandas as pd+ROOT = "/wekafs/ict/hx_624/data/3dcodeverse_meta"+OUT = "/wekafs/ict/hx_624/llm-ft/data/analysis"; os.makedirs(OUT, exist_ok=True)+try:+ from transformers import AutoTokenizer+ tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+except Exception as e:+ tok = None; print("no tokenizer:", e)+random.seed(0)+rows = []+files = sorted(glob.glob(f"{ROOT}/**/metadata.parquet", recursive=True))+print(len(files), "metadata.parquet files")+agg = {}+for f in files:+ rel = os.path.relpath(os.path.dirname(f), ROOT)+ src = rel.split("/")[0]+ df = pd.read_parquet(f)+ n = len(df)+ code = df["code"].fillna("")+ clen = code.str.len().values+ meta0 = json.loads(df["meta_json"].iloc[0]) if "meta_json" in df and n else {}+ lang = df["language"].iloc[0] if "language" in df else meta0.get("language")+ caps = df["captions"].iloc[0] if "captions" in df else None+ cap_keys = list(caps.keys()) if isinstance(caps, dict) else []+ # caption availability+ def has(k): return float(np.mean([isinstance(c, dict) and bool(c.get(k)) for c in df["captions"]])) if "captions" in df else 0.0+ cap_avail = {k: round(has(k), 3) for k in ["detailed", "instruction", "factory", "brief", "synthetic_prompt"]}+ # token estimate on up to 200 random samples+ tok_p50 = tok_p90 = tok_mean = None+ if tok is not None and n:+ idx = random.sample(range(n), min(200, n))+ tl = np.array([len(tok(code.iloc[i]).input_ids) for i in idx])+ tok_p50, tok_p90, tok_mean = int(np.percentile(tl, 50)), int(np.percentile(tl, 90)), float(tl.mean())+ uniq = code.nunique()+ rows.append(dict(subset=rel, source=src, n=n, language=lang, license=meta0.get("license"), multi_file=meta0.get("multi_file"),+ entry=meta0.get("entry"), chars_p50=int(np.percentile(clen, 50)) if n else 0, chars_p90=int(np.percentile(clen, 90)) if n else 0,+ tok_p50=tok_p50, tok_p90=tok_p90, tok_mean=tok_mean, est_total_tokens=(tok_mean * n if tok_mean else None),+ unique_code_frac=round(uniq / max(n, 1), 3), caption_keys=cap_keys, cap_avail=cap_avail, columns=list(df.columns)))+ a = agg.setdefault(src, dict(n=0, est_tokens=0.0, langs=set()))+ a["n"] += n; a["est_tokens"] += (tok_mean * n if tok_mean else 0); a["langs"].add(str(lang))+ print(f"{rel:45s} n={n:6d} lang={str(lang):16s} tok_p50={tok_p50} tok_p90={tok_p90} uniq={uniq/max(n,1):.2f} caps={cap_avail}")+json.dump({"subsets": rows, "by_source": {k: dict(n=v["n"], est_tokens=v["est_tokens"], langs=sorted(v["langs"])) for k, v in agg.items()}},+ open(f"{OUT}/3dcodeverse_survey.json", "w"), indent=1, default=str)+with open(f"{OUT}/3dcodeverse_survey.md", "w") as f:+ f.write("| source | #samples | languages | est. tokens (code only) |\n|---|---|---|---|\n")+ for k, v in sorted(agg.items(), key=lambda kv: -kv[1]["n"]):+ f.write(f"| {k} | {v['n']} | {', '.join(sorted(v['langs']))} | {v['est_tokens']/1e6:.1f}M |\n")+ f.write("\n| subset | n | lang | tok p50 | tok p90 | unique code | captions (detailed/instruction/factory) | license |\n|---|---|---|---|---|---|---|---|\n")+ for r in rows:+ f.write(f"| {r['subset']} | {r['n']} | {r['language']} | {r['tok_p50']} | {r['tok_p90']} | {r['unique_code_frac']} | {r['cap_avail']['detailed']}/{r['cap_avail']['instruction']}/{r['cap_avail']['factory']} | {r['license']} |\n")+print("written", OUT)diff --git a/finetune/scripts/bon8_v1.sh b/finetune/scripts/bon8_v1.shnew file mode 100755index 00000000..2d58f997--- /dev/null+++ b/finetune/scripts/bon8_v1.sh@@ -0,0 +1,18 @@+#!/bin/bash+# best-of-8 for LoRA-v1: 8 sampled generations (T=0.7) on 4 GPUs now; Blender exec deferred until round-2 exec finishes (CPU contention); then pass@n+cd /wekafs/ict/hx_624/llm-ft; PFX=qwen35_9b_lora_v1_T07+scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_bon8.log 2>&1 || { echo "[bon8] EXPORT FAILED"; exit 1; }+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING+for round in 0 1; do+ for g in 0 1 2 3; do i=$((round*4+g+1)); [ $i -le 4 ] && continue # samples 1-4 already exist+ OUT=eval/out/${PFX}_s$i; mkdir -p $OUT+ CUDA_VISIBLE_DEVICES=$g python eval/generate_vllm.py --model runs/lf_qwen35_9b_lora_v1/merged --prompts data/sft_v1/bench_prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 6144 --temperature 0.7 --seed $i > $OUT/gen.log 2>&1 &+ done; wait+done+for i in 5 6 7 8; do cp eval/out/${PFX}_s$i/gens_shard0.jsonl eval/out/${PFX}_s$i/gens.jsonl; grep gen-vllm eval/out/${PFX}_s$i/gen.log | tail -1; done+rm -rf runs/lf_qwen35_9b_lora_v1/merged; echo "[bon8] $(date) gens done (s5-s8)"+until grep -q "\[r2\] .*exec done" logs/self_train_round2.log 2>/dev/null; do sleep 60; done+conda activate llmft+for i in 5 6 7 8; do OUT=eval/out/${PFX}_s$i; python eval/run_bench.py --gen_dir $OUT --workers 32 --timeout 300 > $OUT/exec.log 2>&1; python eval/metrics.py --gen_dir $OUT > $OUT/metrics.log 2>&1; grep "status counts" $OUT/exec.log; done+python eval/pass_at_n.py $PFX 8; python eval/pass_at_n.py $PFX 4; echo "[bon8] DONE"diff --git a/finetune/scripts/build_boot_data.py b/finetune/scripts/build_boot_data.pynew file mode 100644index 00000000..75e81a34--- /dev/null+++ b/finetune/scripts/build_boot_data.py@@ -0,0 +1,37 @@+"""Build bootstrapped SFT sets from execution-verified generations (proper python file; the earlier bash-heredoc version mangled the ```python fence via backtick substitution).+usage: build_boot_data.py <round_dir> <out_name> [--base data/sft_v1]"""+import argparse, glob, hashlib, json, os, random, shutil+ap = argparse.ArgumentParser(); ap.add_argument("round_dir"); ap.add_argument("out_name"); ap.add_argument("--base", default="data/sft_v1"); ap.add_argument("--max_per_prompt", type=int, default=2)+a = ap.parse_args(); random.seed(0)+W = a.round_dir+prompts = {}+for f in glob.glob(f"{W}/prompts*.jsonl"):+ for l in open(f):+ r = json.loads(l); prompts[r["task"]] = r+base = [json.loads(l) for l in open(f"{a.base}/train.jsonl")]+def h_of(code): return hashlib.sha1(("```python\n" + code + "\n```").encode()).hexdigest()+base_hashes = {hashlib.sha1(r["messages"][-1]["content"].encode()).hexdigest() for r in base}+seen = set(base_hashes); new = []; n_ok = 0; dup_base = 0; dup_self = 0; per_prompt = {}+for d in sorted(glob.glob(f"{W}/gen_s*/")) + sorted(glob.glob(f"{W}/gen_s*_g*/")):+ if not os.path.exists(os.path.join(d, "exec_results.jsonl")): continue+ for l in open(os.path.join(d, "exec_results.jsonl")):+ r = json.loads(l)+ if r["status"] != "OK": continue+ n_ok += 1+ code = open(os.path.join(d, r["task"], "code.py")).read().strip()+ if code.count("\n") < 4: continue+ h = h_of(code)+ if h in base_hashes: dup_base += 1; continue+ if h in seen: dup_self += 1; continue+ if per_prompt.get(r["task"], 0) >= a.max_per_prompt: continue+ seen.add(h); per_prompt[r["task"]] = per_prompt.get(r["task"], 0) + 1+ p = prompts[r["task"]]+ new.append({"messages": p["messages"][:2] + [{"role": "assistant", "content": "```python\n" + code + "\n```"}],+ "source": "boot_" + str(p.get("src", p.get("source", "train"))), "id": r["task"], "prompt_kind": "boot"})+out = f"data/{a.out_name}"; os.makedirs(out, exist_ok=True)+allrows = base + new; random.shuffle(allrows)+with open(f"{out}/train.jsonl", "w") as f:+ for r in allrows: f.write(json.dumps(r, ensure_ascii=False) + "\n")+shutil.copy(f"{a.base}/val.jsonl", f"{out}/val.jsonl")+stats = {"n_train": len(allrows), "n_base": len(base), "n_new": len(new), "n_ok_generations": n_ok, "dup_of_training_code": dup_base, "dup_within_generations": dup_self, "prompts_with_new": len(per_prompt)}+json.dump(stats, open(f"{out}/stats.json", "w"), indent=1); print(json.dumps(stats))diff --git a/finetune/scripts/build_md_bal.py b/finetune/scripts/build_md_bal.pynew file mode 100644index 00000000..3153d04d--- /dev/null+++ b/finetune/scripts/build_md_bal.py@@ -0,0 +1,34 @@+"""md_bal: re-balanced mix (~110M tok like md_big) with a much larger Blender share: bio x4, boot x2, CadQuery 35k, GLSL 25k, OpenSCAD x4, three.js."""+import json, random, os, sys+sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/scripts"); from ntok_cache import load_ntok+from transformers import AutoTokenizer+random.seed(13); tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+MD = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; CAP = {"cadquery": 35000, "glsl": 25000, "openscad": 10**9, "blender": 10**9, "threejs": 10**9}; REP = {"openscad": 4, "blender": 4}; BOOT_REP = 2+def ntok(r): return len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)+out_rows = []; stats = {}+for d, cap in CAP.items():+ rows = [json.loads(l) for l in open(f"{MD}/{d}/train.jsonl")]; NT = load_ntok(f"{MD}/{d}/train.jsonl", tok)+ for r, n_ in zip(rows, NT): r["_ntok"] = n_+ random.shuffle(rows); keep = []; toks = 0+ for r in rows:+ if len(keep) >= cap: break+ n = r.pop("_ntok")+ if n <= 8192: keep.append(r); toks += n+ rep = REP.get(d, 1); stats[d] = (len(keep) * rep, toks * rep); out_rows += keep * rep+BOOT = "/wekafs/ict/hx_624/llm-ft/data/sft_v13_boot/train.jsonl"; bl = open(BOOT).read().splitlines(); bn = load_ntok(BOOT, tok)+boot = [(json.loads(l), n) for l, n in zip(bl, bn) if '"source": "boot_' in l]+bt = 0; kb = []+for r, n in boot:+ if n <= 8192: kb.append({"messages": r["messages"], "dialect": "blender", "subset": "boot"}); bt += n+stats["blender_boot"] = (len(kb) * BOOT_REP, bt * BOOT_REP); out_rows += kb * BOOT_REP+# LLaMA-Factory's qwen3_5 (multimodal) template treats literal <image>/<video>/<audio> in text as media placeholders -> escape them+def _san(t): return t.replace("<image>", "<image >").replace("<video>", "<video >").replace("<audio>", "<audio >")+for r in out_rows:+ for m in r["messages"]: m["content"] = _san(m["content"])+random.shuffle(out_rows)+out = "/wekafs/ict/hx_624/llm-ft/data/md_bal"; os.makedirs(out, exist_ok=True)+with open(f"{out}/train.jsonl", "w") as f:+ for r in out_rows: f.write(json.dumps(r, ensure_ascii=False) + "\n")+os.system(f"cp /wekafs/ict/hx_624/llm-ft/data/md_mixed/val.jsonl {out}/val.jsonl")+tot = sum(v[1] for v in stats.values())+print({k: (v[0], round(v[1]/1e6, 1)) for k, v in stats.items()}, "| total samples", len(out_rows), "| tokens/epoch %.1fM" % (tot/1e6))diff --git a/finetune/scripts/build_md_big.py b/finetune/scripts/build_md_big.pynew file mode 100644index 00000000..57821a86--- /dev/null+++ b/finetune/scripts/build_md_big.py@@ -0,0 +1,31 @@+"""md_big: larger multi-dialect SFT set (token-filtered <=8192): blender (multidialect train + v13 bootstrapped), cadquery 40k, glsl 40k, openscad all."""+import json, random, os, sys+sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/scripts"); from ntok_cache import load_ntok+from transformers import AutoTokenizer+random.seed(11); tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+MD = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; CAP = {"cadquery": 40000, "glsl": 40000, "openscad": 5000, "blender": 10000}+def ntok(r): return len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)+out_rows = []; stats = {}+for d, cap in CAP.items():+ rows = [json.loads(l) for l in open(f"{MD}/{d}/train.jsonl")]; NT = load_ntok(f"{MD}/{d}/train.jsonl", tok)+ for r, n_ in zip(rows, NT): r["_ntok"] = n_+ random.shuffle(rows); keep = []; toks = 0+ for r in rows:+ if len(keep) >= cap: break+ n = r.pop("_ntok")+ if n <= 8192: keep.append(r); toks += n+ stats[d] = (len(keep), toks); out_rows += keep+# bootstrapped blender samples (execution-verified, novel prompts)+BOOT = "/wekafs/ict/hx_624/llm-ft/data/sft_v13_boot/train.jsonl"; bl = open(BOOT).read().splitlines(); bn = load_ntok(BOOT, tok)+boot = [(json.loads(l), n) for l, n in zip(bl, bn) if '"source": "boot_' in l]+bt = 0; kb = []+for r, n in boot:+ if n <= 8192: kb.append({"messages": r["messages"], "dialect": "blender", "subset": "boot"}); bt += n+stats["blender_boot"] = (len(kb), bt); out_rows += kb+random.shuffle(out_rows)+out = "/wekafs/ict/hx_624/llm-ft/data/md_big"; os.makedirs(out, exist_ok=True)+with open(f"{out}/train.jsonl", "w") as f:+ for r in out_rows: f.write(json.dumps(r, ensure_ascii=False) + "\n")+os.system(f"cp /wekafs/ict/hx_624/llm-ft/data/md_mixed/val.jsonl {out}/val.jsonl")+tot = sum(v[1] for v in stats.values())+print({k: (v[0], round(v[1]/1e6, 1)) for k, v in stats.items()}, "| total samples", len(out_rows), "| tokens/epoch %.1fM" % (tot/1e6))diff --git a/finetune/scripts/build_md_xl.py b/finetune/scripts/build_md_xl.pynew file mode 100644index 00000000..02d15af2--- /dev/null+++ b/finetune/scripts/build_md_xl.py@@ -0,0 +1,34 @@+"""md_xl: scale-up of md_big — ALL cadquery (deepcad+articraft) + ALL glsl + all blender (+15.7k bootstrapped) + openscad x4 upsampled + threejs, token-filtered <=8192."""+import json, random, os, sys+sys.path.insert(0, "/wekafs/ict/hx_624/llm-ft/scripts"); from ntok_cache import load_ntok+from transformers import AutoTokenizer+random.seed(12); tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+MD = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; CAP = {"cadquery": 10**9, "glsl": 10**9, "openscad": 10**9, "blender": 10**9, "threejs": 10**9}; REP = {"openscad": 4}+def ntok(r): return len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)+out_rows = []; stats = {}+for d, cap in CAP.items():+ rows = [json.loads(l) for l in open(f"{MD}/{d}/train.jsonl")]; NT = load_ntok(f"{MD}/{d}/train.jsonl", tok)+ for r, n_ in zip(rows, NT): r["_ntok"] = n_+ random.shuffle(rows); keep = []; toks = 0+ for r in rows:+ if len(keep) >= cap: break+ n = r.pop("_ntok")+ if n <= 8192: keep.append(r); toks += n+ rep = REP.get(d, 1); stats[d] = (len(keep) * rep, toks * rep); out_rows += keep * rep+BOOT = "/wekafs/ict/hx_624/llm-ft/data/sft_v13_boot/train.jsonl"; bl = open(BOOT).read().splitlines(); bn = load_ntok(BOOT, tok)+boot = [(json.loads(l), n) for l, n in zip(bl, bn) if '"source": "boot_' in l]+bt = 0; kb = []+for r, n in boot:+ if n <= 8192: kb.append({"messages": r["messages"], "dialect": "blender", "subset": "boot"}); bt += n+stats["blender_boot"] = (len(kb), bt); out_rows += kb+# LLaMA-Factory's qwen3_5 (multimodal) template treats literal <image>/<video>/<audio> in text as media placeholders -> escape them+def _san(t): return t.replace("<image>", "<image >").replace("<video>", "<video >").replace("<audio>", "<audio >")+for r in out_rows:+ for m in r["messages"]: m["content"] = _san(m["content"])+random.shuffle(out_rows)+out = "/wekafs/ict/hx_624/llm-ft/data/md_xl"; os.makedirs(out, exist_ok=True)+with open(f"{out}/train.jsonl", "w") as f:+ for r in out_rows: f.write(json.dumps(r, ensure_ascii=False) + "\n")+os.system(f"cp /wekafs/ict/hx_624/llm-ft/data/md_mixed/val.jsonl {out}/val.jsonl")+tot = sum(v[1] for v in stats.values())+print({k: (v[0], round(v[1]/1e6, 1)) for k, v in stats.items()}, "| total samples", len(out_rows), "| tokens/epoch %.1fM" % (tot/1e6))diff --git a/finetune/scripts/build_multidialect.py b/finetune/scripts/build_multidialect.pynew file mode 100644index 00000000..4cdb033f--- /dev/null+++ b/finetune/scripts/build_multidialect.py@@ -0,0 +1,77 @@+"""Build per-dialect SFT sets from ALL 3DCodeVerse metadata parquets + data-quality report.+Dialects: blender (bioinspired3d+factories non-test), cadquery_deepcad, cadquery_articraft, openscad (thingiverse), glsl (shadertoy).+Outputs data/multidialect/<dialect>/{train,test}.jsonl (+ all.jsonl) and data/multidialect/report.json"""+import glob, hashlib, json, os, random, re, sys, unicodedata+import pandas as pd, numpy as np+random.seed(7)+META = "/wekafs/ict/hx_624/data/3dcodeverse_meta"; OUT = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; os.makedirs(OUT, exist_ok=True)+SYS = {+ "blender": "You are an expert in procedural 3D modeling with Blender Python (bpy). Given a description of an object, write a complete, standalone Blender 5 Python script that builds it from scratch (clear the default scene first). Output ONLY the code in one ```python block.",+ "cadquery": "You are an expert CAD programmer. Given a description of a part, write a complete, standalone CadQuery (Python) script that builds it and leaves the final solid in a variable named `result`. Output ONLY the code in one ```python block.",+ "openscad": "You are an expert OpenSCAD programmer. Given a request, write a complete, standalone OpenSCAD (.scad) program that produces the described model. Output ONLY the code in one ```openscad block.",+ "glsl": "You are an expert shader programmer. Given a description, write a complete Shadertoy-style GLSL ES 3.00 fragment shader defining `void mainImage(out vec4 fragColor, in vec2 fragCoord)`. Output ONLY the code in one ```glsl block.",+}+FENCE = {"blender": "python", "cadquery": "python", "openscad": "openscad", "glsl": "glsl"}+def bad_chars(s):+ return sum(1 for ch in s if unicodedata.category(ch) in ("Cc",) and ch not in "\n\t\r") + s.count("�")+def wrap(dialect, caps, name):+ ins = (caps or {}).get("instruction") or ""; det = (caps or {}).get("detailed") or ""+ if dialect == "cadquery":+ return "Write a CadQuery (Python) script that builds the following CAD model: " + (ins or det)+ if dialect == "glsl":+ return ins if ins.lower().startswith(("write", "create", "implement", "make")) else "Write a GLSL fragment shader for Shadertoy that renders: " + (ins or det)+ if dialect == "openscad":+ return det or ins # thingiverse captions are already long OpenSCAD requests+ return ins or ("Create the 3D object described below using Python Blender code: " + det)+sets = {}+def add(dialect, sub, df, code_col="code"):+ rows = sets.setdefault(dialect, [])+ for r in df.itertuples():+ code = getattr(r, code_col) or ""; caps = dict(r.captions) if isinstance(r.captions, dict) else {}+ rows.append(dict(dialect=dialect, subset=sub, id=r.id, code=code, prompt=wrap(dialect, caps, getattr(r, "name", None)), caps=caps))+# --- sources+test_names = {d.replace("_seed0", "") for d in os.listdir("/wekafs/ict/hx_624/data/3dcodebench/data")}+for f in glob.glob(f"{META}/bioinspired3d/*/metadata.parquet"): add("blender", "bioinspired3d", pd.read_parquet(f))+for f in glob.glob(f"{META}/3dcodebench/*/metadata.parquet"):+ df = pd.read_parquet(f); df = df[~df["name"].str.replace("Factory", "", regex=False).isin(test_names)]; add("blender", "factories_nontest", df)+for f in glob.glob(f"{META}/deepcad/*/metadata.parquet"): add("cadquery", "deepcad", pd.read_parquet(f))+for f in glob.glob(f"{META}/articraft/cadquery_single_tex*/metadata.parquet"): add("cadquery", "articraft", pd.read_parquet(f))+add("openscad", "thingiverse", pd.read_parquet(f"{META}/thingiverse/openscad/metadata.parquet"))+for f in glob.glob(f"{META}/shadertoy/*/metadata.parquet"): add("glsl", "shadertoy", pd.read_parquet(f))+# --- QC + build+from transformers import AutoTokenizer+tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+report = {}+for dialect, rows in sets.items():+ n0 = len(rows); issues = dict(empty_code=0, short_code=0, bad_chars=0, no_prompt=0, dup_code=0, too_long=0)+ seen = set(); keep = []+ for s in rows:+ code = s["code"].strip()+ if not code: issues["empty_code"] += 1; continue+ if code.count("\n") < 3: issues["short_code"] += 1; continue+ if bad_chars(code) > 0: issues["bad_chars"] += 1; continue+ if len(s["prompt"]) < 20: issues["no_prompt"] += 1; continue+ h = hashlib.sha1(code.encode()).hexdigest()+ if h in seen: issues["dup_code"] += 1; continue+ seen.add(h); s["code"] = code; keep.append(s)+ # token lengths on a sample+ samp = random.sample(keep, min(500, len(keep)))+ lens = [len(tok(s["prompt"] + "\n" + s["code"]).input_ids) for s in samp]+ p50, p90, p99 = np.percentile(lens, [50, 90, 99]); frac_gt8k = float(np.mean([l > 8192 for l in lens])); frac_gt16k = float(np.mean([l > 16384 for l in lens]))+ random.shuffle(keep)+ n_test = min(200, max(50, len(keep) // 50))+ test, train = keep[:n_test], keep[n_test:]+ d = f"{OUT}/{dialect}"; os.makedirs(d, exist_ok=True)+ for split, rs in [("train", train), ("test", test)]:+ with open(f"{d}/{split}.jsonl", "w") as f:+ for s in rs:+ f.write(json.dumps({"messages": [{"role": "system", "content": SYS[dialect]}, {"role": "user", "content": s["prompt"]},+ {"role": "assistant", "content": f"```{FENCE[dialect]}\n{s['code']}\n```"}],+ "dialect": dialect, "subset": s["subset"], "id": s["id"]}, ensure_ascii=False) + "\n")+ by_sub = {}+ for s in keep: by_sub[s["subset"]] = by_sub.get(s["subset"], 0) + 1+ report[dialect] = dict(raw=n0, kept=len(keep), train=len(train), test=len(test), issues=issues, by_subset=by_sub,+ tok_p50=int(p50), tok_p90=int(p90), tok_p99=int(p99), frac_gt8k=round(frac_gt8k, 3), frac_gt16k=round(frac_gt16k, 3),+ est_train_tokens_M=round(float(np.mean(lens)) * len(train) / 1e6, 1))+ print(dialect, json.dumps(report[dialect]))+json.dump(report, open(f"{OUT}/report.json", "w"), indent=1)diff --git a/finetune/scripts/build_multidialect_train.py b/finetune/scripts/build_multidialect_train.pynew file mode 100644index 00000000..cf6d2e9a--- /dev/null+++ b/finetune/scripts/build_multidialect_train.py@@ -0,0 +1,27 @@+"""Subsample per-dialect train sets (token-filtered ≤ 8192), build LF datasets md_<dialect> and md_mixed."""+import json, random, os, subprocess+from transformers import AutoTokenizer+random.seed(3); tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+MD = "/wekafs/ict/hx_624/llm-ft/data/multidialect"; CAP = {"blender": 6000, "cadquery": 10000, "openscad": 2000, "glsl": 10000}+mixed = []+for d, cap in CAP.items():+ rows = [json.loads(l) for l in open(f"{MD}/{d}/train.jsonl")]; random.shuffle(rows)+ keep = []+ for r in rows:+ if len(keep) >= cap: break+ n = len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)+ if n <= 8192: keep.append(r)+ out = f"/wekafs/ict/hx_624/llm-ft/data/md_{d}"; os.makedirs(out, exist_ok=True)+ nv = max(20, len(keep) // 50); val, train = keep[:nv], keep[nv:]+ for split, rs in [("train", train), ("val", val)]:+ with open(f"{out}/{split}.jsonl", "w") as f:+ for r in rs: f.write(json.dumps(r, ensure_ascii=False) + "\n")+ print(d, "train", len(train), "val", len(val)); mixed += train+random.shuffle(mixed); out = "/wekafs/ict/hx_624/llm-ft/data/md_mixed"; os.makedirs(out, exist_ok=True)+with open(f"{out}/train.jsonl", "w") as f:+ for r in mixed: f.write(json.dumps(r, ensure_ascii=False) + "\n")+# mixed val = concat of per-dialect vals+with open(f"{out}/val.jsonl", "w") as f:+ for d in CAP:+ for l in open(f"/wekafs/ict/hx_624/llm-ft/data/md_{d}/val.jsonl"): f.write(l)+print("mixed train", len(mixed))diff --git a/finetune/scripts/build_report_html.py b/finetune/scripts/build_report_html.pynew file mode 100644index 00000000..514d441d--- /dev/null+++ b/finetune/scripts/build_report_html.py@@ -0,0 +1,62 @@+"""Render REPORT.md into a self-contained, themed HTML page (for publishing as an Artifact)."""+import markdown, re, datetime, sys+SRC = "/wekafs/ict/hx_624/llm-ft/REPORT.md"; DST = "/wekafs/ict/hx_624/llm-ft/REPORT.html"+md = open(SRC).read()+body = markdown.markdown(md, extensions=["tables", "fenced_code", "toc"], extension_configs={"toc": {"toc_depth": "2-3"}})+toc = re.search(r'<div class="toc">(.*?)</div>', body, re.S)+toc_html = toc.group(1) if toc else ""+body = body.replace(toc.group(0), "") if toc else body+css = """+:root{--bg:#f6f4ef;--paper:#fffdf8;--ink:#1f2a2e;--muted:#5b6a70;--line:#d9d4c7;--accent:#0e7c7b;--accent-2:#b5541c;--code:#eef2f1;--head:#13343b}+@media (prefers-color-scheme: dark){:root:not([data-theme="light"]){--bg:#15191b;--paper:#1c2124;--ink:#e6e2d8;--muted:#a3ada9;--line:#2f373a;--accent:#5ec8c5;--accent-2:#e59a62;--code:#242b2e;--head:#dfe9e7}}+:root[data-theme="dark"]{--bg:#15191b;--paper:#1c2124;--ink:#e6e2d8;--muted:#a3ada9;--line:#2f373a;--accent:#5ec8c5;--accent-2:#e59a62;--code:#242b2e;--head:#dfe9e7}+*{box-sizing:border-box}+body{margin:0;background:var(--bg);color:var(--ink);font-family:"IBM Plex Sans","Noto Sans SC","PingFang SC","Microsoft YaHei",system-ui,sans-serif;font-size:16px;line-height:1.7}+.wrap{display:grid;grid-template-columns:minmax(0,1fr) 260px;gap:40px;max-width:1180px;margin:0 auto;padding:40px 28px 80px}+@media (max-width:900px){.wrap{grid-template-columns:1fr}.toc{display:none}}+main{min-width:0;background:var(--paper);border:1px solid var(--line);border-radius:6px;padding:40px 44px}+@media (max-width:600px){main{padding:22px 18px}}+header.hero{margin-bottom:28px;border-bottom:2px solid var(--accent);padding-bottom:18px}+header.hero .eyebrow{font-family:"IBM Plex Mono",ui-monospace,monospace;font-size:12px;letter-spacing:.12em;text-transform:uppercase;color:var(--accent-2)}+h1,h2,h3,h4{color:var(--head);text-wrap:balance;line-height:1.25;font-family:"IBM Plex Sans","Noto Sans SC","PingFang SC",sans-serif}+h1{font-size:30px;margin:6px 0 8px;font-weight:600}+h2{font-size:22px;margin:44px 0 12px;padding-top:14px;border-top:1px solid var(--line);font-weight:600}+h3{font-size:17px;margin:28px 0 8px;font-weight:600}+p,li{max-width:78ch}+blockquote{margin:14px 0;padding:10px 16px;border-left:3px solid var(--accent);background:var(--code);color:var(--muted);border-radius:0 4px 4px 0}+code,pre{font-family:"IBM Plex Mono",ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.88em}+code{background:var(--code);padding:1px 5px;border-radius:3px}+pre{background:var(--code);padding:12px 14px;border-radius:4px;overflow-x:auto}+pre code{background:none;padding:0}+.tbl{overflow-x:auto;margin:14px 0}+table{border-collapse:collapse;width:100%;font-size:14px;font-variant-numeric:tabular-nums}+th,td{border-bottom:1px solid var(--line);padding:7px 10px;text-align:left;vertical-align:top}+th{color:var(--muted);font-weight:600;font-size:12.5px;letter-spacing:.04em;text-transform:uppercase;background:var(--code)}+tr:hover td{background:color-mix(in srgb,var(--accent) 6%,transparent)}+strong{color:var(--head)}+a{color:var(--accent)}+.toc{position:sticky;top:24px;align-self:start;font-size:13.5px;line-height:1.5}+.toc ul{list-style:none;padding-left:0;margin:0}+.toc ul ul{padding-left:12px;margin-top:2px}+.toc li{margin:4px 0}+.toc a{color:var(--muted);text-decoration:none}+.toc a:hover{color:var(--accent)}+.toc .label{font-family:"IBM Plex Mono",monospace;font-size:11px;letter-spacing:.12em;text-transform:uppercase;color:var(--accent-2);margin-bottom:8px}+footer{margin-top:40px;color:var(--muted);font-size:13px}+"""+body = re.sub(r"<table>", '<div class="tbl"><table>', body); body = re.sub(r"</table>", "</table></div>", body)+body = re.sub(r"<h1[^>]*>.*?</h1>", "", body, count=1)+html = f"""<title>3D-code LLM Finetune Lab Notes</title>+<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=IBM+Plex+Sans:wght@400;500;600&family=IBM+Plex+Mono:wght@400;500&family=Noto+Sans+SC:wght@400;500;700&display=swap">+<style>{css}</style>+<div class="wrap">+<main>+<header class="hero"><div class="eyebrow">dgx03 · 4×H100 · {datetime.date.today().isoformat()}</div>+<h1>3D-code LLM finetune 调研报告</h1>+<p style="color:var(--muted);margin:0">Qwen3.5-9B × LLaMA-Factory × 3DCodeVerse → 3DCodeBench(Blender 5.0 执行评测)。工程目录 <code>/wekafs/ict/hx_624/llm-ft</code>。</p></header>+{body}+<footer>Generated from <code>llm-ft/REPORT.md</code> · all numbers reproducible via <code>eval/compare.py</code> / <code>eval/results_table.py</code>.</footer>+</main>+<nav class="toc"><div class="label">Contents</div>{toc_html}</nav>+</div>"""+open(DST, "w").write(html); print("wrote", DST, len(html), "bytes")diff --git a/finetune/scripts/build_variants.sh b/finetune/scripts/build_variants.shnew file mode 100755index 00000000..bcebf979--- /dev/null+++ b/finetune/scripts/build_variants.sh@@ -0,0 +1,11 @@+#!/bin/bash+set -x+source /wekafs/ict/hx_624/llm-ft/env.sh; cd /wekafs/ict/hx_624/llm-ft+python scripts/prep_data.py --out data/sft_v2_indist --include_test_factories > logs/prep_v2.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v2_indist --name v2_indist+python scripts/prep_data.py --out data/sft_v3_bio --sources bio3d > logs/prep_v3.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v3_bio --name v3_bio+python scripts/prep_data.py --out data/sft_v4_cq --sources bio3d,fac,distill,thingi50,deepcad --deepcad_n 6000 > logs/prep_v4.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v4_cq --name v4_cq+python scripts/prep_data.py --out data/sft_v5_instr --prompt_fields instruction:1 > logs/prep_v5.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v5_instr --name v5_instr+python scripts/prep_data.py --out data/sft_v6_detail --prompt_fields detailed:1 > logs/prep_v6.log 2>&1 && python scripts/to_llamafactory.py --data_dir data/sft_v6_detail --name v6_detail+for v in v2_indist v3_bio v4_cq v5_instr v6_detail; do python scripts/make_lora_cfg.py $v $v; done+for v in v2_indist v3_bio v4_cq v5_instr v6_detail; do echo "== $v"; grep -E '"n_train"|"n_val"' data/sft_$v/stats.json | tr -d '\n'; echo; done+echo BUILD_VARIANTS_DONEdiff --git a/finetune/scripts/cache_ntok.py b/finetune/scripts/cache_ntok.pynew file mode 100644index 00000000..f944764f--- /dev/null+++ b/finetune/scripts/cache_ntok.py@@ -0,0 +1,20 @@+"""One-off: cache per-sample token counts (Qwen3.5 chat template) for every multidialect train/test file and the bootstrapped set.+Writes <file>.ntok (one int per line, same order). Build scripts read this instead of re-tokenizing."""+import json, os, sys, glob+from multiprocessing import Pool+FILES = sorted(glob.glob("/wekafs/ict/hx_624/llm-ft/data/multidialect/*/train.jsonl") + glob.glob("/wekafs/ict/hx_624/llm-ft/data/multidialect/*/test.jsonl")) + ["/wekafs/ict/hx_624/llm-ft/data/sft_v13_boot/train.jsonl"]+tok = None+def init():+ global tok+ from transformers import AutoTokenizer+ tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+def count(line):+ r = json.loads(line); return len(tok(tok.apply_chat_template(r["messages"], tokenize=False)).input_ids)+if __name__ == "__main__":+ with Pool(12, initializer=init) as pool:+ for f in FILES:+ out = f + ".ntok"+ if os.path.exists(out) and sum(1 for _ in open(out)) == sum(1 for _ in open(f)): print("cached", f); continue+ lines = open(f).read().splitlines()+ ns = pool.map(count, lines, chunksize=256)+ open(out, "w").write("\n".join(map(str, ns)) + "\n"); print(f, len(ns), "done", flush=True)diff --git a/finetune/scripts/ckpt_sweep.sh b/finetune/scripts/ckpt_sweep.shnew file mode 100755index 00000000..55a3cb6f--- /dev/null+++ b/finetune/scripts/ckpt_sweep.sh@@ -0,0 +1,16 @@+#!/bin/bash+# ckpt_sweep.sh <GPU> <RUN_DIR> <NAME> <ckpt_step> [<ckpt_step> ...]: merge each intermediate adapter checkpoint, vLLM-generate, exec in background+GPU=$1; RUN=$(readlink -f "$2"); NAME=$3; shift 3+cd /wekafs/ict/hx_624/llm-ft+for STEP in "$@"; do+ CK=$RUN/checkpoint-$STEP; [ -d $CK ] || { echo "[ckpt $NAME-$STEP] missing"; continue; }+ OUT=eval/out/qwen35_9b_lora_${NAME}_ckpt$STEP+ sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $CK|" -e "s|export_dir:.*|export_dir: $CK/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_${NAME}_ckpt$STEP.yaml+ scripts/lf_export.sh configs/lf/export_${NAME}_ckpt$STEP.yaml > logs/export_${NAME}_ckpt$STEP.log 2>&1 || { echo "[ckpt $NAME-$STEP] EXPORT FAILED"; continue; }+ ( source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU+ mkdir -p $OUT; python eval/generate_vllm.py --model $CK/merged --prompts data/sft_v1/bench_prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 6144 2>&1 | grep -E "gen-vllm|Error|Traceback" | tail -3 ) > logs/eval_${NAME}_ckpt$STEP.log 2>&1+ cp $OUT/gens_shard0.jsonl $OUT/gens.jsonl; rm -rf $CK/merged+ echo "[ckpt $NAME-$STEP] gen done: $(grep gen-vllm logs/eval_${NAME}_ckpt$STEP.log | tail -1)"+ nohup bash -c "source /wekafs/ict/hx_624/llm-ft/env.sh; cd /wekafs/ict/hx_624/llm-ft; python eval/run_bench.py --gen_dir $OUT --workers 24 --timeout 300 > $OUT/exec.log 2>&1; python eval/metrics.py --gen_dir $OUT > $OUT/metrics.log 2>&1; echo \"[ckpt $NAME-$STEP] eval done\"; cat $OUT/summary.json" >> logs/ckpt_sweep_$NAME.log 2>&1 &+done+echo "CKPT_SWEEP_DONE $NAME"diff --git a/finetune/scripts/contact_sheet_3dcodebench.py b/finetune/scripts/contact_sheet_3dcodebench.pynew file mode 100644index 00000000..4886f710--- /dev/null+++ b/finetune/scripts/contact_sheet_3dcodebench.py@@ -0,0 +1,33 @@+"""Render generated vs GT for a spread of 3DCodeBench tasks and compose a labelled contact sheet.+usage: python scripts/contact_sheet_3dcodebench.py <eval_out_dir> <out_png> [n_per_bucket]"""+import json, os, sys, subprocess, random+from concurrent.futures import ThreadPoolExecutor+from PIL import Image, ImageDraw, ImageFont+BL = "/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; R = os.path.abspath("eval/render_glb.py"); GT = "/wekafs/ict/hx_624/data/3dcodebench/data"+gen_dir, out_png = sys.argv[1], sys.argv[2]; k = int(sys.argv[3]) if len(sys.argv) > 3 else 4+m = [json.loads(l) for l in open(f"{gen_dir}/metrics.jsonl")]; ok = sorted([r for r in m if r["exec"] == "OK" and r.get("[email protected]") is not None], key=lambda r: -r["[email protected]"])+random.seed(3); n = len(ok)+sel = [("best", r) for r in ok[:k]] + [("median", r) for r in ok[n//2 - k//2: n//2 - k//2 + k]] + [("worst", r) for r in ok[-k:]] + [("random", r) for r in random.sample(ok[k:-k], k)]+work = os.path.abspath(f"{gen_dir}/_renders"); os.makedirs(work, exist_ok=True)+def render(glb, png):+ if os.path.exists(png): return+ subprocess.run([BL, "-b", "--factory-startup", "-noaudio", "--python", R, "--", "--glb", glb, "--out", png], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=300, env={**os.environ, "HOME": "/tmp"})+jobs = []+for tag, r in sel:+ t = r["task"]; jobs.append((os.path.abspath(f"{gen_dir}/{t}/out.glb"), f"{work}/{t}_gen.png")); jobs.append((f"{GT}/{t}/glb/{t}.glb", f"{work}/{t}_gt.png"))+with ThreadPoolExecutor(8) as ex: list(ex.map(lambda j: render(*j), jobs))+S = 200; cols = k; rows = 4+sheet = Image.new("RGB", (cols * (2 * S + 16) + 40, rows * (S + 52) + 10), "white"); d = ImageDraw.Draw(sheet)+try: font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", 13); fb = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf", 13)+except Exception: font = fb = ImageFont.load_default()+for i, (tag, r) in enumerate(sel):+ row, col = i // cols, i % cols; x0 = 40 + col * (2 * S + 16); y0 = 10 + row * (S + 52); t = r["task"]+ for j, kind in enumerate(("gen", "gt")):+ p = f"{work}/{t}_{kind}.png"+ try: im = Image.open(p).convert("RGB").resize((S, S))+ except Exception: im = Image.new("RGB", (S, S), (230, 230, 230))+ sheet.paste(im, (x0 + j * S, y0 + 20))+ d.text((x0, y0 + 2), f"{t.replace('_seed0','')} [email protected]={r['[email protected]']:.2f}", fill="black", font=fb)+ d.text((x0, y0 + S + 24), "generated (md_xl)", fill=(60, 60, 60), font=font); d.text((x0 + S, y0 + S + 24), "ground truth", fill=(60, 60, 60), font=font)+ if col == 0: d.text((4, y0 + S // 2), tag, fill=(120, 0, 0), font=fb)+sheet.save(out_png); print("wrote", out_png, sheet.size, "| tasks:", [r["task"] for _, r in sel])diff --git a/finetune/scripts/desc_prompt_eval.sh b/finetune/scripts/desc_prompt_eval.shnew file mode 100755index 00000000..873be1f1--- /dev/null+++ b/finetune/scripts/desc_prompt_eval.sh@@ -0,0 +1,12 @@+#!/bin/bash+# Cross prompt-style eval: 3DCodeBench prompt_description.txt (appearance descriptions) instead of prompt_instruction.txt, for base / v1 / v6_detail+set -uo pipefail; cd /wekafs/ict/hx_624/llm-ft; export PROMPTS=/wekafs/ict/hx_624/llm-ft/data/sft_v1/bench_prompts_desc.jsonl+until grep -q "\[dpo\] DONE\|FAILED" logs/exp_dpo_exec_v1.log 2>/dev/null; do sleep 30; done+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 2,3 | sort -n | tail -1)" -lt 10000 ]; do sleep 20; done+echo "[desc] $(date) start"+[ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_desc.log 2>&1+GPUS=2 eval/run_eval_vllm.sh runs/lf_qwen35_9b_lora_v1/merged eval/out/desc_qwen35_9b_lora_v1 --no_think --max_new_tokens 6144 > logs/eval_desc_v1.log 2>&1 &+GPUS=3 eval/run_eval_vllm.sh /wekafs/ict/hx_624/models/Qwen3.5-9B eval/out/desc_qwen35_9b_base --no_think --max_new_tokens 6144 > logs/eval_desc_base.log 2>&1 &+wait; echo "[desc] v1/base done"; cat eval/out/desc_qwen35_9b_lora_v1/summary.json | tr -d '\n'; echo; cat eval/out/desc_qwen35_9b_base/summary.json | tr -d '\n'; echo+scripts/lf_export.sh configs/lf/export_v6_detail.yaml > logs/export_v6_desc.log 2>&1 && GPUS=2 eval/run_eval_vllm.sh runs/lf_qwen35_9b_lora_v6_detail/merged eval/out/desc_qwen35_9b_lora_v6_detail --no_think --max_new_tokens 6144 > logs/eval_desc_v6.log 2>&1; echo "[desc] v6 done"; cat eval/out/desc_qwen35_9b_lora_v6_detail/summary.json | tr -d '\n'; echo+rm -rf runs/lf_qwen35_9b_lora_v6_detail/merged runs/lf_qwen35_9b_lora_v1/merged; echo "[desc] DONE"diff --git a/finetune/scripts/dpo_round2.sh b/finetune/scripts/dpo_round2.shnew file mode 100755index 00000000..356ef164--- /dev/null+++ b/finetune/scripts/dpo_round2.sh@@ -0,0 +1,43 @@+#!/bin/bash+# Iterative DPO round 2: DPO_v1 model samples K=2 on the 9.9k novel prompts -> Blender filter -> new (OK, FAIL) pairs -> DPO on top of DPO_v1 -> eval+set -uo pipefail; GPU=${GPUS:-1}; cd /wekafs/ict/hx_624/llm-ft; W=data/dpo_round2; mkdir -p $W+echo "[dpo2] $(date) start"+[ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_dpo2.log 2>&1+BASE=runs/lf_qwen35_9b_dpo_exec_v1/merged_r2+sed -e "s|export_dir:.*|export_dir: $(readlink -f runs/lf_qwen35_9b_dpo_exec_v1)/merged_r2|" configs/lf/export_dpo_exec_v1.yaml > configs/lf/export_dpo_exec_v1_r2.yaml+[ -d $BASE ] || scripts/lf_export.sh configs/lf/export_dpo_exec_v1_r2.yaml > logs/export_dpo_r2.log 2>&1 || { echo "[dpo2] EXPORT FAILED"; exit 1; }+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU+for i in 1 2; do python eval/generate_vllm.py --model $BASE --prompts data/selftrain_round2/prompts.jsonl --out $W/gen_s$i --tp 1 --no_think --max_new_tokens 4096 --temperature 0.8 --seed $((300+i)) 2>&1 | grep -E "gen-vllm|Error" | tail -1; done+echo "[dpo2] $(date) sampling done"+conda activate llmft+for i in 1 2; do python eval/run_bench.py --gen_dir $W/gen_s$i --workers 56 --timeout 120 > $W/gen_s$i/exec.log 2>&1; grep "status counts" $W/gen_s$i/exec.log; done+python - <<'PY'+import json, glob, os, random+random.seed(1); W="data/dpo_round2"+prompts={}+for l in open("data/selftrain_round2/prompts.jsonl"): r=json.loads(l); prompts[r["task"]]=r+by={}+for d in sorted(glob.glob(f"{W}/gen_s*/")):+ for l in open(d+"exec_results.jsonl"):+ r=json.loads(l); code=open(f"{d}/{r['task']}/code.py").read().strip()+ if code.count("\n")<3: continue+ by.setdefault(r["task"],{"ok":[],"bad":[]})["ok" if r["status"]=="OK" else "bad"].append(code)+pairs=[]+for t,v in by.items():+ if v["ok"] and v["bad"]:+ ok=random.choice(v["ok"]); bad=random.choice(v["bad"])+ if ok==bad: continue+ p=prompts[t]; pairs.append({"conversations":[{"from":"human","value":p["messages"][1]["content"]}],"system":p["messages"][0]["content"],"chosen":{"from":"gpt","value":"```python\n"+ok+"\n```"},"rejected":{"from":"gpt","value":"```python\n"+bad+"\n```"}})+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_exec_pairs_r2.json","w"), ensure_ascii=False)+info=json.load(open("data/lf/dataset_info.json")); info["dpo_exec_pairs_r2"]={"file_name":"dpo_exec_pairs_r2.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2); print("round2 pairs:", len(pairs), "prompts with any OK:", sum(1 for v in by.values() if v["ok"]), "/", len(by))+PY+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_exec_pairs_r2|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_exec_v2|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_exec_v2.yaml+echo "[dpo2] $(date) train"; GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_exec_v2.yaml gradient_accumulation_steps=16 > logs/train_dpo_exec_v2.log 2>&1+RUN=runs/lf_qwen35_9b_dpo_exec_v2; [ -f $RUN/train_results.json ] || { echo "[dpo2] TRAIN FAILED"; exit 1; }+echo "[dpo2] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_exec_v2.yaml+scripts/lf_export.sh configs/lf/export_dpo_exec_v2.yaml > logs/export_dpo_exec_v2.log 2>&1 || { echo "[dpo2] EXPORT2 FAILED"; exit 1; }+GPUS=$GPU eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_exec_v2 --no_think --max_new_tokens 6144 > logs/eval_dpo_exec_v2.log 2>&1+echo "[dpo2] $(date) eval done"; cat eval/out/qwen35_9b_dpo_exec_v2/summary.json; rm -rf $RUN/merged $BASE; echo "[dpo2] DONE"diff --git a/finetune/scripts/explore_sched.sh b/finetune/scripts/explore_sched.shnew file mode 100755index 00000000..c758f847--- /dev/null+++ b/finetune/scripts/explore_sched.sh@@ -0,0 +1,38 @@+#!/bin/bash+# 8h exploration scheduler: (1) after md_xl sweep + DPO finish -> md_bal LoRA on 4 GPUs (train+evals)+# (2) right after md_bal 'train done' -> full FT on the same md_bal data on GPUs 1,2,3 (ZeRO-3) (3) after md_bal evals free GPU 0 -> geometry-feedback DPO train+eval on GPU 0+set -uo pipefail; cd /wekafs/ict/hx_624/llm-ft+until grep -q "\[bigsweep\] DONE" logs/md_xl_ckpt_sweep.log 2>/dev/null; do sleep 60; done+until grep -qE "\[xldpo\] (DONE|TRAIN FAILED|EXPORT.* FAILED)" logs/md_xl_dpo_round.log 2>/dev/null; do sleep 60; done+echo "[sched] $(date) md_bal LoRA on GPUs 0,1,2,3"+scripts/run_md_exp.sh 0,1,2,3 md_bal > logs/exp_md_bal.log 2>&1 &+until grep -qE "\[md md_bal\] .*train done|TRAIN FAILED" logs/exp_md_bal.log 2>/dev/null; do sleep 60; done+sleep 60+echo "[sched] $(date) full FT md_bal on GPUs 1,2,3 (ZeRO-3, accum 3)"+( GPUS=1,2,3 scripts/lf_train.sh configs/lf/full_md_bal.yaml gradient_accumulation_steps=3 > logs/train_full_md_bal.log 2>&1+ RUN=runs/lf_qwen35_9b_full_md_bal; [ -f $RUN/train_results.json ] || { echo "[sched] FULL TRAIN FAILED"; exit 1; }+ echo "[sched] $(date) full md_bal train done: $(tr -d '\n' < $RUN/train_results.json | cut -c1-200)"+ until grep -q "\[md md_bal\] DONE" logs/exp_md_bal.log 2>/dev/null; do sleep 60; done # wait until GPU 1 is surely free of md_bal evals? (evals run on GPU 0) -> use GPU 1+ GPUS=1 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_md_bal --no_think --max_new_tokens 6144 > logs/eval_full_md_bal.log 2>&1+ echo "[sched] full md_bal 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_full_md_bal/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"+ GPUS=1 eval/run_dialect_eval.sh $RUN full_md_bal > logs/mdeval_full_md_bal.log 2>&1+ for D in cadquery openscad glsl blender; do echo "[sched] full md_bal $D: $(cat eval/out/md_full_md_bal_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done+ echo "[sched] FULL_DONE" ) &+until grep -q "\[md md_bal\] DONE" logs/exp_md_bal.log 2>/dev/null; do sleep 60; done+until grep -q "GEO_PAIRS_READY" logs/geo_dpo_build.log 2>/dev/null; do sleep 60; done+echo "[sched] $(date) geometry-feedback DPO on GPU 0"+GPUS=0 scripts/lf_train.sh configs/lf/dpo_geo_md_xl.yaml gradient_accumulation_steps=8 cutoff_len=4096 > logs/train_dpo_geo_md_xl.log 2>&1+RUN=runs/lf_qwen35_9b_dpo_geo_md_xl; [ -f $RUN/train_results.json ] || { echo "[sched] GEO DPO TRAIN FAILED"; grep -E "Error" logs/train_dpo_geo_md_xl.log | grep -v errors | tail -2 | cut -c1-200; }+if [ -f $RUN/train_results.json ]; then+ echo "[sched] $(date) geo dpo train done: $(tr -d '\n' < $RUN/train_results.json | cut -c1-200)"+ BASE=runs/lf_qwen35_9b_lora_md_xl/merged_geo+ sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_geo_md_xl.yaml+ scripts/lf_export.sh configs/lf/export_dpo_geo_md_xl.yaml > logs/export_dpo_geo_md_xl.log 2>&1+ GPUS=0 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_geo_md_xl --no_think --max_new_tokens 6144 > logs/eval_dpo_geo_md_xl.log 2>&1+ echo "[sched] geo dpo 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_geo_md_xl/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3), 'scored', round(s['[email protected]_mean_scored'],3))")"+ GPUS=0 eval/run_dialect_eval.sh $RUN/merged dpo_geo_md_xl blender cadquery > logs/mdeval_dpo_geo_md_xl.log 2>&1+ for D in blender cadquery; do echo "[sched] geo dpo $D: $(cat eval/out/md_dpo_geo_md_xl_$D/summary.json | tr -d '\n ' | cut -c1-200)"; done+ GPUS=0 eval/best_of_n.sh $RUN/merged dpo_geo_md_xl_T07 4 0.7 2>&1 | grep -E "pass@" | tail -1 | sed 's/^/[sched] geo dpo /'+fi+echo "[sched] GEO_DONE"+wait; echo "[sched] ALL DONE"diff --git a/finetune/scripts/fetch_hf_files.py b/finetune/scripts/fetch_hf_files.pynew file mode 100644index 00000000..a370f5b3--- /dev/null+++ b/finetune/scripts/fetch_hf_files.py@@ -0,0 +1,30 @@+"""Download only files matching a regex from an HF dataset repo (handles tree pagination, no file locks).+usage: python fetch_hf_files.py REPO OUTDIR REGEX [--subdir SUB] [--workers 32]"""+import argparse, os, re, sys, requests+from concurrent.futures import ThreadPoolExecutor+from huggingface_hub import HfApi+ap = argparse.ArgumentParser(); ap.add_argument("repo"); ap.add_argument("out"); ap.add_argument("regex")+ap.add_argument("--subdir", default=None); ap.add_argument("--workers", type=int, default=32)+a = ap.parse_args()+tok = os.environ["HF_TOKEN"]; api = HfApi(token=tok); pat = re.compile(a.regex)+files = [f.path for f in api.list_repo_tree(a.repo, repo_type="dataset", path_in_repo=a.subdir, recursive=True) if getattr(f, "size", None) is not None and pat.search(f.path)]+print(f"[fetch] {len(files)} files match", flush=True)+sess = requests.Session(); sess.headers["Authorization"] = f"Bearer {tok}"+def get(p):+ dst = os.path.join(a.out, p)+ if os.path.exists(dst) and os.path.getsize(dst) > 0: return 0+ os.makedirs(os.path.dirname(dst), exist_ok=True)+ for attempt in range(3):+ try:+ r = sess.get(f"https://huggingface.co/datasets/{a.repo}/resolve/main/{p}", timeout=60)+ if r.status_code == 200:+ open(dst, "wb").write(r.content); return 1+ except Exception as e:+ pass+ print("FAIL", p, flush=True); return -1+n = 0+with ThreadPoolExecutor(a.workers) as ex:+ for i, r in enumerate(ex.map(get, files)):+ n += (r == 1)+ if (i + 1) % 500 == 0: print(f"[fetch] {i+1}/{len(files)}", flush=True)+print(f"[fetch] done: {n} downloaded, {len(files)} total", flush=True)diff --git a/finetune/scripts/fetch_textfiles.sh b/finetune/scripts/fetch_textfiles.shnew file mode 100755index 00000000..e5caf550--- /dev/null+++ b/finetune/scripts/fetch_textfiles.sh@@ -0,0 +1,10 @@+#!/bin/bash+# Fetch only small text files (code/captions/meta/reports) of per-sample HF dataset folders with parallel curl.+# usage: fetch_textfiles.sh <repo_id> <subpath-or-empty> <outdir> <regex-of-files-to-keep>+REPO=$1; SUB=$2; OUT=$3; PAT=$4+mkdir -p "$OUT"; cd "$OUT" || exit 1+curl -s -H "Authorization: Bearer $HF_TOKEN" "https://huggingface.co/api/datasets/$REPO/tree/main/$SUB?recursive=true" \+ | python3 -c "import sys,json,re; pat=re.compile(sys.argv[1]); [print(x['path']) for x in json.load(sys.stdin) if x['type']=='file' and pat.search(x['path'])]" "$PAT" > .filelist+echo "files to fetch: $(wc -l < .filelist)"+cat .filelist | xargs -P 32 -I{} bash -c 'f="{}"; mkdir -p "$(dirname "$f")"; [ -s "$f" ] || curl -sS -L -H "Authorization: Bearer $HF_TOKEN" -o "$f" "https://huggingface.co/datasets/'$REPO'/resolve/main/$f" || echo "FAIL $f"'+echo "FETCH_DONE $(find . -type f ! -name .filelist | wc -l) files"diff --git a/finetune/scripts/geo_dpo_build.sh b/finetune/scripts/geo_dpo_build.shnew file mode 100755index 00000000..12b071b5--- /dev/null+++ b/finetune/scripts/geo_dpo_build.sh@@ -0,0 +1,50 @@+#!/bin/bash+# Geometry-feedback DPO data: sample md_xl (merged_geo) K times on train prompts that have reference code -> execute -> [email protected] vs reference mesh -> (high-F, low-F/FAIL) pairs+set -uo pipefail; cd /wekafs/ict/hx_624/llm-ft; G=${GPUS:-0}; W=data/geo_dpo; M=runs/lf_qwen35_9b_lora_md_xl/merged_geo+until [ -f $M/config.json ] && [ -n "$(ls $M/*.safetensors 2>/dev/null)" ] && [ $(( $(date +%s) - $(stat -c %Y logs/export_md_xl_geo.log) )) -gt 90 ]; do sleep 30; done+until grep -q "\[xlextra\] DONE" logs/md_xl_extra_eval.log 2>/dev/null; do sleep 30; done+echo "[geo] $(date) sampling on GPU $G"+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$G+for i in 1 2 3 4; do python eval/generate_vllm.py --model $M --prompts $W/blender_vllm_prompts.jsonl --out $W/gen_blender_s$i --tp 1 --no_think --max_new_tokens 4096 --temperature 0.8 --seed $((500+i)) 2>&1 | grep -E "gen-vllm|Error" | tail -1; done+for i in 1 2; do python eval/generate_vllm.py --model $M --prompts $W/cadquery_vllm_prompts.jsonl --out $W/gen_cadquery_s$i --tp 1 --no_think --max_new_tokens 4096 --temperature 0.8 --seed $((600+i)) 2>&1 | grep -E "gen-vllm|Error" | tail -1; done+echo "[geo] $(date) sampling done; executing + scoring"+conda activate llmft+for i in 1 2 3 4; do python eval/blender_dialect_eval.py --test $W/blender_prompts.jsonl --gen_dir $W/gen_blender_s$i --workers 40 2>&1 | tail -1 | cut -c1-200; done+for i in 1 2; do python eval/dialect_eval.py --dialect cadquery --test $W/cadquery_prompts.jsonl --gen_dir $W/gen_cadquery_s$i --workers 32 2>&1 | tail -1 | cut -c1-200; done+python - <<'PY'+import json, os, re, random, glob+from collections import Counter, defaultdict+W="data/geo_dpo"; random.seed(7)+def code_of(d, tid):+ p=f"{d}/{tid}/code.py"; return open(p).read().strip() if os.path.exists(p) else None+prompts={}+for dia in ["blender","cadquery"]:+ for l in open(f"{W}/{dia}_prompts.jsonl"):+ r=json.loads(l); prompts[r["id"].replace("/","__")]=(dia, r["messages"][0]["content"], r["messages"][1]["content"])+by=defaultdict(list) # tid -> [(F or None, status, code)]+for d in sorted(glob.glob(f"{W}/gen_*_s*")):+ mp=f"{d}/dialect_metrics.jsonl"+ if not os.path.exists(mp): continue+ for l in open(mp):+ r=json.loads(l); tid=r["id"]; code=code_of(d, tid)+ if not code or code.count("\n")<3: continue+ by[tid].append((r.get("[email protected]"), r.get("gen"), code))+pairs=[]; kinds=Counter(); per=Counter()+for tid, lst in by.items():+ dia, sys_, user=prompts[tid]; ok=[x for x in lst if x[1]=="OK" and x[0] is not None]; bad=[x for x in lst if x[1]!="OK"]+ if not ok: continue+ best=max(ok, key=lambda x:x[0]); worst=min(ok, key=lambda x:x[0])+ def mk(ch, rj, kind):+ if ch==rj: return+ pairs.append({"conversations":[{"from":"human","value":user}],"system":sys_,"chosen":{"from":"gpt","value":ch if ch.startswith("```") else "```\n"+ch+"\n```"},"rejected":{"from":"gpt","value":rj if rj.startswith("```") else "```\n"+rj+"\n```"}}); kinds[kind]+=1; per[dia]+=1+ if best[0]>=0.25 and len(ok)>=2 and best[0]-worst[0]>=0.15: mk(best[2], worst[2], "geo")+ if best[0]>=0.25 and bad: mk(best[2], random.choice(bad)[2], "exec")+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_geo_pairs.json","w"), ensure_ascii=False)+info=json.load(open("data/lf/dataset_info.json")); info["dpo_geo_pairs"]={"file_name":"dpo_geo_pairs.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2)+okF=[x[0] for l in by.values() for x in l if x[1]=="OK" and x[0] is not None]+print("geo pairs:", len(pairs), dict(kinds), dict(per), "| prompts with samples:", len(by), "| mean F of OK samples %.3f" % (sum(okF)/max(1,len(okF))))+PY+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $M)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_geo_pairs|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_geo_md_xl|" -e "s|overwrite_output_dir:.*|overwrite_output_dir: true|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_geo_md_xl.yaml+echo "GEO_PAIRS_READY"diff --git a/finetune/scripts/launch_sft.sh b/finetune/scripts/launch_sft.shnew file mode 100755index 00000000..3b37a0c1--- /dev/null+++ b/finetune/scripts/launch_sft.sh@@ -0,0 +1,14 @@+#!/bin/bash+# usage: GPUS=0,1 ./launch_sft.sh <run_name> [extra train_sft.py args...]+# Runs torchrun on the given GPUs (must be a subset of 0-3 per the user's rule).+set -euo pipefail+source /wekafs/ict/hx_624/llm-ft/env.sh+export CUDA_VISIBLE_DEVICES=${GPUS:-0,1}+RUN=$1; shift+NGPU=$(echo $CUDA_VISIBLE_DEVICES | tr ',' '\n' | wc -l)+OUT=/wekafs/ict/hx_624/llm-ft/runs/$RUN+mkdir -p $OUT+PORT=$((29500 + RANDOM % 500))+echo "[launch] run=$RUN gpus=$CUDA_VISIBLE_DEVICES ngpu=$NGPU port=$PORT out=$OUT"+cd /wekafs/ict/hx_624/llm-ft+torchrun --nproc_per_node=$NGPU --master_port=$PORT scripts/train_sft.py --output_dir $OUT "$@" 2>&1 | tee -a $OUT/train.logdiff --git a/finetune/scripts/lf_export.sh b/finetune/scripts/lf_export.shnew file mode 100755index 00000000..421e3acd--- /dev/null+++ b/finetune/scripts/lf_export.sh@@ -0,0 +1,6 @@+#!/bin/bash+# usage: ./lf_export.sh <export_yaml>+set -euo pipefail+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate lf+export HF_HOME=/wekafs/ict/hx_624/cache/huggingface DISABLE_VERSION_CHECK=1 CUDA_VISIBLE_DEVICES=""+CFG=$(readlink -f "$1"); cd /wekafs/ict/hx_624/tools/LLaMA-Factory && llamafactory-cli export "$CFG"diff --git a/finetune/scripts/lf_train.sh b/finetune/scripts/lf_train.shnew file mode 100755index 00000000..b604b4b8--- /dev/null+++ b/finetune/scripts/lf_train.sh@@ -0,0 +1,19 @@+#!/bin/bash+# usage: GPUS=0,1,2 ./lf_train.sh configs/lf/<cfg>.yaml [extra llamafactory-cli overrides: key=value ...]+set -euo pipefail+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate lf+source /wekafs/ict/hx_624/.secrets/hf.env+export WORKSPACE=/wekafs/ict/hx_624 HF_HOME=/wekafs/ict/hx_624/cache/huggingface HF_XET_HIGH_PERFORMANCE=1+export TORCH_EXTENSIONS_DIR=$WORKSPACE/cache/torch_extensions CUDA_HOME=/usr/local/cuda-12.8+# per-run Triton cache on LOCAL disk (shared cache on wekafs races between concurrent runs); seed it from the shared cache to avoid recompiling+TC=/tmp/hx_624_triton/$(basename "$(python -c "import yaml,sys; print(yaml.safe_load(open(sys.argv[1]))['output_dir'])" "$(readlink -f "$1")")")+[ -d "$TC" ] || { mkdir -p /tmp/hx_624_triton && cp -r $WORKSPACE/cache/triton "$TC" 2>/dev/null || mkdir -p "$TC"; }+export TRITON_CACHE_DIR=$TC+export TOKENIZERS_PARALLELISM=false DISABLE_VERSION_CHECK=1+export CUDA_VISIBLE_DEVICES=${GPUS:-0,1}+CFG=$(readlink -f "$1"); shift+RUN=$(python -c "import yaml,sys; print(yaml.safe_load(open('$CFG'))['output_dir'])")+mkdir -p $RUN+echo "[lf_train] cfg=$CFG gpus=$CUDA_VISIBLE_DEVICES out=$RUN"+cd /wekafs/ict/hx_624/tools/LLaMA-Factory+FORCE_TORCHRUN=1 MASTER_PORT=$((29600 + RANDOM % 300)) llamafactory-cli train $CFG "$@" 2>&1 | tee -a $RUN/train.logdiff --git a/finetune/scripts/make_lora_cfg.py b/finetune/scripts/make_lora_cfg.pynew file mode 100644index 00000000..ed02ec62--- /dev/null+++ b/finetune/scripts/make_lora_cfg.py@@ -0,0 +1,12 @@+"""make_lora_cfg.py NAME DATANAME [--epochs 2] [--lr 1e-4] -> configs/lf/lora_<NAME>.yaml (from the v1 LoRA template)"""+import sys, argparse, yaml+ap = argparse.ArgumentParser(); ap.add_argument("name"); ap.add_argument("dataname"); ap.add_argument("--epochs", type=float, default=2.0); ap.add_argument("--lr", type=float, default=1e-4)+ap.add_argument("--rank", type=int, default=64); ap.add_argument("--cutoff", type=int, default=8192)+a = ap.parse_args()+cfg = yaml.safe_load(open("/wekafs/ict/hx_624/llm-ft/configs/lf/qwen35_9b_lora_sft.yaml"))+cfg.update(dataset=f"{a.dataname}_train", eval_dataset=f"{a.dataname}_val",+ tokenized_path=f"/wekafs/ict/hx_624/llm-ft/data/lf/tokenized_{a.dataname}_len{a.cutoff}_packed",+ output_dir=f"/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_{a.name}",+ num_train_epochs=a.epochs, learning_rate=a.lr, lora_rank=a.rank, lora_alpha=2 * a.rank, cutoff_len=a.cutoff)+out = f"/wekafs/ict/hx_624/llm-ft/configs/lf/lora_{a.name}.yaml"+yaml.safe_dump(cfg, open(out, "w"), sort_keys=False); print(out)diff --git a/finetune/scripts/md_big_ckpt_sweep.sh b/finetune/scripts/md_big_ckpt_sweep.shnew file mode 100755index 00000000..a012b5df--- /dev/null+++ b/finetune/scripts/md_big_ckpt_sweep.sh@@ -0,0 +1,20 @@+#!/bin/bash+# After md_big training: evaluate intermediate checkpoints (training-amount curve) on 3DCodeBench (greedy) + dialect tests (greedy blender; T=0.7 cq/scad/glsl)+set -uo pipefail; GPU=${GPUS:-1}; cd /wekafs/ict/hx_624/llm-ft; RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_big+until grep -q "\[md md_big\] .*train done" logs/exp_md_big.log 2>/dev/null; do sleep 60; done+CKS=$(ls -d $RUN/checkpoint-* 2>/dev/null | sed 's/.*checkpoint-//' | sort -n | tr '\n' ' '); echo "[bigsweep] checkpoints: $CKS"+for STEP in $CKS; do+ CK=$RUN/checkpoint-$STEP; NAME=md_big_ck$STEP+ sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $CK|" -e "s|export_dir:.*|export_dir: $CK/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml+ scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[bigsweep] EXPORT FAILED $STEP"; continue; }+ GPUS=$GPU eval/run_eval_vllm.sh $CK/merged eval/out/qwen35_9b_lora_$NAME --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1+ echo "[bigsweep] ck$STEP 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_lora_$NAME/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"+ GPUS=$GPU eval/run_dialect_eval.sh $CK/merged $NAME blender > logs/mdeval_$NAME.log 2>&1+ source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU+ for D in glsl openscad cadquery; do OUT=eval/out/md_${NAME}_T07_$D; mkdir -p $OUT; cp eval/out/md_md_mixed_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $CK/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done+ conda activate llmft+ for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_${NAME}_T07_$D --workers 32 2>&1 | grep -E "dialect"; done+ echo "[bigsweep] ck$STEP blender: $(cat eval/out/md_${NAME}_blender/summary.json | tr -d '\n ' | cut -c1-140)"+ rm -rf $CK/merged+done+echo "[bigsweep] DONE"diff --git a/finetune/scripts/md_big_dpo_round.sh b/finetune/scripts/md_big_dpo_round.shnew file mode 100755index 00000000..042f8ed2--- /dev/null+++ b/finetune/scripts/md_big_dpo_round.sh@@ -0,0 +1,90 @@+#!/bin/bash+# Multi-dialect execution-feedback DPO on md_big: sample K=2 (T=0.8) on 1500 train prompts per dialect -> execute with each dialect's executor -> (OK, FAIL) pairs -> DPO -> eval (sampling for cq/scad/glsl, greedy for blender & 3DCodeBench)+set -uo pipefail; GPU=${GPUS:-2,3}; G1=$(echo $GPU | cut -d, -f1); G2=$(echo $GPU | cut -d, -f2); cd /wekafs/ict/hx_624/llm-ft; W=data/md_big_dpo_round; mkdir -p $W+echo "[bigdpo] $(date) start"+source /wekafs/ict/hx_624/llm-ft/env.sh+python - <<'PY'+import json, random; random.seed(5)+rows=[]+for d in ["cadquery","openscad","glsl","blender"]:+ rs=[json.loads(l) for l in open(f"data/md_{d}/train.jsonl")]; random.shuffle(rs)+ for i,r in enumerate(rs[:1500]): rows.append({"task": f"{d}_{i:05d}", "dialect": d, "messages": r["messages"][:2]})+random.shuffle(rows)+for g in range(2):+ with open(f"data/md_big_dpo_round/prompts_g{g}.jsonl","w") as f:+ for r in rows[g::2]: f.write(json.dumps(r)+"\n")+print("prompts", len(rows))+PY+BASE=runs/lf_qwen35_9b_lora_md_big/merged+[ -d $BASE ] || scripts/lf_export.sh configs/lf/export_md_big.yaml > logs/export_md_big_dpo.log 2>&1 || { echo "[bigdpo] EXPORT FAILED"; exit 1; }+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING+for i in 1 2; do+ CUDA_VISIBLE_DEVICES=$G1 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g0.jsonl --out $W/gen_s${i}_g0 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((400+i)) > $W/gen_s${i}_g0.log 2>&1 &+ CUDA_VISIBLE_DEVICES=$G2 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g1.jsonl --out $W/gen_s${i}_g1 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((410+i)) > $W/gen_s${i}_g1.log 2>&1 &+ wait; grep -h gen-vllm $W/gen_s${i}_g0.log $W/gen_s${i}_g1.log+done+echo "[bigdpo] $(date) sampling done"+conda activate llmft+python - <<'PY'+# execute every generation with its dialect's executor (parallel), then build pairs+import json, glob, os, sys, random+from concurrent.futures import ThreadPoolExecutor+sys.path.insert(0, "eval"); from dialect_runners import RUNNERS+import subprocess+BL="/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; RUNNER=os.path.abspath("eval/blender_runner.py")+def run_blender(code, d):+ d=os.path.abspath(d); os.makedirs(d, exist_ok=True); sp=os.path.join(d,"code.py"); open(sp,"w").write(code)+ try:+ subprocess.run([BL,"-b","--factory-startup","-noaudio","--python",RUNNER,"--","--script",sp,"--out",os.path.join(d,"out.glb"),"--report",os.path.join(d,"exec.json")],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=120,env={**os.environ,"HOME":"/tmp"})+ r=json.load(open(os.path.join(d,"exec.json"))) if os.path.exists(os.path.join(d,"exec.json")) else {"status":"CRASH"}+ except subprocess.TimeoutExpired: r={"status":"TIMEOUT"}+ return r+W="data/md_big_dpo_round"; prompts={}+for f in glob.glob(f"{W}/prompts_g*.jsonl"):+ for l in open(f): r=json.loads(l); prompts[r["task"]]=r+jobs=[]+for d in sorted(glob.glob(f"{W}/gen_s*_g*/")):+ for t in os.listdir(d):+ cp=os.path.join(d,t,"code.py")+ if os.path.isfile(cp): jobs.append((d,t))+def job(x):+ d,t=x; code=open(os.path.join(d,t,"code.py")).read(); dia=prompts[t]["dialect"]; wd=os.path.join(d,t,"exec")+ r = run_blender(code, wd) if dia=="blender" else RUNNERS[dia](code, wd)+ return (d,t,dia,r.get("status"))+with ThreadPoolExecutor(48) as ex: res=list(ex.map(job, jobs))+by={}+for d,t,dia,st in res:+ code=open(os.path.join(d,t,"code.py")).read().strip()+ if code.count("\n")<3: continue+ by.setdefault(t,{"dia":dia,"ok":[],"bad":[]})["ok" if st=="OK" else "bad"].append(code)+from collections import Counter+print("per-dialect pass rate:", {dia: round(sum(1 for d_,t_,di,st in res if di==dia and st=="OK")/max(1,sum(1 for d_,t_,di,st in res if di==dia)),3) for dia in ["cadquery","openscad","glsl","blender"]})+random.seed(2); pairs=[]; per=Counter()+for t,v in by.items():+ if v["ok"] and v["bad"]:+ ok=random.choice(v["ok"]); bad=random.choice(v["bad"])+ if ok==bad: continue+ p=prompts[t]; per[v["dia"]]+=1+ pairs.append({"conversations":[{"from":"human","value":p["messages"][1]["content"]}],"system":p["messages"][0]["content"],"chosen":{"from":"gpt","value":ok if ok.startswith("```") else "```\n"+ok+"\n```"},"rejected":{"from":"gpt","value":bad if bad.startswith("```") else "```\n"+bad+"\n```"}})+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_md_big_pairs.json","w"), ensure_ascii=False)+info=json.load(open("data/lf/dataset_info.json")); info["dpo_md_big_pairs"]={"file_name":"dpo_md_big_pairs.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2); print("md pairs:", len(pairs), dict(per))+PY+echo "[bigdpo] $(date) pairs built"+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_md_big_pairs|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_big|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_md_big.yaml+GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_md_big.yaml gradient_accumulation_steps=8 cutoff_len=4096 > logs/train_dpo_md_big.log 2>&1+RUN=runs/lf_qwen35_9b_dpo_md_big; [ -f $RUN/train_results.json ] || { echo "[bigdpo] TRAIN FAILED"; exit 1; }+echo "[bigdpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_md_big.yaml+scripts/lf_export.sh configs/lf/export_dpo_md_big.yaml > logs/export_dpo_md_big.log 2>&1 || { echo "[bigdpo] EXPORT2 FAILED"; exit 1; }+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_md_big --no_think --max_new_tokens 6144 > logs/eval_dpo_md_big.log 2>&1+echo "[bigdpo] 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_md_big/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged dpo_md_big > logs/mdeval_dpo_md_big.log 2>&1+# sampling eval for cq/scad/glsl+conda activate vllm; export CUDA_VISIBLE_DEVICES=$G1+for D in glsl openscad cadquery; do OUT=eval/out/md_dpo_md_big_T07_$D; mkdir -p $OUT; cp eval/out/md_dpo_md_big_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $RUN/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done+conda activate llmft+for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_dpo_md_big_T07_$D --workers 32 2>&1 | grep -E "dialect"; done+for D in cadquery openscad glsl blender; do echo "[bigdpo] greedy $D: $(cat eval/out/md_dpo_md_big_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done+rm -rf $RUN/merged $BASE; echo "[bigdpo] DONE"diff --git a/finetune/scripts/md_dpo_resume.sh b/finetune/scripts/md_dpo_resume.shnew file mode 100755index 00000000..80bb068e--- /dev/null+++ b/finetune/scripts/md_dpo_resume.sh@@ -0,0 +1,19 @@+#!/bin/bash+# resume multi-dialect DPO after OOM: train with cutoff 4096 (long GLSL/OpenSCAD pairs), then export + evals+set -uo pipefail; GPU=${GPUS:-2,3}; G1=$(echo $GPU | cut -d, -f1); cd /wekafs/ict/hx_624/llm-ft+BASE=runs/lf_qwen35_9b_lora_md_mixed/merged; RUN=runs/lf_qwen35_9b_dpo_md_mixed+echo "[mddpo] $(date) retrain with cutoff 4096"+GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_md_mixed.yaml gradient_accumulation_steps=8 cutoff_len=4096 > logs/train_dpo_md_mixed.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[mddpo] TRAIN FAILED"; grep -E "Error" logs/train_dpo_md_mixed.log | grep -v errors | tail -2 | cut -c1-200; exit 1; }+echo "[mddpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_md_mixed.yaml+scripts/lf_export.sh configs/lf/export_dpo_md_mixed.yaml > logs/export_dpo_md_mixed.log 2>&1 || { echo "[mddpo] EXPORT2 FAILED"; exit 1; }+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_md_mixed --no_think --max_new_tokens 6144 > logs/eval_dpo_md_mixed.log 2>&1+echo "[mddpo] 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_md_mixed/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged dpo_md_mixed > logs/mdeval_dpo_md_mixed.log 2>&1+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$G1+for D in glsl openscad cadquery; do OUT=eval/out/md_dpo_md_mixed_T07_$D; mkdir -p $OUT; cp eval/out/md_dpo_md_mixed_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $RUN/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done+conda activate llmft+for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_dpo_md_mixed_T07_$D --workers 32 2>&1 | grep -E "dialect"; done+for D in cadquery openscad glsl blender; do echo "[mddpo] greedy $D: $(cat eval/out/md_dpo_md_mixed_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done+rm -rf $RUN/merged $BASE; echo "[mddpo] DONE"diff --git a/finetune/scripts/md_dpo_round.sh b/finetune/scripts/md_dpo_round.shnew file mode 100755index 00000000..38e47570--- /dev/null+++ b/finetune/scripts/md_dpo_round.sh@@ -0,0 +1,90 @@+#!/bin/bash+# Multi-dialect execution-feedback DPO on md_mixed: sample K=2 (T=0.8) on 1500 train prompts per dialect -> execute with each dialect's executor -> (OK, FAIL) pairs -> DPO -> eval (sampling for cq/scad/glsl, greedy for blender & 3DCodeBench)+set -uo pipefail; GPU=${GPUS:-2,3}; G1=$(echo $GPU | cut -d, -f1); G2=$(echo $GPU | cut -d, -f2); cd /wekafs/ict/hx_624/llm-ft; W=data/md_dpo_round; mkdir -p $W+echo "[mddpo] $(date) start"+source /wekafs/ict/hx_624/llm-ft/env.sh+python - <<'PY'+import json, random; random.seed(5)+rows=[]+for d in ["cadquery","openscad","glsl","blender"]:+ rs=[json.loads(l) for l in open(f"data/md_{d}/train.jsonl")]; random.shuffle(rs)+ for i,r in enumerate(rs[:1500]): rows.append({"task": f"{d}_{i:05d}", "dialect": d, "messages": r["messages"][:2]})+random.shuffle(rows)+for g in range(2):+ with open(f"data/md_dpo_round/prompts_g{g}.jsonl","w") as f:+ for r in rows[g::2]: f.write(json.dumps(r)+"\n")+print("prompts", len(rows))+PY+BASE=runs/lf_qwen35_9b_lora_md_mixed/merged+[ -d $BASE ] || scripts/lf_export.sh configs/lf/export_md_mixed.yaml > logs/export_md_mixed_dpo.log 2>&1 || { echo "[mddpo] EXPORT FAILED"; exit 1; }+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING+for i in 1 2; do+ CUDA_VISIBLE_DEVICES=$G1 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g0.jsonl --out $W/gen_s${i}_g0 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((400+i)) > $W/gen_s${i}_g0.log 2>&1 &+ CUDA_VISIBLE_DEVICES=$G2 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g1.jsonl --out $W/gen_s${i}_g1 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((410+i)) > $W/gen_s${i}_g1.log 2>&1 &+ wait; grep -h gen-vllm $W/gen_s${i}_g0.log $W/gen_s${i}_g1.log+done+echo "[mddpo] $(date) sampling done"+conda activate llmft+python - <<'PY'+# execute every generation with its dialect's executor (parallel), then build pairs+import json, glob, os, sys, random+from concurrent.futures import ThreadPoolExecutor+sys.path.insert(0, "eval"); from dialect_runners import RUNNERS+import subprocess+BL="/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; RUNNER=os.path.abspath("eval/blender_runner.py")+def run_blender(code, d):+ d=os.path.abspath(d); os.makedirs(d, exist_ok=True); sp=os.path.join(d,"code.py"); open(sp,"w").write(code)+ try:+ subprocess.run([BL,"-b","--factory-startup","-noaudio","--python",RUNNER,"--","--script",sp,"--out",os.path.join(d,"out.glb"),"--report",os.path.join(d,"exec.json")],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=120,env={**os.environ,"HOME":"/tmp"})+ r=json.load(open(os.path.join(d,"exec.json"))) if os.path.exists(os.path.join(d,"exec.json")) else {"status":"CRASH"}+ except subprocess.TimeoutExpired: r={"status":"TIMEOUT"}+ return r+W="data/md_dpo_round"; prompts={}+for f in glob.glob(f"{W}/prompts_g*.jsonl"):+ for l in open(f): r=json.loads(l); prompts[r["task"]]=r+jobs=[]+for d in sorted(glob.glob(f"{W}/gen_s*_g*/")):+ for t in os.listdir(d):+ cp=os.path.join(d,t,"code.py")+ if os.path.isfile(cp): jobs.append((d,t))+def job(x):+ d,t=x; code=open(os.path.join(d,t,"code.py")).read(); dia=prompts[t]["dialect"]; wd=os.path.join(d,t,"exec")+ r = run_blender(code, wd) if dia=="blender" else RUNNERS[dia](code, wd)+ return (d,t,dia,r.get("status"))+with ThreadPoolExecutor(48) as ex: res=list(ex.map(job, jobs))+by={}+for d,t,dia,st in res:+ code=open(os.path.join(d,t,"code.py")).read().strip()+ if code.count("\n")<3: continue+ by.setdefault(t,{"dia":dia,"ok":[],"bad":[]})["ok" if st=="OK" else "bad"].append(code)+from collections import Counter+print("per-dialect pass rate:", {dia: round(sum(1 for d_,t_,di,st in res if di==dia and st=="OK")/max(1,sum(1 for d_,t_,di,st in res if di==dia)),3) for dia in ["cadquery","openscad","glsl","blender"]})+random.seed(2); pairs=[]; per=Counter()+for t,v in by.items():+ if v["ok"] and v["bad"]:+ ok=random.choice(v["ok"]); bad=random.choice(v["bad"])+ if ok==bad: continue+ p=prompts[t]; per[v["dia"]]+=1+ pairs.append({"conversations":[{"from":"human","value":p["messages"][1]["content"]}],"system":p["messages"][0]["content"],"chosen":{"from":"gpt","value":ok if ok.startswith("```") else "```\n"+ok+"\n```"},"rejected":{"from":"gpt","value":bad if bad.startswith("```") else "```\n"+bad+"\n```"}})+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_md_pairs.json","w"), ensure_ascii=False)+info=json.load(open("data/lf/dataset_info.json")); info["dpo_md_pairs"]={"file_name":"dpo_md_pairs.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2); print("md pairs:", len(pairs), dict(per))+PY+echo "[mddpo] $(date) pairs built"+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_md_pairs|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_mixed|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_md_mixed.yaml+GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_md_mixed.yaml gradient_accumulation_steps=8 > logs/train_dpo_md_mixed.log 2>&1+RUN=runs/lf_qwen35_9b_dpo_md_mixed; [ -f $RUN/train_results.json ] || { echo "[mddpo] TRAIN FAILED"; exit 1; }+echo "[mddpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_md_mixed.yaml+scripts/lf_export.sh configs/lf/export_dpo_md_mixed.yaml > logs/export_dpo_md_mixed.log 2>&1 || { echo "[mddpo] EXPORT2 FAILED"; exit 1; }+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_md_mixed --no_think --max_new_tokens 6144 > logs/eval_dpo_md_mixed.log 2>&1+echo "[mddpo] 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_md_mixed/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged dpo_md_mixed > logs/mdeval_dpo_md_mixed.log 2>&1+# sampling eval for cq/scad/glsl+conda activate vllm; export CUDA_VISIBLE_DEVICES=$G1+for D in glsl openscad cadquery; do OUT=eval/out/md_dpo_md_mixed_T07_$D; mkdir -p $OUT; cp eval/out/md_dpo_md_mixed_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $RUN/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done+conda activate llmft+for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_dpo_md_mixed_T07_$D --workers 32 2>&1 | grep -E "dialect"; done+for D in cadquery openscad glsl blender; do echo "[mddpo] greedy $D: $(cat eval/out/md_dpo_md_mixed_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done+rm -rf $RUN/merged $BASE; echo "[mddpo] DONE"diff --git a/finetune/scripts/md_xl_ckpt_sweep.sh b/finetune/scripts/md_xl_ckpt_sweep.shnew file mode 100755index 00000000..093b5037--- /dev/null+++ b/finetune/scripts/md_xl_ckpt_sweep.sh@@ -0,0 +1,20 @@+#!/bin/bash+# After md_xl training: evaluate intermediate checkpoints (training-amount curve) on 3DCodeBench (greedy) + dialect tests (greedy blender; T=0.7 cq/scad/glsl)+set -uo pipefail; GPU=${GPUS:-1}; cd /wekafs/ict/hx_624/llm-ft; RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_md_xl+until grep -q "\[md md_xl\] .*train done" logs/exp_md_xl.log 2>/dev/null; do sleep 60; done+CKS=$(ls -d $RUN/checkpoint-* 2>/dev/null | sed 's/.*checkpoint-//' | sort -n | tr '\n' ' '); echo "[bigsweep] checkpoints: $CKS"+for STEP in $CKS; do+ CK=$RUN/checkpoint-$STEP; NAME=md_xl_ck$STEP+ sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $CK|" -e "s|export_dir:.*|export_dir: $CK/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml+ scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[bigsweep] EXPORT FAILED $STEP"; continue; }+ GPUS=$GPU eval/run_eval_vllm.sh $CK/merged eval/out/qwen35_9b_lora_$NAME --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1+ echo "[bigsweep] ck$STEP 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_lora_$NAME/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"+ GPUS=$GPU eval/run_dialect_eval.sh $CK/merged $NAME blender > logs/mdeval_$NAME.log 2>&1+ source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU+ for D in glsl openscad cadquery; do OUT=eval/out/md_${NAME}_T07_$D; mkdir -p $OUT; cp eval/out/md_md_mixed_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $CK/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done+ conda activate llmft+ for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_${NAME}_T07_$D --workers 32 2>&1 | grep -E "dialect"; done+ echo "[bigsweep] ck$STEP blender: $(cat eval/out/md_${NAME}_blender/summary.json | tr -d '\n ' | cut -c1-140)"+ rm -rf $CK/merged+done+echo "[bigsweep] DONE"diff --git a/finetune/scripts/md_xl_dpo_round.sh b/finetune/scripts/md_xl_dpo_round.shnew file mode 100755index 00000000..2314030c--- /dev/null+++ b/finetune/scripts/md_xl_dpo_round.sh@@ -0,0 +1,90 @@+#!/bin/bash+# Multi-dialect execution-feedback DPO on md_xl: sample K=2 (T=0.8) on 1500 train prompts per dialect -> execute with each dialect's executor -> (OK, FAIL) pairs -> DPO -> eval (sampling for cq/scad/glsl, greedy for blender & 3DCodeBench)+set -uo pipefail; GPU=${GPUS:-2,3}; G1=$(echo $GPU | cut -d, -f1); G2=$(echo $GPU | cut -d, -f2); cd /wekafs/ict/hx_624/llm-ft; W=data/md_xl_dpo_round; mkdir -p $W+echo "[xldpo] $(date) start"+source /wekafs/ict/hx_624/llm-ft/env.sh+python - <<'PY'+import json, random; random.seed(5)+rows=[]+for d in ["cadquery","openscad","glsl","blender"]:+ rs=[json.loads(l) for l in open(f"data/md_{d}/train.jsonl")]; random.shuffle(rs)+ for i,r in enumerate(rs[:1500]): rows.append({"task": f"{d}_{i:05d}", "dialect": d, "messages": r["messages"][:2]})+random.shuffle(rows)+for g in range(2):+ with open(f"data/md_xl_dpo_round/prompts_g{g}.jsonl","w") as f:+ for r in rows[g::2]: f.write(json.dumps(r)+"\n")+print("prompts", len(rows))+PY+BASE=runs/lf_qwen35_9b_lora_md_xl/merged+[ -d $BASE ] || scripts/lf_export.sh configs/lf/export_md_xl.yaml > logs/export_md_xl_dpo.log 2>&1 || { echo "[xldpo] EXPORT FAILED"; exit 1; }+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING+for i in 1 2; do+ CUDA_VISIBLE_DEVICES=$G1 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g0.jsonl --out $W/gen_s${i}_g0 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((400+i)) > $W/gen_s${i}_g0.log 2>&1 &+ CUDA_VISIBLE_DEVICES=$G2 python eval/generate_vllm.py --model $BASE --prompts $W/prompts_g1.jsonl --out $W/gen_s${i}_g1 --tp 1 --no_think --max_new_tokens 6144 --temperature 0.8 --seed $((410+i)) > $W/gen_s${i}_g1.log 2>&1 &+ wait; grep -h gen-vllm $W/gen_s${i}_g0.log $W/gen_s${i}_g1.log+done+echo "[xldpo] $(date) sampling done"+conda activate llmft+python - <<'PY'+# execute every generation with its dialect's executor (parallel), then build pairs+import json, glob, os, sys, random+from concurrent.futures import ThreadPoolExecutor+sys.path.insert(0, "eval"); from dialect_runners import RUNNERS+import subprocess+BL="/wekafs/ict/hx_624/tools/blender-5.0.1-linux-x64/blender"; RUNNER=os.path.abspath("eval/blender_runner.py")+def run_blender(code, d):+ d=os.path.abspath(d); os.makedirs(d, exist_ok=True); sp=os.path.join(d,"code.py"); open(sp,"w").write(code)+ try:+ subprocess.run([BL,"-b","--factory-startup","-noaudio","--python",RUNNER,"--","--script",sp,"--out",os.path.join(d,"out.glb"),"--report",os.path.join(d,"exec.json")],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=120,env={**os.environ,"HOME":"/tmp"})+ r=json.load(open(os.path.join(d,"exec.json"))) if os.path.exists(os.path.join(d,"exec.json")) else {"status":"CRASH"}+ except subprocess.TimeoutExpired: r={"status":"TIMEOUT"}+ return r+W="data/md_xl_dpo_round"; prompts={}+for f in glob.glob(f"{W}/prompts_g*.jsonl"):+ for l in open(f): r=json.loads(l); prompts[r["task"]]=r+jobs=[]+for d in sorted(glob.glob(f"{W}/gen_s*_g*/")):+ for t in os.listdir(d):+ cp=os.path.join(d,t,"code.py")+ if os.path.isfile(cp): jobs.append((d,t))+def job(x):+ d,t=x; code=open(os.path.join(d,t,"code.py")).read(); dia=prompts[t]["dialect"]; wd=os.path.join(d,t,"exec")+ r = run_blender(code, wd) if dia=="blender" else RUNNERS[dia](code, wd)+ return (d,t,dia,r.get("status"))+with ThreadPoolExecutor(48) as ex: res=list(ex.map(job, jobs))+by={}+for d,t,dia,st in res:+ code=open(os.path.join(d,t,"code.py")).read().strip()+ if code.count("\n")<3: continue+ by.setdefault(t,{"dia":dia,"ok":[],"bad":[]})["ok" if st=="OK" else "bad"].append(code)+from collections import Counter+print("per-dialect pass rate:", {dia: round(sum(1 for d_,t_,di,st in res if di==dia and st=="OK")/max(1,sum(1 for d_,t_,di,st in res if di==dia)),3) for dia in ["cadquery","openscad","glsl","blender"]})+random.seed(2); pairs=[]; per=Counter()+for t,v in by.items():+ if v["ok"] and v["bad"]:+ ok=random.choice(v["ok"]); bad=random.choice(v["bad"])+ if ok==bad: continue+ p=prompts[t]; per[v["dia"]]+=1+ pairs.append({"conversations":[{"from":"human","value":p["messages"][1]["content"]}],"system":p["messages"][0]["content"],"chosen":{"from":"gpt","value":ok if ok.startswith("```") else "```\n"+ok+"\n```"},"rejected":{"from":"gpt","value":bad if bad.startswith("```") else "```\n"+bad+"\n```"}})+random.shuffle(pairs); json.dump(pairs, open("data/lf/dpo_md_xl_pairs.json","w"), ensure_ascii=False)+info=json.load(open("data/lf/dataset_info.json")); info["dpo_md_xl_pairs"]={"file_name":"dpo_md_xl_pairs.json","ranking":True,"formatting":"sharegpt","columns":{"messages":"conversations","system":"system","chosen":"chosen","rejected":"rejected"}}+json.dump(info, open("data/lf/dataset_info.json","w"), indent=2); print("md pairs:", len(pairs), dict(per))+PY+echo "[xldpo] $(date) pairs built"+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|dataset: dpo_exec_pairs|dataset: dpo_md_xl_pairs|" -e "s|output_dir:.*|output_dir: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_dpo_md_xl|" configs/lf/dpo_exec_v1.yaml > configs/lf/dpo_md_xl.yaml+GPUS=$GPU scripts/lf_train.sh configs/lf/dpo_md_xl.yaml gradient_accumulation_steps=8 cutoff_len=4096 > logs/train_dpo_md_xl.log 2>&1+RUN=runs/lf_qwen35_9b_dpo_md_xl; [ -f $RUN/train_results.json ] || { echo "[xldpo] TRAIN FAILED"; exit 1; }+echo "[xldpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|model_name_or_path:.*|model_name_or_path: $(readlink -f $BASE)|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_dpo_md_xl.yaml+scripts/lf_export.sh configs/lf/export_dpo_md_xl.yaml > logs/export_dpo_md_xl.log 2>&1 || { echo "[xldpo] EXPORT2 FAILED"; exit 1; }+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_dpo_md_xl --no_think --max_new_tokens 6144 > logs/eval_dpo_md_xl.log 2>&1+echo "[xldpo] 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_dpo_md_xl/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged dpo_md_xl > logs/mdeval_dpo_md_xl.log 2>&1+# sampling eval for cq/scad/glsl+conda activate vllm; export CUDA_VISIBLE_DEVICES=$G1+for D in glsl openscad cadquery; do OUT=eval/out/md_dpo_md_xl_T07_$D; mkdir -p $OUT; cp eval/out/md_dpo_md_xl_$D/prompts.jsonl $OUT/; python eval/generate_vllm.py --model $RUN/merged --prompts $OUT/prompts.jsonl --out $OUT --tp 1 --no_think --max_new_tokens 8192 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm" | tail -1; done+conda activate llmft+for D in glsl openscad cadquery; do python eval/dialect_eval.py --dialect $D --test data/multidialect/$D/test.jsonl --gen_dir eval/out/md_dpo_md_xl_T07_$D --workers 32 2>&1 | grep -E "dialect"; done+for D in cadquery openscad glsl blender; do echo "[xldpo] greedy $D: $(cat eval/out/md_dpo_md_xl_$D/summary.json | tr -d '\n ' | cut -c1-160)"; done+rm -rf $RUN/merged $BASE; echo "[xldpo] DONE"diff --git a/finetune/scripts/md_xl_extra_eval.sh b/finetune/scripts/md_xl_extra_eval.shnew file mode 100755index 00000000..a4f71b76--- /dev/null+++ b/finetune/scripts/md_xl_extra_eval.sh@@ -0,0 +1,15 @@+#!/bin/bash+# GPU 0 while DPO (GPUs 2,3) and sweep (GPU 1) run: three.js eval (greedy + T=0.7) and 3DCodeBench best-of-4 for md_xl (uses the merged model re-exported by the DPO round)+cd /wekafs/ict/hx_624/llm-ft; M=runs/lf_qwen35_9b_lora_md_xl/merged+until [ -f data/md_xl_dpo_round/gen_s1_g0.log ] && [ -f $M/config.json ]; do sleep 30; done; sleep 60+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=0 PLAYWRIGHT_BROWSERS_PATH=/wekafs/ict/hx_624/cache/ms-playwright+echo "[xlextra] $(date) three.js gens"+mkdir -p eval/out/md_md_xl_threejs eval/out/md_md_xl_T07_threejs+python eval/generate_vllm.py --model $M --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_md_xl_threejs --tp 1 --no_think --max_new_tokens 12288 2>&1 | grep -E "gen-vllm|Error" | tail -1+python eval/generate_vllm.py --model $M --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_md_xl_T07_threejs --tp 1 --no_think --max_new_tokens 12288 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm|Error" | tail -1+echo "[xlextra] $(date) best-of-4"+GPUS=0 eval/best_of_n.sh $M md_xl_T07 4 0.7 2>&1 | grep -E "pass@|OK rate|BEST"+conda activate llmft+for N in md_md_xl_threejs md_md_xl_T07_threejs; do echo "== $N"; python eval/threejs_eval.py --gen_dir eval/out/$N 2>&1 | tail -2; done+echo "[xlextra] DONE"diff --git a/finetune/scripts/md_xl_launch.sh b/finetune/scripts/md_xl_launch.shnew file mode 100755index 00000000..2ea3bb05--- /dev/null+++ b/finetune/scripts/md_xl_launch.sh@@ -0,0 +1,20 @@+#!/bin/bash+cd /wekafs/ict/hx_624/llm-ft; source env.sh+echo "[md_xl] $(date) build data"+python scripts/build_md_xl.py 2>&1 | grep -v Warning | tail -1 | tee logs/build_md_xl.stats+python scripts/to_llamafactory.py --data_dir /wekafs/ict/hx_624/llm-ft/data/md_xl --name md_xl 2>&1 | tail -3 | cut -c1-200+# ~10 checkpoints over the run: steps = tokens / (4 GPUs x 2 accum x 8192)+python3 - <<'PY'+import re, yaml+s=open("logs/build_md_xl.stats").read(); tok=float(re.search(r"tokens/epoch ([0-9.]+)M", s).group(1))*1e6+steps=tok/(8*8192); save=max(50, int(round(steps/10/50))*50)+p="configs/lf/lora_md_xl.yaml"; c=yaml.safe_load(open(p)); c["save_steps"]=save; c["eval_steps"]=save; c["save_total_limit"]=14+yaml.safe_dump(c, open(p,"w"), sort_keys=False); print(f"[md_xl] est. steps {steps:.0f}, save/eval every {save} steps")+PY+echo "BUILD_MD_XL_DONE"+G=0,1,2,3; echo "[md_xl] using GPUs $G"+scripts/run_md_exp.sh $G md_xl 2>&1 | tee -a logs/exp_md_xl.log+grep -q "\[md md_xl\] DONE" logs/exp_md_xl.log || { echo "[md_xl] pipeline failed, not sweeping"; exit 1; }+echo "[md_xl] $(date) starting checkpoint sweep on GPU 1"+GPUS=1 scripts/md_xl_ckpt_sweep.sh > logs/md_xl_ckpt_sweep.log 2>&1+echo "[md_xl] ALL DONE"diff --git a/finetune/scripts/ntok_cache.py b/finetune/scripts/ntok_cache.pynew file mode 100644index 00000000..7884a458--- /dev/null+++ b/finetune/scripts/ntok_cache.py@@ -0,0 +1,12 @@+"""Helper for build scripts: load cached token counts for a jsonl (scripts/cache_ntok.py), falling back to tokenizing."""+import os+def load_ntok(path, tok=None):+ c = path + ".ntok"+ if os.path.exists(c):+ ns = [int(x) for x in open(c).read().split()]+ if ns and sum(1 for _ in open(path)) == len(ns): return ns+ if tok is None:+ from transformers import AutoTokenizer; tok = AutoTokenizer.from_pretrained("/wekafs/ict/hx_624/models/Qwen3.5-9B")+ import json+ ns = [len(tok(tok.apply_chat_template(json.loads(l)["messages"], tokenize=False)).input_ids) for l in open(path)]+ open(c, "w").write("\n".join(map(str, ns)) + "\n"); return nsdiff --git a/finetune/scripts/prep_data.py b/finetune/scripts/prep_data.pynew file mode 100644index 00000000..87141c34--- /dev/null+++ b/finetune/scripts/prep_data.py@@ -0,0 +1,184 @@+"""Build SFT data (chat 'messages' jsonl) for text -> Blender-Python from the 3DCodeVerse-family sources.++Sources (all Blender Python):+ * bioinspired3d geo/tex metadata.parquet (ilabai/3dcodeverse) ~6.3k, short scripts+ * 3dcodebench factories geo/tex parquet (ilabai/3dcodeverse) 243*2, Infinigen style, LONG scripts+ -> by default ONLY the 31 factories that are NOT in the 3DCodeBench test set (avoid contamination)+ * blender_distill (Claude-generated) (ilabai/3dcodeverse) ~60-100+ * thingiverse-openscad-blender-5-0 (ilabai/...) ~1.2k, keep native-CSG only (no mesh fallback)+Outputs: <out>/train.jsonl, <out>/val.jsonl, <out>/stats.json, <out>/bench_prompts.jsonl+"""+import argparse, glob, hashlib, json, os, random, re+import pandas as pd++SYSTEM = ("You are an expert in procedural 3D modeling with Blender Python (bpy). "+ "Given a description of an object, write a complete, standalone Blender 5 Python script that builds it "+ "from scratch (clear the default scene first, create geometry with bpy/bmesh, assign simple materials when "+ "relevant, leave the finished mesh objects in the scene). Output ONLY the code in one ```python block.")++def ans(code):+ return "```python\n" + code.strip("\n") + "\n```"++def wrap_desc(desc):+ starters = ["Write a Blender Python script that builds the following object: ",+ "Create this 3D object with Blender Python (bpy): ",+ "Using Python Blender code, procedurally model: ",+ "Generate a Blender Python (bpy) script for a 3D model matching this description: "]+ return random.choice(starters) + desc.strip()++PROMPT_W = {"instruction": 6, "detailed": 3, "name": 1}+def pick_prompt(name, caps):+ """caps: dict with keys detailed / instruction / factory (some may be missing)."""+ ins = (caps or {}).get("instruction") or ""+ det = (caps or {}).get("detailed") or ""+ opts = []+ if ins and len(ins) > 20: opts += [("instruction", ins)] * PROMPT_W.get("instruction", 0)+ if det and len(det) > 20: opts += [("detailed", wrap_desc(det))] * PROMPT_W.get("detailed", 0)+ if name and len(name) > 15 and not name.lower().startswith("deepcad"): opts += [("name", name)] * PROMPT_W.get("name", 0)+ if not opts: return None, None+ return random.choice(opts)++def code_ok(code, min_lines=4):+ if not code or code.count("\n") < min_lines: return False+ if "import bpy" not in code and "bpy." not in code and "import cadquery" not in code: return False+ if "reference_mesh.json" in code or "MESH_REFERENCE_ASSET" in code: return False # mesh fallback+ return True++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--root", default="/wekafs/ict/hx_624/data")+ ap.add_argument("--out", default="/wekafs/ict/hx_624/llm-ft/data/sft_v1")+ ap.add_argument("--val_frac", type=float, default=0.02)+ ap.add_argument("--seed", type=int, default=42)+ ap.add_argument("--include_test_factories", action="store_true", help="(contaminated) also include the 212 factories that overlap the benchmark")+ ap.add_argument("--max_chars", type=int, default=60000, help="drop samples whose code is longer than this (~15k tokens)")+ ap.add_argument("--tokenizer", default="/wekafs/ict/hx_624/models/Qwen2.5-Coder-7B-Instruct")+ ap.add_argument("--max_tokens", type=int, default=8192, help="drop samples whose full chat exceeds this many tokens")+ ap.add_argument("--sources", default="bio3d,fac,distill,thingi50", help="comma list of sources to include: bio3d,fac,distill,thingi50,deepcad")+ ap.add_argument("--prompt_fields", default="instruction:6,detailed:3,name:1", help="weighted prompt field choice, e.g. 'instruction:1' for instruction-only")+ ap.add_argument("--deepcad_n", type=int, default=20000, help="number of deepcad CadQuery samples when 'deepcad' in --sources")+ args = ap.parse_args()+ random.seed(args.seed)+ os.makedirs(args.out, exist_ok=True)+ SRC = set(args.sources.split(","))+ PROMPT_W.clear(); PROMPT_W.update({k: int(v) for k, v in (x.split(":") for x in args.prompt_fields.split(","))})+ R = args.root+ samples = [] # dict(source, id, prompt_kind, messages)++ # ---- test-set names (for contamination control)+ bench_dir = f"{R}/3dcodebench/data"+ test_names = {d.replace("_seed0", "") for d in os.listdir(bench_dir)}++ # ---- bioinspired3d+ for sub in (["bpy_structures_geo", "bpy_structures_tex"] if "bio3d" in SRC else []):+ df = pd.read_parquet(f"{R}/3dcodeverse/bioinspired3d_{sub}_metadata.parquet")+ for r in df.itertuples():+ if not code_ok(r.code): continue+ kind, p = pick_prompt(r.name, dict(r.captions))+ if not p: continue+ samples.append(dict(source=f"bio3d_{sub[-3:]}", id=r.id, prompt_kind=kind, code=r.code, prompt=p))++ # ---- 3dcodebench factories (Infinigen style)+ for sub in (["factories_geo", "factories_tex"] if "fac" in SRC else []):+ df = pd.read_parquet(f"{R}/3dcodeverse/3dcodebench_{sub}_metadata.parquet")+ for r in df.itertuples():+ fac = r.name.replace("Factory", "")+ if fac in test_names and not args.include_test_factories: continue+ if not code_ok(r.code): continue+ kind, p = pick_prompt(None, dict(r.captions))+ if not p: continue+ samples.append(dict(source=f"fac_{sub[-3:]}" + ("_TEST" if fac in test_names else ""), id=r.id, prompt_kind=kind, code=r.code, prompt=p))++ # ---- blender_distill+ for cp in (glob.glob(f"{R}/3dcodeverse/blender_distill/**/code.py", recursive=True) if "distill" in SRC else []):+ d = os.path.dirname(cp); code = open(cp).read()+ if not code_ok(code): continue+ caps = json.load(open(os.path.join(d, "captions.json"))) if os.path.exists(os.path.join(d, "captions.json")) else {}+ p = caps.get("user_instruction") or (caps.get("instruction") and wrap_desc(caps["instruction"]))+ if not p: continue+ # distill scripts define build() but may not call it -> make sure the script executes standalone+ if re.search(r"^def build\(\):", code, re.M) and not re.search(r"^\s*build\(\)\s*$", code, re.M) and "__main__" not in code:+ code = code.rstrip("\n") + "\n\n\nif __name__ == \"__main__\":\n build()\n"+ samples.append(dict(source="distill", id=os.path.relpath(d, R), prompt_kind="user_instruction", code=code, prompt=p))++ # ---- deepcad (CadQuery) -- cross-dialect data; the code is NOT Blender, system prompt is adapted+ if "deepcad" in SRC:+ dfs = [pd.read_parquet(f) for f in sorted(glob.glob(f"{R}/3dcodeverse_meta/deepcad/*/metadata.parquet"))]+ dd = pd.concat(dfs, ignore_index=True).sample(n=min(args.deepcad_n, sum(len(d) for d in dfs)), random_state=args.seed)+ for r in dd.itertuples():+ code = r.code or ""+ if "import cadquery" not in code or code.count("\n") < 3: continue+ caps = dict(r.captions); ins = caps.get("instruction") or caps.get("detailed") or ""+ if len(ins) < 20: continue+ p = "Write a CadQuery (Python) script that builds the following CAD model: " + ins.strip()+ samples.append(dict(source="deepcad", id=r.id, prompt_kind="instruction", code=code, prompt=p, dialect="cadquery"))+ # ---- thingiverse-openscad-blender-5-0 (native CSG only)+ for cp in (glob.glob(f"{R}/thingiverse_5_0/*/code.py") if "thingi50" in SRC else []):+ d = os.path.dirname(cp); code = open(cp).read()+ rep_p = os.path.join(d, "reports", "translation_report.json")+ status = json.load(open(rep_p)).get("status", "") if os.path.exists(rep_p) else ""+ if "mesh_fallback" in status or not code_ok(code): continue+ caps = json.load(open(os.path.join(d, "captions.json"))) if os.path.exists(os.path.join(d, "captions.json")) else {}+ desc = caps.get("synthetic_prompt") or caps.get("detailed") or ""+ if len(desc) < 40: continue+ p = ("The following request was originally written for OpenSCAD. Implement the same design in Blender Python (bpy) "+ "instead, as a standalone script that builds the geometry with native Blender primitives/booleans.\n\n" + desc.strip())+ samples.append(dict(source="thingi50", id=os.path.basename(d), prompt_kind="synthetic_prompt", code=code, prompt=p))++ # ---- dedup by code hash, length filter+ seen, uniq = set(), []+ for s in samples:+ h = hashlib.sha1(s["code"].strip().encode()).hexdigest()+ if h in seen or len(s["code"]) > args.max_chars: continue+ seen.add(h); uniq.append(s)+ # ---- token length filter (drop too-long samples instead of truncating the completion)+ tok = None+ try:+ from transformers import AutoTokenizer+ tok = AutoTokenizer.from_pretrained(args.tokenizer)+ except Exception as e:+ print("tokenizer unavailable:", e)+ if tok is not None:+ before = len(uniq)+ def ntok(s):+ msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": s["prompt"]}, {"role": "assistant", "content": ans(s["code"])}]+ return len(tok(tok.apply_chat_template(msgs, tokenize=False)).input_ids)+ uniq = [s for s in uniq if ntok(s) <= args.max_tokens]+ print(f"dropped {before - len(uniq)} samples longer than {args.max_tokens} tokens")+ random.shuffle(uniq)+ n_val = max(1, int(len(uniq) * args.val_frac))+ val, train = uniq[:n_val], uniq[n_val:]++ SYSTEM_CQ = ("You are an expert CAD programmer. Given a description of a part, write a complete, standalone CadQuery (Python) "+ "script that builds it. Output ONLY the code in one ```python block.")+ def to_msgs(s):+ return {"messages": [{"role": "system", "content": SYSTEM_CQ if s.get("dialect") == "cadquery" else SYSTEM},+ {"role": "user", "content": s["prompt"]},+ {"role": "assistant", "content": ans(s["code"])}],+ "source": s["source"], "id": s["id"], "prompt_kind": s["prompt_kind"]}++ stats = {"n_train": len(train), "n_val": len(val), "by_source": {}, "token_len": {}}+ for split, rows in [("train", train), ("val", val)]:+ lens = []+ with open(f"{args.out}/{split}.jsonl", "w") as f:+ for s in rows:+ m = to_msgs(s)+ f.write(json.dumps(m, ensure_ascii=False) + "\n")+ stats["by_source"].setdefault(split, {}); stats["by_source"][split][s["source"]] = stats["by_source"][split].get(s["source"], 0) + 1+ if tok is not None:+ lens.append(len(tok(tok.apply_chat_template(m["messages"], tokenize=False)).input_ids))+ if lens:+ import numpy as np+ lens = np.array(lens)+ stats["token_len"][split] = {"p50": int(np.percentile(lens, 50)), "p90": int(np.percentile(lens, 90)), "p99": int(np.percentile(lens, 99)), "max": int(lens.max()), "total": int(lens.sum()), "n_gt_8192": int((lens > 8192).sum()), "n_gt_16384": int((lens > 16384).sum())}++ # ---- benchmark prompts+ with open(f"{args.out}/bench_prompts.jsonl", "w") as f:+ for d in sorted(os.listdir(bench_dir)):+ p = open(f"{bench_dir}/{d}/prompt_instruction.txt").read().strip()+ f.write(json.dumps({"task": d, "messages": [{"role": "system", "content": SYSTEM}, {"role": "user", "content": p}]}) + "\n")+ json.dump(stats, open(f"{args.out}/stats.json", "w"), indent=2)+ print(json.dumps(stats, indent=2))++if __name__ == "__main__":+ main()diff --git a/finetune/scripts/queue.sh b/finetune/scripts/queue.shnew file mode 100755index 00000000..5bb5c90f--- /dev/null+++ b/finetune/scripts/queue.sh@@ -0,0 +1,10 @@+#!/bin/bash+# queue.sh <GPU> <NAME1> [NAME2 ...]: run experiments sequentially on one GPU (waits until no training of ours is using that GPU)+GPU=$1; shift+cd /wekafs/ict/hx_624/llm-ft+for NAME in "$@"; do+ # wait until every GPU in the list has < 10 GB used (previous job finished)+ until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i $GPU | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done+ scripts/run_exp.sh $GPU $NAME 2>&1 | tee -a logs/exp_$NAME.log+done+echo "QUEUE_DONE gpu=$GPU"diff --git a/finetune/scripts/run_dpo_exec.sh b/finetune/scripts/run_dpo_exec.shnew file mode 100755index 00000000..4b7f69cf--- /dev/null+++ b/finetune/scripts/run_dpo_exec.sh@@ -0,0 +1,11 @@+#!/bin/bash+set -uo pipefail; cd /wekafs/ict/hx_624/llm-ft; GPU=${GPUS:-2,3}; NAME=dpo_exec_v1; RUN=runs/lf_qwen35_9b_$NAME+echo "[dpo] $(date) export v1 merged"; [ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_dpo.log 2>&1 || { echo "[dpo] EXPORT FAILED"; exit 1; }+echo "[dpo] $(date) train on GPUs $GPU"; GPUS=$GPU scripts/lf_train.sh configs/lf/$NAME.yaml > logs/train_$NAME.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[dpo] TRAIN FAILED"; grep -E "Error|error" logs/train_$NAME.log | tail -5 | cut -c1-200; exit 1; }+echo "[dpo] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|model_name_or_path:.*|model_name_or_path: /wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_v1/merged|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[dpo] EXPORT2 FAILED"; exit 1; }+echo "[dpo] $(date) merged"; G1=$(echo $GPU | cut -d, -f1)+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_$NAME --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1+echo "[dpo] $(date) eval done"; cat eval/out/qwen35_9b_$NAME/summary.json; rm -rf $RUN/merged; echo "[dpo] DONE"diff --git a/finetune/scripts/run_exp.sh b/finetune/scripts/run_exp.shnew file mode 100755index 00000000..042f5554--- /dev/null+++ b/finetune/scripts/run_exp.sh@@ -0,0 +1,23 @@+#!/bin/bash+# run_exp.sh <GPU(s)> <NAME>: train LoRA (configs/lf/lora_<NAME>.yaml) -> merge -> vLLM generation (GPU) ; Blender exec + metrics run in the background (CPU)+set -uo pipefail+GPU=$1; NAME=$2+NGPU=$(echo $GPU | tr ',' '\n' | wc -l); ACC=$((8 / NGPU))+cd /wekafs/ict/hx_624/llm-ft+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_$NAME; OUT=/wekafs/ict/hx_624/llm-ft/eval/out/qwen35_9b_lora_$NAME+echo "[exp $NAME] $(date) train on GPU(s) $GPU (grad_accum=$ACC)"+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_$NAME.yaml gradient_accumulation_steps=$ACC > logs/train_$NAME.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[exp $NAME] TRAIN FAILED"; exit 1; }+echo "[exp $NAME] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $RUN|" -e "s|export_dir:.*|export_dir: $RUN/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[exp $NAME] EXPORT FAILED"; exit 1; }+echo "[exp $NAME] $(date) merged"+# --- generation on GPU (vLLM), then free the GPU+( source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING TOKENIZERS_PARALLELISM=false CUDA_VISIBLE_DEVICES=$GPU+ mkdir -p $OUT; python eval/generate_vllm.py --model $RUN/merged --prompts data/sft_v1/bench_prompts.jsonl --out $OUT --tp $NGPU --no_think --max_new_tokens 6144 2>&1 | grep -E "gen-vllm|Error|Traceback" | tail -5 ) > logs/eval_$NAME.log 2>&1+cp $OUT/gens_shard0.jsonl $OUT/gens.jsonl 2>/dev/null || { echo "[exp $NAME] GEN FAILED"; cat logs/eval_$NAME.log | tail -5; exit 1; }+echo "[exp $NAME] $(date) gen done: $(grep gen-vllm logs/eval_$NAME.log | tail -1)"+rm -rf $RUN/merged+# --- Blender exec + metrics in background (CPU only) so the GPU queue can continue+nohup bash -c "source /wekafs/ict/hx_624/llm-ft/env.sh; cd /wekafs/ict/hx_624/llm-ft; python eval/run_bench.py --gen_dir $OUT --workers 32 --timeout 300 > $OUT/exec.log 2>&1; python eval/metrics.py --gen_dir $OUT > $OUT/metrics.log 2>&1; echo \"[exp $NAME] \$(date) eval done\"; cat $OUT/summary.json; echo \"[exp $NAME] DONE\"" >> logs/exp_$NAME.log 2>&1 &+echo "[exp $NAME] exec+metrics running in background"diff --git a/finetune/scripts/run_exp_4b.sh b/finetune/scripts/run_exp_4b.shnew file mode 100755index 00000000..d925bd02--- /dev/null+++ b/finetune/scripts/run_exp_4b.sh@@ -0,0 +1,13 @@+#!/bin/bash+# like run_exp.sh but merges onto Qwen3.5-4B+set -uo pipefail; GPU=$1; NAME=$2; cd /wekafs/ict/hx_624/llm-ft+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_$NAME; OUT=/wekafs/ict/hx_624/llm-ft/eval/out/qwen35_9b_lora_$NAME+echo "[exp $NAME] $(date) train on GPU(s) $GPU"+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_$NAME.yaml > logs/train_$NAME.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[exp $NAME] TRAIN FAILED"; exit 1; }+echo "[exp $NAME] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|model_name_or_path:.*|model_name_or_path: /wekafs/ict/hx_624/models/Qwen3.5-4B|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $RUN|" -e "s|export_dir:.*|export_dir: $RUN/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[exp $NAME] EXPORT FAILED"; exit 1; }+echo "[exp $NAME] $(date) merged"+GPUS=$GPU eval/run_eval_vllm.sh $RUN/merged $OUT --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1+echo "[exp $NAME] $(date) eval done"; cat $OUT/summary.json; rm -rf $RUN/merged; echo "[exp $NAME] DONE"diff --git a/finetune/scripts/run_exp_qwen3.sh b/finetune/scripts/run_exp_qwen3.shnew file mode 100755index 00000000..7eaf8223--- /dev/null+++ b/finetune/scripts/run_exp_qwen3.sh@@ -0,0 +1,13 @@+#!/bin/bash+# like run_exp.sh but merges onto Qwen3.5-4B+set -uo pipefail; GPU=$1; NAME=$2; cd /wekafs/ict/hx_624/llm-ft+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_$NAME; OUT=/wekafs/ict/hx_624/llm-ft/eval/out/qwen35_9b_lora_$NAME+echo "[exp $NAME] $(date) train on GPU(s) $GPU"+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_$NAME.yaml > logs/train_$NAME.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[exp $NAME] TRAIN FAILED"; exit 1; }+echo "[exp $NAME] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|model_name_or_path:.*|model_name_or_path: /wekafs/ict/hx_624/models/Qwen3-8B|" -e "s|adapter_name_or_path:.*|adapter_name_or_path: $RUN|" -e "s|export_dir:.*|export_dir: $RUN/merged|" configs/lf/export_lora_template.yaml | sed "s/^template: qwen3_5/template: qwen3/" > configs/lf/export_$NAME.yaml+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[exp $NAME] EXPORT FAILED"; exit 1; }+echo "[exp $NAME] $(date) merged"+GPUS=$GPU eval/run_eval_vllm.sh $RUN/merged $OUT --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1+echo "[exp $NAME] $(date) eval done"; cat $OUT/summary.json; rm -rf $RUN/merged; echo "[exp $NAME] DONE"diff --git a/finetune/scripts/run_full_3gpu.sh b/finetune/scripts/run_full_3gpu.shnew file mode 100755index 00000000..dad87edd--- /dev/null+++ b/finetune/scripts/run_full_3gpu.sh@@ -0,0 +1,15 @@+#!/bin/bash+# wait for self-train sampling to finish (GPU0 free), stop the self-train script (its training would collide), then full FT on GPUs 0,1,3 (ZeRO-3, no offload), then eval.+cd /wekafs/ict/hx_624/llm-ft+until grep -q "\[self\] .*sampling done" logs/self_train_round1.log 2>/dev/null; do sleep 20; done+for p in $(pgrep -u hx_624 -f "scripts/self_train_round.s[h]"); do kill $p; done; echo "[full3] self-train script stopped after sampling (exec/build/train to be run manually)"+sleep 20+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1,3 | sort -n | tail -1)" -lt 10000 ]; do sleep 15; done+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v2_3gpu+echo "[full3] $(date) start full FT on GPUs 0,1,3 (ZeRO-3 no offload, grad_accum=3)"+GPUS=0,1,3 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft_3gpu.yaml > logs/train_full_v2_3gpu.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[full3] TRAIN FAILED"; grep -E "Error|error" logs/train_full_v2_3gpu.log | tail -5 | cut -c1-200; exit 1; }+echo "[full3] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+for f in preprocessor_config.json video_preprocessor_config.json vocab.json merges.txt; do [ -f $RUN/$f ] || cp /wekafs/ict/hx_624/models/Qwen3.5-9B/$f $RUN/; done+GPUS=1 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v2_3gpu --no_think --max_new_tokens 6144 > logs/eval_full_v2_3gpu.log 2>&1+echo "[full3] $(date) eval done"; cat eval/out/qwen35_9b_full_v2_3gpu/summary.json; echo FULL3_DONEdiff --git a/finetune/scripts/run_full_ft.sh b/finetune/scripts/run_full_ft.shnew file mode 100755index 00000000..8bb43da6--- /dev/null+++ b/finetune/scripts/run_full_ft.sh@@ -0,0 +1,13 @@+#!/bin/bash+# Full-parameter SFT (ZeRO-3) on GPUs 0,1,2 after all LoRA queues finish, then eval.+cd /wekafs/ict/hx_624/llm-ft+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "QUEUE_DONE gpu=2" logs/queue_gpu2.log 2>/dev/null; do sleep 60; done+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1,2 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1+echo "[full] $(date) start full FT on GPUs 0,1,2"+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml > logs/train_full_v1.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; exit 1; }+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+if [ -f eval/.vllm_ok ]; then GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1+else GPUS=0,1,2 eval/run_eval.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1; fi+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONEdiff --git a/finetune/scripts/run_full_ft2.sh b/finetune/scripts/run_full_ft2.shnew file mode 100755index 00000000..6ee08257--- /dev/null+++ b/finetune/scripts/run_full_ft2.sh@@ -0,0 +1,13 @@+#!/bin/bash+# Full-parameter SFT (ZeRO-3) on GPUs 0,1,2 after all LoRA queues finish, then eval.+cd /wekafs/ict/hx_624/llm-ft+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "QUEUE_DONE gpu=2" logs/exp_v3_bio_retry.log 2>/dev/null; do sleep 60; done+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1,2 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1+echo "[full] $(date) start full FT on GPUs 0,1,2"+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml > logs/train_full_v1.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; exit 1; }+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+if [ -f eval/.vllm_ok ]; then GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1+else GPUS=0,1,2 eval/run_eval.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1; fi+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONEdiff --git a/finetune/scripts/run_full_ft3.sh b/finetune/scripts/run_full_ft3.shnew file mode 100755index 00000000..b9fc1a6d--- /dev/null+++ b/finetune/scripts/run_full_ft3.sh@@ -0,0 +1,13 @@+#!/bin/bash+# Full-parameter SFT (ZeRO-3) on GPUs 0,1,2 after all LoRA queues finish, then eval.+cd /wekafs/ict/hx_624/llm-ft+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "QUEUE_DONE gpu=2" logs/exp_v3_bio_retry.log 2>/dev/null && grep -q "CKPT_SWEEP_DONE v7_ep4" logs/ckpt_sweep_main.log 2>/dev/null; do sleep 60; done+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1,2 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1+echo "[full] $(date) start full FT on GPUs 0,1,2"+GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml > logs/train_full_v1.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; exit 1; }+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+if [ -f eval/.vllm_ok ]; then GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1+else GPUS=0,1,2 eval/run_eval.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --batch_size 48 --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1; fi+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONEdiff --git a/finetune/scripts/run_full_ft4.sh b/finetune/scripts/run_full_ft4.shnew file mode 100755index 00000000..f9c4bcec--- /dev/null+++ b/finetune/scripts/run_full_ft4.sh@@ -0,0 +1,19 @@+#!/bin/bash+# Full-parameter SFT after LoRA queues + ckpt sweeps; uses GPUs 0,1,2 (ZeRO-3) if GPU 2 is free, else GPUs 0,1 with ZeRO-3 + CPU offload. Then eval, then the postponed v2b LoRA on GPU 0.+cd /wekafs/ict/hx_624/llm-ft+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "CKPT_SWEEP_DONE v7_ep4" logs/ckpt_sweep_main.log 2>/dev/null; do sleep 60; done+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1+if [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 2)" -lt 10000 ]; then+ echo "[full] $(date) GPU 2 free -> 3 GPUs, ZeRO-3 (no offload), grad_accum=3"+ GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml gradient_accumulation_steps=3 > logs/train_full_v1.log 2>&1+else+ echo "[full] $(date) GPU 2 busy (other user) -> 2 GPUs, ZeRO-3 + CPU optimizer offload, grad_accum=4"+ GPUS=0,1 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml deepspeed=/wekafs/ict/hx_624/llm-ft/configs/lf/ds_z3_offload.json gradient_accumulation_steps=4 > logs/train_full_v1.log 2>&1+fi+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; tail -30 logs/train_full_v1.log | cut -c1-200; exit 1; }+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONE+# postponed ablation+scripts/run_exp.sh 0 v2b_indist16k 2>&1 | tee -a logs/exp_v2b_indist16k.logdiff --git a/finetune/scripts/run_full_ft5.sh b/finetune/scripts/run_full_ft5.shnew file mode 100755index 00000000..e98fa64c--- /dev/null+++ b/finetune/scripts/run_full_ft5.sh@@ -0,0 +1,17 @@+#!/bin/bash+# Full-parameter SFT after LoRA queues + ckpt sweeps; uses GPUs 0,1,2 (ZeRO-3) if GPU 2 is free, else GPUs 0,1 with ZeRO-3 + CPU offload. Then eval, then the postponed v2b LoRA on GPU 0.+cd /wekafs/ict/hx_624/llm-ft+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null && grep -q "QUEUE_DONE gpu=1" logs/queue_gpu1.log 2>/dev/null && grep -q "CKPT_SWEEP_DONE v7_ep4" logs/ckpt_sweep_main.log 2>/dev/null && grep -q "CKPT_SWEEP_DONE v3_bio" logs/ckpt_sweep_v3.log 2>/dev/null; do sleep 60; done+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,1 | sort -n | tail -1)" -lt 10000 ]; do sleep 30; done+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1+if [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 2)" -lt 10000 ]; then+ echo "[full] $(date) GPU 2 free -> 3 GPUs, ZeRO-3 (no offload), grad_accum=3"+ GPUS=0,1,2 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml gradient_accumulation_steps=3 > logs/train_full_v1.log 2>&1+else+ echo "[full] $(date) GPU 2 busy (other user) -> 2 GPUs, ZeRO-3 + CPU optimizer offload, grad_accum=4"+ GPUS=0,1 scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml deepspeed=/wekafs/ict/hx_624/llm-ft/configs/lf/ds_z3_offload.json gradient_accumulation_steps=4 > logs/train_full_v1.log 2>&1+fi+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; tail -30 logs/train_full_v1.log | cut -c1-200; exit 1; }+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+GPUS=0 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONEdiff --git a/finetune/scripts/run_full_ft6.sh b/finetune/scripts/run_full_ft6.shnew file mode 100755index 00000000..c02a36ea--- /dev/null+++ b/finetune/scripts/run_full_ft6.sh@@ -0,0 +1,22 @@+#!/bin/bash+# Full-parameter SFT: start as soon as the GPU-0 queue is done; use whichever of GPUs 0,1,2 are free (3 -> ZeRO-3, 2 -> ZeRO-3 + CPU offload). Then eval.+cd /wekafs/ict/hx_624/llm-ft+until grep -q "QUEUE_DONE gpu=0" logs/queue_gpu0.log 2>/dev/null; do sleep 60; done+while true; do+ FREE=$(for g in 0 1 2; do [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i $g)" -lt 10000 ] && echo -n "$g,"; done | sed 's/,$//')+ N=$(echo $FREE | tr ',' '\n' | grep -c .)+ [ "$N" -ge 2 ] && break; sleep 30+done+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1+if [ "$N" -ge 3 ]; then+ echo "[full] $(date) GPUs $FREE -> ZeRO-3 (no offload), grad_accum=3"+ GPUS=$FREE scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml gradient_accumulation_steps=3 > logs/train_full_v1.log 2>&1+else+ echo "[full] $(date) GPUs $FREE -> ZeRO-3 + CPU optimizer offload, grad_accum=4"+ GPUS=$FREE scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml deepspeed=/wekafs/ict/hx_624/llm-ft/configs/lf/ds_z3_offload.json gradient_accumulation_steps=4 > logs/train_full_v1.log 2>&1+fi+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; grep -E "Error|error" logs/train_full_v1.log | tail -5 | cut -c1-200; exit 1; }+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+G1=$(echo $FREE | cut -d, -f1)+GPUS=$G1 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONEdiff --git a/finetune/scripts/run_full_ft7.sh b/finetune/scripts/run_full_ft7.shnew file mode 100755index 00000000..0e9ca0bf--- /dev/null+++ b/finetune/scripts/run_full_ft7.sh@@ -0,0 +1,22 @@+#!/bin/bash+# Full-parameter SFT: start as soon as the GPU-0 queue is done; use whichever of GPUs 0,1,2 are free (3 -> ZeRO-3, 2 -> ZeRO-3 + CPU offload). Then eval.+cd /wekafs/ict/hx_624/llm-ft+until grep -q "\[exp v2b_indist16k\] .*gen done" logs/exp_v2b_indist16k.log 2>/dev/null && grep -q "\[exp v8_r16\] .*gen done" logs/exp_v8_r16.log 2>/dev/null; do sleep 30; done; sleep 30+while true; do+ FREE=$(for g in 0 1 2; do [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i $g)" -lt 10000 ] && echo -n "$g,"; done | sed 's/,$//')+ N=$(echo $FREE | tr ',' '\n' | grep -c .)+ [ "$N" -ge 2 ] && break; sleep 30+done+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_full_v1+if [ "$N" -ge 3 ]; then+ echo "[full] $(date) GPUs $FREE -> ZeRO-3 (no offload), grad_accum=3"+ GPUS=$FREE scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml gradient_accumulation_steps=3 > logs/train_full_v1.log 2>&1+else+ echo "[full] $(date) GPUs $FREE -> ZeRO-3 + CPU optimizer offload, grad_accum=4"+ GPUS=$FREE scripts/lf_train.sh configs/lf/qwen35_9b_full_sft.yaml deepspeed=/wekafs/ict/hx_624/llm-ft/configs/lf/ds_z3_offload.json gradient_accumulation_steps=4 > logs/train_full_v1.log 2>&1+fi+[ -f $RUN/train_results.json ] || { echo "[full] TRAIN FAILED"; grep -E "Error|error" logs/train_full_v1.log | tail -5 | cut -c1-200; exit 1; }+echo "[full] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+G1=$(echo $FREE | cut -d, -f1)+GPUS=$G1 eval/run_eval_vllm.sh $RUN eval/out/qwen35_9b_full_v1 --no_think --max_new_tokens 6144 > logs/eval_full_v1.log 2>&1+echo "[full] $(date) eval done"; cat eval/out/qwen35_9b_full_v1/summary.json; echo FULL_FT_DONEdiff --git a/finetune/scripts/run_md_exp.sh b/finetune/scripts/run_md_exp.shnew file mode 100755index 00000000..e0c8f8a8--- /dev/null+++ b/finetune/scripts/run_md_exp.sh@@ -0,0 +1,16 @@+#!/bin/bash+# run_md_exp.sh <GPU(s)> <NAME>: LoRA train (configs/lf/lora_<NAME>.yaml) -> merge -> 3DCodeBench eval + 4-dialect eval -> rm merged+set -uo pipefail; GPU=$1; NAME=$2; NGPU=$(echo $GPU | tr ',' '\n' | wc -l); ACC=$((8 / NGPU)); cd /wekafs/ict/hx_624/llm-ft+RUN=/wekafs/ict/hx_624/llm-ft/runs/lf_qwen35_9b_lora_$NAME+echo "[md $NAME] $(date) train on GPU(s) $GPU"+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_$NAME.yaml gradient_accumulation_steps=$ACC > logs/train_$NAME.log 2>&1+[ -f $RUN/train_results.json ] || { echo "[md $NAME] TRAIN FAILED"; exit 1; }+echo "[md $NAME] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $RUN|" -e "s|export_dir:.*|export_dir: $RUN/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_$NAME.yaml+scripts/lf_export.sh configs/lf/export_$NAME.yaml > logs/export_$NAME.log 2>&1 || { echo "[md $NAME] EXPORT FAILED"; exit 1; }+G1=$(echo $GPU | cut -d, -f1)+GPUS=$G1 eval/run_eval_vllm.sh $RUN/merged eval/out/qwen35_9b_lora_$NAME --no_think --max_new_tokens 6144 > logs/eval_$NAME.log 2>&1+echo "[md $NAME] $(date) 3DCodeBench: $(python3 -c "import json; s=json.load(open('eval/out/qwen35_9b_lora_$NAME/summary.json')); print(s['exec_ok'], round(s['[email protected]_mean_all(fail=0)'],3))")"+GPUS=$G1 eval/run_dialect_eval.sh $RUN/merged $NAME > logs/mdeval_$NAME.log 2>&1+for D in cadquery openscad glsl blender; do echo "[md $NAME] $D: $(cat eval/out/md_${NAME}_$D/summary.json 2>/dev/null | tr -d '\n ' | cut -c1-200)"; done+rm -rf $RUN/merged; echo "[md $NAME] DONE"diff --git a/finetune/scripts/self_train_finish.sh b/finetune/scripts/self_train_finish.shnew file mode 100755index 00000000..2d24b1cd--- /dev/null+++ b/finetune/scripts/self_train_finish.sh@@ -0,0 +1,39 @@+#!/bin/bash+# Finish the self-training round after its script was stopped post-sampling: exec (CPU) -> build data -> (after 3-GPU full FT) train on GPUs 0,3 -> eval. Then re-run Qwen3-8B on GPU 1.+cd /wekafs/ict/hx_624/llm-ft; W=data/selftrain_round1; K=2+until grep -q "self-train script stopped" logs/exp_full_v2_3gpu.log 2>/dev/null; do sleep 30; done+source /wekafs/ict/hx_624/llm-ft/env.sh+for i in $(seq 1 $K); do [ -f $W/gen_s$i/exec_results.jsonl ] || python eval/run_bench.py --gen_dir $W/gen_s$i --workers 48 --timeout 120 > $W/gen_s$i/exec.log 2>&1; grep "status counts" $W/gen_s$i/exec.log; done+echo "[self] $(date) exec done"+python - <<PY+import json, hashlib, os, random, shutil+random.seed(0); W="$W"; K=$K; SRC="data/sft_v1"+prompts={json.loads(l)["task"]: json.loads(l) for l in open(f"{W}/prompts.jsonl")}+base=[json.loads(l) for l in open(f"{SRC}/train.jsonl")]+seen={hashlib.sha1(r["messages"][-1]["content"].encode()).hexdigest() for r in base}+new=[]; n_ok=0+for i in range(1,K+1):+ for l in open(f"{W}/gen_s{i}/exec_results.jsonl"):+ r=json.loads(l)+ if r["status"]!="OK": continue+ n_ok+=1+ code=open(f"{W}/gen_s{i}/{r['task']}/code.py").read().strip()+ if code.count("\n")<4: continue+ h=hashlib.sha1(("```python\n"+code+"\n```").encode()).hexdigest()+ if h in seen: continue+ seen.add(h); p=prompts[r["task"]]+ new.append({"messages": p["messages"]+[{"role":"assistant","content":"```python\n"+code+"\n```"}], "source":"selftrain", "id":r["task"], "prompt_kind":"self"})+print(f"verified OK generations: {n_ok}, new unique samples: {len(new)}")+out="data/sft_v11_selftrain"; os.makedirs(out, exist_ok=True)+allrows=base+new; random.shuffle(allrows)+with open(f"{out}/train.jsonl","w") as f:+ for r in allrows: f.write(json.dumps(r, ensure_ascii=False)+"\n")+shutil.copy(f"{SRC}/val.jsonl", f"{out}/val.jsonl")+json.dump({"n_train":len(allrows),"n_new":len(new),"n_ok_generations":n_ok}, open(f"{out}/stats.json","w"), indent=1)+PY+python scripts/to_llamafactory.py --data_dir data/sft_v11_selftrain --name v11_selftrain >/dev/null && python scripts/make_lora_cfg.py v11_selftrain v11_selftrain >/dev/null+cat data/sft_v11_selftrain/stats.json; echo "[self] $(date) data built"+until grep -q "FULL3_DONE\|TRAIN FAILED" logs/exp_full_v2_3gpu.log 2>/dev/null; do sleep 30; done+until [ "$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i 0,3 | sort -n | tail -1)" -lt 10000 ]; do sleep 15; done+scripts/run_exp.sh 0,3 v11_selftrain 2>&1 | tee -a logs/exp_v11_selftrain.log+echo "[self] DONE"diff --git a/finetune/scripts/self_train_round.sh b/finetune/scripts/self_train_round.shnew file mode 100755index 00000000..d8da9ff5--- /dev/null+++ b/finetune/scripts/self_train_round.sh@@ -0,0 +1,68 @@+#!/bin/bash+# One round of execution-verified self-training:+# LoRA-v1 merged model -> sample K completions (T) for every training prompt -> run all in Blender -> keep OK -> v1 data + verified samples -> LoRA -> eval+# usage: GPUS=0 scripts/self_train_round.sh <K> <T> (default K=2, T=0.8)+set -uo pipefail+K=${1:-2}; T=${2:-0.8}; GPU=${GPUS:-0}+cd /wekafs/ict/hx_624/llm-ft+NAME=v11_selftrain; SRC=data/sft_v1; WORK=data/selftrain_round1; mkdir -p $WORK+echo "[self] $(date) start (K=$K T=$T GPU=$GPU)"+# 0) prompts file from train.jsonl (system+user only)+source /wekafs/ict/hx_624/llm-ft/env.sh+python - <<PY+import json+rows=[json.loads(l) for l in open("$SRC/train.jsonl")]+with open("$WORK/prompts.jsonl","w") as f:+ for i,r in enumerate(rows):+ f.write(json.dumps({"task": f"t{i:05d}", "messages": r["messages"][:2], "source": r.get("source")})+"\n")+print("prompts:", len(rows))+PY+# 1) merged LoRA-v1+[ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_self.log 2>&1 || { echo "[self] EXPORT FAILED"; exit 1; }+# 2) sample K completions+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU+for i in $(seq 1 $K); do+ python eval/generate_vllm.py --model runs/lf_qwen35_9b_lora_v1/merged --prompts $WORK/prompts.jsonl --out $WORK/gen_s$i --tp 1 --no_think --max_new_tokens 4096 --temperature $T --seed $((100+i)) 2>&1 | grep -E "gen-vllm|Error|Traceback" | tail -3+done+rm -rf runs/lf_qwen35_9b_lora_v1/merged+echo "[self] $(date) sampling done"+# 3) execute everything in Blender (CPU)+conda activate llmft+for i in $(seq 1 $K); do python eval/run_bench.py --gen_dir $WORK/gen_s$i --workers 48 --timeout 120 > $WORK/gen_s$i/exec.log 2>&1; grep "status counts" $WORK/gen_s$i/exec.log; done+echo "[self] $(date) exec done"+# 4) build v11 = v1 + verified samples (dedup by code hash); val = v1 val+python - <<PY+import json, hashlib, os, random+random.seed(0)+W="$WORK"; K=$K+prompts={json.loads(l)["task"]: json.loads(l) for l in open(f"{W}/prompts.jsonl")}+base=[json.loads(l) for l in open("$SRC/train.jsonl")]+seen={hashlib.sha1(r["messages"][-1]["content"].encode()).hexdigest() for r in base}+new=[]; n_ok=0+for i in range(1,K+1):+ for l in open(f"{W}/gen_s{i}/exec_results.jsonl"):+ r=json.loads(l)+ if r["status"]!="OK": continue+ n_ok+=1+ code=open(f"{W}/gen_s{i}/{r['task']}/code.py").read().strip()+ if code.count("\n")<4: continue+ h=hashlib.sha1(("```python\n"+code+"\n```").encode()).hexdigest()+ if h in seen: continue+ seen.add(h)+ p=prompts[r["task"]]+ new.append({"messages": p["messages"]+[{"role":"assistant","content":"```python\n"+code+"\n```"}], "source":"selftrain", "id":r["task"], "prompt_kind":"self"})+print(f"verified OK generations: {n_ok}, new unique samples: {len(new)}")+out="data/sft_v11_selftrain"; os.makedirs(out, exist_ok=True)+allrows=base+new; random.shuffle(allrows)+with open(f"{out}/train.jsonl","w") as f:+ for r in allrows: f.write(json.dumps(r, ensure_ascii=False)+"\n")+import shutil; shutil.copy("$SRC/val.jsonl", f"{out}/val.jsonl")+json.dump({"n_train":len(allrows),"n_new":len(new),"n_ok_generations":n_ok}, open(f"{out}/stats.json","w"), indent=1)+PY+python scripts/to_llamafactory.py --data_dir data/sft_v11_selftrain --name v11_selftrain >/dev/null && python scripts/make_lora_cfg.py $NAME v11_selftrain >/dev/null+cat data/sft_v11_selftrain/stats.json+echo "[self] $(date) data built -> train"+# 5) train + eval (run_exp)+scripts/run_exp.sh $GPU $NAME 2>&1 | tee -a logs/exp_$NAME.log+echo "[self] DONE"diff --git a/finetune/scripts/self_train_round2.sh b/finetune/scripts/self_train_round2.shnew file mode 100755index 00000000..cfa58f2c--- /dev/null+++ b/finetune/scripts/self_train_round2.sh@@ -0,0 +1,59 @@+#!/bin/bash+# Round 2 (novel prompts): LoRA-v1 samples K completions for 9.9k articraft/deepcad-derived Blender prompts on 4 GPUs -> Blender filter -> v13 = v1 + verified -> LoRA (4-GPU DDP) -> eval+set -uo pipefail; K=${1:-2}; T=${2:-0.8}; W=data/selftrain_round2; NAME=v13_boot; cd /wekafs/ict/hx_624/llm-ft+echo "[r2] $(date) start K=$K T=$T"+[ -d runs/lf_qwen35_9b_lora_v1/merged ] || scripts/lf_export.sh configs/lf/export_v1.yaml > logs/export_v1_r2.log 2>&1 || { echo "[r2] EXPORT FAILED"; exit 1; }+source /wekafs/ict/hx_624/llm-ft/env.sh+python - <<PY+import json+rows=[json.loads(l) for l in open("$W/prompts.jsonl")]+for g in range(4):+ with open(f"$W/prompts_g{g}.jsonl","w") as f:+ for r in rows[g::4]: f.write(json.dumps(r)+"\n")+print("sharded", len(rows))+PY+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING+for i in $(seq 1 $K); do+ for g in 0 1 2 3; do+ CUDA_VISIBLE_DEVICES=$g python eval/generate_vllm.py --model runs/lf_qwen35_9b_lora_v1/merged --prompts $W/prompts_g$g.jsonl --out $W/gen_s${i}_g$g --tp 1 --no_think --max_new_tokens 4096 --temperature $T --seed $((200+i*10+g)) > $W/gen_s${i}_g$g.log 2>&1 &+ done; wait+ for g in 0 1 2 3; do grep -E "gen-vllm|Error" $W/gen_s${i}_g$g.log | tail -1; done+done+rm -rf runs/lf_qwen35_9b_lora_v1/merged; echo "[r2] $(date) sampling done"+conda activate llmft+for d in $W/gen_s*_g*/; do python eval/run_bench.py --gen_dir $d --workers 56 --timeout 120 > $d/exec.log 2>&1; grep "status counts" $d/exec.log; done+echo "[r2] $(date) exec done"+python - <<PY+import json, hashlib, os, random, shutil, glob+random.seed(0); W="$W"; SRC="data/sft_v1"+prompts={}+for f in glob.glob(f"{W}/prompts_g*.jsonl"):+ for l in open(f): r=json.loads(l); prompts[r["task"]]=r+base=[json.loads(l) for l in open(f"{SRC}/train.jsonl")]+seen={hashlib.sha1(r["messages"][-1]["content"].encode()).hexdigest() for r in base}+new=[]; n_ok=0; per_src={}+for d in sorted(glob.glob(f"{W}/gen_s*_g*/")):+ if not os.path.exists(d+"exec_results.jsonl"): continue+ for l in open(d+"exec_results.jsonl"):+ r=json.loads(l)+ if r["status"]!="OK": continue+ n_ok+=1+ code=open(f"{d}/{r['task']}/code.py").read().strip()+ if code.count("\n")<4: continue+ h=hashlib.sha1(("```python\n"+code+"\n```").encode()).hexdigest()+ if h in seen: continue+ seen.add(h); p=prompts[r["task"]]; per_src[p["src"]]=per_src.get(p["src"],0)+1+ new.append({"messages": p["messages"]+[{"role":"assistant","content":"```python\n"+code+"\n```"}], "source":"boot_"+p["src"], "id":r["task"], "prompt_kind":"boot"})+print(f"verified OK generations: {n_ok}, new unique samples: {len(new)}, by source: {per_src}")+out="data/sft_v13_boot"; os.makedirs(out, exist_ok=True)+allrows=base+new; random.shuffle(allrows)+with open(f"{out}/train.jsonl","w") as f:+ for r in allrows: f.write(json.dumps(r, ensure_ascii=False)+"\n")+shutil.copy(f"{SRC}/val.jsonl", f"{out}/val.jsonl")+json.dump({"n_train":len(allrows),"n_new":len(new),"n_ok_generations":n_ok,"by_source":per_src}, open(f"{out}/stats.json","w"), indent=1)+PY+python scripts/to_llamafactory.py --data_dir data/sft_v13_boot --name v13_boot >/dev/null && python scripts/make_lora_cfg.py $NAME v13_boot >/dev/null+cat data/sft_v13_boot/stats.json; echo "[r2] $(date) data built -> train on 4 GPUs"+scripts/run_exp.sh 0,1,2,3 $NAME 2>&1 | tee -a logs/exp_$NAME.log+echo "[r2] DONE"diff --git a/finetune/scripts/strip_thingi_fallback.py b/finetune/scripts/strip_thingi_fallback.pynew file mode 100644index 00000000..c06155ee--- /dev/null+++ b/finetune/scripts/strip_thingi_fallback.py@@ -0,0 +1,26 @@+"""Strip the embedded reference-mesh fallback from thingiverse-openscad-blender-5-0 'native_csg_generated' scripts.+Writes <out>/<sample>/code.py (+ copies captions.json) for samples whose stripped script still parses."""+import ast, glob, json, os, re, shutil, sys+SRC = "/wekafs/ict/hx_624/data/thingiverse_5_0"; OUT = "/wekafs/ict/hx_624/data/thingiverse_5_0_native"+os.makedirs(OUT, exist_ok=True); kept = dropped = 0+for cp in sorted(glob.glob(f"{SRC}/*/code.py")):+ d = os.path.dirname(cp); rep = os.path.join(d, "reports/translation_report.json")+ st = json.load(open(rep)).get("status", "") if os.path.exists(rep) else ""+ if st != "native_csg_generated": continue+ code = open(cp).read()+ # 1) remove the fallback function definition+ code2 = re.sub(r"\ndef load_mesh_fallback\(\):.*?(?=\n(?:def |if __name__|[A-Za-z_])|\Z)", "\n", code, flags=re.S)+ # 2) rewrite 'try: <native> except ...: load_mesh_fallback()' -> just the native call (common pattern)+ code2 = re.sub(r"try:\n(\s+)(.*?)\n\s*except [^\n]*:\n\s+(?:[^\n]*\n)*?\s*(?:obj\s*=\s*)?load_mesh_fallback\(\)[^\n]*\n", lambda m: m.group(1) + m.group(2) + "\n", code2, flags=re.S)+ code2 = re.sub(r"^.*load_mesh_fallback\(\).*$", "", code2, flags=re.M)+ code2 = re.sub(r"^MESH_REFERENCE_ASSET\s*=.*$", "", code2, flags=re.M)+ if "load_mesh_fallback" in code2 or "reference_mesh" in code2:+ dropped += 1; continue+ try: ast.parse(code2)+ except SyntaxError: dropped += 1; continue+ od = os.path.join(OUT, os.path.basename(d)); os.makedirs(od, exist_ok=True)+ open(os.path.join(od, "code.py"), "w").write(code2)+ for f in ["captions.json", "meta.json"]:+ if os.path.exists(os.path.join(d, f)): shutil.copy(os.path.join(d, f), od)+ kept += 1+print(f"kept {kept} dropped {dropped} -> {OUT}")diff --git a/finetune/scripts/threejs_check.sh b/finetune/scripts/threejs_check.shnew file mode 100755index 00000000..ab20f345--- /dev/null+++ b/finetune/scripts/threejs_check.sh@@ -0,0 +1,34 @@+#!/bin/bash+# three.js pipeline: wait Playwright -> reference runner check -> base eval (GPU) -> LoRA (300 samples, cutoff 16k) -> eval -> open the md_big gate+set -uo pipefail; GPU=${GPUS:-2}; cd /wekafs/ict/hx_624/llm-ft; source /wekafs/ict/hx_624/llm-ft/env.sh; export PLAYWRIGHT_BROWSERS_PATH=/wekafs/ict/hx_624/cache/ms-playwright+until grep -q PW_INSTALL_DONE /wekafs/ict/hx_624/tools/playwright_install.log 2>/dev/null; do sleep 15; done+echo "[3js] $(date) runner check on 6 reference samples"+python - <<'PY'+import json, sys, re; sys.path.insert(0,"eval"); from threejs_runner import run_threejs+rows=[json.loads(l) for l in open("data/multidialect/threejs/test.jsonl")][:6]+ok=0+for i,r in enumerate(rows):+ code=re.findall(r"```html\n(.*?)```", r["messages"][-1]["content"], flags=re.S)[0]+ rep=run_threejs(code, f"eval/out/_3js_refcheck/{i}"); ok+=rep["status"]=="OK"; print(i, rep["status"], (rep["error"] or "")[:120], rep["latency_s"], "s")+print("REFCHECK_OK", ok, "/", len(rows))+PY+echo "[3js] $(date) base eval"+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm; export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=$GPU+python - <<'PY'+import json+with open("eval/out/_3js_prompts.jsonl","w") as f:+ for l in open("data/multidialect/threejs/test.jsonl"):+ r=json.loads(l); f.write(json.dumps({"task": r["id"].replace("/","__"), "messages": r["messages"][:2]})+"\n")+PY+mkdir -p eval/out/md_base_threejs; python eval/generate_vllm.py --model /wekafs/ict/hx_624/models/Qwen3.5-9B --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_base_threejs --tp 1 --no_think --max_new_tokens 12288 2>&1 | grep -E "gen-vllm|Error" | tail -1+conda activate llmft; python eval/threejs_eval.py --gen_dir eval/out/md_base_threejs 2>&1 | tail -2+echo "[3js] $(date) LoRA train (300 samples, cutoff 16k, 3 epochs) on GPU $GPU"+GPUS=$GPU scripts/lf_train.sh configs/lf/lora_md_threejs.yaml > logs/train_md_threejs.log 2>&1+RUN=runs/lf_qwen35_9b_lora_md_threejs; [ -f $RUN/train_results.json ] || { echo "[3js] TRAIN FAILED"; touch logs/.gate_threejs; exit 1; }+echo "[3js] $(date) train done: $(tr -d '\n' < $RUN/train_results.json)"+sed -e "s|adapter_name_or_path:.*|adapter_name_or_path: $(readlink -f $RUN)|" -e "s|export_dir:.*|export_dir: $(readlink -f $RUN)/merged|" configs/lf/export_lora_template.yaml > configs/lf/export_md_threejs.yaml+scripts/lf_export.sh configs/lf/export_md_threejs.yaml > logs/export_md_threejs.log 2>&1 || { echo "[3js] EXPORT FAILED"; touch logs/.gate_threejs; exit 1; }+conda activate vllm; mkdir -p eval/out/md_md_threejs_threejs; python eval/generate_vllm.py --model $RUN/merged --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_md_threejs_threejs --tp 1 --no_think --max_new_tokens 12288 2>&1 | grep -E "gen-vllm|Error" | tail -1+touch logs/.gate_threejs; echo "[3js] gate opened for md_big"+conda activate llmft; python eval/threejs_eval.py --gen_dir eval/out/md_md_threejs_threejs 2>&1 | tail -2+rm -rf $RUN/merged; echo "[3js] DONE"diff --git a/finetune/scripts/threejs_regen32k.sh b/finetune/scripts/threejs_regen32k.shnew file mode 100755index 00000000..8845d6fc--- /dev/null+++ b/finetune/scripts/threejs_regen32k.sh@@ -0,0 +1,13 @@+#!/bin/bash+# re-merge the three.js LoRA (merged dir was cleaned up), re-generate with a 32k budget on GPU 2, evaluate+cd /wekafs/ict/hx_624/llm-ft+RUN=runs/lf_qwen35_9b_lora_md_threejs+[ -f $RUN/merged/config.json ] || scripts/lf_export.sh configs/lf/export_md_threejs.yaml > logs/export_md_threejs_2.log 2>&1 || { echo "[3js32k] EXPORT FAILED"; exit 1; }+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=2 PLAYWRIGHT_BROWSERS_PATH=/wekafs/ict/hx_624/cache/ms-playwright+echo "[3js32k] $(date) gen LoRA 32k"+mkdir -p eval/out/md_md_threejs_threejs_32k+python eval/generate_vllm.py --model $RUN/merged --prompts eval/out/_3js_prompts.jsonl --out eval/out/md_md_threejs_threejs_32k --tp 1 --no_think --max_new_tokens 30000 --max_model_len 32768 2>&1 | grep -E "gen-vllm|Error" | tail -1+conda activate llmft+python eval/threejs_eval.py --gen_dir eval/out/md_md_threejs_threejs_32k 2>&1 | tail -2+echo "[3js32k] done2"diff --git a/finetune/scripts/threejs_t07.sh b/finetune/scripts/threejs_t07.shnew file mode 100755index 00000000..5029b0c9--- /dev/null+++ b/finetune/scripts/threejs_t07.sh@@ -0,0 +1,10 @@+#!/bin/bash+cd /wekafs/ict/hx_624/llm-ft; RUN=runs/lf_qwen35_9b_lora_md_threejs+source /wekafs/ict/hx_624/anaconda3/etc/profile.d/conda.sh; conda activate vllm+export VLLM_CACHE_ROOT=/wekafs/ict/hx_624/cache/vllm HF_HOME=/wekafs/ict/hx_624/cache/huggingface VLLM_LOGGING_LEVEL=WARNING CUDA_VISIBLE_DEVICES=2 PLAYWRIGHT_BROWSERS_PATH=/wekafs/ict/hx_624/cache/ms-playwright+for M in "$RUN/merged md_md_threejs_T07_threejs" "/wekafs/ict/hx_624/models/Qwen3.5-9B md_base_T07_threejs"; do set -- $M+ mkdir -p eval/out/$2; python eval/generate_vllm.py --model $1 --prompts eval/out/_3js_prompts.jsonl --out eval/out/$2 --tp 1 --no_think --max_new_tokens 12288 --temperature 0.7 --seed 1 2>&1 | grep -E "gen-vllm|Error" | tail -1+done+conda activate llmft+for N in md_md_threejs_T07_threejs md_base_T07_threejs; do echo "== $N"; python eval/threejs_eval.py --gen_dir eval/out/$N 2>&1 | tail -2; done+echo "[3jsT07] done"diff --git a/finetune/scripts/to_llamafactory.py b/finetune/scripts/to_llamafactory.pynew file mode 100644index 00000000..6d7248e9--- /dev/null+++ b/finetune/scripts/to_llamafactory.py@@ -0,0 +1,26 @@+"""Convert our messages-jsonl (system/user/assistant) into LLaMA-Factory sharegpt json + dataset_info.json entries."""+import argparse, json, os+ap = argparse.ArgumentParser()+ap.add_argument("--data_dir", default="/wekafs/ict/hx_624/llm-ft/data/sft_v1")+ap.add_argument("--out_dir", default="/wekafs/ict/hx_624/llm-ft/data/lf")+ap.add_argument("--name", default="blender3d_v1")+a = ap.parse_args()+os.makedirs(a.out_dir, exist_ok=True)+info_path = os.path.join(a.out_dir, "dataset_info.json")+info = json.load(open(info_path)) if os.path.exists(info_path) else {}+for split in ["train", "val"]:+ rows = []+ for l in open(os.path.join(a.data_dir, f"{split}.jsonl")):+ m = json.loads(l)["messages"]+ sys_msg = next((x["content"] for x in m if x["role"] == "system"), None)+ conv = []+ for x in m:+ if x["role"] == "user": conv.append({"from": "human", "value": x["content"]})+ elif x["role"] == "assistant": conv.append({"from": "gpt", "value": x["content"]})+ rows.append({"conversations": conv, "system": sys_msg})+ fn = f"{a.name}_{split}.json"+ json.dump(rows, open(os.path.join(a.out_dir, fn), "w"), ensure_ascii=False)+ info[f"{a.name}_{split}"] = {"file_name": fn, "formatting": "sharegpt", "columns": {"messages": "conversations", "system": "system"}}+ print(split, len(rows), "->", fn)+json.dump(info, open(info_path, "w"), indent=2)+print("dataset_info:", list(info.keys()))diff --git a/finetune/scripts/train_sft.py b/finetune/scripts/train_sft.pynew file mode 100644index 00000000..f13e3b32--- /dev/null+++ b/finetune/scripts/train_sft.py@@ -0,0 +1,129 @@+"""SFT (full or LoRA) of a causal LM on chat-format jsonl using TRL SFTTrainer.++Data: jsonl rows with {"messages":[system,user,assistant]} -> converted to conversational prompt/completion+so that the loss is computed on the assistant completion only.+"""+import argparse, json, os, sys, time+import torch+from datasets import load_dataset+from transformers import AutoTokenizer, AutoModelForCausalLM+from trl import SFTConfig, SFTTrainer++def parse():+ ap = argparse.ArgumentParser()+ ap.add_argument("--model", required=True)+ ap.add_argument("--data_dir", required=True)+ ap.add_argument("--output_dir", required=True)+ ap.add_argument("--max_length", type=int, default=8192)+ ap.add_argument("--epochs", type=float, default=2.0)+ ap.add_argument("--lr", type=float, default=1e-5)+ ap.add_argument("--warmup_ratio", type=float, default=0.03)+ ap.add_argument("--per_device_bs", type=int, default=1)+ ap.add_argument("--grad_accum", type=int, default=8)+ ap.add_argument("--eval_bs", type=int, default=2)+ ap.add_argument("--lora", action="store_true")+ ap.add_argument("--lora_r", type=int, default=64)+ ap.add_argument("--lora_alpha", type=int, default=128)+ ap.add_argument("--lora_dropout", type=float, default=0.05)+ ap.add_argument("--packing", action="store_true")+ ap.add_argument("--no_liger", action="store_true")+ ap.add_argument("--attn", default="auto", help="auto|flash_attention_2|sdpa|eager")+ ap.add_argument("--grad_ckpt", action="store_true", default=True)+ ap.add_argument("--deepspeed", default=None)+ ap.add_argument("--save_steps", type=int, default=200)+ ap.add_argument("--eval_steps", type=int, default=100)+ ap.add_argument("--logging_steps", type=int, default=5)+ ap.add_argument("--seed", type=int, default=42)+ ap.add_argument("--max_train_samples", type=int, default=None)+ ap.add_argument("--report_to", default="tensorboard")+ ap.add_argument("--run_name", default=None)+ ap.add_argument("--weight_decay", type=float, default=0.0)+ ap.add_argument("--neftune", type=float, default=None)+ return ap.parse_args()++def to_prompt_completion(ex):+ msgs = ex["messages"]+ return {"prompt": msgs[:-1], "completion": [msgs[-1]]}++def main():+ a = parse()+ torch.manual_seed(a.seed)+ attn = a.attn+ if attn == "auto":+ try:+ import flash_attn # noqa+ attn = "flash_attention_2"+ except Exception:+ attn = "sdpa"+ tok = AutoTokenizer.from_pretrained(a.model)+ if tok.pad_token is None:+ tok.pad_token = tok.eos_token++ ds = load_dataset("json", data_files={"train": f"{a.data_dir}/train.jsonl", "val": f"{a.data_dir}/val.jsonl"})+ keep = ["messages"]+ ds = ds.map(to_prompt_completion, remove_columns=[c for c in ds["train"].column_names if c not in keep])+ ds = ds.remove_columns(["messages"])+ if a.max_train_samples:+ ds["train"] = ds["train"].select(range(min(a.max_train_samples, len(ds["train"]))))+ print(f"[data] train={len(ds['train'])} val={len(ds['val'])} attn={attn}", flush=True)++ model_kwargs = dict(dtype=torch.bfloat16, attn_implementation=attn)+ model = AutoModelForCausalLM.from_pretrained(a.model, **model_kwargs)+ model.config.use_cache = False++ peft_config = None+ if a.lora:+ from peft import LoraConfig+ peft_config = LoraConfig(r=a.lora_r, lora_alpha=a.lora_alpha, lora_dropout=a.lora_dropout, bias="none",+ task_type="CAUSAL_LM",+ target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"])++ cfg_kwargs = dict(+ output_dir=a.output_dir, run_name=a.run_name or os.path.basename(a.output_dir.rstrip("/")),+ num_train_epochs=a.epochs, learning_rate=a.lr, lr_scheduler_type="cosine", warmup_ratio=a.warmup_ratio,+ weight_decay=a.weight_decay,+ per_device_train_batch_size=a.per_device_bs, per_device_eval_batch_size=a.eval_bs,+ gradient_accumulation_steps=a.grad_accum,+ bf16=True, tf32=True,+ gradient_checkpointing=a.grad_ckpt, gradient_checkpointing_kwargs={"use_reentrant": False},+ logging_steps=a.logging_steps, eval_strategy="steps", eval_steps=a.eval_steps,+ save_strategy="steps", save_steps=a.save_steps, save_total_limit=2,+ report_to=a.report_to, seed=a.seed,+ max_length=a.max_length, packing=a.packing,+ completion_only_loss=True,+ use_liger_kernel=not a.no_liger,+ dataloader_num_workers=2, remove_unused_columns=True,+ deepspeed=a.deepspeed,+ ddp_find_unused_parameters=False,+ save_only_model=True,+ )+ if a.neftune: cfg_kwargs["neftune_noise_alpha"] = a.neftune+ cfg = SFTConfig(**cfg_kwargs)++ trainer = SFTTrainer(model=model, args=cfg, train_dataset=ds["train"], eval_dataset=ds["val"],+ processing_class=tok, peft_config=peft_config)+ if trainer.accelerator.is_main_process:+ n_tr = sum(p.numel() for p in trainer.model.parameters() if p.requires_grad)+ n_all = sum(p.numel() for p in trainer.model.parameters())+ print(f"[model] trainable={n_tr/1e6:.1f}M / total={n_all/1e6:.1f}M ({100*n_tr/max(n_all,1):.2f}%)", flush=True)+ # show one tokenized example length+ ex = trainer.train_dataset[0]+ print("[data] example keys:", list(ex.keys()), "| input_ids len:", len(ex.get("input_ids", [])), flush=True)+ t0 = time.time()+ trainer.train()+ if trainer.accelerator.is_main_process:+ print(f"[done] train time {(time.time()-t0)/60:.1f} min", flush=True)+ trainer.save_model(os.path.join(a.output_dir, "final"))+ tok.save_pretrained(os.path.join(a.output_dir, "final"))+ if a.lora and trainer.accelerator.is_main_process:+ # also save a merged full model for easy eval+ try:+ merged = trainer.model.merge_and_unload()+ merged.save_pretrained(os.path.join(a.output_dir, "final_merged"), safe_serialization=True)+ tok.save_pretrained(os.path.join(a.output_dir, "final_merged"))+ print("[done] merged model saved", flush=True)+ except Exception as e:+ print("merge failed:", e)++if __name__ == "__main__":+ main()