Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat: add 2xT4 Kaggle fine-tuning notebook and remove colab workflow notebook
Nandakishor1332269369d1
diff --git a/README.md b/README.mdindex ade5622d..774403e7 100644--- a/README.md+++ b/README.md@@ -175,10 +175,9 @@ triage = agent.predict({"message": "My payment failed twice"}, laya.triage_quest## Fine-Tuning on Single T4 GPU (Google Colab)-Fine-tune Laya on custom domain data or the `LocalLLaMA/typed-decisions` benchmark on a free T4 GPU:+Fine-tune Laya on your custom domain data or commercial datasets on a free T4 GPU:* **Interactive Fine-Tuning Notebook:** [Fine-Tune on Custom Data](https://colab.research.google.com/drive/15d4Yv__KHeHjshVb-6PRTfqVllxih2S3?usp=sharing) ([`notebooks/laya_finetune_colab.ipynb`](notebooks/laya_finetune_colab.ipynb))-* **Workflow Benchmark Fine-Tuning:** [`notebooks/laya_finetune_typed_decisions_colab.ipynb`](notebooks/laya_finetune_typed_decisions_colab.ipynb) — fine-tunes on `LocalLLaMA/typed-decisions` (1,200 cases), evaluates on 400 test cases, and pushes to Hugging Face.---diff --git a/laya/__init__.py b/laya/__init__.pyindex 6b6ae587..4c693eb2 100644--- a/laya/__init__.py+++ b/laya/__init__.py@@ -13,7 +13,7 @@ from .common import (from .email import clean_email_body, email_questions, email_statefrom .presets import guard_questions, moderation_questions, router_questions, triage_questions-__version__ = "0.1.6"+__version__ = "0.1.7"__all__ = ["Agent","RLAgent",diff --git a/laya/agent.py b/laya/agent.pyindex 656c2460..df0738a1 100644--- a/laya/agent.py+++ b/laya/agent.py@@ -97,6 +97,11 @@ class Agent:model_dir = model_id_or_pathif not os.path.exists(model_dir):+ if model_id_or_path.startswith(("/", "./", "../")) or os.path.isabs(model_id_or_path):+ raise FileNotFoundError(+ f"Local model path not found: {model_id_or_path!r}. "+ f"Check that the directory exists and that training saved the model successfully."+ )from huggingface_hub import snapshot_downloadmodel_dir = snapshot_download(model_id_or_path, token=token or os.environ.get("HF_TOKEN"))diff --git a/notebooks/laya_finetune_typed_decisions_colab.ipynb b/notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynbsimilarity index 59%rename from notebooks/laya_finetune_typed_decisions_colab.ipynbrename to notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynbindex b3fed00b..25ad0ad4 100644--- a/notebooks/laya_finetune_typed_decisions_colab.ipynb+++ b/notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb@@ -4,30 +4,26 @@"cell_type": "markdown","metadata": {},"source": [- "# Fine-Tuning Laya on `LocalLLaMA/typed-decisions` & Evaluation (Single T4 GPU)\n",+ "# Fine-Tuning Laya on `LocalLLaMA/typed-decisions` (Kaggle GPU T4 \u00d72 DDP)\n","\n",- "[](https://colab.research.google.com/github/NandhaKishorM/laya/blob/main/notebooks/laya_finetune_typed_decisions_colab.ipynb)\n","[](https://pypi.org/project/laya/)\n","[](https://huggingface.co/convaiinnovations/laya)\n","[](https://huggingface.co/datasets/LocalLLaMA/typed-decisions)\n","\n",- "This notebook fine-tunes **Laya** (`convaiinnovations/laya`) on the **1,200 training cases (6,000 typed decisions)** of the [LocalLLaMA/typed-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) benchmark on a **single free T4 GPU (16 GB)** using **RLCD (Reinforcement Learning for Calibrated Decisions)**.\n",+ "This notebook fine-tunes **Laya** (`convaiinnovations/laya`, 421M params) on the **1,200 training cases (6,000 typed decisions)** of the [LocalLLaMA/typed-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) benchmark using **both NVIDIA T4 GPUs via Distributed Data Parallel (`torchrun --nproc_per_node=2`)**.\n","\n",- "After fine-tuning (~10-15 minutes), it evaluates the model against the **400 test cases (2,000 decisions)** and compares head-to-head with **TypeSafe Jev 1.13.0**, then lets you push the new checkpoint to Hugging Face.\n",- "\n",- "### The 4 Production Workflows:\n",- "1. `agent_trace_observability` (300 train / 100 test)\n",- "2. `customer_service` (300 train / 100 test)\n",- "3. `invoice_processing` (300 train / 100 test)\n",- "4. `security_incidents` (300 train / 100 test)\n"+ "### Setup on Kaggle:\n",+ "- **Settings -> Accelerator:** `GPU T4 x2`\n",+ "- **Settings -> Internet:** `On`\n",+ "- **Output:** `/kaggle/working/laya_finetuned_typed_decisions`\n"]},{"cell_type": "markdown","metadata": {},"source": [- "## 1. Environment & GPU Check\n",- "Verify the T4 GPU is active (`Runtime -> Change runtime type -> T4 GPU`).\n"+ "## 1. Environment & Dual T4 GPU Check\n",+ "Verify both T4 GPUs are detected.\n"]},{@@ -35,10 +31,13 @@"metadata": {},"source": ["!nvidia-smi\n",- "import os, torch\n",- "assert torch.cuda.is_available(), \"No GPU detected! Set Runtime -> Change runtime type -> T4 GPU\"\n",- "props = torch.cuda.get_device_properties(0)\n",- "print(f\"Connected to: {props.name} | Total VRAM: {props.total_memory / 1e9:.1f} GB\")\n",+ "import os, subprocess, torch\n",+ "n_gpu = torch.cuda.device_count()\n",+ "print(f\"CUDA Available: {torch.cuda.is_available()} | Visible GPUs: {n_gpu}\")\n",+ "for i in range(n_gpu):\n",+ " p = torch.cuda.get_device_properties(i)\n",+ " print(f\" GPU {i}: {p.name} ({p.total_memory / 1e9:.1f} GB)\")\n",+ "assert n_gpu == 2, \"Please select Accelerator: GPU T4 x2 in Kaggle settings!\"\n","os.environ[\"PYTORCH_CUDA_ALLOC_CONF\"] = \"expandable_segments:True\"\n"]},@@ -53,7 +52,7 @@"cell_type": "code","metadata": {},"source": [- "!pip install -q -U \"laya>=0.1.6\" \"transformers>=4.48.0\" \"datasets>=3.0.0\" safetensors huggingface_hub pyarrow pandas scipy accelerate\n",+ "!pip install -q -U \"laya>=0.1.6\" \"transformers>=4.48.0\" \"datasets>=3.0.0\" safetensors huggingface_hub pyarrow pandas scipy accelerate tabulate\n","import laya, transformers, datasets, torch\n","print(\"Laya version :\", laya.__version__)\n","print(\"Transformers version:\", transformers.__version__)\n",@@ -64,8 +63,8 @@"cell_type": "markdown","metadata": {},"source": [- "## 3. Load Base Model & Tokenizer\n",- "We load `convaiinnovations/laya` (421M parameters) in FP16 with gradient checkpointing enabled for single-GPU training.\n"+ "## 3. Download & Preprocess Data for DDP\n",+ "We preprocess all 1,200 training cases into tokenized items and save them to disk so both DDP worker ranks can read them.\n"]},{@@ -73,62 +72,23 @@"metadata": {},"source": ["import os, json, torch\n",+ "from datasets import load_dataset\n",+ "from transformers import AutoTokenizer\n","from huggingface_hub import snapshot_download\n","from laya.agent import _fix_tokenizer_config\n",- "from laya.common import build_model, QTYPES, render_options, build_sequence, proper_reward, confidence_from_probs\n",- "from transformers import AutoTokenizer\n",- "from safetensors.torch import load_file\n",+ "from laya.common import build_sequence, render_options, QTYPES\n","\n","MODEL_ID = \"convaiinnovations/laya\"\n",- "print(f\"Downloading base model weights from {MODEL_ID}...\")\n",+ "print(f\"Fetching tokenizer and config from {MODEL_ID}...\")\n","model_dir = snapshot_download(MODEL_ID)\n","_fix_tokenizer_config(model_dir)\n","\n",+ "tok = AutoTokenizer.from_pretrained(os.path.join(model_dir, \"tokenizer\"))\n","with open(os.path.join(model_dir, \"rl_agent_config.json\")) as f:\n"," cfg = json.load(f)\n","\n",- "# Optimize config for single T4 fine-tuning\n",- "cfg[\"gradient_checkpointing\"] = True\n",- "cfg[\"max_tokens_per_batch\"] = 4096\n",- "cfg[\"head_layers\"] = 2\n",- "cfg[\"max_len\"] = 512\n",- "cfg[\"head_max_len\"] = 192\n",- "\n",- "tok = AutoTokenizer.from_pretrained(os.path.join(model_dir, \"tokenizer\"))\n",- "model = build_model(cfg, encoder_dir=os.path.join(model_dir, \"encoder\"))\n",- "\n",- "# Load fine-tuned weights\n",- "weights = load_file(os.path.join(model_dir, \"model.safetensors\"))\n",- "model.load_state_dict(weights, strict=True)\n",- "\n",- "device = torch.device(\"cuda\")\n",- "model.encoder.gradient_checkpointing_enable(gradient_checkpointing_kwargs={\"use_reentrant\": False})\n",- "model.head_checkpointing = True\n",- "model.to(device)\n",- "model.train()\n",- "\n",- "print(f\"Successfully loaded Laya ({sum(p.numel() for p in model.parameters()) / 1e6:.1f}M params) on {device}!\")\n"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## 4. Download & Preprocess `typed-decisions` (Train Split)\n",- "We download all 1,200 training cases and format them with soft gold targets from the teacher.\n"- ]- },- {- "cell_type": "code",- "metadata": {},- "source": [- "from datasets import load_dataset\n",- "import pandas as pd\n",- "import json\n",- "\n",- "print(\"Downloading LocalLLaMA/typed-decisions (split: train)...\")\n",+ "print(\"Downloading LocalLLaMA/typed-decisions (train split)...\")\n","ds_train = load_dataset(\"LocalLLaMA/typed-decisions\", \"all\", split=\"train\")\n",- "print(f\"Loaded {len(ds_train)} training workflow cases.\")\n","\n","def build_training_item(state, q, gold_q):\n"," t = q[\"type\"]\n",@@ -142,7 +102,6 @@" n_levels = len(crit) if isinstance(crit, list) else 4\n"," target = [gold_q[\"probabilities\"].get(str(i), 0.0) for i in range(n_levels)]\n"," \n",- " # Normalize soft target\n"," s = sum(target)\n"," target = [v / s for v in target] if s > 0 else [1.0 / len(target)] * len(target)\n"," label = target.index(max(target))\n",@@ -159,37 +118,42 @@" \"label\": label\n"," }\n","\n",- "training_items = []\n",- "print(\"Preprocessing training records into tokenized items...\")\n",- "\n",+ "items = []\n","for row in ds_train:\n"," state = json.loads(row[\"state\"])\n"," questions = json.loads(row[\"questions\"])\n"," gold = json.loads(row[\"gold\"])\n",- " \n"," for qid, q in questions.items():\n"," if qid in gold:\n"," it = build_training_item(state, q, gold[qid])\n"," if it:\n",- " training_items.append(it)\n",+ " items.append(it)\n","\n",- "print(f\"Preprocessed {len(training_items)} training sequences across 1,200 cases.\")\n"+ "print(f\"Preprocessed {len(items)} training sequences across 1,200 cases.\")\n",+ "torch.save(items, \"/kaggle/working/train_items.pt\")\n",+ "print(\"Saved preprocessed items to /kaggle/working/train_items.pt\")\n"]},{"cell_type": "markdown","metadata": {},"source": [- "## 5. RLCD Policy Gradient Fine-Tuning Loop\n",- "We train the model using **strictly proper scoring rules** (Log + Spherical + Ranked Probability Score) with a GRPO-style group baseline ($G=4$ samples per question).\n"+ "## 4. DDP Training Script (`train_ddp.py`)\n",+ "We write the multi-GPU distributed RLCD training script using pure policy gradients with proper scoring rules and DDP gradient synchronization.\n"]},{"cell_type": "code","metadata": {},"source": [- "import random, time\n",- "import numpy as np\n",+ "%%writefile /kaggle/working/train_ddp.py\n",+ "import os, sys, time, json, random\n",+ "import torch\n",+ "import torch.distributed as dist\n",+ "from torch.nn.parallel import DistributedDataParallel as DDP\n",+ "from safetensors.torch import load_file, save_file\n",+ "from transformers import AutoTokenizer\n",+ "from laya.common import build_model, proper_reward, QTYPES\n","\n","def collate_train_batch(items, pad_id):\n"," n, L = len(items), max(len(it[\"ids\"]) for it in items)\n",@@ -216,147 +180,192 @@" \"label\": torch.tensor([it[\"label\"] for it in items])\n"," }\n","\n",- "EPOCHS = 2\n",- "BATCH_SIZE = 16 # 16 cases * 5 questions = 80 sequences per batch\n",- "GROUP_SIZE = 4 # Number of noisy candidate logits sampled per question (GRPO baseline)\n",- "LR_ENCODER = 1.5e-5 # Encoder fine-tuning rate\n",- "LR_HEAD = 1.5e-4 # Head fine-tuning rate\n",- "SIGMA_START = 0.5 # Starting exploration noise\n",- "SIGMA_END = 0.2 # Final exploration noise\n",- "\n",- "enc_params = [p for n, p in model.named_parameters() if n.startswith(\"encoder.\")]\n",- "head_params = [p for n, p in model.named_parameters() if not n.startswith(\"encoder.\")]\n",- "\n",- "optimizer = torch.optim.AdamW([\n",- " {\"params\": enc_params, \"lr\": LR_ENCODER},\n",- " {\"params\": head_params, \"lr\": LR_HEAD}\n",- "], weight_decay=0.01)\n",+ "def main():\n",+ " dist.init_process_group(\"nccl\")\n",+ " rank = dist.get_rank()\n",+ " world_size = dist.get_world_size()\n",+ " local_rank = int(os.environ.get(\"LOCAL_RANK\", \"0\"))\n",+ " torch.cuda.set_device(local_rank)\n",+ " device = torch.device(\"cuda\", local_rank)\n","\n",- "scaler = torch.amp.GradScaler(\"cuda\", enabled=True)\n",+ " model_dir = sys.argv[1]\n",+ " output_dir = sys.argv[2]\n",+ " \n",+ " with open(os.path.join(model_dir, \"rl_agent_config.json\")) as f:\n",+ " cfg = json.load(f)\n",+ " cfg[\"gradient_checkpointing\"] = True\n",+ " cfg[\"max_tokens_per_batch\"] = 4096\n","\n",- "print(\"Starting RLCD fine-tuning loop...\")\n",- "t0_train = time.time()\n",+ " tok = AutoTokenizer.from_pretrained(os.path.join(model_dir, \"tokenizer\"))\n",+ " model = build_model(cfg, encoder_dir=os.path.join(model_dir, \"encoder\"))\n",+ " \n",+ " weights = load_file(os.path.join(model_dir, \"model.safetensors\"))\n",+ " model.load_state_dict(weights, strict=True)\n",+ " \n",+ " model.encoder.gradient_checkpointing_enable(gradient_checkpointing_kwargs={\"use_reentrant\": False})\n",+ " model.head_checkpointing = True\n",+ " model.to(device)\n",+ " model.train()\n","\n",- "for epoch in range(EPOCHS):\n",- " random.shuffle(training_items)\n",- " epoch_loss, epoch_rew, n_batches = 0.0, 0.0, 0\n",- " progress = epoch / max(1, EPOCHS - 1)\n",- " sigma = SIGMA_START + (SIGMA_END - SIGMA_START) * progress\n",+ " ddp_model = DDP(model, device_ids=[local_rank], find_unused_parameters=True)\n"," \n",- " # Process in micro-batches\n",- " for b_idx in range(0, len(training_items), BATCH_SIZE * 5):\n",- " chunk = training_items[b_idx:b_idx + BATCH_SIZE * 5]\n",- " if not chunk:\n",- " continue\n",- " \n",- " batch = collate_train_batch(chunk, tok.pad_token_id)\n",- " optimizer.zero_grad()\n",- " \n",- " with torch.autocast(\"cuda\", dtype=torch.float16):\n",- " logits, act = model(\n",- " batch[\"input_ids\"].to(device),\n",- " batch[\"attention_mask\"].to(device),\n",- " batch[\"marker_pos\"].to(device),\n",- " batch[\"marker_mask\"].to(device),\n",- " batch[\"qtype\"].to(device)\n",- " )\n",- " \n",- " logits = logits.float()\n",- " mask = batch[\"marker_mask\"].to(device)\n",- " k = mask.sum(-1, keepdim=True).float()\n",- " \n",- " # 1. Sample G noisy logit distributions with zero-mean projection\n",- " eps = torch.randn((GROUP_SIZE,) + logits.shape, device=device) * sigma * mask\n",- " eps = (eps - eps.sum(-1, keepdim=True) / k) * mask\n",- " z = logits.detach().unsqueeze(0) + eps\n",- " q = torch.softmax(z.masked_fill(~mask, -1e4), -1)\n",- " \n",- " # 2. Evaluate proper scoring reward (Log + Spherical + RPS)\n",- " target = batch[\"target\"].to(device)\n",- " with torch.no_grad():\n",- " r = proper_reward(q, target.unsqueeze(0), batch[\"qtype\"].to(device), mask)\n",- " # 3. GRPO-style Group Mean Baseline Advantage\n",- " adv = r - r.mean(0, keepdim=True)\n",- " adv = adv / (adv.std() + 1e-6)\n",- " \n",- " # 4. Policy gradient loss\n",- " logp = -(((z - logits.unsqueeze(0)) ** 2) * mask).sum(-1) / (2 * sigma ** 2)\n",- " loss = -(adv * logp).mean()\n",- " \n",- " scaler.scale(loss).backward()\n",- " scaler.unscale_(optimizer)\n",- " torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n",- " scaler.step(optimizer)\n",- " scaler.update()\n",+ " all_items = torch.load(\"/kaggle/working/train_items.pt\", weights_only=False)\n",+ " # Split items evenly across ranks\n",+ " my_items = all_items[rank::world_size]\n",+ " \n",+ " EPOCHS = 2\n",+ " MICRO_BATCH = 10 # 10 sequences per forward pass per GPU\n",+ " GRAD_ACCUM = 2 # Effective batch across 2 GPUs = 40 sequences\n",+ " GROUP_SIZE = 4\n",+ " LR_ENCODER = 1.5e-5\n",+ " LR_HEAD = 1.5e-4\n",+ " SIGMA_START = 0.5\n",+ " SIGMA_END = 0.2\n",+ "\n",+ " enc_params = [p for n, p in ddp_model.named_parameters() if \"encoder.\" in n]\n",+ " head_params = [p for n, p in ddp_model.named_parameters() if \"encoder.\" not in n]\n",+ " \n",+ " optimizer = torch.optim.AdamW([\n",+ " {\"params\": enc_params, \"lr\": LR_ENCODER},\n",+ " {\"params\": head_params, \"lr\": LR_HEAD}\n",+ " ], weight_decay=0.01)\n",+ " \n",+ " scaler = torch.amp.GradScaler(\"cuda\", enabled=True)\n",+ " \n",+ " if rank == 0:\n",+ " print(f\"Starting 2xT4 DDP training: {len(all_items)} total items | {len(my_items)} per rank\")\n",+ " t0 = time.time()\n",+ " \n",+ " for epoch in range(EPOCHS):\n",+ " random.seed(42 + epoch + rank)\n",+ " random.shuffle(my_items)\n",+ " epoch_loss, n_batches = 0.0, 0\n",+ " optimizer.zero_grad(set_to_none=True)\n",+ " accum_step = 0\n"," \n",- " epoch_loss += loss.item()\n",- " epoch_rew += r.mean().item()\n",- " n_batches += 1\n",+ " progress = epoch / max(1, EPOCHS - 1)\n",+ " sigma = SIGMA_START + (SIGMA_END - SIGMA_START) * progress\n"," \n",- " if (n_batches % 10) == 0:\n",- " print(f\" Epoch {epoch+1}/{EPOCHS} | Batch {n_batches} | Loss: {loss.item():.4f} | Reward: {r.mean().item():.3f}\")\n",- "\n",- " avg_loss = epoch_loss / max(1, n_batches)\n",- " avg_rew = epoch_rew / max(1, n_batches)\n",- " print(f\"=== Epoch {epoch + 1}/{EPOCHS} Finished in {time.time() - t0_train:.1f}s | Avg Loss: {avg_loss:.4f} | Avg Reward: {avg_rew:.3f} ===\")\n",- "\n",- "print(f\"\\nFine-tuning completed in {time.time() - t0_train:.1f}s!\")\n"+ " for b_idx in range(0, len(my_items), MICRO_BATCH):\n",+ " chunk = my_items[b_idx:b_idx + MICRO_BATCH]\n",+ " if not chunk:\n",+ " continue\n",+ " \n",+ " batch = collate_train_batch(chunk, tok.pad_token_id)\n",+ " \n",+ " with torch.autocast(\"cuda\", dtype=torch.float16):\n",+ " logits, act = ddp_model(\n",+ " batch[\"input_ids\"].to(device),\n",+ " batch[\"attention_mask\"].to(device),\n",+ " batch[\"marker_pos\"].to(device),\n",+ " batch[\"marker_mask\"].to(device),\n",+ " batch[\"qtype\"].to(device)\n",+ " )\n",+ " \n",+ " logits = logits.float()\n",+ " mask = batch[\"marker_mask\"].to(device)\n",+ " k = mask.sum(-1, keepdim=True).float()\n",+ " \n",+ " eps = torch.randn((GROUP_SIZE,) + logits.shape, device=device) * sigma * mask\n",+ " eps = (eps - eps.sum(-1, keepdim=True) / k) * mask\n",+ " z = logits.detach().unsqueeze(0) + eps\n",+ " q = torch.softmax(z.masked_fill(~mask, -1e4), -1)\n",+ " \n",+ " target = batch[\"target\"].to(device)\n",+ " with torch.no_grad():\n",+ " r = proper_reward(q, target.unsqueeze(0), batch[\"qtype\"].to(device), mask)\n",+ " adv = r - r.mean(0, keepdim=True)\n",+ " adv = adv / (adv.std() + 1e-6)\n",+ " \n",+ " logp = -(((z - logits.unsqueeze(0)) ** 2) * mask).sum(-1) / (2 * sigma ** 2)\n",+ " loss = -(adv * logp).mean() / GRAD_ACCUM + 0.0 * act.sum()\n",+ " \n",+ " scaler.scale(loss).backward()\n",+ " accum_step += 1\n",+ " \n",+ " if accum_step % GRAD_ACCUM == 0 or (b_idx + MICRO_BATCH) >= len(my_items):\n",+ " scaler.unscale_(optimizer)\n",+ " torch.nn.utils.clip_grad_norm_(ddp_model.parameters(), 1.0)\n",+ " scaler.step(optimizer)\n",+ " scaler.update()\n",+ " optimizer.zero_grad(set_to_none=True)\n",+ " \n",+ " epoch_loss += loss.item() * GRAD_ACCUM\n",+ " n_batches += 1\n",+ " \n",+ " if rank == 0 and (n_batches % 50) == 0:\n",+ " print(f\" Epoch {epoch+1}/{EPOCHS} | Step {n_batches} | Loss: {loss.item()*GRAD_ACCUM:.4f} | Reward: {r.mean().item():.3f}\")\n",+ "\n",+ " if rank == 0:\n",+ " print(f\"=== Epoch {epoch+1}/{EPOCHS} Completed in {time.time()-t0:.1f}s | Avg Loss: {epoch_loss/max(1, n_batches):.4f} ===\")\n",+ "\n",+ " dist.barrier()\n",+ " if rank == 0:\n",+ " os.makedirs(output_dir, exist_ok=True)\n",+ " sd = {k: v.half().contiguous().cpu() for k, v in model.state_dict().items()}\n",+ " save_file(sd, os.path.join(output_dir, \"model.safetensors\"))\n",+ " model.encoder.config.save_pretrained(os.path.join(output_dir, \"encoder\"))\n",+ " tok.save_pretrained(os.path.join(output_dir, \"tokenizer\"))\n",+ " cfg[\"fine_tuned\"] = True\n",+ " cfg[\"model_name\"] = \"laya-typed-decisions\"\n",+ " with open(os.path.join(output_dir, \"rl_agent_config.json\"), \"w\") as f:\n",+ " json.dump(cfg, f, indent=2)\n",+ " print(f\"Model successfully saved to {output_dir}!\")\n",+ "\n",+ " dist.destroy_process_group()\n",+ "\n",+ "if __name__ == \"__main__\":\n",+ " main()\n"]},{"cell_type": "markdown","metadata": {},"source": [- "## 6. Save the Fine-Tuned Checkpoint\n"+ "## 5. Launch Multi-GPU Fine-Tuning with `torchrun`\n",+ "Runs on both T4 GPUs in parallel (~4 to 6 minutes total).\n"]},{"cell_type": "code","metadata": {},"source": [- "OUTPUT_DIR = \"/content/laya_finetuned_typed_decisions\"\n",- "os.makedirs(OUTPUT_DIR, exist_ok=True)\n",+ "OUTPUT_DIR = \"/kaggle/working/laya_finetuned_typed_decisions\"\n",+ "MODEL_DIR = model_dir\n","\n",- "# 1. Save fp16 weights\n",- "sd = {k: v.half().contiguous().cpu() for k, v in model.state_dict().items()}\n",- "weights_file = os.path.join(OUTPUT_DIR, \"model.safetensors\")\n",- "save_file(sd, weights_file)\n",- "\n",- "# 2. Save encoder config and tokenizer\n",- "model.encoder.config.save_pretrained(os.path.join(OUTPUT_DIR, \"encoder\"))\n",- "tok.save_pretrained(os.path.join(OUTPUT_DIR, \"tokenizer\"))\n",- "\n",- "# 3. Save config with fitted calibration parameters\n",- "cfg[\"fine_tuned\"] = True\n",- "cfg[\"model_name\"] = \"laya-typed-decisions\"\n",- "with open(os.path.join(OUTPUT_DIR, \"rl_agent_config.json\"), \"w\") as f:\n",- " json.dump(cfg, f, indent=2)\n",- "\n",- "print(f\"Model successfully saved to {OUTPUT_DIR} (Total size: {os.path.getsize(weights_file) / 1e6:.1f} MB)!\")\n"+ "cmd = f\"torchrun --standalone --nproc_per_node=2 /kaggle/working/train_ddp.py {MODEL_DIR} {OUTPUT_DIR}\"\n",+ "print(\"Executing DDP training:\", cmd)\n",+ "!{cmd}\n"]},{"cell_type": "markdown","metadata": {},"source": [- "## 7. Evaluate on Benchmark (`test` split: 400 cases / 2,000 decisions)\n",- "We run inference on the official `test` split and calculate the exact benchmark metrics.\n"+ "## 6. Run Benchmark Evaluation on Test Split (400 cases / 2,000 decisions)\n",+ "Evaluate the fine-tuned model against the official `test` split.\n"]},{"cell_type": "code","metadata": {},"source": [+ "import time, json\n",+ "import numpy as np\n",+ "import pandas as pd\n",+ "from datasets import load_dataset\n",+ "import laya\n",+ "from laya.common import ece_score\n",+ "\n","print(\"Loading test split for evaluation...\")\n","ds_test = load_dataset(\"LocalLLaMA/typed-decisions\", \"all\", split=\"test\")\n","\n",- "# Load fine-tuned model with Laya runtime\n","agent_ft = laya.Agent(OUTPUT_DIR, device=\"cuda\")\n","\n","predictions = []\n","latencies_ms = []\n","\n",- "print(f\"Evaluating {len(ds_test)} test cases...\")\n",+ "print(f\"Evaluating {len(ds_test)} test cases on GPU...\")\n","t0_eval = time.time()\n","\n","for i, row in enumerate(ds_test):\n",@@ -379,9 +388,6 @@" \"questions\": questions,\n"," \"latency_ms\": dt_ms\n"," })\n",- " \n",- " if (i + 1) % 50 == 0 or (i + 1) == len(ds_test):\n",- " print(f\" Progress: {i + 1}/{len(ds_test)} cases | Avg Latency: {np.mean(latencies_ms):.1f} ms/case\")\n","\n","print(f\"Evaluated all {len(ds_test)} cases in {time.time() - t0_eval:.1f}s!\")\n"]@@ -390,15 +396,13 @@"cell_type": "markdown","metadata": {},"source": [- "## 8. Compute Official Benchmark Metrics & Compare Head-to-Head\n"+ "## 7. Compute Official Metrics & Comparison Table against TypeSafe Jev\n"]},{"cell_type": "code","metadata": {},"source": [- "from laya.common import ece_score\n",- "\n","accuracies = []\n","soft_accuracies = []\n","brier_scores = []\n",@@ -465,8 +469,6 @@" p_score = p_ans[\"score\"]\n"," g_score = g_ans.get(\"score\", 0.0)\n"," score_maes.append(abs(p_score - g_score))\n",- " \n",- " # Within 1 level metric\n"," within_one.append(float(abs(p_score - g_score) <= 1.0))\n"," \n"," p_lvl = int(round(p_score))\n",@@ -500,7 +502,7 @@" \"Cost/Case\": \"$0.0004 (API)\"\n"," },\n"," {\n",- " \"Model\": \"Laya (Fine-Tuned)\",\n",+ " \"Model\": \"Laya (Fine-Tuned 2xT4)\",\n"," \"Kind\": \"fine-tuned\",\n"," \"Accuracy\": round(laya_acc, 3),\n"," \"Soft Acc\": round(laya_soft_acc, 3),\n",@@ -546,8 +548,7 @@"cell_type": "markdown","metadata": {},"source": [- "## 9. (Optional) Push Fine-Tuned Model to Hugging Face Hub\n",- "Publish your newly fine-tuned checkpoint directly to Hugging Face.\n"+ "## 8. (Optional) Push to Hugging Face Hub\n"]},{@@ -573,11 +574,11 @@}],"metadata": {- "accelerator": "GPU",- "colab": {- "name": "laya_finetune_typed_decisions_colab.ipynb",- "provenance": [],- "gpuType": "T4"+ "accelerator": "nvidiaGpuT4x2",+ "kaggle": {+ "accelerator": "gpuT4x2",+ "isInternetEnabled": true,+ "isGpuEnabled": true},"kernelspec": {"display_name": "Python 3",