Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat: add 2xT4 Kaggle fine-tuning notebook and remove colab workflow notebook

Nandakishor1332269369d1
diff --git a/README.md b/README.md
index ade5622d..774403e7 100644
--- a/README.md
+++ b/README.md
@@ -175,10 +175,9 @@ triage = agent.predict({"message": "My payment failed twice"}, laya.triage_quest
## Fine-Tuning on Single T4 GPU (Google Colab)
-Fine-tune Laya on custom domain data or the `LocalLLaMA/typed-decisions` benchmark on a free T4 GPU:
+Fine-tune Laya on your custom domain data or commercial datasets on a free T4 GPU:
* **Interactive Fine-Tuning Notebook:** [Fine-Tune on Custom Data](https://colab.research.google.com/drive/15d4Yv__KHeHjshVb-6PRTfqVllxih2S3?usp=sharing) ([`notebooks/laya_finetune_colab.ipynb`](notebooks/laya_finetune_colab.ipynb))
-* **Workflow Benchmark Fine-Tuning:** [`notebooks/laya_finetune_typed_decisions_colab.ipynb`](notebooks/laya_finetune_typed_decisions_colab.ipynb) — fine-tunes on `LocalLLaMA/typed-decisions` (1,200 cases), evaluates on 400 test cases, and pushes to Hugging Face.
---
diff --git a/laya/__init__.py b/laya/__init__.py
index 6b6ae587..4c693eb2 100644
--- a/laya/__init__.py
+++ b/laya/__init__.py
@@ -13,7 +13,7 @@ from .common import (
from .email import clean_email_body, email_questions, email_state
from .presets import guard_questions, moderation_questions, router_questions, triage_questions
-__version__ = "0.1.6"
+__version__ = "0.1.7"
__all__ = [
"Agent",
"RLAgent",
diff --git a/laya/agent.py b/laya/agent.py
index 656c2460..df0738a1 100644
--- a/laya/agent.py
+++ b/laya/agent.py
@@ -97,6 +97,11 @@ class Agent:
model_dir = model_id_or_path
if not os.path.exists(model_dir):
+ if model_id_or_path.startswith(("/", "./", "../")) or os.path.isabs(model_id_or_path):
+ raise FileNotFoundError(
+ f"Local model path not found: {model_id_or_path!r}. "
+ f"Check that the directory exists and that training saved the model successfully."
+ )
from huggingface_hub import snapshot_download
model_dir = snapshot_download(model_id_or_path, token=token or os.environ.get("HF_TOKEN"))
diff --git a/notebooks/laya_finetune_typed_decisions_colab.ipynb b/notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb
similarity index 59%
rename from notebooks/laya_finetune_typed_decisions_colab.ipynb
rename to notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb
index b3fed00b..25ad0ad4 100644
--- a/notebooks/laya_finetune_typed_decisions_colab.ipynb
+++ b/notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb
@@ -4,30 +4,26 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "# Fine-Tuning Laya on `LocalLLaMA/typed-decisions` & Evaluation (Single T4 GPU)\n",
+ "# Fine-Tuning Laya on `LocalLLaMA/typed-decisions` (Kaggle GPU T4 \u00d72 DDP)\n",
"\n",
- "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NandhaKishorM/laya/blob/main/notebooks/laya_finetune_typed_decisions_colab.ipynb)\n",
"[![PyPI version](https://img.shields.io/pypi/v/laya.svg)](https://pypi.org/project/laya/)\n",
"[![Hugging Face Model](https://img.shields.io/badge/%F0%9F%A4%97%20Model-convaiinnovations%2Flaya-blue)](https://huggingface.co/convaiinnovations/laya)\n",
"[![Dataset](https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-LocalLLaMA%2Ftyped--decisions-green)](https://huggingface.co/datasets/LocalLLaMA/typed-decisions)\n",
"\n",
- "This notebook fine-tunes **Laya** (`convaiinnovations/laya`) on the **1,200 training cases (6,000 typed decisions)** of the [LocalLLaMA/typed-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) benchmark on a **single free T4 GPU (16 GB)** using **RLCD (Reinforcement Learning for Calibrated Decisions)**.\n",
+ "This notebook fine-tunes **Laya** (`convaiinnovations/laya`, 421M params) on the **1,200 training cases (6,000 typed decisions)** of the [LocalLLaMA/typed-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) benchmark using **both NVIDIA T4 GPUs via Distributed Data Parallel (`torchrun --nproc_per_node=2`)**.\n",
"\n",
- "After fine-tuning (~10-15 minutes), it evaluates the model against the **400 test cases (2,000 decisions)** and compares head-to-head with **TypeSafe Jev 1.13.0**, then lets you push the new checkpoint to Hugging Face.\n",
- "\n",
- "### The 4 Production Workflows:\n",
- "1. `agent_trace_observability` (300 train / 100 test)\n",
- "2. `customer_service` (300 train / 100 test)\n",
- "3. `invoice_processing` (300 train / 100 test)\n",
- "4. `security_incidents` (300 train / 100 test)\n"
+ "### Setup on Kaggle:\n",
+ "- **Settings -> Accelerator:** `GPU T4 x2`\n",
+ "- **Settings -> Internet:** `On`\n",
+ "- **Output:** `/kaggle/working/laya_finetuned_typed_decisions`\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "## 1. Environment & GPU Check\n",
- "Verify the T4 GPU is active (`Runtime -> Change runtime type -> T4 GPU`).\n"
+ "## 1. Environment & Dual T4 GPU Check\n",
+ "Verify both T4 GPUs are detected.\n"
]
},
{
@@ -35,10 +31,13 @@
"metadata": {},
"source": [
"!nvidia-smi\n",
- "import os, torch\n",
- "assert torch.cuda.is_available(), \"No GPU detected! Set Runtime -> Change runtime type -> T4 GPU\"\n",
- "props = torch.cuda.get_device_properties(0)\n",
- "print(f\"Connected to: {props.name} | Total VRAM: {props.total_memory / 1e9:.1f} GB\")\n",
+ "import os, subprocess, torch\n",
+ "n_gpu = torch.cuda.device_count()\n",
+ "print(f\"CUDA Available: {torch.cuda.is_available()} | Visible GPUs: {n_gpu}\")\n",
+ "for i in range(n_gpu):\n",
+ " p = torch.cuda.get_device_properties(i)\n",
+ " print(f\" GPU {i}: {p.name} ({p.total_memory / 1e9:.1f} GB)\")\n",
+ "assert n_gpu == 2, \"Please select Accelerator: GPU T4 x2 in Kaggle settings!\"\n",
"os.environ[\"PYTORCH_CUDA_ALLOC_CONF\"] = \"expandable_segments:True\"\n"
]
},
@@ -53,7 +52,7 @@
"cell_type": "code",
"metadata": {},
"source": [
- "!pip install -q -U \"laya>=0.1.6\" \"transformers>=4.48.0\" \"datasets>=3.0.0\" safetensors huggingface_hub pyarrow pandas scipy accelerate\n",
+ "!pip install -q -U \"laya>=0.1.6\" \"transformers>=4.48.0\" \"datasets>=3.0.0\" safetensors huggingface_hub pyarrow pandas scipy accelerate tabulate\n",
"import laya, transformers, datasets, torch\n",
"print(\"Laya version :\", laya.__version__)\n",
"print(\"Transformers version:\", transformers.__version__)\n",
@@ -64,8 +63,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## 3. Load Base Model & Tokenizer\n",
- "We load `convaiinnovations/laya` (421M parameters) in FP16 with gradient checkpointing enabled for single-GPU training.\n"
+ "## 3. Download & Preprocess Data for DDP\n",
+ "We preprocess all 1,200 training cases into tokenized items and save them to disk so both DDP worker ranks can read them.\n"
]
},
{
@@ -73,62 +72,23 @@
"metadata": {},
"source": [
"import os, json, torch\n",
+ "from datasets import load_dataset\n",
+ "from transformers import AutoTokenizer\n",
"from huggingface_hub import snapshot_download\n",
"from laya.agent import _fix_tokenizer_config\n",
- "from laya.common import build_model, QTYPES, render_options, build_sequence, proper_reward, confidence_from_probs\n",
- "from transformers import AutoTokenizer\n",
- "from safetensors.torch import load_file\n",
+ "from laya.common import build_sequence, render_options, QTYPES\n",
"\n",
"MODEL_ID = \"convaiinnovations/laya\"\n",
- "print(f\"Downloading base model weights from {MODEL_ID}...\")\n",
+ "print(f\"Fetching tokenizer and config from {MODEL_ID}...\")\n",
"model_dir = snapshot_download(MODEL_ID)\n",
"_fix_tokenizer_config(model_dir)\n",
"\n",
+ "tok = AutoTokenizer.from_pretrained(os.path.join(model_dir, \"tokenizer\"))\n",
"with open(os.path.join(model_dir, \"rl_agent_config.json\")) as f:\n",
" cfg = json.load(f)\n",
"\n",
- "# Optimize config for single T4 fine-tuning\n",
- "cfg[\"gradient_checkpointing\"] = True\n",
- "cfg[\"max_tokens_per_batch\"] = 4096\n",
- "cfg[\"head_layers\"] = 2\n",
- "cfg[\"max_len\"] = 512\n",
- "cfg[\"head_max_len\"] = 192\n",
- "\n",
- "tok = AutoTokenizer.from_pretrained(os.path.join(model_dir, \"tokenizer\"))\n",
- "model = build_model(cfg, encoder_dir=os.path.join(model_dir, \"encoder\"))\n",
- "\n",
- "# Load fine-tuned weights\n",
- "weights = load_file(os.path.join(model_dir, \"model.safetensors\"))\n",
- "model.load_state_dict(weights, strict=True)\n",
- "\n",
- "device = torch.device(\"cuda\")\n",
- "model.encoder.gradient_checkpointing_enable(gradient_checkpointing_kwargs={\"use_reentrant\": False})\n",
- "model.head_checkpointing = True\n",
- "model.to(device)\n",
- "model.train()\n",
- "\n",
- "print(f\"Successfully loaded Laya ({sum(p.numel() for p in model.parameters()) / 1e6:.1f}M params) on {device}!\")\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## 4. Download & Preprocess `typed-decisions` (Train Split)\n",
- "We download all 1,200 training cases and format them with soft gold targets from the teacher.\n"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {},
- "source": [
- "from datasets import load_dataset\n",
- "import pandas as pd\n",
- "import json\n",
- "\n",
- "print(\"Downloading LocalLLaMA/typed-decisions (split: train)...\")\n",
+ "print(\"Downloading LocalLLaMA/typed-decisions (train split)...\")\n",
"ds_train = load_dataset(\"LocalLLaMA/typed-decisions\", \"all\", split=\"train\")\n",
- "print(f\"Loaded {len(ds_train)} training workflow cases.\")\n",
"\n",
"def build_training_item(state, q, gold_q):\n",
" t = q[\"type\"]\n",
@@ -142,7 +102,6 @@
" n_levels = len(crit) if isinstance(crit, list) else 4\n",
" target = [gold_q[\"probabilities\"].get(str(i), 0.0) for i in range(n_levels)]\n",
" \n",
- " # Normalize soft target\n",
" s = sum(target)\n",
" target = [v / s for v in target] if s > 0 else [1.0 / len(target)] * len(target)\n",
" label = target.index(max(target))\n",
@@ -159,37 +118,42 @@
" \"label\": label\n",
" }\n",
"\n",
- "training_items = []\n",
- "print(\"Preprocessing training records into tokenized items...\")\n",
- "\n",
+ "items = []\n",
"for row in ds_train:\n",
" state = json.loads(row[\"state\"])\n",
" questions = json.loads(row[\"questions\"])\n",
" gold = json.loads(row[\"gold\"])\n",
- " \n",
" for qid, q in questions.items():\n",
" if qid in gold:\n",
" it = build_training_item(state, q, gold[qid])\n",
" if it:\n",
- " training_items.append(it)\n",
+ " items.append(it)\n",
"\n",
- "print(f\"Preprocessed {len(training_items)} training sequences across 1,200 cases.\")\n"
+ "print(f\"Preprocessed {len(items)} training sequences across 1,200 cases.\")\n",
+ "torch.save(items, \"/kaggle/working/train_items.pt\")\n",
+ "print(\"Saved preprocessed items to /kaggle/working/train_items.pt\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "## 5. RLCD Policy Gradient Fine-Tuning Loop\n",
- "We train the model using **strictly proper scoring rules** (Log + Spherical + Ranked Probability Score) with a GRPO-style group baseline ($G=4$ samples per question).\n"
+ "## 4. DDP Training Script (`train_ddp.py`)\n",
+ "We write the multi-GPU distributed RLCD training script using pure policy gradients with proper scoring rules and DDP gradient synchronization.\n"
]
},
{
"cell_type": "code",
"metadata": {},
"source": [
- "import random, time\n",
- "import numpy as np\n",
+ "%%writefile /kaggle/working/train_ddp.py\n",
+ "import os, sys, time, json, random\n",
+ "import torch\n",
+ "import torch.distributed as dist\n",
+ "from torch.nn.parallel import DistributedDataParallel as DDP\n",
+ "from safetensors.torch import load_file, save_file\n",
+ "from transformers import AutoTokenizer\n",
+ "from laya.common import build_model, proper_reward, QTYPES\n",
"\n",
"def collate_train_batch(items, pad_id):\n",
" n, L = len(items), max(len(it[\"ids\"]) for it in items)\n",
@@ -216,147 +180,192 @@
" \"label\": torch.tensor([it[\"label\"] for it in items])\n",
" }\n",
"\n",
- "EPOCHS = 2\n",
- "BATCH_SIZE = 16 # 16 cases * 5 questions = 80 sequences per batch\n",
- "GROUP_SIZE = 4 # Number of noisy candidate logits sampled per question (GRPO baseline)\n",
- "LR_ENCODER = 1.5e-5 # Encoder fine-tuning rate\n",
- "LR_HEAD = 1.5e-4 # Head fine-tuning rate\n",
- "SIGMA_START = 0.5 # Starting exploration noise\n",
- "SIGMA_END = 0.2 # Final exploration noise\n",
- "\n",
- "enc_params = [p for n, p in model.named_parameters() if n.startswith(\"encoder.\")]\n",
- "head_params = [p for n, p in model.named_parameters() if not n.startswith(\"encoder.\")]\n",
- "\n",
- "optimizer = torch.optim.AdamW([\n",
- " {\"params\": enc_params, \"lr\": LR_ENCODER},\n",
- " {\"params\": head_params, \"lr\": LR_HEAD}\n",
- "], weight_decay=0.01)\n",
+ "def main():\n",
+ " dist.init_process_group(\"nccl\")\n",
+ " rank = dist.get_rank()\n",
+ " world_size = dist.get_world_size()\n",
+ " local_rank = int(os.environ.get(\"LOCAL_RANK\", \"0\"))\n",
+ " torch.cuda.set_device(local_rank)\n",
+ " device = torch.device(\"cuda\", local_rank)\n",
"\n",
- "scaler = torch.amp.GradScaler(\"cuda\", enabled=True)\n",
+ " model_dir = sys.argv[1]\n",
+ " output_dir = sys.argv[2]\n",
+ " \n",
+ " with open(os.path.join(model_dir, \"rl_agent_config.json\")) as f:\n",
+ " cfg = json.load(f)\n",
+ " cfg[\"gradient_checkpointing\"] = True\n",
+ " cfg[\"max_tokens_per_batch\"] = 4096\n",
"\n",
- "print(\"Starting RLCD fine-tuning loop...\")\n",
- "t0_train = time.time()\n",
+ " tok = AutoTokenizer.from_pretrained(os.path.join(model_dir, \"tokenizer\"))\n",
+ " model = build_model(cfg, encoder_dir=os.path.join(model_dir, \"encoder\"))\n",
+ " \n",
+ " weights = load_file(os.path.join(model_dir, \"model.safetensors\"))\n",
+ " model.load_state_dict(weights, strict=True)\n",
+ " \n",
+ " model.encoder.gradient_checkpointing_enable(gradient_checkpointing_kwargs={\"use_reentrant\": False})\n",
+ " model.head_checkpointing = True\n",
+ " model.to(device)\n",
+ " model.train()\n",
"\n",
- "for epoch in range(EPOCHS):\n",
- " random.shuffle(training_items)\n",
- " epoch_loss, epoch_rew, n_batches = 0.0, 0.0, 0\n",
- " progress = epoch / max(1, EPOCHS - 1)\n",
- " sigma = SIGMA_START + (SIGMA_END - SIGMA_START) * progress\n",
+ " ddp_model = DDP(model, device_ids=[local_rank], find_unused_parameters=True)\n",
" \n",
- " # Process in micro-batches\n",
- " for b_idx in range(0, len(training_items), BATCH_SIZE * 5):\n",
- " chunk = training_items[b_idx:b_idx + BATCH_SIZE * 5]\n",
- " if not chunk:\n",
- " continue\n",
- " \n",
- " batch = collate_train_batch(chunk, tok.pad_token_id)\n",
- " optimizer.zero_grad()\n",
- " \n",
- " with torch.autocast(\"cuda\", dtype=torch.float16):\n",
- " logits, act = model(\n",
- " batch[\"input_ids\"].to(device),\n",
- " batch[\"attention_mask\"].to(device),\n",
- " batch[\"marker_pos\"].to(device),\n",
- " batch[\"marker_mask\"].to(device),\n",
- " batch[\"qtype\"].to(device)\n",
- " )\n",
- " \n",
- " logits = logits.float()\n",
- " mask = batch[\"marker_mask\"].to(device)\n",
- " k = mask.sum(-1, keepdim=True).float()\n",
- " \n",
- " # 1. Sample G noisy logit distributions with zero-mean projection\n",
- " eps = torch.randn((GROUP_SIZE,) + logits.shape, device=device) * sigma * mask\n",
- " eps = (eps - eps.sum(-1, keepdim=True) / k) * mask\n",
- " z = logits.detach().unsqueeze(0) + eps\n",
- " q = torch.softmax(z.masked_fill(~mask, -1e4), -1)\n",
- " \n",
- " # 2. Evaluate proper scoring reward (Log + Spherical + RPS)\n",
- " target = batch[\"target\"].to(device)\n",
- " with torch.no_grad():\n",
- " r = proper_reward(q, target.unsqueeze(0), batch[\"qtype\"].to(device), mask)\n",
- " # 3. GRPO-style Group Mean Baseline Advantage\n",
- " adv = r - r.mean(0, keepdim=True)\n",
- " adv = adv / (adv.std() + 1e-6)\n",
- " \n",
- " # 4. Policy gradient loss\n",
- " logp = -(((z - logits.unsqueeze(0)) ** 2) * mask).sum(-1) / (2 * sigma ** 2)\n",
- " loss = -(adv * logp).mean()\n",
- " \n",
- " scaler.scale(loss).backward()\n",
- " scaler.unscale_(optimizer)\n",
- " torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n",
- " scaler.step(optimizer)\n",
- " scaler.update()\n",
+ " all_items = torch.load(\"/kaggle/working/train_items.pt\", weights_only=False)\n",
+ " # Split items evenly across ranks\n",
+ " my_items = all_items[rank::world_size]\n",
+ " \n",
+ " EPOCHS = 2\n",
+ " MICRO_BATCH = 10 # 10 sequences per forward pass per GPU\n",
+ " GRAD_ACCUM = 2 # Effective batch across 2 GPUs = 40 sequences\n",
+ " GROUP_SIZE = 4\n",
+ " LR_ENCODER = 1.5e-5\n",
+ " LR_HEAD = 1.5e-4\n",
+ " SIGMA_START = 0.5\n",
+ " SIGMA_END = 0.2\n",
+ "\n",
+ " enc_params = [p for n, p in ddp_model.named_parameters() if \"encoder.\" in n]\n",
+ " head_params = [p for n, p in ddp_model.named_parameters() if \"encoder.\" not in n]\n",
+ " \n",
+ " optimizer = torch.optim.AdamW([\n",
+ " {\"params\": enc_params, \"lr\": LR_ENCODER},\n",
+ " {\"params\": head_params, \"lr\": LR_HEAD}\n",
+ " ], weight_decay=0.01)\n",
+ " \n",
+ " scaler = torch.amp.GradScaler(\"cuda\", enabled=True)\n",
+ " \n",
+ " if rank == 0:\n",
+ " print(f\"Starting 2xT4 DDP training: {len(all_items)} total items | {len(my_items)} per rank\")\n",
+ " t0 = time.time()\n",
+ " \n",
+ " for epoch in range(EPOCHS):\n",
+ " random.seed(42 + epoch + rank)\n",
+ " random.shuffle(my_items)\n",
+ " epoch_loss, n_batches = 0.0, 0\n",
+ " optimizer.zero_grad(set_to_none=True)\n",
+ " accum_step = 0\n",
" \n",
- " epoch_loss += loss.item()\n",
- " epoch_rew += r.mean().item()\n",
- " n_batches += 1\n",
+ " progress = epoch / max(1, EPOCHS - 1)\n",
+ " sigma = SIGMA_START + (SIGMA_END - SIGMA_START) * progress\n",
" \n",
- " if (n_batches % 10) == 0:\n",
- " print(f\" Epoch {epoch+1}/{EPOCHS} | Batch {n_batches} | Loss: {loss.item():.4f} | Reward: {r.mean().item():.3f}\")\n",
- "\n",
- " avg_loss = epoch_loss / max(1, n_batches)\n",
- " avg_rew = epoch_rew / max(1, n_batches)\n",
- " print(f\"=== Epoch {epoch + 1}/{EPOCHS} Finished in {time.time() - t0_train:.1f}s | Avg Loss: {avg_loss:.4f} | Avg Reward: {avg_rew:.3f} ===\")\n",
- "\n",
- "print(f\"\\nFine-tuning completed in {time.time() - t0_train:.1f}s!\")\n"
+ " for b_idx in range(0, len(my_items), MICRO_BATCH):\n",
+ " chunk = my_items[b_idx:b_idx + MICRO_BATCH]\n",
+ " if not chunk:\n",
+ " continue\n",
+ " \n",
+ " batch = collate_train_batch(chunk, tok.pad_token_id)\n",
+ " \n",
+ " with torch.autocast(\"cuda\", dtype=torch.float16):\n",
+ " logits, act = ddp_model(\n",
+ " batch[\"input_ids\"].to(device),\n",
+ " batch[\"attention_mask\"].to(device),\n",
+ " batch[\"marker_pos\"].to(device),\n",
+ " batch[\"marker_mask\"].to(device),\n",
+ " batch[\"qtype\"].to(device)\n",
+ " )\n",
+ " \n",
+ " logits = logits.float()\n",
+ " mask = batch[\"marker_mask\"].to(device)\n",
+ " k = mask.sum(-1, keepdim=True).float()\n",
+ " \n",
+ " eps = torch.randn((GROUP_SIZE,) + logits.shape, device=device) * sigma * mask\n",
+ " eps = (eps - eps.sum(-1, keepdim=True) / k) * mask\n",
+ " z = logits.detach().unsqueeze(0) + eps\n",
+ " q = torch.softmax(z.masked_fill(~mask, -1e4), -1)\n",
+ " \n",
+ " target = batch[\"target\"].to(device)\n",
+ " with torch.no_grad():\n",
+ " r = proper_reward(q, target.unsqueeze(0), batch[\"qtype\"].to(device), mask)\n",
+ " adv = r - r.mean(0, keepdim=True)\n",
+ " adv = adv / (adv.std() + 1e-6)\n",
+ " \n",
+ " logp = -(((z - logits.unsqueeze(0)) ** 2) * mask).sum(-1) / (2 * sigma ** 2)\n",
+ " loss = -(adv * logp).mean() / GRAD_ACCUM + 0.0 * act.sum()\n",
+ " \n",
+ " scaler.scale(loss).backward()\n",
+ " accum_step += 1\n",
+ " \n",
+ " if accum_step % GRAD_ACCUM == 0 or (b_idx + MICRO_BATCH) >= len(my_items):\n",
+ " scaler.unscale_(optimizer)\n",
+ " torch.nn.utils.clip_grad_norm_(ddp_model.parameters(), 1.0)\n",
+ " scaler.step(optimizer)\n",
+ " scaler.update()\n",
+ " optimizer.zero_grad(set_to_none=True)\n",
+ " \n",
+ " epoch_loss += loss.item() * GRAD_ACCUM\n",
+ " n_batches += 1\n",
+ " \n",
+ " if rank == 0 and (n_batches % 50) == 0:\n",
+ " print(f\" Epoch {epoch+1}/{EPOCHS} | Step {n_batches} | Loss: {loss.item()*GRAD_ACCUM:.4f} | Reward: {r.mean().item():.3f}\")\n",
+ "\n",
+ " if rank == 0:\n",
+ " print(f\"=== Epoch {epoch+1}/{EPOCHS} Completed in {time.time()-t0:.1f}s | Avg Loss: {epoch_loss/max(1, n_batches):.4f} ===\")\n",
+ "\n",
+ " dist.barrier()\n",
+ " if rank == 0:\n",
+ " os.makedirs(output_dir, exist_ok=True)\n",
+ " sd = {k: v.half().contiguous().cpu() for k, v in model.state_dict().items()}\n",
+ " save_file(sd, os.path.join(output_dir, \"model.safetensors\"))\n",
+ " model.encoder.config.save_pretrained(os.path.join(output_dir, \"encoder\"))\n",
+ " tok.save_pretrained(os.path.join(output_dir, \"tokenizer\"))\n",
+ " cfg[\"fine_tuned\"] = True\n",
+ " cfg[\"model_name\"] = \"laya-typed-decisions\"\n",
+ " with open(os.path.join(output_dir, \"rl_agent_config.json\"), \"w\") as f:\n",
+ " json.dump(cfg, f, indent=2)\n",
+ " print(f\"Model successfully saved to {output_dir}!\")\n",
+ "\n",
+ " dist.destroy_process_group()\n",
+ "\n",
+ "if __name__ == \"__main__\":\n",
+ " main()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "## 6. Save the Fine-Tuned Checkpoint\n"
+ "## 5. Launch Multi-GPU Fine-Tuning with `torchrun`\n",
+ "Runs on both T4 GPUs in parallel (~4 to 6 minutes total).\n"
]
},
{
"cell_type": "code",
"metadata": {},
"source": [
- "OUTPUT_DIR = \"/content/laya_finetuned_typed_decisions\"\n",
- "os.makedirs(OUTPUT_DIR, exist_ok=True)\n",
+ "OUTPUT_DIR = \"/kaggle/working/laya_finetuned_typed_decisions\"\n",
+ "MODEL_DIR = model_dir\n",
"\n",
- "# 1. Save fp16 weights\n",
- "sd = {k: v.half().contiguous().cpu() for k, v in model.state_dict().items()}\n",
- "weights_file = os.path.join(OUTPUT_DIR, \"model.safetensors\")\n",
- "save_file(sd, weights_file)\n",
- "\n",
- "# 2. Save encoder config and tokenizer\n",
- "model.encoder.config.save_pretrained(os.path.join(OUTPUT_DIR, \"encoder\"))\n",
- "tok.save_pretrained(os.path.join(OUTPUT_DIR, \"tokenizer\"))\n",
- "\n",
- "# 3. Save config with fitted calibration parameters\n",
- "cfg[\"fine_tuned\"] = True\n",
- "cfg[\"model_name\"] = \"laya-typed-decisions\"\n",
- "with open(os.path.join(OUTPUT_DIR, \"rl_agent_config.json\"), \"w\") as f:\n",
- " json.dump(cfg, f, indent=2)\n",
- "\n",
- "print(f\"Model successfully saved to {OUTPUT_DIR} (Total size: {os.path.getsize(weights_file) / 1e6:.1f} MB)!\")\n"
+ "cmd = f\"torchrun --standalone --nproc_per_node=2 /kaggle/working/train_ddp.py {MODEL_DIR} {OUTPUT_DIR}\"\n",
+ "print(\"Executing DDP training:\", cmd)\n",
+ "!{cmd}\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "## 7. Evaluate on Benchmark (`test` split: 400 cases / 2,000 decisions)\n",
- "We run inference on the official `test` split and calculate the exact benchmark metrics.\n"
+ "## 6. Run Benchmark Evaluation on Test Split (400 cases / 2,000 decisions)\n",
+ "Evaluate the fine-tuned model against the official `test` split.\n"
]
},
{
"cell_type": "code",
"metadata": {},
"source": [
+ "import time, json\n",
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "from datasets import load_dataset\n",
+ "import laya\n",
+ "from laya.common import ece_score\n",
+ "\n",
"print(\"Loading test split for evaluation...\")\n",
"ds_test = load_dataset(\"LocalLLaMA/typed-decisions\", \"all\", split=\"test\")\n",
"\n",
- "# Load fine-tuned model with Laya runtime\n",
"agent_ft = laya.Agent(OUTPUT_DIR, device=\"cuda\")\n",
"\n",
"predictions = []\n",
"latencies_ms = []\n",
"\n",
- "print(f\"Evaluating {len(ds_test)} test cases...\")\n",
+ "print(f\"Evaluating {len(ds_test)} test cases on GPU...\")\n",
"t0_eval = time.time()\n",
"\n",
"for i, row in enumerate(ds_test):\n",
@@ -379,9 +388,6 @@
" \"questions\": questions,\n",
" \"latency_ms\": dt_ms\n",
" })\n",
- " \n",
- " if (i + 1) % 50 == 0 or (i + 1) == len(ds_test):\n",
- " print(f\" Progress: {i + 1}/{len(ds_test)} cases | Avg Latency: {np.mean(latencies_ms):.1f} ms/case\")\n",
"\n",
"print(f\"Evaluated all {len(ds_test)} cases in {time.time() - t0_eval:.1f}s!\")\n"
]
@@ -390,15 +396,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## 8. Compute Official Benchmark Metrics & Compare Head-to-Head\n"
+ "## 7. Compute Official Metrics & Comparison Table against TypeSafe Jev\n"
]
},
{
"cell_type": "code",
"metadata": {},
"source": [
- "from laya.common import ece_score\n",
- "\n",
"accuracies = []\n",
"soft_accuracies = []\n",
"brier_scores = []\n",
@@ -465,8 +469,6 @@
" p_score = p_ans[\"score\"]\n",
" g_score = g_ans.get(\"score\", 0.0)\n",
" score_maes.append(abs(p_score - g_score))\n",
- " \n",
- " # Within 1 level metric\n",
" within_one.append(float(abs(p_score - g_score) <= 1.0))\n",
" \n",
" p_lvl = int(round(p_score))\n",
@@ -500,7 +502,7 @@
" \"Cost/Case\": \"$0.0004 (API)\"\n",
" },\n",
" {\n",
- " \"Model\": \"Laya (Fine-Tuned)\",\n",
+ " \"Model\": \"Laya (Fine-Tuned 2xT4)\",\n",
" \"Kind\": \"fine-tuned\",\n",
" \"Accuracy\": round(laya_acc, 3),\n",
" \"Soft Acc\": round(laya_soft_acc, 3),\n",
@@ -546,8 +548,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## 9. (Optional) Push Fine-Tuned Model to Hugging Face Hub\n",
- "Publish your newly fine-tuned checkpoint directly to Hugging Face.\n"
+ "## 8. (Optional) Push to Hugging Face Hub\n"
]
},
{
@@ -573,11 +574,11 @@
}
],
"metadata": {
- "accelerator": "GPU",
- "colab": {
- "name": "laya_finetune_typed_decisions_colab.ipynb",
- "provenance": [],
- "gpuType": "T4"
+ "accelerator": "nvidiaGpuT4x2",
+ "kaggle": {
+ "accelerator": "gpuT4x2",
+ "isInternetEnabled": true,
+ "isGpuEnabled": true
},
"kernelspec": {
"display_name": "Python 3",