Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
Commits on main
Fused Qwen3 q_norm / k_norm RMSNorm + RoPE Triton kernel (opt-in) (#13890)
701a345 Daniel Han committed 1 month ago
Add optim = "adamw_flash" (FlashAdamW from flashoptim) for lower optimizer memory (#13881)
313bfd9 Daniel Han committed 1 month ago
Keep the plain capability cache on single-GPU hosts (#13962)
6baec3b Daniel Han committed 1 month ago
Installers (AMD, Linux): a Strix APU defers torch to a discrete card, and a lone gfx1153 warns instead of getting kernel-less wheels (#13931)
4a8b727 Daniel Han committed 1 month ago
Studio: proportional image attachments, attachment card and Chat with files polish (#13970)
aafc3df Michael Han committed 1 month ago
FSDP2: first-class LoRA and full finetuning, keep fused LoRA kernels per block (#13865)
ee2ce8c Daniel Han committed 1 month ago
Studio: name the open connection, lead the provider menu with Custom, new OpenRouter logo (#13964)
75e2ea2 Michael Han committed 1 month ago
fix(studio): load audio GGUFs selected from custom folders (#13897)
67f7ab9 Lee Jackson committed 1 month ago
Studio: Deep research sits under Tools in the + menu (#13959)
260d89b Michael Han committed 1 month ago
Add opt-in optimizer state CPU offload (UNSLOTH_OFFLOAD_OPTIMIZER_STATE=1) (#13882)
e539ad5 Daniel Han committed 1 month ago
DPO, ORPO, CPO: opt-in fused preference loss (UNSLOTH_FUSED_PREFERENCE_LOSS=1) (#13870)
85d7482 Daniel Han committed 1 month ago
Fix RMSNorm and LayerNorm backward overwriting the incoming gradient (#13888)
1b8737b Daniel Han committed 1 month ago
Studio: the bot-check page's Unsloth Desktop link opens Get started (#13958)
5870795 Michael Han committed 1 month ago
Core: decide bfloat16, compile and TMA options from every visible NVIDIA GPU on mixed hosts (#13923)
2b72467 Daniel Han committed 1 month ago
Context parallel: Ulysses for Gated DeltaNet hybrids (Qwen3.5, Qwen3-Next) (#13920)
3ef4011 Daniel Han committed 1 month ago
Studio installer: cover the GPUs CUDA_VISIBLE_DEVICES hides when picking the llama.cpp CUDA bundle (#13924)
b17be8b Daniel Han committed 1 month ago
GRPO: exclude masked rows from clip_ratio metrics for one-token completions (#13883)
28e8ad8 Daniel Han committed 1 month ago
Opt-in token-balanced and token-budget SFT micro-batches (#13878)
eac949c Daniel Han committed 1 month ago
Build RoPE cos/sin caches only on the device that uses them (#13855)
aa0b22b Daniel Han committed 1 month ago
GRPO packed logprobs: an OOM caps the packed length instead of disabling packing (#13877)
b2ea078 Daniel Han committed 1 month ago
Context parallel: Ulysses backend with packing and varlen attention (#13848)
ace96a5 Daniel Han committed 1 month ago
Studio installer (Windows AMD): treat hipinfo output as already mask-filtered when picking the gfx target (#13925)
9c4fd2f Daniel Han committed 1 month ago
Studio: automatic image/video loads pick the freest GPU, and GPU picks respect what each card can run (#13928)
c1dc273 Daniel Han committed 1 month ago
GRPO: opt-in parking of optimizer state and gradients on host during rollout (#13866)
c7824b0 Daniel Han committed 1 month ago
Use side stream offload hooks for non-reentrant gradient checkpointing (#13872)
c6f69b9 Daniel Han committed 1 month ago
Add a Triton head_dim 512 attention kernel for Gemma-4 global layers (#13869)
7c4f20c Daniel Han committed 1 month ago
RMSNorm: opt-in saving of the norm output instead of its input (UNSLOTH_RMSNORM_SAVE_OUTPUT) (#13868)
8c9dc1a Daniel Han committed 1 month ago
Refuse an explicit offload_layers swap under FSDP / DeepSpeed (#13857)
efcaebe Daniel Han committed 1 month ago
Prebuilt CUDA wheels: pin NVIDIA's re-published CUDA 13.4.1 manifest (win_arm64) (#13954)
8ca1b8f Daniel Han committed 1 month ago