Skip to content

armgit / kokon

Public Imported from GitHub
0

Edit photos by describing the change, on your own GPU. A free, open-source alternative to Nano Banana and ChatGPT image editing: photos stay on your computer, no daily limit. One installer for Windows, Mac and Linux sets up the best open model (Qwen-Image 2.1). MCP server for AI agents.

Mac engine: fp16-operand attention by default

Gabriele Lozuponed04edcb8af0c
diff --git a/engine-mac/README.md b/engine-mac/README.md
index 7f0d5d8..f4a2888 100644
--- a/engine-mac/README.md
+++ b/engine-mac/README.md
@@ -166,8 +166,9 @@ Mac and shape (`~/Library/Caches/kokon-engine-mac/balance.json`), so the first s
**1024² image, M2 Pro, unsloth's DiT + texture-fix VAE (`generate`), 2026-10-25: 79.9 s** (text encoder 2.0 s, step 1
10.2 s, steps 2–8 8.8–9.1 s, VAE 5.2 s); at the start of this pass 100.2 s; Kokon's sd.cpp ~4.5 min. With
-`KOKON_ATTENTION=half` (fp16 operands in the flash kernel): 76.0 s, the velocity within 6.0e-4 of fp64 at 256²
-(fp32 kernel 1.0e-5); the two 1024² images differ by 0.25 levels on average, 0.18 % of values by more than 8.
+fp16 operands in the flash kernel, **the default since 2026-10-25** (`KOKON_ATTENTION=fp32` for the checks): 76.0 s,
+the velocity within 6.0e-4 of fp64 at 256² (fp32 kernel 1.0e-5; bf16 tools ~1e-2); the two 1024² images differ by 0.25
+levels on average, 0.18 % of values by more than 8. The accuracy figures above are the fp32 kernel's.
**VAE decoder** (`check-vae`, against `tools/reference_vae.py`, diffusers fp32): within 1.1e-5; 1024² in 2.75 s. Its
graph builds and compiles in 0.3 s, but its first run at a size costs ~3.7 s more, GPU time (~1 s of CPU): it cannot
diff --git a/engine-mac/Sources/KokonEngine/Kernels/attention.metal b/engine-mac/Sources/KokonEngine/Kernels/attention.metal
index 077de4e..4158912 100644
--- a/engine-mac/Sources/KokonEngine/Kernels/attention.metal
+++ b/engine-mac/Sources/KokonEngine/Kernels/attention.metal
@@ -150,9 +150,9 @@ kernel void flash_attention(device const float *q [[buffer(0)]], device const fl
}
// The same with q, k, v and the probabilities in fp16 (scores, softmax and O accumulated in fp32; K and V tiles of 64
-// keys side by side in 32 KB), `KOKON_ATTENTION=half`: 94 ms against 120 at 1024² (M2 Pro), head 0 within 6.3e-4 of
-// exact, the velocity within 6.0e-4 of fp64 at 256² (fp32 kernel 1.0e-5). Only V and the probabilities in fp16
-// (q·k fp32): 118 ms, 2.2e-4, so not kept. Off until decided.
+// keys side by side in 32 KB), the engine's default (`KOKON_ATTENTION=fp32` for the one above): 94 ms against 120 at
+// 1024² (M2 Pro), head 0 within 6.3e-4 of exact, the velocity within 6.0e-4 of fp64 at 256² (fp32 kernel 1.0e-5).
+// Only V and the probabilities in fp16 (q·k fp32): 118 ms, 2.2e-4, so not kept.
static inline thread half2 &elementsHalf(thread simdgroup_half8x8 &m) {
return *(thread half2 *)&m.thread_elements();
}
diff --git a/engine-mac/Sources/KokonEngine/QwenDiT.swift b/engine-mac/Sources/KokonEngine/QwenDiT.swift
index b70abff..e1e59b5 100644
--- a/engine-mac/Sources/KokonEngine/QwenDiT.swift
+++ b/engine-mac/Sources/KokonEngine/QwenDiT.swift
@@ -458,10 +458,11 @@ public final class QwenDiT {
}
}
- /// Attention through `Kernels/attention.metal` (default) or MPSGraph (`KOKON_ATTENTION=graph`, the check);
- /// `KOKON_ATTENTION=half`: the kernel with fp16 operands (faster, 6e-4 of fp64; not the default until decided).
+ /// Attention through `Kernels/attention.metal` or MPSGraph (`KOKON_ATTENTION=graph`, the check). The kernel takes
+ /// fp16 operands by default (Gabriele 10-25: 76.0 s against 79.9 s per 1024² image, the velocity within 6e-4 of
+ /// fp64 instead of 1e-5); `KOKON_ATTENTION=fp32` keeps the fp32 kernel (the accuracy checks).
let graphAttention = ProcessInfo.processInfo.environment["KOKON_ATTENTION"] == "graph"
- let flashKernel = ProcessInfo.processInfo.environment["KOKON_ATTENTION"] == "half" ? "flash_attention_half" : "flash_attention"
+ let flashKernel = ProcessInfo.processInfo.environment["KOKON_ATTENTION"] == "fp32" ? "flash_attention" : "flash_attention_half"
struct FlashParams { var queries, keys, causal, stride: UInt32; var scale: Float; var headBase: UInt32 }
/// Queries `[queries, queries + count)` against keys `[0, keys)`, into the same rows of `attended`. The target's