Edit photos by describing the change, on your own GPU. A free, open-source alternative to Nano Banana and ChatGPT image editing: photos stay on your computer, no daily limit. One installer for Windows, Mac and Linux sets up the best open model (Qwen-Image 2.1). MCP server for AI agents.
TypeScript ai-photo-editorcomfyui-alternativegenerative-fillimage-editingimage-generationinpaintinglinuxlocal-aimacosmcpmcp-servernano-banana-alternativeofflinephoto-editorprivacyqwen-imageqwen-image-2-1stable-diffusion-cppvulkanwindows
Mac engine: VAE up blocks as single operations, the last two in tiles
Gabriele Lozupone1dafe36b85ec
diff --git a/engine-mac/README.md b/engine-mac/README.mdindex 39d95a4..685e7dc 100644--- a/engine-mac/README.md+++ b/engine-mac/README.md@@ -253,10 +253,10 @@ cancels within a DiT layer (sd-server answers 409 and qls-core restarts it). A nworking memory in ~0.5 s.What stays loaded between images depends on the Mac (`QwenPipeline.keep*`): from 32 GB everything; below, only the-files' mappings, since on the M2 Pro 16 GB the compiled VAE keeps 7–8 GB of GPU memory after a 1024² decode. With-everything kept there the second image took 121–129 s (pages evicted and read again); now, through the API-(`tools/api_check.py`, 2026-10-25): **first 1024² image 71.1 s, the next ones 73.7 s**, 1.1 GB of footprint while idle,-peak 9.6 GB during the VAE's decode (the one-shot CLI peaks the same; a tiled decode is the next step for 16 GB Macs).+files' mappings. Through the API (`tools/api_check.py`, M2 Pro 16 GB, 1024², 2026-10-25), with the VAE as single+operations and tiles (below): **first image 67.0 s, the next ones 63.9–64.5 s, peak 5.0 GB** (before it: 71.1 s, 73.7 s,+peak 9.6 GB). Keeping the compiled VAE between images now costs only 1.7 GB, but it did not pay on 16 GB: its decode+3.2 s instead of 4.0, the images 68.4–70.3 s (with the DiT kept too 70.8 / 68.3 s; single runs in a row, not alternated).## Per-chip paths@@ -301,9 +301,25 @@ MFA's GEMM (libnnc, 48 × 48 blocks, 2 × 2 SIMD groups) was tried against MPS:that macOS 27's run-time compiler refuses (libnnc ships prebuilt metallibs for it), and without them it ran at3.6–4.0 TFLOP/s against MPS's 5.1–5.6 at 1472 × 4096 × 4096, so MPS stays.-**VAE decoder** (`check-vae`, against `tools/reference_vae.py`, diffusers fp32): within 1.1e-5; 1024² in 2.75 s. Its-graph builds and compiles in 0.3 s, but its first run at a size costs ~3.7 s more, GPU time (~1 s of CPU): it cannot-hide under the DiT, so the pipeline keeps its VAE between images and a long-lived engine pays it once per size.+**VAE decoder** (`check-vae`, against `tools/reference_vae.py`, diffusers fp32): within 1.0–1.4e-5 at 256² and 768²,+1 level off at most. As one MPSGraph it took 2.78 s at 1024² (first run 4.5–5.0 s) but **kept 8.5 GB after the decode**+(peak 9.7 GB): a compiled graph keeps every intermediate tensor (up blocks 1 and 2 alone 2.4 and 4.1 GB). A graph that is+a single convolution keeps nothing beyond the buffers it is given, so the up blocks now run as single operations: each+convolution a graph with its bias, skip and shortcut fused and the weights as inputs (as fast as constants, shared across+sizes), norm + SiLU, nearest 2× and the DupUp shortcut as kernels (`vae.metal`), on pooled buffers, all in one command+buffer. Up blocks 3–4 and the head run in tiles of 288 (2 × 2 at 1024²) that read 10 pixels beyond their share, their+reach: the result is the whole graph's (1.2e-7; a 2-pixel halo is 17 levels off). M2 Pro, 1024², `time-vae`, alternated:++| VAE | decode (first) | kept after | peak |+|---|---|---|---|+| one graph | 2.78 s (4.5–5.0) | 8.5 GB | 9.7 GB |+| operations, untiled | 2.70 s (3.5) | 2.25 GB | 6.9 GB |+| **operations, blocks 3–4 in tiles of 288 (default)** | **3.0 s (3.25)** | **1.67 GB** | **4.6 GB** |+| operations, tiles of 144 | 3.9 s | 1.47 GB | 3.85 GB |+| graphs per block, every block tiled | 3.45 s | 3.8–5 GB | 6.25 GB |++The untiled peak grows with the image; the tiles keep it bounded. Buffers the GPU used leave the footprint up to a second+after their release (`time-vae` waits 1.5 s). `KOKON_VAE_OPS=0`, `KOKON_VAE_SPLIT`, `KOKON_VAE_TILES` to compare.fp16 up blocks (norms and the mid block kept fp32) are wrong (2.7e-1) and 3–11× slower only when MPSGraph may placethem on the Neural Engine (optimization level 1, its default); at level 0 they stay on the GPU and are right butgain little: all five 2.43 s against 2.77 at 1024², for 2.7e-3 of diffusers instead of 1.1e-5 (up to ~19 levels off).diff --git a/engine-mac/Sources/KokonEngine/Checks.swift b/engine-mac/Sources/KokonEngine/Checks.swiftindex 19d0662..9be0f87 100644--- a/engine-mac/Sources/KokonEngine/Checks.swift+++ b/engine-mac/Sources/KokonEngine/Checks.swift@@ -190,7 +190,8 @@ extension Checks {}extension Checks {- /// The VAE decoder against `tools/reference_vae.py`, then its time at `grid` (64 = 1024²).+ /// The VAE decoder against `tools/reference_vae.py` (in tiles when the image needs them), then tiles against the+ /// whole graph on a random latent of `grid` × 1.4 `grid` (default 40), at the default tile and at a small one.public static func vae(path: String, reference directory: String, timeGrid: Int?) throws {let meta = try JSONSerialization.jsonObject(with: Data(contentsOf: URL(fileURLWithPath: directory).appendingPathComponent("meta.json"))) as! [String: Any]@@ -198,30 +199,61 @@ extension Checks {let context = try MetalContext()let vae = try QwenVAE(path: path, context: context)let latent = try reference(directory, "latent"), want = try reference(directory, "image")- var started = DispatchTime.now().uptimeNanoseconds+ let started = DispatchTime.now().uptimeNanosecondslet got = try vae.decode(latent, height: grid, width: grid)let first = Double(DispatchTime.now().uptimeNanoseconds - started) * 1e-9let r = got.withUnsafeBufferPointer { compare($0.baseAddress!, want) }- print(String(format: "VAE %d² → %d²: rel %.2e, max |Δ| %.3g%@ (%.2f s with the graph build)", grid, grid * 16,- r.relative, r.maxAbs, r.finite ? "" : " NON-FINITE", first))- if let t = timeGrid {- started = DispatchTime.now().uptimeNanoseconds- try vae.prepare(height: t, width: t)- print(String(format: "build and compile %d²: %.2f s", t * 16, Double(DispatchTime.now().uptimeNanoseconds - started) * 1e-9))- let random = (0..<(64 * t * t)).map { Float(($0 * 7919) % 1000) / 500 - 1 }- func cpu() -> (user: Double, system: Double) {- var u = rusage()- getrusage(RUSAGE_SELF, &u)- return (Double(u.ru_utime.tv_sec) + Double(u.ru_utime.tv_usec) * 1e-6, Double(u.ru_stime.tv_sec) + Double(u.ru_stime.tv_usec) * 1e-6)- }- for run in 0..<2 {- started = DispatchTime.now().uptimeNanoseconds- let before = cpu()- _ = try vae.decode(random, height: t, width: t)- let after = cpu()- print(String(format: "decode %d² (run %d): %.2f s (CPU user %.2f s, system %.2f s)", t * 16, run + 1,- Double(DispatchTime.now().uptimeNanoseconds - started) * 1e-9, after.user - before.user, after.system - before.system))- }+ print(String(format: "VAE %d² → %d² (%@): rel %.2e, max |Δ| %.3g, %d levels off at most%@ (%.2f s with the graph build)",+ grid, grid * 16, tiles(vae, grid, grid), r.relative, r.maxAbs, levels(got, want, grid * 16, grid * 16),+ r.finite ? "" : " NON-FINITE", first))+ let (h, w) = (timeGrid ?? 40, (timeGrid ?? 40) * 7 / 5)+ let random = (0..<(64 * h * w)).map { Float(($0 * 7919) % 1000) / 500 - 1 }+ let (saved, split) = (vae.tiles, vae.split)+ (vae.tiles, vae.split) = ([], [])+ let whole = try vae.decode(random, height: h, width: w)+ for small in [false, true] {+ // The small tiles: 3–4 times each stage's halo, many seams.+ vae.split = split+ vae.tiles = small ? vae.stages.map { $0.stage.halo * 4 } : saved+ let tiled = try vae.decode(random, height: h, width: w)+ let r = tiled.withUnsafeBufferPointer { compare($0.baseAddress!, whole) }+ print(String(format: "%@ against one graph, %d × %d: rel %.2e, max |Δ| %.3g, %d levels off at most",+ tiles(vae, h, w), h * 16, w * 16, r.relative, r.maxAbs, levels(tiled, whole, h * 16, w * 16)))+ }+ (vae.tiles, vae.split) = (saved, split)+ }++ static func tiles(_ vae: QwenVAE, _ h: Int, _ w: Int) -> String {+ vae.stages.map { stage, tile in+ let scale = QwenVAE.Stage.scale(stage.first)+ guard tile > 2 * stage.halo, scale * max(h, w) > tile else { return "\(stage)" }+ let (y, x) = (QwenVAE.spans(scale * h, tile: tile, halo: stage.halo).count,+ QwenVAE.spans(scale * w, tile: tile, halo: stage.halo).count)+ return "\(stage) in \(y)×\(x) tiles of \(tile)"+ }.joined(separator: ", ")+ }++ /// The largest difference between two decoded images once made 8-bit.+ static func levels(_ a: [Float], _ b: [Float], _ height: Int, _ width: Int) -> Int {+ zip(QwenPipeline.pixels(a, width: width, height: height), QwenPipeline.pixels(b, width: width, height: height))+ .reduce(0) { max($0, abs(Int($1.0) - Int($1.1))) }+ }++ /// The decoder's time and memory at `grid` (64 = 1024²) as the pipeline runs it (`KOKON_VAE_TILE` to choose): the+ /// first decode with its graph builds, then two more; the footprint 1.5 s after each, and the process's peak.+ public static func timeVAE(path: String, grid: Int) throws {+ let context = try MetalContext()+ let vae = try QwenVAE(path: path, context: context)+ let random = (0..<(64 * grid * grid)).map { Float(($0 * 7919) % 1000) / 500 - 1 }+ print("VAE \(grid * 16)² in \(tiles(vae, grid, grid)), \(String(format: "footprint %.2f GB", MemoryUsage.now().footprint))")+ for run in 0..<3 {+ let started = DispatchTime.now().uptimeNanoseconds+ try autoreleasepool { _ = try vae.decode(random, height: grid, width: grid) }+ let seconds = Double(DispatchTime.now().uptimeNanoseconds - started) * 1e-9+ // Buffers the GPU used leave the footprint within a second of their release.+ Thread.sleep(forTimeInterval: 1.5)+ let m = MemoryUsage.now()+ print(String(format: "decode %d: %.2f s, then footprint %.2f GB (peak %.2f GB)", run + 1, seconds, m.footprint, m.footprintPeak))}}}diff --git a/engine-mac/Sources/KokonEngine/Kernels/vae.metal b/engine-mac/Sources/KokonEngine/Kernels/vae.metalnew file mode 100644index 0000000..5e2d0d1--- /dev/null+++ b/engine-mac/Sources/KokonEngine/Kernels/vae.metal@@ -0,0 +1,52 @@+// The VAE decoder's tiles (QwenVAE.swift): a window of channel planes copied between the whole tensor and a tile,+// on the GPU so that every tile of a decode goes in one command buffer.++struct WindowCopy {+ uint channels, rows, cols;+ uint srcH, srcW, srcY, srcX;+ uint dstH, dstW, dstY, dstX;+};++/// dst[c, dstY + y, dstX + x] = src[c, srcY + y, srcX + x] for y < rows, x < cols, c < channels.+kernel void window_copy(device const float *src [[buffer(0)]], device float *dst [[buffer(1)]],+ constant WindowCopy &p [[buffer(2)]], uint3 gid [[thread_position_in_grid]]) {+ if (gid.x >= p.cols || gid.y >= p.rows || gid.z >= p.channels) return;+ ulong s = (ulong(gid.z) * p.srcH + p.srcY + gid.y) * p.srcW + p.srcX + gid.x;+ ulong d = (ulong(gid.z) * p.dstH + p.dstY + gid.y) * p.dstW + p.dstX + gid.x;+ dst[d] = src[s];+}++/// QwenImage21RMS_norm then SiLU, one pixel a thread over the `p.x` channel planes of `p.y` pixels: x / max(‖x‖,+/// 1e-12) · √C · γ, then v · sigmoid(v). `x` and `out` may be the same buffer.+kernel void vae_norm_silu(device const float *x [[buffer(0)]], device float *out [[buffer(1)]],+ device const float *gamma [[buffer(2)]], constant uint2 &p [[buffer(3)]],+ uint i [[thread_position_in_grid]]) {+ if (i >= p.y) return;+ float sum = 0;+ for (uint c = 0; c < p.x; c++) { float v = x[ulong(c) * p.y + i]; sum += v * v; }+ float scale = sqrt(float(p.x)) / max(sqrt(sum), 1e-12f);+ for (uint c = 0; c < p.x; c++) {+ ulong at = ulong(c) * p.y + i;+ float v = x[at] * scale * gamma[c];+ out[at] = v / (1.0f + exp(-v));+ }+}++/// Nearest 2× upsampling of `p.x` planes of `p.y` × `p.z`.+kernel void vae_nearest_up(device const float *x [[buffer(0)]], device float *out [[buffer(1)]],+ constant uint3 &p [[buffer(2)]], uint3 gid [[thread_position_in_grid]]) {+ if (gid.x >= 2 * p.z || gid.y >= 2 * p.y || gid.z >= p.x) return;+ out[(ulong(gid.z) * 2 * p.y + gid.y) * 2 * p.z + gid.x] = x[(ulong(gid.z) * p.y + gid.y / 2) * p.z + gid.x / 2];+}++struct DupUp { uint inC, outC, ft, h, w; };++/// QwenImage21DupUp3D on a first chunk, added to the upsampler's output `out` [outC, 2h, 2w]: output channel o's+/// pixel (2y + i, 2x + j) repeats input channel q / repeats at (y, x), q = ((o · ft + ft − 1) · 2 + i) · 2 + j.+kernel void vae_dup_up_add(device float *out [[buffer(0)]], device const float *x [[buffer(1)]],+ constant DupUp &p [[buffer(2)]], uint3 gid [[thread_position_in_grid]]) {+ if (gid.x >= 2 * p.w || gid.y >= 2 * p.h || gid.z >= p.outC) return;+ uint repeats = p.outC * p.ft * 4 / p.inC;+ uint q = ((gid.z * p.ft + p.ft - 1) * 2 + (gid.y & 1)) * 2 + (gid.x & 1);+ out[(ulong(gid.z) * 2 * p.h + gid.y) * 2 * p.w + gid.x] += x[(ulong(q / repeats) * p.h + gid.y / 2) * p.w + gid.x / 2];+}diff --git a/engine-mac/Sources/KokonEngine/QwenVAE.swift b/engine-mac/Sources/KokonEngine/QwenVAE.swiftindex 5cff9af..c4d83dc 100644--- a/engine-mac/Sources/KokonEngine/QwenVAE.swift+++ b/engine-mac/Sources/KokonEngine/QwenVAE.swift@@ -3,9 +3,11 @@ import Metalimport MetalPerformanceShadersimport MetalPerformanceShadersGraph-/// **The Qwen-Image 2.1 VAE decoder** (`AutoencoderKLQwenImage21.decode`, diffusers 0.42), as one MPSGraph per-/// latent size, fp32. For a single image every causal 3D convolution of the Wan 2.2 design is a 2D one and the-/// temporal convolutions never run (first chunk), so the graph is a plain 2D network:+/// **The Qwen-Image 2.1 VAE decoder** (`AutoencoderKLQwenImage21.decode`, diffusers 0.42), fp32: the latent's+/// conversion and the mid block as one MPSGraph on the whole latent, the up blocks as single operations (`ops`), the+/// last two in tiles (`split`, `tiles`); `build` writes the whole decoder as one graph, the reference. For a single+/// image every causal 3D convolution of the Wan 2.2 design is a 2D one and the temporal convolutions never run (first+/// chunk), so the decoder is a plain 2D network:////// latent · std + mean ─ post_quant_conv (1×1) ─ conv_in (3×3, 64 → 1152)/// mid: residual ─ single-head attention over all pixels ─ residual@@ -35,7 +37,8 @@ public final class QwenVAE {let context: MetalContextlet file: SafetensorsFile- /// One compiled graph per latent size, compiled to the end (`waitForCompilationCompletion`): 2.7 s at 1024² (M2+ /// Compiled graphs: the mid stage per latent size, each convolution per input size. The whole decoder as one graph,+ /// compiled to the end (`waitForCompilationCompletion`): 2.7 s at 1024² (M2/// Pro), mostly the shader compiler, which otherwise ran inside the first decode (6.4 s against 2.75 s; with it/// up front the first decode is 4.15 s). Beside the DiT it slows the steps far more than it saves: the pipeline/// keeps its VAE between images, and an idle engine can prepare ahead. fp16 up blocks (norms and the mid block in@@ -44,6 +47,7 @@ public final class QwenVAE {/// 1024², the first two 2.65, for 2.2–2.7e-3 of diffusers instead of 1.1e-5 (M2 Pro, `check-vae`, 2026-10-25). The/// VAE stays fp32.private var executables: [String: MPSGraphExecutable] = [:]+ private var feedOrders: [String: [Int]] = [:]private let lock = NSLock()public init(path: String, context: MetalContext) throws {@@ -54,29 +58,318 @@ public final class QwenVAE {}}+ /// Where the decoder is cut into graphs, as up block numbers (0: after the mid block; `KOKON_VAE_SPLIT`, e.g.+ /// `0,1,2,3`), and the side of each stage's tiles at its input (0: the whole input; `KOKON_VAE_TILES`, one per+ /// stage). The first stage runs on the whole latent (the mid block's attention sees every pixel); the others may run+ /// on tiles that read `halo` pixels beyond their share, each a graph compiled once for every image size.+ /// By default up blocks 0–2 whole and 3–4 with the head in tiles of 288 (2 × 2 at 1024²): M2 Pro 1024²,+ /// alternated rounds, 2026-10-25: 3.0 s a decode (first 3.25), 1.67 GB kept after it, peak 4.6 GB; one graph 2.8 s+ /// (first 4.5–5.0), 8.5 GB kept, peak 9.7; the up blocks untiled 2.7 s, 2.25 GB kept, but a peak of 6.9 GB that+ /// grows with the image; tiles of 144 3.9 s, peak 3.85.+ public var split = QwenVAE.numbers("KOKON_VAE_SPLIT") ?? [0, 3]+ public var tiles = QwenVAE.numbers("KOKON_VAE_TILES") ?? [0, 0, 288]+ static func numbers(_ key: String) -> [Int]? {+ ProcessInfo.processInfo.environment[key].map { $0.split(separator: ",").compactMap { Int($0) } }+ }++ /// A decoder stage: up blocks `first..<end`, after the latent's conversion and the mid block when `first` is −1,+ /// with the head when `end` is 5.+ public struct Stage: Hashable, Sendable, CustomStringConvertible {+ public let first, end: Int+ public static let whole = Stage(first: -1, end: 5)+ /// Channels at the input of up block `b` (5: the image's 4) and that size relative to the latent.+ static func channels(_ b: Int) -> Int { b < 0 ? 64 : b < 5 ? QwenVAE.dims[b] : 4 }+ static func scale(_ b: Int) -> Int { b < 0 ? 1 : 1 << min(b, 4) }+ /// Output pixels per input pixel along a side.+ var up: Int { Stage.scale(end) / Stage.scale(first) }+ /// How far the stage's 3×3 convolutions reach, in its input's pixels: 6 in each up block at its size, the+ /// upsampler's at twice that, the head's at the image's; a tile needs this much beyond its share to give what+ /// the whole graph computes (up block 3 to the end: 6 + ½ + 3 + ½ = 10).+ var halo: Int {+ let at = { (b: Int) in Double(Stage.scale(b)) / Double(Stage.scale(first)) }+ var reach = 0.0+ for i in max(first, 0)..<end { reach += 6 / at(i) + (i < 4 ? 1 / at(i + 1) : 0) }+ if end == 5 { reach += 1 / at(5) }+ return Int(reach.rounded(.up))+ }+ public var description: String { "\(first)..<\(end)" }+ }++ /// The stages in order, with their tile sides.+ public var stages: [(stage: Stage, tile: Int)] {+ let bounds = [-1] + split.filter { $0 >= 0 && $0 < 5 } + [5]+ return zip(bounds, bounds.dropFirst()).enumerated().map { i, b in+ (Stage(first: b.0, end: b.1), b.0 >= 0 && i < tiles.count ? tiles[i] : 0)+ }+ }++ /// Up blocks as single operations on the engine's buffers (`KOKON_VAE_OPS=0`: as MPSGraph graphs). A compiled graph+ /// keeps every intermediate tensor after it runs: the whole decoder held 8.5 GB after a 1024² decode, its up blocks 1+ /// and 2 alone 2.4 and 4.1 GB; a graph that is one convolution (its bias, skip and shortcut fused) keeps nothing+ /// beyond the buffers it is given (M2 Pro, 2026-10-25). So each convolution is a graph, with weights as inputs+ /// (shared across blocks and sizes, as fast as constants), and the norms, the 2× upsampling and the DupUp shortcut+ /// are kernels (`vae.metal`), all on buffers from a pool that lives for one decode.+ public var ops = ProcessInfo.processInfo.environment["KOKON_VAE_OPS"] != "0"+/// Decodes a normalized latent `[64, h, w]` (the DiT's packed tokens transposed) into RGBA `[4, 16h, 16w]` in- /// [−1, 1].+ /// [−1, 1]. Every stage and tile goes in one command buffer.public func decode(_ latent: [Float], height h: Int, width w: Int) throws -> [Float] {- let executable = try prepare(height: h, width: w)- let input = MPSGraphTensorData(device: MPSGraphDevice(mtlDevice: context.device),- data: latent.withUnsafeBytes { Data($0) },- shape: [1, 64, NSNumber(value: h), NSNumber(value: w)], dataType: .float32)- guard let data = executable.run(with: context.queue, inputs: [input], results: nil, executionDescriptor: nil).first else {- throw MetalContext.Failure.kernel("VAE decode")+ var commands = MPSCommandBuffer(from: context.queue)+ let pool = BufferPool(context: context)+ let memory = ProcessInfo.processInfo.environment["KOKON_VAE_MEMORY"] != nil // waits after each stage to tell+ var current = try pool.take(latent.count)+ current.contents().copyMemory(from: latent, byteCount: 4 * latent.count)+ for (stage, tile) in stages {+ let (H, W) = (Stage.scale(stage.first) * h, Stage.scale(stage.first) * w)+ let next = try pool.take(Stage.channels(stage.end) * stage.up * H * stage.up * W)+ if tile > 2 * stage.halo, H > tile || W > tile {+ try encode(stage, tiles: tile, from: current, to: next, height: H, width: W, pool: pool, commands: commands)+ } else {+ try encode(stage, from: current, to: next, height: H, width: W, pool: pool, commands: commands)+ }+ pool.give(current)+ current = next+ if memory {+ commands.commit()+ commands.waitUntilCompleted()+ commands = MPSCommandBuffer(from: context.queue)+ print(String(format: " after %@: footprint %.2f GB, pool %.2f GB", stage.description, MemoryUsage.now().footprint,+ Double(pool.bytes) / Double(1 << 30)))+ }}+ commands.commit()+ commands.waitUntilCompleted()+ if let error = commands.error { throw MetalContext.Failure.kernel("VAE decode: \(error.localizedDescription)") }var out = [Float](repeating: 0, count: 4 * 16 * h * 16 * w)- out.withUnsafeMutableBytes { data.mpsndarray().readBytes($0.baseAddress!, strideBytes: nil) }+ out.withUnsafeMutableBytes { $0.copyMemory(from: UnsafeRawBufferPointer(start: current.contents(), count: $0.count)) }return out}- /// Builds and compiles the decoder for an `h` × `w` latent, once.+ /// Buffers for one decode, handed out again once given back: the command buffer runs in order, so a buffer's next+ /// user waits for its last one.+ final class BufferPool {+ let context: MetalContext+ private var free: [MTLBuffer] = []+ private(set) var bytes = 0+ init(context: MetalContext) { self.context = context }+ /// A buffer of at least `floats` floats: the smallest free one that fits, else a new one.+ func take(_ floats: Int) throws -> MTLBuffer {+ if let i = free.indices.filter({ free[$0].length >= 4 * floats }).min(by: { free[$0].length < free[$1].length }) {+ return free.remove(at: i)+ }+ bytes += 4 * floats+ return try context.scratch(4 * floats, label: "VAE")+ }+ func give(_ buffer: MTLBuffer) { free.append(buffer) }+ }++ private func data(_ buffer: MTLBuffer, _ shape: [Int]) -> MPSGraphTensorData {+ MPSGraphTensorData(buffer, shape: shape.map { NSNumber(value: $0) }, dataType: .float32)+ }++ /// `stage` on an `H` × `W` input, whole.+ private func encode(_ stage: Stage, from input: MTLBuffer, to output: MTLBuffer, height H: Int, width W: Int,+ pool: BufferPool, commands: MPSCommandBuffer) throws {+ if ops && stage.first >= 0 {+ return try encodeOperations(stage, from: input, to: output, height: H, width: W, pool: pool, commands: commands)+ }+ _ = try prepare(stage, height: H, width: W)+ .encode(to: commands, inputs: [data(input, [1, Stage.channels(stage.first), H, W])],+ results: [data(output, [1, Stage.channels(stage.end), stage.up * H, stage.up * W])], executionDescriptor: nil)+ }++ /// `stage` on `H` × `W` in tiles of side `tile` (the same graphs for all of them), each tile's share written to+ /// `output`.+ private func encode(_ stage: Stage, tiles tile: Int, from input: MTLBuffer, to output: MTLBuffer, height H: Int, width W: Int,+ pool: BufferPool, commands: MPSCommandBuffer) throws {+ let (ci, co, up) = (Stage.channels(stage.first), Stage.channels(stage.end), stage.up)+ let (th, tw) = (min(tile, H), min(tile, W))+ let tileIn = try pool.take(ci * th * tw), tileOut = try pool.take(co * up * th * up * tw)+ for y in QwenVAE.spans(H, tile: th, halo: stage.halo) {+ for x in QwenVAE.spans(W, tile: tw, halo: stage.halo) {+ try window(input, (H, W, y.start, x.start), tileIn, (th, tw, 0, 0), channels: ci, rows: th, cols: tw, commands: commands)+ try encode(stage, from: tileIn, to: tileOut, height: th, width: tw, pool: pool, commands: commands)+ try window(tileOut, (up * th, up * tw, up * (y.core - y.start), up * (x.core - x.start)), output,+ (up * H, up * W, up * y.core, up * x.core), channels: co, rows: up * y.length, cols: up * x.length,+ commands: commands)+ }+ }+ pool.give(tileIn)+ pool.give(tileOut)+ }++ // MARK: - up blocks as operations++ /// Up blocks `stage.first..<stage.end` (and the head) on `[C, H, W]` in `input`, as `QwenVAE.build` writes them.+ private func encodeOperations(_ stage: Stage, from input: MTLBuffer, to output: MTLBuffer, height H: Int, width W: Int,+ pool: BufferPool, commands: MPSCommandBuffer) throws {+ var (x, c, hh, ww) = (input, Stage.channels(stage.first), H, W)+ /// Gives `b` back unless it is the stage's input (the caller's).+ func done(_ b: MTLBuffer) { if b !== input { pool.give(b) } }+ for i in stage.first..<stage.end {+ let (inC, outC) = (QwenVAE.dims[i], QwenVAE.dims[i + 1])+ let blockInput = x+ for j in 0..<3 {+ let name = "decoder.upsamples.\(i).upsamples.\(j)"+ let t = try pool.take(c * hh * ww)+ try normSilu(x, to: t, gamma: name + ".residual.0.gamma", channels: c, pixels: hh * ww, commands: commands)+ let u = try pool.take(outC * hh * ww)+ try convolution(t, name + ".residual.2", inC: c, outC: outC, k: 3, height: hh, width: ww, to: u, commands: commands)+ pool.give(t)+ try normSilu(u, to: u, gamma: name + ".residual.3.gamma", channels: outC, pixels: hh * ww, commands: commands)+ let y = try pool.take(outC * hh * ww)+ try convolution(u, name + ".residual.6", inC: outC, outC: outC, k: 3, height: hh, width: ww, to: y,+ skip: (x, c == outC ? nil : name + ".shortcut", c), commands: commands)+ pool.give(u)+ if x !== blockInput { done(x) }+ (x, c) = (y, outC)+ }+ if i < 4 {+ let up = try pool.take(outC * 4 * hh * ww)+ try dispatch("vae_nearest_up", [x, up], [UInt32(outC), UInt32(hh), UInt32(ww)],+ grid: MTLSize(width: 2 * ww, height: 2 * hh, depth: outC), commands: commands)+ done(x)+ let y = i == stage.end - 1 ? output : try pool.take(outC * 4 * hh * ww)+ try convolution(up, "decoder.upsamples.\(i).upsamples.3.resample.1", inC: outC, outC: outC, k: 3,+ height: 2 * hh, width: 2 * ww, to: y, commands: commands)+ pool.give(up)+ try dispatch("vae_dup_up_add", [y, blockInput], [UInt32(inC), UInt32(outC), UInt32(QwenVAE.temporal[i]), UInt32(hh), UInt32(ww)],+ grid: MTLSize(width: 2 * ww, height: 2 * hh, depth: outC), commands: commands)+ done(blockInput)+ (x, hh, ww) = (y, 2 * hh, 2 * ww)+ } else if blockInput !== x {+ done(blockInput)+ }+ }+ if stage.end == 5 {+ let t = try pool.take(c * hh * ww)+ try normSilu(x, to: t, gamma: "decoder.head.0.gamma", channels: c, pixels: hh * ww, commands: commands)+ done(x)+ try convolution(t, "decoder.head.2", inC: c, outC: 4, k: 3, height: hh, width: ww, to: output, clamp: true, commands: commands)+ pool.give(t)+ } else if x !== output {+ // A stage that ends before the head ends on an upsampler, which writes `output` itself.+ preconditionFailure("VAE stage \(stage) did not end in its output")+ }+ }++ /// Weights as fp32 buffers, each read from the file once.+ private var weightBuffers: [String: MTLBuffer] = [:]+ private func weight(_ name: String) throws -> MTLBuffer {+ try lock.withLock {+ if let b = weightBuffers[name] { return b }+ let values = try file.floats(name)+ let b = try context.scratch(4 * values.count, label: name)+ b.contents().copyMemory(from: values, byteCount: 4 * values.count)+ weightBuffers[name] = b+ return b+ }+ }++ private func dispatch(_ kernel: String, _ buffers: [MTLBuffer], _ parameters: [UInt32], grid: MTLSize,+ commands: MPSCommandBuffer) throws {+ let pipeline = try context.pipeline(kernel)+ var p = parameters+ let e = commands.rootCommandBuffer.makeComputeCommandEncoder()!+ e.setComputePipelineState(pipeline)+ for (i, b) in buffers.enumerated() { e.setBuffer(b, offset: 0, index: i) }+ e.setBytes(&p, length: 4 * p.count, index: buffers.count)+ let width = min(64, grid.width)+ e.dispatchThreads(grid, threadsPerThreadgroup: MTLSize(width: width, height: max(1, min(grid.height, 256 / width)), depth: 1))+ e.endEncoding()+ }++ /// QwenImage21RMS_norm then SiLU, per pixel over `channels` planes (`x` and `out` may be the same).+ private func normSilu(_ x: MTLBuffer, to out: MTLBuffer, gamma: String, channels: Int, pixels: Int, commands: MPSCommandBuffer) throws {+ try dispatch("vae_norm_silu", [x, out, try weight(gamma)], [UInt32(channels), UInt32(pixels)],+ grid: MTLSize(width: pixels, height: 1, depth: 1), commands: commands)+ }++ /// `channels` planes: `rows` × `cols` from (`src`, its size, at y, x) to (`dst`, its size, at y, x).+ private func window(_ src: MTLBuffer, _ srcSize: (Int, Int, Int, Int), _ dst: MTLBuffer, _ dstSize: (Int, Int, Int, Int),+ channels: Int, rows: Int, cols: Int, commands: MPSCommandBuffer) throws {+ try dispatch("window_copy", [src, dst], [channels, rows, cols, srcSize.0, srcSize.1, srcSize.2, srcSize.3,+ dstSize.0, dstSize.1, dstSize.2, dstSize.3].map { UInt32($0) },+ grid: MTLSize(width: cols, height: rows, depth: channels), commands: commands)+ }++ /// `name` (a k × k convolution and its bias) on `[inC, h, w]`, plus `skip` (the residual block's input, through+ /// its 1×1 shortcut when named), clamped to [−1, 1] for the head.+ private func convolution(_ x: MTLBuffer, _ name: String, inC: Int, outC: Int, k: Int, height h: Int, width w: Int,+ to out: MTLBuffer, skip: (buffer: MTLBuffer, shortcut: String?, channels: Int)? = nil,+ clamp: Bool = false, commands: MPSCommandBuffer) throws {+ let kind = skip == nil ? "" : skip!.shortcut == nil ? "+x" : "+1x1(\(skip!.channels))"+ let key = "conv \(inC)→\(outC) \(k)×\(k)\(kind)\(clamp ? " clamp" : "") \(h)×\(w)"+ let (executable, order) = try lock.withLock { () throws -> (MPSGraphExecutable, [Int]) in+ if let e = executables[key], let order = feedOrders[key] { return (e, order) }+ let g = MPSGraph()+ func n(_ v: [Int]) -> [NSNumber] { v.map { NSNumber(value: $0) } }+ var feeds: [MPSGraphTensor: MPSGraphShapedType] = [:], placeholders: [MPSGraphTensor] = []+ func input(_ shape: [Int]) -> MPSGraphTensor {+ let t = g.placeholder(shape: n(shape), dataType: .float32, name: nil)+ feeds[t] = MPSGraphShapedType(shape: n(shape), dataType: .float32)+ placeholders.append(t)+ return t+ }+ func conv(_ x: MPSGraphTensor, _ inC: Int, _ k: Int) -> MPSGraphTensor {+ let d = MPSGraphConvolution2DOpDescriptor(strideInX: 1, strideInY: 1, dilationRateInX: 1, dilationRateInY: 1,+ groups: 1, paddingLeft: k / 2, paddingRight: k / 2, paddingTop: k / 2,+ paddingBottom: k / 2, paddingStyle: .explicit, dataLayout: .NCHW,+ weightsLayout: .OIHW)!+ let y = g.convolution2D(x, weights: input([outC, inC, k, k]), descriptor: d, name: nil)+ return g.addition(y, input([1, outC, 1, 1]), name: nil)+ }+ // Inputs in order: x, weight, bias, then skip, and the shortcut's weight and bias.+ let x = input([1, inC, h, w])+ var y = conv(x, inC, k)+ if let skip {+ let s = input([1, skip.channels, h, w])+ y = g.addition(y, skip.shortcut == nil ? s : conv(s, skip.channels, 1), name: nil)+ }+ if clamp {+ y = g.clamp(y, min: g.constant(-1, dataType: .float32), max: g.constant(1, dataType: .float32), name: nil)+ }+ let descriptor = MPSGraphCompilationDescriptor()+ descriptor.waitForCompilationCompletion = true+ let e = g.compile(with: MPSGraphDevice(mtlDevice: context.device), feeds: feeds, targetTensors: [y],+ targetOperations: nil, compilationDescriptor: descriptor)+ // The executable takes its inputs in its own order (`feedTensors`), not the placeholders'.+ let order = (e.feedTensors ?? placeholders).map { t in placeholders.firstIndex { $0 === t }! }+ executables[key] = e+ feedOrders[key] = order+ return (e, order)+ }+ var inputs = [data(x, [1, inC, h, w]), data(try weight(name + ".weight"), [outC, inC, k, k]),+ data(try weight(name + ".bias"), [1, outC, 1, 1])]+ if let skip {+ inputs.append(data(skip.buffer, [1, skip.channels, h, w]))+ if let shortcut = skip.shortcut {+ inputs += [data(try weight(shortcut + ".weight"), [outC, skip.channels, 1, 1]), data(try weight(shortcut + ".bias"), [1, outC, 1, 1])]+ }+ }+ _ = executable.encode(to: commands, inputs: order.map { inputs[$0] },+ results: [data(out, [1, outC, h, w])], executionDescriptor: nil)+ }++ /// Tiles of side `tile` over `n` pixels: each one's start, and the share it gives (`core`, `length`) at least+ /// `halo` pixels from its edges unless that edge is the image's.+ static func spans(_ n: Int, tile: Int, halo: Int) -> [(start: Int, core: Int, length: Int)] {+ if n <= tile { return [(0, 0, n)] }+ let k = (n + tile - 2 * halo - 1) / (tile - 2 * halo)+ return (0..<k).map { i in+ let (a, b) = (n * i / k, n * (i + 1) / k)+ return (min(max(a - halo, 0), n - tile), a, b - a)+ }+ }++ /// Builds and compiles a stage of the decoder for an `h` × `w` input, once.@discardableResult- public func prepare(height h: Int, width w: Int) throws -> MPSGraphExecutable {+ public func prepare(_ stage: Stage = .whole, height h: Int, width w: Int) throws -> MPSGraphExecutable {try lock.withLock {- let key = "\(h)×\(w)"+ let key = "\(stage) \(h)×\(w)"if let e = executables[key] { return e }- let built = try build(height: h, width: w)- let shape = MPSGraphShapedType(shape: [1, 64, NSNumber(value: h), NSNumber(value: w)], dataType: .float32)+ let built = try build(stage, height: h, width: w)+ let shape = MPSGraphShapedType(shape: built.input.shape, dataType: .float32)let descriptor = MPSGraphCompilationDescriptor()descriptor.waitForCompilationCompletion = truelet e = built.graph.compile(with: MPSGraphDevice(mtlDevice: context.device), feeds: [built.input: shape],@@ -88,7 +381,8 @@ public final class QwenVAE {// MARK: - the graph- private func build(height h: Int, width w: Int) throws -> (graph: MPSGraph, input: MPSGraphTensor, output: MPSGraphTensor) {+ /// The graph of `stage` for an `h` × `w` input.+ private func build(_ stage: Stage, height h: Int, width w: Int) throws -> (graph: MPSGraph, input: MPSGraphTensor, output: MPSGraphTensor) {let g = MPSGraph()func n(_ values: [Int]) -> [NSNumber] { values.map { NSNumber(value: $0) } }func constant(_ values: [Float], _ shape: [Int]) -> MPSGraphTensor {@@ -152,17 +446,24 @@ public final class QwenVAE {return g.reshape(y, shape: n([1, outC, 2 * hh, 2 * ww]), name: nil)}- let input = g.placeholder(shape: n([1, 64, h, w]), dataType: .float32, name: "latent")- var x = g.addition(g.multiplication(input, constant(QwenVAE.latentsStd, [1, 64, 1, 1]), name: nil),- constant(QwenVAE.latentsMean, [1, 64, 1, 1]), name: nil)- x = try conv(x, "conv2", 64, 64, 1) // post_quant_conv+ let input: MPSGraphTensor+ var x: MPSGraphTensorlet dims = QwenVAE.dims- x = try conv(x, "decoder.conv1", 64, dims[0], 3) // conv_in- x = try residual(x, "decoder.middle.0", dims[0], dims[0]) // mid_block.resnets.0- x = try attention(x, "decoder.middle.1", dims[0], h * w, h, w) // mid_block.attentions.0- x = try residual(x, "decoder.middle.2", dims[0], dims[0]) // mid_block.resnets.1var (hh, ww) = (h, w)- for i in 0..<5 { // up_blocks.i+ if stage.first >= 0 {+ input = g.placeholder(shape: n([1, Stage.channels(stage.first), h, w]), dataType: .float32, name: "input")+ x = input+ } else {+ input = g.placeholder(shape: n([1, 64, h, w]), dataType: .float32, name: "latent")+ x = g.addition(g.multiplication(input, constant(QwenVAE.latentsStd, [1, 64, 1, 1]), name: nil),+ constant(QwenVAE.latentsMean, [1, 64, 1, 1]), name: nil)+ x = try conv(x, "conv2", 64, 64, 1) // post_quant_conv+ x = try conv(x, "decoder.conv1", 64, dims[0], 3) // conv_in+ x = try residual(x, "decoder.middle.0", dims[0], dims[0]) // mid_block.resnets.0+ x = try attention(x, "decoder.middle.1", dims[0], h * w, h, w) // mid_block.attentions.0+ x = try residual(x, "decoder.middle.2", dims[0], dims[0]) // mid_block.resnets.1+ }+ for i in max(stage.first, 0)..<stage.end { // up_blocks.ilet (inC, outC) = (dims[i], dims[i + 1])let blockInput = xvar c = inC@@ -178,6 +479,7 @@ public final class QwenVAE {(hh, ww) = (2 * hh, 2 * ww)}}+ if stage.end < 5 { return (g, input, x) }x = silu(try norm(x, "decoder.head.0.gamma", dims[5])) // norm_outx = try conv(x, "decoder.head.2", dims[5], 4, 3) // conv_outx = g.clamp(x, min: g.constant(-1, dataType: .float32), max: g.constant(1, dataType: .float32), name: nil)diff --git a/engine-mac/Sources/kokon-engine-mac/main.swift b/engine-mac/Sources/kokon-engine-mac/main.swiftindex a6652ab..1a6a7f6 100644--- a/engine-mac/Sources/kokon-engine-mac/main.swift+++ b/engine-mac/Sources/kokon-engine-mac/main.swift@@ -8,7 +8,8 @@ kokon-engine-mac: Kokon's Apple Silicon engine (in development)check-dequant <file.gguf> [name...] GPU unpacking against the CPU referencecheck-dit <dit.gguf> <reference dir> one DiT evaluation against tools/reference_dit.pytime-dit <dit.gguf> [grid] [text] [runs] DiT evaluations on random inputs (grid 64 = 1024²)- check-vae <vae.safetensors> <reference dir> [grid] the VAE decoder against tools/reference_vae.py+ check-vae <vae.safetensors> <reference dir> [grid] the VAE decoder against tools/reference_vae.py, tiles against the whole graph+ time-vae <vae.safetensors> [grid] the VAE decoder's time and memory (64 = 1024²; KOKON_VAE_TILE)check-tokenizer <encoder.gguf> <fixture.json> the tokenizer against tools/reference_tokenizer.pycheck-text <encoder.gguf> <reference dir> the text encoder against tools/reference_text_encoder.pycheck-generate <encoder.gguf> <dit.gguf> <vae.safetensors> <reference dir> [text reference dir]@@ -70,6 +71,9 @@ do {case "check-vae":guard arguments.count >= 3 else { fail(usage) }try Checks.vae(path: arguments[1], reference: arguments[2], timeGrid: arguments.count > 3 ? Int(arguments[3]) : nil)+ case "time-vae":+ guard arguments.count >= 2 else { fail(usage) }+ try Checks.timeVAE(path: arguments[1], grid: arguments.count > 2 ? Int(arguments[2]) ?? 64 : 64)case "check-tokenizer":guard arguments.count == 3 else { fail(usage) }try Checks.tokenizer(path: arguments[1], fixture: arguments[2])diff --git a/engine-mac/Tests/KokonEngineTests/QwenPipelineTests.swift b/engine-mac/Tests/KokonEngineTests/QwenPipelineTests.swiftindex 0fbc911..f0b6151 100644--- a/engine-mac/Tests/KokonEngineTests/QwenPipelineTests.swift+++ b/engine-mac/Tests/KokonEngineTests/QwenPipelineTests.swift@@ -36,6 +36,32 @@ final class QwenPipelineTests: XCTestCase {XCTAssert(a.allSatisfy(\.isFinite))}+ /// The VAE's tiles cover every pixel once, each share at least the halo from a tile edge that is not the image's,+ /// and every tile has the same size (one compiled graph).+ func testVAETilesCoverEachPixelOnceWithTheirHalo() {+ let halo = 16+ for tile in [40, 96, 288] {+ for n in [40, 96, 97, 288, 289, 320, 512, 777, 1024, 2048] {+ let spans = QwenVAE.spans(n, tile: tile, halo: halo)+ var covered = [Int](repeating: 0, count: n)+ for s in spans {+ let size = min(tile, n)+ XCTAssert(s.start >= 0 && s.start + size <= n, "tile \(tile), n \(n): \(s)")+ XCTAssert(s.start == 0 || s.core - s.start >= halo, "tile \(tile), n \(n): \(s)")+ XCTAssert(s.start + size == n || s.start + size - (s.core + s.length) >= halo, "tile \(tile), n \(n): \(s)")+ for p in s.core..<(s.core + s.length) { covered[p] += 1 }+ }+ XCTAssertEqual(Set(covered), [1], "tile \(tile), n \(n)")+ }+ }+ XCTAssertEqual(QwenVAE.spans(512, tile: 288, halo: halo).count, 2, "1024²: 2 × 2 tiles")+ // Each stage's reach (its halo), from the architecture: up block 3 to the end 10 pixels, each up block alone 7.+ XCTAssertEqual(QwenVAE.Stage(first: 3, end: 5).halo, 10)+ XCTAssertEqual(QwenVAE.Stage(first: 0, end: 5).halo, 13)+ XCTAssertEqual((0..<4).map { QwenVAE.Stage(first: $0, end: $0 + 1).halo }, [7, 7, 7, 7])+ XCTAssertEqual(QwenVAE.Stage(first: 4, end: 5).halo, 7)+ }+func testPixelsFollowVaeImageProcessor() {// One pixel, RGBA planes: −1 → 0, 1 → 255, beyond the range clamped, 0 → 127.5 rounded to even (128).let image: [Float] = [-1, 1, 3, 0]