Skip to content

armgit / kokon

Public Imported from GitHub
0

Edit photos by describing the change, on your own GPU. A free, open-source alternative to Nano Banana and ChatGPT image editing: photos stay on your computer, no daily limit. One installer for Windows, Mac and Linux sets up the best open model (Qwen-Image 2.1). MCP server for AI agents.

Add Vulkan and Apple Silicon (Metal) engines

Gabriele Lozuponea35c33e743c6
diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 0000000..dfdb8b7
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1 @@
+*.sh text eol=lf
diff --git a/.gitignore b/.gitignore
index d1bc732..7bfb641 100644
--- a/.gitignore
+++ b/.gitignore
@@ -12,3 +12,5 @@ node_modules
__pycache__/
/docs/media/raw
/docs/media/*.mp4
+/src-tauri/binaries/*-apple-darwin
+/target/mac-check
diff --git a/README.md b/README.md
index 69cf997..dd0619f 100644
--- a/README.md
+++ b/README.md
@@ -124,13 +124,17 @@ is checked against its SHA-256 checksum. On a fast connection the whole thing ta
| | Minimum | Recommended |
| --- | --- | --- |
| **OS** | Windows 10 64-bit | Windows 11 |
-| **GPU** | NVIDIA, 12 GB VRAM (RTX 3060 12 GB, 4070) | NVIDIA, 16 GB+ VRAM (RTX 4080, 5070 Ti, 5080, 4090, 5090) |
+| **GPU** | NVIDIA, 12 GB VRAM (RTX 3060 12 GB, 4070) · AMD / Intel 12 GB+ through Vulkan (beta) | NVIDIA, 16 GB+ VRAM (RTX 4080, 5070 Ti, 5080, 4090, 5090) |
| **RAM** | 16 GB | 32 GB |
| **Disk** | 15 GB free | 25 GB free on an SSD |
-| **Time per edit** <!-- TODO re-measure at CFG 1 before launch --> | Fast mode: about half of Balanced | About 1 minute at Balanced on a 16 GB RTX 5070 Ti |
+| **Time per edit** <!-- TODO re-measure at CFG 1 before launch; RTX 5070 12 GB not timed yet --> | A little slower than the RTX 5070 Ti on a 12 GB RTX 5070 | About 1 minute at Balanced on a 16 GB RTX 5070 Ti |
> [!NOTE]
-> Kokon needs an NVIDIA GPU for now. AMD and Intel support (Vulkan) is on the [roadmap](#roadmap).
+> NVIDIA cards run on the CUDA engine, the fastest and tested path. AMD and Intel cards use the
+> **Vulkan engine (beta)**: it works on NVIDIA, but nobody has tried it on a real AMD or Intel
+> card yet, so please [report how it goes](https://github.com/GabrieleLozupone/kokon/issues).
+> Vulkan is slower than CUDA, and its first image takes longer. Apple Silicon Macs are in
+> testing ([platforms](docs/platforms.md)).
<details>
<summary><b>Model presets: what gets downloaded</b></summary>
@@ -140,12 +144,12 @@ The setup screen picks a preset from your VRAM. You can change it later from Set
| Preset | Diffusion model | Text encoder (in RAM) | Download | Tested on real hardware |
| --- | --- | --- | --- | --- |
-| 12 GB | Qwen-Image 2.1 Q4_K_M (4.2 GB) | Qwen3-VL 8B Q4_K_M | ~12 GB | in beta (emulated on a 16 GB card) |
+| 12 GB | Qwen-Image 2.1 Q4_K_M (4.2 GB) | Qwen3-VL 8B Q4_K_M | ~12 GB | ✅ RTX 5070 (12 GB) |
| 16 GB | Qwen-Image 2.1 Q4_K_M | Qwen3-VL 8B Q4_K_M | ~12 GB | ✅ RTX 5070 Ti |
| 24 GB | Qwen-Image 2.1 Q8_0 (7.6 GB) | Qwen3-VL 8B Q4_K_M | ~15 GB | not yet |
| 32 GB+ | Qwen-Image 2.1 Q8_0 | Qwen3-VL 8B Q8_0 (8.7 GB) | ~19 GB | not yet |
-Every preset also gets stable-diffusion.cpp (CUDA 12 build), the Qwen-Image 2.1 VAE, the
+Every preset also gets stable-diffusion.cpp (the CUDA 12 build on NVIDIA, the Vulkan build on AMD and Intel), the Qwen-Image 2.1 VAE, the
Qwen3-VL vision projector and, optionally, the 4x-UltraSharp upscaler. The sources are the
official [Qwen-Image 2.1](https://huggingface.co/Qwen/Qwen-Image-2.1) weights through
[unsloth/Qwen-Image-2.1-GGUF](https://huggingface.co/unsloth/Qwen-Image-2.1-GGUF),
@@ -181,8 +185,8 @@ opened in Explorer. A few things make that loop fast:
Undo returns to the version the current one came from, and nothing is ever deleted.
- **Gallery**: every image keeps its full recipe (prompt, seed, settings, pipeline) and reopens
with one click. The recipe is also embedded in the PNG.
-- **Quality presets**: Fast, Balanced and Quality (native 2K). They follow the official
- Qwen-Image 2.1 settings, so you never need to tune steps or guidance.
+- **Quality presets**: Fast, Balanced, Quality (native 2K) and Ultra (native 2K with the
+ official 40 steps), so you never need to tune steps or guidance.
## How it works
@@ -325,7 +329,7 @@ If you want full control over every node, ComfyUI is the better tool.
| Picks size and memory settings for you | ✅ | ❌ | partly | partly |
| Brush edits the model can see | ✅ | depends on the workflow | ✅ | rarely |
| MCP server for AI agents | ✅ | community nodes | ❌ | ❌ |
-| AMD / Intel GPUs | 🔜 [roadmap](#roadmap) | ✅ | ✅ | varies |
+| AMD / Intel GPUs | beta (Vulkan) | ✅ | ✅ | varies |
| Custom pipelines, node graphs | ❌ | ✅ | partly | ❌ |
| Painting and layers | ❌ | ❌ | ✅ | ❌ |
@@ -353,8 +357,13 @@ Only once, to download the models. After that Kokon works fully offline.
<summary><b>Does it work on AMD or Intel GPUs, or on a Mac?</b></summary>
<br>
-Not yet. stable-diffusion.cpp also runs on Vulkan and Metal, so AMD and Intel support is the
-next big item on the [roadmap](#roadmap), with macOS and Linux after that.
+AMD and Intel: in beta. The setup screen installs stable-diffusion.cpp's Vulkan build for
+them, which uses the graphics driver you already have. It works on NVIDIA cards, but it has not
+been verified on a real AMD or Intel card yet; integrated graphics are not supported. Please
+[open an issue](https://github.com/GabrieleLozupone/kokon/issues) with your card and how it went.
+
+Mac: Apple Silicon support is in testing. See [docs/platforms.md](docs/platforms.md) for the
+planned requirements (24 GB of unified memory recommended).
</details>
@@ -447,11 +456,11 @@ the log from `%LOCALAPPDATA%\KokonData\logs`.
- [x] MCP server and local HTTP API
- [x] Before/after, version undo, private mode, HEIC and HDR
- [ ] Fast mode with a turbo LoRA (an edit in about 15 seconds)
-- [ ] AMD and Intel GPUs through Vulkan
+- [ ] AMD and Intel GPUs through Vulkan (beta now; works on NVIDIA, to verify on AMD/Intel)
- [ ] Background removal and transparent cutouts
- [ ] Batch edits: one instruction for a whole folder
- [ ] Plugins for Krita, GIMP and Photoshop through the local API
-- [ ] macOS and Linux
+- [ ] macOS on Apple Silicon (in testing), then Linux
Have an idea? [Start a discussion](https://github.com/GabrieleLozupone/kokon/discussions) or 👍 an existing one.
diff --git a/crates/qls-core/Cargo.toml b/crates/qls-core/Cargo.toml
index d7ce493..4702a60 100644
--- a/crates/qls-core/Cargo.toml
+++ b/crates/qls-core/Cargo.toml
@@ -27,7 +27,7 @@ uuid = { version = "1.27.0", features = ["v4"] }
zip = { version = "8.6.0", default-features = false, features = ["deflate"] }
[target.'cfg(windows)'.dependencies]
-windows = { version = "0.62", features = ["Win32_Graphics_Imaging", "Win32_System_Com"] }
+windows = { version = "0.62", features = ["Win32_Graphics_Dxgi", "Win32_Graphics_Imaging", "Win32_System_Com"] }
[dev-dependencies]
tempfile = "3"
diff --git a/crates/qls-core/src/catalog.rs b/crates/qls-core/src/catalog.rs
index a8f6b88..b091f42 100644
--- a/crates/qls-core/src/catalog.rs
+++ b/crates/qls-core/src/catalog.rs
@@ -9,13 +9,13 @@ use std::path::{Path, PathBuf};
use serde::Serialize;
-use crate::config::TextEncoderPlacement;
+use crate::config::{GpuBackend, TextEncoderPlacement};
/// Where users report broken download links. Set before publishing the repository.
pub const ISSUES_URL: &str = "https://github.com/GabrieleLozupone/kokon/issues/new";
/// Folder under the stable-diffusion.cpp folder that holds the Qwen weights.
-pub const MODEL_SUBDIR: &str = r"models\qwen21";
+pub const MODEL_SUBDIR: &str = "models/qwen21";
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
#[serde(rename_all = "snake_case")]
@@ -37,27 +37,33 @@ pub struct Download {
pub page: &'static str,
pub size: u64,
pub sha256: &'static str,
- /// Destination relative to the stable-diffusion.cpp folder (for a zip: the archive name,
- /// extracted into the folder and then deleted).
+ /// Destination relative to the stable-diffusion.cpp folder (for a zip: the archive path,
+ /// extracted into its own folder and then deleted).
pub dest: &'static str,
- /// Files a zip provides; their presence means it is installed.
+ /// Files a zip provides (relative to the stable-diffusion.cpp folder); their presence
+ /// means it is installed.
pub provides: &'static [&'static str],
}
+/// `base` + a catalog path written with `/`, as native components.
+pub fn join_rel(base: &Path, rel: &str) -> PathBuf {
+ rel.split('/').filter(|c| !c.is_empty()).fold(base.to_path_buf(), |p, c| p.join(c))
+}
+
impl Download {
pub fn is_zip(&self) -> bool {
!self.provides.is_empty()
}
pub fn path(&self, sdcpp_dir: &Path) -> PathBuf {
- sdcpp_dir.join(self.dest)
+ join_rel(sdcpp_dir, self.dest)
}
/// Installed: a zip's files exist, or the file exists with the expected size. The hash is
/// checked after every download; a size match is enough for files already on disk.
pub fn is_present(&self, sdcpp_dir: &Path) -> bool {
if self.is_zip() {
- return self.provides.iter().all(|f| sdcpp_dir.join(f).is_file());
+ return self.provides.iter().all(|f| join_rel(sdcpp_dir, f).is_file());
}
std::fs::metadata(self.path(sdcpp_dir)).map(|m| m.is_file() && m.len() == self.size).unwrap_or(false)
}
@@ -91,6 +97,44 @@ pub const CUDA_RUNTIME: Download = Download {
provides: &["cudart64_12.dll", "cublas64_12.dll", "cublasLt64_12.dll"],
};
+/// Same commit, Vulkan build: AMD, Intel and NVIDIA cards through the GPU driver's Vulkan
+/// runtime (nothing else to install). Kept in its own folder so it can sit next to the
+/// CUDA build. SHA-256 from GitHub's asset digest, checked against a download 2026-10-09.
+pub const ENGINE_VULKAN: Download = Download {
+ id: "engine_vulkan",
+ label: "stable-diffusion.cpp (Vulkan)",
+ kind: FileKind::Engine,
+ url: "https://github.com/leejet/stable-diffusion.cpp/releases/download/master-929-3f8527a/sd-master-3f8527a-bin-win-vulkan-x64.zip",
+ page: SDCPP_RELEASE,
+ size: 30_067_605,
+ sha256: "60e6850d650417409f18c2170ab5e27335db96da70cd3d1ad1e930bcffc2fd35",
+ dest: "engine-vulkan/sd-master-3f8527a-bin-win-vulkan-x64.zip",
+ provides: &["engine-vulkan/sd-server.exe", "engine-vulkan/stable-diffusion.dll", "engine-vulkan/ggml-vulkan.dll"],
+};
+
+/// Same commit, macOS build: universal binary with the Metal shaders embedded, needs
+/// macOS 26. Ad-hoc signed; downloaded by the app (not a browser) it gets no quarantine flag.
+pub const ENGINE_METAL: Download = Download {
+ id: "engine_metal",
+ label: "stable-diffusion.cpp (Metal)",
+ kind: FileKind::Engine,
+ url: "https://github.com/leejet/stable-diffusion.cpp/releases/download/master-929-3f8527a/sd-master-3f8527a-bin-Darwin-macOS-26.6.2-arm64.zip",
+ page: SDCPP_RELEASE,
+ size: 35_049_086,
+ sha256: "1c8ee6c8e413e3335b1223bbc657ea5d86dae1819f8426a98b84c265587eb192",
+ dest: "engine-metal/sd-master-3f8527a-bin-Darwin-macOS-26.6.2-arm64.zip",
+ provides: &["engine-metal/sd-server", "engine-metal/libstable-diffusion.dylib"],
+};
+
+/// The engine files of a backend, in download order.
+pub fn engine_downloads(backend: GpuBackend) -> Vec<Download> {
+ match backend {
+ GpuBackend::Cuda => vec![ENGINE_CUDA, CUDA_RUNTIME],
+ GpuBackend::Vulkan => vec![ENGINE_VULKAN],
+ GpuBackend::Metal => vec![ENGINE_METAL],
+ }
+}
+
const UNSLOTH: &str = "https://huggingface.co/unsloth/Qwen-Image-2.1-GGUF/tree/main";
const QWEN_TE: &str = "https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct-GGUF/tree/main";
@@ -103,7 +147,7 @@ pub const DIFFUSION_Q4_K_M: Download = Download {
page: UNSLOTH,
size: 4_199_565_024,
sha256: "631d532e7ca71e8d90a87c71d3699761a812039d22e3370e87498d87754660fe",
- dest: r"models\qwen21\qwen-image-2.1-Q4_K_M.gguf",
+ dest: "models/qwen21/qwen-image-2.1-Q4_K_M.gguf",
provides: &[],
};
@@ -116,7 +160,7 @@ pub const DIFFUSION_Q8_0: Download = Download {
page: UNSLOTH,
size: 7_640_860_384,
sha256: "c0ed4b2ffd56cbe9c3df1e4a4098045256484ebe94ba5a7e4338d35ec046baa5",
- dest: r"models\qwen21\qwen-image-2.1-Q8_0.gguf",
+ dest: "models/qwen21/qwen-image-2.1-Q8_0.gguf",
provides: &[],
};
@@ -128,7 +172,7 @@ pub const VAE: Download = Download {
page: "https://huggingface.co/Comfy-Org/Qwen-Image-2.1/tree/main/vae",
size: 675_509_688,
sha256: "bb21f7473051e1ac368515dd3f2e15cd44d7a11748ee8823e1ddca3e4876b7c9",
- dest: r"models\qwen21\qwen_image_2.1_vae_bf16.safetensors",
+ dest: "models/qwen21/qwen_image_2.1_vae_bf16.safetensors",
provides: &[],
};
@@ -141,7 +185,7 @@ pub const TEXT_ENCODER_Q4_K_M: Download = Download {
page: QWEN_TE,
size: 5_027_784_800,
sha256: "67d1659bfe71b89d50b45a4ad1a9e5b997e5bb16ce5da66a6a6167abd569e9e2",
- dest: r"models\qwen21\Qwen3VL-8B-Instruct-Q4_K_M.gguf",
+ dest: "models/qwen21/Qwen3VL-8B-Instruct-Q4_K_M.gguf",
provides: &[],
};
@@ -153,7 +197,7 @@ pub const TEXT_ENCODER_Q8_0: Download = Download {
page: QWEN_TE,
size: 8_709_519_456,
sha256: "0d264b3941185d00a74f75c4245521dae088ff1efc90ab8d1754e83f5844adb0",
- dest: r"models\qwen21\Qwen3VL-8B-Instruct-Q8_0.gguf",
+ dest: "models/qwen21/Qwen3VL-8B-Instruct-Q8_0.gguf",
provides: &[],
};
@@ -166,7 +210,7 @@ pub const VISION_PROJECTOR: Download = Download {
page: QWEN_TE,
size: 1_159_029_824,
sha256: "ca524100ebf825c9a870db1c580d03879e0da0ab2541697e2458e64891cf9d38",
- dest: r"models\qwen21\mmproj-Qwen3VL-8B-Instruct-F16.gguf",
+ dest: "models/qwen21/mmproj-Qwen3VL-8B-Instruct-F16.gguf",
provides: &[],
};
@@ -179,7 +223,7 @@ pub const UPSCALER: Download = Download {
page: "https://huggingface.co/uwg/upscaler/tree/main/ESRGAN",
size: 66_961_958,
sha256: "a5812231fc936b42af08a5edba784195495d303d5b3248c24489ef0c4021fe01",
- dest: r"models\upscalers\4x-UltraSharp.pth",
+ dest: "models/upscalers/4x-UltraSharp.pth",
provides: &[],
};
@@ -208,7 +252,7 @@ pub const PRESETS: &[Preset] = &[
diffusion: DIFFUSION_Q4_K_M,
text_encoder: TEXT_ENCODER_Q4_K_M,
text_encoder_placement: TextEncoderPlacement::Ram,
- verified: false,
+ verified: true,
summary: "4-bit model; text encoder in system RAM; smaller working sizes",
},
Preset {
@@ -257,18 +301,24 @@ pub fn recommended_preset(vram_mib: Option<u64>) -> &'static Preset {
}
}
-/// Everything a preset needs, in download order (small, essential files first).
-pub fn downloads_for(p: &Preset, with_upscaler: bool) -> Vec<Download> {
- let mut v = vec![ENGINE_CUDA, CUDA_RUNTIME, VAE, VISION_PROJECTOR, p.text_encoder, p.diffusion];
+/// Everything a preset needs on `backend`, in download order (small, essential files first).
+pub fn downloads_for(p: &Preset, backend: GpuBackend, with_upscaler: bool) -> Vec<Download> {
+ let mut v = engine_downloads(backend);
+ v.extend([VAE, VISION_PROJECTOR, p.text_encoder, p.diffusion]);
if with_upscaler {
v.push(UPSCALER);
}
v
}
+/// Folder a zip is extracted into: the folder of its `dest`.
+pub fn zip_dir(d: &Download, sdcpp_dir: &Path) -> PathBuf {
+ d.path(sdcpp_dir).parent().map(Path::to_path_buf).unwrap_or_else(|| sdcpp_dir.to_path_buf())
+}
+
/// Message shown when a download keeps failing (moved link, changed file, server down).
pub fn manual_download_hint(d: &Download, sdcpp_dir: &Path) -> String {
- let target = if d.is_zip() { format!("extract it into {}", sdcpp_dir.display()) } else { format!("save it as {}", d.path(sdcpp_dir).display()) };
+ let target = if d.is_zip() { format!("extract it into {}", zip_dir(d, sdcpp_dir).display()) } else { format!("save it as {}", d.path(sdcpp_dir).display()) };
format!(
"Download {} by hand from {} and {target}, then press Check again. Please also open an issue at {ISSUES_URL} so the link can be fixed.",
d.label, d.page
@@ -293,18 +343,22 @@ mod tests {
#[test]
fn catalog_is_well_formed() {
- let all = [ENGINE_CUDA, CUDA_RUNTIME, DIFFUSION_Q4_K_M, DIFFUSION_Q8_0, VAE, TEXT_ENCODER_Q4_K_M, TEXT_ENCODER_Q8_0, VISION_PROJECTOR, UPSCALER];
+ let all = [ENGINE_CUDA, CUDA_RUNTIME, ENGINE_VULKAN, ENGINE_METAL, DIFFUSION_Q4_K_M, DIFFUSION_Q8_0, VAE, TEXT_ENCODER_Q4_K_M, TEXT_ENCODER_Q8_0, VISION_PROJECTOR, UPSCALER];
for d in all {
assert!(d.url.starts_with("https://"), "{}", d.id);
assert_eq!(d.sha256.len(), 64, "{}", d.id);
assert!(d.sha256.chars().all(|c| c.is_ascii_hexdigit() && !c.is_ascii_uppercase()), "{}", d.id);
assert!(d.size > 0);
+ assert!(!d.dest.contains('\\') && d.provides.iter().all(|f| !f.contains('\\')), "portable paths: {}", d.id);
assert_eq!(d.is_zip(), d.kind == FileKind::Engine, "{}", d.id);
}
// The default upscaler matches the one Settings offers first.
assert!(UPSCALER.dest.ends_with(&format!("{}.pth", crate::config::DEFAULT_UPSCALER)));
for p in PRESETS {
- assert_eq!(downloads_for(p, true).len(), 7);
+ assert_eq!(downloads_for(p, GpuBackend::Cuda, true).len(), 7);
+ let vk = downloads_for(p, GpuBackend::Vulkan, false);
+ assert_eq!(vk.len(), 5);
+ assert!(vk.iter().all(|d| d.id != "engine_cuda" && d.id != "cuda_runtime"));
}
}
@@ -312,7 +366,7 @@ mod tests {
fn presence_checks() {
let dir = tempfile::tempdir().unwrap();
assert!(!VAE.is_present(dir.path()));
- std::fs::create_dir_all(dir.path().join(MODEL_SUBDIR)).unwrap();
+ std::fs::create_dir_all(join_rel(dir.path(), MODEL_SUBDIR)).unwrap();
std::fs::write(VAE.path(dir.path()), b"short").unwrap();
assert!(!VAE.is_present(dir.path()), "a partial file is not installed");
for f in ENGINE_CUDA.provides {
@@ -321,5 +375,21 @@ mod tests {
assert!(ENGINE_CUDA.is_present(dir.path()));
let hint = manual_download_hint(&ENGINE_CUDA, dir.path());
assert!(hint.contains("extract it into") && hint.contains(ISSUES_URL));
+ // The Vulkan build lives in its own folder: the CUDA files don't count for it.
+ assert!(!ENGINE_VULKAN.is_present(dir.path()));
+ std::fs::create_dir_all(dir.path().join("engine-vulkan")).unwrap();
+ for f in ENGINE_VULKAN.provides {
+ std::fs::write(join_rel(dir.path(), f), b"x").unwrap();
+ }
+ assert!(ENGINE_VULKAN.is_present(dir.path()));
+ let mut cfg = crate::config::AppConfig::default();
+ cfg.sdcpp_dir = dir.path().to_path_buf();
+ cfg.gpu_backend = GpuBackend::Vulkan;
+ assert_eq!(cfg.server_exe(), join_rel(dir.path(), ENGINE_VULKAN.provides[0]), "installer and launcher agree on the path");
+ cfg.gpu_backend = GpuBackend::Metal;
+ if !cfg!(windows) {
+ assert_eq!(cfg.server_exe(), join_rel(dir.path(), ENGINE_METAL.provides[0]));
+ }
+ assert!(manual_download_hint(&ENGINE_VULKAN, dir.path()).contains("engine-vulkan"));
}
}
diff --git a/crates/qls-core/src/config.rs b/crates/qls-core/src/config.rs
index 0d27a4f..97470d8 100644
--- a/crates/qls-core/src/config.rs
+++ b/crates/qls-core/src/config.rs
@@ -26,7 +26,7 @@ pub const UPSCALER_CHOICES: &[(&str, &str, &str)] = &[
pub const DEFAULT_UPSCALER: &str = UPSCALER_CHOICES[0].0;
/// Default before schema 2; configs still on it are moved to [`DEFAULT_UPSCALER`].
const OLD_DEFAULT_UPSCALER: &str = "RealESRGAN_x4plus";
-const SCHEMA_VERSION: u32 = 5;
+const SCHEMA_VERSION: u32 = 6;
/// Profile id of the official Qwen-Image 2.1 weights (the default since schema 5).
pub const OFFICIAL_PROFILE_ID: &str = "qwen-image-2.1";
@@ -51,6 +51,59 @@ pub enum TextEncoderPlacement {
Cpu,
/// Let stable-diffusion.cpp's auto-fit decide.
Auto,
+ /// Every model's weights reloaded from disk when needed (`--params-backend disk`).
+ /// Least memory, slower; for Macs with 16 GB of unified memory.
+ Disk,
+}
+
+/// Which stable-diffusion.cpp build runs the model. Each build lives in its own folder, so
+/// switching back and forth needs no re-download.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "snake_case")]
+pub enum GpuBackend {
+ /// NVIDIA (CUDA 12 build). The verified path.
+ Cuda,
+ /// AMD, Intel and NVIDIA through Vulkan. Works on NVIDIA; AMD/Intel unverified.
+ Vulkan,
+ /// Apple Silicon (Metal build, macOS 26+). Beta.
+ Metal,
+}
+
+impl Default for GpuBackend {
+ /// The engine of this platform when nothing is configured.
+ fn default() -> Self {
+ if cfg!(target_os = "macos") { GpuBackend::Metal } else { GpuBackend::Cuda }
+ }
+}
+
+impl GpuBackend {
+ /// ggml device-name prefix (`cuda0`, `vulkan0`).
+ pub fn device_prefix(self) -> &'static str {
+ match self {
+ GpuBackend::Cuda => "cuda",
+ GpuBackend::Vulkan => "vulkan",
+ GpuBackend::Metal => "metal",
+ }
+ }
+
+ /// Folder of this build, relative to the stable-diffusion.cpp folder ("" = the folder
+ /// itself, where the CUDA build has always been).
+ pub fn engine_subdir(self) -> &'static str {
+ match self {
+ GpuBackend::Cuda => "",
+ GpuBackend::Vulkan => "engine-vulkan",
+ GpuBackend::Metal => "engine-metal",
+ }
+ }
+
+ /// Engine for a GPU vendor as reported by `diagnostics::query_gpus`.
+ pub fn for_vendor(vendor: &str) -> Self {
+ match vendor {
+ "nvidia" => GpuBackend::Cuda,
+ "apple" => GpuBackend::Metal,
+ _ => GpuBackend::Vulkan,
+ }
+ }
}
/// Effort of a Detailed upscale (low-denoise re-render at ~2K before the upscaler model).
@@ -80,10 +133,12 @@ pub struct RecommendedSettings {
pub steps_fast: u32,
pub steps_balanced: u32,
pub steps_quality: u32,
+ pub steps_ultra: u32,
/// Pixel area used by Auto resolution at each quality level.
pub area_fast: u64,
pub area_balanced: u64,
pub area_quality: u64,
+ pub area_ultra: u64,
pub align: u32,
pub max_side: u32,
pub max_references: u32,
@@ -101,18 +156,21 @@ impl Default for RecommendedSettings {
// Official Qwen-Image 2.1 settings (QwenLM/Qwen-Image-2.1 README, diffusers
// QwenImage21Pipeline): no CFG ("meant to be sampled without guidance"), 40 steps,
// native 2K buckets; flow-matching Euler, whose resolution-dependent schedule
- // sd-server selects automatically. Fast halves the steps for speed (not official).
+ // sd-server selects automatically. Only Ultra runs the official 40 steps; the other
+ // presets use 20 (judged enough by the user, 2026-10-09) and differ in size.
Self {
sampler: "euler".into(),
scheduler: None,
cfg: 1.0,
cfg_edit: 1.0,
steps_fast: 20,
- steps_balanced: 40,
- steps_quality: 40,
+ steps_balanced: 20,
+ steps_quality: 20,
+ steps_ultra: 40,
area_fast: 1024 * 1024,
- area_balanced: 1024 * 1024,
+ area_balanced: 1536 * 1536,
area_quality: 2048 * 2048,
+ area_ultra: 2048 * 2048,
align: 32,
max_side: 4096,
max_references: 10,
@@ -205,18 +263,22 @@ impl ModelProfile {
pub struct AppConfig {
pub schema_version: u32,
pub sdcpp_dir: PathBuf,
+ /// stable-diffusion.cpp build that runs the model.
+ pub gpu_backend: GpuBackend,
pub server_port: u16,
pub service_port: u16,
pub active_profile: String,
pub profiles: Vec<ModelProfile>,
/// ggml device name for diffusion + VAE, e.g. "cuda0" (see `sd-server --list-devices`).
+ /// A name of another engine (e.g. "cuda0" with Vulkan) means device 0 of the active one.
pub device: String,
pub text_encoder_placement: TextEncoderPlacement,
/// VRAM kept free, in GiB (passed as a negative `--max-vram`).
pub vram_reserve_gib: f32,
pub flash_attention: bool,
/// Native CUDA SageAttention in the diffusion model (`--sage-attn`); ~1.5x faster
- /// steps at 2K on an RTX 5070 Ti with near-identical output.
+ /// steps at 2K on an RTX 5070 Ti with near-identical output. CUDA only (SM80+);
+ /// ignored by other engines.
pub sage_attention: bool,
/// VAE tile size in pixels (`--vae-tiling --vae-tile-size N`); 0 disables tiling.
/// Untiled 2K decodes need ~18 GB, OOM and retry with 256px tiles (~22 s); 768px
@@ -263,6 +325,7 @@ impl Default for AppConfig {
profiles: vec![ModelProfile::official(&sdcpp, crate::catalog::preset(crate::catalog::DEFAULT_PRESET).unwrap())],
active_profile: OFFICIAL_PROFILE_ID.into(),
sdcpp_dir: sdcpp,
+ gpu_backend: GpuBackend::default(),
server_port: DEFAULT_SERVER_PORT,
service_port: DEFAULT_SERVICE_PORT,
device: "cuda0".into(),
@@ -422,6 +485,22 @@ impl AppConfig {
}
}
}
+ if c.schema_version < 6 {
+ // 20 steps for Balanced and Quality, Balanced at 1.5K (Ultra takes the
+ // official 40 steps). Edited values stay.
+ for p in &mut c.profiles {
+ let r = &mut p.recommended_settings;
+ if r.steps_balanced == 40 {
+ r.steps_balanced = 20;
+ }
+ if r.steps_quality == 40 {
+ r.steps_quality = 20;
+ }
+ if r.area_balanced == 1024 * 1024 {
+ r.area_balanced = 1536 * 1536;
+ }
+ }
+ }
// Paths into the data folder of the app's old name (models installed by the
// setup screen) follow the folder to its new name.
if std::env::var("QLS_DATA_DIR").is_err() {
@@ -505,8 +584,36 @@ impl AppConfig {
if self.vram_override_mib > 0 { Some(self.vram_override_mib) } else { detected }
}
+ /// Folder of the active stable-diffusion.cpp build.
+ pub fn engine_dir(&self) -> PathBuf {
+ match self.gpu_backend.engine_subdir() {
+ "" => self.sdcpp_dir.clone(),
+ sub => self.sdcpp_dir.join(sub),
+ }
+ }
+
pub fn server_exe(&self) -> PathBuf {
- self.sdcpp_dir.join("sd-server.exe")
+ self.engine_dir().join(if cfg!(windows) { "sd-server.exe" } else { "sd-server" })
+ }
+
+ /// The configured device when it belongs to the active engine, else its first GPU.
+ pub fn effective_device(&self) -> String {
+ let dev = self.device.trim().to_ascii_lowercase();
+ if self.gpu_backend == GpuBackend::Metal {
+ // sd-server resolves "metal" to the Mac's GPU (ggml names it MTL0).
+ return "metal".into();
+ }
+ let prefix = self.gpu_backend.device_prefix();
+ match dev.strip_prefix(prefix) {
+ Some(i) if !i.is_empty() && i.chars().all(|c| c.is_ascii_digit()) => dev,
+ // Engines number their devices independently: fall back to the first one.
+ _ => format!("{prefix}0"),
+ }
+ }
+
+ /// SageAttention is requested and the engine supports it.
+ pub fn uses_sage_attention(&self) -> bool {
+ self.sage_attention && self.gpu_backend == GpuBackend::Cuda
}
pub fn upscalers_dir(&self) -> PathBuf {
@@ -578,6 +685,37 @@ mod tests {
assert_eq!(c.upscalers_dir(), default_sdcpp_dir().join("models").join("upscalers"));
}
+ #[test]
+ fn engine_choice_sets_folder_device_and_flags() {
+ let mut c = AppConfig::default();
+ c.gpu_backend = GpuBackend::Cuda;
+ assert_eq!(c.engine_dir(), c.sdcpp_dir);
+ assert_eq!(c.effective_device(), "cuda0");
+ assert!(c.uses_sage_attention());
+ c.gpu_backend = GpuBackend::Vulkan;
+ assert_eq!(c.engine_dir(), c.sdcpp_dir.join("engine-vulkan"));
+ assert!(c.server_exe().starts_with(c.sdcpp_dir.join("engine-vulkan")));
+ assert_eq!(c.effective_device(), "vulkan0", "a CUDA device name maps to the first Vulkan GPU");
+ c.device = "Vulkan1".into();
+ assert_eq!(c.effective_device(), "vulkan1");
+ c.device = "vulkan".into();
+ assert_eq!(c.effective_device(), "vulkan0");
+ assert!(!c.uses_sage_attention(), "SageAttention is CUDA only");
+ assert_eq!(GpuBackend::for_vendor("nvidia"), GpuBackend::Cuda);
+ assert_eq!(GpuBackend::for_vendor("amd"), GpuBackend::Vulkan);
+ assert_eq!(GpuBackend::for_vendor("apple"), GpuBackend::Metal);
+ c.gpu_backend = GpuBackend::Metal;
+ c.device = "cuda0".into();
+ assert_eq!(c.effective_device(), "metal");
+ assert_eq!(c.engine_dir(), c.sdcpp_dir.join("engine-metal"));
+ assert!(!c.uses_sage_attention());
+ // Configs written before the field existed keep this platform's engine.
+ let old: AppConfig = serde_json::from_str("{}").unwrap();
+ assert_eq!(old.gpu_backend, if cfg!(target_os = "macos") { GpuBackend::Metal } else { GpuBackend::Cuda });
+ let json = serde_json::to_string(&c).unwrap();
+ assert!(json.contains(r#""gpuBackend":"metal""#));
+ }
+
#[test]
fn official_profile_is_added_but_not_forced() {
let dir = tempfile::tempdir().unwrap();
@@ -660,8 +798,8 @@ mod tests {
let old = r#"{"schemaVersion":2,"profiles":[{"id":"a","recommendedSettings":{"steps_balanced":30,"steps_quality":40,"area_quality":2359296}}]}"#;
std::fs::write(&p, old).unwrap();
let r = AppConfig::load_from(&p).profiles[0].recommended_settings.clone();
- // Schema 3 then schema 4 run in sequence and end on the official settings.
- assert_eq!((r.steps_balanced, r.steps_quality, r.area_quality, r.cfg_edit), (40, 40, 2048 * 2048, 1.0));
+ // Schema 3, 4 and 6 run in sequence and end on the current presets.
+ assert_eq!((r.steps_balanced, r.steps_quality, r.area_quality, r.cfg_edit), (20, 20, 2048 * 2048, 1.0));
// Values the user changed are kept.
let custom = r#"{"schemaVersion":2,"profiles":[{"id":"a","recommendedSettings":{"steps_balanced":25,"steps_quality":50}}]}"#;
std::fs::write(&p, custom).unwrap();
@@ -677,7 +815,7 @@ mod tests {
std::fs::write(&p, old).unwrap();
let r = AppConfig::load_from(&p).profiles[0].recommended_settings.clone();
assert_eq!((r.cfg, r.cfg_edit), (1.0, 1.0));
- assert_eq!((r.steps_fast, r.steps_balanced, r.steps_quality, r.area_quality), (20, 40, 40, 2048 * 2048));
+ assert_eq!((r.steps_fast, r.steps_balanced, r.steps_quality, r.area_quality), (20, 20, 20, 2048 * 2048));
assert_eq!(r, RecommendedSettings::default());
let custom = r#"{"schemaVersion":3,"profiles":[{"id":"a","recommendedSettings":{"cfg":3.0,"cfg_edit":2.5,"steps_balanced":28}}]}"#;
std::fs::write(&p, custom).unwrap();
diff --git a/crates/qls-core/src/diagnostics.rs b/crates/qls-core/src/diagnostics.rs
index 6ddf0c5..be22e49 100644
--- a/crates/qls-core/src/diagnostics.rs
+++ b/crates/qls-core/src/diagnostics.rs
@@ -12,14 +12,64 @@ use crate::process::{parse_device_list, run_server_probe};
#[derive(Debug, Clone, Serialize, Default)]
pub struct GpuInfo {
pub name: String,
+ /// "nvidia", "amd", "intel", "apple" or "other"; picks the engine (`GpuBackend::for_vendor`).
+ pub vendor: String,
+ /// Apple Silicon: `memory_total_mib` is unified memory shared with the system.
+ pub unified: bool,
pub memory_total_mib: u64,
pub memory_used_mib: u64,
pub memory_free_mib: u64,
pub driver_version: String,
}
-/// Queries NVIDIA GPUs through nvidia-smi (ships with the driver). None if unavailable.
+impl GpuInfo {
+ /// Memory the planner may use for the diffusion model, as on a discrete card. Apple
+ /// Silicon shares it with the system and the text encoder
+ /// (`planner::unified_memory_planning_mib`).
+ pub fn planning_mib(&self) -> u64 {
+ if self.unified {
+ use crate::catalog::{TEXT_ENCODER_Q4_K_M, VISION_PROJECTOR};
+ let te = (TEXT_ENCODER_Q4_K_M.size + VISION_PROJECTOR.size) / (1024 * 1024);
+ crate::planner::unified_memory_planning_mib(self.memory_total_mib, te)
+ } else {
+ self.memory_total_mib
+ }
+ }
+}
+
+/// The GPUs to plan for: NVIDIA through nvidia-smi (live memory use), otherwise any
+/// graphics card Windows knows (AMD, Intel), or the Apple Silicon GPU. None if there is none.
pub fn query_gpus() -> Option<Vec<GpuInfo>> {
+ query_nvidia().or_else(query_dxgi).or_else(query_apple)
+}
+
+/// Apple Silicon: chip name and unified memory from sysctl (they never change, so read once).
+#[cfg(target_os = "macos")]
+fn query_apple() -> Option<Vec<GpuInfo>> {
+ static INFO: std::sync::OnceLock<Option<GpuInfo>> = std::sync::OnceLock::new();
+ INFO.get_or_init(|| {
+ let read = |key: &str| {
+ let out = std::process::Command::new("/usr/sbin/sysctl").args(["-n", key]).output().ok()?;
+ out.status.success().then(|| String::from_utf8_lossy(&out.stdout).trim().to_string())
+ };
+ if read("hw.optional.arm64").as_deref() != Some("1") {
+ return None; // Intel Mac: not supported
+ }
+ let total = read("hw.memsize")?.parse::<u64>().ok()? / (1024 * 1024);
+ let name = read("machdep.cpu.brand_string").unwrap_or_else(|| "Apple Silicon".into());
+ Some(GpuInfo { name, vendor: "apple".into(), unified: true, memory_total_mib: total, memory_used_mib: 0, memory_free_mib: total, driver_version: String::new() })
+ })
+ .clone()
+ .map(|g| vec![g])
+}
+
+#[cfg(not(target_os = "macos"))]
+fn query_apple() -> Option<Vec<GpuInfo>> {
+ None
+}
+
+/// NVIDIA GPUs through nvidia-smi (ships with the driver). None if unavailable.
+fn query_nvidia() -> Option<Vec<GpuInfo>> {
let mut cmd = std::process::Command::new("nvidia-smi");
cmd.args(["--query-gpu=name,memory.total,memory.used,memory.free,driver_version", "--format=csv,noheader,nounits"]);
#[cfg(windows)]
@@ -43,6 +93,8 @@ pub fn parse_gpu_line(line: &str) -> Option<GpuInfo> {
}
Some(GpuInfo {
name: f[0].to_string(),
+ vendor: "nvidia".into(),
+ unified: false,
memory_total_mib: f[1].parse().ok()?,
memory_used_mib: f[2].parse().ok()?,
memory_free_mib: f[3].parse().ok()?,
@@ -50,6 +102,52 @@ pub fn parse_gpu_line(line: &str) -> Option<GpuInfo> {
})
}
+/// PCI vendor id → vendor name used across the app.
+pub fn vendor_name(id: u32) -> &'static str {
+ match id {
+ 0x10DE => "nvidia",
+ 0x1002 | 0x1022 => "amd",
+ 0x8086 => "intel",
+ _ => "other",
+ }
+}
+
+/// Graphics adapters through DXGI, largest dedicated VRAM first. Skips software adapters
+/// (Microsoft Basic Render Driver). DXGI has no system-wide usage figure, so used = 0.
+/// Integrated GPUs report only their small reserved memory and so fall under the
+/// smallest preset's warning.
+#[cfg(windows)]
+fn query_dxgi() -> Option<Vec<GpuInfo>> {
+ use windows::Win32::Graphics::Dxgi::*;
+ let factory: IDXGIFactory1 = unsafe { CreateDXGIFactory1() }.ok()?;
+ let mut gpus = Vec::new();
+ for i in 0.. {
+ let Ok(adapter) = (unsafe { factory.EnumAdapters1(i) }) else { break };
+ let Ok(d) = (unsafe { adapter.GetDesc1() }) else { continue };
+ if d.Flags & (DXGI_ADAPTER_FLAG_SOFTWARE.0 as u32) != 0 || d.VendorId == 0x1414 {
+ continue;
+ }
+ let len = d.Description.iter().position(|&c| c == 0).unwrap_or(d.Description.len());
+ let total = d.DedicatedVideoMemory as u64 / (1024 * 1024);
+ gpus.push(GpuInfo {
+ name: String::from_utf16_lossy(&d.Description[..len]).trim().to_string(),
+ vendor: vendor_name(d.VendorId).into(),
+ unified: false,
+ memory_total_mib: total,
+ memory_used_mib: 0,
+ memory_free_mib: total,
+ driver_version: String::new(),
+ });
+ }
+ gpus.sort_by(|a, b| b.memory_total_mib.cmp(&a.memory_total_mib));
+ (!gpus.is_empty()).then_some(gpus)
+}
+
+#[cfg(not(windows))]
+fn query_dxgi() -> Option<Vec<GpuInfo>> {
+ None
+}
+
#[derive(Debug, Clone, Serialize)]
pub struct Check {
pub id: &'static str,
@@ -85,7 +183,7 @@ pub fn setup_checks(cfg: &AppConfig, server_ok: Option<bool>) -> SetupReport {
checks.push(Check { id: "sdcpp_dir", label: "stable-diffusion.cpp folder".into(), ok: dir_ok, detail: cfg.sdcpp_dir.display().to_string() });
let exe = cfg.server_exe();
let exe_ok = exe.is_file();
- checks.push(Check { id: "server_exe", label: "sd-server.exe".into(), ok: exe_ok, detail: exe.display().to_string() });
+ checks.push(Check { id: "server_exe", label: "stable-diffusion.cpp server".into(), ok: exe_ok, detail: exe.display().to_string() });
for (label, path, _required) in cfg.profile().files() {
let meta = std::fs::metadata(path).ok().filter(|m| m.is_file());
checks.push(Check {
@@ -99,7 +197,7 @@ pub fn setup_checks(cfg: &AppConfig, server_ok: Option<bool>) -> SetupReport {
});
}
let devices = if exe_ok { list_devices(&exe) } else { vec![] };
- let wanted = cfg.device.to_ascii_lowercase();
+ let wanted = cfg.effective_device();
let dev = devices.iter().find(|(n, _)| n.to_ascii_lowercase() == wanted);
let gpu_ok = dev.is_some();
checks.push(Check {
@@ -109,7 +207,7 @@ pub fn setup_checks(cfg: &AppConfig, server_ok: Option<bool>) -> SetupReport {
detail: match dev {
Some((n, d)) => format!("{n} — {d}"),
None if devices.is_empty() => "No devices reported by sd-server".into(),
- None => format!("{} not found. Available: {}", cfg.device, devices.iter().map(|(n, _)| n.as_str()).collect::<Vec<_>>().join(", ")),
+ None => format!("{wanted} not found. Available: {}", devices.iter().map(|(n, _)| n.as_str()).collect::<Vec<_>>().join(", ")),
},
});
if let Some(running) = server_ok {
@@ -163,6 +261,21 @@ mod tests {
assert_eq!((g.memory_total_mib, g.memory_used_mib, g.memory_free_mib), (16303, 2312, 13991));
assert_eq!(g.driver_version, "617.14");
assert!(parse_gpu_line("garbage").is_none());
+ assert_eq!(g.vendor, "nvidia");
+ assert_eq!(g.planning_mib(), 16303);
+ let mac = GpuInfo { name: "Apple M2 Pro".into(), vendor: "apple".into(), unified: true, memory_total_mib: 16384, ..Default::default() };
+ assert!(mac.planning_mib() < 6 * 1024, "16 GB Mac plans like a small card");
+ assert_eq!(crate::catalog::recommended_preset(Some(mac.planning_mib())).id, "12gb");
+ }
+
+ #[test]
+ fn dxgi_reports_real_adapters() {
+ assert_eq!((vendor_name(0x10DE), vendor_name(0x1002), vendor_name(0x8086), vendor_name(0x5143)), ("nvidia", "amd", "intel", "other"));
+ // Machines without a GPU (CI) report none; otherwise every entry is a real card.
+ for g in query_dxgi().unwrap_or_default() {
+ println!("DXGI: {} ({}) {} MiB", g.name, g.vendor, g.memory_total_mib);
+ assert!(!g.name.is_empty() && !g.name.contains("Basic Render"));
+ }
}
#[test]
diff --git a/crates/qls-core/src/engine.rs b/crates/qls-core/src/engine.rs
index a3dd144..cdbfedb 100644
--- a/crates/qls-core/src/engine.rs
+++ b/crates/qls-core/src/engine.rs
@@ -167,7 +167,8 @@ impl Engine {
new.save()?;
let mut g = self.cfg.write().unwrap();
let restart = crate::process::build_server_args(&g, g.profile()) != crate::process::build_server_args(&new, new.profile())
- || g.sdcpp_dir != new.sdcpp_dir;
+ || g.sdcpp_dir != new.sdcpp_dir
+ || g.gpu_backend != new.gpu_backend;
*g = new;
*self.backend_version.lock().unwrap() = None;
Ok(restart)
@@ -461,7 +462,7 @@ impl Engine {
/// Dry run: the plan a request would get (used by the UI to show working sizes).
pub fn preview_plan(&self, req: &GenerationRequest, sizes: Vec<Size>, mask: Option<(Size, Option<crate::geometry::Rect>, f64)>) -> AppResult<Plan> {
let cfg = self.config();
- let vram = cfg.planning_vram_mib(self.gpu().map(|g| g.memory_total_mib));
+ let vram = cfg.planning_vram_mib(self.gpu().map(|g| g.planning_mib()));
planner::plan(req, &plan_inputs(&cfg, sizes, mask, vram), &cfg.profile().recommended_settings)
}
@@ -506,7 +507,7 @@ impl Engine {
// 2. Plan.
let cfg = self.config();
let profile = cfg.profile().clone();
- let vram = cfg.planning_vram_mib(self.gpu().map(|g| g.memory_total_mib));
+ let vram = cfg.planning_vram_mib(self.gpu().map(|g| g.planning_mib()));
let mask_info = mask.as_ref().map(|(m, _)| (Size::new(m.width(), m.height()), imaging::mask_bbox(m, 16), imaging::mask_coverage(m, 16)));
let plan_inputs = plan_inputs(&cfg, images.iter().map(|i| i.size()).collect(), mask_info, vram);
let plan = planner::plan(&request, &plan_inputs, &profile.recommended_settings)?;
diff --git a/crates/qls-core/src/installer.rs b/crates/qls-core/src/installer.rs
index e3f12f1..dd22e37 100644
--- a/crates/qls-core/src/installer.rs
+++ b/crates/qls-core/src/installer.rs
@@ -9,7 +9,7 @@
//! - Every finished download is hashed; a mismatch deletes it and downloads once more.
//! - A moved link (HTTP 404/403/410) or a second mismatch fails that file with instructions
//! to download it by hand and to open an issue ([`crate::catalog::manual_download_hint`]).
-//! - Zips (the engine) are extracted into the folder and then deleted.
+//! - Zips (the engine) are extracted into the folder of their destination and then deleted.
//!
//! Files already present (by size) are skipped. Network calls only go to the URLs in the
//! catalog (GitHub releases and Hugging Face).
@@ -349,7 +349,7 @@ impl Installer {
}
if d.is_zip() {
self.set(d.id, |i| i.state = ItemState::Extracting);
- let (p, out) = (part.clone(), dir.to_path_buf());
+ let (p, out) = (part.clone(), crate::catalog::zip_dir(d, dir));
tokio::task::spawn_blocking(move || extract_zip(&p, &out))
.await
.map_err(|e| Failure::Transient(e.to_string()))?
@@ -802,6 +802,12 @@ pub fn extract_zip(zip: &Path, out: &Path) -> Result<(), String> {
let mut w = std::fs::File::create(&tmp).map_err(|e| format!("Cannot write {}: {e}", tmp.display()))?;
std::io::copy(&mut entry, &mut w).map_err(|e| format!("Cannot extract {}: {e}", dest.display()))?;
drop(w);
+ // Keep the executable bit of the macOS/Linux engine (sd-server).
+ #[cfg(unix)]
+ if let Some(mode) = entry.unix_mode() {
+ use std::os::unix::fs::PermissionsExt;
+ let _ = std::fs::set_permissions(&tmp, std::fs::Permissions::from_mode(mode & 0o777));
+ }
std::fs::rename(&tmp, &dest).map_err(|e| {
let _ = std::fs::remove_file(&tmp);
format!("Cannot replace {} (is stable-diffusion.cpp still running?): {e}", dest.display())
@@ -852,7 +858,7 @@ mod tests {
std::fs::write(&f, b"abc").unwrap();
assert_eq!(sha256_file(&f).unwrap(), "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad");
let p = crate::catalog::preset("16gb").unwrap();
- let all = crate::catalog::downloads_for(p, false);
+ let all = crate::catalog::downloads_for(p, crate::config::GpuBackend::Cuda, false);
let (dl, disk) = space_needed(dir.path(), &all);
assert_eq!(dl, all.iter().map(|d| d.size).sum::<u64>());
assert!(disk > dl, "zips need room to extract");
diff --git a/crates/qls-core/src/pipeline.rs b/crates/qls-core/src/pipeline.rs
index e7a995e..ede1136 100644
--- a/crates/qls-core/src/pipeline.rs
+++ b/crates/qls-core/src/pipeline.rs
@@ -522,7 +522,7 @@ mod tests {
assert_eq!((v["width"].as_u64(), v["height"].as_u64()), (Some(p.working.width as u64), Some(p.working.height as u64)));
assert!(v.get("hires").map(|h| h.is_null()).unwrap_or(true), "no second pass");
assert!(v.get("cache_mode").map(|c| c.is_null()).unwrap_or(true));
- assert_eq!(v["sample_params"]["sample_steps"], 40);
+ assert_eq!(v["sample_params"]["sample_steps"], 20);
assert_eq!(v["sample_params"]["guidance"]["txt_cfg"], 1.0, "official: no CFG");
assert!(p.working.area() > 2_500_000, "Quality renders toward native 2K: {:?}", p.working);
}
@@ -858,7 +858,7 @@ mod tests {
let r = req("a red fox");
let p = plan(&r, &PlanInputs { vram_total_mib: Some(16303), ..Default::default() }, &RecommendedSettings::default()).unwrap();
let j = prepare(&p, &r.prompt, 5, &[], None, &BackendOptions::default()).unwrap();
- let mut noisy = fake_backend(1024, 1024);
+ let mut noisy = fake_backend(p.working.width, p.working.height);
noisy.put_pixel(3, 3, Rgba([1, 2, 3, 250]));
let out = postprocess(noisy.clone(), &j.post).unwrap();
assert!(out.pixels().all(|p| p[3] == 255));
@@ -878,9 +878,9 @@ mod tests {
let v = serde_json::to_value(&j.request).unwrap();
assert!(v.get("ref_images").is_none() && v.get("init_image").is_none());
assert_eq!(v["seed"], 5);
- assert_eq!(v["width"], 1024);
- let out = postprocess(fake_backend(1024, 1024), &j.post).unwrap();
- assert_eq!(out.dimensions(), (1024, 1024));
+ assert_eq!(v["width"], 1536);
+ let out = postprocess(fake_backend(1536, 1536), &j.post).unwrap();
+ assert_eq!(out.dimensions(), (1536, 1536));
}
#[test]
diff --git a/crates/qls-core/src/planner.rs b/crates/qls-core/src/planner.rs
index 57ac25e..7e10c21 100644
--- a/crates/qls-core/src/planner.rs
+++ b/crates/qls-core/src/planner.rs
@@ -135,6 +135,7 @@ pub enum Quality {
#[default]
Balanced,
Quality,
+ Ultra,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)]
@@ -489,6 +490,7 @@ pub fn resolution_area(mode: ResolutionMode, quality: Quality, rec: &Recommended
Quality::Fast => rec.area_fast,
Quality::Balanced => rec.area_balanced,
Quality::Quality => rec.area_quality,
+ Quality::Ultra => rec.area_ultra,
}),
ResolutionMode::K1 => Some(1024 * 1024),
ResolutionMode::K15 => Some(1536 * 1536),
@@ -513,6 +515,21 @@ pub fn token_budget(vram_total_mib: Option<u64>) -> u64 {
tokens.clamp(4_096.0, 24_576.0) as u64
}
+/// Apple Silicon: the VRAM-equivalent to plan with on a Mac with `ram_mib` of unified
+/// memory, so presets and working sizes follow the same rules as a discrete GPU.
+///
+/// Metal lets the GPU wire about 2/3 of unified memory (about 3/4 from 64 GB up; macOS's
+/// `recommendedMaxWorkingSetSize`, the rest stays for the system). The GPU budgets above
+/// assume the text encoder sits in *separate* system RAM; on a Mac it shares the same pool,
+/// so its weights (`text_encoder_mib`, encoder + vision projector) come off the top.
+/// Not yet measured on a Mac: 16 GB → ~4.5 GiB (lightest preset, 1 MP only),
+/// 24 GB → ~9.8 GiB (12 GB preset), 32 GB → ~15 GiB (16 GB preset, 2K),
+/// 64 GB → ~42 GiB (32 GB+ preset).
+pub fn unified_memory_planning_mib(ram_mib: u64, text_encoder_mib: u64) -> u64 {
+ let gpu_share = if ram_mib >= 64 * 1024 { ram_mib * 3 / 4 } else { ram_mib * 2 / 3 };
+ gpu_share.saturating_sub(text_encoder_mib)
+}
+
pub const PIXELS_PER_TOKEN: u64 = 256; // 8x VAE downscale, 2x2 patches
/// Area of the inpaint guide reference (the region with the painted area marked), as a
@@ -561,6 +578,7 @@ fn steps_for(q: Quality, rec: &RecommendedSettings) -> u32 {
Quality::Fast => rec.steps_fast,
Quality::Balanced => rec.steps_balanced,
Quality::Quality => rec.steps_quality,
+ Quality::Ultra => rec.steps_ultra,
}
}
@@ -960,6 +978,20 @@ pub fn plan(req: &GenerationRequest, inputs: &PlanInputs, rec: &RecommendedSetti
#[cfg(test)]
mod tests {
+ #[test]
+ fn unified_memory_maps_to_presets() {
+ use crate::catalog::{TEXT_ENCODER_Q4_K_M, VISION_PROJECTOR, recommended_preset};
+ let te = (TEXT_ENCODER_Q4_K_M.size + VISION_PROJECTOR.size) / (1024 * 1024);
+ let mac = |gb: u64| unified_memory_planning_mib(gb * 1024, te);
+ assert!(mac(16) < 6 * 1024, "16 GB: only the smallest working sizes");
+ assert_eq!(token_budget(Some(mac(16))), 4_096);
+ assert_eq!(recommended_preset(Some(mac(24))).id, "12gb");
+ assert_eq!(recommended_preset(Some(mac(32))).id, "16gb");
+ assert_eq!(recommended_preset(Some(mac(48))).id, "24gb");
+ assert_eq!(recommended_preset(Some(mac(64))).id, "32gb");
+ assert_eq!(unified_memory_planning_mib(4096, te), 0, "never underflows");
+ }
+
use super::*;
fn rec() -> RecommendedSettings {
@@ -977,10 +1009,10 @@ mod tests {
let p = plan(&req("a lighthouse at dusk"), &inputs(&[]), &rec()).unwrap();
assert_eq!(p.mode, ResolvedMode::Generate);
assert!(p.auto_mode);
- assert_eq!(p.output, Size::new(1024, 1024));
- assert_eq!(p.working, Size::new(1024, 1024));
- assert_eq!((p.steps, p.cfg, p.sampler.as_str()), (40, 1.0, "euler"), "official: 40 steps, no CFG");
- assert!(!p.vae_tiling);
+ assert_eq!(p.output, Size::new(1536, 1536), "Balanced = 1.5K");
+ assert_eq!(p.working, Size::new(1536, 1536));
+ assert_eq!((p.steps, p.cfg, p.sampler.as_str()), (20, 1.0, "euler"), "20 steps, no CFG (official)");
+ assert!(p.vae_tiling, "decodes above ~1.5 MP are tiled");
assert!(p.references.is_empty());
}
@@ -1000,7 +1032,7 @@ mod tests {
#[test]
fn prompt_implies_aspect() {
let p = plan(&req("Generate a hero image for the landing page, 16:9"), &inputs(&[]), &rec()).unwrap();
- assert_eq!(p.output, Size::new(1376, 768));
+ assert_eq!(p.output, Size::new(2048, 1152));
let p = plan(&req("a phone wallpaper of mountains"), &inputs(&[]), &rec()).unwrap();
assert!(p.output.height > p.output.width);
let p = plan(&req("a poster, 1280x720"), &inputs(&[]), &rec()).unwrap();
@@ -1020,15 +1052,21 @@ mod tests {
r.resolution = ResolutionMode::Auto;
r.quality = Quality::Fast;
let p = plan(&r, &inputs(&[]), &rec()).unwrap();
- assert_eq!(p.steps, 20, "Fast halves the official 40 steps");
+ assert_eq!(p.steps, 20);
assert!(p.output.area() <= 1_100_000);
r.quality = Quality::Balanced;
- assert_eq!(plan(&r, &inputs(&[]), &rec()).unwrap().steps, 40);
+ let p = plan(&r, &inputs(&[]), &rec()).unwrap();
+ assert_eq!(p.steps, 20);
+ assert!(p.output.area() > 1_100_000 && p.output.area() <= 1536 * 1536 * 102 / 100, "Balanced = about 1.5K");
r.quality = Quality::Quality;
let p = plan(&r, &inputs(&[]), &rec()).unwrap();
- assert_eq!(p.steps, 40);
+ assert_eq!(p.steps, 20);
assert_eq!(p.working, Size::new(2752, 1536), "Quality = official native 2K bucket");
assert_eq!(p.output, p.working);
+ r.quality = Quality::Ultra;
+ let p = plan(&r, &inputs(&[]), &rec()).unwrap();
+ assert_eq!(p.steps, 40, "Ultra = official 40 steps");
+ assert_eq!(p.working, Size::new(2752, 1536));
}
#[test]
@@ -1087,7 +1125,7 @@ mod tests {
// Any refine setting, any size: one diffusion pass, no cache.
for refine in [RefineMode::Off, RefineMode::Fast, RefineMode::Best] {
for (res, aspect) in [(ResolutionMode::Auto, "16:9"), (ResolutionMode::K15, "1:1"), (ResolutionMode::K2, "9:16")] {
- for q in [Quality::Fast, Quality::Balanced, Quality::Quality] {
+ for q in [Quality::Fast, Quality::Balanced, Quality::Quality, Quality::Ultra] {
let mut r = req("a cozy mountain cabin");
r.aspect_ratio = Some(aspect.into());
r.resolution = res;
@@ -1129,13 +1167,13 @@ mod tests {
fn edits_are_one_pass_and_enlarged_to_the_source() {
let i = PlanInputs { image_sizes: vec![Size::new(5120, 2880)], ..env(true, RefineMode::Fast) };
let mut r = req("make it a dark version");
- r.quality = Quality::Balanced;
+ r.quality = Quality::Fast;
let p = plan(&r, &i, &rec()).unwrap();
assert_eq!(p.mode, ResolvedMode::Edit);
assert_eq!(p.working, Size::new(1376, 768));
assert_eq!(p.references[0].canvas.canvas, p.working);
assert_eq!(p.output, Size::new(5120, 2880));
- assert_eq!((p.steps, p.cfg), (40, 1.0));
+ assert_eq!((p.steps, p.cfg), (20, 1.0));
let esrgan = p.estimated_seconds;
let p2 = plan(&r, &PlanInputs { upscaler: None, ..i.clone() }, &rec()).unwrap();
assert!(esrgan > p2.estimated_seconds, "the enlargement is included in the estimate");
@@ -1148,7 +1186,7 @@ mod tests {
r.quality = Quality::Quality;
let i = PlanInputs { image_sizes: vec![Size::new(5712, 4284)], ..env(true, RefineMode::Best) };
let p = plan(&r, &i, &rec()).unwrap();
- assert_eq!((p.steps, p.cfg, p.cache_threshold), (40, 1.0, None));
+ assert_eq!((p.steps, p.cfg, p.cache_threshold), (20, 1.0, None));
// Native 2K plus a 2K reference exceeds 16 GB: both get half the budget (~2 MP each).
// Each side rounds to 32 px, so allow the same 5 % slack as many_references_share_the_budget.
assert!((1_850_000..=2_202_010).contains(&p.working.area()), "{:?}", p.working);
diff --git a/crates/qls-core/src/process.rs b/crates/qls-core/src/process.rs
index aafabbd..df13b01 100644
--- a/crates/qls-core/src/process.rs
+++ b/crates/qls-core/src/process.rs
@@ -36,8 +36,14 @@ pub fn build_server_args(cfg: &AppConfig, profile: &ModelProfile) -> Vec<String>
a.push("--llm_vision".into());
a.push(p(v));
}
- let dev = cfg.device.to_ascii_lowercase();
- match cfg.text_encoder_placement {
+ let dev = cfg.effective_device();
+ let metal = cfg.gpu_backend == crate::config::GpuBackend::Metal;
+ let mut placement = cfg.text_encoder_placement;
+ if metal && placement == TextEncoderPlacement::Ram {
+ // Unified memory: "system RAM" is the GPU's memory too, so a CPU copy only costs.
+ placement = TextEncoderPlacement::Gpu;
+ }
+ match placement {
TextEncoderPlacement::Ram => {
// Weights in RAM, compute on the GPU. Explicit --params-backend disables
// auto-fit, so DiT/VAE weights live on their compute device.
@@ -46,11 +52,17 @@ pub fn build_server_args(cfg: &AppConfig, profile: &ModelProfile) -> Vec<String>
TextEncoderPlacement::Gpu => a.extend(["--backend".into(), dev.clone()]),
TextEncoderPlacement::Cpu => a.extend(["--backend".into(), format!("diffusion={dev},vae={dev},te=cpu")]),
TextEncoderPlacement::Auto => a.extend(["--backend".into(), format!("diffusion={dev},vae={dev}"), "--auto-fit".into(), "on".into()]),
+ TextEncoderPlacement::Disk => a.extend(["--backend".into(), dev.clone(), "--params-backend".into(), "disk".into()]),
+ }
+ if metal {
+ // Weights mapped from the files instead of copied (fixed for Metal upstream in #2037,
+ // part of the pinned build): about 10 GB less peak memory on Apple Silicon.
+ a.push("--mmap".into());
}
if cfg.flash_attention {
a.push("--fa".into());
}
- if cfg.sage_attention {
+ if cfg.uses_sage_attention() {
// Falls back to flash/default attention for layers it can't handle.
a.push("--sage-attn".into());
}
@@ -68,7 +80,7 @@ pub fn build_server_args(cfg: &AppConfig, profile: &ModelProfile) -> Vec<String>
"off" => a.extend(["--model-args".into(), "qwen_image_2_1_prefix_cache=false".into()]),
t => a.extend(["--model-args".into(), format!("qwen_image_2_1_prefix_cache_type={t}")]),
}
- if cfg.eager_load {
+ if cfg.eager_load && placement != TextEncoderPlacement::Disk {
a.push("--eager-load".into());
}
// ESRGAN models for the hires refine and `/sdcpp/v1/upscale`; the server scans the
@@ -314,7 +326,9 @@ pub struct ServerManager {
lines_tx: broadcast::Sender<String>,
}
+#[cfg(windows)]
const CREATE_NO_WINDOW: u32 = 0x0800_0000;
+#[cfg(windows)]
const CREATE_NEW_PROCESS_GROUP: u32 = 0x0000_0200;
impl ServerManager {
@@ -453,8 +467,8 @@ impl ServerManager {
let exe = cfg.server_exe();
if !exe.is_file() {
return Err(fail(
- AppError::new(ErrorCode::MissingModel, format!("sd-server.exe was not found in {}.", cfg.sdcpp_dir.display()))
- .suggest("Set the stable-diffusion.cpp folder in Settings."),
+ AppError::new(ErrorCode::MissingModel, format!("sd-server was not found in {}.", cfg.engine_dir().display()))
+ .suggest("Install it from Settings → Models → Set up models…, or set the stable-diffusion.cpp folder in Settings."),
));
}
for (label, path, required) in profile.files() {
@@ -821,6 +835,31 @@ mod tests {
assert!(!s.contains("--max-vram"));
}
+ #[test]
+ fn vulkan_launch() {
+ let mut cfg = AppConfig::default();
+ cfg.gpu_backend = crate::config::GpuBackend::Vulkan;
+ let s = build_server_args(&cfg, cfg.profile()).join(" ");
+ assert!(s.contains("--backend vulkan0 --params-backend te=cpu"), "{s}");
+ assert!(s.contains("--fa"));
+ assert!(!s.contains("--sage-attn"), "SageAttention is CUDA only");
+ assert!(s.contains("qwen_image_2_1_prefix_cache_type=q8_0"));
+ }
+
+ #[test]
+ fn metal_launch() {
+ let mut cfg = AppConfig::default();
+ cfg.gpu_backend = crate::config::GpuBackend::Metal;
+ let s = build_server_args(&cfg, cfg.profile()).join(" ");
+ assert!(s.contains("--backend metal --mmap --fa"), "RAM placement runs everything on the GPU: {s}");
+ assert!(!s.contains("--params-backend") && !s.contains("--sage-attn"));
+ assert!(s.contains("--mmap") && s.contains("--eager-load"));
+ cfg.text_encoder_placement = TextEncoderPlacement::Disk;
+ let s = build_server_args(&cfg, cfg.profile()).join(" ");
+ assert!(s.contains("--backend metal --params-backend disk"), "{s}");
+ assert!(!s.contains("--eager-load"), "nothing to preload when weights stream from disk");
+ }
+
#[test]
fn step_bar_parsing() {
assert_eq!(parse_step_bar(" |=====> | 2/20 - 1.99s/it"), Some((2, 20, Some(1.99))));
@@ -889,5 +928,8 @@ mod tests {
parse_device_list(out),
vec![("CUDA0".into(), "NVIDIA GeForce RTX 5070 Ti".into()), ("CPU".into(), "AMD Ryzen 7 7700 8-Core Processor".into())]
);
+ // ggml's Vulkan backend (device names as ggml registers them; not yet captured from a real run).
+ let out = "ggml_vulkan: Found 1 Vulkan devices:\nggml_vulkan: 0 = AMD Radeon RX 7800 XT (AMD proprietary driver) | uma: 0 | fp16: 1\nVulkan0\tAMD Radeon RX 7800 XT\nCPU\tAMD Ryzen 7 7700\n";
+ assert_eq!(parse_device_list(out)[0], ("Vulkan0".into(), "AMD Radeon RX 7800 XT".into()));
}
}
diff --git a/crates/qls-core/src/wic.rs b/crates/qls-core/src/wic.rs
index 8edf1ed..3bb9a9f 100644
--- a/crates/qls-core/src/wic.rs
+++ b/crates/qls-core/src/wic.rs
@@ -1,4 +1,5 @@
-//! Decoding through Windows Imaging Component, for formats the `image` crate lacks (HEIC).
+//! Decoding through the operating system, for formats the `image` crate lacks (HEIC):
+//! Windows Imaging Component on Windows, `sips` on macOS.
//! HEIC needs Microsoft's "HEIF Image Extensions" + "HEVC Video Extensions" (Store).
use image::RgbaImage;
@@ -85,9 +86,38 @@ impl Drop for ComGuard {
}
}
-#[cfg(not(windows))]
+/// macOS: the system's own converter (`sips`, which applies the file's rotation) to PNG,
+/// keeping the embedded ICC profile.
+#[cfg(target_os = "macos")]
+pub fn decode(bytes: &[u8]) -> Result<WicImage, String> {
+ use image::ImageDecoder;
+ let dir = std::env::temp_dir().join(format!("kokon-heic-{}", uuid::Uuid::new_v4()));
+ std::fs::create_dir_all(&dir).map_err(|e| e.to_string())?;
+ let result = (|| {
+ let (src, dst) = (dir.join("in.heic"), dir.join("out.png"));
+ std::fs::write(&src, bytes).map_err(|e| e.to_string())?;
+ let out = std::process::Command::new("/usr/bin/sips")
+ .args(["-s", "format", "png"])
+ .arg(&src)
+ .arg("--out")
+ .arg(&dst)
+ .output()
+ .map_err(|e| format!("Cannot run sips: {e}"))?;
+ if !out.status.success() {
+ return Err(format!("macOS could not convert the image: {}", String::from_utf8_lossy(&out.stderr).trim()));
+ }
+ let mut dec = image::codecs::png::PngDecoder::new(std::io::BufReader::new(std::fs::File::open(&dst).map_err(|e| e.to_string())?)).map_err(|e| e.to_string())?;
+ let icc_profile = dec.icc_profile().ok().flatten();
+ let pixels = image::DynamicImage::from_decoder(dec).map_err(|e| e.to_string())?.to_rgba8();
+ Ok(WicImage { pixels, icc_profile })
+ })();
+ let _ = std::fs::remove_dir_all(&dir);
+ result
+}
+
+#[cfg(not(any(windows, target_os = "macos")))]
pub fn decode(_bytes: &[u8]) -> Result<WicImage, String> {
- Err("HEIC images can only be opened on Windows.".into())
+ Err("HEIC images can only be opened on Windows and macOS.".into())
}
#[cfg(test)]
diff --git a/crates/qls-service/src/api.rs b/crates/qls-service/src/api.rs
index 0c428b3..cc786dd 100644
--- a/crates/qls-service/src/api.rs
+++ b/crates/qls-service/src/api.rs
@@ -18,7 +18,7 @@ use axum::{
use futures_util::stream::Stream;
use qls_core::{
catalog,
- config::AppConfig,
+ config::{AppConfig, GpuBackend},
diagnostics,
engine::{APP_VERSION, JobView},
error::{AppError, ErrorCode},
@@ -206,20 +206,29 @@ async fn setup(State(s): St) -> ApiResult<Json<Value>> {
/// What the setup wizard shows: the GPU, the recommended preset and, per preset, which files
/// are installed in the configured folder and how much is left to download.
-async fn setup_hardware(State(s): St) -> ApiResult<Json<Value>> {
+#[derive(Deserialize)]
+struct HardwareQ {
+ /// Engine to list the downloads for; default: the configured one.
+ backend: Option<GpuBackend>,
+}
+
+async fn setup_hardware(State(s): St, Query(q): Query<HardwareQ>) -> ApiResult<Json<Value>> {
let engine = s.engine.clone();
let v = tokio::task::spawn_blocking(move || {
let cfg = engine.config();
let gpu = engine.gpu();
- let detected = gpu.as_ref().map(|g| g.memory_total_mib);
+ let detected = gpu.as_ref().map(|g| g.planning_mib());
let vram = cfg.planning_vram_mib(detected);
let mut sys = sysinfo::System::new();
sys.refresh_memory();
let dir = cfg.sdcpp_dir.clone();
+ let vendor = gpu.as_ref().map(|g| g.vendor.clone());
+ let recommended_backend = vendor.as_deref().map(GpuBackend::for_vendor).unwrap_or_default();
+ let backend = q.backend.unwrap_or(if cfg.setup_complete { cfg.gpu_backend } else { recommended_backend });
let presets: Vec<Value> = catalog::PRESETS
.iter()
.map(|p| {
- let files = catalog::downloads_for(p, true);
+ let files = catalog::downloads_for(p, backend, true);
let (download, disk) = installer::space_needed(&dir, &files);
json!({
"id": p.id, "label": p.label, "summary": p.summary, "verified": p.verified, "min_vram_mib": p.min_vram_mib,
@@ -229,12 +238,16 @@ async fn setup_hardware(State(s): St) -> ApiResult<Json<Value>> {
.collect();
json!({
"gpu": gpu,
- "nvidia": gpu.is_some(),
+ "nvidia": vendor.as_deref() == Some("nvidia"),
+ "vendor": vendor,
+ "backend": backend,
+ "recommended_backend": recommended_backend,
"vram_mib": vram,
"vram_simulated": cfg.vram_override_mib > 0,
"ram_total_mib": sys.total_memory() / (1024 * 1024),
"recommended": catalog::recommended_preset(vram).id,
"current": cfg.hardware_preset,
+ "current_backend": cfg.gpu_backend,
"presets": presets,
"sdcpp_dir": dir,
"default_sdcpp_dir": qls_core::config::default_sdcpp_dir(),
@@ -250,6 +263,8 @@ async fn setup_hardware(State(s): St) -> ApiResult<Json<Value>> {
#[derive(Deserialize)]
struct InstallBody {
preset: String,
+ /// Engine to install; default: the configured one.
+ backend: Option<GpuBackend>,
#[serde(default = "yes")]
upscaler: bool,
}
@@ -263,6 +278,7 @@ fn yes() -> bool {
async fn install_start(State(s): St, Json(b): Json<InstallBody>) -> ApiResult<Json<Value>> {
let preset = catalog::preset(&b.preset).ok_or_else(|| AppError::invalid(format!("Unknown preset {}.", b.preset)))?;
let cfg = s.engine.config();
+ let backend = b.backend.unwrap_or(cfg.gpu_backend);
// A running server keeps its DLLs open; the engine zip may need to replace them.
let st = s.engine.server.status();
if st.owned && st.state != qls_core::process::ServerState::Stopped {
@@ -273,17 +289,23 @@ async fn install_start(State(s): St, Json(b): Json<InstallBody>) -> ApiResult<Js
s.engine.emit_server();
}
let engine = s.engine.clone();
+ // Below ~8 GiB of planning memory (a 16 GB Mac) only streaming weights from disk fits.
+ let small_unified = s.engine.gpu().is_some_and(|g| g.unified && g.planning_mib() < 8 * 1024);
let on_finish = Box::new(move |complete: bool| {
if !complete {
return;
}
let mut c = engine.config();
c.apply_preset(preset);
+ c.gpu_backend = backend;
+ if backend == GpuBackend::Metal && small_unified {
+ c.text_encoder_placement = qls_core::config::TextEncoderPlacement::Disk;
+ }
if let Err(e) = engine.set_config(c) {
tracing::error!("could not save the installed preset: {}", e.message);
}
});
- s.engine.installer.start(cfg.sdcpp_dir.clone(), preset.id, catalog::downloads_for(preset, b.upscaler), on_finish)?;
+ s.engine.installer.start(cfg.sdcpp_dir.clone(), preset.id, catalog::downloads_for(preset, backend, b.upscaler), on_finish)?;
Ok(Json(serde_json::to_value(s.engine.installer.status()).unwrap()))
}
diff --git a/docs/engineering-notes.md b/docs/engineering-notes.md
index 576e15e..b2f4923 100644
--- a/docs/engineering-notes.md
+++ b/docs/engineering-notes.md
@@ -54,9 +54,10 @@ during development (Qwen Local Studio) and are kept on purpose.
flow schedule (exponential shift 0.5–0.9, terminal 0.02). sd.cpp's own docs use CFG 6, which
runs a second, unconditional model pass every step; the app used that until 2026-10-07.
At CFG 1 the negative prompt is ignored.
- Presets: Fast 20 steps at ~1 MP (a shortcut, not official); Balanced 40 steps at ~1 MP;
- Quality 40 steps at the official 2K buckets (2048², 2752×1536…; edits share that budget
- with their reference images, so they land near 2 MP on 16 GB).
+ Presets (since 2026-10-09; 20 steps judged enough, only Ultra keeps the official 40):
+ Fast 20 steps at ~1 MP; Balanced 20 steps at ~1.5K (1536²); Quality 20 steps and Ultra
+ 40 steps at the official 2K buckets (2048², 2752×1536…; edits share that budget with their
+ reference images, so they land near 2 MP on 16 GB).
- Default launch: `--backend cuda0 --params-backend te=cpu --fa --sage-attn --vae-tiling
--vae-tile-size 768 --vae-tile-overlap 0.125 --max-vram -1` plus `--hires-upscalers-dir`
when the upscalers folder exists (text-encoder weights in RAM, computed on the GPU). Measured
@@ -75,8 +76,8 @@ during development (Qwen Local Studio) and are kept on purpose.
## Resolution and upscaling policy
-Every job runs a single diffusion pass at the working size (Auto: ~1 MP for Fast/Balanced,
-native 2K for Quality; or the 1K/1.5K/2K/Source/Custom choice), capped by the VRAM token budget
+Every job runs a single diffusion pass at the working size (Auto: ~1 MP for Fast, ~1.5K for
+Balanced, native 2K for Quality and Ultra; or the 1K/1.5K/2K/Source/Custom choice), capped by the VRAM token budget
that the target shares with its reference images. The earlier two-pass mode (≈1 MP render →
ESRGAN → low-denoise refine at 2K) was removed on 2026-10-06: in A/B tests the refine made
edits look worse than a single pass.
diff --git a/docs/platforms.md b/docs/platforms.md
new file mode 100644
index 0000000..43fe89e
--- /dev/null
+++ b/docs/platforms.md
@@ -0,0 +1,108 @@
+# Platforms: engines, requirements and what is verified
+
+Kokon runs Qwen-Image 2.1 through stable-diffusion.cpp. Each platform uses a prebuilt engine
+from the same pinned release ([master-929-3f8527a](https://github.com/leejet/stable-diffusion.cpp/releases/tag/master-929-3f8527a)),
+so model behaviour is identical everywhere; only speed and memory differ.
+
+| Platform | Engine | Status |
+| --- | --- | --- |
+| Windows + NVIDIA | CUDA 12 build | ✅ verified (RTX 5070 Ti 16 GB, RTX 5070 12 GB) |
+| Windows + AMD / Intel | Vulkan build (30 MB, uses the driver's Vulkan runtime) | 🟡 beta: works on NVIDIA (RTX 5070 Ti, 2K/20 steps, slower than CUDA); AMD / Intel to verify |
+| macOS + Apple Silicon | Metal build (universal binary, shaders embedded) | 🟡 built, first run on an M2 Pro 16 GB pending (`scripts/dev/mac-dev.sh`) |
+| Windows + AMD via ROCm | ROCm build | ✗ not planned: open crashes (#2008), DLL conflicts (#2012), black images with k-quants (#1385) |
+| Linux | Vulkan / CUDA builds exist | later |
+
+## How the app sizes itself (all platforms)
+
+One number drives everything: the memory the GPU can use for the diffusion model.
+
+- **Discrete GPU** (NVIDIA, AMD, Intel): its VRAM. NVIDIA through `nvidia-smi`; any other card
+ through DXGI (dedicated video memory). The text encoder lives in system RAM.
+- **Apple Silicon**: unified memory is shared, so the planner uses
+ `planner::unified_memory_planning_mib`: Metal's GPU share of RAM (≈ 2/3, ≈ 3/4 from 64 GB)
+ minus the text encoder, which sits in the same pool.
+
+From that number the existing logic picks the preset (which weights to download) and the token
+budget (largest working size per quality level). Nothing is tuned per platform.
+
+## Windows: AMD and Intel (Vulkan, beta)
+
+- Engine: `sd-master-3f8527a-bin-win-vulkan-x64.zip`, installed into `<sdcpp>\engine-vulkan`,
+ next to the CUDA build (switching engines needs no re-download).
+- Launch: `--backend vulkan0`, flash attention, q8_0 prefix cache. No SageAttention (CUDA only).
+- Requirements: a discrete GPU with **12 GB+ VRAM** (RX 6700 XT 12 GB / 7700 XT and up,
+ Arc A770 16 GB / B580 12 GB), current driver, 16 GB RAM.
+- Known upstream issues:
+ - integrated GPUs report 0 MB and fail (#2022), so they are not supported;
+ - an edit with 2 reference images can crash (#1991, fix in ggml not yet synced);
+ - no published Qwen 2.1 speeds on AMD/Intel yet.
+
+## macOS: Apple Silicon (in testing)
+
+Evidence: [qwen-image-studio](https://github.com/sanketshinde3001/qwen-image-studio) runs Qwen 2.1
+t2i and edits on a 16 GB M5 with **our exact engine build**. Its numbers: 10–12 GB peak
+(`--mmap --params-backend disk`), about 75 s/step at 1024² with CFG 6, so about 37 s per model
+pass. Kokon uses CFG 1, which is one pass per step.
+
+### Draft system requirements
+
+| | Minimum | Recommended |
+| --- | --- | --- |
+| **Chip** | Apple M1 or newer (Intel Macs: no) | M-series Pro / Max / Ultra |
+| **macOS** | 26 Tahoe (the engine binary's minimum) | latest |
+| **Unified memory** | 24 GB (Fast/Balanced, up to ~1.5K) | 32 GB+ (Quality 2K); 48 GB+ gets the 8-bit model |
+| **Disk** | 15 GB free | 25 GB on the internal SSD |
+
+16 GB Macs can run it only in the lightest setup: weights streamed from disk, 1 MP, much
+slower. It should be offered as "not supported, try at your own risk" rather than blocked.
+
+### Memory math
+
+Weights: diffusion Q4_K_M 4.2 GB + text encoder Q4_K_M 5.0 GB + vision projector 1.2 GB +
+VAE 0.25 GB ≈ 10.7 GB.
+
+| Unified memory | GPU share | − text encoder | Plans like a | Preset |
+| --- | --- | --- | --- | --- |
+| 16 GB | ~10.7 GB | ~4.5 GB | 4–6 GB card | lightest, 1 MP |
+| 24 GB | ~16 GB | ~9.8 GB | 10 GB card | 12 GB preset |
+| 32 GB | ~21 GB | ~15 GB | 16 GB card | 16 GB preset (2K) |
+| 48 GB | ~32 GB | ~26 GB | 24 GB+ card | 24 GB preset (Q8 model) |
+| 64 GB+ | ~48 GB | ~42 GB | 32 GB+ card | 32 GB+ preset |
+
+### Expected speed
+
+These are **estimates, not measured**. They scale the M5 result by GPU size and are for
+1 MP, 20 steps, CFG 1.
+
+| Chip class | GPU cores | Bandwidth | Per image (rough) |
+| --- | --- | --- | --- |
+| M1–M5 (base) | 7–10 | 68–153 GB/s | 10–15 min |
+| Pro | 14–20 | 150–273 GB/s | 4–6 min |
+| Max | 30–40 | 300–546 GB/s | 2–3 min |
+| Ultra | 48–80 | ~800 GB/s | 1–2 min |
+
+The setup screen should show the chip, its memory and this expected time before downloading,
+so nobody with a base chip is surprised.
+
+### Work needed (✅ = done 2026-10-09, runs from source with `scripts/dev/mac-dev.sh`)
+
+1. ✅ Detection: `sysctl hw.memsize`, `machdep.cpu.brand_string`, GPU cores from
+ `system_profiler SPDisplaysDataType -json`; feed `unified_memory_planning_mib` into the
+ planner and presets.
+2. ✅ Engine: the macOS zip (`sd-server`, `libstable-diffusion.dylib`) in `<sdcpp>/engine-metal`;
+ launch with `--backend metal` (sd-server maps "metal" to ggml's MTL device), `--fa`, `--mmap`,
+ no SageAttention; RAM placement becomes "everything on the GPU" (same memory), and Macs
+ that plan below 8 GiB (16 GB) get `--params-backend disk`.
+3. ✅ Portability fixes (all Rust type-checks for aarch64-apple-darwin):
+ - catalog paths use `\` (switch to `/`);
+ - HEIC decoding uses Windows WIC (macOS: `sips`);
+ - Tauri sidecar names and the `.exe` suffix;
+ - data folders (`dirs` already maps them to `~/Library/...`).
+4. Packaging (todo; `tauri.macos.conf.json` prepared): a GitHub Actions `macos-14` (arm64) runner builds the `.dmg`. Without an Apple
+ Developer ID ($99/year) the app is unsigned, so users must right-click → Open the first
+ time. The engine itself, downloaded by the app over HTTP, carries no quarantine flag and
+ runs, because it is ad-hoc signed.
+5. Known Metal issue: transparent (RGBA) output decodes as white (#2024, fix not merged).
+ Kokon outputs RGB, so only transparency-preserving edits are affected.
+6. Testing: needs a real Apple Silicon Mac with 24 GB+ (GitHub's runners have 7 GB: build
+ only). Without one, ship as "beta, untested".
diff --git a/mcp/src/tools.ts b/mcp/src/tools.ts
index cf5933a..26e4cc8 100644
--- a/mcp/src/tools.ts
+++ b/mcp/src/tools.ts
@@ -5,7 +5,9 @@ import { z } from "zod";
import { allowedRoots, checkInput, checkOutput, PathError } from "./paths.js";
import { ServiceClient, ServiceError } from "./service.js";
-const quality = z.enum(["fast", "balanced", "quality"]).describe("fast = ~1 MP, 20 steps; balanced = ~1 MP, 40 steps (the default); quality = native 2K, 40 steps (slower; edits share 2K with their references)");
+const quality = z
+ .enum(["fast", "balanced", "quality", "ultra"])
+ .describe("fast = ~1 MP, 20 steps; balanced = ~1.5K, 20 steps (the default); quality = native 2K, 20 steps (slower; edits share 2K with their references); ultra = native 2K, 40 steps (slowest)");
const seed = z.number().int().min(-1).max(2_147_483_647).describe("Seed for reproducible results; omit or -1 for random");
const dim = z.number().int().min(64).max(4096);
const aspect = z
diff --git a/mcp/test/tools.test.ts b/mcp/test/tools.test.ts
index 414e807..4f487a9 100644
--- a/mcp/test/tools.test.ts
+++ b/mcp/test/tools.test.ts
@@ -14,7 +14,8 @@ describe("input validation", () => {
expect(obj("generate_image").safeParse({ prompt: "x", width: 5000, height: 512 }).success).toBe(false);
expect(obj("generate_image").safeParse({ prompt: "x", aspect_ratio: "16:9" }).success).toBe(true);
expect(obj("generate_image").safeParse({ prompt: "x", aspect_ratio: "wide" }).success).toBe(false);
- expect(obj("generate_image").safeParse({ prompt: "x", quality: "ultra" }).success).toBe(false);
+ expect(obj("generate_image").safeParse({ prompt: "x", quality: "ultra" }).success).toBe(true);
+ expect(obj("generate_image").safeParse({ prompt: "x", quality: "max" }).success).toBe(false);
expect(obj("generate_image").safeParse({ prompt: "x", seed: -2 }).success).toBe(false);
});
it("edit_images limits and order", () => {
diff --git a/scripts/dev/mac-dev.sh b/scripts/dev/mac-dev.sh
new file mode 100755
index 0000000..19108c8
--- /dev/null
+++ b/scripts/dev/mac-dev.sh
@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+# Runs Kokon from source on an Apple Silicon Mac (development build, no installer).
+#
+# bash scripts/dev/mac-dev.sh # build and start the app
+# bash scripts/dev/mac-dev.sh --check # only check the prerequisites
+#
+# Needs: macOS 26+, Xcode Command Line Tools, Rust (rustup), Node.js 22+.
+set -euo pipefail
+cd "$(dirname "$0")/../.."
+ROOT="$(pwd)"
+TRIPLE=aarch64-apple-darwin
+
+fail() { echo "✗ $1"; exit 1; }
+ok() { echo "✓ $1"; }
+
+[ "$(uname -s)" = Darwin ] || fail "This script is for macOS."
+[ "$(uname -m)" = arm64 ] || fail "Kokon on Mac needs Apple Silicon (M1 or newer)."
+MACOS="$(sw_vers -productVersion)"
+[ "${MACOS%%.*}" -ge 26 ] || fail "macOS $MACOS: the stable-diffusion.cpp build needs macOS 26 (Tahoe) or newer."
+ok "macOS $MACOS on $(sysctl -n machdep.cpu.brand_string), $(( $(sysctl -n hw.memsize) / 1073741824 )) GB unified memory"
+xcode-select -p >/dev/null 2>&1 || fail "Xcode Command Line Tools missing: run xcode-select --install"
+ok "Xcode Command Line Tools"
+command -v cargo >/dev/null || fail "Rust missing: run curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh then open a new terminal"
+ok "$(cargo --version)"
+command -v node >/dev/null || fail "Node.js missing: install Node 22 LTS from https://nodejs.org (or brew install node@22)"
+NODE_MAJOR="$(node -p 'process.versions.node.split(".")[0]')"
+[ "$NODE_MAJOR" -ge 22 ] || fail "Node $(node -v): version 22 or newer is needed."
+ok "Node $(node -v)"
+[ "${1:-}" = "--check" ] && exit 0
+
+echo "→ JavaScript packages"
+npm ci --no-audit --no-fund
+(cd mcp && npm ci --no-audit --no-fund && npm run build)
+
+echo "→ kokon-service (Rust, first build takes a few minutes)"
+cargo build -p qls-service
+
+# Tauri expects its sidecars under src-tauri/binaries with the target triple.
+mkdir -p src-tauri/binaries
+cp target/debug/kokon-service "src-tauri/binaries/kokon-service-$TRIPLE"
+# Dev stand-in for the MCP executable (the Windows build packs Node into one .exe).
+cat > "src-tauri/binaries/kokon-mcp-$TRIPLE" <<EOF
+#!/bin/sh
+exec node "$ROOT/mcp/build/kokon-mcp.cjs" "\$@"
+EOF
+chmod +x "src-tauri/binaries/kokon-mcp-$TRIPLE"
+
+echo "→ Starting Kokon (close the window to stop)"
+npx tauri dev
diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs
index 3b4bf48..e4f4df6 100644
--- a/src-tauri/src/lib.rs
+++ b/src-tauri/src/lib.rs
@@ -68,7 +68,8 @@ fn ping(port: u16) -> bool {
fn service_exe() -> Option<PathBuf> {
let me = std::env::current_exe().ok()?;
let dir = me.parent()?;
- for name in ["kokon-service.exe", "kokon-service-x86_64-pc-windows-msvc.exe"] {
+ // Bundled sidecar (Tauri strips the target suffix) or a dev build next to the shell.
+ for name in ["kokon-service.exe", "kokon-service-x86_64-pc-windows-msvc.exe", "kokon-service", "kokon-service-aarch64-apple-darwin"] {
let p = dir.join(name);
if p.is_file() {
return Some(p);
@@ -78,7 +79,7 @@ fn service_exe() -> Option<PathBuf> {
}
fn spawn_service() -> Result<(), String> {
- let exe = service_exe().ok_or("kokon-service.exe was not found next to the application.")?;
+ let exe = service_exe().ok_or("kokon-service was not found next to the application.")?;
let mut cmd = std::process::Command::new(exe);
cmd.stdin(std::process::Stdio::null()).stdout(std::process::Stdio::null()).stderr(std::process::Stdio::null());
#[cfg(windows)]
@@ -108,7 +109,7 @@ fn ensure_service(shared: &Shared) -> Result<ServiceInfo, String> {
return Ok(i);
}
}
- Err("The Kokon service did not start. See %LOCALAPPDATA%\\KokonData\\logs\\service.log.".into())
+ Err(format!("The Kokon service did not start. See {}.", data_dir().join("logs").join("service.log").display()))
}
fn heartbeat(info: &ServiceInfo) -> bool {
diff --git a/src-tauri/tauri.macos.conf.json b/src-tauri/tauri.macos.conf.json
new file mode 100644
index 0000000..304f0a5
--- /dev/null
+++ b/src-tauri/tauri.macos.conf.json
@@ -0,0 +1,7 @@
+{
+ "bundle": {
+ "targets": ["app", "dmg"],
+ "icon": ["icons/32x32.png", "icons/128x128.png", "icons/[email protected]", "icons/icon.icns"],
+ "macOS": { "minimumSystemVersion": "26.0" }
+ }
+}
diff --git a/src/App.tsx b/src/App.tsx
index 12baf72..22a23e5 100644
--- a/src/App.tsx
+++ b/src/App.tsx
@@ -15,6 +15,7 @@ import { Setup } from "./components/Setup";
import { ProjectTitle, Sidebar } from "./components/Sidebar";
import { Stage } from "./components/Stage";
import { IconButton } from "./components/ui";
+import { shortcut } from "./lib/platform";
const IMAGE_EXT = /\.(png|jpe?g|webp|bmp|gif|tiff?|heic|heif)$/i;
@@ -68,10 +69,10 @@ function HistoryButtons() {
if (!canUndo && !canRedo) return null;
return (
<>
- <IconButton title="Undo: back to the previous version (Ctrl+Z)" onClick={undoVersion} disabled={!canUndo}>
+ <IconButton title={`Undo: back to the previous version (${shortcut("Z")})`} onClick={undoVersion} disabled={!canUndo}>
<Undo2 size={18} />
</IconButton>
- <IconButton title="Redo (Ctrl+Y)" onClick={redoVersion} disabled={!canRedo}>
+ <IconButton title={`Redo (${shortcut("Y")})`} onClick={redoVersion} disabled={!canRedo}>
<Redo2 size={18} />
</IconButton>
</>
diff --git a/src/components/Advanced.tsx b/src/components/Advanced.tsx
index ba75bf2..d9f3b93 100644
--- a/src/components/Advanced.tsx
+++ b/src/components/Advanced.tsx
@@ -1,22 +1,12 @@
import { Dices, RotateCcw } from "lucide-react";
-import { planPipeline } from "../lib/logic";
import { updateComposer, useStore } from "../lib/store";
-import type { Advanced, OutputFormat, Plan, ResolutionMode } from "../lib/types";
+import type { Advanced, OutputFormat, Plan } from "../lib/types";
import { Seg, Switch } from "./ui";
// Verified against sd-server 3f8527a capabilities; replaced by the live list when loaded.
const FALLBACK_SAMPLERS = ["euler", "euler_a", "heun", "dpm2", "dpm++2s_a", "dpm++2m", "dpm++2mv2", "ipndm", "ipndm_v", "lcm", "ddim_trailing", "tcd", "res_multistep", "res_2s", "er_sde"];
const FALLBACK_SCHEDULERS = ["discrete", "karras", "exponential", "ays", "gits", "sgm_uniform", "simple", "smoothstep", "kl_optimal", "lcm", "beta", "flux"];
-const RESOLUTIONS: { value: ResolutionMode; label: string }[] = [
- { value: "auto", label: "Auto" },
- { value: "source", label: "Source" },
- { value: "1k", label: "1K" },
- { value: "1.5k", label: "1.5K" },
- { value: "2k", label: "2K" },
- { value: "custom", label: "Custom" },
-];
-
function NumField(props: { label: string; value: number | null | undefined; placeholder: string; min: number; max: number; step?: number; onChange: (v: number | null) => void }) {
return (
<label className="field-row">
@@ -49,19 +39,6 @@ export function AdvancedPanel({ plan }: { plan: Plan | null }) {
return (
<>
- <div className="field">
- <span>Resolution</span>
- <Seg value={c.resolution} options={RESOLUTIONS} onChange={(v) => updateComposer({ resolution: v })} />
- {c.resolution === "custom" && (
- <div style={{ display: "flex", gap: 8, alignItems: "center" }}>
- <input className="input" type="number" min={64} max={4096} value={c.customWidth} onChange={(e) => updateComposer({ customWidth: Math.max(64, Math.min(4096, Number(e.target.value) || 64)) })} style={{ width: 90 }} />
- <span className="muted">×</span>
- <input className="input" type="number" min={64} max={4096} value={c.customHeight} onChange={(e) => updateComposer({ customHeight: Math.max(64, Math.min(4096, Number(e.target.value) || 64)) })} style={{ width: 90 }} />
- </div>
- )}
- {plan && <span className="muted">{planPipeline(plan)}</span>}
- </div>
-
<label className="field-row">
<span>Seed</span>
<div style={{ display: "flex", gap: 6, alignItems: "center" }}>
@@ -150,7 +127,7 @@ export function AdvancedPanel({ plan }: { plan: Plan | null }) {
<textarea className="input" rows={2} value={c.negative} placeholder="Optional; used only when CFG is above 1" onChange={(e) => updateComposer({ negative: e.target.value })} />
</label>
- <button className="btn" onClick={() => updateComposer({ advanced: {}, seed: null, negative: "", resolution: "auto" })}>
+ <button className="btn" onClick={() => updateComposer({ advanced: {}, seed: null, negative: "" })}>
<RotateCcw size={14} /> Reset to automatic
</button>
</>
diff --git a/src/components/Compare.tsx b/src/components/Compare.tsx
index c115d39..e53b360 100644
--- a/src/components/Compare.tsx
+++ b/src/components/Compare.tsx
@@ -117,6 +117,11 @@ export function CompareFrame({ g, base, fullscreen = false }: { g: GenerationRec
</>
)}
{label && <span className="compare-tag left">{label}</span>}
+ {!fullscreen && (
+ <div className="compare-overlay" onPointerDown={(e) => e.stopPropagation()} onDoubleClick={(e) => e.stopPropagation()}>
+ <CompareBar />
+ </div>
+ )}
{!fullscreen && (
<FullscreenButton
hint="Hold B for the original"
diff --git a/src/components/Composer.tsx b/src/components/Composer.tsx
index 736674c..78fb6d7 100644
--- a/src/components/Composer.tsx
+++ b/src/components/Composer.tsx
@@ -1,19 +1,21 @@
-import { ArrowUp, Brush, ChevronDown, Expand, FolderOpen, Gauge, ImagePlus, RectangleHorizontal, RectangleVertical, SlidersHorizontal, Sparkles, Square, X } from "lucide-react";
+import { ArrowUp, Brush, ChevronDown, Expand, FolderOpen, Gauge, ImagePlus, Maximize, Pencil, SlidersHorizontal, Sparkles, X } from "lucide-react";
import { useEffect, useMemo, useRef, useState } from "react";
import { api, urls } from "../lib/api";
-import { activeOverrides, buildRequest, expansionPixels, formatDuration, hasOutpaint, moveRef, needsPrompt, versionLabels, type Ref } from "../lib/logic";
+import { activeOverrides, aspectLabel, buildRequest, expansionPixels, formatDuration, hasOutpaint, imageName, moveRef, needsPrompt, parseRatio, type Ref } from "../lib/logic";
import { native } from "../lib/native";
import {
- addAsReference, addInputs, getState, itemSource, selectedItem, selectItem, setState, submitComposer, toastError, updateComposer, useStore, type SessionItem,
+ addAsReference, addInputs, editShownImage, getState, itemSource, selectedItem, sessionLabels, setState, startNewImage, submitComposer, toastError, updateComposer, useStore, type SessionItem,
} from "../lib/store";
import type { InpaintAction, Plan, Quality } from "../lib/types";
import { AdvancedPanel } from "./Advanced";
+import { AspectGrid, pickAspect, RatioIcon, resolutionLabel, ResolutionMenu } from "./SizeMenus";
import { Option, Popover, Seg } from "./ui";
const QUALITY: { value: Quality; label: string; sub: string }[] = [
{ value: "fast", label: "Fast", sub: "~1 MP, 20 steps" },
- { value: "balanced", label: "Balanced", sub: "~1 MP, 40 steps · recommended" },
- { value: "quality", label: "Quality", sub: "native 2K, 40 steps · slower" },
+ { value: "balanced", label: "Balanced", sub: "~1.5K, 20 steps · recommended" },
+ { value: "quality", label: "Quality", sub: "native 2K, 20 steps · slower" },
+ { value: "ultra", label: "Ultra", sub: "native 2K, 40 steps · slowest" },
];
const INPAINT_ACTIONS: { value: InpaintAction; label: string; placeholder: string }[] = [
@@ -25,14 +27,6 @@ const INPAINT_ACTIONS: { value: InpaintAction; label: string; placeholder: strin
const INPAINT_VERB: Record<InpaintAction, string> = { edit: "Changing", remove: "Clearing", replace: "Replacing", add: "Adding to" };
-const ASPECT_ICON: Record<string, React.ReactNode> = {
- "1:1": <Square size={15} />,
- "4:3": <RectangleHorizontal size={15} />,
- "16:9": <RectangleHorizontal size={15} />,
- "3:4": <RectangleVertical size={15} />,
- "9:16": <RectangleVertical size={15} />,
-};
-
export async function importPaths(paths: string[]) {
const recs = [];
for (const p of paths) {
@@ -45,15 +39,15 @@ export async function importPaths(paths: string[]) {
addInputs(recs);
}
-/** Version name of a reference when it is part of the project ("v2", "Original"). */
+/** Name of a reference when it is part of the project ("Image 1.2"). */
function useRefNames() {
const session = useStore((s) => s.session);
return useMemo(() => {
- const labels = versionLabels(session);
+ const labels = sessionLabels(session);
const bySource = new Map<string, string>();
for (const i of session) {
const s = itemSource(i);
- if (s) bySource.set(s, labels[i.key]);
+ if (s) bySource.set(s, imageName(labels[i.key]));
}
return (r: Ref) => bySource.get(r.source) ?? null;
}, [session]);
@@ -128,35 +122,45 @@ function RefStrip() {
);
}
+/** Edit image | New image: whether the prompt changes an image or makes a new one. */
+function ModeSwitch({ editing }: { editing: boolean }) {
+ const sel = useStore(selectedItem);
+ const session = useStore((s) => s.session);
+ const img = sel && (sel.gen || sel.input) ? sel : [...session].reverse().find((i) => i.gen || i.input);
+ const name = img ? imageName(sessionLabels(session)[img.key]).toLowerCase() : "";
+ return (
+ <div className="mode-switch" role="radiogroup" aria-label="What the prompt does">
+ <button className={editing ? "" : "on"} role="radio" aria-checked={!editing} onClick={() => editing && startNewImage()} title="Make a new image from the prompt alone; your images stay in the timeline">
+ <Sparkles size={14} /> New image
+ </button>
+ <button className={editing ? "on" : ""} role="radio" aria-checked={editing} onClick={() => !editing && editShownImage()} title={editing ? "The prompt edits this image" : `Edit ${name}, the image shown`}>
+ <Pencil size={14} /> Edit image
+ </button>
+ </div>
+ );
+}
+
/** Says, in words, what the prompt is going to do, and offers the tools for that image. */
function TargetBar() {
const c = useStore((s) => s.composer);
- const sel = useStore(selectedItem);
const session = useStore((s) => s.session);
const nameOf = useRefNames();
if (!c.refs.length) {
- const img = sel && (sel.gen || sel.input) ? sel : [...session].reverse().find((i) => i.gen || i.input);
- if (!img) return null;
+ if (!session.some((i) => i.gen || i.input)) return null;
return (
<div className="target create">
- <span className="target-icon">
- <Sparkles size={15} />
- </span>
+ <ModeSwitch editing={false} />
<span className="target-text">
<span className="what">New image</span>
<span className="sub">Made from the prompt alone</span>
</span>
- <span className="spacer" />
- <button className="tool" onClick={() => selectItem(img.key)}>
- Edit {versionLabels(session)[img.key]} instead
- </button>
</div>
);
}
const first = c.refs[0];
- const name = nameOf(first) ?? (first.source.startsWith("gen:") ? "this result" : first.name);
+ const name = nameOf(first)?.toLowerCase() ?? (first.source.startsWith("gen:") ? "this result" : first.name);
const outPx = hasOutpaint(c.outpaint) ? expansionPixels(c.outpaint!, first.width, first.height) : null;
const what =
c.refs.length > 1 ? `Combining ${c.refs.length} images` : c.mask ? `${INPAINT_VERB[c.inpaintAction]} the painted area of ${name}` : outPx ? `Expanding ${name}` : `Editing ${name}`;
@@ -169,6 +173,7 @@ function TargetBar() {
return (
<div className="target">
+ <ModeSwitch editing />
<RefStrip />
<span className="target-text">
<span className="what">{what}</span>
@@ -195,9 +200,6 @@ function TargetBar() {
</button>
)}
</span>
- <button className="tool" title="Create a new image from the prompt instead" onClick={() => updateComposer({ refs: [], mask: null, outpaint: null })}>
- New image
- </button>
</div>
);
}
@@ -206,7 +208,7 @@ function TargetBar() {
function AddImage() {
const session = useStore((s) => s.session);
const refs = useStore((s) => s.composer.refs);
- const labels = useMemo(() => versionLabels(session), [session]);
+ const labels = useMemo(() => sessionLabels(session), [session]);
const used = new Set(refs.map((r) => r.source));
const candidates = session.filter((i) => {
const s = itemSource(i);
@@ -233,10 +235,10 @@ function AddImage() {
>
{(close) => (
<div className="add-menu">
- <div className="label">{refs.length ? "Add a version as reference" : "Edit a version"}</div>
+ <div className="label">{refs.length ? "Add an image as reference" : "Edit an image"}</div>
<div className="add-grid">
{candidates.map((it: SessionItem) => (
- <button key={it.key} title={it.gen?.prompt ?? labels[it.key]} onClick={() => (addAsReference(it), close())}>
+ <button key={it.key} title={it.gen?.prompt ?? imageName(labels[it.key])} onClick={() => (addAsReference(it), close())}>
<img src={urls.sourceThumb(itemSource(it)!)} alt="" draggable={false} />
<span>{labels[it.key]}</span>
</button>
@@ -323,7 +325,12 @@ export function Composer() {
.filter(Boolean)
.join("\n")
: "";
- const shaped = c.resolution !== "custom" && !c.mask && !outPx;
+ const shaped = !c.mask && !outPx;
+ const custom = c.resolution === "custom";
+ const aspectText = custom ? aspectLabel(c.customWidth, c.customHeight) : c.aspect === "auto" ? (plan ? plan.aspect_label : "Auto") : c.aspect;
+ const aspectRatioNow = custom
+ ? c.customWidth / c.customHeight
+ : (parseRatio(c.aspect) ?? (plan ? plan.output.width / plan.output.height : 1));
return (
<div className="composer-wrap">
@@ -360,25 +367,32 @@ export function Composer() {
{shaped && (
<Popover
+ className="wide"
trigger={(open, toggle) => (
<button className={`pill ${open ? "on" : ""}`} onClick={toggle} title="Aspect ratio">
- {ASPECT_ICON[c.aspect] ?? <RectangleHorizontal size={15} />}
- {c.aspect === "auto" ? (plan ? plan.aspect_label : "Auto") : c.aspect}
+ <RatioIcon ratio={aspectRatioNow} size={15} />
+ {aspectText}
<ChevronDown size={13} />
</button>
)}
>
- {(close) => (
- <>
- {["auto", "1:1", "4:3", "3:4", "16:9", "9:16"].map((a) => (
- <Option key={a} selected={c.aspect === a} label={a === "auto" ? "Auto" : a} sub={a === "auto" ? (hasImages ? "Same as image 1" : "From the prompt") : undefined} icon={ASPECT_ICON[a]} onClick={() => (updateComposer({ aspect: a }), close())} />
- ))}
- <Option label="Custom size…" icon={<SlidersHorizontal size={15} />} onClick={() => (updateComposer({ resolution: "custom" }), close())} />
- </>
- )}
+ {(close) => <AspectGrid value={custom ? aspectLabel(c.customWidth, c.customHeight) : c.aspect} onPick={(r) => (pickAspect(r), close())} auto={{ sub: hasImages ? "Same as image 1" : "From the prompt, else square" }} />}
</Popover>
)}
+ <Popover
+ className="wide"
+ trigger={(open, toggle) => (
+ <button className={`pill ${open ? "on" : ""}`} onClick={toggle} title="Resolution">
+ <Maximize size={14} />
+ {resolutionLabel(c.resolution, c.customWidth, c.customHeight)}
+ <ChevronDown size={13} />
+ </button>
+ )}
+ >
+ {(close) => <ResolutionMenu plan={plan} close={close} />}
+ </Popover>
+
<Popover
trigger={(open, toggle) => (
<button className={`pill ${open ? "on" : ""}`} onClick={toggle} title="Quality">
diff --git a/src/components/ExpandEditor.tsx b/src/components/ExpandEditor.tsx
index 76a0758..d74edec 100644
--- a/src/components/ExpandEditor.tsx
+++ b/src/components/ExpandEditor.tsx
@@ -1,14 +1,43 @@
-import { ArrowDown, ArrowLeft, ArrowRight, ArrowUp, Check, RectangleHorizontal, RectangleVertical, RotateCcw, Square, X } from "lucide-react";
+import { ArrowDown, ArrowLeft, ArrowRight, ArrowUp, Check, ChevronDown, Maximize, RotateCcw, X } from "lucide-react";
import { useEffect, useLayoutEffect, useRef, useState } from "react";
import { urls } from "../lib/api";
-import { expansionPixels, expansionPreset } from "../lib/logic";
+import { aspectLabel, expandToRatio, expandToSize, expansionPixels, expansionPreset, parseRatio } from "../lib/logic";
import { focusPrompt, setState, updateComposer, useStore } from "../lib/store";
import type { OutpaintSpec, Units } from "../lib/types";
-import { IconButton, Seg } from "./ui";
+import { AspectGrid, RatioIcon } from "./SizeMenus";
+import { IconButton, Popover, Seg } from "./ui";
type Side = "left" | "right" | "top" | "bottom";
const MAX_FINAL = 16384;
+/** Exact final size: each side can only grow (an expansion never crops). */
+function SizeForm(props: { w: number; h: number; minW: number; minH: number; onApply: (w: number, h: number) => void }) {
+ const [w, setW] = useState(props.w);
+ const [h, setH] = useState(props.h);
+ const ok = w >= props.minW && h >= props.minH && w <= MAX_FINAL && h <= MAX_FINAL && (w > props.minW || h > props.minH);
+ return (
+ <div className="expand-size">
+ <span className="pair">
+ <input className="input" type="number" min={props.minW} max={MAX_FINAL} value={w} onChange={(e) => setW(Math.round(Number(e.target.value) || 0))} onKeyDown={(e) => e.key === "Enter" && ok && props.onApply(w, h)} aria-label="Final width" />
+ <span className="muted">×</span>
+ <input className="input" type="number" min={props.minH} max={MAX_FINAL} value={h} onChange={(e) => setH(Math.round(Number(e.target.value) || 0))} onKeyDown={(e) => e.key === "Enter" && ok && props.onApply(w, h)} aria-label="Final height" />
+ </span>
+ {w > 0 && h > 0 && (
+ <span className="ratio-tag">
+ <RatioIcon ratio={w / h} size={14} />
+ {aspectLabel(w, h)}
+ </span>
+ )}
+ <span className="muted">
+ At least {props.minW}×{props.minH} (the image itself); the extra space is split evenly.
+ </span>
+ <button className="btn" disabled={!ok} onClick={() => props.onApply(w, h)}>
+ Use this size
+ </button>
+ </div>
+ );
+}
+
export function ExpandEditor() {
const ref0 = useStore((s) => s.composer.refs[0]);
const initial = useStore((s) => s.composer.outpaint);
@@ -72,9 +101,8 @@ export function ExpandEditor() {
setFrozenScale(null);
};
- const preset = (k: Parameters<typeof expansionPreset>[0]) => {
- const cur = { ...px, units: "pixels" as Units };
- const next = expansionPreset(k, W, H, cur);
+ const preset = (k: Parameters<typeof expansionPreset>[0]) => applyPixels(expansionPreset(k, W, H, { ...px, units: "pixels" as Units }));
+ const applyPixels = (next: OutpaintSpec) => {
if (spec.units === "percent") {
setSpec({ left: toPct(next.left, W), right: toPct(next.right, W), top: toPct(next.top, H), bottom: toPct(next.bottom, H), units: "percent" });
} else setSpec(next);
@@ -97,7 +125,7 @@ export function ExpandEditor() {
<div className="overlay-head">
<h2>Expand the image</h2>
<span className="muted">
- {W}×{H} → {finalW}×{finalH}
+ {W}×{H} ({aspectLabel(W, H)}) → <strong className="expand-final">{finalW}×{finalH} ({aspectLabel(finalW, finalH)})</strong>
</span>
<span className="spacer" />
<IconButton title="Cancel (Esc)" onClick={() => setState({ overlay: null })}>
@@ -133,15 +161,49 @@ export function ExpandEditor() {
</IconButton>
</div>
<div className="group">
- <IconButton title="Landscape 16:9" onClick={() => preset("landscape")}>
- <RectangleHorizontal size={17} />
- </IconButton>
- <IconButton title="Portrait 9:16" onClick={() => preset("portrait")}>
- <RectangleVertical size={17} />
- </IconButton>
- <IconButton title="Square" onClick={() => preset("square")}>
- <Square size={17} />
- </IconButton>
+ <Popover
+ className="wide"
+ trigger={(open, toggle) => (
+ <button className={`pill ${open ? "on" : ""}`} onClick={toggle} title="Expand to an aspect ratio">
+ <RatioIcon ratio={finalW / finalH} size={15} />
+ {aspectLabel(finalW, finalH)}
+ <ChevronDown size={13} />
+ </button>
+ )}
+ >
+ {(close) => (
+ <AspectGrid
+ value={aspectLabel(finalW, finalH)}
+ onPick={(r) => {
+ const ratio = parseRatio(r);
+ if (ratio) applyPixels(expandToRatio(W, H, ratio));
+ close();
+ }}
+ />
+ )}
+ </Popover>
+ <Popover
+ trigger={(open, toggle) => (
+ <button className={`pill ${open ? "on" : ""}`} onClick={toggle} title="Expand to an exact size">
+ <Maximize size={14} />
+ {finalW}×{finalH}
+ <ChevronDown size={13} />
+ </button>
+ )}
+ >
+ {(close) => (
+ <SizeForm
+ w={finalW}
+ h={finalH}
+ minW={W}
+ minH={H}
+ onApply={(w, h) => {
+ applyPixels(expandToSize(W, H, w, h));
+ close();
+ }}
+ />
+ )}
+ </Popover>
<IconButton title="Reset" onClick={() => setSpec({ left: 0, right: 0, top: 0, bottom: 0, units: spec.units })}>
<RotateCcw size={17} />
</IconButton>
diff --git a/src/components/InpaintEditor.tsx b/src/components/InpaintEditor.tsx
index b37afeb..7d5bfc5 100644
--- a/src/components/InpaintEditor.tsx
+++ b/src/components/InpaintEditor.tsx
@@ -3,6 +3,7 @@ import { useCallback, useEffect, useLayoutEffect, useRef, useState } from "react
import { api, urls } from "../lib/api";
import { focusPrompt, getState, setState, toastError, updateComposer, useStore } from "../lib/store";
import { IconButton } from "./ui";
+import { shortcut } from "../lib/platform";
type Op =
| { kind: "stroke"; erase: boolean; size: number; points: [number, number][] }
@@ -253,10 +254,10 @@ export function InpaintEditor() {
</label>
</div>
<div className="group">
- <IconButton title="Undo (Ctrl+Z)" onClick={undo} disabled={!ops.length}>
+ <IconButton title={`Undo (${shortcut("Z")})`} onClick={undo} disabled={!ops.length}>
<Undo2 size={17} />
</IconButton>
- <IconButton title="Redo (Ctrl+Y)" onClick={redoOp} disabled={!redo.length}>
+ <IconButton title={`Redo (${shortcut("Y")})`} onClick={redoOp} disabled={!redo.length}>
<Redo2 size={17} />
</IconButton>
<IconButton title="Invert mask" onClick={() => push({ kind: "invert" })}>
diff --git a/src/components/Settings.tsx b/src/components/Settings.tsx
index dea89cb..9e7c144 100644
--- a/src/components/Settings.tsx
+++ b/src/components/Settings.tsx
@@ -226,6 +226,11 @@ export function Settings() {
{tab === "performance" && (
<>
+ <div className="field">
+ <span>Engine</span>
+ <span>{{ cuda: "CUDA (NVIDIA)", vulkan: "Vulkan", metal: "Metal (Apple Silicon, beta)" }[cfg.gpuBackend]}</span>
+ <span className="muted">To switch engine, use Models → Set up models…: it downloads the other engine if needed. Both can stay installed.</span>
+ </div>
<label className="field-row">
<span>GPU device</span>
<select className="select" value={cfg.device} onChange={(e) => set({ device: e.target.value })}>
@@ -246,11 +251,12 @@ export function Settings() {
{ value: "gpu", label: "GPU" },
{ value: "cpu", label: "CPU" },
{ value: "auto", label: "Auto" },
+ { value: "disk", label: "Disk" },
]}
onChange={(v) => set({ textEncoderPlacement: v })}
/>
<span className="muted">
- RAM (recommended): weights in system memory, computed on the GPU. CPU uses the least VRAM but makes image editing several times slower.
+ RAM (recommended): weights in system memory, computed on the GPU. CPU uses the least VRAM but makes image editing several times slower. Disk reads all weights from disk when needed: least memory, slower (for 16 GB Macs). On a Mac, RAM and GPU are the same memory.
</span>
</div>
<label className="field-row">
@@ -263,7 +269,7 @@ export function Settings() {
</div>
<div className="field-row">
<span>SageAttention (faster sampling)</span>
- <Switch on={cfg.sageAttention} onChange={(v) => set({ sageAttention: v })} />
+ <Switch on={cfg.sageAttention && cfg.gpuBackend === "cuda"} onChange={(v) => set({ sageAttention: v })} disabled={cfg.gpuBackend !== "cuda"} />
</div>
<label className="field-row">
<span>VAE tile size (px, 0 = off)</span>
diff --git a/src/components/Setup.tsx b/src/components/Setup.tsx
index c9fefda..db0cbb0 100644
--- a/src/components/Setup.tsx
+++ b/src/components/Setup.tsx
@@ -4,11 +4,25 @@ import { api } from "../lib/api";
import { formatBytes, installItemShort, installProgress, installStep, installTimeLeft } from "../lib/logic";
import { native } from "../lib/native";
import { refreshStatus, setState, toastError, useStore } from "../lib/store";
-import type { HardwareReport, InstallItem, InstallStatus, SetupReport } from "../lib/types";
+import type { GpuBackend, HardwareReport, InstallItem, InstallStatus, SetupReport } from "../lib/types";
import { Seg, Switch } from "./ui";
const gb = (mib: number) => `${Math.round(mib / 1024)} GB`;
+function engineOptions(vendor: string | null): { value: GpuBackend; label: string }[] {
+ if (vendor === "apple") return [{ value: "metal", label: "Metal (beta)" }];
+ const vulkan = { value: "vulkan" as GpuBackend, label: vendor === "nvidia" ? "Vulkan" : "Vulkan (beta)" };
+ return vendor === "nvidia" ? [{ value: "cuda", label: "CUDA" }, vulkan] : [vulkan];
+}
+
+function engineNote(b: GpuBackend, nvidia: boolean): string {
+ if (b === "cuda") return "NVIDIA's CUDA build: the fastest and tested path. Recommended.";
+ if (b === "metal") return "Beta. Runs on the Mac's GPU through Metal. Needs macOS 26 or newer.";
+ return nvidia
+ ? "The same model through Vulkan: works on NVIDIA, but slower than CUDA (no SageAttention) and the first image takes longer. Useful to try the AMD / Intel path; both engines can stay installed."
+ : "Beta. AMD and Intel cards run the model through Vulkan, with the graphics driver you already have. Not yet tested on a real AMD or Intel card: please report how it goes.";
+}
+
function ItemIcon({ it }: { it: InstallItem }) {
switch (it.state) {
case "present":
@@ -72,17 +86,19 @@ export function Setup() {
const [report, setReport] = useState<SetupReport | null>(null);
const [inst, setInst] = useState<InstallStatus | null>(null);
const [preset, setPreset] = useState<string | null>(null);
+ const [backend, setBackend] = useState<GpuBackend | null>(null);
const [upscaler, setUpscaler] = useState(true);
const [showFiles, setShowFiles] = useState(false);
const [checking, setChecking] = useState(false);
const wasRunning = useRef(false);
const setupDone = useStore((s) => s.status?.setup_complete ?? false);
- const refresh = async () => {
+ const refresh = async (engine: GpuBackend | null = backend) => {
setChecking(true);
try {
- const [h, r, i] = await Promise.all([api.hardware(), api.setup(), api.installStatus()]);
+ const [h, r, i] = await Promise.all([api.hardware(engine ?? undefined), api.setup(), api.installStatus()]);
setHw(h);
+ setBackend(h.backend);
setReport(r);
setInst(i);
setPreset((p) => p ?? (i.running && i.preset ? i.preset : h.recommended));
@@ -125,7 +141,7 @@ export function Setup() {
const install = async () => {
if (!preset) return;
try {
- setInst(await api.install(preset, upscaler));
+ setInst(await api.install(preset, upscaler, backend ?? hw?.backend ?? "cuda"));
} catch (e) {
toastError(e);
}
@@ -179,19 +195,30 @@ export function Setup() {
) : (
<>
<div className="setup-card setup-gpu">
- <strong>{hw.gpu ? hw.gpu.name : "No NVIDIA GPU found"}</strong>
+ <strong>{hw.gpu ? hw.gpu.name : "No graphics card found"}</strong>
<span className="muted">
- {vram ? `${gb(vram)} VRAM${hw.vram_simulated ? " (simulated)" : ""} · ` : ""}
- {gb(hw.ram_total_mib)} RAM{hw.gpu ? ` · driver ${hw.gpu.driver_version}` : ""}
+ {hw.gpu?.unified
+ ? `${gb(hw.gpu.memory_total_mib)} unified memory${hw.vram_simulated ? ` (planning as ${gb(vram ?? 0)})` : ""}`
+ : `${vram ? `${gb(vram)} VRAM${hw.vram_simulated ? " (simulated)" : ""} · ` : ""}${gb(hw.ram_total_mib)} RAM`}{hw.gpu?.driver_version ? ` · driver ${hw.gpu.driver_version}` : ""}
</span>
</div>
- {!hw.nvidia && (
+ {!hw.gpu && (
<div className="setup-note">
- This version needs an NVIDIA GPU (RTX 20 series or newer) with at least 12 GB of VRAM and a recent driver. Support for AMD and Intel cards (Vulkan) is planned.
+ Kokon needs a graphics card with at least 12 GB of VRAM: NVIDIA (RTX 20 series or newer), or AMD / Intel through Vulkan (beta), or a Mac with Apple Silicon. Make sure its driver is installed.
</div>
)}
- {vram !== null && vram < 11 * 1024 && <div className="setup-note">Your GPU has {gb(vram)} of VRAM. 12 GB is the smallest size this app is made for; it may fail or run slowly.</div>}
- {hw.ram_total_mib < 15 * 1024 && <div className="setup-note">The text encoder runs from system RAM; with {gb(hw.ram_total_mib)} of RAM, 16 GB or more is recommended.</div>}
+ {hw.gpu?.unified ? (
+ hw.gpu.memory_total_mib < 23 * 1024 && (
+ <div className="setup-note">
+ With {gb(hw.gpu.memory_total_mib)} of unified memory Kokon runs in its lightest mode: model weights are read from disk as needed and images are made at about 1 megapixel. Expect several minutes per image; 24 GB or more is recommended. Close other apps while generating.
+ </div>
+ )
+ ) : (
+ <>
+ {vram !== null && vram < 11 * 1024 && <div className="setup-note">Your GPU has {gb(vram)} of VRAM. 12 GB is the smallest size this app is made for; it may fail or run slowly.</div>}
+ {hw.ram_total_mib < 15 * 1024 && <div className="setup-note">The text encoder runs from system RAM; with {gb(hw.ram_total_mib)} of RAM, 16 GB or more is recommended.</div>}
+ </>
+ )}
{running || paused ? (
<div className="muted">
@@ -199,6 +226,20 @@ export function Setup() {
</div>
) : (
<>
+ {hw.gpu && (
+ <div className="field">
+ <span>Engine</span>
+ <Seg<GpuBackend>
+ value={backend ?? hw.backend}
+ options={engineOptions(hw.vendor)}
+ onChange={(b) => {
+ setBackend(b);
+ refresh(b);
+ }}
+ />
+ <span className="muted">{engineNote(backend ?? hw.backend, hw.nvidia)}</span>
+ </div>
+ )}
<div className="field">
<span>Preset for your graphics card</span>
<Seg<string> value={preset ?? hw.recommended} options={hw.presets.map((p) => ({ value: p.id, label: p.label }))} onChange={setPreset} />
@@ -297,10 +338,10 @@ export function Setup() {
<Download size={16} /> {failed.length ? "Try again" : `Install · ${formatBytes(toDownload)}`}
</button>
)}
- {hw && presetInstalled && !running && chosen && hw.current !== chosen.id && (
- // Files of another preset are already here: switch to it without downloading.
+ {hw && presetInstalled && !running && chosen && (hw.current !== chosen.id || hw.current_backend !== hw.backend) && (
+ // Files of another preset or engine are already here: switch without downloading.
<button className="btn" onClick={install}>
- Use the {chosen.label} preset
+ {hw.current_backend !== hw.backend ? `Use ${{ cuda: "CUDA", vulkan: "Vulkan", metal: "Metal" }[hw.backend]} with the ${chosen.label} preset` : `Use the ${chosen.label} preset`}
</button>
)}
{(ready || presetInstalled) && !running && (
@@ -312,7 +353,7 @@ export function Setup() {
<button className="btn ghost" onClick={() => setState({ overlay: "settings" })} disabled={running}>
<Settings2 size={14} /> Settings
</button>
- <button className="btn ghost" onClick={refresh} disabled={checking || running}>
+ <button className="btn ghost" onClick={() => refresh()} disabled={checking || running}>
<RefreshCw size={14} /> Check again
</button>
</div>
diff --git a/src/components/Sidebar.tsx b/src/components/Sidebar.tsx
index 820cbf8..4f8b020 100644
--- a/src/components/Sidebar.tsx
+++ b/src/components/Sidebar.tsx
@@ -55,7 +55,7 @@ Double-click or F2 to rename`}
<span className="proj-title">{p.title}</span>
)}
<span className="proj-sub">
- {p.count} {p.count === 1 ? "version" : "versions"} · {timeAgo(p.updatedAt)}
+ {p.count} {p.count === 1 ? "image" : "images"} · {timeAgo(p.updatedAt)}
</span>
</span>
<button
diff --git a/src/components/SizeMenus.tsx b/src/components/SizeMenus.tsx
new file mode 100644
index 0000000..b2475a1
--- /dev/null
+++ b/src/components/SizeMenus.tsx
@@ -0,0 +1,133 @@
+import { ArrowLeftRight, Check } from "lucide-react";
+import { useState } from "react";
+import { ASPECTS, aspectLabel, parseRatio, planPipeline, sizeForRatio } from "../lib/logic";
+import { getState, updateComposer, useStore } from "../lib/store";
+import type { Plan, ResolutionMode } from "../lib/types";
+import { Option } from "./ui";
+
+/** A small rectangle with the ratio's shape. */
+export function RatioIcon({ ratio, size = 16 }: { ratio: number; size?: number }) {
+ const w = ratio >= 1 ? size : Math.max(4, size * ratio);
+ const h = ratio >= 1 ? Math.max(4, size / ratio) : size;
+ return (
+ <span className="ratio-icon" style={{ width: size, height: size }} aria-hidden>
+ <i style={{ width: w, height: h }} />
+ </span>
+ );
+}
+
+/** Two numbers typed by hand (a ratio such as 7:5, or a size such as 1920×1080). */
+function PairFields(props: { a: number | ""; b: number | ""; sep: string; max: number; onA: (v: number | "") => void; onB: (v: number | "") => void; label: string; onEnter?: () => void }) {
+ const num = (v: string): number | "" => (v === "" ? "" : Math.max(0, Math.min(props.max, Number(v) || 0)));
+ const key = (e: React.KeyboardEvent) => e.key === "Enter" && props.onEnter?.();
+ return (
+ <span className="pair" aria-label={props.label}>
+ <input className="input" type="number" min={1} max={props.max} value={props.a} onChange={(e) => props.onA(num(e.target.value))} onKeyDown={key} />
+ <span className="muted">{props.sep}</span>
+ <input className="input" type="number" min={1} max={props.max} value={props.b} onChange={(e) => props.onB(num(e.target.value))} onKeyDown={key} />
+ </span>
+ );
+}
+
+/** Grid of common ratios plus a ratio of your own. `value` is the current ratio ("16:9",
+ * "7:5"), "auto" or null. */
+export function AspectGrid(props: { value: string | null; onPick: (ratio: string) => void; auto?: { sub: string } }) {
+ const custom = props.value && props.value !== "auto" && !(ASPECTS as readonly string[]).includes(props.value) ? props.value.split(/[:x×/]/) : null;
+ const [a, setA] = useState<number | "">(custom ? Number(custom[0]) : "");
+ const [b, setB] = useState<number | "">(custom ? Number(custom[1]) : "");
+ const typed = a && b ? `${a}:${b}` : "";
+ const ok = !!typed && parseRatio(typed) !== null;
+ return (
+ <div className="aspect-menu">
+ {props.auto && (
+ <button className={`aspect-auto ${props.value === "auto" ? "on" : ""}`} onClick={() => props.onPick("auto")}>
+ <span>Auto</span>
+ <span className="sub">{props.auto.sub}</span>
+ {props.value === "auto" && <Check size={14} />}
+ </button>
+ )}
+ <div className="aspect-grid">
+ {ASPECTS.map((r) => (
+ <button key={r} className={props.value === r ? "on" : ""} onClick={() => props.onPick(r)} title={r}>
+ <RatioIcon ratio={parseRatio(r)!} />
+ {r}
+ </button>
+ ))}
+ </div>
+ <div className="aspect-custom">
+ <span className="muted">Other</span>
+ <PairFields a={a} b={b} sep=":" max={100} onA={setA} onB={setB} label="Custom ratio" onEnter={() => ok && props.onPick(typed)} />
+ <button className="btn" disabled={!ok} onClick={() => props.onPick(typed)} title={typed && !ok ? "Use a ratio between 1:8 and 8:1" : undefined}>
+ Use
+ </button>
+ </div>
+ </div>
+ );
+}
+
+const RES_OPTIONS: { value: ResolutionMode; label: string; sub: string }[] = [
+ { value: "auto", label: "Auto", sub: "Set by the quality preset" },
+ { value: "1k", label: "1K", sub: "~1 MP" },
+ { value: "1.5k", label: "1.5K", sub: "~2.4 MP" },
+ { value: "2k", label: "2K", sub: "~4 MP · the model's native size" },
+ { value: "source", label: "Source", sub: "Same size as image 1" },
+];
+
+export const resolutionLabel = (r: ResolutionMode, w: number, h: number) => (r === "custom" ? `${w}×${h}` : RES_OPTIONS.find((o) => o.value === r)?.label ?? "Auto");
+
+/** Output size: a preset level or an exact width × height (whose ratio is then shown). */
+export function ResolutionMenu({ plan, close }: { plan: Plan | null; close: () => void }) {
+ const c = useStore((s) => s.composer);
+ const editing = c.refs.length > 0;
+ const [w, setW] = useState<number | "">(c.customWidth);
+ const [h, setH] = useState<number | "">(c.customHeight);
+ const valid = typeof w === "number" && typeof h === "number" && w >= 64 && h >= 64;
+ const apply = (nw: number | "", nh: number | "") => {
+ setW(nw);
+ setH(nh);
+ if (typeof nw === "number" && typeof nh === "number" && nw >= 64 && nh >= 64) updateComposer({ resolution: "custom", customWidth: nw, customHeight: nh });
+ };
+ return (
+ <div className="res-menu">
+ {RES_OPTIONS.filter((o) => o.value !== "source" || editing).map((o) => (
+ <Option key={o.value} selected={c.resolution === o.value} label={o.label} sub={o.sub} onClick={() => (updateComposer({ resolution: o.value }), close())} />
+ ))}
+ <div className={`res-custom ${c.resolution === "custom" ? "on" : ""}`}>
+ <span className="res-custom-head">
+ <span className="check">{c.resolution === "custom" && <Check size={15} />}</span>
+ Exact size
+ </span>
+ <span className="res-custom-row">
+ <PairFields a={w} b={h} sep="×" max={4096} onA={(v) => apply(v, h)} onB={(v) => apply(w, v)} label="Width and height" onEnter={close} />
+ <button className="icon-btn sm" title="Swap width and height" onClick={() => apply(h, w)}>
+ <ArrowLeftRight size={14} />
+ </button>
+ {valid && (
+ <span className="ratio-tag">
+ <RatioIcon ratio={(w as number) / (h as number)} size={14} />
+ {aspectLabel(w as number, h as number)}
+ </span>
+ )}
+ </span>
+ </div>
+ {plan && (
+ <div className="res-foot muted">
+ Output {plan.output.width}×{plan.output.height} ({aspectLabel(plan.output.width, plan.output.height)}) · {planPipeline(plan)}
+ </div>
+ )}
+ </div>
+ );
+}
+
+/** Picking a ratio while an exact size is set keeps about the same number of pixels; Auto
+ * goes back to automatic size too. */
+export function pickAspect(ratio: string) {
+ const c = getState().composer;
+ const r = parseRatio(ratio);
+ if (c.resolution !== "custom") updateComposer({ aspect: ratio });
+ else if (!r) updateComposer({ aspect: "auto", resolution: "auto" });
+ else {
+ const s = sizeForRatio(r, c.customWidth * c.customHeight);
+ updateComposer({ aspect: ratio, customWidth: s.width, customHeight: s.height });
+ }
+}
diff --git a/src/components/Stage.tsx b/src/components/Stage.tsx
index a8ea2a0..2df71c5 100644
--- a/src/components/Stage.tsx
+++ b/src/components/Stage.tsx
@@ -1,20 +1,21 @@
import {
- Brush, ClipboardPaste, Combine, Copy, Download, Ellipsis, Expand, FolderOpen, Heart, Info, Loader2, MousePointerClick, Pencil, RotateCcw, Shuffle, Square, TextCursorInput, Trash2, Wand2, X, ZoomIn,
+ Brush, ClipboardPaste, Combine, Copy, Download, Ellipsis, Expand, FolderOpen, Heart, ImageUpscale, Info, Loader2, MousePointerClick, Pencil, RefreshCw, RotateCcw, SlidersHorizontal, Square, TextCursorInput, Trash2, Wand2, X,
} from "lucide-react";
-import { Fragment, useEffect, useMemo, useRef, useState } from "react";
+import { Fragment, useEffect, useLayoutEffect, useMemo, useRef, useState } from "react";
import { api, urls } from "../lib/api";
import {
- compareBase, composerFromRecord, formatDuration, jobFraction, jobLabel, MAX_UPSCALE_AREA, MAX_UPSCALE_SIDE, MODE_LABEL, pipelineSummary, remainingSeconds, sizeForWidth, upscaleRequest, upscalerLabel,
- upscaleTargets, variationRequest, versionLabels,
+ compareBase, composerFromRecord, formatDuration, jobFraction, jobLabel, MAX_UPSCALE_AREA, MAX_UPSCALE_SIDE, MODE_LABEL, pipelineSummary, remainingSeconds, sizeForWidth, upscaleRequest, arcLayout,
+ upscalerLabel, upscaleTargets, variationRequest, imageName, type LinkKind, type TimelineLink,
} from "../lib/logic";
import { native } from "../lib/native";
import {
- cancelJob, focusPrompt, getState, itemSource, removeSessionItem, selectedItem, selectItem, setState, submitRequest, toastError, toastInfo, useStore, type SessionItem,
+ addAsReference, cancelJob, focusPrompt, getState, itemSource, removeSessionItem, reusePrompt, selectedItem, selectItem, sessionLabels, sessionLinks, setState, submitRequest, toastError, toastInfo, useStore, type SessionItem,
} from "../lib/store";
import type { GenerationRecord, JobView, Mode, Size, UpscaleMethod } from "../lib/types";
-import { CompareBar, CompareFrame } from "./Compare";
+import { CompareFrame } from "./Compare";
import { IconButton, Popover, Seg, useFit } from "./ui";
import { FullscreenButton, ZoomImage } from "./Viewer";
+import { fileManager, shortcut } from "../lib/platform";
export const MODE_ICON: Record<Mode, React.ReactNode> = {
auto: <Wand2 size={11} />,
@@ -23,7 +24,7 @@ export const MODE_ICON: Record<Mode, React.ReactNode> = {
multi_reference: <Combine size={11} />,
inpaint: <Brush size={11} />,
outpaint: <Expand size={11} />,
- upscale: <ZoomIn size={11} />,
+ upscale: <ImageUpscale size={11} />,
};
const MODE_DOING: Record<Mode, string> = {
@@ -46,7 +47,7 @@ function Empty() {
<MousePointerClick size={14} /> Drop images here
</span>
<span>
- <ClipboardPaste size={14} /> Paste with Ctrl+V
+ <ClipboardPaste size={14} /> Paste with {shortcut("V")}
</span>
</div>
</div>
@@ -65,7 +66,7 @@ function Pending({ job, item, labels }: { job: JobView; item: SessionItem; label
const from = src ? session.find((i) => itemSource(i) === src) : undefined;
const mode = job.plan?.mode;
// "v3 · Editing v2", "v1 · Creating"
- const title = [labels[item.key], mode && `${MODE_DOING[mode]}${from && mode !== "multi_reference" ? ` ${labels[from.key]}` : ""}`].filter(Boolean).join(" · ");
+ const title = [imageName(labels[item.key]), mode && `${MODE_DOING[mode]}${from && mode !== "multi_reference" ? ` ${imageName(labels[from.key]).toLowerCase()}` : ""}`].filter(Boolean).join(" · ");
// The source keeps its own shape inside an expansion; elsewhere it fills the frame.
const outpaint = item.request?.outpaint;
return (
@@ -211,15 +212,21 @@ function UpscaleMenu(props: { source: string; size: Size; original?: Size | null
);
}
+/** A main action under the image: icon and word (the word hides when the stage is narrow). */
+function ActionButton(props: { title: string; icon: React.ReactNode; label: string; onClick: () => void; active?: boolean }) {
+ return (
+ <button className={`act ${props.active ? "active" : ""}`} title={props.title} aria-label={props.label} onClick={props.onClick}>
+ {props.icon}
+ <span className="act-label">{props.label}</span>
+ </button>
+ );
+}
+
function UpscaleButton(props: { source: string; size: Size; original?: Size | null; parentId?: string | null }) {
return (
<Popover
placement="top-end"
- trigger={(open, toggle) => (
- <IconButton title="Upscale" onClick={toggle} active={open}>
- <ZoomIn size={17} />
- </IconButton>
- )}
+ trigger={(open, toggle) => <ActionButton title="Upscale: make a larger copy (2×, 4K or a custom size)" icon={<ImageUpscale size={16} />} label="Upscale" onClick={toggle} active={open} />}
>
{(close) => <UpscaleMenu {...props} close={close} />}
</Popover>
@@ -250,7 +257,7 @@ function MoreMenu({ item, path }: { item: SessionItem; path: string | null }) {
}}
>
<span className="check">
- <TextCursorInput size={15} />
+ <SlidersHorizontal size={15} />
</span>
<span>Reuse prompt and settings</span>
</button>
@@ -260,7 +267,7 @@ function MoreMenu({ item, path }: { item: SessionItem; path: string | null }) {
<span className="check">
<FolderOpen size={15} />
</span>
- <span>Show in Explorer</span>
+ <span>Show in {fileManager}</span>
</button>
)}
<button className="opt" onClick={() => (close(), removeSessionItem(item.key))}>
@@ -280,15 +287,25 @@ function GenActions({ item, g }: { item: SessionItem; g: GenerationRecord }) {
const [fav, setFav] = useState(g.favorite);
return (
<div className="actions">
- <IconButton
- title="Variation (same prompt, new seed)"
- onClick={() => {
- const r = variationRequest(g);
- if (r) submitRequest(r);
- }}
- >
- <Shuffle size={17} />
- </IconButton>
+ {g.prompt && (
+ <ActionButton
+ title="Reuse prompt: put this prompt back in the box to change it a little. It works on the same image this one was made from; your current settings stay as they are."
+ icon={<TextCursorInput size={16} />}
+ label="Reuse prompt"
+ onClick={() => reusePrompt(g)}
+ />
+ )}
+ {g.request && g.mode !== "upscale" && (
+ <ActionButton
+ title="Regenerate: same prompt and settings, new seed"
+ icon={<RefreshCw size={16} />}
+ label="Regenerate"
+ onClick={() => {
+ const r = variationRequest(g);
+ if (r) submitRequest(r);
+ }}
+ />
+ )}
<UpscaleButton source={`gen:${g.id}`} size={{ width: g.width, height: g.height }} original={original} parentId={g.id} />
<span className="sep" />
<IconButton
@@ -375,17 +392,16 @@ function PlainFrame({ src, alt, w, h }: { src: string; alt: string; w: number; h
);
}
-/** One line under the image: which version this is, what made it, and what you can do with it. */
+/** Under the image: which image this is and what made it, then what you can do with it. */
function StageBar({ item, label }: { item: SessionItem; label: string }) {
const g = item.gen;
- const base = g ? compareBase(g) : null;
const w = g?.width ?? item.input!.width;
const h = g?.height ?? item.input!.height;
const name = item.input?.original_name ? safeDecode(item.input.original_name) : "Imported image";
return (
<div className="stage-bar">
<div className="stage-meta">
- <span className="ver">{label}</span>
+ <span className="ver">{imageName(label)}</span>
{g && <span className="mode">{MODE_LABEL[g.mode]}</span>}
<span className="stage-prompt" title={g?.prompt ?? name}>
{g ? g.prompt || (g.mode === "upscale" ? `${g.references[0]?.width ?? "?"}×${g.references[0]?.height ?? "?"} → ${w}×${h}` : "") : name}
@@ -394,7 +410,6 @@ function StageBar({ item, label }: { item: SessionItem; label: string }) {
{w}×{h}
</span>
</div>
- {base && <CompareBar />}
{g ? <GenActions key={`actions-${g.id}`} item={item} g={g} /> : <InputActions item={item} />}
</div>
);
@@ -408,63 +423,180 @@ function safeDecode(s: string) {
}
}
-/** The project's versions in order: Original → v1 → v2 … The image the prompt edits carries a pencil. */
+/** Arcs above the thumbnails join a version to the earlier ones it was made from: one level
+ * per arc that spans another, `LANE` px apart, rounded corners of `R` px. */
+const LANE = 9;
+const R = 5;
+
+/** What joins a version to the one before it: a line when it continues from it (dashed for a
+ * new seed of it), a break when it starts something new, a plain gap when it comes from
+ * other versions (an arc above shows which). */
+function Joint({ kind, hot }: { kind: LinkKind; hot: boolean }) {
+ if (kind === "start") return <span className="joint break" aria-hidden />;
+ if (kind === "branch" || kind === "combine") return <span className="joint gap" aria-hidden />;
+ return <span className={`joint link ${kind === "variant" ? "dashed" : ""} ${hot ? "hot" : ""}`} aria-hidden />;
+}
+
+function linkText(link: TimelineLink | undefined, labels: Record<string, string>): string {
+ if (!link || link.kind === "start") return "";
+ const names = link.parents.map((k) => imageName(labels[k]).toLowerCase()).join(" + ");
+ return link.kind === "variant" ? `new seed of ${names}` : `from ${names}`;
+}
+
+/** The project's versions in order. A version continues the line of the one it was made from;
+ * a new image starts a new line; versions made from earlier ones (or from several) are joined
+ * to them by arcs. The image the prompt edits carries a pencil. */
function Timeline({ labels }: { labels: Record<string, string> }) {
const session = useStore((s) => s.session);
const selected = useStore((s) => s.selected);
const jobs = useStore((s) => s.jobs);
const target = useStore((s) => s.composer.refs[0]?.source);
const boxRef = useRef<HTMLDivElement>(null);
+ const trackRef = useRef<HTMLDivElement>(null);
+ const [centers, setCenters] = useState<Record<string, number>>({});
+ const [hovered, setHovered] = useState<string | null>(null);
+ const links = useMemo(() => sessionLinks(session), [session]);
+ // Dragging a thumbnail onto the prompt box adds it to the prompt.
+ const drag = useRef<{ key: string; src: string; x0: number; y0: number; moved: boolean } | null>(null);
+ const dragged = useRef(false);
+ const [ghost, setGhost] = useState<{ src: string; x: number; y: number; over: boolean } | null>(null);
+ const overPrompt = (x: number, y: number) => !!document.elementFromPoint(x, y)?.closest(".composer");
+ const hintDrop = (on: boolean) => document.querySelector(".composer")?.classList.toggle("drag", on);
+ const endDrag = () => {
+ drag.current = null;
+ setGhost(null);
+ hintDrop(false);
+ };
useEffect(() => {
boxRef.current?.querySelector(".frame.sel")?.scrollIntoView({ block: "nearest", inline: "nearest" });
}, [selected, session.length]);
+ // Thumbnail centres, for the arcs.
+ useLayoutEffect(() => {
+ const out: Record<string, number> = {};
+ trackRef.current?.querySelectorAll<HTMLElement>(".frame").forEach((f) => {
+ if (f.dataset.key) out[f.dataset.key] = f.offsetLeft + f.offsetWidth / 2;
+ });
+ setCenters(out);
+ }, [session, labels]);
+
if (!session.length) return null;
+ const focus = hovered ?? selected;
+ const selParents = new Set(focus ? links[focus]?.parents : []);
+ const index = new Map(session.map((it, i) => [it.key, i]));
+ const arcs = session
+ .flatMap((it) => {
+ const l = links[it.key];
+ if (!l || (l.kind !== "branch" && l.kind !== "combine")) return [];
+ return l.parents.map((p) => ({ from: p, to: it.key, hot: it.key === focus || p === focus }));
+ })
+ .filter((a) => centers[a.from] !== undefined && centers[a.to] !== undefined);
+ const places = arcLayout(arcs.map((a) => ({ a: index.get(a.from)!, b: index.get(a.to)! })));
+ const band = arcs.length ? 10 + (Math.max(...places.map((p) => p.lane)) + 1) * LANE : 0;
+ const arcPath = (x1: number, x2: number, top: number) => {
+ const y = band - 1;
+ const d = x2 > x1 ? 1 : -1;
+ // Up from the source, across at its level, down into the result, with an arrowhead.
+ return `M ${x1} ${y} L ${x1} ${top + R} Q ${x1} ${top} ${x1 + d * R} ${top} L ${x2 - d * R} ${top} Q ${x2} ${top} ${x2} ${top + R} L ${x2} ${y} M ${x2 - 3.5} ${y - 4.5} L ${x2} ${y} L ${x2 + 3.5} ${y - 4.5}`;
+ };
+
return (
<div className="timeline" ref={boxRef} aria-label="Versions">
- {session.map((it, i) => {
- const job = it.jobId ? jobs[it.jobId] : undefined;
- const src = itemSource(it);
- const pending = it.kind === "job";
- const failed = pending && job && ["failed", "cancelled"].includes(job.status);
- const frac = pending && job ? jobFraction(job) : null;
- const mode: Mode | undefined = it.gen?.mode ?? job?.plan?.mode;
- const from = it.gen?.parent_id ? session.find((x) => x.gen?.id === it.gen!.parent_id) : undefined;
- const pendingSrc = pending && it.request?.mode !== "generate" ? it.request?.images?.[0] : undefined;
- const prompt = it.gen?.prompt || it.request?.prompt || "";
- const tip = [labels[it.key], mode ? MODE_LABEL[mode] : it.input ? "Imported" : "", from && session[i - 1] !== from ? `from ${labels[from.key]}` : "", prompt ? `— ${prompt}` : ""].filter(Boolean).join(" ");
- return (
- <Fragment key={it.key}>
- {i > 0 && <span className="link" />}
- <button className={`frame ${selected === it.key ? "sel" : ""}`} onClick={() => selectItem(it.key)} title={tip}>
- <span className="frame-img">
- {src && <img src={urls.sourceThumb(src)} alt="" draggable={false} />}
- {pendingSrc && <img className="dim" src={urls.sourceThumb(pendingSrc)} alt="" draggable={false} />}
- {pending && !failed && (
- <span className="frame-state">
- {frac !== null ? <Ring frac={frac} /> : <Loader2 className="spin" size={16} />}
- </span>
- )}
- {failed && (
- <span className="frame-state">
- <X size={16} />
- </span>
- )}
- {src && src === target && (
- <span className="frame-badge" title="The prompt edits this image">
- <Pencil size={10} />
- </span>
- )}
- </span>
- <span className="frame-label">
- {mode && MODE_ICON[mode]}
- {labels[it.key]}
- </span>
- </button>
- </Fragment>
- );
- })}
+ {/* Room for the selection outline even when there are no arcs (the scroll box clips). */}
+ <div className="track" ref={trackRef} style={{ paddingTop: Math.max(band, 6) }} onMouseLeave={() => setHovered(null)}>
+ {arcs.length > 0 && (
+ <svg className="arcs" height={band} aria-hidden>
+ {arcs
+ .map((a, i) => ({ a, p: places[i] }))
+ .sort((x, y) => Number(x.a.hot) - Number(y.a.hot))
+ .map(({ a, p }) => (
+ <path key={`${a.from}-${a.to}`} className={a.hot ? "hot" : ""} d={arcPath(centers[a.from] + p.dx1, centers[a.to] + p.dx2, band - 8 - (p.lane + 1) * LANE)} />
+ ))}
+ </svg>
+ )}
+ {session.map((it, i) => {
+ const job = it.jobId ? jobs[it.jobId] : undefined;
+ const src = itemSource(it);
+ const pending = it.kind === "job";
+ const failed = pending && job && ["failed", "cancelled"].includes(job.status);
+ const frac = pending && job ? jobFraction(job) : null;
+ const mode: Mode | undefined = it.gen?.mode ?? job?.plan?.mode;
+ const link = links[it.key];
+ const pendingSrc = pending && it.request?.mode !== "generate" ? it.request?.images?.[0] : undefined;
+ const prompt = it.gen?.prompt || it.request?.prompt || "";
+ const what = mode ? (mode === "generate" && link?.kind === "start" ? "New image" : MODE_LABEL[mode]) : it.input ? "Imported" : "";
+ const tip = [[imageName(labels[it.key]), what, linkText(link, labels)].filter(Boolean).join(" · "), prompt, src ? "Drag onto the prompt box to use it there" : ""].filter(Boolean).join("\n");
+ const hot = it.key === focus || link?.parents[0] === focus;
+ return (
+ <Fragment key={it.key}>
+ {i > 0 && <Joint kind={link?.kind ?? "next"} hot={hot} />}
+ <button
+ data-key={it.key}
+ className={`frame ${selected === it.key ? "sel" : ""} ${selParents.has(it.key) ? "src" : ""}`}
+ title={tip}
+ onMouseEnter={() => setHovered(it.key)}
+ onClick={() => {
+ if (dragged.current) dragged.current = false;
+ else selectItem(it.key);
+ }}
+ onPointerDown={(e) => {
+ if (e.button === 0 && src) drag.current = { key: it.key, src, x0: e.clientX, y0: e.clientY, moved: false };
+ }}
+ onPointerMove={(e) => {
+ const d = drag.current;
+ if (!d) return;
+ if (!d.moved) {
+ if (Math.hypot(e.clientX - d.x0, e.clientY - d.y0) < 6) return;
+ d.moved = true;
+ e.currentTarget.setPointerCapture(e.pointerId);
+ }
+ const over = overPrompt(e.clientX, e.clientY);
+ hintDrop(over);
+ setGhost({ src: d.src, x: e.clientX, y: e.clientY, over });
+ }}
+ onPointerUp={(e) => {
+ const d = drag.current;
+ endDrag();
+ if (!d?.moved) return;
+ dragged.current = true;
+ const dropped = session.find((x) => x.key === d.key);
+ if (dropped && overPrompt(e.clientX, e.clientY)) {
+ addAsReference(dropped);
+ focusPrompt();
+ }
+ }}
+ onPointerCancel={endDrag}
+ >
+ <span className="frame-img">
+ {src && <img src={urls.sourceThumb(src)} alt="" draggable={false} />}
+ {pendingSrc && <img className="dim" src={urls.sourceThumb(pendingSrc)} alt="" draggable={false} />}
+ {pending && !failed && (
+ <span className="frame-state">
+ {frac !== null ? <Ring frac={frac} /> : <Loader2 className="spin" size={16} />}
+ </span>
+ )}
+ {failed && (
+ <span className="frame-state">
+ <X size={16} />
+ </span>
+ )}
+ {src && src === target && (
+ <span className="frame-badge" title="The prompt edits this image">
+ <Pencil size={10} />
+ </span>
+ )}
+ </span>
+ <span className="frame-label">
+ {mode && MODE_ICON[mode]}
+ {labels[it.key]}
+ </span>
+ </button>
+ </Fragment>
+ );
+ })}
+ </div>
+ {ghost && <img className={`drag-ghost ${ghost.over ? "over" : ""}`} src={urls.sourceThumb(ghost.src)} style={{ left: ghost.x, top: ghost.y }} alt="" draggable={false} />}
</div>
);
}
@@ -484,7 +616,7 @@ export function Stage() {
const item = useStore(selectedItem);
const session = useStore((s) => s.session);
const job = useStore((s) => (item?.jobId ? s.jobs[item.jobId] : undefined));
- const labels = useMemo(() => versionLabels(session), [session]);
+ const labels = useMemo(() => sessionLabels(session), [session]);
if (!item) {
return (
diff --git a/src/components/ui.tsx b/src/components/ui.tsx
index 7af9b84..d2a57ee 100644
--- a/src/components/ui.tsx
+++ b/src/components/ui.tsx
@@ -82,8 +82,8 @@ export function Seg<T extends string>(props: { value: T; options: { value: T; la
);
}
-export function Switch(props: { on: boolean; onChange: (v: boolean) => void; label?: string }) {
- return <button className={`switch ${props.on ? "on" : ""}`} role="switch" aria-checked={props.on} aria-label={props.label} onClick={() => props.onChange(!props.on)} />;
+export function Switch(props: { on: boolean; onChange: (v: boolean) => void; label?: string; disabled?: boolean }) {
+ return <button className={`switch ${props.on ? "on" : ""}`} role="switch" aria-checked={props.on} aria-label={props.label} disabled={props.disabled} onClick={() => props.onChange(!props.on)} />;
}
/** Text that turns into a field while renaming: Enter or leaving saves, Escape cancels. */
diff --git a/src/lib/api.ts b/src/lib/api.ts
index 01b62c5..f9d66b1 100644
--- a/src/lib/api.ts
+++ b/src/lib/api.ts
@@ -2,6 +2,7 @@
// token as a query parameter) so the webview streams them directly — no base64 copies.
import type {
+ GpuBackend,
AppConfig, AppErrorBody, GenerationRecord, GenerationRequest, HardwareReport, InputRecord, InstallStatus, JobView, Plan, ServiceEvent, SetupReport, StatusResponse,
} from "./types";
@@ -52,8 +53,8 @@ const json = (body: unknown) => JSON.stringify(body);
export const api = {
status: () => call<StatusResponse>("/status"),
setup: () => call<SetupReport>("/setup"),
- hardware: () => call<HardwareReport>("/setup/hardware"),
- install: (preset: string, upscaler: boolean) => call<InstallStatus>("/setup/install", { method: "POST", body: json({ preset, upscaler }) }),
+ hardware: (backend?: GpuBackend) => call<HardwareReport>(backend ? `/setup/hardware?backend=${backend}` : "/setup/hardware"),
+ install: (preset: string, upscaler: boolean, backend: GpuBackend) => call<InstallStatus>("/setup/install", { method: "POST", body: json({ preset, upscaler, backend }) }),
installStatus: () => call<InstallStatus>("/setup/install"),
cancelInstall: () => call<InstallStatus>("/setup/install/cancel", { method: "POST" }),
config: () => call<AppConfig>("/config"),
diff --git a/src/lib/logic.test.ts b/src/lib/logic.test.ts
index 9fc5975..f0ceb5f 100644
--- a/src/lib/logic.test.ts
+++ b/src/lib/logic.test.ts
@@ -2,7 +2,7 @@ import { describe, expect, it } from "vitest";
import {
activeOverrides, buildRequest, compareBase, diffPixels, emptyComposer, expansionPixels, expansionPreset, jobFraction, jobLabel, moveRef, needsPrompt, persistentAdvanced, pipelineSummary,
planPipeline, remainingSeconds, sizeForWidth, timeAgo, upscaleRequest, upscaleTargets, versionLabels, type Ref,
- formatBytes, installItemShort, installItemText, installProgress, installStep, installTimeLeft, undoTarget,
+ formatBytes, installItemShort, installItemText, installProgress, installStep, installTimeLeft, timelineLinks, undoTarget, aspectLabel, parseRatio, sizeForRatio, expandToRatio, expandToSize, arcLayout,
} from "./logic";
import type { GenerationRecord, InstallItem, JobView, Plan } from "./types";
@@ -193,15 +193,21 @@ describe("before / after comparison", () => {
});
describe("project versions", () => {
- it("names imports Original and numbers results and pending jobs", () => {
+ it("numbers each new image, and the images made from it within it", () => {
+ // import, edit, edit; new image, its edit; new seed of the new image; edit of the first
+ // import again; combine led by the second image; a pending edit of 1.1.
const items = [
- { key: "a", kind: "input" as const },
- { key: "b", kind: "gen" as const },
- { key: "c", kind: "job" as const },
- { key: "d", kind: "input" as const },
- { key: "e", kind: "gen" as const },
+ { key: "a", source: "input:a", from: [] },
+ { key: "b", source: "gen:b", from: ["input:a"] },
+ { key: "c", source: "gen:c", from: ["gen:b"] },
+ { key: "d", source: "gen:d", from: [] },
+ { key: "e", source: "gen:e", from: ["gen:d"] },
+ { key: "f", source: "gen:f", from: ["gen:d"], variant: true },
+ { key: "g", source: "gen:g", from: ["input:a"] },
+ { key: "h", source: "gen:h", from: ["gen:d", "gen:c"] },
+ { key: "i", source: null, from: ["gen:b"] },
];
- expect(versionLabels(items)).toEqual({ a: "Original", b: "v1", c: "v2", d: "Original 2", e: "v3" });
+ expect(versionLabels(items)).toEqual({ a: "1", b: "1.1", c: "1.2", d: "2", e: "2.1", f: "3", g: "1.3", h: "2.2", i: "1.4" });
});
it("formats project age", () => {
@@ -264,3 +270,86 @@ describe("undo between versions", () => {
expect(undoTarget(items, "missing")).toBeNull();
});
});
+
+describe("timeline links", () => {
+ // Original -> v1 (edit) -> v2 (edit); v3 new image; v4 edits v3; v5 edits v1 again;
+ // v6 combines v2 and v4; v7 is a new seed of v3; v8 comes from a file outside the project.
+ const items = [
+ { key: "orig", source: "input:a", from: [] },
+ { key: "v1", source: "gen:1", from: ["input:a"] },
+ { key: "v2", source: "gen:2", from: ["gen:1"] },
+ { key: "v3", source: "gen:3", from: [] },
+ { key: "v4", source: "gen:4", from: ["gen:3"] },
+ { key: "v5", source: "gen:5", from: ["gen:1"] },
+ { key: "v6", source: null, from: ["gen:2", "gen:4"] },
+ { key: "v7", source: "gen:7", from: ["gen:3"], variant: true },
+ { key: "v8", source: "gen:8", from: ["C:\elsewhere.png"] },
+ ];
+ const l = timelineLinks(items);
+ it("continues a line, and breaks it for a new image", () => {
+ expect(l.orig).toEqual({ kind: "start", parents: [] });
+ expect(l.v1).toEqual({ kind: "next", parents: ["orig"] });
+ expect(l.v2.kind).toBe("next");
+ expect(l.v3.kind).toBe("start");
+ expect(l.v4).toEqual({ kind: "next", parents: ["v3"] });
+ });
+ it("joins versions made from earlier or several versions", () => {
+ expect(l.v5).toEqual({ kind: "branch", parents: ["v1"] });
+ expect(l.v6).toEqual({ kind: "combine", parents: ["v2", "v4"] });
+ expect(l.v7).toEqual({ kind: "branch", parents: ["v3"] });
+ expect(l.v8.kind).toBe("start");
+ });
+ it("marks a new seed of the version just before", () => {
+ const v = timelineLinks([
+ { key: "a", source: "gen:1", from: [] },
+ { key: "b", source: "gen:2", from: ["gen:1"], variant: true },
+ ]);
+ expect(v.b).toEqual({ kind: "variant", parents: ["a"] });
+ });
+});
+
+describe("aspect ratios", () => {
+ it("parses ratios within the service's 1:8 to 8:1 range", () => {
+ expect(parseRatio("16:9")).toBeCloseTo(16 / 9);
+ expect(parseRatio("2.35 : 1")).toBeCloseTo(2.35);
+ expect(parseRatio("1920x1080")).toBeCloseTo(16 / 9);
+ expect(parseRatio("10:1")).toBeNull();
+ expect(parseRatio("wide")).toBeNull();
+ });
+ it("names sizes by their ratio", () => {
+ expect(aspectLabel(1920, 1080)).toBe("16:9");
+ expect(aspectLabel(2752, 1536)).toBe("16:9");
+ expect(aspectLabel(1400, 1000)).toBe("7:5");
+ expect(aspectLabel(1850, 1000)).toBe("1.85:1");
+ expect(aspectLabel(1000, 1850)).toBe("1:1.85");
+ });
+ it("keeps the pixel count when the ratio changes", () => {
+ const s = sizeForRatio(16 / 9, 1024 * 1024);
+ expect(s.width % 16 + (s.height % 16)).toBe(0);
+ expect(Math.abs((s.width * s.height) / (1024 * 1024) - 1)).toBeLessThan(0.03);
+ expect(aspectLabel(s.width, s.height)).toBe("16:9");
+ });
+ it("expands to a ratio or an exact size without cropping", () => {
+ expect(expandToRatio(1000, 1000, 2)).toEqual({ left: 500, right: 500, top: 0, bottom: 0, units: "pixels" });
+ expect(expandToRatio(2000, 1000, 1)).toEqual({ left: 0, right: 0, top: 500, bottom: 500, units: "pixels" });
+ expect(expandToSize(1000, 800, 1501, 700)).toEqual({ left: 250, right: 251, top: 0, bottom: 0, units: "pixels" });
+ });
+});
+
+describe("timeline arcs", () => {
+ it("puts an arc above every shorter arc it overlaps", () => {
+ // 1->3 and 3->5 only touch at 3 (overlap), 0->6 spans both, 7->8 is apart.
+ const p = arcLayout([{ a: 1, b: 3 }, { a: 3, b: 5 }, { a: 0, b: 6 }, { a: 7, b: 8 }]);
+ expect(p[0].lane).toBe(0);
+ expect(p[1].lane).toBe(1);
+ expect(p[2].lane).toBe(2);
+ expect(p[3].lane).toBe(0);
+ });
+ it("spreads ends that meet at one item, keeping the order of the other ends", () => {
+ // Both arcs end at 6 (a combine of 2 and 4); the one from further left ends further left.
+ const p = arcLayout([{ a: 4, b: 6 }, { a: 2, b: 6 }]);
+ expect(p[1].dx2).toBeLessThan(p[0].dx2);
+ expect(p[0].dx2 + p[1].dx2).toBe(0);
+ expect(p[0].dx1).toBe(0);
+ });
+});
diff --git a/src/lib/logic.ts b/src/lib/logic.ts
index a2c7c8e..edbd875 100644
--- a/src/lib/logic.ts
+++ b/src/lib/logic.ts
@@ -180,21 +180,100 @@ export const MODE_LABEL: Record<Mode, string> = {
upscale: "Upscale",
};
-/** Version names within a project: imported images are "Original" ("Original 2", ...), and
- * results and pending jobs are numbered v1, v2... in project order. */
-export function versionLabels(items: { key: string; kind: "gen" | "input" | "job" }[]): Record<string, string> {
+/** Short names within a project. Every image that starts something new (a new image, a new
+ * seed of one, an import) gets the next number: "1", "2"… Everything made from it (edits,
+ * edits of edits, combinations led by it, upscales) is numbered within it: "1.1", "1.2"…
+ * `from`/`variant` as in timelineLinks. */
+export function versionLabels(items: { key: string; source: string | null; from: string[]; variant?: boolean }[]): Record<string, string> {
+ const links = timelineLinks(items);
const out: Record<string, string> = {};
- let v = 0;
- let inputs = 0;
+ const family: Record<string, string> = {};
+ const count: Record<string, number> = {};
+ let images = 0;
for (const it of items) {
- if (it.kind === "input") {
- inputs++;
- out[it.key] = inputs === 1 ? "Original" : `Original ${inputs}`;
- } else out[it.key] = `v${++v}`;
+ const parent = links[it.key].parents[0];
+ if (!parent || it.variant) {
+ const n = String(++images);
+ family[it.key] = n;
+ out[it.key] = n;
+ } else {
+ const f = family[parent];
+ family[it.key] = f;
+ count[f] = (count[f] ?? 0) + 1;
+ out[it.key] = `${f}.${count[f]}`;
+ }
}
return out;
}
+export interface ArcPlace {
+ /** 0 = lowest level; an arc sits above every shorter arc it spans or overlaps. */
+ lane: number;
+ /** Horizontal shift of each end from its thumbnail's centre, so arcs meeting at one
+ * thumbnail do not share a point. */
+ dx1: number;
+ dx2: number;
+}
+
+/** Places arcs between items at positions `a` -> `b` (indexes along the timeline). */
+export function arcLayout(arcs: { a: number; b: number }[], spread = 8, maxShift = 16): ArcPlace[] {
+ const lanes: { lo: number; hi: number }[][] = [];
+ const order = arcs.map((_, i) => i).sort((i, j) => Math.abs(arcs[i].b - arcs[i].a) - Math.abs(arcs[j].b - arcs[j].a) || Math.min(arcs[i].a, arcs[i].b) - Math.min(arcs[j].a, arcs[j].b));
+ const out: ArcPlace[] = arcs.map(() => ({ lane: 0, dx1: 0, dx2: 0 }));
+ for (const i of order) {
+ const lo = Math.min(arcs[i].a, arcs[i].b);
+ const hi = Math.max(arcs[i].a, arcs[i].b);
+ let lane = 0;
+ while (lanes[lane]?.some((s) => lo <= s.hi && s.lo <= hi)) lane++;
+ (lanes[lane] ??= []).push({ lo, hi });
+ out[i].lane = lane;
+ }
+ // Ends at the same item, ordered by where the other end lies (left to right), spread out.
+ const ends = new Map<number, { arc: number; end: "dx1" | "dx2"; other: number }[]>();
+ arcs.forEach((r, i) => {
+ for (const [at, end, other] of [[r.a, "dx1", r.b], [r.b, "dx2", r.a]] as const) {
+ if (!ends.has(at)) ends.set(at, []);
+ ends.get(at)!.push({ arc: i, end, other });
+ }
+ });
+ for (const list of ends.values()) {
+ list.sort((p, q) => p.other - q.other);
+ const step = list.length > 1 ? Math.min(spread, (2 * maxShift) / (list.length - 1)) : 0;
+ list.forEach((e, k) => (out[e.arc][e.end] = (k - (list.length - 1) / 2) * step));
+ }
+ return out;
+}
+
+/** "Image 1.2" for a short name, for sentences and headings. */
+export const imageName = (label: string | undefined) => (label ? `Image ${label}` : "Image");
+
+/** How a version joins the timeline: "start" begins a new line (a new image or an import),
+ * "next" continues from the version just before it, "variant" is the version before made
+ * again with a new seed, "branch" comes from an earlier version, "combine" from several. */
+export type LinkKind = "start" | "next" | "variant" | "branch" | "combine";
+
+export interface TimelineLink {
+ kind: LinkKind;
+ /** Keys of the versions this one was made from (in reference order). */
+ parents: string[];
+}
+
+/** Links between versions. `from` are the "gen:<id>"/"input:<id>" sources an item was made
+ * from; `variant` marks a new seed of `from[0]` rather than an edit of it. Sources that are
+ * not in the project are ignored. */
+export function timelineLinks(items: { key: string; source: string | null; from: string[]; variant?: boolean }[]): Record<string, TimelineLink> {
+ const bySource = new Map<string, string>();
+ const out: Record<string, TimelineLink> = {};
+ items.forEach((it, i) => {
+ const parents = [...new Set(it.from.map((f) => bySource.get(f)).filter((k): k is string => !!k && k !== it.key))];
+ const prev = items[i - 1]?.key;
+ const kind: LinkKind = !parents.length ? "start" : parents.length > 1 ? "combine" : parents[0] !== prev ? "branch" : it.variant ? "variant" : "next";
+ out[it.key] = { kind, parents };
+ if (it.source) bySource.set(it.source, it.key);
+ });
+ return out;
+}
+
/** "1m ago"-style age for the project list. */
export function timeAgo(ms: number, now = Date.now()): string {
const s = Math.max(0, (now - ms) / 1000);
@@ -439,7 +518,55 @@ export function installTimeLeft(remaining: number, bytesPerSec: number): string
return s < 60 ? "less than a minute left" : `about ${Math.ceil(s / 60)} min left`;
}
-export const ASPECTS = ["auto", "1:1", "4:3", "3:4", "16:9", "9:16", "custom"] as const;
+/** Aspect ratios offered in the menus (landscape, then its portrait twin). */
+export const ASPECTS = ["1:1", "5:4", "4:5", "4:3", "3:4", "3:2", "2:3", "16:9", "9:16", "2:1", "1:2", "21:9", "9:21"] as const;
+
+/** "16:9" -> 1.777…; null when it is not a usable ratio (the service accepts 1:8 to 8:1). */
+export function parseRatio(s: string): number | null {
+ const m = /^\s*(\d+(?:\.\d+)?)\s*[:x×/]\s*(\d+(?:\.\d+)?)\s*$/i.exec(s);
+ if (!m) return null;
+ const r = Number(m[1]) / Number(m[2]);
+ return Number.isFinite(r) && r >= 0.125 && r <= 8 ? r : null;
+}
+
+const gcd = (a: number, b: number): number => (b ? gcd(b, a % b) : a);
+
+/** Readable ratio of a size: "16:9" when it is (within 1 %) a common ratio, the reduced
+ * fraction when that is short ("7:5"), else "1.85:1". */
+export function aspectLabel(w: number, h: number): string {
+ if (w <= 0 || h <= 0) return "";
+ const r = w / h;
+ const known = ASPECTS.find((a) => Math.abs(parseRatio(a)! / r - 1) < 0.01);
+ if (known) return known;
+ const g = gcd(Math.round(w), Math.round(h));
+ const a = Math.round(w) / g;
+ const b = Math.round(h) / g;
+ if (a <= 32 && b <= 32) return `${a}:${b}`;
+ return r >= 1 ? `${r.toFixed(2)}:1` : `1:${(1 / r).toFixed(2)}`;
+}
+
+/** Width × height with ratio `r` and about the given pixel area, rounded to multiples of 16. */
+export function sizeForRatio(r: number, area: number, max = 4096): Size {
+ const r16 = (v: number) => Math.max(64, Math.min(max, Math.round(v / 16) * 16));
+ return { width: r16(Math.sqrt(area * r)), height: r16(Math.sqrt(area / r)) };
+}
+
+/** Expansion (source pixels, centred) that turns a w×h image into ratio `r` without cropping. */
+export function expandToRatio(w: number, h: number, r: number): OutpaintSpec {
+ if (w / h < r) {
+ const extra = Math.max(0, Math.round(h * r) - w);
+ return { left: Math.floor(extra / 2), right: extra - Math.floor(extra / 2), top: 0, bottom: 0, units: "pixels" };
+ }
+ const extra = Math.max(0, Math.round(w / r) - h);
+ return { left: 0, right: 0, top: Math.floor(extra / 2), bottom: extra - Math.floor(extra / 2), units: "pixels" };
+}
+
+/** Expansion (centred) to an exact final size; a side smaller than the source stays as it is. */
+export function expandToSize(w: number, h: number, W: number, H: number): OutpaintSpec {
+ const ex = Math.max(0, Math.round(W) - w);
+ const ey = Math.max(0, Math.round(H) - h);
+ return { left: Math.floor(ex / 2), right: ex - Math.floor(ex / 2), top: Math.floor(ey / 2), bottom: ey - Math.floor(ey / 2), units: "pixels" };
+}
/** Expansion presets in source pixels. */
export function expansionPreset(kind: "left" | "right" | "top" | "bottom" | "landscape" | "portrait" | "square", w: number, h: number, cur: OutpaintSpec): OutpaintSpec {
diff --git a/src/lib/platform.ts b/src/lib/platform.ts
new file mode 100644
index 0000000..87e6498
--- /dev/null
+++ b/src/lib/platform.ts
@@ -0,0 +1,8 @@
+// Platform-specific wording for labels (shortcut handlers accept Ctrl and ⌘ everywhere).
+
+export const isMac = typeof navigator !== "undefined" && /Mac/i.test(navigator.platform || navigator.userAgent);
+
+/** "Ctrl+Z" on Windows, "⌘Z" on macOS. */
+export const shortcut = (key: string) => (isMac ? `⌘${key}` : `Ctrl+${key}`);
+
+export const fileManager = isMac ? "Finder" : "Explorer";
diff --git a/src/lib/store.ts b/src/lib/store.ts
index caca228..e54c637 100644
--- a/src/lib/store.ts
+++ b/src/lib/store.ts
@@ -7,7 +7,7 @@
import { useSyncExternalStore } from "react";
import { api, ApiError, setPrivateInputs, subscribe } from "./api";
-import { buildRequest, emptyComposer, needsPrompt, newKey, persistentAdvanced, refFromGen, refFromInput, undoTarget, type CompareMode, type Composer, type Ref } from "./logic";
+import { buildRequest, emptyComposer, needsPrompt, newKey, persistentAdvanced, refFromGen, refFromInput, timelineLinks, undoTarget, versionLabels, type CompareMode, type Composer, type Ref } from "./logic";
import type { AppErrorBody, GenerationRecord, GenerationRequest, InputRecord, JobView, ServerState, StatusResponse } from "./types";
export type Overlay = null | "gallery" | "settings" | "inpaint" | "expand";
@@ -553,19 +553,89 @@ function retarget(c: Composer, ref: Ref): Composer {
return { ...c, refs: [ref, ...rest], mask: null, outpaint: null, aspect: "auto" };
}
-/** Shows a version; an image also becomes the one the prompt edits. Choosing a version
+const refOf = (it: SessionItem | undefined) => (it?.gen ? refFromGen(it.gen) : it?.input ? refFromInput(it.input) : null);
+
+/** Shows a version. While editing, the shown image also becomes the one the prompt edits;
+ * in "New image" mode browsing only shows it (Edit image switches back). Choosing a version
* by hand ends the current Undo/Redo walk. */
export function selectItem(key: string, keepRedo = false) {
setState((s) => {
const it = s.session.find((i) => i.key === key);
if (!it) return {};
- const ref = it.gen ? refFromGen(it.gen) : it.input ? refFromInput(it.input) : null;
+ const ref = s.composer.refs.length ? refOf(it) : null;
return { selected: key, composer: ref ? retarget(s.composer, ref) : s.composer, ...(keepRedo ? {} : { redo: [] }) };
});
}
-const undoItems = (session: SessionItem[]) =>
- session.map((i) => ({ key: i.key, source: itemSource(i), from: (i.gen?.request?.images?.[0] ?? (i.gen?.parent_id ? `gen:${i.gen.parent_id}` : null)) ?? i.request?.images?.[0] ?? null }));
+/** "New image": the prompt creates an image from scratch; the versions stay to look at. */
+export function startNewImage() {
+ setState((s) => ({ composer: { ...s.composer, refs: [], mask: null, outpaint: null } }));
+ focusPrompt();
+}
+
+/** "Edit image": the prompt edits the version shown (or the latest image). */
+export function editShownImage() {
+ setState((s) => {
+ const shown = s.session.find((i) => i.key === s.selected);
+ const ref = refOf(shown) ?? refOf([...s.session].reverse().find((i) => i.gen || i.input));
+ return ref ? { composer: retarget(s.composer, ref) } : {};
+ });
+ focusPrompt();
+}
+
+/** The sources ("gen:<id>"/"input:<id>") a version was made from; `variant` when it is the
+ * same recipe again with a new seed (a variation of a new image) rather than an edit. */
+export function itemOrigin(i: SessionItem): { from: string[]; variant: boolean } {
+ const req = i.gen?.request ?? i.request;
+ const mode = i.gen?.mode ?? req?.mode;
+ const images = mode === "generate" ? [] : req?.images ?? [];
+ if (images.length) return { from: images, variant: false };
+ const parent = i.gen?.parent_id ?? req?.parent_id;
+ return parent ? { from: [`gen:${parent}`], variant: true } : { from: [], variant: false };
+}
+
+const lineage = (session: SessionItem[]) => session.map((i) => ({ key: i.key, source: itemSource(i), ...itemOrigin(i) }));
+
+/** Short names of a project's images: "1", "1.1", "2"… */
+export const sessionLabels = (session: SessionItem[]) => versionLabels(lineage(session));
+
+/** How each version joins the timeline. */
+export const sessionLinks = (session: SessionItem[]) => timelineLinks(lineage(session));
+
+/** Puts a result's prompt back in the prompt box to try it again with small changes: it
+ * works on the same images the result was made from (none for a new image), and keeps the
+ * quality, size and advanced settings as they are now. */
+export function reusePrompt(g: GenerationRecord) {
+ setState((s) => {
+ const c = s.composer;
+ const sources = itemOrigin({ key: "", kind: "gen", gen: g });
+ const refs: Ref[] = [];
+ if (!sources.variant) {
+ sources.from.forEach((src, i) => {
+ const here = refOf(s.session.find((x) => itemSource(x) === src));
+ const rec = g.references[i];
+ const ref = here ?? (rec ? { key: newKey(), source: `input:${rec.input_id}`, width: rec.width, height: rec.height, name: `Image ${i + 1}` } : null);
+ if (ref) refs.push(ref);
+ });
+ }
+ const mask = g.mask_input_id && refs[0] ? { inputId: g.mask_input_id, refKey: refs[0].key } : null;
+ return {
+ composer: {
+ ...c,
+ prompt: g.prompt,
+ refs,
+ mask,
+ inpaintAction: mask ? g.request?.inpaint_action ?? "edit" : c.inpaintAction,
+ outpaint: g.outpaint && refs.length ? { ...g.outpaint, units: "pixels" } : null,
+ aspect: refs.length && !c.refs.length ? "auto" : c.aspect,
+ seed: null,
+ },
+ };
+ });
+ focusPrompt(true);
+}
+
+const undoItems = (session: SessionItem[]) => session.map((i) => ({ key: i.key, source: itemSource(i), from: itemOrigin(i).from[0] ?? null }));
/** The version Undo goes back to (what the selected one was made from), if any. */
export function undoKey(s: State): string | null {
@@ -651,11 +721,23 @@ export async function cancelJob(jobId: string) {
export function addAsReference(item: SessionItem) {
const ref = item.gen ? refFromGen(item.gen) : item.input ? refFromInput(item.input) : null;
if (!ref) return;
+ if (getState().composer.refs.some((r) => r.source === ref.source)) {
+ toastInfo("That image is already in the prompt.");
+ return;
+ }
+ if (getState().composer.refs.length >= 10) {
+ toastInfo("Up to 10 reference images are supported.");
+ return;
+ }
setState((s) => ({ composer: { ...s.composer, refs: [...s.composer.refs, ref].slice(0, 10), outpaint: null, aspect: s.composer.refs.length ? s.composer.aspect : "auto" } }));
}
-export function focusPrompt() {
- window.setTimeout(() => document.getElementById("prompt")?.focus(), 30);
+export function focusPrompt(cursorToEnd = false) {
+ window.setTimeout(() => {
+ const el = document.getElementById("prompt") as HTMLTextAreaElement | null;
+ el?.focus();
+ if (el && cursorToEnd) el.setSelectionRange(el.value.length, el.value.length);
+ }, 30);
}
export function removeSessionItem(key: string) {
diff --git a/src/lib/types.ts b/src/lib/types.ts
index 57f7eb6..531900e 100644
--- a/src/lib/types.ts
+++ b/src/lib/types.ts
@@ -4,7 +4,7 @@ export type Mode = "auto" | "generate" | "edit" | "inpaint" | "outpaint" | "mult
export type ResolvedMode = Exclude<Mode, "auto">;
/** What an inpaint does in the painted area. */
export type InpaintAction = "edit" | "remove" | "replace" | "add";
-export type Quality = "fast" | "balanced" | "quality";
+export type Quality = "fast" | "balanced" | "quality" | "ultra";
export type ResolutionMode = "auto" | "source" | "1k" | "1.5k" | "2k" | "custom";
export type OutputFormat = "png" | "jpeg" | "webp";
export type Units = "pixels" | "percent";
@@ -174,7 +174,7 @@ export interface StatusResponse {
service: { version: string; pid: number; uptime_s: number };
server: { state: ServerState; owned: boolean; pid: number | null; port: number; model: string | null; model_loaded: boolean; error: AppErrorBody | null };
profile: { id: string; display_name: string };
- gpu: { name: string; memory_total_mib: number; memory_used_mib: number; memory_free_mib: number; driver_version: string } | null;
+ gpu: { name: string; vendor: string; unified: boolean; memory_total_mib: number; memory_used_mib: number; memory_free_mib: number; driver_version: string } | null;
active_job: JobView | null;
queue: { queued: number; jobs: JobView[] };
upscaler?: { model: string; available: boolean; refine_mode: RefineMode };
@@ -184,7 +184,7 @@ export interface StatusResponse {
export interface Check { id: string; label: string; ok: boolean; detail: string }
export interface SetupReport { ready: boolean; checks: Check[]; devices: [string, string][]; backend_version: string | null }
-export type TextEncoderPlacement = "ram" | "gpu" | "cpu" | "auto";
+export type TextEncoderPlacement = "ram" | "gpu" | "cpu" | "auto" | "disk";
export interface ModelProfile {
id: string;
@@ -198,9 +198,13 @@ export interface ModelProfile {
capabilities: Record<string, boolean>;
}
+/** stable-diffusion.cpp build: CUDA (NVIDIA) or Vulkan (AMD, Intel, also NVIDIA). */
+export type GpuBackend = "cuda" | "vulkan" | "metal";
+
export interface AppConfig {
schemaVersion: number;
sdcppDir: string;
+ gpuBackend: GpuBackend;
serverPort: number;
servicePort: number;
activeProfile: string;
@@ -268,6 +272,12 @@ export interface HardwarePreset {
export interface HardwareReport {
gpu: StatusResponse["gpu"];
nvidia: boolean;
+ vendor: string | null;
+ /** Engine the presets' file lists are for. */
+ backend: GpuBackend;
+ recommended_backend: GpuBackend;
+ /** Engine in use now. */
+ current_backend: GpuBackend;
vram_mib: number | null;
vram_simulated: boolean;
ram_total_mib: number;
diff --git a/src/styles.css b/src/styles.css
index 6496035..06cae6b 100644
--- a/src/styles.css
+++ b/src/styles.css
@@ -111,19 +111,27 @@ svg { display: block; flex: none; }
/* ---------- main ---------- */
.main { flex: 1; display: flex; min-height: 0; position: relative; }
-.stage { flex: 1; min-width: 0; display: flex; flex-direction: column; align-items: center; justify-content: center; padding: 8px 24px 12px; position: relative; }
+.stage { container: stage / inline-size; flex: 1; min-width: 0; display: flex; flex-direction: column; align-items: center; justify-content: center; padding: 8px 24px 12px; position: relative; }
.stage-frame { position: relative; flex: 1; min-height: 0; width: 100%; display: flex; align-items: center; justify-content: center; }
.stage-img { max-width: 100%; max-height: 100%; object-fit: contain; border-radius: 6px; background-image: conic-gradient(var(--checker) 25%, transparent 0 50%, var(--checker) 0 75%, transparent 0); background-size: 16px 16px; }
/* the line under the image: version, what made it, compare, actions */
-.stage-bar { width: 100%; max-width: 820px; display: flex; align-items: center; gap: 10px; margin-top: 8px; min-height: 36px; }
-.stage-meta { flex: 1; min-width: 0; display: flex; align-items: center; gap: 8px; font-size: 12px; color: var(--text-3); }
+/* under the image: what it is (one centred line), then what you can do with it */
+.stage-bar { width: 100%; max-width: 820px; display: flex; flex-direction: column; align-items: center; gap: 4px; margin-top: 10px; }
+.stage-meta { max-width: 100%; justify-content: center; min-height: 24px; min-width: 0; display: flex; align-items: center; gap: 8px; font-size: 12px; color: var(--text-3); }
.stage-meta .ver { font-weight: 600; color: var(--text); font-size: 13px; flex: none; }
.stage-meta .mode { flex: none; padding: 1px 7px; border-radius: 6px; background: var(--surface); color: var(--text-2); }
-.stage-meta .stage-prompt { flex: 1; min-width: 0; color: var(--text-2); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
+.stage-meta .stage-prompt { flex: 0 1 auto; min-width: 0; color: var(--text-2); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.stage-meta .dims { flex: none; }
.actions { display: flex; gap: 2px; flex: none; }
.actions .sep { width: 1px; background: var(--border); margin: 6px 4px; }
+/* main actions: icon + word */
+.act { height: 34px; padding: 0 10px; border-radius: 10px; display: inline-flex; align-items: center; gap: 6px; font-size: 12.5px; color: var(--text-2); white-space: nowrap; }
+.act:hover, .act.active { background: var(--surface); color: var(--text); }
+@container stage (max-width: 600px) {
+ .act { width: 34px; padding: 0; justify-content: center; }
+ .act-label { display: none; }
+}
.empty { display: flex; flex-direction: column; align-items: center; gap: 14px; color: var(--text-3); text-align: center; }
.empty h1 { font-size: 26px; font-weight: 600; letter-spacing: -0.02em; color: var(--text); margin: 0; }
@@ -146,10 +154,22 @@ svg { display: block; flex: none; }
.failed { display: flex; flex-direction: column; align-items: center; gap: 10px; max-width: 460px; text-align: center; color: var(--text-2); }
.failed strong { color: var(--text); font-weight: 600; }
-/* version timeline: Original -> v1 -> v2 ... */
-.timeline { flex: none; display: flex; align-items: flex-start; max-width: 100%; overflow-x: auto; padding: 12px 4px 2px; scrollbar-width: none; }
+/* version timeline: a line per image and the versions made from it; arcs join versions made
+ from earlier ones or from several */
+.timeline { flex: none; display: flex; max-width: 100%; overflow-x: auto; padding: 0 8px 4px; margin: 14px 0 10px; scrollbar-width: none; }
.timeline::-webkit-scrollbar { display: none; }
-.timeline .link { flex: none; width: 14px; height: 1.5px; background: var(--surface-3); margin-top: 24px; }
+.track { position: relative; display: flex; align-items: flex-start; width: max-content; }
+.arcs { position: absolute; left: 0; top: 0; width: 100%; overflow: visible; pointer-events: none; }
+.arcs path { fill: none; stroke: var(--text-3); stroke-opacity: 0.45; stroke-width: 1.5; stroke-linecap: round; stroke-linejoin: round; transition: stroke-opacity 0.15s; }
+.arcs path.hot { stroke: var(--text); stroke-opacity: 1; stroke-width: 2; }
+.joint { flex: none; }
+.joint.link { width: 18px; height: 1.5px; background: var(--text-3); opacity: 0.45; margin-top: 24px; }
+.joint.link.hot { opacity: 1; background: var(--text); }
+.joint.link.dashed { background: none; border-top: 1.5px dashed var(--text-3); height: 0; }
+.joint.link.dashed.hot { border-top-color: var(--text); }
+.joint.gap { width: 16px; }
+/* a new image: the line breaks */
+.joint.break { width: 29px; height: 32px; margin-top: 8px; background: linear-gradient(var(--border), var(--border)) center / 1px 100% no-repeat; }
.frame { flex: none; display: flex; flex-direction: column; align-items: center; gap: 5px; padding: 0 2px; color: var(--text-3); }
.frame-img { width: 48px; height: 48px; border-radius: 10px; overflow: hidden; background: var(--surface); position: relative; display: grid; place-items: center; outline: 2px solid transparent; outline-offset: 2px; }
.frame-img img { position: absolute; inset: 0; width: 100%; height: 100%; object-fit: cover; }
@@ -157,6 +177,8 @@ svg { display: block; flex: none; }
.frame-state { position: relative; color: var(--text); display: grid; place-items: center; }
.frame:hover .frame-img { outline-color: var(--surface-3); }
.frame.sel .frame-img { outline-color: var(--text); }
+/* what the shown version was made from */
+.frame.src .frame-img { outline: 2px dashed var(--text-3); }
.frame.sel { color: var(--text); }
.frame-label { display: flex; align-items: center; gap: 3px; font-size: 11px; line-height: 1; white-space: nowrap; }
.frame-badge { position: absolute; right: 3px; bottom: 3px; width: 17px; height: 17px; border-radius: 5px; background: var(--invert-bg); color: var(--invert-text); display: grid; place-items: center; }
@@ -168,6 +190,11 @@ svg { display: block; flex: none; }
/* what the prompt works on */
.target { display: flex; align-items: center; gap: 10px; padding: 2px 0 10px; margin-bottom: 6px; border-bottom: 1px solid var(--border); min-width: 0; }
.target .spacer { flex: 1; }
+/* Edit image | New image */
+.mode-switch { display: flex; flex: none; background: var(--bg); border-radius: 12px; padding: 3px; gap: 2px; }
+.mode-switch button { height: 30px; padding: 0 10px; border-radius: 9px; display: inline-flex; align-items: center; gap: 6px; font-size: 12.5px; color: var(--text-2); white-space: nowrap; }
+.mode-switch button:hover:not(.on) { color: var(--text); }
+.mode-switch button.on { background: var(--invert-bg); color: var(--invert-text); }
.target-icon { width: 40px; height: 40px; border-radius: 10px; background: var(--surface-2); display: grid; place-items: center; color: var(--text-2); flex: none; }
.target-text { display: flex; flex-direction: column; min-width: 0; }
.target-text .what { font-size: 13px; font-weight: 600; color: var(--text); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
@@ -229,6 +256,7 @@ input[type="range"] { width: 100%; accent-color: var(--text); }
.switch::after { content: ""; position: absolute; left: 3px; top: 3px; width: 14px; height: 14px; border-radius: 50%; background: var(--bg); transition: left 0.15s; }
.switch.on { background: var(--text); }
.switch.on::after { left: 17px; }
+.switch:disabled { opacity: 0.4; cursor: not-allowed; }
.btn { height: 34px; padding: 0 14px; border-radius: 10px; background: var(--surface); display: inline-flex; align-items: center; gap: 8px; font-size: 13px; }
.btn:hover:not(:disabled) { background: var(--surface-2); }
.btn.primary { background: var(--invert-bg); color: var(--invert-text); }
@@ -339,6 +367,14 @@ pre.diag { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font
/* before / after comparison */
.compare-bar { display: flex; gap: 8px; align-items: center; flex: none; }
.compare-bar .seg { min-width: 260px; }
+/* comparison views, on the image itself */
+.compare-overlay { position: absolute; left: 50%; bottom: 10px; transform: translateX(-50%); z-index: 2; opacity: 0.85; transition: opacity 0.15s; }
+.compare-overlay:hover { opacity: 1; }
+.compare-overlay .compare-bar { gap: 4px; }
+.compare-overlay .seg { min-width: 0; background: rgba(0, 0, 0, 0.55); backdrop-filter: blur(10px); border-radius: 10px; }
+.compare-overlay .seg button { height: 26px; padding: 0 10px; color: rgba(255, 255, 255, 0.8); }
+.compare-overlay .seg button:hover { color: #fff; }
+.compare-overlay .seg button.on { background: rgba(255, 255, 255, 0.92); color: #111; box-shadow: none; }
.compare-box { position: relative; border-radius: 6px; overflow: hidden; touch-action: none; user-select: none; background-image: conic-gradient(var(--checker) 25%, transparent 0 50%, var(--checker) 0 75%, transparent 0); background-size: 16px 16px; }
.compare-box.slider { cursor: ew-resize; }
.compare-img, .compare-diff { position: absolute; inset: 0; width: 100%; height: 100%; display: block; }
@@ -376,3 +412,35 @@ pre.diag { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font
.viewer-tag { color: #fff; font-weight: 600; }
.viewer-close { width: 36px; height: 36px; border-radius: 10px; display: grid; place-items: center; color: #fff; background: rgba(255, 255, 255, 0.12); }
.viewer-close:hover { background: rgba(255, 255, 255, 0.22); }
+
+/* ---------- aspect ratio and size menus ---------- */
+.popover.wide { min-width: 0; }
+.ratio-icon { display: inline-grid; place-items: center; flex: none; }
+.ratio-icon i { display: block; border: 1.5px solid currentColor; border-radius: 3px; box-sizing: border-box; }
+.aspect-menu { width: 296px; display: flex; flex-direction: column; gap: 6px; }
+.aspect-auto { display: flex; align-items: center; gap: 10px; width: 100%; padding: 8px 10px; border-radius: 9px; font-size: 13px; text-align: left; }
+.aspect-auto:hover, .aspect-auto.on { background: var(--surface); }
+.aspect-auto .sub { color: var(--text-3); font-size: 12px; margin-left: auto; }
+.aspect-grid { display: grid; grid-template-columns: repeat(4, 1fr); gap: 4px; }
+.aspect-grid button { display: flex; flex-direction: column; align-items: center; gap: 5px; padding: 8px 2px 6px; border-radius: 9px; font-size: 12px; color: var(--text-2); }
+.aspect-grid button:hover { background: var(--surface); color: var(--text); }
+.aspect-grid button.on { background: var(--invert-bg); color: var(--invert-text); }
+.aspect-custom { display: flex; align-items: center; gap: 8px; padding: 6px 4px 2px 10px; border-top: 1px solid var(--border); font-size: 12px; }
+.aspect-custom .btn { height: 30px; margin-left: auto; }
+.pair { display: inline-flex; align-items: center; gap: 6px; }
+.pair .input { width: 72px; height: 30px; padding: 0 8px; }
+.ratio-tag { display: inline-flex; align-items: center; gap: 5px; font-size: 12px; color: var(--text-2); white-space: nowrap; }
+.res-menu { width: 340px; }
+.res-custom { display: flex; flex-direction: column; gap: 6px; padding: 8px 10px; border-radius: 9px; }
+.res-custom.on { background: var(--surface); }
+.res-custom-head { display: flex; align-items: center; gap: 10px; font-size: 13px; }
+.res-custom-head .check { width: 16px; display: grid; place-items: center; }
+.res-custom-row { display: flex; align-items: center; gap: 6px; padding-left: 26px; }
+.res-foot { padding: 8px 10px 4px; border-top: 1px solid var(--border); margin-top: 4px; font-size: 12px; }
+.expand-size { width: 280px; display: flex; flex-direction: column; gap: 8px; padding: 6px; font-size: 12px; }
+.expand-final { color: var(--text); font-weight: 600; }
+.editor-tools .group .pill:hover { background: var(--surface-2); }
+
+/* a thumbnail dragged from the timeline */
+.drag-ghost { position: fixed; z-index: 100; width: 56px; height: 56px; object-fit: cover; border-radius: 10px; pointer-events: none; transform: translate(-50%, -60%) rotate(-3deg); box-shadow: var(--shadow); opacity: 0.85; }
+.drag-ghost.over { opacity: 1; transform: translate(-50%, -60%) scale(1.08); }