Skip to content

armgit / kokon

Public Imported from GitHub
0

Edit photos by describing the change, on your own GPU. A free, open-source alternative to Nano Banana and ChatGPT image editing: photos stay on your computer, no daily limit. One installer for Windows, Mac and Linux sets up the best open model (Qwen-Image 2.1). MCP server for AI agents.

0.12 work: AI marking (EU AI Act), Prompt+ for areas and Expand, lab fixes

Gabriele Lozupone843d4617cb48
diff --git a/.gitignore b/.gitignore
index 767ff31..dbbd116 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,6 +7,7 @@ node_modules
/src-tauri/binaries/*.exe
# Model files bundled with the app, fetched at build time (scripts/release/fetch-bundled.mjs)
/src-tauri/resources/*.pth
+/src-tauri/resources/*.onnx
/src-tauri/resources/engine-mac/
*.log
/.test-data
diff --git a/NOTICE b/NOTICE
index 9e46911..efa7d01 100644
--- a/NOTICE
+++ b/NOTICE
@@ -35,6 +35,32 @@ Third-party software included in the Kokon installer
https://creativecommons.org/licenses/by/4.0/). Included unmodified; Setup downloads the same
file from a mirror (https://github.com/GabrieleLozupone/kokon/releases/tag/models-1) if the
installed copy is missing.
+- TrustMark by Adobe (https://github.com/adobe/trustmark), MIT: the P variant's encoder and
+ decoder models (encoder_P.onnx, decoder_P.onnx), included unmodified for the AI watermark,
+ and its BCH error-correction code (crates/qls-core/src/watermark/bch.rs, from the Rust
+ implementation), with an image pipeline ported from its Python reference:
+
+ MIT License
+
+ Copyright (c) 2024 Adobe
+
+ Permission is hereby granted, free of charge, to any person obtaining a copy
+ of this software and associated documentation files (the "Software"), to deal
+ in the Software without restriction, including without limitation the rights
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ copies of the Software, and to permit persons to whom the Software is
+ furnished to do so, subject to the following conditions:
+
+ The above copyright notice and this permission notice shall be included in all
+ copies or substantial portions of the Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ SOFTWARE.
Downloaded by the setup screen (not part of this product)
diff --git a/README.md b/README.md
index 9d95353..e5a866f 100644
--- a/README.md
+++ b/README.md
@@ -495,6 +495,8 @@ Kokon makes only two kinds of network request:
- **Setup:** `github.com` for the stable-diffusion.cpp release (and the MoSR upscaler, if the copy
that comes with the app is missing) and `huggingface.co` for the models and the other upscalers.
+ The AI watermark's models come with the app; Adobe's model host (`cai-watermark.adobe.net`) is
+ contacted only if that copy is missing.
- **Update check:** at most once a day, the app fetches `latest.json` from this repository's
latest GitHub release, a small file with the newest version number. Nothing about you or your
images is sent. Turn it off on the welcome screen or in Settings → General. Updates are
@@ -845,6 +847,13 @@ place to start. Please run the tests above before opening a pull request.
Kokon edits real photos. Don't use it to deceive, harass or impersonate people, or to make
images of someone without their consent. You are responsible for what you create and share.
+Every image Kokon makes or changes with AI is marked as such, as the EU AI Act asks: in the file's
+metadata (IPTC Digital Source Type) and with an invisible watermark ([TrustMark](https://github.com/adobe/trustmark),
+the text `Kokon-AI`) that survives JPEG, resizing and screenshots. Please don't remove these
+marks. If you publish a realistic image of real people, places or events, say that it was made
+or changed with AI. Settings → General → **Check an image** reads the marks of any image;
+[docs/ai-marking.md](docs/ai-marking.md) has the details.
+
## License
Kokon's source code is released under the [GNU Affero General Public License v3.0](LICENSE)
diff --git a/crates/qls-core/examples/watermark_check.rs b/crates/qls-core/examples/watermark_check.rs
new file mode 100644
index 0000000..5059661
--- /dev/null
+++ b/crates/qls-core/examples/watermark_check.rs
@@ -0,0 +1,275 @@
+//! The AI watermark (qls_core::watermark) by hand, for docs/ai-marking.md's measurements:
+//!
+//! cargo run --release -p qls-core --example watermark_check -- <sdcpp_dir> mark <in> <out.png>
+//! cargo run --release -p qls-core --example watermark_check -- <sdcpp_dir> check <image>...
+//! cargo run --release -p qls-core --example watermark_check -- <sdcpp_dir> bench <folder> [max]
+//! cargo run --release -p qls-core --example watermark_check -- <sdcpp_dir> negatives <folder> [max]
+//!
+//! `bench` marks every PNG/JPEG/WebP in the folder (CPU), prints the PSNR and the largest pixel
+//! change, then checks the mark after common changes (JPEG, resizing, crops, brightness, noise,
+//! blur, mirror) and that the unmarked images are not read as marked. `negatives` only reads
+//! unmarked images (photos, screenshots): how many are taken for Kokon's mark (should be none) or
+//! for any TrustMark. Nothing is written.
+
+use std::path::{Path, PathBuf};
+
+use image::{RgbaImage, imageops::FilterType};
+use qls_core::watermark;
+
+fn load(p: &Path) -> RgbaImage {
+ image::open(p)
+ .unwrap_or_else(|e| panic!("{}: {e}", p.display()))
+ .to_rgba8()
+}
+
+fn psnr(a: &RgbaImage, b: &RgbaImage) -> (f64, u8) {
+ let (mut se, mut n, mut max) = (0f64, 0f64, 0u8);
+ for (p, q) in a.pixels().zip(b.pixels()) {
+ for c in 0..3 {
+ let d = p[c].abs_diff(q[c]);
+ max = max.max(d);
+ se += (d as f64).powi(2);
+ n += 1.0;
+ }
+ }
+ (
+ if se == 0.0 {
+ f64::INFINITY
+ } else {
+ 10.0 * (255f64.powi(2) / (se / n)).log10()
+ },
+ max,
+ )
+}
+
+fn jpeg(img: &RgbaImage, q: u8) -> RgbaImage {
+ image::load_from_memory(&qls_core::imaging::encode_jpeg(img, q).unwrap())
+ .unwrap()
+ .to_rgba8()
+}
+
+fn scale(img: &RgbaImage, f: f32) -> RgbaImage {
+ image::imageops::resize(
+ img,
+ ((img.width() as f32 * f) as u32).max(1),
+ ((img.height() as f32 * f) as u32).max(1),
+ FilterType::Triangle,
+ )
+}
+
+/// Keeps `keep` of the area, cut from the right and bottom (the centre moves).
+fn crop(img: &RgbaImage, keep: f32) -> RgbaImage {
+ let s = keep.sqrt();
+ image::imageops::crop_imm(
+ img,
+ 0,
+ 0,
+ (img.width() as f32 * s) as u32,
+ (img.height() as f32 * s) as u32,
+ )
+ .to_image()
+}
+
+fn map(img: &RgbaImage, f: impl Fn(u32, u32, u8) -> f32) -> RgbaImage {
+ let mut o = img.clone();
+ for (x, y, p) in o.enumerate_pixels_mut() {
+ for c in 0..3 {
+ p.0[c] = f(x, y, p.0[c]).round().clamp(0.0, 255.0) as u8;
+ }
+ }
+ o
+}
+
+fn noise(x: u32, y: u32) -> f32 {
+ // Deterministic ±6 noise.
+ let h = (x.wrapping_mul(73856093) ^ y.wrapping_mul(19349663)).wrapping_mul(2654435761);
+ (h >> 24) as f32 / 255.0 * 12.0 - 6.0
+}
+
+fn images(dir: &Path, max: usize) -> Vec<PathBuf> {
+ let mut v: Vec<PathBuf> = walk(dir)
+ .into_iter()
+ .filter(|p| {
+ p.extension().and_then(|e| e.to_str()).is_some_and(|e| {
+ matches!(
+ e.to_ascii_lowercase().as_str(),
+ "png" | "jpg" | "jpeg" | "webp"
+ )
+ })
+ })
+ .collect();
+ v.sort();
+ v.truncate(max);
+ v
+}
+
+fn walk(dir: &Path) -> Vec<PathBuf> {
+ let mut out = vec![];
+ for e in std::fs::read_dir(dir).into_iter().flatten().flatten() {
+ let p = e.path();
+ if p.is_dir() {
+ out.extend(walk(&p))
+ } else {
+ out.push(p)
+ }
+ }
+ out
+}
+
+fn main() {
+ let args: Vec<String> = std::env::args().skip(1).collect();
+ let dir = PathBuf::from(&args[0]);
+ match args[1].as_str() {
+ "mark" => {
+ let img = load(Path::new(&args[2]));
+ let out = watermark::embed(&dir, &img).unwrap().unwrap_or(img.clone());
+ let (p, m) = psnr(&img, &out);
+ out.save(&args[3]).unwrap();
+ println!(
+ "marked {} -> {} (PSNR {p:.1} dB, max change {m})",
+ args[2], args[3]
+ );
+ }
+ "check" => {
+ for f in &args[2..] {
+ println!(
+ "{f}: {:?}",
+ watermark::detect(&dir, &load(Path::new(f))).unwrap()
+ );
+ }
+ }
+ "bench" => {
+ let max = args
+ .get(3)
+ .and_then(|s| s.parse().ok())
+ .unwrap_or(usize::MAX);
+ let files = images(Path::new(&args[2]), max);
+ type T = (&'static str, Box<dyn Fn(&RgbaImage) -> RgbaImage>);
+ let tests: Vec<T> = vec![
+ ("none (PNG)", Box::new(|i| i.clone())),
+ ("JPEG 90", Box::new(|i| jpeg(i, 90))),
+ ("JPEG 75", Box::new(|i| jpeg(i, 75))),
+ ("JPEG 50", Box::new(|i| jpeg(i, 50))),
+ ("half size", Box::new(|i| scale(i, 0.5))),
+ ("quarter size", Box::new(|i| scale(i, 0.25))),
+ ("1.5x size", Box::new(|i| scale(i, 1.5))),
+ (
+ "screenshot-like (75% + JPEG 85)",
+ Box::new(|i| jpeg(&scale(i, 0.75), 85)),
+ ),
+ ("crop 10% (right/bottom)", Box::new(|i| crop(i, 0.9))),
+ ("crop 20% (right/bottom)", Box::new(|i| crop(i, 0.8))),
+ (
+ "brightness +10%",
+ Box::new(|i| map(i, |_, _, v| v as f32 * 1.1)),
+ ),
+ (
+ "noise ±6",
+ Box::new(|i| map(i, |x, y, v| v as f32 + noise(x, y))),
+ ),
+ (
+ "blur (half then back)",
+ Box::new(|i| {
+ image::imageops::resize(
+ &scale(i, 0.5),
+ i.width(),
+ i.height(),
+ FilterType::Triangle,
+ )
+ }),
+ ),
+ ("mirror", Box::new(|i| image::imageops::flip_horizontal(i))),
+ ];
+ let mut hits = vec![0usize; tests.len()];
+ let (mut fp, mut psnrs, mut maxes, mut ms) = (0, vec![], vec![], vec![]);
+ for f in &files {
+ let img = load(f);
+ if watermark::detect(&dir, &img)
+ .unwrap()
+ .is_some_and(|x| x.kokon)
+ {
+ fp += 1;
+ println!("already marked (or a false positive): {}", f.display());
+ }
+ let t = std::time::Instant::now();
+ let Some(marked) = watermark::embed(&dir, &img).unwrap() else {
+ continue;
+ };
+ ms.push(t.elapsed().as_millis());
+ let (p, m) = psnr(&img, &marked);
+ psnrs.push(p);
+ maxes.push(m);
+ let row: Vec<bool> = tests
+ .iter()
+ .map(|(_, t)| {
+ watermark::detect(&dir, &t(&marked))
+ .unwrap()
+ .is_some_and(|x| x.kokon)
+ })
+ .collect();
+ for (h, ok) in hits.iter_mut().zip(&row) {
+ *h += *ok as usize;
+ }
+ println!(
+ "{} {}x{}: PSNR {p:.1} dB, max {m}, {} ms, read after: {}",
+ f.file_name().unwrap().to_string_lossy(),
+ img.width(),
+ img.height(),
+ ms.last().unwrap(),
+ row.iter()
+ .map(|b| if *b { '+' } else { '-' })
+ .collect::<String>()
+ );
+ }
+ let n = psnrs.len();
+ if n == 0 {
+ println!("no images");
+ return;
+ }
+ psnrs.sort_by(|a, b| a.partial_cmp(b).unwrap());
+ println!(
+ "\n{n} images marked; PSNR min {:.1} / median {:.1} dB; largest change {} levels; median {} ms",
+ psnrs[0],
+ psnrs[n / 2],
+ maxes.iter().max().unwrap(),
+ {
+ ms.sort();
+ ms[n / 2]
+ }
+ );
+ println!("unmarked images read as Kokon's: {fp} of {}", files.len());
+ for ((name, _), h) in tests.iter().zip(hits) {
+ println!(" {name:34} {h}/{n} ({:.0}%)", 100.0 * h as f64 / n as f64);
+ }
+ }
+ "negatives" => {
+ let max = args
+ .get(3)
+ .and_then(|s| s.parse().ok())
+ .unwrap_or(usize::MAX);
+ let (mut n, mut kokon, mut other) = (0, 0, 0);
+ for f in images(Path::new(&args[2]), max) {
+ let Ok(img) = image::open(&f) else { continue };
+ n += 1;
+ match watermark::detect(&dir, &img.to_rgba8()).unwrap() {
+ Some(x) if x.kokon => {
+ kokon += 1;
+ println!("KOKON (false positive?): {}", f.display());
+ }
+ Some(x) => {
+ other += 1;
+ println!(
+ "other TrustMark {:?} {:?}: {}",
+ x.schema,
+ x.text,
+ f.display()
+ );
+ }
+ None => {}
+ }
+ }
+ println!("{n} images: {kokon} read as Kokon's, {other} as another TrustMark");
+ }
+ _ => eprintln!("mark | check | bench | negatives"),
+ }
+}
diff --git a/crates/qls-core/src/catalog.rs b/crates/qls-core/src/catalog.rs
index de6c23d..95bab4f 100644
--- a/crates/qls-core/src/catalog.rs
+++ b/crates/qls-core/src/catalog.rs
@@ -576,6 +576,39 @@ pub const UPSCALER: Download = Download {
engine_tag: None,
};
+/// Adobe TrustMark, variant P (MIT code and weights): the invisible AI watermark every result
+/// gets (watermark.rs; AI Act Art. 50(2), with the XMP of provenance.rs). Bundled with the app
+/// like the upscaler (copied into place at start) so marking never waits for a download; Adobe's
+/// model host is the fallback. Pinned by size and SHA-256 (downloaded 2026-11-04).
+pub const TRUSTMARK_P_ENCODER: Download = Download {
+ id: "trustmark_p_encoder",
+ label: "TrustMark P encoder (AI watermark)",
+ kind: FileKind::Model,
+ url: "https://cai-watermark.adobe.net/watermarking/trustmark-models/encoder_P.onnx",
+ fallback_urls: &[],
+ page: "https://github.com/adobe/trustmark",
+ size: 17_312_208,
+ sha256: "053441c9c9f05fc158ccba71c610d9d58fcd2c82d1912bf0ffcee988cf2f74c8",
+ dest: "models/watermark/encoder_P.onnx",
+ provides: &[],
+ engine_tag: None,
+};
+
+/// The TrustMark P decoder: reads the watermark (skip an image already marked, "Check an image").
+pub const TRUSTMARK_P_DECODER: Download = Download {
+ id: "trustmark_p_decoder",
+ label: "TrustMark P decoder (AI watermark check)",
+ kind: FileKind::Model,
+ url: "https://cai-watermark.adobe.net/watermarking/trustmark-models/decoder_P.onnx",
+ fallback_urls: &[],
+ page: "https://github.com/adobe/trustmark",
+ size: 47_400_467,
+ sha256: "be6d7c33f8a7b376f179e75f3f7c58ff816a9ac7bb6d37fd0a729a635f624c35",
+ dest: "models/watermark/decoder_P.onnx",
+ provides: &[],
+ engine_tag: None,
+};
+
const ESRGAN_MODELS: &str = "https://huggingface.co/uwg/upscaler/tree/main/ESRGAN";
/// 4x_NMKD-Superscale-SP_178000_G by Nmkd (ESRGAN, WTFPL): the default upscaler since
@@ -753,6 +786,7 @@ pub fn recommended_preset(vram_mib: Option<u64>, unified: bool) -> &'static Pres
pub fn downloads_for(p: &Preset, backend: GpuBackend, sdcpp_dir: &Path) -> Vec<Download> {
let mut v = engine_downloads(backend);
v.push(UPSCALER);
+ v.extend([TRUSTMARK_P_ENCODER, TRUSTMARK_P_DECODER]);
if backend != GpuBackend::Metal {
v.push(SUPERSCALE);
}
@@ -781,6 +815,7 @@ pub fn profile_model_downloads(profile: &crate::config::ModelProfile) -> Vec<Dow
pub fn downloads_for_profile(profile: &crate::config::ModelProfile, backend: GpuBackend) -> Vec<Download> {
let mut v = engine_downloads(backend);
v.push(UPSCALER);
+ v.extend([TRUSTMARK_P_ENCODER, TRUSTMARK_P_DECODER]);
if backend != GpuBackend::Metal {
v.push(SUPERSCALE);
}
@@ -854,10 +889,10 @@ mod tests {
assert!(metal.iter().all(|d| d.id != "upscaler_superscale"), "Macs default to Lanczos: no Superscale download");
for p in PRESETS {
let cuda = downloads_for(p, GpuBackend::Cuda, Path::new("new"));
- assert_eq!(cuda.len(), 8);
+ assert_eq!(cuda.len(), 10);
assert!(cuda.iter().any(|d| d.id == "upscaler") && cuda.iter().any(|d| d.id == "upscaler_superscale"), "MoSR (bundled) and the default model");
let vk = downloads_for(p, GpuBackend::Vulkan, Path::new("new"));
- assert_eq!(vk.len(), 7);
+ assert_eq!(vk.len(), 9);
assert!(vk.iter().all(|d| d.id != "engine_cuda" && d.id != "cuda_runtime"));
}
}
@@ -993,7 +1028,7 @@ mod tests {
std::fs::write(&vae, b"mine").unwrap();
assert_eq!(profile_model_downloads(&p).len(), 1);
let all = downloads_for_profile(&p, GpuBackend::Vulkan);
- assert_eq!(all.iter().map(|d| d.id).collect::<Vec<_>>(), ["engine_vulkan", "upscaler", "upscaler_superscale", "vision_projector"]);
+ assert_eq!(all.iter().map(|d| d.id).collect::<Vec<_>>(), ["engine_vulkan", "upscaler", "trustmark_p_encoder", "trustmark_p_decoder", "upscaler_superscale", "vision_projector"]);
}
#[test]
diff --git a/crates/qls-core/src/engine.rs b/crates/qls-core/src/engine.rs
index 64f1469..4f7f681 100644
--- a/crates/qls-core/src/engine.rs
+++ b/crates/qls-core/src/engine.rs
@@ -22,7 +22,7 @@ use crate::paths;
use crate::pipeline;
use crate::planner::{self, GenerationRequest, Mode, OutputFormat, Plan, PlanInputs, ResolutionMode, ResolvedMode};
use crate::process::{BackendProgress, BackendStage, ProgressParser, ServerManager, ServerState};
-use crate::sdcpp::{BackendJobStatus, CancelOutcome, Capabilities, SdcppClient, UpscaleRequest};
+use crate::sdcpp::{BackendJobStatus, CancelOutcome, Capabilities, SdcppClient, TextGenRequest, UpscaleRequest};
use crate::store::{GenerationRecord, InputRecord, PipelineRecord, ReferenceRecord, Store, new_id, now_rfc3339};
pub const APP_VERSION: &str = env!("CARGO_PKG_VERSION");
@@ -321,6 +321,7 @@ impl Engine {
*g = new;
drop(g);
self.place_bundled_upscaler();
+ self.place_bundled_watermark();
Ok(restart)
}
@@ -413,6 +414,54 @@ impl Engine {
/// Copies the upscaler shipped with the app into the configured folder when it is missing
/// there (new install, another folder chosen). Failures leave it to the setup download.
+ /// The invisible AI watermark (watermark.rs) in a result's pixels when it is AI-made
+ /// (provenance::is_ai). An upscale or LUT of an import becomes AI-made when the import declared
+ /// it (XMP or C2PA: kept, the Code of Practice's Measure 1.2) or still carries Kokon's mark.
+ /// Never fails the job: without the model the file keeps its XMP and a note says so.
+ pub(crate) fn mark<'a>(&self, img: &'a image::RgbaImage, rec: &mut GenerationRecord, inputs: &[InputRecord]) -> std::borrow::Cow<'a, image::RgbaImage> {
+ use std::borrow::Cow;
+ let dir = self.config().sdcpp_dir;
+ // An upscale or LUT changes no content: it is what its parent was (a LUT of a LUT of a
+ // camera photo is no more AI-made than the photo).
+ if matches!(rec.mode.as_str(), "upscale" | "lut")
+ && rec.source_type.is_none()
+ && let Some(parent) = rec.parent_id.as_deref().and_then(|p| self.gen_store(p).get_generation(p).ok())
+ {
+ rec.source_type = Some(crate::provenance::source_type(&parent).into());
+ }
+ if !crate::provenance::is_ai(rec) {
+ let declared = inputs.iter().find_map(|i| std::fs::read(&i.path).ok().and_then(|b| crate::provenance::declared_ai_type(&b)));
+ let kokon = || crate::watermark::detect(&dir, img).ok().flatten().is_some_and(|f| f.kokon);
+ match declared {
+ Some(t) => rec.source_type = Some(t.into()),
+ None if kokon() => rec.source_type = Some(crate::provenance::AI_TYPES[1].into()),
+ None => return Cow::Borrowed(img),
+ }
+ }
+ match crate::watermark::embed(&dir, img) {
+ Ok(Some(m)) => Cow::Owned(m),
+ Ok(None) => Cow::Borrowed(img),
+ Err(e) => {
+ tracing::warn!("no AI watermark on {}: {} {:?}", rec.id, e.message, e.detail);
+ rec.notes.push("The invisible AI watermark could not be added (its model is missing or failed); the file's metadata still marks it as made with AI.".into());
+ Cow::Borrowed(img)
+ }
+ }
+ }
+
+ /// Copies the AI watermark's models (TrustMark P) from the app's bundle when they are missing.
+ pub fn place_bundled_watermark(&self) {
+ let Some(bundle) = crate::installer::bundle_dir() else { return };
+ let dir = self.config().sdcpp_dir;
+ for d in crate::watermark::TRUSTMARK_P {
+ match crate::installer::install_bundled(&d, &dir, &bundle) {
+ Ok(true) => tracing::info!("bundled {} copied into {}", d.file_name(), dir.display()),
+ Ok(false) => {}
+ Err(e) => tracing::warn!("could not copy the bundled {} into {}: {e}", d.file_name(), dir.display()),
+ }
+ }
+ }
+
pub fn place_bundled_upscaler(&self) {
let Some(bundle) = crate::installer::bundle_dir() else { return };
let dir = self.config().sdcpp_dir;
@@ -694,9 +743,10 @@ impl Engine {
pub fn submit(&self, mut request: GenerationRequest, workspace: Option<PathBuf>) -> AppResult<JobView> {
self.touch();
let upscale = request.mode == Mode::Upscale;
- // Removing a painted area needs no description (the planner writes the prompt).
+ // Removing a painted area needs no description (the planner writes the prompt), nor does
+ // Expand (it continues the scene).
let removal = request.mask.is_some() && request.inpaint_action == planner::InpaintAction::Remove;
- if request.prompt.trim().is_empty() && !upscale && !removal && request.read_scene.is_none() {
+ if request.prompt.trim().is_empty() && !upscale && !removal && request.outpaint.is_none() && request.read_scene.is_none() {
return Err(AppError::invalid("Describe what to generate or change."));
}
if upscale && request.images.is_empty() {
@@ -1004,10 +1054,13 @@ impl Engine {
}),
pipeline: None,
hidden: false,
+ source_type: None,
};
+ let marked = self.mark(&out, &mut rec, std::slice::from_ref(&input));
+ let out: &image::RgbaImage = &marked;
rec.duration_ms = started.elapsed().as_millis() as u64;
- write_atomic(&file, &encode_output(&out, format, &rec, img.icc_profile.as_deref())?)?;
- if let Ok(j) = imaging::encode_jpeg(&imaging::thumbnail(&out, 512), 85) {
+ write_atomic(&file, &encode_output(out, format, &rec, img.icc_profile.as_deref())?)?;
+ if let Ok(j) = imaging::encode_jpeg(&imaging::thumbnail(out, 512), 85) {
let _ = std::fs::write(store.thumb_path(&id), j);
}
store.insert_generation(&rec)?;
@@ -1026,7 +1079,9 @@ impl Engine {
None => None,
};
let private = private || parent.as_ref().is_some_and(|p| Arc::ptr_eq(self.gen_store(&p.id), &self.private_store));
- let (img, input) = self.resolve_source(image, None, private)?;
+ let (img, flat) = self.resolve_source(image, None, private)?;
+ // Compare's "before" is the photo the lab was opened on, not the saved image itself.
+ let input = from.and_then(|f| self.resolve_source(f, None, private).ok()).map(|(_, r)| r).unwrap_or(flat);
let cfg = self.config();
let id = new_id();
let now = chrono::Local::now();
@@ -1035,7 +1090,7 @@ impl Engine {
std::fs::create_dir_all(&dir).map_err(|e| AppError::io("create the output folder", &dir, e))?;
let file = paths::unique_path(dir.join(format!("{}-layers.png", now.format("%H%M%S"))), &[]);
let out = img.pixels;
- let rec = GenerationRecord {
+ let mut rec = GenerationRecord {
id: id.clone(),
created_at: now_rfc3339(),
mode: "layers".into(),
@@ -1072,9 +1127,12 @@ impl Engine {
request: serde_json::json!({ "mode": "layers", "images": [from.unwrap_or(image)], "layers_image": image, "parent_id": parent.as_ref().map(|p| p.id.clone()), "private": private }),
pipeline: None,
hidden: false,
+ source_type: None,
};
- write_atomic(&file, &encode_output(&out, OutputFormat::Png, &rec, img.icc_profile.as_deref())?)?;
- if let Ok(j) = imaging::encode_jpeg(&imaging::thumbnail(&out, 512), 85) {
+ let marked = self.mark(&out, &mut rec, std::slice::from_ref(&input));
+ let out: &image::RgbaImage = &marked;
+ write_atomic(&file, &encode_output(out, OutputFormat::Png, &rec, img.icc_profile.as_deref())?)?;
+ if let Ok(j) = imaging::encode_jpeg(&imaging::thumbnail(out, 512), 85) {
let _ = std::fs::write(store.thumb_path(&id), j);
}
store.insert_generation(&rec)?;
@@ -1236,11 +1294,13 @@ impl Engine {
let plan2 = plan.clone();
let prompt = request.prompt.clone();
let mask_img: Option<GrayImage> = mask.as_ref().map(|(m, _)| m.clone());
- let opts = pipeline::BackendOptions { server_vae_tile_size: cfg.server_vae_tile_size(), marks };
+ // A prompt Prompt+ wrote and the user kept (reviewed in the box) is a whole instruction.
+ let written = request.enhanced_from.is_some();
+ let opts = pipeline::BackendOptions { server_vae_tile_size: cfg.server_vae_tile_size(), marks, written };
let prepared = tokio::task::spawn_blocking(move || pipeline::prepare(&plan2, &prompt, seed, &images, mask_img.as_ref(), &opts))
.await
.map_err(AppError::internal)??;
- let pipeline::PreparedJob { request: mut sd_req, mut post } = prepared;
+ let pipeline::PreparedJob { request: mut sd_req, mut post, enhancer: view } = prepared;
if let Some(what) = caps.mac_engine_unsupported(&sd_req) {
return Err(AppError::new(ErrorCode::Unsupported, format!("Kokon's Mac engine (experimental) does not do {what} yet."))
.suggest("Turn it off in Settings → Performance to use the standard engine for this."));
@@ -1261,15 +1321,29 @@ impl Engine {
// 4b. Prompt enhancer: rewrites the user's prompt, looking at the references
// exactly as the image model gets them.
if request.enhance && enhancer::applies_to(plan.mode, plan.marks) {
- // Marks: an edit of the marked region (its one reference).
- let as_mode = if plan.marks { ResolvedMode::Edit } else { plan.mode };
- enhanced = self.enhance_prompt(id, &client, &caps, &cfg, as_mode, &request.prompt, &sd_req.ref_images, seed).await?;
- if let Some(text) = &enhanced {
- sd_req.prompt = pipeline::enhanced_prompt(&request.prompt, text);
- // The rewrite understood a transparency request the keywords missed: keep the alpha.
- post.keep_alpha |= pipeline::rewrite_wants_transparency(text);
+ // A painted area or an expansion: Prompt+ writes what goes there, from its own view,
+ // and Kokon's instruction is kept round it.
+ if let Some(v) = &view {
+ let req = match &v.kind {
+ pipeline::EnhancerKind::Area { action, color } => enhancer::area_request(*action, color, &request.prompt, v.image.clone(), seed),
+ pipeline::EnhancerKind::Expand { sides } => enhancer::expand_request(sides, &request.prompt, v.image.clone(), seed),
+ };
+ enhanced = self.enhance_prompt(id, &client, &caps, &cfg, plan.mode, &request.prompt, &[], seed, Some(req), request.enhance_detail).await?;
+ if let Some(text) = &enhanced {
+ sd_req.prompt = v.prompt(&plan, text);
+ }
+ } else {
+ // Marks: an edit of the marked region (its one reference).
+ let as_mode = if plan.marks { ResolvedMode::Edit } else { plan.mode };
+ enhanced = self.enhance_prompt(id, &client, &caps, &cfg, as_mode, &request.prompt, &sd_req.ref_images, seed, None, request.enhance_detail).await?;
+ if let Some(text) = &enhanced {
+ sd_req.prompt = pipeline::enhanced_prompt(&request.prompt, text);
+ // The rewrite understood a transparency request the keywords missed: keep the alpha.
+ post.keep_alpha |= pipeline::rewrite_wants_transparency(text);
+ }
}
}
+ drop(view);
// Prompt+ review: the prompt is the result; the user's next job makes the image.
if request.enhance_only {
if enhanced.is_none() {
@@ -1510,7 +1584,7 @@ impl Engine {
/// new prompt, or None when the job keeps the user's prompt: the engine cannot write text,
/// or the enhancer failed (both noted on the job). Errors only for a cancel or a crash.
#[allow(clippy::too_many_arguments)]
- async fn enhance_prompt(&self, id: &str, client: &SdcppClient, caps: &Capabilities, cfg: &AppConfig, mode: ResolvedMode, prompt: &str, images: &[String], seed: i64) -> AppResult<Option<String>> {
+ async fn enhance_prompt(&self, id: &str, client: &SdcppClient, caps: &Capabilities, cfg: &AppConfig, mode: ResolvedMode, prompt: &str, images: &[String], seed: i64, own: Option<TextGenRequest>, detail: bool) -> AppResult<Option<String>> {
// A retry after running out of memory keeps the first prompt.
if let Some(done) = self.jobs.lock().unwrap().get(id).and_then(|j| j.enhanced.clone()) {
return Ok(Some(done));
@@ -1527,7 +1601,10 @@ impl Engine {
let started = Instant::now();
// A new image from references: each reference is labelled alone first, then the scene is
// written from the labels without the images (see `enhancer::COMPOSE_SYSTEM_PROMPT`).
- let req = if mode == ResolvedMode::Compose {
+ let req = if let Some(r) = own {
+ // A painted area or an expansion: its request is made with its own view.
+ r
+ } else if mode == ResolvedMode::Compose {
let mut labels = Vec::with_capacity(images.len());
for img in images {
let r = client.text_gen(&enhancer::label_request(img.clone(), prompt, seed), Duration::from_secs(120), |_| {}, || self.is_cancelled(id)).await;
@@ -1546,6 +1623,7 @@ impl Engine {
} else {
enhancer::request(cfg, mode, prompt, images.to_vec(), seed)
};
+ let req = if detail { enhancer::detailed(req, mode) } else { req };
let mut text = String::new();
let mut last_emit = Instant::now();
let on_delta = |d: &str| {
@@ -1845,10 +1923,15 @@ impl Engine {
request: serde_json::to_value(req).unwrap_or_default(),
pipeline,
hidden: req.hidden,
+ source_type: None,
};
rec.notes.extend(hdr_note);
+ // The invisible AI watermark goes into the pixels every copy is made from (the file, the
+ // MCP copy, the thumbnail).
+ let marked = self.mark(img, &mut rec, inputs);
+ let img: &image::RgbaImage = &marked;
let bytes = match &hdr {
- Some(gm) => crate::hdr::encode_ultra_hdr(img, icc, gm, 95).map_err(|e| AppError::new(ErrorCode::Internal, "Failed to encode the HDR JPEG.").detail(e))?,
+ Some(gm) => crate::hdr::encode_ultra_hdr(img, icc, gm, 95, &crate::provenance::attributes(&rec)).map_err(|e| AppError::new(ErrorCode::Internal, "Failed to encode the HDR JPEG.").detail(e))?,
None => encode_output(img, format, &rec, icc)?,
};
write_atomic(&file, &bytes)?;
@@ -1934,13 +2017,19 @@ fn numbered(path: &str, i: usize) -> String {
pub fn encode_output(img: &image::RgbaImage, format: OutputFormat, rec: &GenerationRecord, icc: Option<&[u8]>) -> AppResult<Vec<u8>> {
match format {
+ // Each format carries the AI marking (provenance.rs).
OutputFormat::Png => {
let text = rec.png_text();
- let pairs: Vec<(&str, &str)> = text.iter().map(|(k, v)| (*k, v.as_str())).collect();
+ let xmp = crate::provenance::xmp(rec);
+ let mut pairs: Vec<(&str, &str)> = text.iter().map(|(k, v)| (*k, v.as_str())).collect();
+ pairs.push(("XML:com.adobe.xmp", &xmp));
imaging::encode_png(img, &pairs, icc, false)
}
- OutputFormat::Jpeg => imaging::encode_jpeg(img, 95),
- OutputFormat::Webp => imaging::encode_webp_lossless(img),
+ OutputFormat::Jpeg => Ok(crate::provenance::jpeg_with_xmp(&imaging::encode_jpeg(img, 95)?, &crate::provenance::xmp(rec))),
+ OutputFormat::Webp => {
+ let alpha = img.pixels().any(|p| p[3] < 255);
+ Ok(crate::provenance::webp_with_xmp(&imaging::encode_webp_lossless(img)?, &crate::provenance::xmp(rec), img.width(), img.height(), alpha))
+ }
}
}
@@ -2000,7 +2089,7 @@ mod tests {
let share: f64 = note.split("kept on ").nth(1).unwrap().split('%').next().unwrap().parse().unwrap();
assert!((35.0..65.0).contains(&share), "{note}");
let icc = crate::heif::primary_icc(&bytes);
- let out = crate::hdr::encode_ultra_hdr(&edited, icc.as_deref(), &gm, 95).unwrap();
+ let out = crate::hdr::encode_ultra_hdr(&edited, icc.as_deref(), &gm, 95, "").unwrap();
if let Ok(d) = std::env::var("QLS_TEST_OUT") {
std::fs::write(Path::new(&d).join("edit_ultrahdr.jpg"), &out).unwrap();
}
@@ -2204,7 +2293,7 @@ mod tests {
let id = submit();
let mut events = e.events.subscribe();
let (port, body) = fake_server(200, "text/event-stream", sse_answer(&["Change the cat's", " fur to black."], "stop"), Duration::from_millis(150)).await;
- let out = e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Edit, "rendi il gatto nero", &images, 42).await.unwrap();
+ let out = e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Edit, "rendi il gatto nero", &images, 42, None, false).await.unwrap();
assert_eq!(out.as_deref(), Some("Change the cat's fur to black."));
let mut partial = false;
while let Ok(ev) = events.try_recv() {
@@ -2219,31 +2308,31 @@ mod tests {
assert_eq!((sent["images"][0].as_str(), sent["sampling"]["seed"].as_i64()), (Some("PNG1"), Some(42)), "the references and the job's seed");
assert_eq!(sent["system_prompt"], enhancer::EDIT_SYSTEM_PROMPT);
// A retry (out of memory) keeps that prompt without asking the server again.
- let again = e.enhance_prompt(&id, &SdcppClient::new(1), &caps, &cfg, ResolvedMode::Edit, "rendi il gatto nero", &images, 43).await.unwrap();
+ let again = e.enhance_prompt(&id, &SdcppClient::new(1), &caps, &cfg, ResolvedMode::Edit, "rendi il gatto nero", &images, 43, None, false).await.unwrap();
assert_eq!(again, out);
// A failure keeps the user's prompt, with a note.
let id = submit();
let chunks = vec!["event: delta\ndata: {\"text\":\"Chan\"}\n\n".into(), "event: error\ndata: {\"error\":\"out of memory\"}\n\n".into()];
let (port, _) = fake_server(200, "text/event-stream", chunks, Duration::from_millis(1)).await;
- assert_eq!(e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Generate, "un gatto", &[], 1).await.unwrap(), None);
+ assert_eq!(e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Generate, "un gatto", &[], 1, None, false).await.unwrap(), None);
let v = e.job(&id).unwrap();
assert_eq!(v.enhanced_prompt, None);
assert!(v.notes.last().is_some_and(|n| n.contains("prompt enhancer failed") && n.contains("out of memory") && n.ends_with("used as written.")), "{:?}", v.notes);
// An empty answer too.
let id = submit();
let (port, _) = fake_server(200, "text/event-stream", sse_answer(&[" "], "stop"), Duration::ZERO).await;
- assert_eq!(e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Generate, "un gatto", &[], 1).await.unwrap(), None);
+ assert_eq!(e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Generate, "un gatto", &[], 1, None, false).await.unwrap(), None);
assert!(e.job(&id).unwrap().notes.last().unwrap().contains("wrote nothing"));
// A cut-off answer is used, with a note.
let id = submit();
let (port, _) = fake_server(200, "text/event-stream", sse_answer(&["A cat on a"], "length"), Duration::ZERO).await;
- assert_eq!(e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Generate, "un gatto", &[], 1).await.unwrap().as_deref(), Some("A cat on a"));
+ assert_eq!(e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Generate, "un gatto", &[], 1, None, false).await.unwrap().as_deref(), Some("A cat on a"));
assert!(e.job(&id).unwrap().notes.last().unwrap().contains("length limit"));
// An engine without text generation (official builds): note, no request.
let id = submit();
- assert_eq!(e.enhance_prompt(&id, &SdcppClient::new(1), &Capabilities::default(), &cfg, ResolvedMode::Generate, "un gatto", &[], 1).await.unwrap(), None);
+ assert_eq!(e.enhance_prompt(&id, &SdcppClient::new(1), &Capabilities::default(), &cfg, ResolvedMode::Generate, "un gatto", &[], 1, None, false).await.unwrap(), None);
assert!(e.job(&id).unwrap().notes.last().unwrap().contains("no prompt enhancer"));
assert_eq!(e.job(&id).unwrap().status, JobStatus::Queued, "status untouched");
@@ -2255,7 +2344,7 @@ mod tests {
tokio::time::sleep(Duration::from_millis(300)).await;
e2.cancel(&id2).unwrap();
});
- let err = e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Generate, "un gatto", &[], 1).await.unwrap_err();
+ let err = e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Generate, "un gatto", &[], 1, None, false).await.unwrap_err();
assert_eq!(err.code, ErrorCode::Cancelled);
}
@@ -2312,7 +2401,7 @@ mod tests {
let id = e.submit(GenerationRequest { prompt: "lei sul divano con il gatto".into(), enhance: true, ..Default::default() }, None).unwrap().id;
let (port, seen) = fake_text_server(vec!["person, female, long brown hair.\n", "cat, orange tabby", "A photograph of the woman from <image1> on a sofa."]).await;
let images = vec!["PNG1".to_string(), "PNG2".to_string()];
- let out = e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Compose, "lei sul divano con il gatto", &images, 9).await.unwrap();
+ let out = e.enhance_prompt(&id, &SdcppClient::new(port), &caps, &cfg, ResolvedMode::Compose, "lei sul divano con il gatto", &images, 9, None, false).await.unwrap();
assert_eq!(out.as_deref(), Some("A photograph of the woman from <image1> on a sofa."));
let seen = seen.lock().unwrap();
assert_eq!(seen.len(), 3);
@@ -2416,6 +2505,25 @@ mod tests {
assert!(!req.private, "off unless asked");
}
+ #[test]
+ fn saved_results_carry_both_ai_marks() {
+ // Needs the TrustMark models: KOKON_TEST_MODELS is a folder holding models/watermark/.
+ let Ok(models) = std::env::var("KOKON_TEST_MODELS") else { return };
+ let dir = tempfile::tempdir().unwrap();
+ let cfg = AppConfig { output_dir: dir.path().join("out"), sdcpp_dir: PathBuf::from(models), ..Default::default() };
+ let e = Engine::new(cfg, dir.path()).unwrap();
+ let img = image::RgbaImage::from_fn(400, 300, |x, y| image::Rgba([(x / 2) as u8, (y / 2) as u8, 90, 255]));
+ let input = e.store.put_input(&imaging::encode_png(&img, &[], None, false).unwrap(), Some("lab.png"), None).unwrap();
+ let rec = e.save_layers(&format!("input:{}", input.id), None, false).unwrap();
+ let bytes = std::fs::read(&rec.file_path).unwrap();
+ assert_eq!(crate::provenance::declared_ai_type(&bytes), Some("compositeWithTrainedAlgorithmicMedia"));
+ let saved = imaging::decode_image(&bytes).unwrap().pixels;
+ assert!(crate::watermark::detect(&e.config().sdcpp_dir, &saved).unwrap().unwrap().kokon);
+ assert!(rec.notes.iter().all(|n| !n.contains("watermark")), "{:?}", rec.notes);
+ let report = crate::provenance::check(&e.config().sdcpp_dir, &bytes).unwrap();
+ assert_eq!((report["ai"].as_bool(), report["watermark"]["kokon"].as_bool()), (Some(true), Some(true)));
+ }
+
#[test]
fn lut_results_are_new_versions_of_their_source() {
let dir = tempfile::tempdir().unwrap();
@@ -2442,6 +2550,18 @@ mod tests {
let rec = e.apply_lut(&src, &lut.name, 0.5, false).unwrap();
assert_eq!((rec.mode.as_str(), rec.width, rec.height), ("lut", 16, 8));
+ // A LUT of a camera photo is not AI-made (XMP "enhanced", no watermark), nor is a LUT of that.
+ assert!(!crate::provenance::is_ai(&rec) && rec.source_type.is_none());
+ assert_eq!(crate::provenance::declared_ai_type(&std::fs::read(&rec.file_path).unwrap()), None);
+ let twice = e.apply_lut(&format!("gen:{}", rec.id), &lut.name, 0.5, false).unwrap();
+ assert_eq!((twice.source_type.as_deref(), crate::provenance::is_ai(&twice)), (Some("algorithmicallyEnhanced"), false));
+ // An imported image that says it was AI-made stays marked so (the Code's Measure 1.2).
+ let ai_rec = test_record("ai", 1, "ai.jpg");
+ let ai = crate::provenance::jpeg_with_xmp(&imaging::encode_jpeg(&image::RgbaImage::from_pixel(16, 8, image::Rgba([9, 9, 9, 255])), 90).unwrap(), &crate::provenance::xmp(&GenerationRecord { mode: "generate".into(), ..ai_rec }));
+ let ai_input = e.store.put_input(&ai, Some("from-another-app.jpg"), None).unwrap();
+ let kept = e.apply_lut(&format!("input:{}", ai_input.id), &lut.name, 0.5, false).unwrap();
+ assert_eq!(kept.source_type.as_deref(), Some("trainedAlgorithmicMedia"));
+ assert_eq!(crate::provenance::declared_ai_type(&std::fs::read(&kept.file_path).unwrap()), Some("trainedAlgorithmicMedia"));
assert_eq!(rec.request["images"][0], src.as_str(), "made from the source (timeline link)");
assert_eq!(rec.request["lut"]["intensity"], 0.5);
assert!(Path::new(&rec.file_path).starts_with(dir.path().join("out")) && rec.file_path.ends_with(".png"));
@@ -2601,9 +2721,14 @@ mod tests {
let rec = test_record("x", 1, "f");
let png = encode_output(&img, OutputFormat::Png, &rec, None).unwrap();
assert!(imaging::read_png_text(&png).iter().any(|(k, _)| k == "qls:generation"));
+ // The AI marking (provenance.rs) in every format.
+ assert!(imaging::read_png_text(&png).iter().any(|(k, v)| k == "XML:com.adobe.xmp" && v.contains("Iptc4xmpExt:DigitalSourceType")));
let jpg = encode_output(&img, OutputFormat::Jpeg, &rec, None).unwrap();
assert_eq!(&jpg[..2], &[0xFF, 0xD8]);
let webp = encode_output(&img, OutputFormat::Webp, &rec, None).unwrap();
assert_eq!(&webp[..4], b"RIFF");
+ for f in [&jpg, &webp] {
+ assert!(f.windows(17).any(|w| w == b"DigitalSourceType"));
+ }
}
}
diff --git a/crates/qls-core/src/enhancer.rs b/crates/qls-core/src/enhancer.rs
index 854439a..464940d 100644
--- a/crates/qls-core/src/enhancer.rs
+++ b/crates/qls-core/src/enhancer.rs
@@ -6,7 +6,7 @@
//! Design and the A/B behind these texts: docs/engineering-notes.md, "Prompt+".
use crate::config::AppConfig;
-use crate::planner::ResolvedMode;
+use crate::planner::{InpaintAction, ResolvedMode};
use crate::sdcpp::{TextGenRequest, TextSampling};
/// Built-in system prompt for text-to-image (engineering notes, "Prompt+"; decided 2026-10-18, revised
@@ -68,6 +68,63 @@ Fill in the pattern for its kind, replacing each <...> with what you see and lea
- style, <medium>, <colour palette>, <kind of light>, <texture>
Describe the thing alone: never its action, pose, who holds it, what it rests on, its surroundings or background (except as the place or style when that is what the request takes)."#;
+/// Built-in system prompt for a painted area (a brushed edit without coloured marks), added
+/// 2026-11-04. The enhancer sees the untouched region with the area outlined (the image model's own
+/// copy is blurred there for Remove and Replace) and writes the whole instruction for the action;
+/// for Remove also what is seen in the object's place (only its name made Qwen paint it again).
+/// Sent as it is in pipeline::written_inpaint_prompt.
+pub const AREA_SYSTEM_PROMPT: &str = r#"You write the instruction for an image editing model that changes one area of a picture. You see the picture with that area outlined by a bright line; what is inside the line is shown as it is. The model will get the picture with that area blurred out and must redraw it from your words alone. The user's message gives the line's colour, the action, the words your text must start with, and the user's own words, in any language, or none.
+- Start with exactly the given words and write in English, 20 to 60 words in all.
+- Remove: name everything inside the line as one thing (the main object with whatever is on it or in it), then, in a second sentence, describe what is seen there instead: the background that continues from just outside the line, with its surface, material, colours, texture or pattern and light, as if the thing had never been there, with no shadow or mark of it left.
+- Change: say what inside the line changes and what it becomes, and that the rest of it stays as it is.
+- Replace: name what is inside the line and what replaces it, then describe the new thing: what it is, its colours and materials, its size fitting the area and the scene's perspective, light matching the scene and its shadow on the surface under it.
+- Add: describe the new thing put inside the line: what it is, its colours and materials, where it stands, its size fitting the scene's perspective, light matching the scene and its shadow.
+- Keep everything the user asked for and never contradict it; base every detail on what you see. With no words from the user, choose what fits the scene best.
+- Never describe anything outside the line except as what continues into it. Never mention the line, its colour, a mask, a blur or an area.
+Output only the instruction."#;
+
+/// Built-in system prompt for Expand (outpaint), added 2026-11-04: the enhancer sees the canvas
+/// with the new borders still empty and writes what they show; the outpaint instruction
+/// (pipeline::effective_prompt) goes in front of it, as for typed words.
+pub const EXPAND_SYSTEM_PROMPT: &str = r#"You help an image model extend a picture beyond its borders. You see the picture on a larger canvas whose new border areas are still empty and flat. The user's message says which sides grow and gives the user's own wishes, in any language, or none.
+- Write one English paragraph of 30 to 90 words describing what the new areas show so that the whole picture continues naturally past each growing side: what continues there and how it ends at the new edge.
+- Keep the same medium, style, light, colours and perspective as the picture. Keep everything the user asked for and put it in the new areas.
+- Describe only what the new areas add. Never add new people, animals or main subjects unless the user asks.
+- Never mention borders, empty areas, a canvas or extending.
+Output only the description."#;
+
+/// The area and Expand texts are short (at most ~90 words).
+pub const AREA_MAX_TOKENS: u32 = 256;
+
+/// Prompt+ "Detailed" (2026-11-04, Gabriele: "give more context to the generator"): added after
+/// any system prompt, built-in or the user's, it replaces the length asked there. More context
+/// only about what is in the picture or what changes, never what is absent or untouched (a list of
+/// what stays makes Qwen redraw it; a "no X" puts X in). Not A/B-tested yet.
+pub const DETAIL_SCENE: &str = "\n\nDetailed mode: this replaces the length given above. Write 200 to 350 words, up to about 450 when there are several people or many objects. Give the image generator more context about everything that is in the picture: each subject's materials, textures, small details and expression, the setting and the objects in the background, the camera (distance, angle, lens, depth of field), the light and how it falls on each surface, and the colours. Describe only what is there; never write what is absent or what to avoid.";
+pub const DETAIL_EDIT: &str = "\n\nDetailed mode: this replaces the length given above. Write 60 to 150 words, up to about 250 for several changes. Give the image editor more context about the change only: the new things' materials, textures and colours, their exact place and size, how the light of the scene falls on them, their shadows and reflections, and how they meet what is around them. Never describe the parts that stay, and never write what must not be added.";
+pub const DETAIL_AREA: &str = "\n\nDetailed mode: this replaces the length given above. Write 50 to 120 words, with more context about what is seen inside the line after the edit: materials, textures, colours, light, shadows and how it meets what is around it. Never write what must not be added.";
+pub const DETAIL_EXPAND: &str = "\n\nDetailed mode: this replaces the length given above. Write 80 to 180 words, with more context about what the new areas show: materials, textures, colours, the light and how far things are. Never write what is absent.";
+
+/// The longest a Detailed text runs (~450 words; ~1.4 tokens a word).
+const DETAIL_MAX_TOKENS: u32 = 1024;
+
+/// A Prompt+ request made "Detailed": its system prompt ends with the length and context of the
+/// kind it is (a new image, an edit, a painted area, an expansion), and longer texts are allowed.
+pub fn detailed(mut req: TextGenRequest, mode: ResolvedMode) -> TextGenRequest {
+ let add = if req.system_prompt == AREA_SYSTEM_PROMPT {
+ DETAIL_AREA
+ } else if req.system_prompt == EXPAND_SYSTEM_PROMPT {
+ DETAIL_EXPAND
+ } else if matches!(mode, ResolvedMode::Generate | ResolvedMode::Compose) {
+ DETAIL_SCENE
+ } else {
+ DETAIL_EDIT
+ };
+ req.system_prompt.push_str(add);
+ req.max_tokens = req.max_tokens.max(DETAIL_MAX_TOKENS);
+ req
+}
+
/// Longest label kept (a model that rambles is cut, the label is only a hint).
pub const LABEL_MAX_TOKENS: u32 = 96;
@@ -88,10 +145,49 @@ pub fn applies(mode: ResolvedMode) -> bool {
matches!(mode, ResolvedMode::Generate | ResolvedMode::Compose | ResolvedMode::Edit | ResolvedMode::MultiReference)
}
-/// [`applies`], and inpaints with coloured marks: written as an edit of the marked region (the
-/// edit prompt knows marks named by colour).
-pub fn applies_to(mode: ResolvedMode, marks: bool) -> bool {
- applies(mode) || (marks && mode == ResolvedMode::Inpaint)
+/// [`applies`], and inpaints and expansions: coloured marks written as an edit of the marked
+/// region (the edit prompt knows marks named by colour), a painted area and an expansion with
+/// their own prompts ([`area_request`], [`expand_request`]) inside Kokon's instructions.
+pub fn applies_to(mode: ResolvedMode, _marks: bool) -> bool {
+ applies(mode) || matches!(mode, ResolvedMode::Inpaint | ResolvedMode::Outpaint)
+}
+
+/// What the user's words were, for the area and Expand prompts.
+fn words(prompt: &str) -> String {
+ let p = prompt.trim();
+ if p.is_empty() { "(none)".into() } else { p.to_string() }
+}
+
+/// A painted area (no marks): `image` is the untouched region with the area outlined in `color`.
+pub fn area_request(action: InpaintAction, color: &str, prompt: &str, image: String, seed: i64) -> TextGenRequest {
+ let (act, start) = match action {
+ InpaintAction::Remove => ("remove", "Remove"),
+ InpaintAction::Edit => ("change", "Change"),
+ InpaintAction::Replace => ("replace", "Replace"),
+ InpaintAction::Add => ("add", "Add"),
+ };
+ // The app puts the action's words in front of what the user types: only the rest is theirs.
+ let typed = if action == InpaintAction::Edit { prompt.trim().strip_prefix("Change").map(str::trim).unwrap_or(prompt.trim()).to_string() } else { action.subject(prompt) };
+ TextGenRequest {
+ prompt: format!("Line colour: {color}\nAction: {act}\nStart with: {start}\nThe user's words: {}", words(&typed)),
+ system_prompt: AREA_SYSTEM_PROMPT.into(),
+ images: vec![image],
+ max_tokens: AREA_MAX_TOKENS,
+ sampling: TextSampling { seed, ..SAMPLING },
+ stream: true,
+ }
+}
+
+/// An expansion: `image` is the canvas with the new areas empty; `sides` names them ("left and top").
+pub fn expand_request(sides: &str, prompt: &str, image: String, seed: i64) -> TextGenRequest {
+ TextGenRequest {
+ prompt: format!("Growing sides: {sides}\nThe user's wishes: {}", words(prompt)),
+ system_prompt: EXPAND_SYSTEM_PROMPT.into(),
+ images: vec![image],
+ max_tokens: AREA_MAX_TOKENS,
+ sampling: TextSampling { seed, ..SAMPLING },
+ stream: true,
+ }
}
/// The system prompt for `mode`: the one saved in Settings, or the built-in one.
@@ -154,7 +250,7 @@ pub fn compose_request(cfg: &AppConfig, labels: &[String], prompt: &str, seed: i
/// whole text. None when nothing usable is left.
pub fn clean(text: &str) -> Option<String> {
let mut t = text.trim();
- for label in ["Prompt:", "Instruction:", "Editing instruction:"] {
+ for label in ["Prompt:", "Instruction:", "Editing instruction:", "Description:"] {
if t.get(..label.len()).is_some_and(|head| head.eq_ignore_ascii_case(label)) {
t = t[label.len()..].trim_start();
}
@@ -174,6 +270,8 @@ mod tests {
fn modes_and_prompts() {
assert!(applies(ResolvedMode::Generate) && applies(ResolvedMode::Edit) && applies(ResolvedMode::MultiReference) && applies(ResolvedMode::Compose));
assert!(!applies(ResolvedMode::Inpaint) && !applies(ResolvedMode::Outpaint) && !applies(ResolvedMode::Upscale));
+ // A painted area and Expand have prompts of their own; an upscale has no prompt.
+ assert!(applies_to(ResolvedMode::Inpaint, false) && applies_to(ResolvedMode::Outpaint, false) && !applies_to(ResolvedMode::Upscale, false));
let mut cfg = AppConfig::default();
assert_eq!(system_prompt(&cfg, ResolvedMode::Generate), T2I_SYSTEM_PROMPT);
assert_eq!(system_prompt(&cfg, ResolvedMode::Edit), EDIT_SYSTEM_PROMPT);
@@ -193,7 +291,7 @@ mod tests {
fn builtin_prompts_have_nothing_the_model_copies() {
// Seen in real outputs (bench sysprompt-v2-20261019-155557): example colours ended up in
// almost every prompt, and the text rule became "No text appears in the image."
- for p in [T2I_SYSTEM_PROMPT, EDIT_SYSTEM_PROMPT, COMPOSE_SYSTEM_PROMPT, COMPOSE_LABEL_PROMPT] {
+ for p in [T2I_SYSTEM_PROMPT, EDIT_SYSTEM_PROMPT, COMPOSE_SYSTEM_PROMPT, COMPOSE_LABEL_PROMPT, AREA_SYSTEM_PROMPT, EXPAND_SYSTEM_PROMPT] {
for leak in ["deep navy", "pale cream", "copper", "No text"] {
assert!(!p.contains(leak), "{leak:?} in a built-in prompt");
}
@@ -223,6 +321,41 @@ mod tests {
assert_eq!(clean_label(" cat,\n orange tabby.\n"), "cat, orange tabby");
}
+ #[test]
+ fn detailed_asks_for_more_context_of_its_kind() {
+ let cfg = AppConfig::default();
+ let t = detailed(request(&cfg, ResolvedMode::Generate, "un gatto", vec![], 1), ResolvedMode::Generate);
+ assert!(t.system_prompt.starts_with(T2I_SYSTEM_PROMPT) && t.system_prompt.ends_with(DETAIL_SCENE));
+ let e = detailed(request(&cfg, ResolvedMode::Edit, "x", vec!["P".into()], 1), ResolvedMode::Edit);
+ assert!(e.system_prompt.ends_with(DETAIL_EDIT));
+ let a = detailed(area_request(InpaintAction::Remove, "cyan", "", "P".into(), 1), ResolvedMode::Inpaint);
+ assert!(a.system_prompt.ends_with(DETAIL_AREA) && a.max_tokens == DETAIL_MAX_TOKENS);
+ let x = detailed(expand_request("left", "", "P".into(), 1), ResolvedMode::Outpaint);
+ assert!(x.system_prompt.ends_with(DETAIL_EXPAND));
+ // Only what is there: no "keep" lists, no "no X".
+ for d in [DETAIL_SCENE, DETAIL_EDIT, DETAIL_AREA, DETAIL_EXPAND] {
+ assert!(d.contains("replaces the length") && !d.contains("unchanged"));
+ }
+ assert!(450.0 * 1.4 < DETAIL_MAX_TOKENS as f64);
+ }
+
+ #[test]
+ fn area_and_expand_requests() {
+ let r = area_request(InpaintAction::Remove, "magenta", " ", "PNG".into(), 7);
+ assert_eq!(r.prompt, "Line colour: magenta\nAction: remove\nStart with: Remove\nThe user's words: (none)");
+ // The action's words the app puts in front are not the user's.
+ assert!(area_request(InpaintAction::Remove, "magenta", "Remove ", "PNG".into(), 7).prompt.ends_with("(none)"));
+ assert!(area_request(InpaintAction::Remove, "magenta", "Remove the lamp", "PNG".into(), 7).prompt.ends_with("words: the lamp"));
+ assert_eq!((r.images.len(), r.system_prompt.as_str(), r.sampling.seed, r.max_tokens), (1, AREA_SYSTEM_PROMPT, 7, AREA_MAX_TOKENS));
+ let r = area_request(InpaintAction::Add, "cyan", " un vaso di fiori ", "PNG".into(), 1);
+ assert!(r.prompt.ends_with("Action: add\nStart with: Add\nThe user's words: un vaso di fiori"));
+ let r = expand_request("left and right", "", "PNG".into(), 3);
+ assert_eq!(r.prompt, "Growing sides: left and right\nThe user's wishes: (none)");
+ assert!(r.system_prompt == EXPAND_SYSTEM_PROMPT && r.images.len() == 1);
+ // The instruction around it is Kokon's: the text is only what goes in the area.
+ assert!(AREA_SYSTEM_PROMPT.contains("Never mention the line") && EXPAND_SYSTEM_PROMPT.contains("Never mention borders"));
+ }
+
#[test]
fn compose_prompt_makes_a_new_picture() {
// "Keep everything else unchanged" turns a composition into an edit of image 1.
diff --git a/crates/qls-core/src/examples.rs b/crates/qls-core/src/examples.rs
index cc17db4..fcb482b 100644
--- a/crates/qls-core/src/examples.rs
+++ b/crates/qls-core/src/examples.rs
@@ -111,7 +111,6 @@ impl Engine {
continue;
}
let file = dir.join(format!("{}.jpg", s.name));
- std::fs::write(&file, s.image).map_err(|e| AppError::io("store the example", &file, e))?;
let img = imaging::decode_image(s.image)?.pixels;
// The previous version as this one's reference, the way an edit records it.
let reference: Option<InputRecord> = match &prev {
@@ -137,7 +136,7 @@ impl Engine {
..Default::default()
};
let mode = serde_json::to_value(s.mode).ok().and_then(|v| v.as_str().map(str::to_owned)).unwrap_or_default();
- let rec = GenerationRecord {
+ let mut rec = GenerationRecord {
id: id.clone(),
// Older than anything the user makes, in project order.
created_at: format!("2026-10-10T09:{:02}:{:02}.000Z", pi * 5, si * 10),
@@ -177,7 +176,12 @@ impl Engine {
request: serde_json::to_value(&request).unwrap_or_default(),
pipeline: None,
hidden: false,
+ source_type: None,
};
+ // Model-made like any result: the XMP and the invisible watermark (provenance.rs).
+ let img = self.mark(&img, &mut rec, &[]).into_owned();
+ let bytes = crate::provenance::jpeg_with_xmp(&imaging::encode_jpeg(&img, 92)?, &crate::provenance::xmp(&rec));
+ std::fs::write(&file, bytes).map_err(|e| AppError::io("store the example", &file, e))?;
if let Ok(j) = imaging::encode_jpeg(&imaging::thumbnail(&img, 512), 85) {
let _ = std::fs::write(store.thumb_path(&id), j);
}
diff --git a/crates/qls-core/src/hdr.rs b/crates/qls-core/src/hdr.rs
index 26a4a64..e43ca4c 100644
--- a/crates/qls-core/src/hdr.rs
+++ b/crates/qls-core/src/hdr.rs
@@ -146,7 +146,8 @@ const XMP_NS: &[u8] = b"http://ns.adobe.com/xap/1.0/\0";
const ISO_NS: &[u8] = b"urn:iso:std:iso:ts:21496:-1\0";
/// Encodes an Ultra HDR JPEG: `sdr` as the primary image (with `icc`), `gm` as the gain map.
-pub fn encode_ultra_hdr(sdr: &RgbaImage, icc: Option<&[u8]>, gm: &GainMap, quality: u8) -> Result<Vec<u8>, String> {
+/// `provenance`: more attributes for the primary's XMP description (Kokon's AI marking, provenance.rs).
+pub fn encode_ultra_hdr(sdr: &RgbaImage, icc: Option<&[u8]>, gm: &GainMap, quality: u8, provenance: &str) -> Result<Vec<u8>, String> {
let primary = jpeg(&DynamicImage::ImageRgba8(sdr.clone()).into_rgb8(), icc, quality)?;
let gain = jpeg(&gm.image, None, quality.min(90))?;
@@ -155,7 +156,7 @@ pub fn encode_ultra_hdr(sdr: &RgbaImage, icc: Option<&[u8]>, gm: &GainMap, quali
let gain_file = insert_after_soi(&gain, &gain_segments);
// Primary: XMP directory + ISO version marker + MPF index, then the encoder's own segments.
- let mut head = app(1, &[XMP_NS, primary_xmp(gain_file.len()).as_bytes()].concat());
+ let mut head = app(1, &[XMP_NS, primary_xmp(gain_file.len(), provenance).as_bytes()].concat());
head.extend(app(2, &[ISO_NS, &[0, 0, 0, 0]].concat()));
let mpf_at = 2 + head.len() + 4 + 4; // SOI + segments + APP2 marker/length + "MPF\0"
let mpf_len = app(2, &mpf(0, 0, 0)).len();
@@ -216,9 +217,9 @@ fn mpf(primary_len: u32, gain_len: u32, gain_offset: u32) -> Vec<u8> {
v
}
-fn primary_xmp(gain_len: usize) -> String {
+fn primary_xmp(gain_len: usize, extra: &str) -> String {
format!(
- r#"<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Kokon"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description xmlns:Container="http://ns.google.com/photos/1.0/container/" xmlns:Item="http://ns.google.com/photos/1.0/container/item/" xmlns:hdrgm="http://ns.adobe.com/hdr-gain-map/1.0/" hdrgm:Version="1.0"><Container:Directory><rdf:Seq><rdf:li rdf:parseType="Resource"><Container:Item Item:Semantic="Primary" Item:Mime="image/jpeg"/></rdf:li><rdf:li rdf:parseType="Resource"><Container:Item Item:Semantic="GainMap" Item:Mime="image/jpeg" Item:Length="{gain_len}"/></rdf:li></rdf:Seq></Container:Directory></rdf:Description></rdf:RDF></x:xmpmeta>"#
+ r#"<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Kokon"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description xmlns:Container="http://ns.google.com/photos/1.0/container/" xmlns:Item="http://ns.google.com/photos/1.0/container/item/" xmlns:hdrgm="http://ns.adobe.com/hdr-gain-map/1.0/" hdrgm:Version="1.0" {extra}><Container:Directory><rdf:Seq><rdf:li rdf:parseType="Resource"><Container:Item Item:Semantic="Primary" Item:Mime="image/jpeg"/></rdf:li><rdf:li rdf:parseType="Resource"><Container:Item Item:Semantic="GainMap" Item:Mime="image/jpeg" Item:Length="{gain_len}"/></rdf:li></rdf:Seq></Container:Directory></rdf:Description></rdf:RDF></x:xmpmeta>"#
)
}
@@ -313,7 +314,7 @@ mod tests {
let sdr = RgbaImage::from_pixel(64, 32, image::Rgba([200, 30, 30, 255]));
let gm = GainMap { image: GrayImage::from_pixel(32, 16, Luma([128])), metadata: meta() };
let icc = vec![7u8; 600];
- let out = encode_ultra_hdr(&sdr, Some(&icc), &gm, 90).unwrap();
+ let out = encode_ultra_hdr(&sdr, Some(&icc), &gm, 90, "").unwrap();
// Find the MPF segment and check that its second entry points at a JPEG with gain map XMP.
let pos = out.windows(4).position(|w| w == b"MPF\0").unwrap();
@@ -351,7 +352,7 @@ mod tests {
let base = crate::wic::decode(&bytes).unwrap().pixels;
println!("base {:?} gain map {:?}", base.dimensions(), gm.image.dimensions());
assert!(same_framing(base.dimensions(), gm.image.dimensions()), "gain map oriented like the base");
- let out = encode_ultra_hdr(&base, crate::heif::primary_icc(&bytes).as_deref(), &adapt_gain_map(&gm, &base, &base).0, 92).unwrap();
+ let out = encode_ultra_hdr(&base, crate::heif::primary_icc(&bytes).as_deref(), &adapt_gain_map(&gm, &base, &base).0, 92, "").unwrap();
if let Ok(dir) = std::env::var("QLS_TEST_OUT") {
std::fs::write(Path::new(&dir).join("ultrahdr_test.jpg"), &out).unwrap();
}
diff --git a/crates/qls-core/src/imaging.rs b/crates/qls-core/src/imaging.rs
index f22f7d3..af68cd9 100644
--- a/crates/qls-core/src/imaging.rs
+++ b/crates/qls-core/src/imaging.rs
@@ -691,6 +691,23 @@ pub fn pick_highlight(img: &RgbaImage, mask: &GrayImage) -> (&'static str, [u8;
HIGHLIGHTS[k]
}
+/// `img` with the area where `mask` > 127 only outlined in `color` (a line just outside it), what
+/// is inside left as it is: for Prompt+ to see and name it. A tint over it made the 8B read the
+/// tint as a thing ("a red cloth covering the bowl", 2026-11-04).
+pub fn outline(img: &RgbaImage, mask: &GrayImage, color: [u8; 3]) -> RgbaImage {
+ assert_eq!(img.dimensions(), mask.dimensions());
+ let inside = binarize(mask, 127);
+ let ring = dilate(&inside, (img.width().max(img.height()) / 160).max(3));
+ RgbaImage::from_fn(img.width(), img.height(), |x, y| {
+ let p = img.get_pixel(x, y);
+ if inside.get_pixel(x, y)[0] == 0 && ring.get_pixel(x, y)[0] > 0 {
+ Rgba([color[0], color[1], color[2], 255])
+ } else {
+ Rgba([p[0], p[1], p[2], 255])
+ }
+ })
+}
+
/// Guide image: `img` with the area where `mask` > 127 tinted with `color` and outlined,
/// so a prompt can refer to "the area marked in red". Fully opaque.
pub fn highlight(img: &RgbaImage, mask: &GrayImage, color: [u8; 3]) -> RgbaImage {
diff --git a/crates/qls-core/src/lib.rs b/crates/qls-core/src/lib.rs
index 38bd16b..1515667 100644
--- a/crates/qls-core/src/lib.rs
+++ b/crates/qls-core/src/lib.rs
@@ -16,14 +16,17 @@ pub mod lut;
pub mod matting;
pub mod mac_bench;
pub mod modelfile;
+pub mod onnx;
pub mod paths;
pub mod pipeline;
pub mod planner;
pub mod process;
pub mod projects;
+pub mod provenance;
pub mod remote;
pub mod scene;
pub mod sdcpp;
pub mod speed;
pub mod store;
+pub mod watermark;
pub mod wic;
diff --git a/crates/qls-core/src/matting.rs b/crates/qls-core/src/matting.rs
index f98c283..54a6c45 100644
--- a/crates/qls-core/src/matting.rs
+++ b/crates/qls-core/src/matting.rs
@@ -3,7 +3,7 @@
//! sticker with the user's selection (`src/lib/matte.ts`: the selection decides what, the matte the
//! edge). Spike s17 (2026-11-02): cleaner edges than SAM, deterministic, ~4 s per object on a
//! 16-thread CPU, no VRAM taken from the image engine. The webview could not run it (WebGPU kernels
-//! rejected, WASM out of memory), hence here.
+//! rejected, WASM out of memory), hence here. Also the MobileSAM files the lab's Select uses.
use std::path::{Path, PathBuf};
use std::sync::Mutex;
@@ -31,6 +31,49 @@ pub const BIREFNET_LITE: Download = Download {
engine_tag: None,
};
+/// Click-to-select in the layers lab: MobileSAM (Apache-2.0) in Acly's ONNX export (MIT), pinned.
+/// It runs in the webview (src/lib/sam.ts, onnxruntime-web); kokon-service downloads the two files
+/// the first time the lab is opened and serves them to it (`/segment/models/<file>`), since 0.12
+/// (before: the dev server only, so a release had no Select).
+pub const MOBILE_SAM: [Download; 2] = [
+ Download {
+ id: "mobile_sam_encoder",
+ label: "MobileSAM image encoder (click to select)",
+ kind: FileKind::Model,
+ url: "https://huggingface.co/Acly/MobileSAM/resolve/0d3b403339b4674a82493d5e97964dd78089ddc8/mobile_sam_image_encoder.onnx",
+ fallback_urls: &[],
+ page: "https://huggingface.co/Acly/MobileSAM",
+ size: 28_157_093,
+ sha256: "580f5fb648ea1062c0aabc26217aed56921985f03f0cbbd852bba81d760cc749",
+ dest: "models/segment/mobile_sam_image_encoder.onnx",
+ provides: &[],
+ engine_tag: None,
+ },
+ Download {
+ id: "mobile_sam_decoder",
+ label: "MobileSAM mask decoder (click to select)",
+ kind: FileKind::Model,
+ url: "https://huggingface.co/Acly/MobileSAM/resolve/0d3b403339b4674a82493d5e97964dd78089ddc8/sam_mask_decoder_multi.onnx",
+ fallback_urls: &[],
+ page: "https://huggingface.co/Acly/MobileSAM",
+ size: 16_496_559,
+ sha256: "8976b90a87ba50a6a72217a5ff994f7d25ce16f2229fcc1ed259e1294c622ffe",
+ dest: "models/segment/sam_mask_decoder_multi.onnx",
+ provides: &[],
+ engine_tag: None,
+ },
+];
+
+/// A MobileSAM file by its name (the last part of its `dest`), for the webview; None for any other name.
+pub fn sam_file(sdcpp_dir: &Path, name: &str) -> Option<PathBuf> {
+ MOBILE_SAM.iter().find(|d| d.dest.rsplit('/').next() == Some(name)).map(|d| join_rel(sdcpp_dir, d.dest))
+}
+
+/// Both MobileSAM files are there.
+pub fn sam_installed(sdcpp_dir: &Path) -> bool {
+ MOBILE_SAM.iter().all(|d| join_rel(sdcpp_dir, d.dest).is_file())
+}
+
/// The model's input side (the export is fixed at 1024²).
const SIDE: u32 = 1024;
const MEAN: [f32; 3] = [0.485, 0.456, 0.406];
@@ -80,7 +123,7 @@ pub fn matte(sdcpp_dir: &Path, img: &RgbaImage) -> AppResult<GrayImage> {
let mut guard = SESSION.lock().unwrap_or_else(|p| p.into_inner());
if guard.as_ref().is_none_or(|(p, _, _)| p != &path) {
let threads = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(4);
- let env = ort::init().with_name("kokon-stickers").build().map_err(ort_err)?;
+ let env = crate::onnx::environment().map_err(ort_err)?;
let s = Session::builder(&env)
.map_err(ort_err)?
.with_optimization_level(GraphOptimizationLevel::Level3)
@@ -105,6 +148,17 @@ pub fn matte(sdcpp_dir: &Path, img: &RgbaImage) -> AppResult<GrayImage> {
mod tests {
use super::*;
+ #[test]
+ fn only_the_sam_files_are_served() {
+ let d = Path::new("/kokon");
+ assert_eq!(sam_file(d, "sam_mask_decoder_multi.onnx"), Some(join_rel(d, "models/segment/sam_mask_decoder_multi.onnx")));
+ assert!(sam_file(d, "mobile_sam_image_encoder.onnx").is_some());
+ for bad in ["birefnet_lite.onnx", "../models/segment/sam_mask_decoder_multi.onnx", "", "segment"] {
+ assert_eq!(sam_file(d, bad), None, "{bad}");
+ }
+ assert!(!sam_installed(d));
+ }
+
#[test]
fn input_is_normalised_chw() {
let img = RgbaImage::from_pixel(8, 8, image::Rgba([255, 0, 128, 255]));
@@ -157,3 +211,21 @@ mod real_model {
}
}
}
+
+#[cfg(test)]
+mod shared_runtime_tests {
+ /// With the real models (KOKON_TEST_SDCPP: a folder holding models/matting/birefnet_lite.onnx and
+ /// models/watermark/*.onnx): a result is watermarked, then a fast sticker is matted, in one
+ /// process. Before crate::onnx, the second to start failed ("only one environment is allowed
+ /// per process"): every quick cut-out after a result (Windows), no watermark after a sticker (Mac).
+ #[test]
+ fn watermark_then_sticker_in_one_process() {
+ let Ok(dir) = std::env::var("KOKON_TEST_SDCPP") else { return };
+ let dir = std::path::PathBuf::from(dir);
+ let img = image::RgbaImage::from_fn(400, 300, |x, y| image::Rgba([(x / 2) as u8, (y / 2) as u8, if (150..250).contains(&x) { 230 } else { 40 }, 255]));
+ let marked = crate::watermark::embed(&dir, &img).unwrap().unwrap();
+ let m = super::matte(&dir, &marked).unwrap();
+ assert_eq!(m.dimensions(), (400, 300));
+ assert!(crate::watermark::detect(&dir, &marked).unwrap().unwrap().kokon);
+ }
+}
diff --git a/crates/qls-core/src/onnx.rs b/crates/qls-core/src/onnx.rs
new file mode 100644
index 0000000..7d90933
--- /dev/null
+++ b/crates/qls-core/src/onnx.rs
@@ -0,0 +1,26 @@
+//! The process's one ONNX Runtime environment. `ort` allows a single environment per process: the
+//! fast stickers (matting.rs) and the AI watermark (watermark/) each built their own, so whichever
+//! ran second failed ("only one environment is allowed per process"; seen on the Mac 2026-11-04: a
+//! sticker first, then every result unwatermarked). Both take it from here.
+
+use std::sync::OnceLock;
+
+use ort::environment::Environment;
+
+static ENV: OnceLock<Result<Environment, String>> = OnceLock::new();
+
+/// The shared environment, built on first use (a cheap handle: clones share it).
+pub fn environment() -> Result<Environment, String> {
+ ENV.get_or_init(|| ort::init().with_name("kokon").build().map_err(|e| e.to_string())).clone()
+}
+
+#[cfg(test)]
+mod tests {
+ #[test]
+ fn one_environment_for_everyone() {
+ let a = super::environment().unwrap();
+ let b = super::environment().unwrap();
+ drop((a, b));
+ assert!(super::environment().is_ok());
+ }
+}
diff --git a/crates/qls-core/src/pipeline.rs b/crates/qls-core/src/pipeline.rs
index bf0893e..e79bad4 100644
--- a/crates/qls-core/src/pipeline.rs
+++ b/crates/qls-core/src/pipeline.rs
@@ -19,11 +19,14 @@ pub struct BackendOptions {
pub server_vae_tile_size: u32,
/// An inpaint's coloured marks (`GenerationRequest::marks`, any size: stretched over the image).
pub marks: Option<RgbaImage>,
+ /// The prompt was written by Prompt+ (reviewed in the box: `enhanced_from`): for a painted
+ /// area it is a whole instruction, sent in [`written_inpaint_prompt`].
+ pub written: bool,
}
impl Default for BackendOptions {
fn default() -> Self {
- Self { server_vae_tile_size: 0, marks: None }
+ Self { server_vae_tile_size: 0, marks: None, written: false }
}
}
@@ -49,6 +52,52 @@ pub fn vae_tiling_params(tiling: bool, server_tile: u32, job_tile: Option<(u32,
pub struct PreparedJob {
pub request: ImgGenRequest,
pub post: PostProcess,
+ /// What Prompt+ looks at for a painted area or an expansion (other modes: the references).
+ pub enhancer: Option<EnhancerView>,
+}
+
+/// Prompt+ for a painted area or an expansion (enhancer::area_request, expand_request): the
+/// picture it sees, and how its text goes back into Kokon's instruction ([`EnhancerView::prompt`]).
+pub struct EnhancerView {
+ /// PNG, base64, as the references.
+ pub image: String,
+ pub kind: EnhancerKind,
+}
+
+pub enum EnhancerKind {
+ /// The untouched region with the area highlighted in `color` (named in the model's instruction).
+ Area { action: InpaintAction, color: String },
+ /// The canvas with the new areas empty; `sides`: "left and top".
+ Expand { sides: String },
+}
+
+impl EnhancerView {
+ /// The image model's prompt with Prompt+'s text in place of the user's words.
+ pub fn prompt(&self, plan: &Plan, text: &str) -> String {
+ match &self.kind {
+ EnhancerKind::Area { action, color } => written_inpaint_prompt(*action, text, color),
+ EnhancerKind::Expand { .. } => effective_prompt(plan, text),
+ }
+ }
+}
+
+/// The sides an expansion grows, in words ("left, right and bottom").
+pub fn growing_sides(e: &crate::geometry::Expansion) -> String {
+ let sides: Vec<&str> = [("left", e.left), ("right", e.right), ("top", e.top), ("bottom", e.bottom)].iter().filter(|(_, px)| *px > 0).map(|(n, _)| *n).collect();
+ match sides.as_slice() {
+ [] => "none".into(),
+ [one] => one.to_string(),
+ [rest @ .., last] => format!("{} and {last}", rest.join(", ")),
+ }
+}
+
+/// `img` no larger than about 1 MP, for Prompt+ to look at (as the edit references are).
+fn for_enhancer(img: &RgbaImage) -> RgbaImage {
+ let k = (1_048_576.0 / (img.width() as f64 * img.height() as f64)).sqrt();
+ if k >= 1.0 {
+ return img.clone();
+ }
+ imaging::resize(img, ((img.width() as f64 * k).round() as u32).max(1), ((img.height() as f64 * k).round() as u32).max(1))
}
pub enum PostKind {
@@ -158,6 +207,22 @@ pub fn inpaint_prompt(action: InpaintAction, prompt: &str, color: &str) -> Strin
}
}
+/// Inpaint instruction around a prompt Prompt+ wrote for the area (enhancer::AREA_SYSTEM_PROMPT):
+/// its text is already the whole instruction for the action (for Remove: what goes and what is
+/// seen there instead), so it is sent as it is, in the same frame as [`inpaint_prompt`]. Putting
+/// only the object's name in "blurred out to remove …" made Qwen paint it again (2026-11-04,
+/// a basket of fruit came back on both tries).
+pub fn written_inpaint_prompt(action: InpaintAction, text: &str, color: &str) -> String {
+ let area = format!("the area marked in {color} in <image2>");
+ let keep = "Keep everything outside the marked area exactly as it is.";
+ let text = text.trim();
+ match action {
+ InpaintAction::Remove | InpaintAction::Replace => format!("In <image1>, {area} has been blurred out. {text} {keep}"),
+ InpaintAction::Edit => format!("In <image1>, change only {area}. {text} Blend the change seamlessly into the surrounding scene. {keep}"),
+ InpaintAction::Add => format!("In <image1>, inside {area}: {text} {keep}"),
+ }
+}
+
/// `top` drawn over `base` by its alpha (the same size).
fn draw_over(base: &RgbaImage, top: &RgbaImage) -> RgbaImage {
let mut out = base.clone();
@@ -245,6 +310,7 @@ pub fn prepare(plan: &Plan, prompt: &str, seed: i64, images: &[LoadedImage], mas
embed_image_metadata: false,
};
let mut kind = PostKind::Plain;
+ let mut enhancer = None;
let keep_alpha = wants_transparency(prompt) || images.iter().any(|i| i.has_transparency);
let need_images = |n: usize| -> AppResult<()> {
if images.len() < n {
@@ -335,6 +401,10 @@ pub fn prepare(plan: &Plan, prompt: &str, seed: i64, images: &[LoadedImage], mas
let (color_name, color) = imaging::pick_highlight(&content, &m_content);
let guide_mask = imaging::resize_mask(&m_work, gc.content.width, gc.content.height);
let guide = imaging::highlight(&imaging::resize(&image1, gc.content.width, gc.content.height), &guide_mask, color);
+ // Prompt+ sees the region as it is, the area only outlined: image 1 is blurred there
+ // for Remove and Replace, and a tint over it was read as a thing of its own.
+ let view = imaging::outline(&imaging::resize(&content, gc.content.width, gc.content.height), &guide_mask, color);
+ enhancer = Some(EnhancerView { image: png_b64(&for_enhancer(&view))?, kind: EnhancerKind::Area { action, color: color_name.to_string() } });
Some((imaging::pad_edge(&guide, gc.canvas, gc.content_offset), color_name))
};
drop(content);
@@ -352,6 +422,7 @@ pub fn prepare(plan: &Plan, prompt: &str, seed: i64, images: &[LoadedImage], mas
req.ref_image_args = Some("resize_before_vae=false".into());
}
req.prompt = match &guide {
+ Some((_, color_name)) if opts.written => written_inpaint_prompt(action, prompt, color_name),
Some((_, color_name)) => inpaint_prompt(action, prompt, color_name),
None => effective_prompt(plan, prompt),
};
@@ -373,6 +444,7 @@ pub fn prepare(plan: &Plan, prompt: &str, seed: i64, images: &[LoadedImage], mas
let canvas = imaging::pad_solid(&scaled, op.working.canvas, (r.x, r.y), mean_color(src));
let mask = imaging::outpaint_mask(op.working.canvas, r, op.overlap_working);
req.init_image = Some(png_b64(&canvas)?);
+ enhancer = Some(EnhancerView { image: png_b64(&for_enhancer(&canvas))?, kind: EnhancerKind::Expand { sides: growing_sides(&op.expansion) } });
req.mask_image = Some(gray_png_b64(&mask)?);
req.ref_images.push(png_b64(&to_canvas(src, &plan.references[0].canvas))?);
req.strength = plan.strength;
@@ -388,6 +460,7 @@ pub fn prepare(plan: &Plan, prompt: &str, seed: i64, images: &[LoadedImage], mas
Ok(PreparedJob {
request: req,
post: PostProcess { kind, keep_alpha, working: plan.working, content: plan.content, output: plan.output, mode: plan.mode, outpaint: plan.outpaint },
+ enhancer,
})
}
@@ -1118,12 +1191,53 @@ mod tests {
assert_eq!(out.get_pixel(400, 300), &Rgba([120, 120, 120, 255]));
}
+ #[test]
+ fn a_written_prompt_is_sent_whole() {
+ let (img, m, _) = inpaint_job("x", InpaintAction::Remove);
+ let mut r = req("Remove the white box. The bare table continues there, same grain and light.");
+ r.mask = Some("input:m".into());
+ r.inpaint_action = InpaintAction::Remove;
+ let inputs = PlanInputs {
+ image_sizes: vec![img.size()],
+ mask: Some((Size::new(900, 700), imaging::mask_bbox(&m, 127), imaging::mask_coverage(&m, 127))),
+ vram_total_mib: Some(16303),
+ ..Default::default()
+ };
+ let p = plan(&r, &inputs, &RecommendedSettings::default()).unwrap();
+ let j = prepare(&p, &r.prompt, 1, std::slice::from_ref(&img), Some(&m), &BackendOptions { written: true, ..Default::default() }).unwrap();
+ assert!(j.request.prompt.contains("blurred out. Remove the white box. The bare table continues there"), "{}", j.request.prompt);
+ assert!(!j.request.prompt.contains("to remove"), "{}", j.request.prompt);
+ }
+
#[test]
fn remove_needs_no_prompt() {
let (_, _, j) = inpaint_job("", InpaintAction::Remove);
assert!(j.request.prompt.contains("to remove everything that was there"));
}
+ #[test]
+ fn prompt_plus_sees_the_area_unblurred_and_its_text_goes_in_kokons_instruction() {
+ let (_, _, j) = inpaint_job("", InpaintAction::Remove);
+ let v = j.enhancer.as_ref().expect("an area has a Prompt+ view");
+ let EnhancerKind::Area { action, color } = &v.kind else { panic!("not an area") };
+ assert_eq!(*action, InpaintAction::Remove);
+ assert_ne!(v.image, j.request.ref_images[0], "not the blurred image 1");
+ // Expand: Prompt+ sees the canvas, and Kokon's outpaint instruction goes in front of its text.
+ let img = photo(800, 600);
+ let mut r = req("");
+ r.outpaint = Some(OutpaintSpec { left: 0.0, right: 50.0, top: 0.0, bottom: 0.0, units: ExpansionUnits::Percent });
+ let plan = plan(&r, &PlanInputs { image_sizes: vec![img.size()], vram_total_mib: Some(16303), ..Default::default() }, &RecommendedSettings::default()).unwrap();
+ let p = v.prompt(&plan, "Remove the white box. The bare table continues there.");
+ assert!(p.contains("blurred out. Remove the white box. The bare table continues there. Keep everything") && p.contains(color.as_str()), "{p}");
+ let x = prepare(&plan, "", 1, std::slice::from_ref(&img), None, &BackendOptions::default()).unwrap();
+ let xv = x.enhancer.expect("an expansion has a Prompt+ view");
+ assert!(matches!(&xv.kind, EnhancerKind::Expand { sides } if sides == "right"));
+ assert!(xv.prompt(&plan, "More of the meadow.").starts_with("Extend the image naturally") && xv.prompt(&plan, "More of the meadow.").ends_with("More of the meadow."));
+ use crate::geometry::Expansion;
+ assert_eq!(growing_sides(&Expansion { left: 10, right: 10, top: 0, bottom: 10 }), "left, right and bottom");
+ assert_eq!(growing_sides(&Expansion { left: 0, right: 0, top: 20, bottom: 0 }), "top");
+ }
+
#[test]
fn inpaint_action_subjects() {
assert_eq!(InpaintAction::Edit.resolve("please remove the girl"), InpaintAction::Remove);
diff --git a/crates/qls-core/src/planner.rs b/crates/qls-core/src/planner.rs
index 4c1ee81..ab6e497 100644
--- a/crates/qls-core/src/planner.rs
+++ b/crates/qls-core/src/planner.rs
@@ -285,6 +285,10 @@ pub struct GenerationRequest {
/// `JobView::enhanced_prompt` and no image, so the user can read and change it first.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub enhance_only: bool,
+ /// Prompt+ "Detailed": a longer prompt with more context about what is in the image or what
+ /// changes (enhancer::detailed). Standard lengths otherwise.
+ #[serde(default, skip_serializing_if = "std::ops::Not::not")]
+ pub enhance_detail: bool,
/// Layers lab "Blend into scene": only read the place the object with this name stands in
/// (`scene`, the Prompt+ model). The job ends with the reading in `JobView::scene` and no image.
#[serde(default, skip_serializing_if = "Option::is_none")]
@@ -921,15 +925,16 @@ pub fn plan(req: &GenerationRequest, inputs: &PlanInputs, rec: &RecommendedSetti
let (mode, auto_mode, mode_reason) = detect_mode(req, inputs)?;
let marks = mode == ResolvedMode::Inpaint && req.marks.is_some();
if req.enhance_only && !(req.enhance && crate::enhancer::applies_to(mode, marks)) {
- return Err(AppError::invalid("Prompt+ writes prompts for new images, edits and combines only."));
+ return Err(AppError::invalid("Prompt+ does not write prompts for upscales."));
}
// Marks: the prompt names things by colour, never an action of its own (a Remove would blur the marks away).
let inpaint_action = (mode == ResolvedMode::Inpaint).then(|| if marks { InpaintAction::Edit } else { req.inpaint_action.resolve(&req.prompt) });
if req.read_scene.is_some() && (mode != ResolvedMode::Edit || inputs.image_sizes.len() != 1) {
return Err(AppError::invalid("The scene is read from one image, as an edit."));
}
- // Upscales and removals need no description, nor a scene reading (the object's name is its own field).
- let prompt_optional = mode == ResolvedMode::Upscale || inpaint_action == Some(InpaintAction::Remove) || req.read_scene.is_some();
+ // Upscales and removals need no description, nor a scene reading (the object's name is its own
+ // field), nor an expansion (it continues the scene; Prompt+ can write what the new areas show).
+ let prompt_optional = mode == ResolvedMode::Upscale || mode == ResolvedMode::Outpaint || inpaint_action == Some(InpaintAction::Remove) || req.read_scene.is_some();
if req.prompt.trim().is_empty() && !prompt_optional {
return Err(AppError::invalid("Describe what to generate or change."));
}
diff --git a/crates/qls-core/src/projects.rs b/crates/qls-core/src/projects.rs
index fad2bfe..55dd913 100644
--- a/crates/qls-core/src/projects.rs
+++ b/crates/qls-core/src/projects.rs
@@ -107,9 +107,27 @@ fn source(i: &Item) -> Option<String> {
.or_else(|| i.input_id.as_ref().map(|r| format!("input:{r}")))
}
-/// The cover (the latest image) and the number of versions, as the UI computes them.
+/// The images the project's layers lab made (its `made` list, see src/lib/layerDoc.ts): steps
+/// such as one object's crop, never the project's cover.
+fn lab_made(p: &Project) -> HashSet<&str> {
+ p.extra
+ .get("lab")
+ .and_then(|l| l.get("made"))
+ .and_then(Value::as_array)
+ .map(|a| a.iter().filter_map(Value::as_str).collect())
+ .unwrap_or_default()
+}
+
+/// The cover (the latest image that is not a lab step) and the number of versions, as the UI
+/// computes them.
fn refresh_meta(p: &mut Project) {
- p.meta.cover = p.items.iter().rev().find_map(source);
+ let lab = lab_made(p);
+ p.meta.cover = p
+ .items
+ .iter()
+ .rev()
+ .filter_map(source)
+ .find(|s| !lab.contains(s.as_str()));
p.meta.count = p.items.iter().filter(|i| i.kind != "job").count() as u32;
}
@@ -410,6 +428,15 @@ mod tests {
k.iter().map(|s| s.to_string()).collect()
}
+ #[test]
+ fn the_cover_is_never_a_layers_lab_step() {
+ let mut p = project("p", vec![img("a"), img("b")]);
+ p.extra.insert("lab".into(), serde_json::json!({ "v": 2, "docs": [], "made": ["gen:gb"] }));
+ let (p, _) = ProjectStore::in_memory().put("p", p, &known(&[])).unwrap();
+ assert_eq!(p.meta.cover.as_deref(), Some("gen:ga"));
+ assert_eq!(p.meta.count, 2);
+ }
+
#[test]
fn a_new_project_is_stored_with_its_cover_and_count() {
let s = ProjectStore::in_memory();
diff --git a/crates/qls-core/src/provenance.rs b/crates/qls-core/src/provenance.rs
new file mode 100644
index 0000000..af86beb
--- /dev/null
+++ b/crates/qls-core/src/provenance.rs
@@ -0,0 +1,306 @@
+//! AI marking of the images Kokon saves: an XMP packet with IPTC's Digital Source Type (the
+//! vocabulary Google, Meta and Adobe read to label AI images) and Kokon as the creator tool, in
+//! every PNG, JPEG and WebP Kokon writes (results, Save as, the copies for MCP). Unsigned: like any
+//! metadata it can be removed; a signed C2PA manifest would need a certificate (decided 2026-11-04:
+//! XMP only for 0.12; the Code of Practice on marking AI content, Sub-measure 1.1.1, leaves the
+//! signature out "in a local deployment scenario"). The second layer, in the pixels, is the
+//! invisible watermark of watermark.rs (docs/ai-marking.md).
+
+use crate::store::GenerationRecord;
+
+const IPTC_TYPES: &str = "http://cv.iptc.org/newscodes/digitalsourcetype/";
+const XMP_NS: &[u8] = b"http://ns.adobe.com/xap/1.0/\0";
+
+/// The IPTC Digital Source Types that say "made or changed with generative AI".
+pub const AI_TYPES: [&str; 2] = ["trainedAlgorithmicMedia", "compositeWithTrainedAlgorithmicMedia"];
+
+/// The IPTC Digital Source Type of a result: made by the model from words (and references), an
+/// image changed by it, or (an upscale or a colour LUT of an imported photo) only enhanced. A
+/// type set on the record wins: an upscale or LUT takes its parent's (engine.rs `mark`), an
+/// enhanced import that was AI-made already keeps AI.
+pub fn source_type(rec: &GenerationRecord) -> &'static str {
+ const KNOWN: [&str; 3] = [AI_TYPES[0], AI_TYPES[1], "algorithmicallyEnhanced"];
+ if let Some(t) = rec.source_type.as_deref().and_then(|t| KNOWN.iter().find(|a| **a == t)) {
+ return t;
+ }
+ match rec.mode.as_str() {
+ "generate" | "compose" => "trainedAlgorithmicMedia",
+ // Of a Kokon result: still the model's picture; of an imported photo: enhanced only.
+ "upscale" | "lut" if rec.parent_id.is_none() => "algorithmicallyEnhanced",
+ _ => "compositeWithTrainedAlgorithmicMedia",
+ }
+}
+
+/// The attributes for an rdf:Description (also added to the Ultra HDR JPEG's own XMP).
+pub fn attributes(rec: &GenerationRecord) -> String {
+ format!(
+ r#"xmlns:Iptc4xmpExt="http://iptc.org/std/Iptc4xmpExt/2008-02-29/" xmlns:xmp="http://ns.adobe.com/xap/1.0/" Iptc4xmpExt:DigitalSourceType="{IPTC_TYPES}{}" xmp:CreatorTool="Kokon {}" xmp:CreateDate="{}""#,
+ source_type(rec),
+ escape(&rec.app_version),
+ escape(&rec.created_at),
+ )
+}
+
+/// The whole XMP packet.
+pub fn xmp(rec: &GenerationRecord) -> String {
+ format!(
+ "<?xpacket begin=\"\u{feff}\" id=\"W5M0MpCehiHzreSzNTczkc9d\"?><x:xmpmeta xmlns:x=\"adobe:ns:meta/\" x:xmptk=\"Kokon\"><rdf:RDF xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\"><rdf:Description rdf:about=\"\" {}/></rdf:RDF></x:xmpmeta><?xpacket end=\"w\"?>",
+ attributes(rec)
+ )
+}
+
+/// Is the result AI-generated or AI-manipulated (marked with the watermark too)? Only an
+/// enhancement of an imported image is not (the Commission's Art. 50 guidelines: rescaling, colour
+/// adjustments and sharpening are standard editing), unless that import was AI-made already.
+pub fn is_ai(rec: &GenerationRecord) -> bool {
+ AI_TYPES.contains(&source_type(rec))
+}
+
+/// The AI Digital Source Type an image file declares (XMP, or a C2PA manifest's actions, which
+/// carry the same IPTC term as text), if any. Read from the bytes: no parser needed for one term.
+pub fn declared_ai_type(bytes: &[u8]) -> Option<&'static str> {
+ const KEY: &[u8] = b"digitalsourcetype/";
+ let lower: Vec<u8> = bytes.iter().map(u8::to_ascii_lowercase).collect();
+ let mut best = None;
+ let mut from = 0;
+ while let Some(i) = lower[from..].windows(KEY.len()).position(|w| w == KEY) {
+ let at = from + i + KEY.len();
+ let tail = &bytes[at..bytes.len().min(at + 64)];
+ let word: String = tail.iter().take_while(|b| b.is_ascii_alphanumeric()).map(|&b| b as char).collect();
+ // The composite term names the whole when both appear (an edited AI image).
+ match AI_TYPES.iter().find(|t| **t == word) {
+ Some(&"compositeWithTrainedAlgorithmicMedia") => return Some(AI_TYPES[1]),
+ Some(t) => best = Some(*t),
+ None => {}
+ }
+ from = at;
+ }
+ best
+}
+
+/// Any IPTC Digital Source Type term the bytes declare (the composite AI term first when several).
+fn declared_type(bytes: &[u8]) -> Option<String> {
+ if let Some(t) = declared_ai_type(bytes) {
+ return Some(t.into());
+ }
+ const KEY: &[u8] = b"digitalsourcetype/";
+ let lower: Vec<u8> = bytes.iter().map(u8::to_ascii_lowercase).collect();
+ let i = lower.windows(KEY.len()).position(|w| w == KEY)? + KEY.len();
+ let word: String = bytes[i..bytes.len().min(i + 64)].iter().take_while(|b| b.is_ascii_alphanumeric()).map(|&b| b as char).collect();
+ (!word.is_empty()).then_some(word)
+}
+
+/// The XMP CreatorTool (attribute or element form), if any.
+fn creator_tool(bytes: &[u8]) -> Option<String> {
+ let text = String::from_utf8_lossy(bytes);
+ let i = text.find("xmp:CreatorTool")? + "xmp:CreatorTool".len();
+ let rest = &text[i..];
+ let v = if let Some(r) = rest.strip_prefix("=\"") { r.split('"').next() } else { rest.strip_prefix('>').and_then(|r| r.split('<').next()) }?;
+ let v = v.trim();
+ (!v.is_empty() && v.len() < 200).then(|| v.to_string())
+}
+
+/// What an image's AI markings say, for "Check an image" (the Code of Practice's Commitment 2: a
+/// free, local detection anyone can run; nothing is kept). Reads the metadata (IPTC Digital Source
+/// Type in XMP or C2PA, CreatorTool, whether a C2PA manifest is present: not verified) and the
+/// TrustMark P watermark. `ai`: one of them says made or changed with AI.
+pub fn check(sdcpp_dir: &std::path::Path, bytes: &[u8]) -> crate::error::AppResult<serde_json::Value> {
+ use sha2::Digest;
+ let img = crate::imaging::decode_image(bytes)?.pixels;
+ let source = declared_type(bytes);
+ let declared_ai = source.as_deref().is_some_and(|t| AI_TYPES.contains(&t));
+ let c2pa = bytes.windows(4).any(|w| w == b"jumb") && bytes.windows(4).any(|w| w == b"c2pa");
+ let watermark = if crate::watermark::installed(sdcpp_dir) {
+ // Another app's TrustMark only with the strongest error correction: the weaker schemas
+ // accept ~2 % of unmarked images by chance (32 of 1567 on 2026-11-04, none as Kokon's).
+ match crate::watermark::detect(sdcpp_dir, &img)? {
+ Some(f) if f.kokon || f.schema == "BCH_SUPER" => serde_json::json!({ "checked": true, "found": true, "kokon": f.kokon, "text": f.text, "schema": f.schema }),
+ _ => serde_json::json!({ "checked": true, "found": false, "kokon": false }),
+ }
+ } else {
+ serde_json::json!({ "checked": false })
+ };
+ let wm_kokon = watermark["kokon"].as_bool().unwrap_or(false);
+ Ok(serde_json::json!({
+ "ai": declared_ai || wm_kokon,
+ "metadata": { "digital_source_type": source, "ai": declared_ai, "creator_tool": creator_tool(bytes), "c2pa_manifest": c2pa },
+ "watermark": watermark,
+ "width": img.width(),
+ "height": img.height(),
+ "sha256": hex::encode(sha2::Sha256::digest(bytes)),
+ "checked_at": chrono::Local::now().to_rfc3339(),
+ "checker": format!("Kokon {} (XMP/C2PA reader, TrustMark P decoder)", crate::engine::APP_VERSION),
+ }))
+}
+
+fn escape(s: &str) -> String {
+ s.replace('&', "&amp;")
+ .replace('"', "&quot;")
+ .replace('<', "&lt;")
+ .replace('>', "&gt;")
+}
+
+/// A JPEG with the XMP as an APP1 segment after SOI (and after a JFIF APP0, which comes first).
+pub fn jpeg_with_xmp(jpeg: &[u8], xmp: &str) -> Vec<u8> {
+ if jpeg.len() < 4 || jpeg[..2] != [0xFF, 0xD8] {
+ return jpeg.to_vec();
+ }
+ let mut at = 2;
+ if jpeg[2..4] == [0xFF, 0xE0] && jpeg.len() >= 6 {
+ at = (4 + u16::from_be_bytes([jpeg[4], jpeg[5]]) as usize).min(jpeg.len());
+ }
+ let payload = [XMP_NS, xmp.as_bytes()].concat();
+ let mut out = Vec::with_capacity(jpeg.len() + payload.len() + 4);
+ out.extend_from_slice(&jpeg[..at]);
+ out.extend_from_slice(&[0xFF, 0xE1]);
+ out.extend_from_slice(&((payload.len() + 2) as u16).to_be_bytes());
+ out.extend_from_slice(&payload);
+ out.extend_from_slice(&jpeg[at..]);
+ out
+}
+
+/// A WebP with the XMP in an "XMP " chunk: a simple file (one VP8/VP8L chunk) becomes an extended
+/// one (VP8X first, with the XMP flag, and the alpha flag when `alpha`).
+pub fn webp_with_xmp(webp: &[u8], xmp: &str, width: u32, height: u32, alpha: bool) -> Vec<u8> {
+ if webp.len() < 20 || &webp[..4] != b"RIFF" || &webp[8..12] != b"WEBP" {
+ return webp.to_vec();
+ }
+ let body = &webp[12..];
+ let mut chunks = Vec::with_capacity(webp.len() + xmp.len() + 40);
+ if &body[..4] == b"VP8X" {
+ chunks.extend_from_slice(body);
+ chunks[8] |= 0x04;
+ } else {
+ let mut vp8x = [0u8; 10];
+ vp8x[0] = 0x04 | if alpha { 0x10 } else { 0 };
+ vp8x[4..7].copy_from_slice(&(width.max(1) - 1).to_le_bytes()[..3]);
+ vp8x[7..10].copy_from_slice(&(height.max(1) - 1).to_le_bytes()[..3]);
+ chunks.extend_from_slice(b"VP8X");
+ chunks.extend_from_slice(&10u32.to_le_bytes());
+ chunks.extend_from_slice(&vp8x);
+ chunks.extend_from_slice(body);
+ }
+ chunks.extend_from_slice(b"XMP ");
+ chunks.extend_from_slice(&(xmp.len() as u32).to_le_bytes());
+ chunks.extend_from_slice(xmp.as_bytes());
+ if xmp.len() % 2 == 1 {
+ chunks.push(0);
+ }
+ let mut out = Vec::with_capacity(chunks.len() + 12);
+ out.extend_from_slice(b"RIFF");
+ out.extend_from_slice(&((chunks.len() + 4) as u32).to_le_bytes());
+ out.extend_from_slice(b"WEBP");
+ out.extend_from_slice(&chunks);
+ out
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ fn rec(mode: &str, parent: Option<&str>) -> GenerationRecord {
+ let mut r: GenerationRecord = serde_json::from_value(serde_json::json!({
+ "id": "g", "created_at": "2026-11-04T10:00:00+01:00", "mode": mode, "prompt": "", "negative_prompt": "",
+ "seed": 1, "width": 8, "height": 8, "working_width": 8, "working_height": 8, "model": "", "sampler": "",
+ "steps": 8, "cfg": 1.0, "quality": "", "resolution": "", "references": [], "file_path": "f", "app_version": "0.12.0", "favorite": false, "duration_ms": 0, "notes": [], "extra_paths": [], "request": null,
+ }))
+ .unwrap();
+ r.parent_id = parent.map(str::to_string);
+ r
+ }
+
+ #[test]
+ fn source_types() {
+ assert_eq!(
+ source_type(&rec("generate", None)),
+ "trainedAlgorithmicMedia"
+ );
+ assert_eq!(
+ source_type(&rec("compose", None)),
+ "trainedAlgorithmicMedia"
+ );
+ for m in ["edit", "inpaint", "outpaint", "multi_reference", "layers"] {
+ assert_eq!(
+ source_type(&rec(m, None)),
+ "compositeWithTrainedAlgorithmicMedia",
+ "{m}"
+ );
+ }
+ assert_eq!(
+ source_type(&rec("upscale", None)),
+ "algorithmicallyEnhanced"
+ );
+ assert_eq!(
+ source_type(&rec("lut", Some("p"))),
+ "compositeWithTrainedAlgorithmicMedia"
+ );
+ // An enhanced import that was AI-made keeps its type, and is AI.
+ let mut up = rec("upscale", None);
+ assert!(!is_ai(&up));
+ up.source_type = Some("trainedAlgorithmicMedia".into());
+ assert_eq!(source_type(&up), "trainedAlgorithmicMedia");
+ assert!(is_ai(&up) && is_ai(&rec("edit", None)));
+ up.source_type = Some("made up".into());
+ assert_eq!(source_type(&up), "algorithmicallyEnhanced");
+ let x = xmp(&rec("generate", None));
+ assert!(x.contains(r#"Iptc4xmpExt:DigitalSourceType="http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia""#));
+ assert!(
+ x.contains(r#"xmp:CreatorTool="Kokon 0.12.0""#) && x.ends_with("<?xpacket end=\"w\"?>")
+ );
+ }
+
+ #[test]
+ fn reads_the_declared_type() {
+ let x = xmp(&rec("generate", None));
+ assert_eq!(declared_ai_type(format!("junk{x}junk").as_bytes()), Some("trainedAlgorithmicMedia"));
+ assert_eq!(declared_ai_type(xmp(&rec("edit", None)).as_bytes()), Some("compositeWithTrainedAlgorithmicMedia"));
+ assert_eq!(declared_ai_type(xmp(&rec("upscale", None)).as_bytes()), None);
+ // A C2PA action names the type in its own casing; a camera photo has none.
+ assert_eq!(declared_ai_type(b"..http://cv.iptc.org/newscodes/DigitalSourceType/trainedAlgorithmicMedia\x00.."), Some("trainedAlgorithmicMedia"));
+ assert_eq!(declared_ai_type(b"digitalsourcetype/digitalCapture"), None);
+ assert_eq!(declared_ai_type(b""), None);
+ }
+
+ #[test]
+ fn check_reads_the_metadata_without_the_models() {
+ let img = image::RgbaImage::from_pixel(9, 7, image::Rgba([10, 20, 30, 255]));
+ let jpg = jpeg_with_xmp(&crate::imaging::encode_jpeg(&img, 90).unwrap(), &xmp(&rec("edit", None)));
+ let r = check(std::path::Path::new("no-models-here"), &jpg).unwrap();
+ assert_eq!(r["ai"], true);
+ assert_eq!(r["metadata"]["digital_source_type"], "compositeWithTrainedAlgorithmicMedia");
+ assert_eq!(r["metadata"]["creator_tool"], "Kokon 0.12.0");
+ assert_eq!(r["metadata"]["c2pa_manifest"], false);
+ assert_eq!(r["watermark"]["checked"], false);
+ assert_eq!(r["sha256"].as_str().unwrap().len(), 64);
+ let plain = check(std::path::Path::new("no-models-here"), &crate::imaging::encode_jpeg(&img, 90).unwrap()).unwrap();
+ assert_eq!((plain["ai"].as_bool(), plain["metadata"]["digital_source_type"].is_null()), (Some(false), true));
+ assert_eq!(creator_tool(b"<xmp:CreatorTool>Adobe Photoshop 26.0</xmp:CreatorTool>").as_deref(), Some("Adobe Photoshop 26.0"));
+ }
+
+ #[test]
+ fn jpeg_and_webp_carry_it_and_still_decode() {
+ let img = image::RgbaImage::from_pixel(9, 7, image::Rgba([10, 20, 30, 128]));
+ let x = xmp(&rec("edit", None));
+ let jpg = jpeg_with_xmp(&crate::imaging::encode_jpeg(&img, 90).unwrap(), &x);
+ assert!(jpg.windows(XMP_NS.len()).any(|w| w == XMP_NS));
+ assert_eq!(image::load_from_memory(&jpg).unwrap().width(), 9);
+ let webp = webp_with_xmp(
+ &crate::imaging::encode_webp_lossless(&img).unwrap(),
+ &x,
+ 9,
+ 7,
+ true,
+ );
+ assert_eq!(&webp[12..16], b"VP8X");
+ assert_eq!(
+ u32::from_le_bytes(webp[4..8].try_into().unwrap()) as usize,
+ webp.len() - 8
+ );
+ assert!(webp.windows(4).any(|w| w == b"XMP "));
+ let back = image::load_from_memory(&webp).unwrap().to_rgba8();
+ assert_eq!(
+ (back.width(), back.height(), back.get_pixel(0, 0)[3]),
+ (9, 7, 128)
+ );
+ }
+}
diff --git a/crates/qls-core/src/store.rs b/crates/qls-core/src/store.rs
index 2c3e0ce..f4bcb5e 100644
--- a/crates/qls-core/src/store.rs
+++ b/crates/qls-core/src/store.rs
@@ -95,6 +95,10 @@ pub struct GenerationRecord {
/// it is shown). The project still opens it by id.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub hidden: bool,
+ /// The IPTC Digital Source Type when it is not the one of its mode (provenance.rs): an upscale
+ /// or a LUT of an imported image that was already AI-made stays marked as AI-made.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub source_type: Option<String>,
}
impl GenerationRecord {
@@ -564,6 +568,7 @@ mod tests {
request: serde_json::json!({"prompt": prompt}),
pipeline: None,
hidden: false,
+ source_type: None,
}
}
diff --git a/crates/qls-core/src/watermark/LICENSE-trustmark b/crates/qls-core/src/watermark/LICENSE-trustmark
new file mode 100644
index 0000000..cec325d
--- /dev/null
+++ b/crates/qls-core/src/watermark/LICENSE-trustmark
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Adobe
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/crates/qls-core/src/watermark/bch.rs b/crates/qls-core/src/watermark/bch.rs
new file mode 100644
index 0000000..7b6fba5
--- /dev/null
+++ b/crates/qls-core/src/watermark/bch.rs
@@ -0,0 +1,1085 @@
+// Vendored from Adobe TrustMark (https://github.com/adobe/trustmark, rust/src/bits/bch.rs, MIT
+// License, Copyright (c) 2024 Adobe; see crates/qls-core/src/watermark/LICENSE-trustmark). Unchanged
+// except this note and the lint allowance below.
+#![allow(clippy::all, dead_code, unused_parens)]
+
+// Copyright 2024 Adobe
+// All Rights Reserved.
+//
+// NOTICE: Adobe permits you to use, modify, and distribute this file in
+// accordance with the terms of the Adobe license agreement accompanying
+// it.
+
+// This module is a direct port of trustmark's BCH encoding/decoding code[^1].
+//
+// TODO: Refactor this to be clearer/more maintainable.
+//
+// [^1]: https://github.com/adobe/trustmark/blob/0a45b8dc4e515068e5d043c9a1086e094814b5d7/python/trustmark/bchecc.py
+
+pub(super) const POLYNOMIAL: u32 = 137;
+
+#[derive(Debug)]
+pub(super) struct EccState {
+ m: u32,
+ t: u32,
+ _poly: u32,
+ n: Option<u32>,
+ ecc_bytes: Option<u32>,
+ cyclic_tab: Option<Vec<u32>>,
+ syn: Option<Vec<u32>>,
+ elp: Option<Vec<u32>>,
+ errloc: Option<Vec<u32>>,
+ exponents: Option<Vec<u32>>,
+ logarithms: Option<Vec<u32>>,
+ elp_pre: Option<Vec<u32>>,
+ ecc_bits: Option<u32>,
+ ecc_buf: Option<Vec<u32>>,
+}
+
+#[derive(Debug, Clone)]
+struct Polynomial {
+ deg: u32,
+ c: Option<Vec<u32>>,
+}
+
+impl Polynomial {
+ fn new(deg: u32) -> Self {
+ Self { deg, c: None }
+ }
+}
+
+// def __init__(self, t, poly):
+pub(super) fn bch_init(t: u32, poly: u32) -> EccState {
+ // tmp = poly;
+ // m = 0;
+ // while (tmp >> 1):
+ // tmp =tmp >> 1
+ // m +=1
+ let m = poly.ilog2();
+
+ // self.ECCstate=self.params(m=m,t=t,poly=poly)
+ let mut ecc_state = EccState::new(m, t, poly);
+
+ // self.ECCstate.n=pow(2,m)-1
+ ecc_state.n = Some(2_u32.pow(m) - 1);
+ // words = self.ceilop(m*t,32)
+ let words = (m * t).div_ceil(32);
+ // self.ECCstate.ecc_bytes = self.ceilop(m*t,8)
+ ecc_state.ecc_bytes = Some((m * t).div_ceil(8));
+ // self.ECCstate.cyclic_tab=[0]*(words*1024)
+ ecc_state.cyclic_tab = Some(vec![0; (words * 1024).try_into().unwrap()]);
+ // self.ECCstate.syn=[0]*(2*t)
+ ecc_state.syn = Some(vec![0; (2 * t).try_into().unwrap()]);
+ // self.ECCstate.elp=[0]*(t+1)
+ ecc_state.elp = Some(vec![0; (t + 1).try_into().unwrap()]);
+ // self.ECCstate.errloc=[0] * t
+ ecc_state.errloc = Some(vec![0; t.try_into().unwrap()]);
+
+ // x=1
+ // k=pow(2,self.deg(poly))
+ // if k != pow(2,self.ECCstate.m):
+ // return -1
+ let mut x = 1;
+ let k = 2_u32.pow(poly.ilog2());
+ if k != 2_u32.pow(ecc_state.m) {
+ panic!("k ({k}) should be equal to {}", 2_u32.pow(ecc_state.m));
+ }
+
+ // self.ECCstate.exponents=[0]*(1+self.ECCstate.n)
+ ecc_state.exponents = Some(vec![0; (1 + ecc_state.n.unwrap()).try_into().unwrap()]);
+ // self.ECCstate.logarithms=[0]*(1+self.ECCstate.n)
+ ecc_state.logarithms = Some(vec![0; (1 + ecc_state.n.unwrap()).try_into().unwrap()]);
+ // self.ECCstate.elp_pre=[0]*(1+self.ECCstate.m)
+ ecc_state.elp_pre = Some(vec![0; (1 + ecc_state.m).try_into().unwrap()]);
+
+ // for i in range(0,self.ECCstate.n):
+ // self.ECCstate.exponents[i]=x
+ // self.ECCstate.logarithms[x]=i
+ // if i and x==1:
+ // return -1
+ // x*= 2
+ // if (x & k):
+ // x=x^poly
+ for i in 0..ecc_state.n.unwrap() {
+ ecc_state.exponents.as_mut().unwrap()[i as usize] = x;
+ ecc_state.logarithms.as_mut().unwrap()[x as usize] = i;
+ if i != 0 && x == 1 {
+ panic!("i ({i}) was nonzero and x was 1");
+ }
+ x *= 2;
+ if x & k != 0 {
+ x ^= poly;
+ }
+ }
+
+ // self.ECCstate.logarithms[0]=0
+ // self.ECCstate.exponents[self.ECCstate.n]=1
+ ecc_state.logarithms.as_mut().unwrap()[0] = 0;
+ ecc_state.exponents.as_mut().unwrap()[ecc_state.n.unwrap() as usize] = 1;
+
+ // n=0
+ // g=self.polynomial(deg=0)
+ // g.c=[0]*((m*t)+1)
+ // roots=[0]*(self.ECCstate.n+1)
+ // genpoly=[0]*self.ceilop(m*t+1,32)
+ let mut g = Polynomial::new(0);
+ g.c = Some(vec![0; (m * t + 1).try_into().unwrap()]);
+ let mut roots = vec![0; (ecc_state.n.unwrap() + 1).try_into().unwrap()];
+ let mut genpoly = vec![0; (m * t + 1).div_ceil(32).try_into().unwrap()];
+
+ // # enum all roots
+ // for i in range(0,t):
+ // r=2*i+1
+ // for j in range(0,m):
+ // roots[r]=1
+ // r=self.mod(2*r)
+ for i in 0..t {
+ let mut r = 2 * i + 1;
+ for _j in 0..m {
+ roots[r as usize] = 1;
+ r = (2 * r) % ecc_state.n.unwrap();
+ }
+ }
+
+ // # build g(x)
+ // g.deg=0
+ // g.c[0]=1
+ g.c.as_mut().unwrap()[0] = 1;
+ // for i in range(0,self.ECCstate.n):
+ // if roots[i]:
+ // r=self.ECCstate.exponents[i]
+ // g.c[g.deg+1]=1
+ // for j in range(g.deg,0,-1):
+ // g.c[j]=self.g_mul(g.c[j],r)^g.c[j-1]
+ // g.c[0]=self.g_mul(g.c[0],r)
+ // g.deg += 1
+ for i in 0..ecc_state.n.unwrap() {
+ if roots[i as usize] != 0 {
+ let r = ecc_state.exponents.as_ref().unwrap()[i as usize];
+ g.c.as_mut().unwrap()[(g.deg + 1) as usize] = 1;
+ for j in (1..=g.deg).rev() {
+ g.c.as_mut().unwrap()[j as usize] = ecc_state
+ .g_mul(g.c.as_ref().unwrap()[j as usize], r)
+ ^ g.c.as_ref().unwrap()[(j - 1) as usize];
+ }
+ g.c.as_mut().unwrap()[0] = ecc_state.g_mul(g.c.as_ref().unwrap()[0], r);
+ g.deg += 1;
+ }
+ }
+
+ // # store
+ // n = g.deg+1
+ // i = 0
+ let mut n = g.deg + 1;
+ let mut i = 0;
+
+ // while (n>0) :
+
+ // if n>32:
+ // nbits=32
+ // else:
+ // nbits=n
+
+ // word=0
+ // for j in range (0,nbits):
+ // if g.c[n-1-j] :
+ // word = word | pow(2,31-j)
+ // genpoly[i]=word
+ // i += 1
+ // n -= nbits
+ while n > 0 {
+ let nbits = u32::min(n, 32);
+ let mut word = 0;
+ for j in 0..nbits {
+ if g.c.as_ref().unwrap()[(n - 1 - j) as usize] != 0 {
+ word |= 2_u32.pow(31 - j);
+ }
+ }
+ genpoly[i] = word;
+ i += 1;
+ n -= nbits;
+ }
+ // self.ECCstate.ecc_bits=g.deg
+ ecc_state.ecc_bits = Some(g.deg);
+
+ // self.build_cyclic(genpoly);
+ ecc_state.build_cyclic(&genpoly);
+
+ // sum=0
+ // aexp=0
+ // for i in range(0,m):
+ // for j in range(0,m):
+ // sum = sum ^ self.g_pow(i*pow(2,j))
+ // if sum:
+ // aexp=self.ECCstate.exponents[i]
+ // break
+ let mut sum = 0;
+ let mut aexp = 0;
+ for i in 0..m {
+ for j in 0..m {
+ sum ^= ecc_state.g_pow((i * 2_u32.pow(j)) as usize)
+ }
+ if sum != 0 {
+ aexp = ecc_state.exponents.as_ref().unwrap()[i as usize];
+ break;
+ }
+ }
+
+ // x=0
+ // precomp=[0] * 31
+ // remaining=m
+ let mut x = 0;
+ let mut precomp = [0; 31];
+ let mut remaining = m;
+
+ // while (x<= self.ECCstate.n and remaining):
+ // y=self.g_sqrt(x)^x
+ // for i in range(0,2):
+ // r=self.g_log(y)
+ // if (y and (r<m) and not precomp[r]):
+ // self.ECCstate.elp_pre[r]=x
+ // precomp[r]=1
+ // remaining -=1
+ // break
+ // y=y^aexp
+ // x += 1
+ while x <= ecc_state.n.unwrap() && remaining != 0 {
+ let mut y = ecc_state.g_sqrt(x) ^ x;
+ for _i in 0..2 {
+ let r = ecc_state.logarithms.as_ref().unwrap()[y as usize];
+ if y != 0 && r < m && precomp[r as usize] == 0 {
+ ecc_state.elp_pre.as_mut().unwrap()[r as usize] = x;
+ precomp[r as usize] = 1;
+ remaining -= 1;
+ break;
+ }
+ y ^= aexp;
+ }
+ x += 1;
+ }
+
+ // Commentary: The original function was a class initializer, so we implicitly return the
+ // `EccState` we've been building up here.
+ ecc_state
+}
+
+// def encode(self,data):
+pub(super) fn bch_encode(ecc_state: &mut EccState, data: &[u8]) -> Vec<u8> {
+ // datalen=len(data)
+ // l=self.ceilop(self.ECCstate.m*self.ECCstate.t, 32)-1
+ let l = (ecc_state.m * ecc_state.t).div_ceil(32) - 1;
+
+ // ecc= [0]*self.ECCstate.ecc_bytes
+ // Commentary: This doesn't seem to be used, so I'm deleting it.
+
+ // ecc_max_words=self.ceilop(31*64, 32)
+ // r = [0]*ecc_max_words
+ let ecc_max_words: usize = (31_usize * 64).div_ceil(32);
+ let mut r = vec![0; ecc_max_words];
+
+ // tab0idx=0
+ // tab1idx=tab0idx+256*(l+1)
+ // tab2idx=tab1idx+256*(l+1)
+ // tab3idx=tab2idx+256*(l+1)
+ let tab0idx = 0;
+ let tab1idx = tab0idx + 256 * (l + 1);
+ let tab2idx = tab1idx + 256 * (l + 1);
+ let tab3idx = tab2idx + 256 * (l + 1);
+
+ // mlen=int(datalen/4) # how many whole words
+ // offset=0
+ let mut mlen = data.len() / 4;
+ let mut offset = 0;
+ // while (mlen>0):
+ // w=self.load4bytes(data[offset:(offset+4)])
+ // w=w^r[0]
+ // p0=tab0idx+(l+1)*((w>>0) & 0xff)
+ // p1=tab1idx+(l+1)*((w>>8) & 0xff)
+ // p2=tab2idx+(l+1)*((w>>16) & 0xff)
+ // p3=tab3idx+(l+1)*((w>>24) & 0xff)
+
+ // for i in range(0,l):
+ // r[i]=r[i+1] ^ self.ECCstate.cyclic_tab[p0+i] ^ self.ECCstate.cyclic_tab[p1+i] ^ self.ECCstate.cyclic_tab[p2+i] ^ self.ECCstate.cyclic_tab[p3+i]
+
+ // r[l] = self.ECCstate.cyclic_tab[p0+l]^self.ECCstate.cyclic_tab[p1+l]^self.ECCstate.cyclic_tab[p2+l]^self.ECCstate.cyclic_tab[p3+l];
+ // mlen -=1
+ // offset +=4
+ while mlen > 0 {
+ let mut w = u32::from_be_bytes(data[offset..offset + 4].try_into().unwrap());
+ w ^= r[0];
+ let p0 = tab0idx + (l + 1) * (w & 0xff);
+ let p1 = tab1idx + (l + 1) * ((w >> 8) & 0xff);
+ let p2 = tab2idx + (l + 1) * ((w >> 16) & 0xff);
+ let p3 = tab3idx + (l + 1) * ((w >> 24) & 0xff);
+ let cyclic_tab = ecc_state.cyclic_tab.as_ref().unwrap();
+ for i in 0..l {
+ r[i as usize] = r[(i + 1) as usize]
+ ^ cyclic_tab[(p0 + i) as usize]
+ ^ cyclic_tab[(p1 + i) as usize]
+ ^ cyclic_tab[(p2 + i) as usize]
+ ^ cyclic_tab[(p3 + i) as usize];
+ }
+ r[l as usize] = cyclic_tab[(p0 + l) as usize]
+ ^ cyclic_tab[(p1 + l) as usize]
+ ^ cyclic_tab[(p2 + l) as usize]
+ ^ cyclic_tab[(p3 + l) as usize];
+ mlen -= 1;
+ offset += 4;
+ }
+
+ // data=data[offset:]
+ // leftdata=len(data)
+ let data = &data[offset..];
+ let mut leftdata = data.len();
+
+ // ecc=r
+ // posn=0
+ let mut ecc = r;
+ let mut posn = 0;
+ // while (leftdata):
+ // tmp=data[posn]
+ // posn += 1
+ // pidx = (l+1)*(((ecc[0] >> 24)^(tmp)) & 0xff)
+ // for i in range(0,l):
+ // ecc[i]=(((ecc[i] << 8)&0xffffffff)|ecc[i+1]>>24)^(self.ECCstate.cyclic_tab[pidx])
+ // pidx += 1
+ // ecc[l]=((ecc[l] << 8)&0xffffffff)^(self.ECCstate.cyclic_tab[pidx])
+ // leftdata -= 1
+ while leftdata != 0 {
+ let tmp = data[posn];
+ let cyclic_tab = ecc_state.cyclic_tab.as_ref().unwrap();
+ posn += 1;
+ let mut pidx = (l + 1) * (((ecc[0] >> 24) ^ (tmp as u32)) & 0xff);
+ for i in 0..l {
+ ecc[i as usize] = ((ecc[i as usize] << 8) | (ecc[(i + 1) as usize] >> 24))
+ ^ cyclic_tab[pidx as usize];
+ pidx += 1;
+ }
+ ecc[l as usize] = (ecc[l as usize] << 8) ^ cyclic_tab[pidx as usize];
+ leftdata -= 1;
+ }
+
+ // self.ECCstate.ecc_buf=ecc
+ // eccout=[]
+ ecc_state.ecc_buf = Some(ecc.clone());
+ let mut eccout: Vec<u8> = vec![];
+ // for e in r:
+ // eccout.append((e >> 24) & 0xff)
+ // eccout.append((e >> 16) & 0xff)
+ // eccout.append((e >> 8) & 0xff)
+ // eccout.append((e >> 0) & 0xff)
+ // Commentary: In the Python version, `ecc` and `r` reference the same value.
+ for e in ecc {
+ eccout.extend(e.to_be_bytes())
+ }
+
+ // eccout=eccout[0:self.ECCstate.ecc_bytes]
+ eccout.truncate(ecc_state.ecc_bytes.unwrap().try_into().unwrap());
+
+ // eccbytes=(bytearray(bytes(eccout)))
+ // return eccbytes
+ eccout
+}
+
+// def decode(self,data,recvecc):
+pub(super) fn bch_decode(ecc_state: &mut EccState, data: &mut [u8], recvecc: &[u8]) -> u8 {
+ // calc_ecc=self.encode(data)
+ let _calc_ecc = bch_encode(ecc_state, data);
+ let mut corrected_ecc = recvecc.to_vec();
+
+ // self.ECCstate.errloc=[]
+ ecc_state.errloc = Some(vec![]);
+
+ // ecclen=len(recvecc)
+ // mlen=int(ecclen/4) # how many whole words
+ // eccbuf=[]
+ // offset=0
+ let ecclen = recvecc.len();
+ let mut mlen = ecclen / 4;
+ let mut eccbuf = vec![];
+ let mut offset = 0;
+
+ // while (mlen>0):
+ while mlen > 0 {
+ // w=self.load4bytes(recvecc[offset:(offset+4)])
+ // eccbuf.append(w)
+ // offset+=4
+ // mlen -=1
+ let w = u32::from_be_bytes(recvecc[offset..offset + 4].try_into().unwrap());
+ eccbuf.push(w);
+ offset += 4;
+ mlen -= 1;
+ }
+
+ // recvecc=recvecc[offset:]
+ // leftdata=len(recvecc)
+ let mut recvecc = recvecc[offset..].to_vec();
+ let leftdata = recvecc.len();
+
+ // if leftdata>0: #pad it to 4
+ if leftdata > 0 {
+ // recvecc=recvecc+bytes([0]*(4-leftdata))
+ // w=self.load4bytes(recvecc)
+ // eccbuf.append(w)
+ recvecc.extend(std::iter::repeat_n(0, 4 - leftdata));
+ let w = u32::from_be_bytes(recvecc.clone().try_into().unwrap());
+ eccbuf.push(w);
+ }
+
+ // eccwords=self.ceilop(self.ECCstate.m*self.ECCstate.t, 32)
+ let eccwords = (ecc_state.m * ecc_state.t).div_ceil(32);
+
+ // sum=0
+ let mut sum = 0;
+ // for i in range(0,eccwords):
+ for i in 0..eccwords {
+ // self.ECCstate.ecc_buf[i] = self.ECCstate.ecc_buf[i] ^ eccbuf[i]
+ // sum = sum | self.ECCstate.ecc_buf[i]
+ ecc_state.ecc_buf.as_mut().unwrap()[i as usize] ^= eccbuf[i as usize];
+ sum |= ecc_state.ecc_buf.as_ref().unwrap()[i as usize];
+ }
+ // if sum==0:
+ // return 0 # no bit flips
+ if sum == 0 {
+ return 0;
+ }
+
+ // s=self.ECCstate.ecc_bits
+ // t=self.ECCstate.t
+ // syn=[0]*(2*t)
+ let mut s: i32 = ecc_state.ecc_bits.unwrap() as i32;
+ let t = ecc_state.t;
+ let mut syn = vec![0; 2 * (t as usize)];
+
+ // m= s & 31
+ let m = s & 31;
+
+ // synbuf=self.ECCstate.ecc_buf
+ let mut synbuf = ecc_state.ecc_buf.clone().unwrap();
+
+ // if (m):
+ // synbuf[int(s/32)] = synbuf[int(s/32)] & ~(pow(2,32-m)-1)
+ if m != 0 {
+ synbuf[(s / 32) as usize] &= !(2_u32.pow(32 - (m as u32)) - 1);
+ }
+
+ // synptr=0
+ let mut synptr = 0;
+ // while(s>0 or synptr==0):
+ while s > 0 || synptr == 0 {
+ // poly=synbuf[synptr]
+ // synptr += 1
+ // s-= 32
+ let mut poly = synbuf[synptr];
+ synptr += 1;
+ s -= 32;
+ // while (poly):
+ while poly != 0 {
+ // i=self.deg(poly)
+ let i = poly.ilog2();
+ // for j in range(0,(2*t),2):
+ for j in (0..2 * t).step_by(2) {
+ // syn[j]=syn[j] ^ self.g_pow((j+1)*(i+s))
+ syn[j as usize] ^= ecc_state.g_pow(((j + 1) * (((i as i32) + s) as u32)) as usize);
+ }
+ // poly = poly ^ pow(2,i)
+ poly ^= 2_u32.pow(i);
+ }
+ }
+
+ // for i in range(0,t):
+ // syn[2*i+1]=self.g_sqrt(syn[i])
+ for i in 0..t {
+ syn[(2 * i + 1) as usize] = ecc_state.g_sqrt(syn[i as usize]);
+ }
+
+ // n=self.ECCstate.n
+ // t=self.ECCstate.t
+ // pp=-1
+ // pd=1
+ let n = ecc_state.n.unwrap();
+ let t = ecc_state.t;
+ let mut pp: i32 = -1;
+ let mut pd = 1;
+
+ // pelp=self.polynomial(deg=0)
+ // pelp.deg=0
+ // pelp.c= [0]*(2*t)
+ // pelp.c[0]=1
+ let mut pelp = Polynomial::new(0);
+ pelp.deg = 0;
+ pelp.c = Some(vec![0; (2 * t) as usize]);
+ pelp.c.as_mut().unwrap()[0] = 1;
+
+ // elp=self.polynomial(deg=0)
+ // elp.c= [0]*(2*t)
+ // elp.c[0]=1
+ let mut elp = Polynomial::new(0);
+ elp.c = Some(vec![0; (2 * t) as usize]);
+ elp.c.as_mut().unwrap()[0] = 1;
+
+ // d=syn[0]
+ let mut d = syn[0];
+
+ // elp_copy=self.polynomial(deg=0)
+ let mut elp_copy;
+ // for i in range(0,t):
+ for i in 0..t {
+ // if (elp.deg>t):
+ // break
+ if elp.deg > t {
+ break;
+ }
+ // if d:
+ if d != 0 {
+ // k=2*i-pp
+ // elp_copy=deepcopy(elp)
+ // tmp=self.g_log(d)+n-self.g_log(pd)
+ let k = (2 * (i as i32) - pp) as u32;
+ elp_copy = elp.clone();
+ let tmp = ecc_state.g_log(d) + n - ecc_state.g_log(pd);
+ // for j in range(0,(pelp.deg+1)):
+ for j in 0..pelp.deg + 1 {
+ // if (pelp.c[j]):
+ // l=self.g_log(pelp.c[j])
+ // elp.c[j+k]=elp.c[j+k] ^ self.g_pow(tmp+l)
+ if pelp.c.as_ref().unwrap()[j as usize] != 0 {
+ let l = ecc_state.g_log(pelp.c.as_ref().unwrap()[j as usize]);
+ elp.c.as_mut().unwrap()[(j + k) as usize] ^=
+ ecc_state.g_pow((tmp + l) as usize);
+ }
+ }
+
+ // tmp=pelp.deg+k
+ let tmp = pelp.deg + k;
+ // if tmp>elp.deg:
+ if tmp > elp.deg {
+ // elp.deg=tmp
+ // pelp=deepcopy(elp_copy)
+ // pd=d
+ // pp=2*i
+ elp.deg = tmp;
+ pelp = elp_copy.clone();
+ pd = d;
+ pp = (2 * i) as i32;
+ }
+ }
+ // if (i<t-1):
+ if i < t - 1 {
+ // d=syn[2*i+2]
+ // for j in range(1,(elp.deg+1)):
+ // d = d ^ self.g_mul(elp.c[j],syn[2*i+2-j])
+ d = syn[(2 * i + 2) as usize];
+ for j in 1..elp.deg + 1 {
+ d ^= ecc_state.g_mul(
+ elp.c.as_ref().unwrap()[j as usize],
+ syn[(2 * i + 2 - j) as usize],
+ );
+ }
+ }
+ }
+ // self.ECCstate.elp=elp
+ // Commentary: `ecc_state.elp` is an `Option<Vec<u32>>`, I'm going to leave this assignment
+ // out for now and see where it gets me. That means that below, I will refer to `elp` instead
+ // of `ecc_state.elp`.
+
+ // nroots = self.getroots(len(data),self.ECCstate.elp)
+ // datalen=len(data)
+ // nbits=(datalen*8)+self.ECCstate.ecc_bits
+ let nroots = ecc_state.getroots(data.len() as u32, &elp);
+ // Python version of getroots() returned -1 on failure. Then, when we range over (0,-1), no
+ // iteration would occur.
+ // Rust version returns u32::MAX. If we try to iterate over 0..u32::MAX, we'll try to access an
+ // element from ecc_state.errloc, which is empty.
+ // Hence, we check for the sentinal value here.
+ if nroots != elp.deg || nroots > t {
+ return u8::MAX;
+ }
+ let datalen = data.len() as u32;
+ let nbits = (datalen * 8) + ecc_state.ecc_bits.unwrap();
+
+ // for i in range(0,nroots):
+ // if self.ECCstate.errloc[i] >= nbits:
+ // return -1
+ // self.ECCstate.errloc[i]=nbits-1-self.ECCstate.errloc[i]
+ // self.ECCstate.errloc[i]=(self.ECCstate.errloc[i] & ~7) | (7-(self.ECCstate.errloc[i] & 7))
+ for i in 0..nroots {
+ if ecc_state.errloc.as_mut().unwrap()[i as usize] >= nbits {
+ return u8::MAX;
+ }
+ ecc_state.errloc.as_mut().unwrap()[i as usize] =
+ nbits - 1 - ecc_state.errloc.as_ref().unwrap()[i as usize];
+ ecc_state.errloc.as_mut().unwrap()[i as usize] =
+ (ecc_state.errloc.as_ref().unwrap()[i as usize] & !7)
+ | (7 - (ecc_state.errloc.as_ref().unwrap()[i as usize] & 7))
+ }
+
+ // for bitflip in self.ECCstate.errloc:
+ // byte= int (bitflip / 8)
+ // bit = pow(2,(bitflip & 7))
+ // if bitflip < (len(data)+len(recvecc))*8:
+ // if byte<len(data):
+ // data[byte] = data[byte] ^ bit
+ // else:
+ // recvecc[byte - len(data)] = recvecc[byte - len(data)] ^ bit
+ for bitflip in ecc_state.errloc.as_ref().unwrap() {
+ let byte = bitflip / 8;
+ let bit = 2_u32.pow(bitflip & 7);
+ if (*bitflip as usize) < (data.len() + corrected_ecc.len()) * 8 {
+ if (byte as usize) < data.len() {
+ data[byte as usize] ^= bit as u8;
+ } else {
+ corrected_ecc[(byte - data.len() as u32) as usize] ^= bit as u8;
+ }
+ }
+ }
+
+ let reencoded_ecc = bch_encode(ecc_state, data);
+ let ecc_bits = ecc_state.ecc_bits.unwrap() as usize;
+ let whole_bytes = ecc_bits / 8;
+ if reencoded_ecc[..whole_bytes] != corrected_ecc[..whole_bytes] {
+ return u8::MAX;
+ }
+ // Trailing byte-alignment bits are not protected BCH parity.
+ if !ecc_bits.is_multiple_of(8) {
+ let mask = u8::MAX << (8 - ecc_bits % 8);
+ if (reencoded_ecc[whole_bytes] ^ corrected_ecc[whole_bytes]) & mask != 0 {
+ return u8::MAX;
+ }
+ }
+
+ nroots as u8
+}
+
+impl EccState {
+ fn new(m: u32, t: u32, _poly: u32) -> Self {
+ Self {
+ m,
+ t,
+ _poly,
+ n: None,
+ ecc_bytes: None,
+ cyclic_tab: None,
+ syn: None,
+ elp: None,
+ errloc: None,
+ exponents: None,
+ logarithms: None,
+ elp_pre: None,
+ ecc_bits: None,
+ ecc_buf: None,
+ }
+ }
+
+ // def build_cyclic(self, g):
+ fn build_cyclic(&mut self, g: &[u32]) {
+ // l=self.ceilop(self.ECCstate.m*self.ECCstate.t, 32)
+ let l = (self.m * self.t).div_ceil(32);
+
+ // plen=self.ceilop(self.ECCstate.ecc_bits+1,32)
+ // ecclen=self.ceilop(self.ECCstate.ecc_bits,32)
+ let plen = (self.ecc_bits.unwrap() + 1).div_ceil(32);
+ let ecclen = self.ecc_bits.unwrap().div_ceil(32);
+
+ // self.ECCstate.cyclic_tab = [0] * 4*256*l
+ self.cyclic_tab = Some(vec![0; (4 * 256 * l).try_into().unwrap()]);
+
+ // for i in range(0,256):
+ for i in 0..256 {
+ // for b in range(0,4):
+ for b in 0..4 {
+ // offset= (b*256+i)*l
+ // data = i << 8*b
+ let offset = (b * 256 + i) * l;
+ let mut data = i << (8 * b);
+
+ // while (data):
+ while data != 0 {
+ // d=self.deg(data)
+ let d = data.ilog2();
+ // data = data ^ (g[0] >> (31-d))
+ data ^= g[0] >> (31 - d);
+ // for j in range(0,ecclen):
+ for j in 0..ecclen {
+ // if d<31:
+ // hi=(g[j] << (d+1)) & 0xffffffff
+ // else:
+ // hi=0
+ let hi = if d < 31 { g[j as usize] << (d + 1) } else { 0 };
+ // if j+1 < plen:
+ // lo= g[j+1] >> (31-d)
+ // else:
+ // lo= 0
+ let lo = if j + 1 < plen {
+ g[(j + 1) as usize] >> (31 - d)
+ } else {
+ 0
+ };
+ // self.ECCstate.cyclic_tab[j+offset] = self.ECCstate.cyclic_tab[j+offset] ^ (hi | lo)
+ self.cyclic_tab.as_mut().unwrap()[(j + offset) as usize] ^= hi | lo;
+ }
+ }
+ }
+ }
+ }
+
+ // def g_mul(self, a,b):
+
+ // if (a>0 and b>0):
+ // res=self.mod(self.ECCstate.logarithms[a]+self.ECCstate.logarithms[b])
+ // return (self.ECCstate.exponents[res])
+ // else:
+ // return 0
+ fn g_mul(&self, a: u32, b: u32) -> u32 {
+ if a > 0 && b > 0 {
+ let res = (self.logarithms.as_ref().unwrap()[a as usize]
+ + self.logarithms.as_ref().unwrap()[b as usize])
+ % self.n.unwrap();
+ self.exponents.as_ref().unwrap()[res as usize]
+ } else {
+ 0
+ }
+ }
+
+ // def g_pow(self, i):
+ // return self.ECCstate.exponents[self.modn(i)]
+ fn g_pow(&self, i: usize) -> u32 {
+ self.exponents.as_ref().unwrap()[self.modn(i.try_into().unwrap()) as usize]
+ }
+
+ // def modn(self, v):
+ // n=self.ECCstate.n
+ // while (v>=n):
+ // v -= n
+ // v = (v & n) + (v >> self.ECCstate.m)
+ // return v
+ fn modn(&self, mut v: u32) -> u32 {
+ while v >= self.n.unwrap() {
+ v -= self.n.unwrap();
+ v = (v & self.n.unwrap()) + (v >> self.m);
+ }
+ v
+ }
+
+ // def g_sqrt(self, a):
+ // if a:
+ // return self.ECCstate.exponents[self.mod(2*self.ECCstate.logarithms[a])]
+ // else:
+ // return 0
+ fn g_sqrt(&self, a: u32) -> u32 {
+ if a != 0 {
+ self.exponents.as_ref().unwrap()
+ [(2 * self.logarithms.as_ref().unwrap()[a as usize] % self.n.unwrap()) as usize]
+ } else {
+ 0
+ }
+ }
+
+ // def g_log(self, x):
+ // return self.ECCstate.logarithms[x]
+ fn g_log(&self, a: u32) -> u32 {
+ self.logarithms.as_ref().unwrap()[a as usize]
+ }
+
+ // def mod(self, v):
+ // if v<self.ECCstate.n:
+ // return v
+ // else:
+ // return v-self.ECCstate.n
+ fn r#mod(&self, v: u32) -> u32 {
+ if v < self.n.unwrap() {
+ v
+ } else {
+ v - self.n.unwrap()
+ }
+ }
+
+ // def g_div(self,a,b):
+ // if a:
+ // return self.ECCstate.exponents[self.mod(self.ECCstate.logarithms[a]+self.ECCstate.n-self.ECCstate.logarithms[b])]
+ // else:
+ // return 0
+ fn g_div(&self, a: u32, b: u32) -> u32 {
+ if a != 0 {
+ self.exponents.as_ref().unwrap()[self.r#mod(
+ self.logarithms.as_ref().unwrap()[a as usize] + self.n.unwrap()
+ - self.logarithms.as_ref().unwrap()[b as usize],
+ ) as usize]
+ } else {
+ 0
+ }
+ }
+
+ // def getroots(self, k, poly):
+ fn getroots(&mut self, mut k: u32, poly: &Polynomial) -> u32 {
+ // roots=[]
+ let mut roots = vec![];
+
+ // if poly.deg>2:
+ if poly.deg > 2 {
+ // k=k*8+self.ECCstate.ecc_bits
+ k = k * 8 + self.ecc_bits.unwrap();
+
+ // rep=[0]*(self.ECCstate.t*2)
+ // d=poly.deg
+ // l=self.ECCstate.n-self.g_log(poly.c[poly.deg])
+ let mut rep: Vec<i32> = vec![0; (self.t * 2) as usize];
+ let d = poly.deg;
+ let l = self.n.unwrap() - self.g_log(poly.c.as_ref().unwrap()[poly.deg as usize]);
+ // for i in range(0,d):
+ for i in 0..d {
+ // if poly.c[i]:
+ // rep[i]=self.mod(self.g_log(poly.c[i])+l)
+ // else:
+ // rep[i]=-1
+ if poly.c.as_ref().unwrap()[i as usize] != 0 {
+ rep[i as usize] =
+ self.r#mod(self.g_log(poly.c.as_ref().unwrap()[i as usize]) + l) as i32;
+ } else {
+ rep[i as usize] = -1;
+ }
+ }
+
+ // rep[poly.deg]=0
+ // syn0=self.g_div(poly.c[0],poly.c[poly.deg])
+ rep[poly.deg as usize] = 0;
+ let syn0 = self.g_div(
+ poly.c.as_ref().unwrap()[0],
+ poly.c.as_ref().unwrap()[poly.deg as usize],
+ );
+ // for i in range(self.ECCstate.n-k+1, self.ECCstate.n+1):
+ for i in self.n.unwrap() - k + 1..self.n.unwrap() + 1 {
+ // syn=syn0
+ let mut syn = syn0;
+ // for j in range(1,poly.deg+1):
+ // m=rep[j]
+ // if m>=0:
+ // syn = syn ^ self.g_pow(m+j*i)
+ for j in 1..poly.deg + 1 {
+ let m = rep[j as usize];
+ if m >= 0 {
+ syn ^= self.g_pow(((m as u32) + j * i) as usize);
+ }
+ }
+ // if syn==0:
+ // roots.append(self.ECCstate.n-i)
+ // if len(roots)==poly.deg:
+ // break
+ if syn == 0 {
+ roots.push(self.n.unwrap() - i);
+ if roots.len() == poly.deg as usize {
+ break;
+ }
+ }
+ }
+ // if len(roots)<poly.deg:
+ // # not enough roots to correct
+ // self.ECCstate.errloc=[]
+ // return -1
+ if roots.len() < poly.deg as usize {
+ self.errloc = Some(vec![]);
+ return u32::MAX;
+ }
+ }
+
+ // if poly.deg==1:
+ // if (poly.c[0]):
+ // roots.append(self.mod(self.ECCstate.n-self.ECCstate.logarithms[poly.c[0]]+self.ECCstate.logarithms[poly.c[1]]) )
+ if poly.deg == 1 && poly.c.as_ref().unwrap()[0] != 0 {
+ roots.push(self.r#mod(
+ self.n.unwrap()
+ - self.logarithms.as_ref().unwrap()[poly.c.as_ref().unwrap()[0] as usize]
+ + self.logarithms.as_ref().unwrap()[poly.c.as_ref().unwrap()[1] as usize],
+ ));
+ }
+
+ // if poly.deg==2:
+ if poly.deg == 2 {
+ // if (poly.c[0] and poly.c[1]):
+ if poly.c.as_ref().unwrap()[0] != 0 && poly.c.as_ref().unwrap()[1] != 0 {
+ // l0=self.ECCstate.logarithms[poly.c[0]]
+ // l1=self.ECCstate.logarithms[poly.c[1]]
+ // l2=self.ECCstate.logarithms[poly.c[2]]
+ let l0 = self.logarithms.as_ref().unwrap()[poly.c.as_ref().unwrap()[0] as usize];
+ let l1 = self.logarithms.as_ref().unwrap()[poly.c.as_ref().unwrap()[1] as usize];
+ let l2 = self.logarithms.as_ref().unwrap()[poly.c.as_ref().unwrap()[2] as usize];
+
+ // u=self.g_pow(l0+l2+2*(self.ECCstate.n-l1))
+ // r=0
+ // v=u
+ let u = self.g_pow((l0 + l2 + 2 * (self.n.unwrap() - l1)) as usize);
+ let mut r = 0;
+ let mut v = u;
+ // while (v):
+ // i=self.deg(v)
+ // r = r ^ self.ECCstate.elp_pre[i]
+ // v = v ^ pow(2,i)
+ while v != 0 {
+ let i = v.ilog(2);
+ r ^= self.elp_pre.as_ref().unwrap()[i as usize];
+ v ^= 2_u32.pow(i);
+ }
+ // if self.g_sqrt(r)^r == u:
+ // roots.append(self.modn(2*self.ECCstate.n-l1-self.ECCstate.logarithms[r]+l2))
+ // roots.append(self.modn(2*self.ECCstate.n-l1-self.ECCstate.logarithms[r^1]+l2))
+ if self.g_sqrt(r) ^ r == u {
+ roots.push(self.modn(
+ 2 * self.n.unwrap() - l1 - self.logarithms.as_ref().unwrap()[r as usize]
+ + l2,
+ ));
+ roots.push(self.modn(
+ 2 * self.n.unwrap()
+ - l1
+ - self.logarithms.as_ref().unwrap()[(r ^ 1) as usize]
+ + l2,
+ ));
+ }
+ }
+ }
+
+ // self.ECCstate.errloc=roots
+ // return len(roots)
+ let lenroots = roots.len() as u32;
+ self.errloc = Some(roots);
+ lenroots
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn init() {
+ let ecc_state = bch_init(5, 137);
+ assert_eq!(ecc_state.m, 7);
+ assert_eq!(ecc_state.t, 5);
+ assert_eq!(ecc_state._poly, 137);
+ assert_eq!(ecc_state.n, Some(127));
+ assert_eq!(ecc_state.ecc_bytes, Some(5));
+ assert_eq!(
+ ecc_state.cyclic_tab.as_ref().map(|x| &x[..5]),
+ Some([0_u32, 0, 2498495642, 3758096384, 3174305199].as_slice())
+ );
+ assert_eq!(
+ ecc_state.cyclic_tab.as_ref().map(|x| &x[x.len() - 3..]),
+ Some([0, 1839291269, 3758096384].as_slice())
+ );
+ assert_eq!(
+ ecc_state.syn.as_ref().map(|x| &x[..]),
+ Some([0; 10].as_slice())
+ );
+ assert_eq!(
+ ecc_state.elp.as_ref().map(|x| &x[..]),
+ Some([0; 6].as_slice())
+ );
+ assert_eq!(
+ ecc_state.errloc.as_ref().map(|x| &x[..]),
+ Some([0; 5].as_slice())
+ );
+ assert_eq!(
+ ecc_state.exponents.as_ref().map(|x| &x[..10]),
+ Some([1, 2, 4, 8, 16, 32, 64, 9, 18, 36].as_slice())
+ );
+ assert_eq!(
+ ecc_state.logarithms.as_ref().map(|x| &x[..10]),
+ Some([0, 0, 1, 31, 2, 62, 32, 103, 3, 7].as_slice())
+ );
+ assert_eq!(
+ ecc_state.elp_pre.as_ref().map(|x| &x[..]),
+ Some([0, 16, 18, 102, 22, 40, 110, 0].as_slice())
+ );
+ assert_eq!(ecc_state.ecc_bits, Some(35),);
+ }
+
+ #[test]
+ fn encode_zeros() {
+ let mut ecc_state = bch_init(8, 137);
+ let ecc = bch_encode(&mut ecc_state, &[0, 0, 0, 0, 0, 0, 0, 0]);
+ assert_eq!(ecc, &[0, 0, 0, 0, 0, 0, 0]);
+ }
+
+ #[test]
+ fn encode_data() {
+ let mut ecc_state = bch_init(4, 137);
+ let ecc = bch_encode(&mut ecc_state, &[133, 20, 228, 249, 11, 172, 165, 151, 0]);
+ assert_eq!(ecc, &[115, 32, 10, 0]);
+ }
+
+ #[test]
+ fn quadratic_zero_roots_are_rejected() {
+ let packet = "0000101101110101001111100000111101110001010010000000110001111101101000001101100101010010111111011111";
+ let pack = |bits: &str| {
+ bits.as_bytes()
+ .chunks(8)
+ .map(|chunk| {
+ chunk.iter().enumerate().fold(0_u8, |byte, (index, bit)| {
+ byte | ((bit - b'0') << (7 - index))
+ })
+ })
+ .collect::<Vec<_>>()
+ };
+ let mut data = pack(&packet[..75]);
+ let ecc = pack(&packet[75..96]);
+ let mut ecc_state = bch_init(3, POLYNOMIAL);
+ assert_eq!(bch_decode(&mut ecc_state, &mut data, &ecc), u8::MAX);
+ assert!(ecc_state.errloc.as_ref().unwrap().is_empty());
+ }
+
+ #[test]
+ fn byte_payloads_keep_the_full_correction_radius() {
+ for radius in [8, 5, 4, 3] {
+ let mut ecc_state = bch_init(radius, POLYNOMIAL);
+ for size in [1, 5, 8] {
+ let original: Vec<_> = (0..size)
+ .map(|index| (index as u8).wrapping_mul(37) | 31)
+ .collect();
+ let parity = bch_encode(&mut ecc_state, &original);
+ for flips in 0..=radius as usize {
+ for layout in 0..3 {
+ let mut data = original.clone();
+ let mut ecc = parity.clone();
+ for index in 0..flips {
+ let location = match layout {
+ 0 => index,
+ 1 => size * 8 + index,
+ _ if index % 2 == 1 => size * 8 + index / 2,
+ _ => index / 2,
+ };
+ if location < size * 8 {
+ data[location / 8] ^= 1 << (7 - location % 8);
+ } else {
+ let bit = location - size * 8;
+ ecc[bit / 8] ^= 1 << (7 - bit % 8);
+ }
+ }
+ assert_eq!(
+ bch_decode(&mut ecc_state, &mut data, &ecc),
+ flips as u8,
+ "radius={radius} size={size} flips={flips} layout={layout}"
+ );
+ assert_eq!(data, original);
+ }
+ }
+ }
+ }
+ }
+
+ #[test]
+ fn ecc_byte_padding_remains_ignored_during_correction() {
+ for radius in [8, 5, 4, 3] {
+ let mut ecc_state = bch_init(radius, POLYNOMIAL);
+ let original = [255, 7];
+ let mut parity = bch_encode(&mut ecc_state, &original);
+ let padding = parity.len() * 8 - ecc_state.ecc_bits.unwrap() as usize;
+ if padding != 0 {
+ *parity.last_mut().unwrap() |= ((1_u16 << padding) - 1) as u8;
+ }
+ let mut data = original;
+ data[0] ^= 128;
+ parity[0] ^= 128;
+ assert_eq!(bch_decode(&mut ecc_state, &mut data, &parity), 2);
+ assert_eq!(data, original);
+ }
+ }
+}
diff --git a/crates/qls-core/src/watermark/mod.rs b/crates/qls-core/src/watermark/mod.rs
new file mode 100644
index 0000000..858ac56
--- /dev/null
+++ b/crates/qls-core/src/watermark/mod.rs
@@ -0,0 +1,446 @@
+//! The invisible watermark in every image Kokon makes or changes with the model (AI Act Art. 50(2):
+//! the second marking layer next to the XMP of provenance.rs; the Code of Practice on marking
+//! AI-generated content, Sub-measure 1.1.2). It is Adobe's TrustMark, variant P (the "perceptual"
+//! one, ~50 dB PSNR: not visible), MIT code and weights, listed in C2PA's soft-binding algorithm
+//! list (`com.adobe.trustmark.P`), so anyone can read it with Adobe's open decoder:
+//! `TrustMark(model_type='P').decode(img)` returns the text "Kokon-AI" (docs/ai-marking.md).
+//!
+//! This is a port of the Python reference (`python/trustmark/trustmark.py` and `datalayer.py`,
+//! adobe/trustmark, MIT) on our own ONNX Runtime (the `trustmark` crate pins an older `ort`):
+//! the centre square of the image, resized to 256², goes through the encoder with the payload;
+//! the change it makes (its mean per channel removed) is scaled back to the square and added,
+//! the edges feathered. The decoder reads the centre square at 224². The BCH error correction is
+//! vendored from the Rust crate (bch.rs, MIT).
+
+mod bch;
+
+use std::path::{Path, PathBuf};
+use std::sync::Mutex;
+
+use image::{RgbaImage, imageops::FilterType};
+use ort::session::Session;
+use ort::session::builder::GraphOptimizationLevel;
+use ort::value::Tensor;
+
+use crate::catalog::{Download, join_rel};
+use crate::error::{AppError, AppResult, ErrorCode};
+
+/// The text Kokon writes in the watermark (TrustMark text mode: 7-bit ASCII, BCH_5). Adobe's
+/// decoder shows it as is. A future format would change the last character.
+pub const PAYLOAD: &str = "Kokon-AI";
+
+/// TrustMark P (catalog.rs: shipped inside the app, Adobe's host as the fallback): encoder, decoder.
+pub const TRUSTMARK_P: [Download; 2] = [
+ crate::catalog::TRUSTMARK_P_ENCODER,
+ crate::catalog::TRUSTMARK_P_DECODER,
+];
+
+/// Both model files are there.
+pub fn installed(sdcpp_dir: &Path) -> bool {
+ TRUSTMARK_P.iter().all(|d| d.is_present(sdcpp_dir))
+}
+
+/// The variant's sizes and strength (P: the encoder at 256², the decoder at 224², strength x1.25).
+const ENC_SIDE: u32 = 256;
+const DEC_SIDE: u32 = 224;
+const P_STRENGTH: f32 = 1.25;
+/// Kokon's strength (the reference's default is 1.0). Measured on Kokon outputs (see
+/// docs/ai-marking.md): invisible, and read back after JPEG 75, half size, a 10 % crop.
+pub const STRENGTH: f32 = 1.0;
+/// Below this side the reference is not reliable: smaller images are left unmarked (Kokon never
+/// makes them; XMP still marks them).
+pub const MIN_SIDE: u32 = 150;
+/// The mark fades in over this share of the square's side from each edge (smoothstep). The
+/// reference feathers 1 % (at most 50 px), which leaves a faint seam on flat colour (seen on a
+/// purple background, 2026-11-04); the decoder reads the whole square, so a wider fade costs little.
+const FEATHER: f32 = 0.06;
+
+/// What the watermark check found.
+#[derive(Debug, Clone, PartialEq, serde::Serialize)]
+pub struct Found {
+ /// The text read (only when the error correction accepted it).
+ pub text: String,
+ /// TrustMark's error-correction schema (BCH_SUPER, BCH_5, BCH_4, BCH_3).
+ pub schema: &'static str,
+ /// It is Kokon's mark.
+ pub kokon: bool,
+}
+
+struct Sessions {
+ _env: ort::environment::Environment,
+ dir: PathBuf,
+ encoder: Option<Session>,
+ decoder: Option<Session>,
+}
+
+/// Loaded on first use and kept (~150 MB of RAM); each model only when needed.
+static SESSIONS: Mutex<Option<Sessions>> = Mutex::new(None);
+
+fn ort_err(e: impl std::fmt::Display) -> AppError {
+ AppError::new(ErrorCode::Internal, format!("The AI watermark failed: {e}"))
+}
+
+fn with_session<T>(
+ sdcpp_dir: &Path,
+ decoder: bool,
+ f: impl FnOnce(&mut Session) -> AppResult<T>,
+) -> AppResult<T> {
+ let d = &TRUSTMARK_P[decoder as usize];
+ let path = join_rel(sdcpp_dir, d.dest);
+ if !path.is_file() {
+ return Err(AppError::new(
+ ErrorCode::MissingModel,
+ "The AI watermark model is not installed.",
+ ));
+ }
+ let mut guard = SESSIONS.lock().unwrap_or_else(|p| p.into_inner());
+ if guard.as_ref().is_none_or(|s| s.dir != sdcpp_dir) {
+ let env = crate::onnx::environment().map_err(ort_err)?;
+ *guard = Some(Sessions {
+ _env: env,
+ dir: sdcpp_dir.to_path_buf(),
+ encoder: None,
+ decoder: None,
+ });
+ }
+ let s = guard.as_mut().unwrap();
+ let slot = if decoder {
+ &mut s.decoder
+ } else {
+ &mut s.encoder
+ };
+ if slot.is_none() {
+ let threads = std::thread::available_parallelism()
+ .map(|n| n.get())
+ .unwrap_or(4)
+ .min(8);
+ let session = Session::builder(&s._env)
+ .map_err(ort_err)?
+ .with_optimization_level(GraphOptimizationLevel::Level3)
+ .map_err(ort_err)?
+ .with_intra_threads(threads)
+ .map_err(ort_err)?
+ .commit_from_file(&path)
+ .map_err(ort_err)?;
+ *slot = Some(session);
+ }
+ f(slot.as_mut().unwrap())
+}
+
+/// The centre square the variant works on: (left, top, side).
+fn centre_square(w: u32, h: u32) -> (u32, u32, u32) {
+ let m = w.min(h);
+ ((w - m) / 2, (h - m) / 2, m)
+}
+
+/// RGB of `img` (alpha ignored) resized to `side`² (bilinear with antialiasing, as PIL's), in
+/// [-1, 1], CHW.
+fn model_input(img: &RgbaImage, side: u32) -> Vec<f32> {
+ let small = image::imageops::resize(img, side, side, FilterType::Triangle);
+ let n = (side * side) as usize;
+ let mut x = vec![0f32; 3 * n];
+ for (i, p) in small.pixels().enumerate() {
+ for k in 0..3 {
+ x[k * n + i] = p.0[k] as f32 / 127.5 - 1.0;
+ }
+ }
+ x
+}
+
+/// Bilinear resize of one CHW plane set (align_corners = false, as torch's `interpolate`).
+fn upsample(src: &[f32], sw: usize, sh: usize, dw: usize, dh: usize) -> Vec<f32> {
+ let mut out = vec![0f32; 3 * dw * dh];
+ let axis = |d: usize, dn: usize, sn: usize| {
+ let s = ((d as f32 + 0.5) * sn as f32 / dn as f32 - 0.5).max(0.0);
+ let i0 = (s.floor() as usize).min(sn - 1);
+ let i1 = (i0 + 1).min(sn - 1);
+ (i0, i1, s - i0 as f32)
+ };
+ let xs: Vec<_> = (0..dw).map(|x| axis(x, dw, sw)).collect();
+ for c in 0..3 {
+ let plane = &src[c * sw * sh..(c + 1) * sw * sh];
+ for y in 0..dh {
+ let (y0, y1, fy) = axis(y, dh, sh);
+ let (r0, r1) = (
+ &plane[y0 * sw..(y0 + 1) * sw],
+ &plane[y1 * sw..(y1 + 1) * sw],
+ );
+ let o = &mut out[c * dw * dh + y * dw..c * dw * dh + (y + 1) * dw];
+ for (x, &(x0, x1, fx)) in xs.iter().enumerate() {
+ let top = r0[x0] + (r0[x1] - r0[x0]) * fx;
+ let bot = r1[x0] + (r1[x1] - r1[x0]) * fx;
+ o[x] = top + (bot - top) * fy;
+ }
+ }
+ }
+ out
+}
+
+/// The 100 bits TrustMark carries for `text` in text mode with BCH_5 (datalayer.py): 61 data
+/// bits (7-bit ASCII, zero padded), 35 error-correction bits, then the schema "0001".
+fn payload_bits(text: &str) -> Vec<f32> {
+ const DATA: usize = 61;
+ const ECC: usize = 35;
+ let mut data: String = text
+ .chars()
+ .map(|c| format!("{:07b}", (c as u32) & 127))
+ .collect();
+ data.truncate(DATA);
+ data.push_str(&"0".repeat(DATA - data.len()));
+ let padded = format!("{data}{}", "0".repeat((8 - DATA % 8) % 8));
+ let bytes: Vec<u8> = padded
+ .as_bytes()
+ .chunks(8)
+ .map(|c| u8::from_str_radix(std::str::from_utf8(c).unwrap(), 2).unwrap())
+ .collect();
+ let mut state = bch::bch_init(5, bch::POLYNOMIAL);
+ let ecc = bch::bch_encode(&mut state, &bytes);
+ let mut ecc: String = ecc.iter().map(|b| format!("{b:08b}")).collect();
+ ecc.truncate(ECC);
+ format!("{data}{ecc}0001")
+ .chars()
+ .map(|c| if c == '1' { 1.0 } else { 0.0 })
+ .collect()
+}
+
+/// The schemas: (name, data bits, bit flips corrected).
+const SCHEMAS: [(&str, usize, u32); 4] = [
+ ("BCH_SUPER", 40, 8),
+ ("BCH_5", 61, 5),
+ ("BCH_4", 68, 4),
+ ("BCH_3", 75, 3),
+];
+
+/// Reads 100 decoder bits as datalayer.py does: the schema from the last 2 bits; the error
+/// correction must accept the data (and its padding stay zero), else nothing was found. Kokon's
+/// mark is only accepted as BCH_5 with exactly its text, so a photo whose noise happens to pass a
+/// weaker check is never called Kokon's.
+fn read_bits(bits: &[bool]) -> Option<Found> {
+ if bits.len() != 100 {
+ return None;
+ }
+ let version = (bits[98] as usize) << 1 | bits[99] as usize;
+ let (schema, data_bits, t) = SCHEMAS[version];
+ let ecc_bits = 100 - 4 - data_bits;
+ let pack = |b: &[bool]| -> Vec<u8> {
+ b.chunks(8)
+ .map(|c| {
+ c.iter()
+ .enumerate()
+ .fold(0u8, |acc, (i, &v)| acc | ((v as u8) << (7 - i)))
+ })
+ .collect()
+ };
+ let mut data = pack(&bits[..data_bits]);
+ let ecc = pack(&bits[data_bits..data_bits + ecc_bits]);
+ let mut state = bch::bch_init(t, bch::POLYNOMIAL);
+ let flips = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
+ bch::bch_decode(&mut state, &mut data, &ecc)
+ }))
+ .ok()?;
+ let pad = (8 - data_bits % 8) % 8;
+ if flips as u32 > t || (pad != 0 && data.last().copied().unwrap_or(0) & ((1u8 << pad) - 1) != 0)
+ {
+ return None;
+ }
+ let all: String = data.iter().map(|b| format!("{b:08b}")).collect();
+ let text: String = all
+ .as_bytes()
+ .chunks(7)
+ .filter(|c| c.len() == 7)
+ .map(|c| u8::from_str_radix(std::str::from_utf8(c).unwrap(), 2).unwrap() as char)
+ .collect();
+ let text = text.trim_end_matches('\0').trim().to_string();
+ let kokon = schema == "BCH_5" && text == PAYLOAD;
+ Some(Found {
+ text,
+ schema,
+ kokon,
+ })
+}
+
+/// Reads the watermark of `img` (None: no TrustMark P watermark the error correction accepts).
+pub fn detect(sdcpp_dir: &Path, img: &RgbaImage) -> AppResult<Option<Found>> {
+ let (w, h) = img.dimensions();
+ if w.min(h) < 16 {
+ return Ok(None);
+ }
+ let (left, top, m) = centre_square(w, h);
+ let square = image::imageops::crop_imm(img, left, top, m, m).to_image();
+ let x = model_input(&square, DEC_SIDE);
+ let logits = with_session(sdcpp_dir, true, |s| {
+ let input = Tensor::from_array((
+ [1usize, 3, DEC_SIDE as usize, DEC_SIDE as usize],
+ x.into_boxed_slice(),
+ ))
+ .map_err(ort_err)?;
+ let out = s.run(ort::inputs!["image" => input]).map_err(ort_err)?;
+ let (_, v) = out["output"].try_extract_tensor::<f32>().map_err(ort_err)?;
+ Ok(v.to_vec())
+ })?;
+ let bits: Vec<bool> = logits.iter().map(|&v| v > 0.0).collect();
+ Ok(read_bits(&bits))
+}
+
+/// `img` with Kokon's watermark, or None when it already carries it (an edit of a Kokon image
+/// keeps the mark it has: marking twice would only add noise) or is too small to carry one.
+pub fn embed(sdcpp_dir: &Path, img: &RgbaImage) -> AppResult<Option<RgbaImage>> {
+ let (w, h) = img.dimensions();
+ if w.min(h) < MIN_SIDE {
+ return Ok(None);
+ }
+ if detect(sdcpp_dir, img)?.is_some_and(|f| f.kokon) {
+ return Ok(None);
+ }
+ let (left, top, m) = centre_square(w, h);
+ let square = image::imageops::crop_imm(img, left, top, m, m).to_image();
+ let cover = model_input(&square, ENC_SIDE);
+ let bits = payload_bits(PAYLOAD);
+ let side = ENC_SIDE as usize;
+ let stego = with_session(sdcpp_dir, false, |s| {
+ let image = Tensor::from_array(([1usize, 3, side, side], cover.clone().into_boxed_slice()))
+ .map_err(ort_err)?;
+ let secret =
+ Tensor::from_array(([1usize, 100], bits.into_boxed_slice())).map_err(ort_err)?;
+ let out = s
+ .run(ort::inputs!["onnx::Concat_0" => image, "onnx::Gemm_1" => secret])
+ .map_err(ort_err)?;
+ let (_, v) = out["image"].try_extract_tensor::<f32>().map_err(ort_err)?;
+ Ok(v.to_vec())
+ })?;
+ // The change, without a colour shift (its mean per channel removed), at the square's size.
+ let n = side * side;
+ let mut residual: Vec<f32> = stego
+ .iter()
+ .zip(&cover)
+ .map(|(s, c)| s.clamp(-1.0, 1.0) - c)
+ .collect();
+ for c in 0..3 {
+ let plane = &mut residual[c * n..(c + 1) * n];
+ let mean = plane.iter().sum::<f32>() / n as f32;
+ plane.iter_mut().for_each(|v| *v -= mean);
+ }
+ let mm = m as usize;
+ let up = upsample(&residual, side, side, mm, mm);
+ let k = STRENGTH * P_STRENGTH;
+ let band = (m as f32 * FEATHER).max(1.0);
+ let ramp = |i: usize| {
+ let t = ((i as f32 + 0.5) / band).min(1.0);
+ t * t * (3.0 - 2.0 * t)
+ };
+ let mut out = img.clone();
+ for y in 0..mm {
+ let ay = ramp(y.min(mm - 1 - y));
+ for x in 0..mm {
+ let a = ay * ramp(x.min(mm - 1 - x));
+ let p = out.get_pixel_mut(left + x as u32, top + y as u32);
+ for c in 0..3 {
+ let orig = p.0[c] as f32;
+ let marked = ((up[c * mm * mm + y * mm + x] * k + orig / 127.5 - 1.0)
+ .clamp(-1.0, 1.0)
+ + 1.0)
+ * 127.5;
+ p.0[c] = (a * marked + (1.0 - a) * orig).round().clamp(0.0, 255.0) as u8;
+ }
+ }
+ }
+ Ok(Some(out))
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ fn bools(v: &[f32]) -> Vec<bool> {
+ v.iter().map(|&b| b > 0.5).collect()
+ }
+
+ #[test]
+ fn payload_round_trips_and_survives_five_flips() {
+ let bits = payload_bits(PAYLOAD);
+ assert_eq!(bits.len(), 100);
+ assert_eq!(&bools(&bits)[96..], &[false, false, false, true]);
+ let found = read_bits(&bools(&bits)).unwrap();
+ assert_eq!(
+ found,
+ Found {
+ text: PAYLOAD.into(),
+ schema: "BCH_5",
+ kokon: true
+ }
+ );
+ let mut b = bools(&bits);
+ for i in [0, 13, 40, 60, 77] {
+ b[i] = !b[i];
+ }
+ assert!(read_bits(&b).unwrap().kokon);
+ for i in [5, 20, 33, 50, 70, 90] {
+ b[i] = !b[i];
+ }
+ assert!(read_bits(&b).is_none_or(|f| !f.kokon));
+ }
+
+ #[test]
+ fn random_bits_are_never_kokon() {
+ // A cheap LCG: no rand dependency.
+ let mut s: u64 = 0x2545F4914F6CDD1D;
+ let mut accepted = 0;
+ for _ in 0..20_000 {
+ let b: Vec<bool> = (0..100)
+ .map(|_| {
+ s = s
+ .wrapping_mul(6364136223846793005)
+ .wrapping_add(1442695040888963407);
+ s >> 63 == 1
+ })
+ .collect();
+ if let Some(f) = read_bits(&b) {
+ accepted += 1;
+ assert!(!f.kokon);
+ }
+ }
+ // Weaker schemas sometimes accept noise (datalayer.py does too): never as Kokon's mark.
+ assert!(accepted < 2_000, "{accepted}");
+ }
+
+ #[test]
+ fn upsample_matches_torch_bilinear() {
+ // torch.nn.functional.interpolate([[0, 1]], size=4, mode="bilinear", align_corners=False)
+ // along x: [0, 0.25, 0.75, 1].
+ let src = vec![0.0, 1.0, 0.0, 1.0, 0.0, 1.0];
+ let out = upsample(&src, 2, 1, 4, 1);
+ assert_eq!(&out[..4], &[0.0, 0.25, 0.75, 1.0]);
+ }
+
+ /// With the models in KOKON_TEST_MODELS (a folder holding models/watermark/*.onnx): marks,
+ /// reads back, does not mark twice, survives JPEG 75 and half size, and leaves alpha alone.
+ #[test]
+ fn embeds_and_detects_with_the_models() {
+ let Ok(dir) = std::env::var("KOKON_TEST_MODELS") else {
+ return;
+ };
+ let dir = PathBuf::from(dir);
+ let img = RgbaImage::from_fn(640, 480, |x, y| {
+ image::Rgba([
+ (x / 3) as u8,
+ (y / 2) as u8,
+ ((x + y) / 5) as u8,
+ if x < 20 { 0 } else { 255 },
+ ])
+ });
+ assert!(detect(&dir, &img).unwrap().is_none_or(|f| !f.kokon));
+ let marked = embed(&dir, &img).unwrap().unwrap();
+ assert!(detect(&dir, &marked).unwrap().unwrap().kokon);
+ assert!(embed(&dir, &marked).unwrap().is_none());
+ assert!(marked.pixels().zip(img.pixels()).all(|(a, b)| a[3] == b[3]));
+ let jpg = crate::imaging::encode_jpeg(&marked, 75).unwrap();
+ assert!(
+ detect(&dir, &image::load_from_memory(&jpg).unwrap().to_rgba8())
+ .unwrap()
+ .unwrap()
+ .kokon
+ );
+ let half = image::imageops::resize(&marked, 320, 240, FilterType::Triangle);
+ assert!(detect(&dir, &half).unwrap().unwrap().kokon);
+ }
+}
diff --git a/crates/qls-service/src/api.rs b/crates/qls-service/src/api.rs
index ebf86c0..10b2e85 100644
--- a/crates/qls-service/src/api.rs
+++ b/crates/qls-service/src/api.rs
@@ -82,6 +82,9 @@ pub fn router(state: Arc<AppState>, lan: bool) -> Router {
.route("/upscalers/{name}/download", post(download_upscaler))
.route("/matte", get(matte_status).post(matte_image))
.route("/matte/download", post(matte_download))
+ .route("/segment", get(segment_status))
+ .route("/segment/download", post(segment_download))
+ .route("/segment/models/{file}", get(segment_model))
.route("/config", get(get_config).put(put_config))
.route("/enhancer/defaults", get(enhancer_defaults))
.route("/mac-engine/bench", get(mac_bench_status).post(mac_bench_start).delete(mac_bench_clear))
@@ -136,7 +139,10 @@ pub fn router(state: Arc<AppState>, lan: bool) -> Router {
.route("/remote/password", post(remote_password).delete(remote_password_clear))
.layer(middleware::from_fn_with_state(ctx.clone(), auth))
.layer(DefaultBodyLimit::max(4 * 1024 * 1024));
- let uploads = Router::new().route("/inputs", post(upload_input)).layer(DefaultBodyLimit::max(210 * 1024 * 1024));
+ let uploads = Router::new()
+ .route("/inputs", post(upload_input))
+ .route("/provenance/check", post(check_provenance))
+ .layer(DefaultBodyLimit::max(210 * 1024 * 1024));
Router::new()
.route("/api/v1/ping", get(ping))
.route("/api/v1/auth/session", get(web::session))
@@ -278,7 +284,11 @@ async fn status(State(s): St) -> Json<Value> {
async fn setup(State(s): St) -> ApiResult<Json<Value>> {
let e = s.engine.clone();
- let _ = tokio::task::spawn_blocking(move || e.place_bundled_upscaler()).await;
+ let _ = tokio::task::spawn_blocking(move || {
+ e.place_bundled_upscaler();
+ e.place_bundled_watermark();
+ })
+ .await;
let cfg = s.engine.config();
let srv_ok = s.engine.server.status().state == qls_core::process::ServerState::Ready;
let e = s.engine.clone();
@@ -440,7 +450,11 @@ async fn install_start(State(s): St, Json(b): Json<InstallBody>) -> ApiResult<Js
});
// The bundled upscaler first, so it is not downloaded again.
let e = s.engine.clone();
- let _ = tokio::task::spawn_blocking(move || e.place_bundled_upscaler()).await;
+ let _ = tokio::task::spawn_blocking(move || {
+ e.place_bundled_upscaler();
+ e.place_bundled_watermark();
+ })
+ .await;
let label = if user { format!("user:{}", preset.id) } else { preset.id.to_string() };
s.engine.installer.start(cfg.sdcpp_dir.clone(), &label, downloads, on_finish)?;
Ok(Json(serde_json::to_value(s.engine.installer.status()).unwrap()))
@@ -481,6 +495,27 @@ async fn matte_download(State(s): St) -> ApiResult<Json<Value>> {
Ok(Json(serde_json::to_value(s.engine.installer.status()).unwrap()))
}
+/// The layers lab's click-to-select model (MobileSAM, run in the webview): there or not, and its size.
+async fn segment_status(State(s): St) -> Json<Value> {
+ let dir = s.engine.config().sdcpp_dir;
+ let bytes: u64 = qls_core::matting::MOBILE_SAM.iter().map(|d| d.size).sum();
+ Json(json!({ "installed": qls_core::matting::sam_installed(&dir), "bytes": bytes }))
+}
+
+async fn segment_download(State(s): St) -> ApiResult<Json<Value>> {
+ let dir = s.engine.config().sdcpp_dir;
+ s.engine.installer.start(dir, "segment", qls_core::matting::MOBILE_SAM.to_vec(), Box::new(|_| {}))?;
+ Ok(Json(serde_json::to_value(s.engine.installer.status()).unwrap()))
+}
+
+/// One MobileSAM file, for onnxruntime-web in the webview (only these two names are served).
+async fn segment_model(State(s): St, AxPath(file): AxPath<String>) -> ApiResult<Response> {
+ let path = qls_core::matting::sam_file(&s.engine.config().sdcpp_dir, &file)
+ .filter(|p| p.is_file())
+ .ok_or_else(|| AppError::new(ErrorCode::NotFound, "The selection model is not downloaded."))?;
+ serve_file(&path, Some("application/octet-stream")).await
+}
+
#[derive(Deserialize)]
struct MatteBody {
/// "input:<id>" / "gen:<id>".
@@ -490,8 +525,15 @@ struct MatteBody {
/// The matte of an image (grey PNG at its size) for a fast sticker.
async fn matte_image(State(s): St, Json(b): Json<MatteBody>) -> ApiResult<Response> {
let engine = s.engine.clone();
- let png = tokio::task::spawn_blocking(move || engine.matte(&b.image)).await.map_err(AppError::internal)??;
- Ok(([(header::CONTENT_TYPE, "image/png")], png).into_response())
+ let image = b.image.clone();
+ let started = std::time::Instant::now();
+ let res = tokio::task::spawn_blocking(move || engine.matte(&b.image)).await.map_err(AppError::internal)?;
+ // Logged: a quick cut-out that fails or crawls is otherwise invisible (the lab runs it in the window).
+ match &res {
+ Ok(_) => tracing::info!("quick cut-out matte of {image} in {} ms", started.elapsed().as_millis()),
+ Err(e) => tracing::warn!("quick cut-out matte of {image} failed after {} ms: {} {:?}", started.elapsed().as_millis(), e.message, e.detail),
+ }
+ Ok(([(header::CONTENT_TYPE, "image/png")], res?).into_response())
}
/// Upscaler models of earlier versions in the app's own folder (Settings offers to delete
@@ -959,6 +1001,14 @@ struct PrivateQ {
private: bool,
}
+/// "Check an image": what its AI markings say (provenance::check). The image is read in memory
+/// and never stored (the Code of Practice's zero retention for detection).
+async fn check_provenance(State(s): St, body: Bytes) -> ApiResult<Json<Value>> {
+ let dir = s.engine.config().sdcpp_dir;
+ let report = tokio::task::spawn_blocking(move || qls_core::provenance::check(&dir, &body)).await.map_err(AppError::internal)??;
+ Ok(Json(report))
+}
+
async fn upload_input(State(s): St, Query(q): Query<PrivateQ>, headers: HeaderMap, body: Bytes) -> ApiResult<Json<Value>> {
if body.is_empty() {
return Err(AppError::invalid("The upload was empty.").into());
diff --git a/crates/qls-service/src/main.rs b/crates/qls-service/src/main.rs
index 38ce0a0..1e92575 100644
--- a/crates/qls-service/src/main.rs
+++ b/crates/qls-service/src/main.rs
@@ -295,7 +295,11 @@ async fn run(args: Vec<String>) {
};
// The upscaler ships with the app: put it where sd-server looks (no download needed).
let e = engine.clone();
- let _ = tokio::task::spawn_blocking(move || e.place_bundled_upscaler()).await;
+ let _ = tokio::task::spawn_blocking(move || {
+ e.place_bundled_upscaler();
+ e.place_bundled_watermark();
+ })
+ .await;
// The original VAE or Q4_K model left behind when a busy server held it open during the update.
engine.remove_unused_original_vae();
engine.remove_unused_old_dit();
diff --git a/docs/STATUS.md b/docs/STATUS.md
index 7e959f6..bfcd73a 100644
--- a/docs/STATUS.md
+++ b/docs/STATUS.md
@@ -1,6 +1,6 @@
# Status
-**Updated 2026-10-28.** What is in progress, what comes next, and where help is welcome. Kept current at the end
+**Updated 2026-11-04.** What is in progress, what comes next, and where help is welcome. Kept current at the end
of every task (see [AGENTS.md](../AGENTS.md#keeping-docsstatusmd-current)); GitHub is the source of truth.
## Current version: [0.11.2](https://github.com/GabrieleLozupone/kokon/releases/tag/v0.11.2) (2026-10-28)
@@ -9,7 +9,11 @@ Windows, macOS (Apple Silicon, 15+) and Linux (`.deb`, `.rpm`). Installed copies
## Next
-- **0.12: reliability foundations** (details soon).
+- **0.12: reliability foundations** (details soon). Being finished for it: Prompt+ for brushed areas and Expand
+ (it can name what to remove or write what the new borders show, for you to approve), Delete for several versions
+ of a project at once, a clearer project list, the click-to-select model downloaded on first use, and AI marking
+ as the EU AI Act asks: the IPTC Digital Source Type in every file's XMP plus an invisible TrustMark watermark,
+ and Settings → General → Check an image ([docs/ai-marking.md](ai-marking.md)).
- After 0.11: the desktop app as a client of a Kokon server elsewhere, and a Docker image for AMD and Intel
(Vulkan) once it can be tested.
- On the list, no dates yet: a custom upscaler picker
diff --git a/docs/ai-marking.md b/docs/ai-marking.md
new file mode 100644
index 0000000..ddc815b
--- /dev/null
+++ b/docs/ai-marking.md
@@ -0,0 +1,135 @@
+# AI marking
+
+Every image Kokon makes or changes with its model is marked as AI-generated in two ways, as
+Article 50(2) of the EU AI Act asks of generative AI systems: a line in the file's metadata, and an
+invisible watermark in the pixels. This page says what the marks contain, how anyone can read them,
+and how well the watermark survives common changes. It follows the European Commission's
+[Code of Practice on marking and labelling AI-generated content](https://digital-strategy.ec.europa.eu/en/policies/code-practice-ai-generated-content)
+(June 2026) and its [guidelines on Article 50](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-50) (July 2026).
+
+**Please don't remove or alter these marks** when you share Kokon images, except for legitimate
+reasons such as security research. They tell people the image was made with AI.
+
+## What is marked
+
+| Result | Metadata (IPTC Digital Source Type) | Watermark |
+|---|---|---|
+| Generate, Compose | `trainedAlgorithmicMedia` | yes |
+| Edit, brushed areas, Expand, references, Layers lab | `compositeWithTrainedAlgorithmicMedia` | yes |
+| Upscale or colour LUT of a Kokon result | the type of that result | yes |
+| Upscale or colour LUT of an imported image | `algorithmicallyEnhanced` | no (standard editing) |
+| ... of an imported image that was already AI-made (its metadata says so, or it carries Kokon's watermark) | the type it declared | yes |
+| Layers lab, Export PSD | `compositeWithTrainedAlgorithmicMedia` (metadata only) | in the layers it came from |
+
+Upscaling and colour changes of your own photos are not marked as AI-generated: the Commission's
+guidelines list rescaling, colour adjustments and sharpening as standard editing, outside Article
+50(2). Images smaller than 150 px on their shortest side get the metadata only (Kokon does not make
+them).
+
+## Layer 1: metadata
+
+An XMP packet in every PNG (`iTXt` `XML:com.adobe.xmp`), JPEG (APP1), WebP (`XMP ` chunk), Ultra HDR
+JPEG and PSD (image resource 1060):
+
+```xml
+<rdf:Description rdf:about=""
+ xmlns:Iptc4xmpExt="http://iptc.org/std/Iptc4xmpExt/2008-02-29/"
+ xmlns:xmp="http://ns.adobe.com/xap/1.0/"
+ Iptc4xmpExt:DigitalSourceType="http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia"
+ xmp:CreatorTool="Kokon 0.12.0"
+ xmp:CreateDate="2026-11-04T10:00:00+01:00"/>
+```
+
+The [IPTC Digital Source Type](https://cv.iptc.org/newscodes/digitalsourcetype/) is the term Google,
+Meta, Adobe and others read to label AI images. It is not signed: Kokon runs on your own computer,
+where a signing key could not be kept secret (the Code of Practice leaves signatures out "in a
+local deployment scenario"). Metadata is easy to strip, so it is not the only mark.
+
+Code: [crates/qls-core/src/provenance.rs](../crates/qls-core/src/provenance.rs),
+[src/lib/psd.ts](../src/lib/psd.ts).
+
+## Layer 2: invisible watermark
+
+[TrustMark](https://github.com/adobe/trustmark) by Adobe, variant **P** (the high-quality one), MIT
+licensed code and model weights. It is an open, documented watermark listed in the
+[C2PA soft-binding algorithm list](https://github.com/c2pa-org/softbinding-algorithm-list)
+(`com.adobe.trustmark.P`), so anyone can read it with Adobe's free decoder; no key is needed.
+
+- **Payload:** the text `Kokon-AI`, in TrustMark's text mode (7-bit ASCII) with the `BCH_5` error
+ correction (61 data bits, up to 5 wrong bits corrected). Nothing about you, your prompt or your
+ computer is in it.
+- **Where:** the largest centred square of the image, faded in over its outer 6 %. The alpha
+ channel is never changed.
+- **Strength:** TrustMark's default (1.0).
+- **Once:** an image that already carries Kokon's mark (an edit of a Kokon result) is not marked
+ again.
+- **Models:** `encoder_P.onnx` and `decoder_P.onnx` from Adobe, included in the installer (pinned by
+ size and SHA-256 in [catalog.rs](../crates/qls-core/src/catalog.rs)).
+
+Code: [crates/qls-core/src/watermark/](../crates/qls-core/src/watermark/), a port of Adobe's Python
+reference on Kokon's ONNX Runtime.
+
+## Check an image
+
+- **In Kokon:** Settings → General → AI marking → **Check an image**. It reads the metadata (and
+ whether a C2PA manifest is present, without verifying it) and the watermark, and says what they
+ mean. The image is read on the computer running Kokon and not kept. **Copy report** gives the
+ result with the file's SHA-256, the time and the checker's version.
+- **Through the API:** `POST /api/v1/provenance/check` with the image bytes as the body (see
+ [http-api.md](http-api.md)).
+- **With Adobe's decoder**, without Kokon:
+
+ ```python
+ # pip install trustmark
+ from PIL import Image
+ from trustmark import TrustMark
+ tm = TrustMark(model_type="P")
+ print(tm.decode(Image.open("image.png").convert("RGB")))
+ # ('Kokon-AI', True, 1): Kokon's mark, found, schema BCH_5
+ ```
+
+- **The metadata** with any XMP reader, e.g. `exiftool -XMP-iptcExt:DigitalSourceType image.png`.
+
+## How well it works
+
+Measured on 2026-11-04 with
+[examples/watermark_check.rs](../crates/qls-core/examples/watermark_check.rs) (`bench` and
+`negatives`), on 33 Kokon results from 1024×1024 to 5504×3072 (photos, illustrations, edits,
+expansions). Kokon's mark and Adobe's Python reference read each other's marks.
+
+**Visibility:** PSNR 49.0 dB at worst, 51.6 dB median (above ~45 dB a change is not visible); the
+largest change to any pixel is 17 of 255 levels, in detailed areas. Compared side by side at 100 %,
+flat colour included, no difference was visible. Marking takes about 0.1 s (0.36 s for a
+5504×3072 image) on the CPU.
+
+**Read back after** (out of 33):
+
+| Change | Read |
+|---|---|
+| None (PNG) | 33 |
+| JPEG quality 90 / 75 / 50 | 32 / 32 / 30 |
+| Half size / quarter size / 1.5× size | 33 / 33 / 33 |
+| Screenshot-like (75 % size, JPEG 85) | 32 |
+| Crop 10 % / 20 % of the area (right and bottom) | 33 / 33 |
+| Brightness +10 %, noise ±6 levels, blur, mirror | 33 each |
+
+The one image that lost the mark after JPEG compression is mostly flat purple: there is little
+texture to hold it. Its metadata still marks it.
+
+**False alarms:** none. Of 1,567 unmarked images (Kokon results made before the watermark,
+older AI images, screenshots, game captures), none was read as Kokon's mark. Kokon's mark is
+accepted only with exactly its text and error-correction schema; a random 100-bit pattern lands
+within 5 bits of it about once in 10²¹ reads. (TrustMark's weaker schemas accept about 2 % of unmarked
+images as *some* watermark, with random text, so "Check an image" names another app's TrustMark
+only when it uses the strongest schema.)
+
+**Limits:** like every watermark, it can be removed on purpose (for example by regenerating the
+image with another model), and a large crop that cuts away the centre removes it. A file that
+lost both marks cannot be told apart from any other image.
+
+## Our process
+
+Before each release that changes how images are written, we run the tests above and the unit
+tests in `watermark/` and `provenance.rs` (a marked image is read back after JPEG and resizing;
+random bits are never read as Kokon's mark). Reports of a mark that is missing, visible or misread
+are welcome as [issues](https://github.com/GabrieleLozupone/kokon/issues).
diff --git a/docs/engineering-notes.md b/docs/engineering-notes.md
index 2207105..e77ba64 100644
--- a/docs/engineering-notes.md
+++ b/docs/engineering-notes.md
@@ -295,6 +295,15 @@ edits) before the image is made, so there is no second model in memory.
- **Review first:** `enhance_only` runs a job that stops after the prompt (no image, nothing saved); the
app streams it into the prompt box. The next job sends the text with `enhanced_from` (the user's
words), so the record keeps both and nothing is rewritten twice.
+- **A painted area and Expand (0.12, not A/B-tested yet):** two more built-in prompts (no Settings
+ field). For a brushed area the enhancer sees the region as it is with the area highlighted (the
+ image model's own copy is blurred there for Remove and Replace) and writes what goes in the area,
+ or names what is removed, after the action's words ("Remove", "Change", "Replace with", "Add": the app
+ puts them in the prompt, the service takes them off inside its instruction); for Expand it sees the canvas with the new borders empty and writes
+ what they show. For an area it writes the whole instruction of the action (Remove: what goes and what
+ is seen in its place; a bare name in the removal sentence made Qwen paint the object again), sent in
+ `pipeline::written_inpaint_prompt`; Expand's text goes after the outpaint instruction (`pipeline::EnhancerView`). With nothing typed for a removal or an expansion, the app has Prompt+
+ write the prompt into the box first (`enhanced_from: ""`), to approve or change.
### New image from references (compose)
diff --git a/docs/http-api.md b/docs/http-api.md
index 0c7bf6e..498e035 100644
--- a/docs/http-api.md
+++ b/docs/http-api.md
@@ -81,6 +81,10 @@ Full requests (`/jobs`) also accept `"mode": "upscale"` with `"upscale": {"scale
`"mode": "compose"` (the references case of `/generate`), and with `"enhance": true` (Prompt+)
`"enhance_only": true`: the job only writes the prompt (`enhanced_prompt` in the job, no image), so it
can be read and changed; send it back as `prompt` with `"enhanced_from"` set to your original words.
+Prompt+ works on every mode but upscales; for a painted area (`mask`) and an expansion it writes only
+what goes in the area (for a removal: what is removed) or in the new borders, and the prompt may be
+empty for a removal and an expansion. `"enhance_detail": true` asks Prompt+ for a longer prompt with more context
+about what is in the image or what changes.
## Input images
@@ -91,6 +95,7 @@ can be read and changed; send it back as `prompt` with `"enhanced_from"` set to
| GET | `/inputs/{id}` | The stored input's record |
| GET | `/inputs/{id}/image`, `/inputs/{id}/thumb` | The image, or its thumbnail |
| GET | `/inputs/{id}/preview` | A screen-sized copy (at most 1280 px on the long side, JPEG; PNG if it has transparency), made on first request |
+| POST | `/provenance/check` | What an image's AI markings say: raw image bytes as the body (up to 210 MB), nothing stored. Answers `ai`, `metadata` (`digital_source_type`, `creator_tool`, `c2pa_manifest`), `watermark` (`checked`, `found`, `kokon`, `text`), `sha256`, `checked_at`; see [ai-marking.md](ai-marking.md) |
| POST | `/image-info` | Size, aspect, type and embedded generation metadata of an image |
| POST | `/private/clear` | End private mode: delete every private input and result |
diff --git a/scripts/release/fetch-bundled.mjs b/scripts/release/fetch-bundled.mjs
index 780e369..f957518 100644
--- a/scripts/release/fetch-bundled.mjs
+++ b/scripts/release/fetch-bundled.mjs
@@ -4,7 +4,7 @@
// node scripts/release/fetch-bundled.mjs
//
// Downloads the upscaler of crates/qls-core/src/catalog.rs (`UPSCALER`: our mirror first, then
-// the author's release) into src-tauri/resources/ unless it is already there, and checks its
+// the author's release) and the AI watermark models (`TRUSTMARK_P_*`) into src-tauri/resources/ unless it is already there, and checks its
// size and SHA-256 against the catalog. Runs before every `tauri dev` / `tauri build`.
import crypto from "node:crypto";
import fs from "node:fs";
@@ -53,3 +53,6 @@ async function fetchBundled(name) {
}
await fetchBundled("UPSCALER");
+// The AI watermark (crates/qls-core/src/watermark): TrustMark P, MIT.
+await fetchBundled("TRUSTMARK_P_ENCODER");
+await fetchBundled("TRUSTMARK_P_DECODER");
diff --git a/src-tauri/tauri.conf.json b/src-tauri/tauri.conf.json
index 5384f51..9e4b196 100644
--- a/src-tauri/tauri.conf.json
+++ b/src-tauri/tauri.conf.json
@@ -50,7 +50,9 @@
"binaries/kokon-mcp"
],
"resources": {
- "resources/4xNomos2_hq_mosr.pth": "bundled/4xNomos2_hq_mosr.pth"
+ "resources/4xNomos2_hq_mosr.pth": "bundled/4xNomos2_hq_mosr.pth",
+ "resources/encoder_P.onnx": "bundled/encoder_P.onnx",
+ "resources/decoder_P.onnx": "bundled/decoder_P.onnx"
},
"windows": {
"nsis": {
diff --git a/src/App.tsx b/src/App.tsx
index 348e310..45aa2e4 100644
--- a/src/App.tsx
+++ b/src/App.tsx
@@ -21,6 +21,7 @@ import { Stage } from "./components/Stage";
import { Tour } from "./components/Tour";
import { PromptTips } from "./components/PromptTips";
import { Shortcuts } from "./components/Shortcuts";
+import { AiCheck } from "./components/AiCheck";
import { WhatsNew, whatsNewDue } from "./components/WhatsNew";
import { MacEngineOffer } from "./components/MacEngineOffer";
import { UpscalerCompare } from "./components/Upscalers";
@@ -354,6 +355,7 @@ export default function App() {
{overlay === "mac-engine" && <MacEngineOffer />}
{overlay === "prompt-tips" && <PromptTips />}
{overlay === "shortcuts" && <Shortcuts />}
+ {overlay === "ai-check" && <AiCheck />}
{compareUpscalers && <UpscalerCompare />}
</>
);
diff --git a/src/components/AiCheck.tsx b/src/components/AiCheck.tsx
new file mode 100644
index 0000000..6a5dcce
--- /dev/null
+++ b/src/components/AiCheck.tsx
@@ -0,0 +1,144 @@
+// "Check an image": what an image's AI markings say (the service's /provenance/check). Kokon marks
+// everything it makes or changes with the model twice, in the file's metadata and with an
+// invisible watermark (EU AI Act Art. 50(2), docs/ai-marking.md); this is the free check anyone
+// can run on any image. The image is read on the computer running Kokon and not kept.
+
+import { Copy, ScanSearch, X } from "lucide-react";
+import { useEffect, useState } from "react";
+import { api, type AiCheckReport } from "../lib/api";
+import { native } from "../lib/native";
+import { setState, toastError, toastInfo } from "../lib/store";
+
+/** The IPTC Digital Source Type in plain words. */
+const SOURCE_WORDS: Record<string, string> = {
+ trainedAlgorithmicMedia: "made with generative AI",
+ compositeWithTrainedAlgorithmicMedia: "made or changed with generative AI",
+ algorithmicallyEnhanced: "enhanced by software (not generated)",
+ digitalCapture: "a photo from a camera",
+ algorithmicMedia: "made by an algorithm (not AI)",
+ compositeSynthetic: "a composite with synthetic parts",
+ composite: "a composite of several images",
+ screenCapture: "a screenshot",
+};
+
+/** The headline and its tone. */
+function verdict(r: AiCheckReport): { title: string; tone: "ai" | "none" } {
+ if (r.watermark.kokon) return { title: "Made or changed with AI in Kokon", tone: "ai" };
+ if (r.metadata.ai) return { title: `Marked as ${SOURCE_WORDS[r.metadata.digital_source_type ?? ""] ?? "made with AI"}`, tone: "ai" };
+ return { title: "No AI marking found", tone: "none" };
+}
+
+export function AiCheck() {
+ const close = () => setState({ overlay: null });
+ const [busy, setBusy] = useState(false);
+ const [name, setName] = useState<string | null>(null);
+ const [report, setReport] = useState<AiCheckReport | null>(null);
+
+ useEffect(() => {
+ const onKey = (e: KeyboardEvent) => e.key === "Escape" && close();
+ window.addEventListener("keydown", onKey);
+ return () => window.removeEventListener("keydown", onKey);
+ }, []);
+
+ const choose = async () => {
+ const [file] = await native.pickImageFiles();
+ if (!file) return;
+ setBusy(true);
+ setReport(null);
+ setName(file.name);
+ try {
+ setReport(await api.checkImage(file));
+ } catch (e) {
+ toastError(e);
+ } finally {
+ setBusy(false);
+ }
+ };
+
+ const copy = () => {
+ if (!report) return;
+ native.copyText(JSON.stringify({ file: name, ...report }, null, 2)).then(() => toastInfo("Report copied"), toastError);
+ };
+
+ const v = report && verdict(report);
+ const wm = report?.watermark;
+ const md = report?.metadata;
+ return (
+ <div className="modal-scrim" onMouseDown={(e) => e.target === e.currentTarget && close()}>
+ <div className="tips ai-check" role="dialog" aria-label="Check an image for AI marking">
+ <div className="tips-head">
+ <ScanSearch size={20} />
+ <h1>Check an image</h1>
+ <span className="spacer" />
+ <button className="icon-btn" onClick={close} aria-label="Close">
+ <X size={18} />
+ </button>
+ </div>
+ <div className="tips-body">
+ <p className="muted">
+ Kokon marks every image it makes or changes with AI twice: in the file's information, and with an invisible watermark that stays when the file is copied,
+ resized or turned into a JPEG. Check any image here. It is read on the computer running Kokon and not kept.
+ </p>
+ <div className="ai-check-pick">
+ <button className="btn primary" onClick={choose} disabled={busy}>
+ {busy ? "Checking…" : report ? "Check another image" : "Choose an image"}
+ </button>
+ {name && <span className="muted ai-check-name">{name}</span>}
+ </div>
+ {report && v && wm && md && (
+ <section className="ai-check-result" aria-live="polite">
+ <h2 className={v.tone === "ai" ? "ai-yes" : ""}>{v.title}</h2>
+ {v.tone === "none" && <p className="muted">That does not prove it was not made with AI: markings can be removed, and not every app adds them.</p>}
+ <dl>
+ <dt>Invisible watermark</dt>
+ <dd>
+ {!wm.checked
+ ? "Not checked: the watermark model is not installed."
+ : wm.kokon
+ ? `Kokon's (TrustMark, reads "${wm.text}")`
+ : wm.found
+ ? `A TrustMark watermark from another app (reads "${wm.text}")`
+ : "None found"}
+ </dd>
+ <dt>File information</dt>
+ <dd>{md.digital_source_type ? `Says it is ${SOURCE_WORDS[md.digital_source_type] ?? md.digital_source_type}` : "Says nothing about how it was made"}</dd>
+ {md.creator_tool && (
+ <>
+ <dt>Made with</dt>
+ <dd>{md.creator_tool}</dd>
+ </>
+ )}
+ <dt>Content Credentials</dt>
+ <dd>{md.c2pa_manifest ? "Present (not verified here)" : "None"}</dd>
+ </dl>
+ <div className="ai-check-foot">
+ <span className="muted" title={report.sha256}>
+ {report.width}×{report.height} · SHA-256 {report.sha256.slice(0, 12)}…
+ </span>
+ <button className="btn" onClick={copy}>
+ <Copy size={14} /> Copy report
+ </button>
+ </div>
+ </section>
+ )}
+ </div>
+ </div>
+ </div>
+ );
+}
+
+/** Settings → General: what Kokon marks, and the check. */
+export function AiMarkingCard() {
+ return (
+ <div className="repo-card ai-marking-card">
+ <ScanSearch size={22} className="support-mark" aria-hidden />
+ <div className="body">
+ <strong>AI marking</strong>
+ <span className="muted">Images Kokon makes or changes say so in their file information and in an invisible watermark, as the EU AI Act asks. Please don't remove them.</span>
+ </div>
+ <button className="btn" onClick={() => setState({ overlay: "ai-check" })}>
+ Check an image
+ </button>
+ </div>
+ );
+}
diff --git a/src/components/BlendChoice.tsx b/src/components/BlendChoice.tsx
index 7fc859a..ad49e96 100644
--- a/src/components/BlendChoice.tsx
+++ b/src/components/BlendChoice.tsx
@@ -50,10 +50,13 @@ export function BlendChoice({ rows, onChange, onBlend, onCancel }: { rows: Blend
<X size={18} />
</IconButton>
</span>
- <h2 id="blend-choice-title">Blend into the scene</h2>
+ <h2 id="blend-choice-title">
+ Blend into the scene <span className="exp-tag">Experimental</span>
+ </h2>
<p className="split-choice-lead">
The AI looked at where {rows.length === 1 ? "the object stands" : "each object stands"}. It redraws the scene with what that place does: the contact shadow, the
- shadow, a reflection on wet or shiny ground. Check what it read and change anything that is wrong.
+ shadow, a reflection on wet or shiny ground. Check what it read and change anything that is wrong. It is
+ experimental and often does not work yet: blending all the objects at once works best.
</p>
<div className="blend-rows">
{rows.map((row) => {
diff --git a/src/components/Composer.tsx b/src/components/Composer.tsx
index 8202dd2..6a8751f 100644
--- a/src/components/Composer.tsx
+++ b/src/components/Composer.tsx
@@ -1,12 +1,12 @@
-import { ArrowUp, Brush, Check, ChevronDown, Expand, Eye, FolderOpen, Gauge, ImagePlus, Layers, Lightbulb, Maximize, Pencil, Settings2, SlidersHorizontal, Sparkles, Square, Undo2, WandSparkles, X } from "lucide-react";
+import { ArrowUp, Brush, Check, ChevronDown, Expand, Eye, FolderOpen, Gauge, ImagePlus, Layers, Lightbulb, Maximize, Pencil, Settings, Settings2, SlidersHorizontal, Sparkles, Square, Undo2, WandSparkles, X } from "lucide-react";
import { useEffect, useMemo, useRef, useState } from "react";
import { api, urls } from "../lib/api";
-import { activeLoras, activeOverrides, aspectLabel, buildRequest, composing, editing, enhancerApplies, expansionPixels, formatDuration, hasOutpaint, imageName, MAX_LORAS, moveRef, needsPrompt, parseRatio, toggleLora, wantsPromptReview, type Ref } from "../lib/logic";
+import { activeLoras, activeOverrides, aspectLabel, buildRequest, composing, editing, expansionPixels, formatDuration, hasOutpaint, imageName, MAX_LORAS, moveRef, needsPrompt, parseRatio, toggleLora, typedWords, wantsPromptReview, withActionVerb, type Ref } from "../lib/logic";
import { native } from "../lib/native";
import { haptic } from "../lib/haptics";
import { inBrowser, useNarrow } from "../lib/platform";
import {
- addAsReference, addInputs, openEditor, cancelPromptReview, editShownImage, enhancerAvailable, getState, itemSource, openSettings, selectedItem, sessionLabels, setProjectLoras, setState, startNewImage, submitComposer, toastError, undoEnhancedPrompt, updateComposer, useStore, type SessionItem,
+ addAsReference, addInputs, openEditor, cancelPromptReview, editShownImage, enhancerAvailable, focusPrompt, getState, itemSource, openSettings, selectedItem, sessionLabels, setProjectLoras, setState, startNewImage, submitComposer, toastError, undoEnhancedPrompt, updateComposer, useStore, type SessionItem,
} from "../lib/store";
import type { InpaintAction, Plan, Quality } from "../lib/types";
import { AdvancedPanel } from "./Advanced";
@@ -22,8 +22,8 @@ const QUALITY: { value: Quality; label: string; sub: string }[] = [
];
const INPAINT_ACTIONS: { value: InpaintAction; label: string; placeholder: string }[] = [
- { value: "edit", label: "Change", placeholder: "Describe the change in the painted area…" },
{ value: "remove", label: "Remove", placeholder: "What to remove (optional)…" },
+ { value: "edit", label: "Change", placeholder: "Describe the change in the painted area…" },
{ value: "replace", label: "Replace", placeholder: "What to put there instead…" },
{ value: "add", label: "Add", placeholder: "What to add in the painted area…" },
];
@@ -173,6 +173,7 @@ function TargetBar() {
const c = useStore((s) => s.composer);
const session = useStore((s) => s.session);
const nameOf = useRefNames();
+ const narrow = useNarrow();
if (!c.refs.length) {
if (!session.some((i) => i.gen || i.input)) return null;
@@ -242,9 +243,10 @@ function TargetBar() {
</button>
)}
</span>
- {c.refs.length === 1 && (
+ {/* The lab needs a mouse and room: not in the phone layout. */}
+ {c.refs.length === 1 && !narrow && (
<span className="tool-group">
- <button className="tool" onClick={() => setState({ overlay: "layers" })} title="Experimental: put the objects into layers and save each one, and the background, as a new version">
+ <button className="tool lab" onClick={() => setState({ overlay: "layers" })} title="Experimental: put the objects into layers and save each one, and the background, as a new version">
<Layers size={14} /> <span className="lbl">Layers (lab)</span>
</button>
</span>
@@ -433,13 +435,19 @@ function PromptPlus() {
>
{(close) => (
<>
- <div className="label">Prompt+</div>
+ <div className="menu-title">Prompt+</div>
<Option selected={mode === "off"} label="Off" onClick={() => (pick("off"), close())} />
<Option selected={mode === "write"} label="On" sub="writes, then makes" onClick={() => (pick("write"), close())} />
<Option selected={mode === "review"} label="Review first" sub="you edit it first" onClick={() => (pick("review"), close())} />
- <button className="opt" onClick={() => (close(), openSettings("general"))}>
+ {/* How much context Prompt+ gives the image model: only about what is there or what changes. */}
+ <div className="menu-sep" />
+ <div className="label">Detail</div>
+ <Option selected={!c.enhanceDetail} label="Standard" sub="short, focused" onClick={() => (updateComposer({ enhanceDetail: false }), close())} />
+ <Option selected={c.enhanceDetail} label="Detailed" sub="more context" onClick={() => (updateComposer({ enhance: true, enhanceDetail: true }), close())} />
+ <div className="menu-sep" />
+ <button className="opt" onClick={() => (close(), openSettings("general", "prompt-plus"))}>
<span className="check">
- <Settings2 size={15} />
+ <Settings size={15} />
</span>
<span>Prompt+ instructions…</span>
</button>
@@ -502,7 +510,7 @@ export function Composer() {
const taRef = useRef<HTMLTextAreaElement>(null);
const hasImages = c.refs.length > 0;
const overrides = activeOverrides(c.advanced);
- const canEnhance = useStore(enhancerAvailable) && enhancerApplies(c);
+ const canEnhance = useStore(enhancerAvailable);
const newImage = !editing(c);
// Prompt+ review: the prompt it is writing streams into the box, which is read-only meanwhile.
const review = useStore((s) => s.review);
@@ -647,7 +655,7 @@ export function Composer() {
className="send"
title={wantsPromptReview(c) ? "Write the prompt with Prompt+ (Enter)" : newImage ? "Create (Enter)" : "Apply (Enter)"}
aria-label={wantsPromptReview(c) ? "Write the prompt" : "Generate"}
- disabled={!c.prompt.trim() && needsPrompt(c)}
+ disabled={!typedWords(c) && needsPrompt(c)}
onClick={submit}
>
{wantsPromptReview(c) ? <WandSparkles size={17} strokeWidth={2.2} /> : <ArrowUp size={18} strokeWidth={2.4} />}
@@ -689,7 +697,7 @@ export function Composer() {
<AddImage />
{c.mask && !c.mask.marks && (
- <Seg value={c.inpaintAction} options={INPAINT_ACTIONS} onChange={(v) => updateComposer({ inpaintAction: v })} />
+ <Seg value={c.inpaintAction} options={INPAINT_ACTIONS} onChange={(v) => (updateComposer({ inpaintAction: v, prompt: withActionVerb(c.prompt, v), enhancedFrom: null }), focusPrompt(true))} />
)}
{shaped && (
diff --git a/src/components/ExpandEditor.tsx b/src/components/ExpandEditor.tsx
index 3328338..cc1a2d4 100644
--- a/src/components/ExpandEditor.tsx
+++ b/src/components/ExpandEditor.tsx
@@ -1,7 +1,7 @@
import { ArrowDown, ArrowLeft, ArrowRight, ArrowUp, Check, ChevronDown, Lock, LockOpen, Maximize, RotateCcw, X } from "lucide-react";
import { useEffect, useLayoutEffect, useRef, useState } from "react";
import { urls } from "../lib/api";
-import { LONG_SIDE_PRESETS, aspectLabel, expandToRatio, expandToSize, expansionPixels, expansionPreset, parseRatio, sizeForLongSide } from "../lib/logic";
+import { LONG_SIDE_PRESETS, aspectLabel, expandToRatio, expandToSize, expansionPixels, expansionPreset, parseRatio, placeInCanvas, sizeForLongSide } from "../lib/logic";
import { closeEditor, focusPrompt, updateComposer, useStore } from "../lib/store";
import type { OutpaintSpec, Units } from "../lib/types";
import { AspectGrid, RatioIcon } from "./SizeMenus";
@@ -77,7 +77,7 @@ function SizeForm(props: { w: number; h: number; minW: number; minH: number; onA
</span>
)}
<span className="muted">
- At least {props.minW}×{props.minH} (the image itself); the extra space is split evenly.
+ At least {props.minW}×{props.minH} (the image itself); the image starts centred: drag it, or use the grid, to place it.
</span>
<button className="btn" disabled={!ok} onClick={() => props.onApply(w, h)}>
Use this size
@@ -86,6 +86,15 @@ function SizeForm(props: { w: number; h: number; minW: number; minH: number; onA
);
}
+/** How far the canvas (`size` px of a view `room` px) moves from the centre while `pin`'s side is
+ * dragged: its opposite edge stays at `pin.at`, as far as the view allows. */
+function pinShift(pin: { side: Side; at: number } | null, axis: "x" | "y", size: number, room: number): number {
+ if (!pin || (axis === "x") !== (pin.side === "left" || pin.side === "right")) return 0;
+ const start = pin.side === "right" || pin.side === "bottom" ? pin.at : pin.at - size;
+ const clamped = Math.min(Math.max(start, Math.min(0, room - size)), Math.max(0, room - size));
+ return clamped - (room - size) / 2;
+}
+
export function ExpandEditor() {
const ref0 = useStore((s) => s.composer.refs[0]);
const initial = useStore((s) => s.composer.outpaint);
@@ -93,7 +102,17 @@ export function ExpandEditor() {
const stageRef = useRef<HTMLDivElement>(null);
const [stage, setStage] = useState({ w: 800, h: 600 });
const [frozenScale, setFrozenScale] = useState<number | null>(null);
- const drag = useRef<{ side: Side; start: number; startVal: number } | null>(null);
+ // The side being dragged, and where the canvas's opposite edge was then (px from the view's
+ // left/top): it holds still there, slides only once the canvas reaches the window's other edge.
+ const [pin, setPin] = useState<{ side: Side; at: number } | null>(null);
+ // A side being dragged: `s0` the view's scale when it was grabbed, `pos` the pointer now, `extra`
+ // what holding it at the window's edge has added (source px).
+ const drag = useRef<{ side: Side; start: number; startVal: number; s0: number; pos: number; extra: number; moved: boolean } | null>(null);
+ const raf = useRef(0);
+ const liveScale = useRef(1);
+ useEffect(() => () => cancelAnimationFrame(raf.current), []);
+ // Moving the image inside the new canvas: the same final size, the extra space shifted between sides.
+ const move = useRef<{ x: number; y: number; px: { left: number; right: number; top: number; bottom: number } } | null>(null);
useLayoutEffect(() => {
const el = stageRef.current;
@@ -119,7 +138,10 @@ export function ExpandEditor() {
const finalW = W + px.left + px.right;
const finalH = H + px.top + px.bottom;
const fitScale = Math.min(stage.w / finalW, stage.h / finalH, 1.5);
- const scale = frozenScale ?? fitScale;
+ // Dragging a side: the view keeps its scale while the canvas fits, and shrinks smoothly once it
+ // reaches the window's edge (zooming out at the grab was a jarring jump, 11-04).
+ const scale = frozenScale ?? (drag.current ? Math.min(fitScale, drag.current.s0) : fitScale);
+ liveScale.current = scale;
const src = ref0.source.startsWith("gen:") ? urls.gen(ref0.source.slice(4)) : urls.input(ref0.source.slice(6));
const toUnits = (pixels: number, side: Side) => {
@@ -129,25 +151,72 @@ export function ExpandEditor() {
};
const setSide = (side: Side, v: number) => setSpec((s) => ({ ...s, [side]: Math.max(0, Number.isFinite(v) ? v : 0) }));
+ /** The dragged side's size: the pointer's way out (at the scale it was grabbed at) plus what
+ * holding it at the edge added, snapped to 8 px. */
+ const applyDrag = (d: NonNullable<typeof drag.current>) => {
+ const delta = (d.pos - d.start) / d.s0;
+ const outward = (d.side === "left" || d.side === "top" ? -delta : delta) + d.extra;
+ setSide(d.side, toUnits(Math.max(0, Math.round((d.startVal + outward) / 8) * 8), d.side));
+ };
const onHandleDown = (side: Side) => (e: React.PointerEvent) => {
(e.target as HTMLElement).setPointerCapture(e.pointerId);
+ const cw0 = finalW * scale;
+ const ch0 = finalH * scale;
+ setPin({ side, at: side === "right" ? (stage.w - cw0) / 2 : side === "left" ? (stage.w + cw0) / 2 : side === "bottom" ? (stage.h - ch0) / 2 : (stage.h + ch0) / 2 });
+ const pos = side === "left" || side === "right" ? e.clientX : e.clientY;
+ const d = { side, start: pos, startVal: px[side], s0: scale, pos, extra: 0, moved: false };
+ drag.current = d;
+ // Held at the window's edge (or past it), the side keeps growing, faster the further out the
+ // pointer is: no letting go and grabbing again to go on.
+ let last = performance.now();
+ const loop = (now: number) => {
+ if (drag.current !== d) return;
+ const dt = Math.min(0.05, (now - last) / 1000);
+ last = now;
+ const r = stageRef.current?.getBoundingClientRect();
+ if (r && d.moved) {
+ const edge = 28;
+ const over = side === "right" ? d.pos - (r.right - edge) : side === "left" ? r.left + edge - d.pos : side === "bottom" ? d.pos - (r.bottom - edge) : r.top + edge - d.pos;
+ if (over > 0) {
+ d.extra += ((160 + 8 * over) * dt) / liveScale.current;
+ applyDrag(d);
+ }
+ }
+ raf.current = requestAnimationFrame(loop);
+ };
+ raf.current = requestAnimationFrame(loop);
+ };
+ const onImageDown = (e: React.PointerEvent) => {
+ if (px.left + px.right + px.top + px.bottom === 0) return;
+ (e.currentTarget as HTMLElement).setPointerCapture(e.pointerId);
setFrozenScale(scale);
- drag.current = { side, start: side === "left" || side === "right" ? e.clientX : e.clientY, startVal: px[side] };
+ move.current = { x: e.clientX, y: e.clientY, px };
};
const onHandleMove = (e: React.PointerEvent) => {
+ const m = move.current;
+ if (m) {
+ applyPixels(placed(m.px, m.px.left + (e.clientX - m.x) / (frozenScale ?? scale), m.px.top + (e.clientY - m.y) / (frozenScale ?? scale)));
+ return;
+ }
const d = drag.current;
if (!d) return;
- const pos = d.side === "left" || d.side === "right" ? e.clientX : e.clientY;
- const delta = (pos - d.start) / (frozenScale ?? scale);
- const outward = d.side === "left" || d.side === "top" ? -delta : delta;
- // Snap to 8 px for tidy numbers.
- const pixels = Math.max(0, Math.round((d.startVal + outward) / 8) * 8);
- setSide(d.side, toUnits(pixels, d.side));
+ d.pos = d.side === "left" || d.side === "right" ? e.clientX : e.clientY;
+ if (Math.abs(d.pos - d.start) > 4) d.moved = true;
+ applyDrag(d);
};
const onHandleUp = () => {
+ cancelAnimationFrame(raf.current);
drag.current = null;
+ move.current = null;
setFrozenScale(null);
+ setPin(null);
};
+ const placed = (p: typeof px, left: number, top: number): OutpaintSpec => placeInCanvas(p, left, top);
+ // Where the image sits: 0 = left/top, 0.5 = centre, 1 = right/bottom (the 3x3 grid).
+ const exX = px.left + px.right;
+ const exY = px.top + px.bottom;
+ const anchorAt = (ax: number, ay: number) => applyPixels({ left: Math.round(exX * ax), right: exX - Math.round(exX * ax), top: Math.round(exY * ay), bottom: exY - Math.round(exY * ay), units: "pixels" });
+ const anchored = (ax: number, ay: number) => Math.abs(px.left - Math.round(exX * ax)) <= 4 && Math.abs(px.top - Math.round(exY * ay)) <= 4;
const preset = (k: Parameters<typeof expansionPreset>[0]) => applyPixels(expansionPreset(k, W, H, { ...px, units: "pixels" as Units }));
const applyPixels = (next: OutpaintSpec) => {
@@ -181,9 +250,23 @@ export function ExpandEditor() {
</IconButton>
</div>
<div className="editor-stage" ref={stageRef} onPointerMove={onHandleMove} onPointerUp={onHandleUp}>
- <div className="expand-area" style={{ width: cw, height: ch }}>
+ <div
+ className="expand-area"
+ style={{
+ width: cw,
+ height: ch,
+ // While a side is dragged its opposite edge holds still, so the handle follows the
+ // pointer (centred, it moved at half its speed).
+ transform: `translate(${pinShift(pin, "x", cw, stage.w)}px, ${pinShift(pin, "y", ch, stage.h)}px)`,
+ }}
+ >
<div className="expand-canvas" style={{ left: 0, top: 0, width: cw, height: ch }} />
- <div className="expand-src" style={{ left: px.left * scale, top: px.top * scale, width: W * scale, height: H * scale }}>
+ <div
+ className={`expand-src ${any ? "movable" : ""}`}
+ style={{ left: px.left * scale, top: px.top * scale, width: W * scale, height: H * scale }}
+ onPointerDown={onImageDown}
+ title={any ? "Drag to place the image in the new canvas" : undefined}
+ >
<img src={src} alt="" draggable={false} />
</div>
<div className="handle h" style={{ left: 0, top: ch / 2 }} onPointerDown={onHandleDown("left")} title="Drag to extend left" />
@@ -252,6 +335,22 @@ export function ExpandEditor() {
/>
)}
</Popover>
+ {/* Where the image goes in the new canvas: the extra space goes to the other sides. */}
+ <div className="anchor-grid" role="group" aria-label="Place the image">
+ {[0, 0.5, 1].map((ay) =>
+ [0, 0.5, 1].map((ax) => (
+ <button
+ key={`${ax}-${ay}`}
+ className={any && anchored(ax, ay) ? "on" : ""}
+ disabled={!any || (exX === 0 && ax !== 0.5) || (exY === 0 && ay !== 0.5)}
+ onClick={() => anchorAt(ax, ay)}
+ title={`Image ${ay === 0 ? "top" : ay === 1 ? "bottom" : ""}${ay !== 0.5 && ax !== 0.5 ? " " : ""}${ax === 0 ? "left" : ax === 1 ? "right" : ""}${ax === 0.5 && ay === 0.5 ? "centred" : ""}: the new space on the ${[ax === 0 ? "right" : ax === 1 ? "left" : exX ? "left and right" : "", ay === 0 ? "bottom" : ay === 1 ? "top" : exY ? "top and bottom" : ""].filter(Boolean).join(", ")}`}
+ >
+ <span />
+ </button>
+ )),
+ )}
+ </div>
<IconButton title="Reset" onClick={() => setSpec({ left: 0, right: 0, top: 0, bottom: 0, units: spec.units })}>
<RotateCcw size={17} />
</IconButton>
diff --git a/src/components/InpaintEditor.tsx b/src/components/InpaintEditor.tsx
index c08ac4a..5bd22d7 100644
--- a/src/components/InpaintEditor.tsx
+++ b/src/components/InpaintEditor.tsx
@@ -2,6 +2,7 @@ import { Brush, Check, Contrast, Eraser, Hand, Loader2, MousePointerClick, Palet
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { api, urls } from "../lib/api";
import { area, dilate, fromLogits, maskOrder, samSelection } from "../lib/layers";
+import { withActionVerb } from "../lib/logic";
import { inBrowser, shortcut } from "../lib/platform";
import { Segmenter, type SamResult } from "../lib/sam";
import { MARK_COLOURS, marksPrompt, type MarkColour } from "../lib/sceneEdit";
@@ -63,6 +64,8 @@ export function InpaintEditor() {
const [colour, setColour] = useState<MarkColour>("red");
const [wants, setWantsState] = useState<Partial<Record<MarkColour, string>>>(() => lastWants.get(ref0?.source ?? "") ?? {});
const [used, setUsed] = useState<MarkColour[]>([]);
+ /** Where each colour is on the image (0-1): its box goes there, above it. */
+ const [where, setWhere] = useState<Partial<Record<MarkColour, Box>>>({});
const [tool, setTool] = useState<"brush" | "eraser" | "pan" | "select">("brush");
const [spacePressed, setSpacePressed] = useState(false);
// Right button held: the other tool is in use, and the toolbar shows it.
@@ -70,6 +73,8 @@ export function InpaintEditor() {
const [brush, setBrush] = useState(48); // in screen px
const [busy, setBusy] = useState(false);
const [sam, setSam] = useState<"idle" | "loading" | "ready" | "missing" | "error">("idle");
+ // The selection model being downloaded (the first time), in %.
+ const [samPct, setSamPct] = useState<number | null>(null);
const [selecting, setSelecting] = useState(false);
const [cursor, setCursor] = useState<{ x: number; y: number } | null>(null);
const canvasRef = useRef<HTMLCanvasElement>(null);
@@ -157,7 +162,11 @@ export function InpaintEditor() {
useEffect(() => {
redraw(ops);
- if (colours) setUsed(coloursOn(canvasRef.current, mw, mh));
+ if (colours) {
+ const on = coloursOn(canvasRef.current, mw, mh);
+ setUsed(on.used);
+ setWhere(on.where);
+ }
}, [ops, redraw, colours, mw, mh]);
/** Colours <-> Area: the drawing starts again (one kind per request). */
@@ -192,8 +201,9 @@ export function InpaintEditor() {
setSam("loading");
(async () => {
try {
- const s = await Segmenter.load();
+ const s = await Segmenter.load(true, (pct) => open.current && setSamPct(pct));
if (!open.current) return;
+ setSamPct(null);
if (!s) return setSam("missing");
const img = new Image();
img.crossOrigin = "anonymous";
@@ -230,7 +240,7 @@ export function InpaintEditor() {
const select = async (x: number, y: number, remove: boolean) => {
const s = segmenter.current;
if (!s) {
- toastInfo(sam === "missing" ? "The selection model is not in this build yet: paint with the brush." : sam === "error" ? "The selection model could not start: paint with the brush." : "The selection model is still getting ready.");
+ toastInfo(sam === "missing" ? "The selection model could not be downloaded: paint with the brush." : samPct != null ? `Downloading the selection model… ${samPct}%` : sam === "error" ? "The selection model could not start: paint with the brush." : "The selection model is still getting ready.");
return;
}
const prev = lastClick.current;
@@ -458,7 +468,10 @@ export function InpaintEditor() {
// Colours: the prompt names each colour's thing (Prompt+, when on, rewrites it seeing the marks).
const named = used.some((u) => wants[u]?.trim());
const prompt = marks && (named || !getState().composer.prompt.trim()) ? marksPrompt(used.map((u) => ({ colour: u, what: wants[u] ?? "" }))) : undefined;
- updateComposer({ mask: { inputId: rec.id, refKey, ...(marks ? { marks } : {}) }, outpaint: null, mode: "auto", ...(prompt ? { prompt, enhancedFrom: null } : {}) });
+ // Area: the action's words start the prompt (Remove by default), unless something is typed.
+ const c0 = getState().composer;
+ const verb = !marks && !c0.prompt.trim() ? withActionVerb("", c0.inpaintAction) : undefined;
+ updateComposer({ mask: { inputId: rec.id, refKey, ...(marks ? { marks } : {}) }, outpaint: null, mode: "auto", ...(prompt ? { prompt, enhancedFrom: null } : verb ? { prompt: verb, enhancedFrom: null } : {}) });
}
closeEditor();
focusPrompt();
@@ -479,9 +492,11 @@ export function InpaintEditor() {
const selectTitle =
sam === "missing"
- ? "Select (S): the selection model is not in this build yet"
+ ? "Select (S): the selection model could not be downloaded (open the editor again to retry)"
: sam === "loading"
- ? "Select (S): getting the selection model ready…"
+ ? samPct != null
+ ? `Select (S): downloading the selection model (44 MB, once)… ${samPct}%`
+ : "Select (S): getting the selection model ready…"
: `Select (S): click an object to ${colours ? "fill it with the colour" : "paint it"} · click again for a bigger part · right-click takes it out`;
return (
@@ -543,6 +558,34 @@ export function InpaintEditor() {
onPointerCancel={onUp}
onPointerLeave={() => setCursor(null)}
/>
+ {/* What goes at each colour, on the image above its area (below when that is at the top). */}
+ {colours &&
+ used.map((c) => {
+ const b = where[c];
+ if (!b) return null;
+ const below = b.y0 < 0.1;
+ return (
+ <label
+ key={c}
+ className="marks-want on-stage"
+ style={{
+ left: `${((b.x0 + b.x1) / 2) * 100}%`,
+ top: `${(below ? b.y1 : b.y0) * 100}%`,
+ transform: `translate(-50%, ${below ? "8px" : "calc(-100% - 8px)"}) scale(${1 / k})`,
+ transformOrigin: below ? "top center" : "bottom center",
+ }}
+ onPointerDown={(e) => e.stopPropagation()}
+ >
+ <span className="doc-swatch" style={{ background: cssOf(c) }} />
+ <input
+ value={wants[c] ?? ""}
+ placeholder={`What goes here, e.g. “a green velvet armchair”`}
+ onChange={(e) => setWants({ ...wants, [c]: e.target.value })}
+ spellCheck={false}
+ />
+ </label>
+ );
+ })}
{cursor && !isPanningActive && tool !== "select" && (
<div
className="brush-cursor"
@@ -564,20 +607,9 @@ export function InpaintEditor() {
<div className="marks-panel">
<p className="marks-help">
{used.length
- ? "Say what goes at each colour. Works best when the colour fills the whole area the thing should take."
+ ? "Say what goes at each colour in its box on the image. Works best when the colour fills the whole area the thing should take."
: "Fill the whole area where something should go, one colour per thing (Select fills an object in one click). The rest of the photo stays as it is."}
</p>
- {used.map((c) => (
- <label key={c} className="marks-want">
- <span className="doc-swatch" style={{ background: cssOf(c) }} />
- <input
- value={wants[c] ?? ""}
- placeholder={`What goes at the ${c} area, e.g. “a green velvet armchair”`}
- onChange={(e) => setWants({ ...wants, [c]: e.target.value })}
- spellCheck={false}
- />
- </label>
- ))}
</div>
)}
@@ -668,13 +700,17 @@ export function InpaintEditor() {
);
}
-/** The mark colours on the canvas (sampled; an opaque pixel goes to the nearest colour). */
-function coloursOn(c: HTMLCanvasElement | null, w: number, h: number): MarkColour[] {
+type Box = { x0: number; y0: number; x1: number; y1: number };
+
+/** The mark colours on the canvas and the box of each (0-1 of the image; sampled, an opaque pixel goes
+ * to the nearest colour). */
+function coloursOn(c: HTMLCanvasElement | null, w: number, h: number): { used: MarkColour[]; where: Partial<Record<MarkColour, Box>> } {
const ctx = c?.getContext("2d");
- if (!ctx) return [];
+ const where: Partial<Record<MarkColour, Box>> = {};
+ if (!ctx) return { used: [], where };
const d = ctx.getImageData(0, 0, w, h).data;
const seen = new Set<MarkColour>();
- for (let i = 0; i < d.length && seen.size < MARK_COLOURS.length; i += 4 * 7) {
+ for (let i = 0; i < d.length; i += 4 * 7) {
if (d[i + 3] < 200) continue;
let best: (typeof MARK_COLOURS)[number] = MARK_COLOURS[0];
let bd = Infinity;
@@ -683,8 +719,15 @@ function coloursOn(c: HTMLCanvasElement | null, w: number, h: number): MarkColou
if (dd < bd) [best, bd] = [m, dd];
}
seen.add(best.name);
+ const x = ((i / 4) % w) / w;
+ const y = ((i / 4 / w) | 0) / h;
+ const b = (where[best.name] ??= { x0: x, y0: y, x1: x, y1: y });
+ b.x0 = Math.min(b.x0, x);
+ b.x1 = Math.max(b.x1, x);
+ b.y0 = Math.min(b.y0, y);
+ b.y1 = Math.max(b.y1, y);
}
- return MARK_COLOURS.map((m) => m.name).filter((n) => seen.has(n));
+ return { used: MARK_COLOURS.map((m) => m.name).filter((n) => seen.has(n)), where };
}
/** A selection as a canvas in its colour (made once per selection). */
diff --git a/src/components/LabScene.ts b/src/components/LabScene.ts
index c492ec2..7b9b314 100644
--- a/src/components/LabScene.ts
+++ b/src/components/LabScene.ts
@@ -182,6 +182,53 @@ function fillBorder(d: Float32Array, valid: Uint8Array, w: number, h: number) {
}
}
+/** The edit put back (RGBA w×h), carried to the frame's edges: where it leaves no pixels, and its own
+ * `m` outermost rows (off, see fillBorder), the pixel `m` inside along the column, then the row, made
+ * opaque. Blend pastes the edit as it is, so without it a shadow stopped a few px short of the edge
+ * with the original showing in the strip (Gabriele 11-03, the girl and the spider). In place. */
+function extendToEdges(q: Uint8ClampedArray, w: number, h: number, m: number) {
+ const copy = (to: number, from: number) => {
+ q[to * 4] = q[from * 4];
+ q[to * 4 + 1] = q[from * 4 + 1];
+ q[to * 4 + 2] = q[from * 4 + 2];
+ q[to * 4 + 3] = 255;
+ };
+ const done = new Uint8Array(w * h);
+ for (let x = 0; x < w; x++) {
+ let top = -1;
+ let bottom = -1;
+ for (let y = 0; y < h; y++) {
+ if (q[(y * w + x) * 4 + 3] <= 250) continue;
+ if (top < 0) top = y;
+ bottom = y;
+ }
+ if (top < 0 || bottom - top < 4 * m) continue;
+ top += m;
+ bottom -= m;
+ for (let y = 0; y < h; y++) {
+ done[y * w + x] = 1;
+ if (y < top) copy(y * w + x, top * w + x);
+ else if (y > bottom) copy(y * w + x, bottom * w + x);
+ }
+ }
+ for (let y = 0; y < h; y++) {
+ let left = -1;
+ let right = -1;
+ for (let x = 0; x < w; x++) {
+ if (!done[y * w + x]) continue;
+ if (left < 0) left = x;
+ right = x;
+ }
+ if (left < 0 || right - left < 4 * m) continue;
+ left += m;
+ right -= m;
+ for (let x = 0; x < w; x++) {
+ if (x < left) copy(y * w + x, y * w + left);
+ else if (x > right) copy(y * w + x, y * w + right);
+ }
+ }
+}
+
/** The darkening the edit added (tone change divided out), 0-1 per pixel; tone measured on the
* pixels farther than `reach` from every object and outside `busy` (the marks). */
function addedDarkness(p: Pair, objects: Uint8Array[], reach: number, busy?: Uint8Array): Float32Array {
@@ -278,7 +325,14 @@ export async function readScene(doc: LayerDoc, e: SceneEdit, result: string): Pr
if (targets.length && e.blend) {
const all = new Uint8Array(w * h);
for (const t of targets) for (let i = 0; i < t.m.length; i++) if (t.m[i]) all[i] = 255;
- const r = blendKeep(p.before, p.after, p.valid, all, w, h);
+ // The edit carried to the frame's edges, so what is pasted reaches them (extendToEdges).
+ const m = Math.max(3, Math.round(Math.max(w, h) * 0.006));
+ const after = Uint8ClampedArray.from(p.after);
+ extendToEdges(after, w, h, m);
+ const valid = Uint8Array.from(p.valid, (_, i) => (after[i * 4 + 3] > 250 && p.before[i * 4 + 3] > 250 ? 1 : 0));
+ const r = blendKeep(p.before, after, valid, all, w, h);
+ // The objects not blended were not in what Qwen got: the Blend layer on top leaves them showing.
+ for (const [id, m] of feet) if (!e.shadows.includes(id)) for (let i = 0; i < m.length; i++) if (m[i]) r.alpha[i] = 0;
if (r.n < w * h * 0.0003) {
out.notes.push("Nothing was added round the objects. Blend again, or check what the AI read.");
return out;
@@ -288,6 +342,7 @@ export async function readScene(doc: LayerDoc, e: SceneEdit, result: string): Pr
const fctx = full.getContext("2d")!;
const px = fctx.getImageData(0, 0, W, H);
const q = px.data;
+ extendToEdges(q, W, H, Math.max(3, Math.round((m * W) / w)));
for (let i = 0; i < q.length; i += 4) {
const l = 0.2126 * q[i] + 0.7152 * q[i + 1] + 0.0722 * q[i + 2];
const k = (r.lut[Math.min(255, Math.round(l))] + 1) / (l + 1);
diff --git a/src/components/LabStage.tsx b/src/components/LabStage.tsx
index d2de586..9279cb1 100644
--- a/src/components/LabStage.tsx
+++ b/src/components/LabStage.tsx
@@ -3,12 +3,14 @@
// own versions; clicking it opens that image's layers.
import { Layers, Loader2, X } from "lucide-react";
-import { useEffect, useMemo, useRef, useState, useSyncExternalStore } from "react";
+import { useEffect, useMemo, useState, useSyncExternalStore } from "react";
import { docId, docsVersion, jobLayer, waitingMarks, labItems, leftovers, loadLab, saveLab, subscribeDocs, type LabStore, type LayerDoc } from "../lib/layerDoc";
import { editSource, itemOrigin, itemSource, removeSessionItem, setState, useStore, type SessionItem } from "../lib/store";
import type { JobView } from "../lib/types";
import { urls } from "../lib/api";
import { useJobPercent } from "./jobMotion";
+import { DevelopImg } from "./Develop";
+import { useNarrow } from "../lib/platform";
import { renderDoc } from "./LayerDoc";
/** Opens `doc`'s layers: the prompt goes to its photo (or one of its layers), which picks the
@@ -56,30 +58,46 @@ export const docById = (lab: Lab, id: string | null | undefined) => (id ? lab.st
/** The layered image as an object URL (redrawn when a layer changes; the old one stays until then). */
export function useLabImage(doc: LayerDoc | null, maxSide: number): string | null {
- const key = doc ? JSON.stringify([doc.width, doc.height, doc.layers.map((l) => [l.source, l.hidden, l.dx, l.dy, l.s, l.follows, l.at])]) : "";
- const [url, setUrl] = useState<string | null>(null);
- const shown = useRef<string | null>(null);
+ const key = doc ? JSON.stringify([maxSide, doc.width, doc.height, doc.layers.map((l) => [l.source, l.hidden, l.dx, l.dy, l.s, l.follows, l.at])]) : "";
+ // Shown at once when it was drawn before (coming back to the frame), else the last one drawn
+ // until the new one is ready.
+ const [url, setUrl] = useState<string | null>(() => rendered.get(key) ?? null);
useEffect(() => {
if (!doc) return;
+ const done = rendered.get(key);
+ if (done) {
+ setUrl(done);
+ return;
+ }
let gone = false;
renderDoc(doc, maxSide)
.then((c) => new Promise<Blob | null>((res) => c.toBlob(res, "image/png")))
.then((b) => {
- if (gone || !b) return;
- if (shown.current) URL.revokeObjectURL(shown.current);
- shown.current = URL.createObjectURL(b);
- setUrl(shown.current);
+ if (!b) return;
+ const u = URL.createObjectURL(b);
+ remember(key, u);
+ if (!gone) setUrl(u);
})
.catch(() => {});
return () => {
gone = true;
};
// eslint-disable-next-line react-hooks/exhaustive-deps
- }, [key, maxSide]);
- useEffect(() => () => void (shown.current && URL.revokeObjectURL(shown.current)), []);
+ }, [key]);
return url;
}
+/** Composites already drawn (the frame and its thumbnail per document state), the oldest let go. */
+const rendered = new Map<string, string>();
+function remember(key: string, url: string) {
+ rendered.set(key, url);
+ while (rendered.size > 24) {
+ const [k, u] = rendered.entries().next().value!;
+ rendered.delete(k);
+ URL.revokeObjectURL(u);
+ }
+}
+
export type Lab = { store: LabStore; hidden: Map<string, string | null> };
export interface LabJob {
item: SessionItem;
@@ -152,16 +170,22 @@ export function LabJobs({ lab, doc, className }: { lab: Lab; doc: LayerDoc; clas
/** The Stage while a lab version is selected: the layered image to look at; its button opens the lab. */
export function LabFrame({ doc, lab }: { doc: LayerDoc; lab: Lab }) {
const url = useLabImage(doc, 1600);
+ // The version strip's thumbnail (drawn already) as the placeholder, as for other versions.
+ const thumb = useLabImage(doc, 160);
+ // The lab needs a mouse and room: on a phone its layers are only looked at.
+ const narrow = useNarrow();
return (
<div className="stage-frame">
<LabJobs lab={lab} doc={doc} className="on-stage" />
<div className="lab-frame" style={{ aspectRatio: `${doc.width} / ${doc.height}` }}>
- {url ? <img src={url} alt="The layered image" draggable={false} /> : <Loader2 className="spin" size={22} />}
+ <DevelopImg key={docId(doc)} src={url ?? undefined} placeholder={thumb ?? undefined} alt="The layered image" />
{/* The pencil's marks sent for new objects, until the objects are there. */}
{url && waitingMarks(doc).map((m) => <img key={m} className="lab-marks" src={urls.input(m)} alt="" draggable={false} />)}
- <button className="lab-open" onClick={() => openLab(doc)} title="Open this image's layers in the Layers lab">
- <Layers size={14} /> Open Layers lab
- </button>
+ {!narrow && (
+ <button className="lab-open" onClick={() => openLab(doc)} title="Open this image's layers in the Layers lab">
+ <Layers size={14} /> Open Layers lab
+ </button>
+ )}
</div>
</div>
);
diff --git a/src/components/LayerDoc.tsx b/src/components/LayerDoc.tsx
index dcfb034..df86417 100644
--- a/src/components/LayerDoc.tsx
+++ b/src/components/LayerDoc.tsx
@@ -13,13 +13,13 @@
import { Brush, Eraser, Eye, EyeOff, FileDown, GripVertical, Group, Hand, History, ImageDown, Link2, Loader2, Lock, MousePointer2, Paintbrush, Pencil, Redo2, RotateCcw, Scissors, Sparkles, Sun, Trash2, Undo2, Ungroup, WandSparkles, X } from "lucide-react";
import { useEffect, useMemo, useRef, useState } from "react";
import { api, urls } from "../lib/api";
-import { focusPixels, foreground, modelMatte, stickerAlpha } from "../lib/matte";
+import { focusPixels, foreground, keptShare, MIN_KEPT, modelMatte, stickerAlpha } from "../lib/matte";
import { blendPrompt, DEFAULT_READING } from "../lib/blend";
import { dilate, erode, fileSafe } from "../lib/layers";
-import { centre, copiedShare, CROP_MAX_RATIO, cropFor, docId, extOf, unionRect, type Rect, waitingMarks, shadowAlpha, mapBox, reorder, unmapPoint, withGroup, xfOf, type Box, type DocLayer, type Effect, type Feed, type LayerDoc, type SceneEdit, type Sticker, type Xf } from "../lib/layerDoc";
+import { centre, copiedShare, CROP_MAX_RATIO, cropFor, docId, extOf, unionRect, type Rect, waitingMarks, shadowAlpha, strengthPixels, mapBox, reorder, unmapPoint, withGroup, xfOf, type Box, type DocLayer, type Effect, type Feed, type LayerDoc, type SceneEdit, type Sticker, type Xf } from "../lib/layerDoc";
import { buildRequest, refFromInput } from "../lib/logic";
import { native } from "../lib/native";
-import { writePsd } from "../lib/psd";
+import { aiMarkingXmp, writePsd } from "../lib/psd";
import { MARK_COLOURS, marksPrompt, type MarkColour } from "../lib/sceneEdit";
import { placesOf } from "./LabScene";
import { shortcut } from "../lib/platform";
@@ -731,8 +731,12 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
for (let i = 0; i < sel.length; i++) sel[i] = own[i * 4 + 3] > 127 ? 255 : 0;
// The object in its scene (a new object drawn by the lab is over the photo too).
const rgba = scene(w, h, true).getContext("2d")!.getImageData(0, 0, w, h).data;
+ if (!sel.some((v) => v)) throw new Error(`${l.name} has no pixels to cut out.`);
const matte = await modelMatte(focusPixels(rgba, sel, w, h), w, h);
const { alpha } = stickerAlpha(matte, sel, w, h);
+ // Checked before anything changes: a cut-out that kept (almost) nothing leaves the layer as it was.
+ const kept = keptShare(alpha, sel);
+ if (kept < MIN_KEPT) throw new Error(`The quick cut-out found almost nothing of ${l.name} (${Math.round(kept * 100)} % kept): the layer is unchanged. Try Clean edges.`);
const small = document.createElement("canvas");
small.width = w;
small.height = h;
@@ -784,6 +788,7 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
try {
await ensureMatteModel();
await fastSticker(l);
+ toastInfo(`${l.name}: quick cut-out done.`);
} catch (e) {
toastError(e);
} finally {
@@ -794,7 +799,8 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
// Stickers: after a split with stickers on, every object is cleaned once. Quality: each goes to
// Clean edges on its own (a crop at object size and the sticker prompt, sent as lab steps; the
// service runs them one after the other and each result comes back like any Clean edges). Fast:
- // the background's fill is sent first, then each object is matted here (fastSticker).
+ // each object is matted here (fastSticker), the background's fill sent before (PC: side by side)
+ // or after them (Mac).
const stickersStarted = useRef(false);
const setSticker = (layer: string, p: Partial<Sticker>) => {
const d = liveRef.current;
@@ -829,8 +835,11 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
}
};
if (fast) {
- // The fill runs on the image engine meanwhile; the stickers on the CPU, one after the other.
- await sendFill();
+ // A PC: the fill is sent first and runs on the graphics card while the stickers are made on
+ // the CPU, side by side. A Mac (unified memory): the image engine uses the CPU too and they
+ // fought (11-04: stickers waited on the fill), so the stickers first, then the fill.
+ const sideBySide = !getState().status?.gpu?.unified;
+ if (sideBySide) await sendFill();
let ready = true;
try {
await ensureMatteModel();
@@ -851,8 +860,10 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
} catch (e) {
console.error(e);
setSticker(l.id, { failed: true, local: undefined });
+ toastError(e);
}
}
+ if (!sideBySide) await sendFill();
putBack();
return;
}
@@ -977,8 +988,10 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [stickersLeft, stickerRows.length]);
- // Shadow strength: the Automatic shadows layer redrawn from the model's version, its alpha as k of
- // them stacked (1 - (1 - a)^k: darker stays a soft shadow, lighter fades evenly), as its next version.
+ // Strength: the Blend (or older Shadows) layer redrawn from the model's version, as its next version:
+ // the change it makes over the layers under it, times k (0: none, 1: as made, 2: twice as dark).
+ // The Blend layer is a paste, opaque where it changed, so scaling its alpha alone (stacking it,
+ // before 11-04) left it as it was: past alpha 1 its colours move on from what is under it.
// The Blend (or older Shadows) layer shown, not the in-front piece.
const autoShadows = live.layers.find((l) => l.auto && l.effect && l.blend !== "front" && !l.hidden) ?? live.layers.find((l) => l.auto && l.effect && l.blend !== "front");
const [strengthShown, setStrengthShown] = useState(100);
@@ -997,7 +1010,20 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
const ctx = c.getContext("2d")!;
ctx.drawImage(img, 0, 0, W, H);
const d = ctx.getImageData(0, 0, W, H);
- for (let i = 3; i < d.data.length; i += 4) d.data[i] = Math.round(255 * (1 - Math.pow(1 - d.data[i] / 255, k)));
+ // What is under the layer, in the layer's own place (it may have been moved or scaled).
+ const at = live.layers.indexOf(l);
+ const under = await render(W, H, undefined, live.layers.slice(at + 1));
+ const info = await sourceInfo(l.source, W, H);
+ const t = xfOf(live.layers, l, (src) => (src === l.source ? info.box : infos[src]?.box), W, H);
+ const u = document.createElement("canvas");
+ u.width = W;
+ u.height = H;
+ const uctx = u.getContext("2d")!;
+ uctx.translate(t.c.x, t.c.y);
+ uctx.scale(1 / t.s, 1 / t.s);
+ uctx.translate(-t.c.x - t.dx, -t.c.y - t.dy);
+ uctx.drawImage(under, 0, 0);
+ d.data.set(strengthPixels(d.data, uctx.getImageData(0, 0, W, H).data, k));
ctx.putImageData(d, 0, 0);
const rec = await api.upload(await toPng(c), "Shadows.png");
const src = `input:${rec.id}`;
@@ -1441,6 +1467,8 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
const keep = { selected: getState().selected, refs: getState().composer.refs };
try {
const skip = new Set(live.layers.filter((l) => l.effect && (o.shadows.includes(l.follows ?? "") || (!l.follows && o.shadows.length > 0))).map((l) => l.id));
+ // Blend: the background and the objects being blended, nothing else (Gabriele 11-03).
+ if (o.blend) for (const l of live.layers) if (!l.base && !l.effect && !o.shadows.includes(l.id)) skip.add(l.id);
const flat = document.createElement("canvas");
flat.width = W;
flat.height = H;
@@ -1543,9 +1571,9 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
if (!chosen.length) return toastInfo("No visible objects to blend in.");
setBusy(true);
try {
- // What the AI reads: the layers as they are, without the earlier Blend or Shadows layers, on grey
- // (holes would make the image transparent).
- const skip = new Set(live.layers.filter((l) => l.effect && l.auto).map((l) => l.id));
+ // What the AI reads: the background and the chosen objects (as sendScene sends them), without the
+ // earlier Blend or Shadows layers, on grey (holes would make the image transparent).
+ const skip = new Set(live.layers.filter((l) => (l.effect && l.auto) || (!l.base && !l.effect && !chosen.includes(l))).map((l) => l.id));
const flat = document.createElement("canvas");
flat.width = W;
flat.height = H;
@@ -1646,7 +1674,8 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
fx.fillStyle = "#fff";
fx.fillRect(0, 0, W, H);
fx.drawImage(await render(W, H), 0, 0);
- const path = await native.savePsd(writePsd(W, H, layers, fx.getImageData(0, 0, W, H).data));
+ const xmp = aiMarkingXmp(getState().status?.service.version ?? "");
+ const path = await native.savePsd(writePsd(W, H, layers, fx.getImageData(0, 0, W, H).data, xmp));
if (path) toastInfo(`Saved ${path.split(/[\\/]/).pop()}: each layer is its own layer in Photoshop.`);
} catch (e) {
toastError(e);
@@ -1725,7 +1754,7 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
? "Shadow brush: select a shadow layer in the list"
: tool === "sketch"
? "Pencil: draw where a new object goes, one colour per object · [ ] size · then say what it is on the right"
- : "Click a layer to select it · Shift+click adds · drag to move · corner scales · Ctrl+A all · Ctrl+G group";
+ : "Click a layer to select it · Shift+click adds · drag to move · a corner scales · Ctrl+A all · Ctrl+G group";
return (
<div className="overlay">
@@ -1872,16 +1901,19 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
})}
{box && tool !== "sketch" && (
<div className="doc-sel" style={{ left: box.x0, top: box.y0, width: box.x1 - box.x0, height: box.y1 - box.y0, borderWidth: 1.5 / k }}>
- {canScale && (
- <div
- className="doc-handle"
- title="Drag to scale"
- style={{ width: 12 / k, height: 12 / k, right: -6 / k, bottom: -6 / k, borderWidth: 1.5 / k }}
- onPointerDown={(e) => startDrag(e, "scale", sels)}
- onPointerMove={onMove}
- onPointerUp={onUp}
- />
- )}
+ {/* A handle on every corner: scaling is about the centre, so any of them works the same. */}
+ {canScale &&
+ (["tl", "tr", "bl", "br"] as const).map((at) => (
+ <div
+ key={at}
+ className={`doc-handle ${at}`}
+ title="Drag to scale"
+ style={{ width: 12 / k, height: 12 / k, [at[0] === "t" ? "top" : "bottom"]: -6 / k, [at[1] === "l" ? "left" : "right"]: -6 / k, borderWidth: 1.5 / k }}
+ onPointerDown={(e) => startDrag(e, "scale", sels)}
+ onPointerMove={onMove}
+ onPointerUp={onUp}
+ />
+ ))}
</div>
)}
</div>
@@ -2141,15 +2173,16 @@ export function LayerDoc({ projectId, doc, onChange, onNewSplit }: { projectId:
title={
blendStale
? "Something moved since the blend: its shadows and reflections no longer fit. Blend again (the selected objects, else all)"
- : "The AI reads where each object stands (you can correct it), then redraws the scene with their shadows, reflections on wet or shiny ground, grass round their feet; only that is kept, as the Blend layer (the selected objects, else all)"
+ : "Experimental: often does not work yet. The AI reads where each object stands (you can correct it), then redraws the scene with their shadows, reflections on wet or shiny ground, grass round their feet; only that is kept, as the Blend layer (the selected objects, else all)"
}
>
<Sun size={15} /> {blendStale ? "Blend again" : "Blend into scene"}
+ <span className="exp-tag">Experimental</span>
</button>
{autoShadows && (
- <label className="slider" title="How dark the Shadows layer is, without making it again (100 %: as the model made it)">
+ <label className="slider" title="How strong the blend's shadows and reflections are, without making them again (0 %: none, 100 %: as the model made them, 200 %: twice as strong)">
Strength
- <input type="range" min={50} max={200} step={10} value={strengthShown} onChange={(e) => setStrengthShown(Number(e.target.value))} />
+ <input type="range" min={0} max={200} step={10} value={strengthShown} onChange={(e) => setStrengthShown(Number(e.target.value))} />
<span className="doc-cmd-val">{strengthShown}%</span>
</label>
)}
diff --git a/src/components/LayerSplit.tsx b/src/components/LayerSplit.tsx
index caa9b11..b44cffd 100644
--- a/src/components/LayerSplit.tsx
+++ b/src/components/LayerSplit.tsx
@@ -128,6 +128,8 @@ export function LayerSplit({ onSplit, onBack }: { onSplit: (doc: LayerDoc) => vo
const [spacePressed, setSpacePressed] = useState(false);
const [cursor, setCursor] = useState<{ x: number; y: number } | null>(null);
const [sam, setSam] = useState<SamState>("loading");
+ // The selection model being downloaded (the first time), in %.
+ const [samPct, setSamPct] = useState<number | null>(null);
const [busy, setBusy] = useState<string | null>(null);
const [tick, setTick] = useState(0);
const [areas, setAreas] = useState<Record<string, number>>({});
@@ -166,8 +168,9 @@ export function LayerSplit({ onSplit, onBack }: { onSplit: (doc: LayerDoc) => vo
full.current = loadImage(src);
(async () => {
try {
- const s = await Segmenter.load();
+ const s = await Segmenter.load(true, (pct) => !gone && setSamPct(pct));
if (gone) return;
+ setSamPct(null);
if (!s) return setSam("missing");
setSam("encoding");
await s.encode(await full.current!, mw, mh);
@@ -278,7 +281,7 @@ export function LayerSplit({ onSplit, onBack }: { onSplit: (doc: LayerDoc) => vo
const select = async (x: number, y: number, remove: boolean) => {
const s = segmenter.current;
if (!s) {
- toastInfo(sam === "missing" ? "The selection model is not there: paint with the brush." : "The selection model is still getting ready.");
+ toastInfo(sam === "missing" ? "The selection model could not be downloaded: paint with the brush." : "The selection model is still getting ready.");
return;
}
const layer = activeRef.current;
@@ -553,10 +556,10 @@ export function LayerSplit({ onSplit, onBack }: { onSplit: (doc: LayerDoc) => vo
const status =
busy ??
{
- loading: "Loading the selection model…",
+ loading: samPct != null ? `Downloading the selection model (44 MB, once)… ${samPct}%` : "Loading the selection model…",
encoding: "Reading the image…",
ready: tool === "select" ? `Click an object to add it to “${activeName}” · again on the same spot: bigger part · right-click: take it out` : `Painting “${activeName}”`,
- missing: "Selection model not found (lab-models/mobilesam): paint the layers with the brush",
+ missing: "The selection model could not be downloaded: paint the layers with the brush (open the lab again to retry)",
error: "The selection model failed to start (see the console): paint the layers with the brush",
}[sam];
const brushCursor = tool === "brush" || tool === "eraser";
diff --git a/src/components/PhoneApp.tsx b/src/components/PhoneApp.tsx
index 0f69af0..f4e7070 100644
--- a/src/components/PhoneApp.tsx
+++ b/src/components/PhoneApp.tsx
@@ -9,7 +9,7 @@ import { useEffect, useRef, useState, type ReactNode } from "react";
import { createPortal } from "react-dom";
import { urls } from "../lib/api";
import { haptic } from "../lib/haptics";
-import { timeAgo } from "../lib/logic";
+import { projectTitle, timeAgo } from "../lib/logic";
import { native } from "../lib/native";
import { deleteProject, deleteProjects, newProject, renameProject, setState, switchProject, useStore, type ProjectMeta } from "../lib/store";
import { Composer } from "./Composer";
@@ -62,6 +62,10 @@ export function PhoneApp() {
// On a phone the sidebar's place is the library, a screen of its own.
const library = useStore((s) => s.sidebar) || overlay === "gallery";
useVisualViewport();
+ // No Layers lab on a phone (a window narrowed with it open): its layers are only looked at.
+ useEffect(() => {
+ if (overlay === "layers") setState({ overlay: null });
+ }, [overlay]);
return (
<div className="phone-app">
{library ? <Library /> : <ProjectScreen />}
@@ -315,10 +319,8 @@ function ProjectCard({ p, current, picked, onOpen, onHold }: { p: ProjectMeta; c
{p.cover && <DevelopImg src={urls.sourceThumb(p.cover)} lazy />}
{picked !== null && <span className="pick-mark">{picked && <Check size={14} strokeWidth={3} />}</span>}
</span>
- <span className="lib-card-title">{p.title}</span>
- <span className="lib-card-sub">
- {p.count} {p.count === 1 ? "image" : "images"} · {timeAgo(p.updatedAt)}
- </span>
+ <span className="lib-card-title">{projectTitle(p.title)}</span>
+ <span className="lib-card-sub">{timeAgo(p.updatedAt)}</span>
</button>
);
}
diff --git a/src/components/Settings.tsx b/src/components/Settings.tsx
index 464b571..7230353 100644
--- a/src/components/Settings.tsx
+++ b/src/components/Settings.tsx
@@ -2,6 +2,7 @@ import { Activity, Copy, Cpu, Download, FolderOpen, HardDrive, Heart, Layers, Mo
import { useEffect, useRef, useState } from "react";
import { writeText } from "@tauri-apps/plugin-clipboard-manager";
import { api } from "../lib/api";
+import { AiMarkingCard } from "./AiCheck";
import { native } from "../lib/native";
import { inBrowser } from "../lib/platform";
import { getState, openModelSetup, refreshStatus, setState, startTour, toastError, toastInfo, useStore } from "../lib/store";
@@ -273,6 +274,16 @@ export function Settings() {
});
useEffect(() => setState({ settingsTab: null }), []);
const [cfg, setCfg] = useState<AppConfig | null>(null);
+ // Opened on a section (e.g. Prompt+ instructions, from the Prompt+ menu): scrolled to once it is drawn.
+ const focus = useRef(getState().settingsFocus);
+ useEffect(() => {
+ if (!focus.current || !cfg) return;
+ const el = document.getElementById(`settings-${focus.current}`);
+ if (!el) return;
+ focus.current = null;
+ setState({ settingsFocus: null });
+ window.requestAnimationFrame(() => el.scrollIntoView({ block: "start", behavior: "smooth" }));
+ }, [cfg, tab]);
const [devices, setDevices] = useState<[string, string][]>([]);
const [upscaler, setUpscaler] = useState<Check | null>(null);
// The saved model setup: what is missing, whether the official files are downloaded.
@@ -429,6 +440,7 @@ export function Settings() {
<>
<RepoCard />
<SupportCard />
+ <AiMarkingCard />
<PathField label="Save images to" value={cfg.outputDir} folder open onChange={(v) => set({ outputDir: v })} />
<div className="field">
<span>Appearance</span>
@@ -496,7 +508,7 @@ export function Settings() {
</button>
</div>
- <h4 style={{ margin: "14px 0 0", fontSize: 14, fontWeight: 600 }}>Prompt+</h4>
+ <h4 id="settings-prompt-plus" style={{ margin: "14px 0 0", fontSize: 14, fontWeight: 600, scrollMarginTop: 12 }}>Prompt+</h4>
<span className="muted" style={{ marginTop: -8 }}>
With Prompt+ on (next to the prompt), the model's own language model rewrites your prompt before making the image, following these instructions. It
always writes in English. Its menu also lets you read and change the prompt before the image is made.
diff --git a/src/components/Sidebar.tsx b/src/components/Sidebar.tsx
index 62cbd78..24db921 100644
--- a/src/components/Sidebar.tsx
+++ b/src/components/Sidebar.tsx
@@ -1,12 +1,14 @@
-import { Check, Images, PanelLeftClose, Pencil, Settings, SquarePen, Trash2, X } from "lucide-react";
+import { Check, ChevronDown, Folder, FolderOpen, Images, ListChecks, PanelLeftClose, Pencil, Settings, SquarePen, Trash2, X } from "lucide-react";
import { useEffect, useState } from "react";
import { urls } from "../lib/api";
-import { timeAgo } from "../lib/logic";
+import { dayGroup, projectTitle, timeAgo } from "../lib/logic";
import { native } from "../lib/native";
import { deleteProject, deleteProjects, newProject, renameProject, setState, switchProject, useStore } from "../lib/store";
import { IconButton, NameField } from "./ui";
import { KokonMark } from "./Welcome";
+const CLOSED_KEY = "kokon.sidebarClosed";
+
/** The Kokon mark and name: the sidebar's head, and the top bar's while the sidebar is hidden. */
export function Wordmark() {
return (
@@ -28,6 +30,26 @@ export function Sidebar() {
const [renaming, setRenaming] = useState<string | null>(null);
// Selection: null = off; otherwise the picked project ids (and the last one clicked, for Shift).
const [picked, setPicked] = useState<Set<string> | null>(null);
+ // Collapsed: "Projects" (the whole list) and day headings, remembered on this device.
+ const [closed, setClosed] = useState<Set<string>>(() => {
+ try {
+ return new Set(JSON.parse(localStorage.getItem(CLOSED_KEY) ?? "[]") as string[]);
+ } catch {
+ return new Set();
+ }
+ });
+ const fold = (name: string) =>
+ setClosed((c) => {
+ const next = new Set(c);
+ if (!next.delete(name)) next.add(name);
+ try {
+ localStorage.setItem(CLOSED_KEY, JSON.stringify([...next]));
+ } catch {
+ /* storage blocked: folded until the page closes */
+ }
+ return next;
+ });
+ const listOpen = !closed.has("Projects");
const [anchor, setAnchor] = useState<string | null>(null);
const selecting = picked !== null;
@@ -69,34 +91,16 @@ export function Sidebar() {
setPicked(null);
};
- return (
- <aside className={`sidebar ${selecting ? "selecting" : ""}`}>
- <div className="side-head" data-tauri-drag-region>
- <Wordmark />
- <IconButton title="Hide sidebar" onClick={() => setState({ sidebar: false })}>
- <PanelLeftClose size={18} />
- </IconButton>
- </div>
- <nav className="side-nav">
- <button className={`side-item ${empty && !gallery ? "on" : ""}`} onClick={newProject}>
- <SquarePen size={16} /> New image
- </button>
- <button className={`side-item ${gallery ? "on" : ""}`} onClick={() => setState({ overlay: gallery ? null : "gallery" })}>
- <Images size={16} /> All images
- </button>
- </nav>
- {projects.length > 0 && (
- <div className="side-label">
- <span>Projects</span>
- <button className="side-select" onClick={() => setPicked(selecting ? null : new Set())} title={selecting ? "Stop selecting (Esc)" : "Select several projects to remove them at once (or Ctrl/⌘-click)"}>
- {selecting ? "Done" : "Select"}
- </button>
- </div>
- )}
- <div className="side-list">
- {projects.map((p) => {
- const on = picked?.has(p.id) ?? false;
- return (
+ // Under a heading per day (the list is newest first), to find a project by when.
+ const sections: { group: string; items: typeof projects }[] = [];
+ for (const p of projects) {
+ const group = dayGroup(p.updatedAt);
+ if (sections[sections.length - 1]?.group === group) sections[sections.length - 1].items.push(p);
+ else sections.push({ group, items: [p] });
+ }
+ const row = (p: (typeof projects)[number]) => {
+ const on = picked?.has(p.id) ?? false;
+ return (
<div
key={p.id}
role={selecting ? "checkbox" : "button"}
@@ -117,7 +121,7 @@ export function Sidebar() {
} else if (e.key === "F2" && !selecting) setRenaming(p.id);
else if ((e.key === "Delete" || e.key === "Backspace") && selecting) removePicked();
}}
- title={selecting ? p.title : `${p.title}\nDouble-click or F2 to rename · Ctrl/⌘-click to select several`}
+ title={selecting ? p.title : `${p.title}\n${p.count} ${p.count === 1 ? "version" : "versions"} · ${timeAgo(p.updatedAt)}\nDouble-click or F2 to rename · Ctrl/⌘-click to select several`}
>
<span className="cover">
{p.cover && <img src={urls.sourceThumb(p.cover)} alt="" draggable={false} loading="lazy" />}
@@ -127,11 +131,8 @@ export function Sidebar() {
{renaming === p.id ? (
<NameField value={p.title} onSave={(v) => renameProject(p.id, v)} onDone={() => setRenaming(null)} />
) : (
- <span className="proj-title">{p.title}</span>
+ <span className="proj-title">{projectTitle(p.title)}</span>
)}
- <span className="proj-sub">
- {p.count} {p.count === 1 ? "image" : "images"} · {timeAgo(p.updatedAt)}
- </span>
</span>
{!selecting && (
<>
@@ -160,8 +161,53 @@ export function Sidebar() {
</>
)}
</div>
- );
- })}
+ );
+ };
+
+ return (
+ <aside className={`sidebar ${selecting ? "selecting" : ""}`}>
+ <div className="side-head" data-tauri-drag-region>
+ <Wordmark />
+ <IconButton title="Hide sidebar" onClick={() => setState({ sidebar: false })}>
+ <PanelLeftClose size={18} />
+ </IconButton>
+ </div>
+ <nav className="side-nav">
+ <button className={`side-item ${empty && !gallery ? "on" : ""}`} onClick={newProject}>
+ <SquarePen size={16} /> New image
+ </button>
+ <button className={`side-item ${gallery ? "on" : ""}`} onClick={() => setState({ overlay: gallery ? null : "gallery" })}>
+ <Images size={16} /> All images
+ </button>
+ </nav>
+ {projects.length > 0 && (
+ <div className="side-label">
+ <button className={`side-item side-fold ${listOpen ? "" : "closed"}`} onClick={() => fold("Projects")} aria-expanded={listOpen} title={listOpen ? "Collapse projects" : "Show projects"}>
+ {listOpen ? <FolderOpen size={16} /> : <Folder size={16} />}
+ Projects
+ <ChevronDown size={14} className="fold-chev" />
+ </button>
+ <button className="side-select" onClick={() => setPicked(selecting ? null : new Set())} title={selecting ? "Stop selecting (Esc)" : "Select several projects to remove them at once (or Ctrl/⌘-click)"}>
+ {!selecting && <ListChecks size={13} />}
+ {selecting ? "Done" : "Select"}
+ </button>
+ </div>
+ )}
+ <div className="side-list">
+ {/* Under "Projects": a line from its folder icon, a rounded branch to each day. */}
+ {listOpen && projects.length > 0 && (
+ <div className="side-tree">
+ {sections.map(({ group, items }) => (
+ <section key={group} className="side-sec">
+ <button className={`side-group ${closed.has(group) ? "closed" : ""}`} onClick={() => fold(group)} aria-expanded={!closed.has(group)}>
+ {group}
+ <ChevronDown size={12} />
+ </button>
+ {!closed.has(group) && items.map(row)}
+ </section>
+ ))}
+ </div>
+ )}
</div>
{selecting && (
<div className="select-bar">
diff --git a/src/components/Stage.tsx b/src/components/Stage.tsx
index 4de7e26..c9cb9bb 100644
--- a/src/components/Stage.tsx
+++ b/src/components/Stage.tsx
@@ -1,5 +1,5 @@
import {
- Brush, ClipboardPaste, Combine, Copy, Download, Ellipsis, Expand, FolderOpen, Heart, ImageUpscale, Info, Layers, Loader2, MousePointerClick, Palette, Pencil, RefreshCw, RotateCcw, SlidersHorizontal, Square, TextCursorInput, Trash2, Wand2, WandSparkles, X,
+ Brush, Check, ClipboardPaste, Combine, Copy, Download, Ellipsis, Expand, FolderOpen, Heart, ImageUpscale, Info, Layers, Loader2, MousePointerClick, Palette, Pencil, RefreshCw, RotateCcw, SlidersHorizontal, Square, TextCursorInput, Trash2, Wand2, WandSparkles, X,
} from "lucide-react";
import { Fragment, useEffect, useLayoutEffect, useMemo, useRef, useState } from "react";
import { api, urls } from "../lib/api";
@@ -9,7 +9,7 @@ import {
} from "../lib/logic";
import { canCopyImage, native } from "../lib/native";
import {
- addAsReference, addResult, cancelJob, focusPrompt, getState, itemSource, removeSessionItem, reusePrompt, selectedItem, selectItem, sessionLabels, sessionLinks, setState, stageImage, submitRequest, toastError, toastInfo, useStore, type SessionItem,
+ addAsReference, addResult, cancelJob, focusPrompt, getState, itemSource, removeSessionItem, removeSessionItems, reusePrompt, selectedItem, selectItem, sessionLabels, sessionLinks, setState, stageImage, submitRequest, toastError, toastInfo, useStore, type SessionItem,
} from "../lib/store";
import type { GenerationRecord, GenerationRequest, JobView, LutEntry, Mode, Size } from "../lib/types";
import { Cocoon, Hatch } from "./Cocoon";
@@ -667,6 +667,48 @@ function Timeline({ labels, lab }: { labels: Record<string, string>; lab: Lab })
const selected = useStore((s) => s.selected);
const jobs = useStore((s) => s.jobs);
const target = useStore((s) => s.composer.refs[0]?.source);
+ // Ctrl/⌘-click picks several versions (Shift-click a run of them), Delete removes them from the
+ // project (or the shown one, with nothing picked), after asking. A version being made can't be.
+ const [picked, setPicked] = useState<Set<string>>(new Set());
+ const anchor = useRef<string | null>(null);
+ const removable = (it: SessionItem) => it.kind !== "job" || ["failed", "cancelled"].includes(jobs[it.jobId ?? ""]?.status ?? "failed");
+ const pick = (key: string, range: boolean) => {
+ const from = anchor.current ?? selected;
+ const next = new Set(picked.size ? picked : selected && session.some((i) => i.key === selected) ? [selected] : []);
+ if (range && from) {
+ const a = session.findIndex((i) => i.key === from);
+ const b = session.findIndex((i) => i.key === key);
+ if (a >= 0 && b >= 0) for (const it of session.slice(Math.min(a, b), Math.max(a, b) + 1)) next.add(it.key);
+ } else if (next.has(key)) next.delete(key);
+ else next.add(key);
+ if (!range) anchor.current = key;
+ setPicked(new Set([...next].filter((k) => session.some((i) => i.key === k && removable(i)))));
+ };
+ // Versions gone (removed here or elsewhere) are no longer picked.
+ useEffect(() => {
+ if ([...picked].some((k) => !session.some((i) => i.key === k))) setPicked(new Set([...picked].filter((k) => session.some((i) => i.key === k))));
+ }, [session, picked]);
+ useEffect(() => {
+ const down = async (e: KeyboardEvent) => {
+ if (isTyping(e.target) || getState().overlay) return;
+ if (e.key === "Escape" && picked.size) {
+ setPicked(new Set());
+ return;
+ }
+ if (e.key !== "Delete" && e.key !== "Backspace") return;
+ const keys = picked.size ? [...picked] : session.filter((i) => i.key === selected && removable(i)).map((i) => i.key);
+ if (!keys.length) return;
+ e.preventDefault();
+ const n = keys.length;
+ const ok = await native.confirm(n === 1 ? "Remove this version from the project? Its image stays in All images." : `Remove ${n} versions from the project? Their images stay in All images.`, "Remove from project");
+ if (!ok) return;
+ removeSessionItems(keys);
+ setPicked(new Set());
+ };
+ window.addEventListener("keydown", down);
+ return () => window.removeEventListener("keydown", down);
+ // eslint-disable-next-line react-hooks/exhaustive-deps
+ }, [picked, session, selected, jobs]);
const boxRef = useRef<HTMLDivElement>(null);
const trackRef = useRef<HTMLDivElement>(null);
const [centers, setCenters] = useState<Record<string, number>>({});
@@ -802,12 +844,18 @@ function Timeline({ labels, lab }: { labels: Record<string, string>; lab: Lab })
{(i > 0 || labFirst.length > 0) && <Joint kind={link?.kind ?? "next"} hot={hot} />}
<button
data-key={it.key}
- className={`frame ${selected === it.key ? "sel" : ""} ${selParents.has(it.key) ? "src" : ""}`}
+ className={`frame ${selected === it.key ? "sel" : ""} ${selParents.has(it.key) ? "src" : ""} ${picked.has(it.key) ? "picked" : ""}`}
title={tip}
+ aria-pressed={picked.size ? picked.has(it.key) : undefined}
onMouseEnter={() => setHovered(it.key)}
- onClick={() => {
+ onClick={(e) => {
if (dragged.current) dragged.current = false;
- else selectItem(it.key);
+ else if (e.ctrlKey || e.metaKey || e.shiftKey) pick(it.key, e.shiftKey);
+ else {
+ if (picked.size) setPicked(new Set());
+ anchor.current = it.key;
+ selectItem(it.key);
+ }
}}
onPointerDown={(e) => {
// A finger scrolls the timeline; dragging onto the prompt is for a mouse.
@@ -861,6 +909,11 @@ function Timeline({ labels, lab }: { labels: Record<string, string>; lab: Lab })
</svg>
)}
</span>
+ {picked.has(it.key) && (
+ <span className="frame-pick" aria-hidden>
+ <Check size={11} strokeWidth={3} />
+ </span>
+ )}
<span className="frame-label">
{mode && MODE_ICON[mode]}
{labels[it.key]}
diff --git a/src/lib/api.ts b/src/lib/api.ts
index f4acd13..f1a9d9c 100644
--- a/src/lib/api.ts
+++ b/src/lib/api.ts
@@ -81,6 +81,18 @@ export interface RemoteStatus {
const json = (body: unknown) => JSON.stringify(body);
+/** The answer of "Check an image" (crates/qls-core/src/provenance.rs `check`). */
+export interface AiCheckReport {
+ ai: boolean;
+ metadata: { digital_source_type: string | null; ai: boolean; creator_tool: string | null; c2pa_manifest: boolean };
+ watermark: { checked: boolean; found?: boolean; kokon?: boolean; text?: string; schema?: string };
+ width: number;
+ height: number;
+ sha256: string;
+ checked_at: string;
+ checker: string;
+}
+
export const api = {
status: () => call<StatusResponse>("/status"),
setup: () => call<SetupReport>("/setup"),
@@ -113,6 +125,10 @@ export const api = {
upscalers: () => call<{ selected: string; active: string | null; items: UpscalerItem[] }>("/upscalers"),
/** Progress in `installStatus` (preset "upscaler:<name>"). */
downloadUpscaler: (name: string) => call<InstallStatus>(`/upscalers/${encodeURIComponent(name)}/download`, { method: "POST" }),
+ /** The click-to-select model (MobileSAM, run here in the webview): there or not, and its size. */
+ segmentStatus: () => call<{ installed: boolean; bytes: number }>("/segment"),
+ /** Progress in `installStatus` (preset "segment"). */
+ downloadSegment: () => call<InstallStatus>("/segment/download", { method: "POST" }),
/** The fast-sticker model of the layers lab: there or not, and its download size. */
matteStatus: () => call<{ installed: boolean; bytes: number }>("/matte"),
/** Progress in `installStatus` (preset "matting"). */
@@ -158,6 +174,9 @@ export const api = {
cancel: (id: string) => call<JobView>(`/jobs/${id}/cancel`, { method: "POST" }),
upload: (bytes: Blob | ArrayBuffer | Uint8Array, name?: string) =>
call<InputRecord>(`/inputs${privateQuery()}`, { method: "POST", body: bytes as BodyInit, headers: { "Content-Type": "application/octet-stream", ...(name ? { "x-filename": encodeURIComponent(name) } : {}) } }),
+ /** What an image's AI markings say (metadata and Kokon's invisible watermark); nothing is kept. */
+ checkImage: (bytes: Blob | ArrayBuffer | Uint8Array) =>
+ call<AiCheckReport>("/provenance/check", { method: "POST", body: bytes as BodyInit, headers: { "Content-Type": "application/octet-stream" } }),
importPath: (path: string) => call<InputRecord>(`/inputs/import${privateQuery()}`, { method: "POST", body: json({ path }) }),
clearPrivate: () => call<{ cleared: number }>("/private/clear", { method: "POST" }),
input: (id: string) => call<InputRecord>(`/inputs/${id}`),
@@ -235,6 +254,8 @@ export const urls = {
lutPreview: (name: string, source: string, max: number) => `${base}/luts/${encodeURIComponent(name)}/preview?source=${encodeURIComponent(source)}&max=${max}${tq("&")}`,
sourcePreview: (s: string) => (s.startsWith("gen:") ? urls.genPreview(s.slice(4)) : urls.inputPreview(s.slice(6))),
sourceThumb: (s: string) => (s.startsWith("gen:") ? urls.genThumb(s.slice(4)) : urls.inputThumb(s.slice(6))),
+ /** A MobileSAM file kokon-service downloaded (`segmentStatus`). */
+ segmentModel: (file: string) => `${base}/segment/models/${file}${tq("?")}`,
};
/** Server-sent events with automatic reconnect. Returns an unsubscribe function. */
diff --git a/src/lib/layerDoc.test.ts b/src/lib/layerDoc.test.ts
index b999c56..715e1e2 100644
--- a/src/lib/layerDoc.test.ts
+++ b/src/lib/layerDoc.test.ts
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
-import { copiedShare, cropFor, expandedRect, extOf, unionRect, docFor, keepLabsIn, labSources, loadLab, saveLab, waitingMarks, jobLayer, labItems, leftovers, putDoc, type LabStore, shadowAlpha, mapPoint, placed, reconcile, reorder, toLayer, xfOf, type DocLayer, type LayerDoc } from "./layerDoc";
+import { copiedShare, cropFor, expandedRect, extOf, unionRect, docFor, keepLabsIn, labSources, loadLab, saveLab, waitingMarks, jobLayer, labItems, leftovers, putDoc, type LabStore, shadowAlpha, strengthPixels, mapPoint, placed, reconcile, reorder, toLayer, xfOf, type DocLayer, type LayerDoc } from "./layerDoc";
const layer = (id: string, source: string, base = false): DocLayer => ({ id, name: id, source, history: [source], hidden: false, base, dx: 0, dy: 0, s: 1 });
const doc = (layers: DocLayer[]): LayerDoc => ({ width: 100, height: 100, layers, known: [] });
@@ -76,6 +76,18 @@ describe("layer document", () => {
expect(xfOf([shadow], shadow, () => box, 100, 100)).toEqual({ c: { x: 50, y: 50 }, s: 1, dx: 0, dy: 0 });
});
+ it("strengthPixels: 0 shows nothing, 1 as made, 2 doubles an opaque paste's change over what is under it", () => {
+ const made = [60, 60, 60, 255, 0, 0, 0, 128];
+ const under = [100, 100, 100, 255, 200, 200, 200, 255];
+ expect([...strengthPixels(made, under, 0)]).toEqual([60, 60, 60, 0, 0, 0, 0, 0]);
+ expect([...strengthPixels(made, under, 1)]).toEqual([60, 60, 60, 255, 0, 0, 0, 128]);
+ expect([...strengthPixels(made, under, 0.5)].slice(0, 4)).toEqual([60, 60, 60, 128]);
+ const two = [...strengthPixels(made, under, 2)];
+ expect(two.slice(0, 4)).toEqual([20, 20, 20, 255]);
+ // A half-transparent shadow at 2: fully opaque, the same darkness as the paste's own.
+ expect(two[7]).toBe(255);
+ });
+
it("shadowAlpha: how much darker, inside the painted area only; brighter pixels are left out", () => {
const px = (v: number) => [v, v, v, 255];
const before = [...px(200), ...px(200), ...px(200), ...px(200), ...px(100)];
diff --git a/src/lib/layerDoc.ts b/src/lib/layerDoc.ts
index f261c65..288a2b0 100644
--- a/src/lib/layerDoc.ts
+++ b/src/lib/layerDoc.ts
@@ -480,6 +480,23 @@ export function shadowAlpha(before: ArrayLike<number>, after: ArrayLike<number>,
return out;
}
+/** A Blend or Shadows layer at strength `k` (RGBA, `made` = as the model made it, `under` = the
+ * layers under it): the change it makes, times k. Up to fully opaque the alpha carries it; past
+ * that the colours move on from what is under it (k = 2 on an opaque paste: twice the darkening). */
+export function strengthPixels(made: ArrayLike<number>, under: ArrayLike<number>, k: number): Uint8ClampedArray {
+ const out = new Uint8ClampedArray(made.length);
+ for (let j = 0; j < made.length; j += 4) {
+ const t = (made[j + 3] / 255) * Math.max(0, k);
+ if (t <= 1) {
+ out.set([made[j], made[j + 1], made[j + 2], Math.round(t * 255)], j);
+ continue;
+ }
+ for (let c = 0; c < 3; c++) out[j + c] = Math.round(under[j + c] + t * (made[j + c] - under[j + c]));
+ out[j + 3] = 255;
+ }
+ return out;
+}
+
/** The pencil's marks sent with New objects / Edit together whose result has not come back yet
* (input ids): shown over the layers, here and on the Stage, until the objects are there. */
export const waitingMarks = (doc: LayerDoc): string[] => (doc.scenes ?? []).filter((e) => e.marks && !e.made).map((e) => e.marks!);
diff --git a/src/lib/logic.test.ts b/src/lib/logic.test.ts
index b8809f1..0dec330 100644
--- a/src/lib/logic.test.ts
+++ b/src/lib/logic.test.ts
@@ -5,7 +5,8 @@ import {
modelOfferDue,
whatsNewDue,
macBenchProgress,
- activeLoras, activeOverrides, buildRequest, composing, editing, reviewRequest, wantsPromptReview, toggleLora, MAX_LORAS, jobPercent, compareBase, composerFromRecord, enhancerApplies, diffPixels, emptyComposer, expansionPixels, expansionPreset, jobLabel, jobSecondsLeft, lutInfo, moveRef, needsPrompt, normalizeQuality, persistentAdvanced, pipelineSummary,
+ dayGroup, isGenericName, placeInCanvas, projectTitle, typedWords, withActionVerb,
+ activeLoras, activeOverrides, buildRequest, composing, editing, reviewRequest, wantsPromptReview, toggleLora, MAX_LORAS, jobPercent, compareBase, composerFromRecord, diffPixels, emptyComposer, expansionPixels, expansionPreset, jobLabel, jobSecondsLeft, lutInfo, moveRef, needsPrompt, normalizeQuality, persistentAdvanced, pipelineSummary,
planPipeline, remainingSeconds, sizeForWidth, timeAgo, upscaleRequest, upscaleTargets, versionLabels, type JobClock, type Ref,
neighbourAfterRemoval, pageOutdated, previewIndex, starPromptDue, STAR_PROMPT_AFTER, formatBytes, formatDuration, installItemShort, installItemText, installProgress, installStep, installTimeLeft, timelineLinks, undoTarget, aspectLabel, parseRatio, sizeForRatio, sizeForLongSide, expandToRatio, expandToSize, arcLayout,
goneVersions, PREVIEW_SIDE, stageFitsPreview, viewerWantsFull,
@@ -75,7 +76,6 @@ describe("buildRequest", () => {
it("coloured marks go with the mask, as an edit Prompt+ can write", () => {
const plain = { ...emptyComposer(), prompt: "x", refs: [ref(1)], mask: { inputId: "m", refKey: "k1" }, inpaintAction: "remove" as const, enhance: true };
expect(buildRequest(plain).marks).toBeUndefined();
- expect(enhancerApplies(plain)).toBe(false);
expect(needsPrompt(plain)).toBe(false);
const marked = { ...plain, mask: { inputId: "m", refKey: "k1", marks: "k" } };
const r = buildRequest(marked);
@@ -122,11 +122,53 @@ describe("buildRequest", () => {
const on = { ...off, enhance: true };
expect(buildRequest(on).enhance).toBe(true);
expect(buildRequest({ ...on, refs: [ref(1), ref(2)] }).enhance).toBe(true);
+ // A painted area and Expand too: Prompt+ writes what goes there inside Kokon's instruction.
const masked = { ...on, refs: [ref(1)], mask: { inputId: "m", refKey: "k1" } };
- expect(enhancerApplies(masked)).toBe(false);
- expect("enhance" in buildRequest(masked)).toBe(false);
+ expect(buildRequest(masked).enhance).toBe(true);
const expanded = { ...on, refs: [ref(1)], outpaint: { left: 10, right: 0, top: 0, bottom: 0, units: "percent" as const } };
- expect("enhance" in buildRequest(expanded)).toBe(false);
+ expect(buildRequest(expanded).enhance).toBe(true);
+ expect(needsPrompt(expanded)).toBe(false);
+ });
+ it("each painted-area action puts its words at the start of the prompt; Remove is the default", () => {
+ expect(emptyComposer().inpaintAction).toBe("remove");
+ expect(withActionVerb("", "remove")).toBe("Remove ");
+ expect(withActionVerb("Remove the lamp", "replace")).toBe("Replace with the lamp");
+ expect(withActionVerb("Replace with a cat", "add")).toBe("Add a cat");
+ expect(withActionVerb("a red sofa", "edit")).toBe("Change a red sofa");
+ const area = { ...emptyComposer(), refs: [ref(1)], mask: { inputId: "m", refKey: "k1" }, enhance: true };
+ // The action's words alone are nothing typed: Remove + Prompt+ has it written, Change still needs words.
+ expect(typedWords({ ...area, prompt: "Remove " })).toBe("");
+ expect(wantsPromptReview({ ...area, prompt: "Remove " })).toBe(true);
+ expect(needsPrompt({ ...area, prompt: "Change ", inpaintAction: "edit" as const }) && !typedWords({ ...area, prompt: "Change ", inpaintAction: "edit" as const })).toBe(true);
+ expect(typedWords({ ...area, prompt: "Remove the lamp" })).toBe("the lamp");
+ // Not for a new image or an edit without a painted area.
+ expect(typedWords({ ...emptyComposer(), prompt: "Add a hat" })).toBe("Add a hat");
+ });
+ it("Prompt+ Detailed goes with the request only while Prompt+ writes", () => {
+ const c = { ...emptyComposer(), prompt: "un gatto", enhance: true, enhanceDetail: true };
+ expect(buildRequest(c)).toMatchObject({ enhance: true, enhance_detail: true });
+ expect(reviewRequest(c)).toMatchObject({ enhance_only: true, enhance_detail: true });
+ expect("enhance_detail" in buildRequest({ ...c, enhanceDetail: false })).toBe(false);
+ expect("enhance_detail" in buildRequest({ ...c, enhance: false })).toBe(false);
+ // Already written: sent as it is.
+ expect("enhance_detail" in buildRequest({ ...c, enhancedFrom: "un gatto" })).toBe(false);
+ });
+ it("nothing typed for a removal or Expand: Prompt+ writes it into the box first, once", () => {
+ const remove = { ...emptyComposer(), enhance: true, enhanceReview: false, refs: [ref(1)], mask: { inputId: "m", refKey: "k1" }, inpaintAction: "remove" as const };
+ expect(wantsPromptReview(remove)).toBe(true);
+ expect(wantsPromptReview({ ...remove, enhance: false })).toBe(false);
+ // Typed words follow the user's choice (write, or review first).
+ expect(wantsPromptReview({ ...remove, prompt: "the lamp" })).toBe(false);
+ expect(wantsPromptReview({ ...remove, prompt: "the lamp", enhanceReview: true })).toBe(true);
+ // Written from nothing: sent as it is, not written again.
+ const written = { ...remove, prompt: "the brass floor lamp on the left", enhancedFrom: "" };
+ expect(wantsPromptReview(written)).toBe(false);
+ expect(buildRequest(written)).toMatchObject({ enhanced_from: "" });
+ expect("enhance" in buildRequest(written)).toBe(false);
+ const expand = { ...emptyComposer(), enhance: true, refs: [ref(1)], outpaint: { left: 10, right: 0, top: 0, bottom: 0, units: "percent" as const } };
+ expect(wantsPromptReview(expand)).toBe(true);
+ // A change still needs words.
+ expect(wantsPromptReview({ ...remove, inpaintAction: "edit" as const })).toBe(false);
});
it("reusing an enhanced result's settings gives the user's prompt with the enhancer on", () => {
const g = { prompt: "A tabby cat sleeping on a sunlit windowsill.", original_prompt: "un gatto", references: [], mask_input_id: null, outpaint: null, quality: "fast", request: null } as unknown as GenerationRecord;
@@ -167,7 +209,8 @@ describe("buildRequest", () => {
expect(wantsPromptReview({ ...c, enhanceReview: false })).toBe(false);
expect(wantsPromptReview({ ...c, enhance: false })).toBe(false);
expect(wantsPromptReview({ ...c, prompt: " " })).toBe(false);
- expect(wantsPromptReview({ ...c, refs: [ref(1)], mask: { inputId: "m", refKey: "k1" } })).toBe(false);
+ // A painted area is written too (what goes there), as any edit.
+ expect(wantsPromptReview({ ...c, refs: [ref(1)], mask: { inputId: "m", refKey: "k1" } })).toBe(true);
const r = reviewRequest(c);
expect([r.enhance, r.enhance_only, r.prompt]).toEqual([true, true, "un gatto"]);
// The written prompt is sent as is, with the user's words, and not rewritten again.
@@ -703,3 +746,33 @@ describe("modelOfferDue", () => {
expect(modelOfferDue(dit(null), {}, "0.11.1")).toBe(false);
});
});
+
+describe("project list", () => {
+ it("projectTitle drops an image file's extension only", () => {
+ expect(projectTitle("4.png")).toBe("4");
+ expect(projectTitle("Kitchen.JPG")).toBe("Kitchen");
+ expect(projectTitle("make it night.")).toBe("make it night.");
+ });
+ it("isGenericName: names that say nothing about the picture", () => {
+ for (const n of ["image", "image.png", "4.png", "IMG_1234.JPG", "Screenshot 2026-10-01 at 10.00.00.png", "WhatsApp Image 2026-10-01 at 10.00.00.jpeg", "image (3).png", "PXL_20261001_101010.jpg", "Imported image"]) expect(isGenericName(n), n).toBe(true);
+ for (const n of ["Kitchen Scene", "photo of my dog", "grandma.jpg", "Halloween Project", "imagery"]) expect(isGenericName(n), n).toBe(false);
+ });
+ it("dayGroup: by calendar day", () => {
+ const now = new Date(2026, 10, 4, 10).getTime();
+ expect(dayGroup(new Date(2026, 10, 4, 1).getTime(), now)).toBe("Today");
+ expect(dayGroup(new Date(2026, 10, 3, 23).getTime(), now)).toBe("Yesterday");
+ expect(dayGroup(new Date(2026, 9, 30).getTime(), now)).toBe("Previous 7 days");
+ expect(dayGroup(new Date(2026, 9, 10).getTime(), now)).toBe("Previous 30 days");
+ expect(dayGroup(new Date(2026, 0, 1).getTime(), now)).toBe("Older");
+ });
+});
+
+describe("expand: placing the image", () => {
+ it("moves the extra space between sides, keeps the size, snaps to 8 px and to the edges", () => {
+ const centred = { left: 100, right: 100, top: 50, bottom: 50 };
+ expect(placeInCanvas(centred, 0, 50)).toEqual({ left: 0, right: 200, top: 48, bottom: 52, units: "pixels" });
+ expect(placeInCanvas(centred, 197, 103)).toEqual({ left: 200, right: 0, top: 100, bottom: 0, units: "pixels" });
+ expect(placeInCanvas(centred, -40, 999)).toEqual({ left: 0, right: 200, top: 100, bottom: 0, units: "pixels" });
+ expect(placeInCanvas({ left: 0, right: 0, top: 30, bottom: 30 }, 20, 0).left).toBe(0);
+ });
+});
diff --git a/src/lib/logic.ts b/src/lib/logic.ts
index d32d67d..331efdd 100644
--- a/src/lib/logic.ts
+++ b/src/lib/logic.ts
@@ -30,11 +30,13 @@ export interface Composer {
customHeight: number;
seed: number | null;
advanced: Advanced;
- /** Prompt enhancer on (remembered; used only where `enhancerApplies`). */
+ /** Prompt enhancer on (remembered). */
enhance: boolean;
/** Prompt+ writes its prompt into the box first, to read and change before making the image
* (remembered); otherwise it writes and the image follows. */
enhanceReview: boolean;
+ /** Prompt+ "Detailed": longer prompts with more context (remembered). */
+ enhanceDetail: boolean;
/** The user's own words when the prompt in the box was written by Prompt+ (a review, or a
* reused enhanced prompt): sent as is, without rewriting it again. */
enhancedFrom: string | null;
@@ -57,7 +59,8 @@ export const emptyComposer = (prefs?: Partial<Composer>): Composer => ({
prompt: "",
refs: [],
mask: null,
- inpaintAction: "edit",
+ // Remove first: the most common brushed edit, and the one that needs no words.
+ inpaintAction: "remove",
outpaint: null,
mode: "auto",
aspect: prefs?.aspect ?? "auto",
@@ -69,6 +72,7 @@ export const emptyComposer = (prefs?: Partial<Composer>): Composer => ({
advanced: newChatAdvanced(prefs?.advanced),
enhance: prefs?.enhance === true,
enhanceReview: prefs?.enhanceReview === true,
+ enhanceDetail: prefs?.enhanceDetail === true,
enhancedFrom: null,
create: false,
loras: null,
@@ -142,7 +146,8 @@ export function buildRequest(c: Composer, parentId?: string | null): GenerationR
const custom = c.resolution === "custom";
// Inpaint and expand keep the source's shape; create, edit and combine take the chosen ratio.
const shaped = !maskValid && !outpaintInUse(c);
- const reviewed = !!c.enhancedFrom && !!c.prompt.trim();
+ // Written by Prompt+ already (from the user's words, or from none: "" for a removal or Expand).
+ const reviewed = c.enhancedFrom != null && !!c.prompt.trim();
return {
prompt: c.prompt.trim(),
mode: composing(c) ? "compose" : c.mode,
@@ -162,13 +167,19 @@ export function buildRequest(c: Composer, parentId?: string | null): GenerationR
client: "ui",
parent_id: parentId ?? null,
// A prompt Prompt+ wrote already is sent as it is, with the user's words kept beside it.
- ...(reviewed ? { enhanced_from: c.enhancedFrom } : c.enhance && enhancerApplies(c) ? { enhance: true } : {}),
+ ...(reviewed ? { enhanced_from: c.enhancedFrom } : c.enhance ? { enhance: true, ...(c.enhanceDetail ? { enhance_detail: true } : {}) } : {}),
};
}
-/** Prompt+ in review mode: Enter first has the prompt written into the box (no image yet). */
+/** Prompt+ in review mode: Enter first has the prompt written into the box (no image yet). A
+ * removal or an expansion with nothing typed is always written into the box first: Prompt+ names
+ * what is removed, or what the new areas show, for the user to approve or change. Prompt+ works on
+ * everything the prompt box makes: new images, edits, combines, coloured marks, a painted area
+ * (crates/qls-core/src/enhancer.rs AREA_SYSTEM_PROMPT) and Expand (EXPAND_SYSTEM_PROMPT). */
export function wantsPromptReview(c: Composer): boolean {
- return c.enhance && c.enhanceReview && enhancerApplies(c) && !c.enhancedFrom && !!c.prompt.trim();
+ if (!c.enhance || c.enhancedFrom != null) return false;
+ if (!typedWords(c)) return !needsPrompt(c) && (maskInUse(c) || outpaintInUse(c));
+ return c.enhanceReview;
}
/** The request that only writes the prompt (Prompt+ review). */
@@ -176,18 +187,28 @@ export function reviewRequest(c: Composer): GenerationRequest {
return { ...buildRequest(c), enhance: true, enhance_only: true };
}
-/** The prompt enhancer works on new images, edits, combines and coloured marks; a painted area
- * or an expansion wraps the prompt in its own instruction. */
-export function enhancerApplies(c: Composer): boolean {
- return (!maskInUse(c) || marksInUse(c)) && !outpaintInUse(c);
-}
-
/** The painted area in use is coloured marks (the prompt names them by colour). */
export const marksInUse = (c: Composer) => maskInUse(c) && !!c.mask!.marks;
-/** False only for a removal in a painted area, which needs no description. */
+/** The words each action of a painted area puts at the start of the prompt (the service takes
+ * them off again: planner.rs InpaintAction::subject). */
+export const ACTION_VERB: Record<InpaintAction, string> = { remove: "Remove", edit: "Change", replace: "Replace with", add: "Add" };
+const VERB_RE = /^\s*(?:remove|change|replace with|replace|add)\b[ \t]*/i;
+
+/** The prompt with `action`'s words in front, in place of another action's. */
+export function withActionVerb(prompt: string, action: InpaintAction): string {
+ return `${ACTION_VERB[action]} ${prompt.replace(VERB_RE, "")}`;
+}
+
+/** What the user typed: for a painted area, without the action's own words ("Remove " alone is nothing). */
+export function typedWords(c: Composer): string {
+ return maskInUse(c) && !c.mask!.marks ? c.prompt.replace(VERB_RE, "").trim() : c.prompt.trim();
+}
+
+/** False for a removal in a painted area, which needs no description, and for Expand, which
+ * continues the scene (Prompt+ can write what the new areas show). */
export function needsPrompt(c: Composer): boolean {
- return !(maskInUse(c) && !c.mask!.marks && c.inpaintAction === "remove");
+ return !((maskInUse(c) && !c.mask!.marks && c.inpaintAction === "remove") || outpaintInUse(c));
}
/** Advanced fields that change how the model samples. They last for the session only: a value
@@ -386,6 +407,37 @@ export function timeAgo(ms: number, now = Date.now()): string {
return new Date(ms).toLocaleDateString();
}
+/** A project's title as the list shows it: an image file's name without its extension. */
+export function projectTitle(title: string): string {
+ return title.replace(/\.(png|jpe?g|webp|heic|heif|avif|gif|bmp|tiff?)$/i, "").trim() || title;
+}
+
+/** A file name that says nothing about the picture ("image.png", "4.png", "IMG_1234.JPG",
+ * "Screenshot 2026-10-01 at 10.00.00"): a project named after it takes its first prompt instead. */
+export function isGenericName(name: string): boolean {
+ // "Screenshot 2026-10-01 at 10.00.00": the "at" of a date says nothing either.
+ const n = projectTitle(name).toLowerCase().replace(/ (at|alle) /g, " ");
+ return (
+ /^[\d\s_.()-]*$/.test(n) ||
+ /^(image|img|photo|pic|picture|untitled|unnamed|download|file|imported image|whatsapp image|screenshot|screen shot|schermata|immagine|foto)[\s\d_.()\-:,]*$/.test(n) ||
+ /^(img|dsc|dscn|pxl|mvimg)[_-]?\d[\d_.-]*$/.test(n)
+ );
+}
+
+/** The heading a project goes under in the list, by when it was last changed. */
+export function dayGroup(ms: number, now = Date.now()): string {
+ const day = (t: number) => {
+ const d = new Date(t);
+ return new Date(d.getFullYear(), d.getMonth(), d.getDate()).getTime();
+ };
+ const days = Math.round((day(now) - day(ms)) / 86400000);
+ if (days <= 0) return "Today";
+ if (days === 1) return "Yesterday";
+ if (days < 7) return "Previous 7 days";
+ if (days < 30) return "Previous 30 days";
+ return "Older";
+}
+
/** Short truthful status text for a job. */
export function jobLabel(j: JobView): string {
switch (j.status) {
@@ -821,6 +873,17 @@ export function expandToSize(w: number, h: number, W: number, H: number): Outpai
return { left: Math.floor(ex / 2), right: ex - Math.floor(ex / 2), top: Math.floor(ey / 2), bottom: ey - Math.floor(ey / 2), units: "pixels" };
}
+/** The image moved to `left`, `top` (source px) inside the same expanded canvas: the extra space
+ * shifts between opposite sides, the final size stays. Snapped to 8 px; within 4 px of an edge, on it. */
+export function placeInCanvas(p: { left: number; right: number; top: number; bottom: number }, left: number, top: number): OutpaintSpec {
+ const snap = (v: number, max: number) => (v >= max - 4 ? max : Math.min(max, Math.max(0, Math.round(v / 8) * 8)));
+ const ex = p.left + p.right;
+ const ey = p.top + p.bottom;
+ const l = snap(left, ex);
+ const t = snap(top, ey);
+ return { left: l, right: ex - l, top: t, bottom: ey - t, units: "pixels" };
+}
+
/** Expansion presets in source pixels. */
export function expansionPreset(kind: "left" | "right" | "top" | "bottom" | "landscape" | "portrait" | "square", w: number, h: number, cur: OutpaintSpec): OutpaintSpec {
const base: OutpaintSpec = { left: cur.left, right: cur.right, top: cur.top, bottom: cur.bottom, units: "pixels" };
diff --git a/src/lib/matte.test.ts b/src/lib/matte.test.ts
index 1f3ca07..3011e30 100644
--- a/src/lib/matte.test.ts
+++ b/src/lib/matte.test.ts
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
-import { foreground, stickerAlpha } from "./matte";
+import { foreground, keptShare, MIN_KEPT, stickerAlpha } from "./matte";
const W = 64;
const H = 64;
@@ -36,6 +36,17 @@ describe("fast sticker alpha", () => {
});
});
+describe("quick cut-out check", () => {
+ it("counts how much of the selection the sticker keeps", () => {
+ const sel = Uint8Array.from([255, 255, 255, 255, 0, 0]);
+ expect(keptShare(Float32Array.from([1, 1, 0.9, 0.6, 1, 1]), sel)).toBe(1);
+ expect(keptShare(Float32Array.from([1, 0, 0, 0, 1, 1]), sel)).toBe(0.25);
+ // An empty matte (the model failed or saw nothing): fails the check, the layer stays.
+ expect(keptShare(new Float32Array(6), sel)).toBeLessThan(MIN_KEPT);
+ expect(keptShare(new Float32Array(6), new Uint8Array(6))).toBe(0);
+ });
+});
+
describe("foreground colours", () => {
it("takes the background's tint out of a soft edge", () => {
// White object on red: the edge pixels are half white, half red; the estimate gets nearer white.
diff --git a/src/lib/matte.ts b/src/lib/matte.ts
index 8e99d45..2243044 100644
--- a/src/lib/matte.ts
+++ b/src/lib/matte.ts
@@ -124,6 +124,23 @@ export function stickerAlpha(matte: Float32Array, sel: Mask, w: number, h: numbe
return dropsPiece(a, sel, w, h) ? { alpha: bandAlpha(matte, sel, w, h), fellBack: true } : { alpha: a, fellBack: false };
}
+/** How much of the selection the sticker keeps (0-1): the share of selected pixels with alpha over
+ * one half. A quick cut-out that keeps almost nothing failed (an empty or wrong matte), and must
+ * say so instead of putting an invisible layer in place of the object. */
+export function keptShare(alpha: Float32Array, sel: Mask): number {
+ let inside = 0;
+ let kept = 0;
+ for (let i = 0; i < sel.length; i++) {
+ if (sel[i] <= 127) continue;
+ inside++;
+ if (alpha[i] > 0.5) kept++;
+ }
+ return inside ? kept / inside : 0;
+}
+
+/** Below this share of its selection kept, a quick cut-out counts as failed. */
+export const MIN_KEPT = 0.2;
+
/** Foreground colours of `rgba` under `alpha` (0-1): what the edge pixels would be without the
* background mixed in (multilevel estimation of Germer, Uelwer, Conrad, Harmeling 2020, as in
* pymatting's estimate_foreground_ml: coarse to fine, F and B solved per pixel from the neighbours). */
diff --git a/src/lib/psd.test.ts b/src/lib/psd.test.ts
index 2ae5abd..967a804 100644
--- a/src/lib/psd.test.ts
+++ b/src/lib/psd.test.ts
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
-import { packBits, writePsd } from "./psd";
+import { aiMarkingXmp, packBits, writePsd } from "./psd";
function unpack(p: ArrayLike<number>, at: number, len: number): number[] {
const out: number[] = [];
@@ -36,7 +36,19 @@ function readPsd(b: Uint8Array) {
const width = u32();
expect([u16(), u16()]).toEqual([8, 3]);
skip();
- skip();
+ // Image resources: 8BIM blocks (id, even-padded Pascal name, even-padded data).
+ const resources = new Map<number, Uint8Array>();
+ const resEnd = u32() + o;
+ while (o < resEnd) {
+ expect(s(4)).toBe("8BIM");
+ const id = u16();
+ const nl = u8();
+ o += nl + ((nl + 1) % 2);
+ const len = u32();
+ resources.set(id, b.slice(o, o + len));
+ o += len + (len % 2);
+ }
+ expect(o).toBe(resEnd);
const masksLen = u32();
const masksEnd = o + masksLen;
const infoLen = u32();
@@ -100,7 +112,7 @@ function readPsd(b: Uint8Array) {
o += n;
}
expect(o).toBe(b.length);
- return { width, height, channels, recs, flat };
+ return { width, height, channels, recs, flat, resources };
}
describe("psd", () => {
@@ -139,4 +151,20 @@ describe("psd", () => {
expect(out.flat.length).toBe(3 * W * H);
expect(out.flat.slice(0, W)).toEqual([0, 1, 2, 3, 4, 5]);
});
+
+ it("carries the AI marking as XMP (resource 1060) and still reads back", () => {
+ const px = new Uint8ClampedArray([1, 2, 3, 255, 4, 5, 6, 255, 7, 8, 9, 255]);
+ const x = aiMarkingXmp("0.12.0", new Date(2026, 10, 4, 9, 5, 7));
+ expect(x).toContain('Iptc4xmpExt:DigitalSourceType="http://cv.iptc.org/newscodes/digitalsourcetype/compositeWithTrainedAlgorithmicMedia"');
+ expect(x).toContain('xmp:CreatorTool="Kokon 0.12.0"');
+ expect(x).toMatch(/xmp:CreateDate="2026-11-04T09:05:07[+-]\d\d:\d\d"/);
+ expect(aiMarkingXmp('a"<b')).toContain('Kokon a&quot;&lt;b"');
+ // Odd and even packet lengths both keep the sections aligned.
+ for (const packet of [x, x + " "]) {
+ const out = readPsd(writePsd(3, 1, [{ name: "L", hidden: false, rgba: px }], px, packet));
+ expect(new TextDecoder().decode(out.resources.get(1060))).toBe(packet);
+ expect(out.flat.slice(0, 3)).toEqual([1, 4, 7]);
+ }
+ expect(readPsd(writePsd(3, 1, [], px)).resources.size).toBe(0);
+ });
});
diff --git a/src/lib/psd.ts b/src/lib/psd.ts
index f26c61f..0509f20 100644
--- a/src/lib/psd.ts
+++ b/src/lib/psd.ts
@@ -1,6 +1,24 @@
// Layered Photoshop file (.psd, 8-bit RGB) of the Layers lab's document: each layer a pixel layer
// cropped to what it shows, RLE (PackBits) channels, and the flattened image for apps that read
-// only that. Layer names in Unicode ('luni'), hidden layers kept hidden.
+// only that. Layer names in Unicode ('luni'), hidden layers kept hidden. The AI marking (as in
+// crates/qls-core/src/provenance.rs) is an XMP packet in image resource 1060, where Photoshop keeps its own.
+
+/** The XMP marking a layered lab file: IPTC Digital Source Type "composite with trained algorithmic media"
+ * (the lab's layers come from the model: split, filled behind, blended), Kokon as the creator tool, the date. */
+export function aiMarkingXmp(appVersion: string, date = new Date()): string {
+ const pad = (n: number) => String(Math.floor(Math.abs(n))).padStart(2, "0");
+ const off = -date.getTimezoneOffset();
+ const when =
+ `${date.getFullYear()}-${pad(date.getMonth() + 1)}-${pad(date.getDate())}T${pad(date.getHours())}:${pad(date.getMinutes())}:${pad(date.getSeconds())}` +
+ `${off < 0 ? "-" : "+"}${pad(off / 60)}:${pad(off % 60)}`;
+ const esc = (s: string) => s.replace(/&/g, "&amp;").replace(/"/g, "&quot;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
+ return (
+ `<?xpacket begin="\u{feff}" id="W5M0MpCehiHzreSzNTczkc9d"?><x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Kokon"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">` +
+ `<rdf:Description rdf:about="" xmlns:Iptc4xmpExt="http://iptc.org/std/Iptc4xmpExt/2008-02-29/" xmlns:xmp="http://ns.adobe.com/xap/1.0/" ` +
+ `Iptc4xmpExt:DigitalSourceType="http://cv.iptc.org/newscodes/digitalsourcetype/compositeWithTrainedAlgorithmicMedia" ` +
+ `xmp:CreatorTool="Kokon ${esc(appVersion)}" xmp:CreateDate="${when}"/></rdf:RDF></x:xmpmeta><?xpacket end="w"?>`
+ );
+}
export interface PsdLayer {
name: string;
@@ -114,8 +132,9 @@ function opaqueBox(rgba: Uint8ClampedArray, W: number, H: number) {
return x1 < 0 ? null : { x0, y0, w: x1 - x0 + 1, h: y1 - y0 + 1 };
}
-/** The .psd file: `layers` top first (as the lab lists them), `flat` the flattened image (opaque). */
-export function writePsd(width: number, height: number, layers: PsdLayer[], flat: Uint8ClampedArray): Uint8Array {
+/** The .psd file: `layers` top first (as the lab lists them), `flat` the flattened image (opaque), `xmp` the
+ * metadata packet (aiMarkingXmp) as image resource 1060. */
+export function writePsd(width: number, height: number, layers: PsdLayer[], flat: Uint8ClampedArray, xmp?: string): Uint8Array {
const w = new Writer();
// Header: RGB, 8 bits, 3 channels in the flattened image.
w.ascii("8BPS");
@@ -127,7 +146,18 @@ export function writePsd(width: number, height: number, layers: PsdLayer[], flat
w.u16(8);
w.u16(3);
w.u32(0); // colour mode data
- w.u32(0); // image resources
+ // Image resources: one block, "8BIM", id 1060 (XMP), an empty Pascal name padded to even, the UTF-8 packet padded to even.
+ const endResources = w.length();
+ if (xmp) {
+ const data = new TextEncoder().encode(xmp);
+ w.ascii("8BIM");
+ w.u16(1060);
+ w.u16(0);
+ w.u32(data.length);
+ w.bytes(data);
+ if (data.length % 2) w.u8(0);
+ }
+ endResources();
const endMasks = w.length();
const endInfo = w.length();
diff --git a/src/lib/sam.ts b/src/lib/sam.ts
index 6d5df86..301eea1 100644
--- a/src/lib/sam.ts
+++ b/src/lib/sam.ts
@@ -1,15 +1,65 @@
// Click-to-select for the layers lab (components/LayersEditor.tsx): MobileSAM (Apache-2.0, Acly's
-// ONNX export, MIT) run in the webview with onnxruntime-web. Experimental: the model files are not
-// shipped; the vite dev server serves them from `lab-models/mobilesam/` (gitignored). The planned
-// app version runs SAM in kokon-service instead.
+// ONNX export, MIT) run in the webview with onnxruntime-web. kokon-service downloads the two files
+// the first time Select is used (44 MB, pinned: crates/qls-core/src/matting.rs MOBILE_SAM) and serves
+// them; without the service's copy, the vite dev server's `lab-models/mobilesam/` (gitignored).
//
// The encoder runs once per image (longest side 1024, 0-255 RGB, HWC); the decoder runs per click
// on the cached embeddings and returns SAM's 4 masks at the size asked for: index 0 = the single-mask
// output, 1-3 = sub-part / part / whole (their order and the predicted IoU are unreliable, S1).
import type * as Ort from "onnxruntime-web";
+import { api, urls } from "./api";
-const MODEL_DIR = "/lab-models/mobilesam/";
+const DEV_DIR = "/lab-models/mobilesam/";
+const ENCODER = "mobile_sam_image_encoder.onnx";
+const DECODER = "sam_mask_decoder_multi.onnx";
+
+/** Everyone waiting for the one download (the lab's split and scene both load the model at once). */
+const progressListeners = new Set<(pct: number) => void>();
+let downloading: Promise<void> | null = null;
+
+/** Downloads the files through kokon-service and waits for it. A request the service refuses
+ * because an install is already running (the other caller's) still waits for that install: before,
+ * the refusal read as "could not be downloaded" while the download went on (Mac, 2026-11-04). */
+async function download() {
+ try {
+ await api.downloadSegment();
+ } catch (e) {
+ console.warn("segment download not started (another install running?)", e);
+ }
+ progressListeners.forEach((f) => f(0));
+ for (;;) {
+ await new Promise((r) => window.setTimeout(r, 700));
+ const i = await api.installStatus().catch(() => null);
+ if (!i) break;
+ const items = i.items.filter((x) => x.id.startsWith("mobile_sam"));
+ if (items.some((x) => x.state === "failed" || x.state === "cancelled")) break;
+ const size = items.reduce((n, x) => n + (x.size || 0), 0);
+ if (size) {
+ const pct = Math.floor((100 * items.reduce((n, x) => n + x.received, 0)) / size);
+ progressListeners.forEach((f) => f(pct));
+ }
+ if (!i.running) break;
+ }
+}
+
+/** Where the model files are: kokon-service's copy (downloaded now if it isn't there, `onProgress`
+ * 0-100), else the dev server's. Null when neither has them. */
+async function modelUrls(onProgress?: (pct: number) => void): Promise<((file: string) => string) | null> {
+ const st = await api.segmentStatus().catch(() => null);
+ if (st && !st.installed) {
+ if (onProgress) progressListeners.add(onProgress);
+ try {
+ await (downloading ??= download().finally(() => (downloading = null)));
+ } finally {
+ if (onProgress) progressListeners.delete(onProgress);
+ }
+ }
+ if ((await api.segmentStatus().catch(() => null))?.installed) return urls.segmentModel;
+ const head = await fetch(DEV_DIR + DECODER, { method: "HEAD" }).catch(() => null);
+ if (head?.ok && !head.headers.get("content-type")?.includes("text/html")) return (f) => DEV_DIR + f;
+ return null;
+}
const SIDE = 1024;
let ort: Promise<typeof Ort> | null = null;
@@ -33,10 +83,10 @@ export class Segmenter {
readonly device: string,
) {}
- /** Null when the model files are not there (a release build). */
- static async load(webgpu = true): Promise<Segmenter | null> {
- const head = await fetch(MODEL_DIR + "sam_mask_decoder_multi.onnx", { method: "HEAD" }).catch(() => null);
- if (!head?.ok || head.headers.get("content-type")?.includes("text/html")) return null;
+ /** Null when the model files could not be had (no service copy, the download failed). */
+ static async load(webgpu = true, onProgress?: (pct: number) => void): Promise<Segmenter | null> {
+ const at = await modelUrls(onProgress);
+ if (!at) return null;
const o = await loadOrt();
o.env.logLevel = "error";
// WebGPU when the webview has it, else WASM (one thread: the page is not cross-origin isolated).
@@ -44,14 +94,14 @@ export class Segmenter {
let device = "wasm";
if (webgpu && "gpu" in navigator) {
try {
- enc = await o.InferenceSession.create(MODEL_DIR + "mobile_sam_image_encoder.onnx", { executionProviders: ["webgpu"] });
+ enc = await o.InferenceSession.create(at(ENCODER), { executionProviders: ["webgpu"] });
device = "webgpu";
} catch {
enc = null;
}
}
- enc ??= await o.InferenceSession.create(MODEL_DIR + "mobile_sam_image_encoder.onnx", { executionProviders: ["wasm"] });
- const dec = await o.InferenceSession.create(MODEL_DIR + "sam_mask_decoder_multi.onnx", { executionProviders: ["wasm"] });
+ enc ??= await o.InferenceSession.create(at(ENCODER), { executionProviders: ["wasm"] });
+ const dec = await o.InferenceSession.create(at(DECODER), { executionProviders: ["wasm"] });
return new Segmenter(o, enc, dec, device);
}
diff --git a/src/lib/store.ts b/src/lib/store.ts
index ba362fb..c6a3723 100644
--- a/src/lib/store.ts
+++ b/src/lib/store.ts
@@ -9,11 +9,11 @@ import { inBrowser, isNarrow } from "./platform";
import { useSyncExternalStore } from "react";
import { markFresh } from "../components/Cocoon";
import { api, ApiError, setPrivateInputs, subscribe, urls } from "./api";
-import { activeLoras, buildRequest, composing, emptyComposer, goneVersions, macEngineOfferDue, needsPrompt, newKey, pageOutdated, persistentAdvanced, refFromGen, refFromInput, reviewRequest, stageFitsPreview, starPromptDue, timelineLinks, undoTarget, versionLabels, wantsPromptReview, type CompareMode, type Composer, type Ref } from "./logic";
+import { activeLoras, buildRequest, composing, emptyComposer, goneVersions, isGenericName, macEngineOfferDue, needsPrompt, newKey, pageOutdated, persistentAdvanced, refFromGen, refFromInput, reviewRequest, stageFitsPreview, starPromptDue, timelineLinks, typedWords, undoTarget, versionLabels, wantsPromptReview, type CompareMode, type Composer, type Ref } from "./logic";
import { keepLabsIn, labsChanged, labSources, loadLab, type LabStore } from "./layerDoc";
import type { AppErrorBody, GenerationRecord, GenerationRequest, InputRecord, JobView, LoraItem, MacBenchState, ServerState, StatusResponse } from "./types";
-export type Overlay = null | "gallery" | "settings" | "inpaint" | "expand" | "layers" | "welcome" | "whats-new" | "mac-engine" | "prompt-tips" | "shortcuts";
+export type Overlay = null | "gallery" | "settings" | "inpaint" | "expand" | "layers" | "welcome" | "whats-new" | "mac-engine" | "prompt-tips" | "shortcuts" | "ai-check";
export interface SessionItem {
key: string;
@@ -98,6 +98,8 @@ export interface State {
loras: LoraItem[];
/** Tab Settings opens on (then cleared). */
settingsTab: string | null;
+ /** A section of that tab to scroll to once open (its element id is `settings-<name>`). */
+ settingsFocus: string | null;
/** Prompt+ is writing a prompt into the box for review: its job, and the user's words. */
review: { jobId: string; prompt: string } | null;
}
@@ -118,9 +120,9 @@ function loadPrefs(): Partial<Composer> {
function savePrefs(c: Composer) {
try {
- const { quality, resolution, aspect, customWidth, customHeight, enhance, enhanceReview } = c;
+ const { quality, resolution, aspect, customWidth, customHeight, enhance, enhanceReview, enhanceDetail } = c;
const advanced = persistentAdvanced(c.advanced);
- localStorage.setItem(PREFS_KEY, JSON.stringify({ quality, resolution, aspect, customWidth, customHeight, advanced, enhance, enhanceReview }));
+ localStorage.setItem(PREFS_KEY, JSON.stringify({ quality, resolution, aspect, customWidth, customHeight, advanced, enhance, enhanceReview, enhanceDetail }));
} catch {
/* storage unavailable */
}
@@ -202,7 +204,8 @@ keepLabsIn({
return;
}
labsHere.delete(id);
- projects[id] = { ...projects[id], lab };
+ const p = projects[id];
+ projects[id] = { ...p, lab, meta: { ...p.meta, cover: lastCover(p.items, lab) } };
hideLabSteps(lab);
persistProjects([id]);
},
@@ -330,10 +333,16 @@ const toStored = (i: SessionItem): StoredItem => ({
});
function titleOf(items: SessionItem[]): string | null {
+ const prompt = (i: SessionItem) => i.gen?.original_prompt || i.gen?.prompt || i.request?.prompt;
for (const i of items) {
- const p = i.gen?.original_prompt || i.gen?.prompt || i.request?.prompt;
+ const p = prompt(i);
if (p) return p;
- if (i.input) return i.input.original_name ? safeDecode(i.input.original_name) : "Imported image";
+ if (i.input) {
+ const name = i.input.original_name ? safeDecode(i.input.original_name) : "Imported image";
+ // A file name that says nothing ("image.png", "IMG_1234.jpg"): the project's first prompt,
+ // so it can be told apart in the list.
+ return isGenericName(name) ? (items.map(prompt).find(Boolean) ?? name) : name;
+ }
}
return null;
}
@@ -346,10 +355,14 @@ function safeDecode(s: string) {
}
}
-const lastCover = (items: StoredItem[]) => {
+/** The cover: the latest image that is not a layers lab step (one object's crop, a fill...): the
+ * lab's `made` list (lib/layerDoc.ts), as the service does (crates/qls-core/src/projects.rs). */
+const lastCover = (items: StoredItem[], lab?: unknown) => {
+ const made = (lab as { made?: unknown } | undefined)?.made;
+ const steps = new Set(Array.isArray(made) ? (made as string[]) : []);
for (let k = items.length - 1; k >= 0; k--) {
const s = storedSource(items[k]);
- if (s) return s;
+ if (s && !steps.has(s)) return s;
}
return null;
};
@@ -374,7 +387,7 @@ function syncProject(prev: State) {
title: old?.meta.named ? old.meta.title : titleOf(state.session) ?? old?.meta.title ?? "Untitled",
named: old?.meta.named,
updatedAt: edited || !old ? Date.now() : old.meta.updatedAt,
- cover: lastCover(items),
+ cover: lastCover(items, old?.lab ?? labsHere.get(id)),
count: countOf(items),
},
items,
@@ -470,6 +483,7 @@ let state: State = {
starPrompt: false,
loras: [],
settingsTab: null,
+ settingsFocus: null,
review: null,
};
@@ -698,7 +712,7 @@ function onJob(job: JobView) {
p.items = [...p.items.slice(0, k), ...stored, ...p.items.slice(k + 1)];
if (wasSelected && stored[0]) p.selected = stored[0].key;
// Shown at once; the service made the same change and saves it (no save from here).
- p.meta = { ...p.meta, cover: lastCover(p.items), count: countOf(p.items), updatedAt: Date.now() };
+ p.meta = { ...p.meta, cover: lastCover(p.items, p.lab), count: countOf(p.items), updatedAt: Date.now() };
return { jobs, projects: projectList() };
}
// A job from another client (e.g. MCP): track it but do not add it to a project.
@@ -867,8 +881,8 @@ export async function refreshLoras() {
}
/** Opens Settings on one of its tabs. */
-export function openSettings(tab: string) {
- setState({ overlay: "settings", settingsTab: tab });
+export function openSettings(tab: string, focus: string | null = null) {
+ setState({ overlay: "settings", settingsTab: tab, settingsFocus: focus });
}
/** Adds the example projects that ship with Kokon (made by Kokon itself), except the ones
@@ -995,7 +1009,7 @@ export function forgetGeneration(id: string) {
for (const p of Object.values(projects)) {
if (!p.items.some((i) => i.genId === id)) continue;
p.items = p.items.filter((i) => i.genId !== id);
- p.meta = { ...p.meta, cover: lastCover(p.items), count: countOf(p.items) };
+ p.meta = { ...p.meta, cover: lastCover(p.items, p.lab), count: countOf(p.items) };
changed.push(p.meta.id);
}
persistProjects(changed);
@@ -1238,7 +1252,7 @@ export async function submitComposer() {
const req: GenerationRequest = { ...buildRequest(s.composer, parent), loras: activeLoras(s.composer, s.loras) };
// Made from the layers lab: a step, out of the gallery (only the image saved from the lab shows).
if (s.overlay === "layers" || isLabStep(target)) req.hidden = true;
- if (!req.prompt && needsPrompt(s.composer)) {
+ if (!typedWords(s.composer) && needsPrompt(s.composer)) {
toastInfo("Describe what to create or change.");
return;
}
@@ -1313,13 +1327,20 @@ export function focusPrompt(cursorToEnd = false) {
}
export function removeSessionItem(key: string) {
+ removeSessionItems([key]);
+}
+
+/** Versions taken out of the open project (their images stay in All images). */
+export function removeSessionItems(keys: string[]) {
const s = getState();
- const removed = s.session.find((i) => i.key === key);
- const session = s.session.filter((i) => i.key !== key);
- const src = removed ? itemSource(removed) : null;
- const refs = src ? s.composer.refs.filter((r) => r.source !== src) : s.composer.refs;
+ const gone = new Set(keys);
+ const removed = s.session.filter((i) => gone.has(i.key));
+ if (!removed.length) return;
+ const session = s.session.filter((i) => !gone.has(i.key));
+ const srcs = new Set(removed.map(itemSource).filter(Boolean));
+ const refs = s.composer.refs.filter((r) => !srcs.has(r.source));
setState({ session, composer: refs.length === s.composer.refs.length ? s.composer : { ...s.composer, refs, mask: null, outpaint: null } });
- if (s.selected === key) {
+ if (s.selected && gone.has(s.selected)) {
const next = session[session.length - 1];
if (next) selectItem(next.key);
else setState({ selected: null });
diff --git a/src/lib/types.ts b/src/lib/types.ts
index 24c48af..3907d31 100644
--- a/src/lib/types.ts
+++ b/src/lib/types.ts
@@ -64,6 +64,8 @@ export interface GenerationRequest {
enhance?: boolean;
/** Prompt+ review: only write the prompt (the job ends with `enhanced_prompt` and no image). */
enhance_only?: boolean;
+ /** Prompt+ "Detailed": a longer prompt with more context about what is there or what changes. */
+ enhance_detail?: boolean;
/** Layers lab "Blend into scene": only read the place the object with this name stands in
* (the job ends with `scene` and no image). */
read_scene?: string;
diff --git a/src/styles.css b/src/styles.css
index 7045f13..c2e78c4 100644
--- a/src/styles.css
+++ b/src/styles.css
@@ -125,18 +125,41 @@ svg { display: block; flex: none; }
.side-item { display: flex; align-items: center; gap: 10px; height: 38px; padding: 0 12px; border-radius: 12px; color: var(--text-2); font-size: 14px; text-align: left; transition: background 0.15s, box-shadow 0.2s, color 0.15s; }
.side-item:hover { background: var(--surface); color: var(--text); }
.side-item.on { background: var(--bg); color: var(--text); box-shadow: var(--lift); font-weight: 500; }
-.side-label { display: flex; align-items: center; justify-content: space-between; font-size: 11.5px; color: var(--text-3); padding: 8px 14px 4px 20px; letter-spacing: 0.02em; }
-.side-select { height: 24px; padding: 0 8px; border-radius: 7px; font-size: 11.5px; color: var(--text-3); opacity: 0; transition: opacity 0.15s, background 0.15s, color 0.15s; }
-.sidebar:hover .side-select, .sidebar.selecting .side-select, .side-select:focus-visible { opacity: 1; }
+.side-label { display: flex; align-items: center; justify-content: space-between; gap: 6px; padding: 0 12px 2px 8px; }
+/* "Projects": a row like New image / All images (its folder icon at the same place); it and the day
+ * headings fold their list (Sidebar.tsx), the chevron turns when folded. */
+.side-fold { flex: 1; min-width: 0; }
+.side-group { display: inline-flex; align-items: center; gap: 4px; border-radius: 7px; color: var(--text-3); font-weight: 600; transition: color 0.15s, background 0.15s; }
+.side-group:hover { color: var(--text); background: var(--surface); }
+.side-fold .fold-chev, .side-group svg { opacity: 0; color: var(--text-3); transition: opacity 0.15s, transform 0.2s var(--ease); }
+.side-fold:hover .fold-chev, .side-group:hover svg, .side-fold.closed .fold-chev, .side-group.closed svg, .side-fold:focus-visible .fold-chev, .side-group:focus-visible svg { opacity: 1; }
+.side-fold.closed .fold-chev, .side-group.closed svg { transform: rotate(-90deg); }
+/* Under "Projects": a trunk from its folder icon (centre 8 + 12 + 8 px), a rounded branch to each
+ * day heading; the last heading's branch ends the trunk. */
+/* Opaque (mixed into the sidebar's colour): where the trunk and a branch overlap it is not darker. */
+.side-tree { --tree-line: color-mix(in srgb, var(--text) 16%, var(--ground)); display: flex; flex-direction: column; margin-left: 19px; }
+.side-sec { position: relative; display: flex; flex-direction: column; gap: 2px; padding-left: 14px; }
+.side-sec::before { content: ""; position: absolute; left: 0; top: 0; bottom: 0; border-left: 2px solid var(--tree-line); }
+.side-sec:last-child::before { bottom: auto; height: 9px; }
+.side-sec:first-child:last-child::before { height: 3px; }
+.side-sec > .side-group { position: relative; }
+.side-sec > .side-group::before { content: ""; position: absolute; left: -14px; top: calc(50% - 10px); width: 11px; height: 10px; border-left: 2px solid var(--tree-line); border-bottom: 2px solid var(--tree-line); border-bottom-left-radius: 9px; pointer-events: none; }
+/* Always visible: a small outlined button, not text that shows on hover. */
+.side-select { height: 24px; padding: 0 8px; display: inline-flex; align-items: center; gap: 5px; border-radius: 7px; border: 1px solid var(--border-2); font-size: 11.5px; font-weight: 500; color: var(--text-2); transition: background 0.15s, color 0.15s, border-color 0.15s; }
.side-select:hover { background: var(--surface); color: var(--text); }
-.side-list { flex: 1; overflow-y: auto; padding: 2px 8px 12px; display: flex; flex-direction: column; gap: 2px; }
-.proj { display: flex; align-items: center; gap: 11px; padding: 7px; border-radius: 14px; cursor: pointer; position: relative; outline-offset: -2px; transition: background 0.15s, box-shadow 0.2s; }
+.sidebar.selecting .side-select { background: var(--invert-bg); color: var(--invert-text); border-color: transparent; }
+.side-list { flex: 1; overflow-y: auto; padding: 2px 8px 12px; display: flex; flex-direction: column; }
+.proj { display: flex; align-items: center; gap: 10px; padding: 6px; border-radius: 14px; cursor: pointer; position: relative; outline-offset: -2px; transition: background 0.15s, box-shadow 0.2s; }
.proj:hover { background: var(--surface); }
.proj.on { background: var(--bg); box-shadow: var(--lift); }
-.proj .cover { width: 42px; height: 42px; border-radius: 10px; overflow: hidden; background: var(--surface-3); flex: none; }
+.proj .cover { width: 46px; height: 46px; border-radius: 11px; overflow: hidden; background: var(--surface-3); flex: none; }
.proj .cover img { width: 100%; height: 100%; object-fit: cover; display: block; }
.proj-text { flex: 1; min-width: 0; display: flex; flex-direction: column; }
-.proj-title { font-size: 13px; font-weight: 500; color: var(--text); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
+/* One line, cut with an ellipsis: the hover buttons only shorten it, never wrap it. */
+.proj-title { font-size: 13.5px; font-weight: 500; line-height: 1.3; color: var(--text); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
+/* Today, Yesterday...: the list's headings (Sidebar.tsx) */
+.side-group { align-self: flex-start; font-size: 11.5px; height: 22px; padding: 0 6px; margin: 8px 0 2px 0; }
+.side-sec:first-child > .side-group { margin-top: 2px; }
.proj-sub { font-size: 11.5px; color: var(--text-3); white-space: nowrap; }
.proj-rm { width: 24px; height: 24px; border-radius: 7px; display: none; place-items: center; color: var(--text-2); flex: none; }
.proj:hover .proj-rm { display: grid; }
@@ -357,6 +380,10 @@ svg { display: block; flex: none; }
}
/* what the shown version was made from */
.frame.src .frame-img { outline: 1.6px dashed var(--text-3); outline-offset: 3.2px; }
+/* picked with Ctrl/⌘-click (Delete removes them): a solid outline and a tick */
+.frame.picked .frame-img, .frame.picked:hover .frame-img { outline: 2px solid var(--text); outline-offset: 3px; }
+.frame.picked { position: relative; }
+.frame-pick { position: absolute; right: -6px; top: -6px; width: 18px; height: 18px; border-radius: 50%; display: grid; place-items: center; background: var(--invert-bg); color: var(--invert-text); box-shadow: var(--lift); pointer-events: none; }
.frame.sel { color: var(--text); }
.frame-label { display: flex; align-items: center; gap: 3px; font-size: 11px; line-height: 1; white-space: nowrap; font-variant-numeric: tabular-nums; }
.frame-badge { position: absolute; right: 3px; bottom: 3px; width: 17px; height: 17px; border-radius: 6px; background: var(--invert-bg); color: var(--invert-text); display: grid; place-items: center; }
@@ -393,9 +420,13 @@ svg { display: block; flex: none; }
.tool { height: 30px; padding: 0 11px; border-radius: 15px; border: 1px solid var(--border-2); transition: background 0.15s, color 0.15s; display: inline-flex; align-items: center; gap: 6px; font-size: 12px; color: var(--text-2); white-space: nowrap; flex: none; }
.tool:hover { background: var(--surface); color: var(--text); }
.tool.on { background: var(--surface-2); color: var(--text); border-color: transparent; }
+
.tool-group .tool.on { border-radius: 15px 0 0 15px; padding-right: 6px; }
.tool-x { height: 30px; padding: 0 9px 0 4px; border-radius: 0 15px 15px 0; background: var(--surface-2); color: var(--text-2); display: grid; place-items: center; }
.tool-x:hover { color: var(--text); }
+/* The Layers lab (experimental): gold, as its Experimental badge. */
+.tool.lab { color: var(--okay); background: var(--okay-bg); border-color: color-mix(in srgb, var(--okay) 35%, transparent); }
+.tool.lab:hover { color: var(--okay); background: color-mix(in srgb, var(--okay) 20%, transparent); border-color: color-mix(in srgb, var(--okay) 55%, transparent); }
.refs { display: flex; gap: 6px; overflow-x: auto; scrollbar-width: none; flex: none; max-width: 45%; padding: 6px 6px 0 0; margin-top: -6px; }
.ref { position: relative; width: 40px; height: 40px; border-radius: 10px; overflow: visible; flex: none; cursor: grab; }
.ref img { width: 100%; height: 100%; object-fit: cover; border-radius: 11px; box-shadow: var(--lift); display: block; background: var(--surface-2); }
@@ -546,6 +577,8 @@ pre.diag { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font
.marks-want { display: flex; align-items: center; gap: 8px; width: min(480px, 100%); }
.marks-want input { flex: 1; min-width: 0; height: 30px; padding: 0 8px; border: 1px solid var(--border); border-radius: 8px; background: var(--bg); color: var(--text); font: inherit; }
.marks-want input:focus { outline: none; border-color: var(--border-2); }
+.marks-want.on-stage { position: absolute; z-index: 2; width: 320px; padding: 4px 4px 4px 8px; border-radius: 10px; background: var(--surface); box-shadow: 0 4px 16px rgba(0, 0, 0, 0.35); cursor: auto; }
+.marks-want.on-stage input { height: 28px; }
.marks-colours { gap: 6px; padding: 5px 8px; }
.layers-panel { width: 300px; flex: none; display: flex; flex-direction: column; gap: 6px; padding: 0 0 4px; overflow-y: auto; font-size: 12px; }
.layers-title { display: flex; align-items: center; gap: 6px; font-weight: 600; font-size: 13px; }
@@ -573,6 +606,7 @@ pre.diag { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font
.doc-mask-chip { position: absolute; top: 12px; left: 50%; transform: translateX(-50%); z-index: 3; padding: 6px 12px; border-radius: 999px; background: #ff3b6b; color: #fff; font-size: 12px; font-weight: 500; white-space: nowrap; pointer-events: none; box-shadow: var(--lift); }
.doc-sel { position: absolute; border: 1.5px dashed #fff; outline: 1px solid rgba(0, 0, 0, 0.45); pointer-events: none; box-sizing: border-box; }
.doc-handle { position: absolute; background: #fff; border: 1.5px solid #000; border-radius: 2px; pointer-events: auto; cursor: nwse-resize; box-sizing: border-box; }
+.doc-handle.tr, .doc-handle.bl { cursor: nesw-resize; }
.brush-cursor.pencil { mix-blend-mode: normal; }
.doc-sel-part { border-style: solid; outline: none; opacity: 0.6; }
.layers-body .mask-wrap canvas.doc-marks { pointer-events: none; opacity: 0.9; }
@@ -628,7 +662,9 @@ pre.diag { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font
.doc-grid { display: grid; grid-template-columns: 1fr 1fr; gap: 6px; }
.doc-grid .btn { justify-content: center; height: 34px; min-width: 0; white-space: nowrap; }
.doc-grid > :only-child { grid-column: 1 / -1; }
-.editor-tools .pill.doc-cmd { color: var(--text); font-weight: 500; }
+/* The lab's commands: the tool buttons' corners (.icon-btn, .btn), not a pill. */
+.editor-tools .pill.doc-cmd { color: var(--text); font-weight: 500; height: 34px; padding: 0 12px; border-radius: 11px; }
+.exp-tag { display: inline-flex; align-items: center; margin-left: 6px; padding: 1px 7px; border-radius: 999px; font-size: 10.5px; font-weight: 600; letter-spacing: 0.01em; vertical-align: middle; color: var(--okay); background: var(--okay-bg); }
.editor-tools .pill.doc-cmd.stale { box-shadow: inset 0 0 0 1.5px var(--okay); }
.editor-tools .pill.doc-cmd.primary { background: var(--invert-bg); color: var(--invert-text); }
.editor-tools .pill.doc-cmd.primary:hover:not(:disabled) { opacity: 0.88; background: var(--invert-bg); }
@@ -650,6 +686,16 @@ pre.diag { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font
.expand-area { position: relative; }
.expand-canvas { position: absolute; border: 1.5px dashed var(--text-2); border-radius: 4px; background-image: conic-gradient(var(--checker) 25%, transparent 0 50%, var(--checker) 0 75%, transparent 0); background-size: 16px 16px; }
.expand-src { position: absolute; }
+.expand-src.movable { cursor: grab; touch-action: none; }
+.expand-src.movable:active { cursor: grabbing; }
+/* Place the image: 3x3 dots, the image's spot filled (ExpandEditor.tsx). */
+.anchor-grid { display: grid; grid-template-columns: repeat(3, 12px); grid-template-rows: repeat(3, 12px); gap: 2px; padding: 3px 6px; }
+.anchor-grid button { display: grid; place-items: center; border-radius: 3px; }
+.anchor-grid button span { width: 6px; height: 6px; border-radius: 2px; background: var(--text-3); opacity: 0.5; transition: opacity 0.15s, background 0.15s, transform 0.15s; }
+.anchor-grid button:hover:not(:disabled) span { opacity: 1; background: var(--text); }
+.anchor-grid button.on span { opacity: 1; background: var(--text); transform: scale(1.35); }
+.anchor-grid button:disabled { cursor: default; }
+.anchor-grid button:disabled span { opacity: 0.2; }
.expand-src img { width: 100%; height: 100%; display: block; }
.handle { position: absolute; background: var(--text); border-radius: 3px; z-index: 2; }
.handle.h { width: 6px; height: 40px; cursor: ew-resize; transform: translate(-50%, -50%); }
@@ -870,8 +916,12 @@ pre.diag { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font
.info-prompt > div { flex: 1; min-width: 0; }
.upscale-menu { width: 300px; }
/* Prompt+ menu: one line per choice */
-.popover.prompt-plus { width: 260px; }
+.popover.prompt-plus { width: 280px; }
.popover.prompt-plus .opt > span:not(.check) { white-space: nowrap; }
+.popover.prompt-plus .opt .check { flex: none; }
+/* The menu's own title (Prompt+), above its sections' small labels (Detail). */
+.popover .menu-title { font-size: 13px; font-weight: 600; color: var(--text); padding: 6px 10px 4px; }
+.popover .menu-sep { height: 1px; margin: 6px 4px; background: var(--border); }
.ratio-icon { display: inline-grid; place-items: center; flex: none; }
.ratio-icon i { display: block; border: 1.5px solid currentColor; border-radius: 3px; box-sizing: border-box; }
.aspect-menu { width: 296px; display: flex; flex-direction: column; gap: 6px; }
@@ -1003,6 +1053,7 @@ pre.diag { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font
/* prompt tips (lightbulb in the composer) */
:root { --good: #1a7f37; --good-bg: rgba(26, 127, 55, 0.1); --bad: #c62828; --bad-bg: rgba(198, 40, 40, 0.09); --okay: #9a6700; --okay-bg: rgba(191, 135, 0, 0.12); }
:root[data-theme="dark"] { --good: #7ee2a0; --good-bg: rgba(74, 222, 128, 0.12); --bad: #ff8f8f; --bad-bg: rgba(255, 99, 99, 0.12); --okay: #f5d77a; --okay-bg: rgba(245, 197, 24, 0.12); }
+
button.link { color: inherit; text-decoration: underline; text-underline-offset: 2px; font-size: inherit; padding: 0; }
button.link:hover { color: var(--text); }
.tips { width: min(860px, 94vw); max-height: 90vh; display: flex; flex-direction: column; gap: 12px; background: var(--bg); border-radius: 22px; box-shadow: var(--shadow); padding: 22px 24px 18px; animation: welcome-in 0.25s ease-out; }
@@ -1018,6 +1069,20 @@ button.link:hover { color: var(--text); }
.shortcuts { width: min(760px, 94vw); }
.shortcuts-body { display: grid; grid-template-columns: repeat(auto-fit, minmax(320px, 1fr)); align-content: start; }
.shortcuts-body h2 { margin: 0 0 8px; font-size: 13px; font-weight: 600; }
+/* "Check an image" for AI marking (Settings → General) */
+.ai-check { width: min(560px, 94vw); }
+.ai-check .tips-body > p { margin: 0; font-size: 13px; line-height: 1.45; }
+.ai-check-pick { display: flex; align-items: center; gap: 12px; min-width: 0; }
+.ai-check-name { font-size: 12.5px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
+.ai-check-result { background: var(--surface); border-radius: 16px; padding: 14px 16px; display: flex; flex-direction: column; gap: 10px; }
+.ai-check-result h2 { margin: 0; font-size: 16px; }
+.ai-check-result p { margin: -4px 0 0; font-size: 12.5px; }
+.ai-check-result dl { margin: 0; display: grid; grid-template-columns: max-content 1fr; gap: 6px 14px; font-size: 13px; }
+.ai-check-result dt { color: var(--text-2); }
+.ai-check-result dd { margin: 0; overflow-wrap: anywhere; }
+.ai-check-foot { display: flex; align-items: center; justify-content: space-between; gap: 10px; font-size: 12px; }
+.ai-check-foot .btn { display: inline-flex; align-items: center; gap: 6px; }
+.ai-marking-card { background: transparent; }
.shortcut-row { display: flex; align-items: center; justify-content: space-between; gap: 12px; padding: 5px 0; font-size: 13px; color: var(--text-2); }
.shortcut-row .keys { display: flex; align-items: center; gap: 4px; flex: none; font-size: 12px; }
kbd { min-width: 22px; height: 22px; padding: 0 6px; border-radius: 6px; display: inline-grid; place-items: center; font: inherit; font-size: 11.5px; color: var(--text); background: var(--bg); box-shadow: inset 0 -1px 0 var(--border), 0 0 0 1px var(--border); }