LightCraft: An open-source non-destructive photo editor written by scratch in pure Rust
Fix #366: the SAM 3 dialog without a download location shows how to install by hand
Claude60a98354d977
diff --git a/crates/ui-egui/locales/de.json b/crates/ui-egui/locales/de.jsonindex 84a033c28..ed685f812 100644--- a/crates/ui-egui/locales/de.json+++ b/crates/ui-egui/locales/de.json@@ -649,6 +649,7 @@"Horizontal": "Horizontal","Horizontal text": "Horizontaler Text","Hours": "Stunden",+ "How to install the model": "So installierst du das Modell","Hue": "Farbton","Hue Range": "Farbtonbereich","ISO Speed": "ISO-Empfindlichkeit",@@ -694,6 +695,7 @@"Info overlay": "Infoüberlagerung","Insert at the cursor": "An Cursorposition einfügen","Inspect Photo": "Foto untersuchen",+ "Install the SAM 3 Model": "SAM-3-Modell installieren","Instant ’70s": "Sofortbild der 70er","Interface": "Oberfläche","Interface language": "Sprache der Oberfläche",@@ -1455,7 +1457,7 @@"They move to Recently Deleted and can be restored; no file on disk is touched.": "Sie werden nach „Zuletzt gelöscht“ verschoben und können wiederhergestellt werden; keine Datei auf dem Datenträger wird verändert.","Thirds": "Drittel","This Computer": "Dieser Computer",- "This build has no download location for the model yet: put the files in the folder above yourself (see docs/ai-masks.md).": "Diese Version enthält noch keine Downloadquelle für das Modell: Lege die Dateien selbst im obigen Ordner ab (siehe docs/ai-masks.md).",+ "This build can't download the model yet. To install it by hand, put model.safetensors, vocab.json and merges.txt from Meta's facebook/sam3 in the folder above: Object and Describe work as soon as they are there.": "Diese Version kann das Modell noch nicht herunterladen. Zum Installieren von Hand lege model.safetensors, vocab.json und merges.txt aus Metas facebook/sam3 in den obigen Ordner: Objekt und Beschreiben funktionieren, sobald sie dort liegen.","This folder is not available (drive disconnected?). Smart previews are not built or used until it is back or you choose another folder.": "Dieser Ordner ist nicht verfügbar (Laufwerk getrennt?). Smart-Vorschauen werden erst wieder erstellt oder verwendet, wenn er verfügbar ist oder du einen anderen Ordner auswählst.","This photo can't be opened": "Dieses Foto kann nicht geöffnet werden","Threshold": "Schwellenwert",diff --git a/crates/ui-egui/locales/ja.json b/crates/ui-egui/locales/ja.jsonindex d3e39f27b..cbde1b82c 100644--- a/crates/ui-egui/locales/ja.json+++ b/crates/ui-egui/locales/ja.json@@ -499,6 +499,7 @@"High Contrast B&W": "ハイコントラスト白黒","Home": "ホーム","Hours": "時間",+ "How to install the model": "モデルのインストール方法","ISO Speed": "ISO感度","ISO speed": "ISO感度","Import Point Curve Presets…": "ポイントカーブのプリセットを読み込む…",@@ -512,6 +513,7 @@"Info Overlay: File": "情報オーバーレイ:ファイル","Insert at the cursor": "カーソル位置に挿入","Inspect Photo": "写真を調べる",+ "Install the SAM 3 Model": "SAM 3モデルのインストール","Instant ’70s": "インスタント’70s","Invert Component": "構成要素を反転","JPEG (Large)": "JPEG(大)",@@ -1319,7 +1321,7 @@"The photo has these settings": "この写真に適用されている設定","They can be restored from Recently Deleted until it is emptied.": "「最近削除した写真」を空にするまでは復元できます","Thirds": "三分割",- "This build has no download location for the model yet: put the files in the folder above yourself (see docs/ai-masks.md).": "このビルドにはモデルのダウンロード元がまだありません。上のフォルダにファイルを手動で置いてください(docs/ai-masks.md を参照)",+ "This build can't download the model yet. To install it by hand, put model.safetensors, vocab.json and merges.txt from Meta's facebook/sam3 in the folder above: Object and Describe work as soon as they are there.": "このビルドではまだモデルをダウンロードできません。手動でインストールするには、Meta の facebook/sam3 の model.safetensors、vocab.json、merges.txt を上のフォルダに置いてください。置いた時点で「オブジェクト」と「説明で選択」が使えるようになります。","This folder is not available (drive disconnected?). Smart previews are not built or used until it is back or you choose another folder.": "このフォルダにアクセスできません。ドライブを接続するか別のフォルダを選ぶまで、スマートプレビューは作成・使用されません","This photo can't be opened": "この写真を開けません","Threshold": "しきい値",diff --git a/crates/ui-egui/locales/pt-br.json b/crates/ui-egui/locales/pt-br.jsonindex 6e7045bf5..9ae6bfe30 100644--- a/crates/ui-egui/locales/pt-br.json+++ b/crates/ui-egui/locales/pt-br.json@@ -623,6 +623,7 @@"Horizontal": "Horizontal","Horizontal text": "Texto horizontal","Hours": "Horas",+ "How to install the model": "Como instalar o modelo","Hue": "Matiz","Hue Range": "Faixa de matiz","ISO Speed": "Sensibilidade ISO",@@ -663,6 +664,7 @@"Info overlay": "Sobreposição de informações","Insert at the cursor": "Inserir no cursor","Inspect Photo": "Inspecionar foto",+ "Install the SAM 3 Model": "Instalar o modelo SAM 3","Instant ’70s": "Instantâneo anos 70","Interface": "Interface","Interface language": "Idioma da interface",@@ -1401,7 +1403,7 @@"They move to Recently Deleted and can be restored; no file on disk is touched.": "Elas vão para Excluídos recentemente e podem ser restauradas; nenhum arquivo no disco é alterado.","Thirds": "Terços","This Computer": "Este computador",- "This build has no download location for the model yet: put the files in the folder above yourself (see docs/ai-masks.md).": "Esta versão ainda não tem um local de download para o modelo: coloque os arquivos você mesmo na pasta acima (veja docs/ai-masks.md).",+ "This build can't download the model yet. To install it by hand, put model.safetensors, vocab.json and merges.txt from Meta's facebook/sam3 in the folder above: Object and Describe work as soon as they are there.": "Esta versão ainda não consegue baixar o modelo. Para instalá-lo manualmente, coloque model.safetensors, vocab.json e merges.txt do facebook/sam3 da Meta na pasta acima: Objeto e Descrever funcionam assim que eles estiverem lá.","This folder is not available (drive disconnected?). Smart previews are not built or used until it is back or you choose another folder.": "Esta pasta não está disponível (unidade desconectada?). As pré-visualizações inteligentes não são geradas nem usadas até ela voltar ou você escolher outra pasta.","This photo can't be opened": "Não foi possível abrir esta foto","Threshold": "Limite",diff --git a/crates/ui-egui/locales/ru.json b/crates/ui-egui/locales/ru.jsonindex 6cb076dbb..61d35e483 100644--- a/crates/ui-egui/locales/ru.json+++ b/crates/ui-egui/locales/ru.json@@ -509,6 +509,7 @@"History & Activity (Y)": "История и активность (Y)","Horizontal": "По горизонтали","Horizontal text": "Горизонтальный текст",+ "How to install the model": "Как установить модель","Hue": "Тон","Hue Range": "Диапазон тонов","If file exists": "Если файл существует",@@ -541,6 +542,7 @@"Info (I)": "Информация (I)","Info overlay": "Информационная накладка","Inspect Photo": "Просмотр фото",+ "Install the SAM 3 Model": "Установка модели SAM 3","Instant ’70s": "Мгновенный стиль 70-х","Interface": "Интерфейс","Interface language": "Язык интерфейса",@@ -1151,6 +1153,7 @@"Theme": "Тема","They can be restored from Recently Deleted until it is emptied.": "Их можно восстановить из «Недавно удалённых», пока этот раздел не очищен.","Thirds": "Правило третей",+ "This build can't download the model yet. To install it by hand, put model.safetensors, vocab.json and merges.txt from Meta's facebook/sam3 in the folder above: Object and Describe work as soon as they are there.": "Эта сборка пока не умеет скачивать модель. Чтобы установить её вручную, положите model.safetensors, vocab.json и merges.txt из facebook/sam3 от Meta в папку выше: «Объект» и «Описать» заработают, как только файлы окажутся там.","This folder is not available (drive disconnected?). Smart previews are not built or used until it is back or you choose another folder.": "Эта папка недоступна (диск отключён?). Смарт-просмотры не создаются и не используются, пока она не появится снова или вы не выберете другую папку.","This photo can't be opened": "Это фото не удаётся открыть","Threshold": "Порог",diff --git a/crates/ui-egui/locales/zh-hans.json b/crates/ui-egui/locales/zh-hans.jsonindex c42e2abd3..fe2e86723 100644--- a/crates/ui-egui/locales/zh-hans.json+++ b/crates/ui-egui/locales/zh-hans.json@@ -505,6 +505,7 @@"Has GPS": "有 GPS","Home": "主文件夹","Hours": "小时",+ "How to install the model": "如何安装模型","ISO Speed": "ISO 感光度","ISO speed": "ISO 感光度","Import as new": "作为新照片导入",@@ -513,6 +514,7 @@"Info Overlay: Exposure": "信息叠加:曝光","Info Overlay: File": "信息叠加:文件","Insert at the cursor": "在光标处插入",+ "Install the SAM 3 Model": "安装 SAM 3 模型","Invert Component": "反相组件","Join the ArtCraft community on Discord": "加入 Discord 上的 ArtCraft 社区","Label": "标签",@@ -1388,7 +1390,7 @@"Theme": "主题","They can be restored from Recently Deleted until it is emptied.": "在“最近删除”被清空之前,都可以从那里恢复。","Thirds": "三分法",- "This build has no download location for the model yet: put the files in the folder above yourself (see docs/ai-masks.md).": "此版本尚未提供模型的下载地址:请自行将文件放入上方的文件夹(参见 docs/ai-masks.md)。",+ "This build can't download the model yet. To install it by hand, put model.safetensors, vocab.json and merges.txt from Meta's facebook/sam3 in the folder above: Object and Describe work as soon as they are there.": "此版本尚无法下载模型。如需手动安装,请将 Meta 的 facebook/sam3 中的 model.safetensors、vocab.json 和 merges.txt 放入上方的文件夹:放入后即可使用“对象”和“描述”。","This folder is not available (drive disconnected?). Smart previews are not built or used until it is back or you choose another folder.": "此文件夹不可用(驱动器已断开连接?)。在它恢复或您选择其他文件夹之前,不会构建或使用智能预览。","This photo can't be opened": "无法打开此照片","Threshold": "阈值",diff --git a/crates/ui-egui/locales/zh-hant.json b/crates/ui-egui/locales/zh-hant.jsonindex c097f4f68..4a1050a09 100644--- a/crates/ui-egui/locales/zh-hant.json+++ b/crates/ui-egui/locales/zh-hant.json@@ -505,6 +505,7 @@"Has GPS": "有 GPS","Home": "個人資料夾","Hours": "小時",+ "How to install the model": "如何安裝模型","ISO Speed": "ISO 感光度","ISO speed": "ISO 感光度","Import as new": "以新照片匯入",@@ -513,6 +514,7 @@"Info Overlay: Exposure": "資訊疊加:曝光","Info Overlay: File": "資訊疊加:檔案","Insert at the cursor": "在游標處插入",+ "Install the SAM 3 Model": "安裝 SAM 3 模型","Invert Component": "反轉元件","Join the ArtCraft community on Discord": "加入 Discord 上的 ArtCraft 社群","Label": "標籤",@@ -1388,7 +1390,7 @@"Theme": "主題","They can be restored from Recently Deleted until it is emptied.": "清空「最近刪除」之前,可以從中還原照片。","Thirds": "三分法",- "This build has no download location for the model yet: put the files in the folder above yourself (see docs/ai-masks.md).": "此版本尚未提供模型的下載位置:請自行將檔案放入上方的資料夾(請參閱 docs/ai-masks.md)。",+ "This build can't download the model yet. To install it by hand, put model.safetensors, vocab.json and merges.txt from Meta's facebook/sam3 in the folder above: Object and Describe work as soon as they are there.": "此版本尚無法下載模型。如需手動安裝,請將 Meta 的 facebook/sam3 中的 model.safetensors、vocab.json 和 merges.txt 放入上方的資料夾:放入後即可使用「物件」與「描述」。","This folder is not available (drive disconnected?). Smart previews are not built or used until it is back or you choose another folder.": "此資料夾無法使用(磁碟可能已中斷連線)。恢復連線或選擇其他資料夾之前,不會建立或使用智慧型預覽。","This photo can't be opened": "無法開啟此照片","Threshold": "臨界值",diff --git a/crates/ui-egui/src/panels/dialogs.rs b/crates/ui-egui/src/panels/dialogs.rsindex af4635c32..735cb6c82 100644--- a/crates/ui-egui/src/panels/dialogs.rs+++ b/crates/ui-egui/src/panels/dialogs.rs@@ -99,6 +99,8 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) {Dialog::ConfirmDelete { .. } => "Delete Photos",Dialog::RemoveFolder { disk: true, .. } => "Remove Disk from Library",Dialog::RemoveFolder { .. } => "Remove Folder from Library",+ // (nothing to download from in this build: the dialog explains the manual install)+ Dialog::SamModel { .. } if sam_by_hand(&app.session.segmenter) => "Install the SAM 3 Model",Dialog::SamModel { .. } => "Download the SAM 3 Model?",Dialog::About => "About LightCraft",Dialog::Shortcuts => "Keyboard Shortcuts",@@ -829,7 +831,7 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) {let sam = &app.session.segmenter;let (sam_installed, sam_running, sam_failed) = (sam.installed(), sam.download_status().running, sam.download_status().error.is_some());// no download location in this build: nothing to offer but the manual install- let sam_nowhere = !sam_installed && !sam_running && sam.mirrors().is_empty();+ let sam_nowhere = sam_by_hand(sam);let cancel = match &dlg {Dialog::SamModel { .. } if sam_running || sam_installed || sam_nowhere => "Close",Dialog::SamModel { .. } => "Not Now",@@ -925,6 +927,22 @@ pub fn keeps_open(app: &LightcraftApp, dlg: &Dialog) -> bool {matches!(dlg, Dialog::SamModel { .. }) && !app.session.segmenter.installed()}+/// No SAM 3 model, no download running and nowhere to download it from: installing it by hand is+/// all the dialog can offer.+fn sam_by_hand(sam: &lightcraft_engine::segment::Segmenter) -> bool {+ !sam.installed() && !sam.download_status().running && sam.mirrors().is_empty()+}++/// The model installation guide.+const SAM_HELP: &str = "https://github.com/storytold/lightcraft/blob/main/docs/ai-masks.md#getting-the-model";++/// Show the SAM 3 model folder in the file manager (created first, so there is something to show).+fn show_model_folder(app: &mut LightcraftApp, dir: &std::path::Path) -> Result<(), String> {+ std::fs::create_dir_all(dir).map_err(|e| format!("{}: {e}", dir.display()))?;+ let reveal = app.services.reveal.as_mut().ok_or("not available here")?;+ reveal(&dir.to_string_lossy())+}+/// The SAM 3 dialog: what the model is, its size and licence, and the download's progress.fn sam_model_body(app: &mut LightcraftApp, ui: &mut egui::Ui, error: Option<&str>) {use lightcraft_engine::segment::{LICENSE_NAME, LICENSE_URL, MODEL_BYTES};@@ -971,12 +989,32 @@ fn sam_model_body(app: &mut LightcraftApp, ui: &mut egui::Ui, error: Option<&strreturn;}if seg.mirrors().is_empty() {+ // nothing to download from: say how to install it by hand, with the folder and the guide a click away+ let dir = seg.dir.clone();ui.label(egui::RichText::new(crate::i18n::tr(- "This build has no download location for the model yet: put the files in the folder above yourself (see docs/ai-masks.md).",+ "This build can't download the model yet. To install it by hand, put model.safetensors, vocab.json and merges.txt from Meta's facebook/sam3 in the folder above: Object and Describe work as soon as they are there.",)).color(t.text_dim),);+ ui.horizontal_wrapped(|ui| {+ if let Some(dir) = dir.filter(|_| app.services.reveal.is_some()) {+ let r = ui.button(crate::i18n::tr(crate::menus::reveal_label()));+ crate::widgets::register(ui.ctx(), "button:samFolder", r.rect);+ if r.clicked()+ && let Err(e) = show_model_folder(app, &dir)+ {+ app.toast(ui.ctx(), e);+ }+ }+ let r = ui.link(crate::i18n::tr("How to install the model")).on_hover_text(SAM_HELP);+ crate::widgets::register(ui.ctx(), "link:samHelp", r.rect);+ if r.clicked()+ && let Err(e) = crate::links::open(app, SAM_HELP)+ {+ app.toast(ui.ctx(), e);+ }+ });}if let Some(e) = error.map(str::to_string).or(d.error) {ui.label(egui::RichText::new(format!("{} {e}", crate::i18n::tr("The download didn't work:"))).color(egui::Color32::from_rgb(230, 90, 80)));diff --git a/crates/ui-egui/src/tests_masking.rs b/crates/ui-egui/src/tests_masking.rsindex 8e2c81391..0729a854f 100644--- a/crates/ui-egui/src/tests_masking.rs+++ b/crates/ui-egui/src/tests_masking.rs@@ -649,6 +649,20 @@ fn ai_masks_without_the_model_offer_the_download() {// no location configured in this build: no Download button (only Close), and an agent// confirming anyway gets the reason; the dialog staysh.step();+ // …but a way to install it by hand: the guide, and the model folder (created on demand)+ assert!(h.app.widgets.iter().any(|(id, _)| id == "link:samHelp"), "no installation guide link");+ let shown = std::sync::Arc::new(std::sync::Mutex::new(Vec::<String>::new()));+ let log = shown.clone();+ h.app.services.reveal = Some(Box::new(move |p: &str| {+ log.lock().unwrap().push(p.to_string());+ Ok(())+ }));+ h.step();+ let r = h.request("ui.clickWidget", json!({"id": "button:samFolder"}), T);+ assert_eq!(r["ok"], true, "{r}");+ assert!(dir.is_dir(), "the model folder is created to be shown");+ assert_eq!(*shown.lock().unwrap(), vec![dir.to_string_lossy().to_string()]);+ let _ = std::fs::remove_dir_all(&dir);let r = h.request("ui.clickWidget", json!({"id": "button:dialogOk"}), T);assert_eq!(r["ok"], false, "{r}");let r = h.request("ui.dialog.confirm", json!({}), T);diff --git a/docs/ai-masks.md b/docs/ai-masks.mdindex 5a32d6902..7e760aa1b 100644--- a/docs/ai-masks.md+++ b/docs/ai-masks.md@@ -80,7 +80,9 @@ The model is fetched from an ordered list of mirrors: base URLs where `<base>/mo> **Maintainers:** the built-in list is **empty** until LightCraft's own CDN locations exist> (see the `TODO(maintainer)` there): add them in order of preference, host the three files> unchanged, and pin `vocab.json` / `merges.txt` (size + SHA-256) in `SAM3_FILES` at the same-> time. Until then the in-app download needs a user-configured mirror, and the dialog says so.+> time. Until then the in-app download needs a user-configured mirror, and the dialog says so:+> it names the three files to put in the model folder by hand, with a button that shows that folder+> (created if needed) and a link to this guide.Hugging Face's `facebook/sam3` can't be a default: it is **gated** (each person must accept thelicence there, wait for approval and download with their own token). Someone with access candiff --git a/docs/parity.md b/docs/parity.mdindex 5054c1139..492b09238 100644--- a/docs/parity.md+++ b/docs/parity.md@@ -342,7 +342,7 @@ under the UI command that wraps it (a no-op error); the UI command now wins.| LR-MASK-SUBJECT | Select subject | P2 | 🟡 | `cmd:mask.add` (`subject`), `crates/pipeline/src/masks.rs` | saliency heuristic, no segmentation model || LR-MASK-SKY | Select sky | P2 | 🟡 | `cmd:mask.add` (`sky`) | heuristic || LR-MASK-BACKGROUND | Select background | P2 | 🟡 | `cmd:mask.add` (`background`) | inverse of the subject heuristic |-| LR-MASK-OBJECTS | Object selection | P2 | 🟡 | `cmd:mask.add` (`object`, `prompt`), `cmd:mask.objectPoint`, `cmd:mask.refineDetail`, `cmd:segment.prepare`, `cmd:segment.model.status`, `cmd:segment.model.download`, `cmd:segment.model.cancel`, `crates/segment`, `crates/engine/src/segment/mod.rs` | SAM 3 in pure Rust (candle; Metal on macOS, CPU elsewhere): Object tile → click to include, ⌥-click to leave out; Describe tile → a text prompt selects every instance ("sky", "the red car"); both also as Add/Subtract/Intersect components; + / − under the selected mask; comma lists (`car, road`); a zoomed-in detail pass for 5–10× finer edges on small objects; per-selection Edge (hard ↔ soft). The model runs on its own worker thread (the UI never waits; panics become errors), is unloaded after 10 min idle, and is optional: the segmentation is stored with the mask (288² logits), so renders and exports never need it. Weights are never bundled (SAM License): without them the app offers a consented background download (mirrors, resume, timeouts, SHA-256) — **but no default download location is configured yet** (users set `LIGHTCRAFT_SAM3_MIRRORS` or install by hand; see docs/ai-masks.md). Not verified against Lightroom's Select Object; no brush/box object mode; the first click on a photo waits for its analysis (~4 s on an M4 Pro, much longer on CPU) |+| LR-MASK-OBJECTS | Object selection | P2 | 🟡 | `cmd:mask.add` (`object`, `prompt`), `cmd:mask.objectPoint`, `cmd:mask.refineDetail`, `cmd:segment.prepare`, `cmd:segment.model.status`, `cmd:segment.model.download`, `cmd:segment.model.cancel`, `crates/segment`, `crates/engine/src/segment/mod.rs` | SAM 3 in pure Rust (candle; Metal on macOS, CPU elsewhere): Object tile → click to include, ⌥-click to leave out; Describe tile → a text prompt selects every instance ("sky", "the red car"); both also as Add/Subtract/Intersect components; + / − under the selected mask; comma lists (`car, road`); a zoomed-in detail pass for 5–10× finer edges on small objects; per-selection Edge (hard ↔ soft). The model runs on its own worker thread (the UI never waits; panics become errors), is unloaded after 10 min idle, and is optional: the segmentation is stored with the mask (288² logits), so renders and exports never need it. Weights are never bundled (SAM License): without them the app offers a consented background download (mirrors, resume, timeouts, SHA-256) — **but no default download location is configured yet** (users set `LIGHTCRAFT_SAM3_MIRRORS` or install by hand: the dialog names the three files, shows the model folder and links docs/ai-masks.md). Not verified against Lightroom's Select Object; no brush/box object mode; the first click on a photo waits for its analysis (~4 s on an M4 Pro, much longer on CPU) || LR-MASK-PEOPLE | People parts | P2 | ⬜ | | || LR-MASK-LANDSCAPE | Landscape classes | P2 | ⬜ | | shape exists, evaluates empty || LR-MASK-BRUSH | Brush mask | P0 | ✅ | `cmd:tool.brush`, `cmd:mask.brushStroke` (`autoMask`), `crates/pipeline/src/masks.rs` | size/feather/flow/density/erase; Auto Mask: dabs weighted by similarity to the colour under the dab centre, refined by a guided filter on luminance (CPU + GPU); no A/B brushes, no pressure |