π End-to-end encrypted cloud for everything.
Dart 2faandroidauthye2eeencryptionend-to-end-encryptionfluttergolanggoogle-photosgoogle-photos-alternativeiosopensourcephotosprivacyprivacy-toolstypescriptzero-knowledge
napi codegen without compilation, and clearer separation of script functions
laurenspriem531b4b1b23f9
diff --git a/.github/workflows/desktop-lint.yml b/.github/workflows/desktop-lint.ymlindex 4b0e4e3c78..3dfec2335b 100644--- a/.github/workflows/desktop-lint.yml+++ b/.github/workflows/desktop-lint.yml@@ -43,7 +43,9 @@ jobs:save-if: false# tsc typechecks against the generated rust-bindings/index.d.ts.- - name: Generate ML addon bindings+ # This generates only the declarations (via cargo check); it does+ # not build the addon.+ - name: Generate N-API rust bindingsworking-directory: rustrun: cargo codegen napidiff --git a/desktop/.gitignore b/desktop/.gitignoreindex 98133c7f9b..583286b838 100644--- a/desktop/.gitignore+++ b/desktop/.gitignore@@ -15,11 +15,12 @@ dist/# We download it on demand, if needed for the particular OS/arch.build/vips*-# Downloaded by scripts/ml-native.js for the particular OS/arch.+# Staged by scripts/ort.js for the particular OS/arch.build/onnxruntime/-# Rust ML addons staged for packaging by scripts/ml-native.js.-build/ml-native/+# Rust N-API addons staged for packaging by scripts/napi.js.+build/napi/-# The Rust ML addon and its generated bindings, built by `cargo codegen napi`.+# The Rust ML addon built by scripts/napi.js, and the declarations generated by+# `cargo codegen napi`.rust-bindings/diff --git a/desktop/README.md b/desktop/README.mdindex 8c43ddce39..e6879792d9 100644--- a/desktop/README.md+++ b/desktop/README.md@@ -32,7 +32,7 @@ npm run postinstall> [!NOTE]>-> `npm run postinstall` requires a Rust toolchain and initially runs `cargo codegen napi`. When changing the Rust Photos ML code, rebuild the desktop addon manually:+> `npm run postinstall` fetches binary dependencies but does not compile the Rust ML addon. `npm run dev` builds an optimized addon automatically. When changing the Rust Photos ML code, regenerate the bindings with:>> ```sh> (cd "$(git rev-parse --show-toplevel)/rust" && cargo codegen napi)diff --git a/desktop/docs/dependencies.md b/desktop/docs/dependencies.mdindex 7b20e462d1..16978d921a 100644--- a/desktop/docs/dependencies.md+++ b/desktop/docs/dependencies.md@@ -74,9 +74,9 @@ On macOS, we use the `sips` CLI tool for these tasks, but that is already availa### ML-The ML pipeline (natural language search using CLIP, and face detection and recognition) is implemented by the Rust crate shared with the mobile apps (`rust/crates/photos`), exposed to us as a Node native addon built from source by `cargo codegen napi` (see "[Note: ML with Rust]").+The ML pipeline (natural language search using CLIP, and face detection and recognition) is implemented by the Rust crate shared with the mobile apps (`rust/crates/photos`), exposed to us as a Node native addon built from source by `scripts/napi.js` (see "[Note: ML with Rust]"). `cargo codegen napi` generates only the TypeScript declarations for that addon.-The addon loads [ONNX Runtime](https://onnxruntime.ai) dynamically at runtime, using Ente's pinned custom builds (CoreML-enabled on macOS) that `scripts/ml-native.js` downloads into the extra resources (`build`) folder. See "[Note: ONNX Runtime binaries]".+The addon loads [ONNX Runtime](https://onnxruntime.ai) dynamically at runtime, using Ente's pinned custom builds (CoreML-enabled on macOS) that `scripts/ort.js` installs during postinstall and stages into the extra resources (`build`) folder. See "[Note: ONNX Runtime binaries]".### ZIPdiff --git a/desktop/docs/dev.md b/desktop/docs/dev.mdindex 60cf9c1afc..59176ebaae 100644--- a/desktop/docs/dev.md+++ b/desktop/docs/dev.md@@ -51,9 +51,9 @@ Installs skip dependency lifecycle scripts (`ignore-scripts` in `.npmrc`) so tha- Downloads the `vips` binary used during development (see `scripts/vips.js`).-- Downloads the ONNX Runtime library used by the ML pipeline (see `scripts/ml-native.js`).+- Downloads the ONNX Runtime libraries used by the ML pipeline (see `scripts/ort.js`).-- Builds the Rust ML addon from source by running `cargo codegen napi`, which places the built addon and its generated TypeScript declarations in the gitignored `rust-bindings/` directory. Rerun this codegen (from `../rust`) after changing the Rust ML code.+Postinstall does not compile the Rust ML addon. `npm run dev` builds an optimized host addon through `scripts/napi.js`; packaging builds optimized target addons through electron-builder's `beforeBuild` hook. `cargo codegen napi` only regenerates the TypeScript declarations in the gitignored `rust-bindings/` directory.### lint, lint:fixdiff --git a/desktop/electron-builder.yml b/desktop/electron-builder.ymlindex 45f9904213..9cd676777f 100644--- a/desktop/electron-builder.yml+++ b/desktop/electron-builder.yml@@ -42,7 +42,7 @@ mac:category: public.app-category.photographyhardenedRuntime: truemergeASARs: false- # The Rust ML addon and the ONNX Runtime library are shipped as separate+ # The Rust N-API addon and the ONNX Runtime library are shipped as separate# per-arch Mach-Os (in both slices of the universal build) instead of fat# binaries; the app picks the one matching the process arch at runtime.- x64ArchFiles: "**/{ml-native,onnxruntime}/**"+ x64ArchFiles: "**/{napi,onnxruntime}/**"diff --git a/desktop/package.json b/desktop/package.jsonindex 1dcba5d40e..8d30cc8d26 100644--- a/desktop/package.json+++ b/desktop/package.json@@ -8,14 +8,15 @@"main": "app/main.js","scripts": {"build": "npm run build-renderer && npm run build-main",- "build:ci": "npm run build-renderer:ci && tsc",+ "build:ci": "npm run build-renderer:ci && npm run codegen:napi && tsc","build:quick": "npm run build-renderer && npm run build-main:quick",- "build-main": "tsc && electron-builder",- "build-main:quick": "tsc && electron-builder --dir --config.compression=store --config.mac.identity=null",+ "build-main": "npm run codegen:napi && tsc && electron-builder",+ "build-main:quick": "npm run codegen:napi && tsc && electron-builder --dir --config.compression=store --config.mac.identity=null","build-renderer": "npm run check-web-deps && cross-env-shell _ENTE_IS_DESKTOP=1 \"cd ../web && npm run build:photos && cd ../desktop && shx rm -rf out && shx cp -r ../web/apps/photos/out out\"","build-renderer:ci": "cross-env-shell _ENTE_IS_DESKTOP=1 \"cd ../web && npm ci && npm run build:photos && cd ../desktop && shx rm -rf out && shx cp -r ../web/apps/photos/out out\"","check-web-deps": "node ../web/scripts/check-deps.mjs",- "dev": "node scripts/postinstall.mjs --if-needed && concurrently --kill-others --success first --names 'main,rndr' \"npm run dev-main\" \"npm run dev-renderer\"",+ "codegen:napi": "cross-env-shell \"cd ../rust && cargo codegen napi\"",+ "dev": "node scripts/postinstall.mjs --if-needed && node scripts/napi.js build && concurrently --kill-others --success first --names 'main,rndr' \"npm run dev-main\" \"npm run dev-renderer\"","dev-main": "tsc && electron .","dev-renderer": "npm run check-web-deps && cross-env-shell _ENTE_IS_DESKTOP=1 \"cd ../web && npm exec --workspace photos -- next dev --webpack -p 3008\"","postinstall": "node scripts/postinstall.mjs",diff --git a/desktop/scripts/beforeBuild.js b/desktop/scripts/beforeBuild.jsindex cd00aeb245..a045bdaf6b 100755--- a/desktop/scripts/beforeBuild.js+++ b/desktop/scripts/beforeBuild.js@@ -1,13 +1,14 @@const fsp = require("fs/promises");-const { ensureONNXRuntime, stageMLAddons } = require("./ml-native");+const { stageNapiAddons } = require("./napi");+const { stageONNXRuntime } = require("./ort");/*** This hook is invoked during the initial build (e.g. when triggered by* "npm run build"), and importantly, on each rebuild for a different- * architecture during the build. We use it to ensure the vips binary, the- * ONNX Runtime library, and the Rust ML addon match the current architecture.+ * architecture during the build. We use it to stage the vips binary, the ONNX+ * Runtime library, and the Rust N-API addon for the current architecture.* See "[Note: vips]", "[Note: ONNX Runtime binaries]" and "[Note: Packaging- * the ML addon]" for more details.+ * the N-API addon]" for more details.** The documentation for this hook is at:* https://www.electron.build/app-builder-lib.interface.configuration#beforebuild@@ -33,12 +34,10 @@ const { ensureONNXRuntime, stageMLAddons } = require("./ml-native");module.exports = async (context) => {const { appDir, platform, arch } = context;- // Match the ONNX Runtime library and the staged Rust ML addon to the- // architecture being packaged. Unlike the vips handling below, this also- // has work to do when the build arch is the host arch: the addon must be- // staged from `rust-bindings/` into `build/` in all cases.- await ensureONNXRuntime(platform.nodeName, arch, appDir);- await stageMLAddons(appDir, platform.nodeName, arch);+ // Stage the already-downloaded ONNX Runtime library, then build and stage+ // the release Rust addon for the architecture being packaged.+ await stageONNXRuntime(platform.nodeName, arch, appDir);+ await stageNapiAddons(appDir, platform.nodeName, arch);// The arch used by Electron Builder is not the same as the arch used by// Node's process, but for the two cases that we care about, "x64" anddiff --git a/desktop/scripts/ml-native.js b/desktop/scripts/ml-native.jsdeleted file mode 100644index 9fbfaf2334..0000000000--- a/desktop/scripts/ml-native.js+++ /dev/null@@ -1,285 +0,0 @@-/**- * [Note: Packaging the ML addon]- *- * The Rust ML addon (see [Note: ML with Rust]) is built from source by- * `cargo codegen napi` (which our postinstall runs) into the gitignored- * `rust-bindings/` directory, where the development app loads it from.- *- * When packaging, the addon(s) for the architectures being built are staged- * into `build/ml-native/`, which the "extraFiles" clause in- * `electron-builder.yml` copies into the packaged app's resources. On macOS- * the packaged app is universal, so both the arm64 and x64 addons must be- * present (and identical) in every per-arch build that gets merged; anything- * that is not the host architecture is cross-compiled, so this script installs- * the corresponding Rust target as needed (`rustup target add ...`).- */--/**- * [Note: ONNX Runtime binaries]- *- * The Rust ML addon (see [Note: ML with Rust]) loads the ONNX Runtime- * dynamic library at runtime. We use Ente's pinned custom builds β CoreML- * enabled on macOS, WebGPU-capable elsewhere β published at- * https://github.com/laurens-pilot/ort-packaging/releases- *- * This script downloads the archive for the relevant OS/arch combination,- * verifies it against the SHA-256 checksums pinned below, and extracts it- * into `build/onnxruntime/<arch>`. From there it gets used during- * development, and copied into the packaged app's resources by the- * "extraFiles" clause in `electron-builder.yml`.- *- * On macOS both architectures are downloaded since the packaged app is a- * universal binary. Like `vips.js`, this script runs during "npm install"- * (via postinstall), while `beforeBuild.js` additionally invokes it for the- * specific architecture being packaged (which on CI might differ from the- * architecture we're running on).- */--const crypto = require("node:crypto");-const fsp = require("node:fs/promises");-const path = require("node:path");-const { execFileSync, execSync } = require("node:child_process");--const ortVersion = "1.27.0-r4";--const ortReleaseURL = `https://github.com/laurens-pilot/ort-packaging/releases/download/ort-${ortVersion}`;--/**- * SHA-256 checksums of the release assets, pinned here so that the library we- * package cannot change without a corresponding (reviewed) change to this- * file. When bumping {@link ortVersion}, update these with the values from the- * release's `.sha256` sidecar files after verifying them against the built- * archives.- */-const ortAssetSHA256s = {- "darwin-arm64":- "dbb243c3b43963fda6c9475a978d0954b69490d386bc3ecb54a471b2a13ba043",- "darwin-x64":- "d655686380c1976a9067a48d82645a381e4bc4bdfcc44c88f300befc24c4ae8e",- "linux-arm64":- "0b0367bddd96fb0263781022f715ec531a9856e14d8ed91beea80da8d9bb6a2e",- "linux-x64":- "d417b8d92498f3e754e23fbeac0751f13375061c6c31cc83e869f33a37fca8ad",- "win32-arm64":- "9e3179a985e08700b37f15af7049bca4fc007cb4d5a7e2ff1d81df10e330544e",- "win32-x64":- "0534f35981fe3174379c7e80f93977edd990ce67c40f6cbdde1e6890eb798911",-};--const ortAssetName = (platform, arch) => {- switch (platform) {- case "darwin":- return `onnxruntime-coreml-macos-${arch}-${ortVersion}.tar.gz`;- case "linux":- return `onnxruntime-webgpu-linux-${arch}-${ortVersion}.tar.gz`;- case "win32":- return `onnxruntime-webgpu-windows-${arch}-${ortVersion}.zip`;- default:- throw new Error(`Unsupported platform: ${platform}`);- }-};--const archesForTarget = (platform, arch) =>- platform == "darwin" ? ["arm64", "x64"] : [arch];--/**- * Download and extract the ONNX Runtime library for the given OS/arch into- * `<appDir>/build/onnxruntime/<arch>`, no-oping if it is already there.- */-const downloadONNXRuntimeIfNeeded = async (platform, arch, appDir) => {- const asset = ortAssetName(platform, arch);- const outDir = path.join(appDir, "build", "onnxruntime", arch);- const stampPath = path.join(outDir, ".ente-ort-stamp");-- try {- if ((await fsp.readFile(stampPath, "utf8")) == asset) return;- } catch {}-- const expected = ortAssetSHA256s[`${platform}-${arch}`];- if (!expected) throw new Error(`No pinned SHA-256 for ${platform}-${arch}`);-- console.log(`Downloading ${asset}`);- const res = await fetch(`${ortReleaseURL}/${asset}`);- if (!res.ok)- throw new Error(`Failed to download ${asset}: HTTP ${res.status}`);- const archive = Buffer.from(await res.arrayBuffer());-- const actual = crypto.createHash("sha256").update(archive).digest("hex");- if (actual != expected)- throw new Error(- `SHA-256 mismatch for ${asset}: expected ${expected}, got ${actual}`,- );-- await fsp.rm(outDir, { recursive: true, force: true });- await fsp.mkdir(outDir, { recursive: true });- const archivePath = path.join(outDir, asset);- await fsp.writeFile(archivePath, archive);- // tar is available on all the platforms we build on (bsdtar on macOS and- // Windows, GNU tar on Linux), and handles both .tar.gz and .zip.- execFileSync("tar", ["-xf", asset], { cwd: outDir });- await fsp.rm(archivePath);- await fsp.writeFile(stampPath, asset);-};--/**- * Ensure `<appDir>/build/onnxruntime` contains exactly the ONNX Runtime- * libraries needed for the given OS/arch combination.- *- * On macOS the packaged app is universal, so both architectures are kept. On- * Linux and Windows only the given arch is kept: unlike vips (a single file- * that gets overwritten in place per arch), the libraries live in per-arch- * directories, so directories left over from earlier passes for other- * architectures need to be removed lest they also get copied into the- * package.- */-const ensureONNXRuntime = async (platform, arch, appDir) => {- const arches = archesForTarget(platform, arch);- for (const a of arches)- await downloadONNXRuntimeIfNeeded(platform, a, appDir);-- const outDir = path.join(appDir, "build", "onnxruntime");- for (const entry of await fsp.readdir(outDir))- if (!arches.includes(entry))- await fsp.rm(path.join(outDir, entry), { recursive: true });-};--/** napi-cli's name for the addon built for the given platform-arch. */-const napiTriple = (platform, arch) => {- switch (`${platform}-${arch}`) {- case "darwin-arm64":- return "darwin-arm64";- case "darwin-x64":- return "darwin-x64";- case "linux-x64":- return "linux-x64-gnu";- case "linux-arm64":- return "linux-arm64-gnu";- case "win32-x64":- return "win32-x64-msvc";- case "win32-arm64":- return "win32-arm64-msvc";- default:- throw new Error(`Unsupported platform-arch: ${platform}-${arch}`);- }-};--/** The Rust target triple for the given platform-arch. */-const rustTriple = (platform, arch) => {- switch (`${platform}-${arch}`) {- case "darwin-arm64":- return "aarch64-apple-darwin";- case "darwin-x64":- return "x86_64-apple-darwin";- case "linux-x64":- return "x86_64-unknown-linux-gnu";- case "linux-arm64":- return "aarch64-unknown-linux-gnu";- case "win32-x64":- return "x86_64-pc-windows-msvc";- case "win32-arm64":- return "aarch64-pc-windows-msvc";- default:- throw new Error(`Unsupported platform-arch: ${platform}-${arch}`);- }-};--/** Install the given Rust target for the active toolchain if it is missing. */-const ensureRustTarget = (target, appDir) => {- const installed = execFileSync(- "rustup",- ["target", "list", "--installed"],- { cwd: appDir, encoding: "utf8" },- ).split(/\r?\n/);- if (installed.includes(target)) return;-- console.log(`Installing Rust target ${target}`);- execFileSync("rustup", ["target", "add", target], {- cwd: appDir,- stdio: "inherit",- });-};--/**- * Build the addon for the given platform-arch into `rust-bindings/`.- *- * The build always runs so that edits to the Rust sources cannot be missed- * when packaging; cargo's incremental compilation makes this cheap when- * nothing has changed. Linux always uses napi-cli's cross toolchain, including- * for the host architecture, to keep the addon's glibc requirement at 2.17.- */-const buildAddon = async (appDir, platform, arch) => {- if (arch != process.arch)- ensureRustTarget(rustTriple(platform, arch), appDir);-- const addonPath = path.join(- appDir,- "rust-bindings",- `index.${napiTriple(platform, arch)}.node`,- );-- const cmd = [- "npm exec -- napi build",- "--manifest-path ../rust/bindings/napi/photos/Cargo.toml",- "--target-dir ../rust/target",- "--release --strip --platform --no-js",- "--dts index.d.ts",- "--output-dir rust-bindings",- ...(platform == "linux" || arch != process.arch- ? [- `--target ${rustTriple(platform, arch)}`,- ...(platform == "linux" ? ["--use-napi-cross"] : []),- ]- : []),- ].join(" ");- console.log(`> ${cmd}`);- execSync(cmd, { cwd: appDir, stdio: "inherit" });-- await fsp.access(addonPath);- return addonPath;-};--/**- * Ensure the addon(s) needed when packaging for the given platform-arch are- * built and staged into `build/ml-native/`.- */-const stageMLAddons = async (appDir, platform, arch) => {- // On macOS the packaged app is universal, so both architectures must be- // staged regardless of which arch this build pass is for.- const arches = archesForTarget(platform, arch);-- const stageDir = path.join(appDir, "build", "ml-native");- await fsp.mkdir(stageDir, { recursive: true });-- const wanted = [];- for (const a of arches) {- const addonPath = await buildAddon(appDir, platform, a);- wanted.push(path.basename(addonPath));- await fsp.copyFile(- addonPath,- path.join(stageDir, path.basename(addonPath)),- );- }-- // Remove addons staged by earlier passes for other platforms or- // architectures so that they don't get copied into this package.- for (const entry of await fsp.readdir(stageDir))- if (!wanted.includes(entry))- await fsp.rm(path.join(stageDir, entry), { recursive: true });-};--const main = async () => {- const appDir = path.resolve(__dirname, "..");- await ensureONNXRuntime(process.platform, process.arch, appDir);- execSync("cargo codegen napi", {- cwd: path.resolve(appDir, "..", "rust"),- stdio: "inherit",- });-};--module.exports = { ensureONNXRuntime, napiTriple, stageMLAddons };--if (require.main === module)- main().catch((e) => {- console.error(e);- process.exit(1);- });diff --git a/desktop/scripts/napi.js b/desktop/scripts/napi.jsnew file mode 100644index 0000000000..1f90444a2d--- /dev/null+++ b/desktop/scripts/napi.js@@ -0,0 +1,187 @@+/**+ * [Note: Packaging the N-API addon]+ *+ * This script owns the two napi-rs operations used by desktop:+ *+ * - `node scripts/napi.js dts <typeDefDir> <outPath>` renders declarations+ * emitted by the `cargo check` performed by `cargo codegen napi`.+ * - `node scripts/napi.js build` builds the host addon into the gitignored+ * `rust-bindings/` directory for development.+ *+ * Packaging calls {@link stageNapiAddons} from `beforeBuild.js`. Every addon+ * build goes through {@link buildNapiAddon}, which always uses Cargo's+ * release profile. This script neither downloads nor installs ONNX Runtime;+ * that is owned by `ort.js` and postinstall.+ */++const fsp = require("node:fs/promises");+const path = require("node:path");+const { execFileSync, execSync } = require("node:child_process");++const archesForTarget = (platform, arch) =>+ platform == "darwin" ? ["arm64", "x64"] : [arch];++/** napi-cli's name for the addon built for the given platform-arch. */+const napiTriple = (platform, arch) => {+ switch (`${platform}-${arch}`) {+ case "darwin-arm64":+ return "darwin-arm64";+ case "darwin-x64":+ return "darwin-x64";+ case "linux-x64":+ return "linux-x64-gnu";+ case "linux-arm64":+ return "linux-arm64-gnu";+ case "win32-x64":+ return "win32-x64-msvc";+ case "win32-arm64":+ return "win32-arm64-msvc";+ default:+ throw new Error(`Unsupported platform-arch: ${platform}-${arch}`);+ }+};++/** The Rust target triple for the given platform-arch. */+const rustTriple = (platform, arch) => {+ switch (`${platform}-${arch}`) {+ case "darwin-arm64":+ return "aarch64-apple-darwin";+ case "darwin-x64":+ return "x86_64-apple-darwin";+ case "linux-x64":+ return "x86_64-unknown-linux-gnu";+ case "linux-arm64":+ return "aarch64-unknown-linux-gnu";+ case "win32-x64":+ return "x86_64-pc-windows-msvc";+ case "win32-arm64":+ return "aarch64-pc-windows-msvc";+ default:+ throw new Error(`Unsupported platform-arch: ${platform}-${arch}`);+ }+};++/** Install the given Rust target for the active toolchain if it is missing. */+const ensureRustTarget = (target, appDir) => {+ const installed = execFileSync(+ "rustup",+ ["target", "list", "--installed"],+ { cwd: appDir, encoding: "utf8" },+ ).split(/\r?\n/);+ if (installed.includes(target)) return;++ console.log(`Installing Rust target ${target}`);+ execFileSync("rustup", ["target", "add", target], {+ cwd: appDir,+ stdio: "inherit",+ });+};++/**+ * Build the addon for the given platform-arch into `rust-bindings/`.+ *+ * The build always runs so that edits to the Rust sources cannot be missed+ * when packaging; Cargo's incremental compilation makes this cheap when+ * nothing has changed. Linux always uses napi-cli's cross toolchain, including+ * for the host architecture, to keep the addon's glibc requirement at 2.17.+ */+const buildNapiAddon = async (appDir, platform, arch) => {+ if (arch != process.arch)+ ensureRustTarget(rustTriple(platform, arch), appDir);++ const addonPath = path.join(+ appDir,+ "rust-bindings",+ `index.${napiTriple(platform, arch)}.node`,+ );++ const cmd = [+ "npm exec -- napi build",+ "--manifest-path ../rust/bindings/napi/photos/Cargo.toml",+ "--target-dir ../rust/target",+ "--release --strip --platform --no-js",+ "--dts index.d.ts",+ "--output-dir rust-bindings",+ ...(platform == "linux" || arch != process.arch+ ? [+ `--target ${rustTriple(platform, arch)}`,+ ...(platform == "linux" ? ["--use-napi-cross"] : []),+ ]+ : []),+ ].join(" ");+ console.log(`> ${cmd}`);+ execSync(cmd, { cwd: appDir, stdio: "inherit" });++ await fsp.access(addonPath);+ return addonPath;+};++/**+ * Ensure the release addon(s) for a package target are built and staged into+ * `build/napi/`.+ */+const stageNapiAddons = async (appDir, platform, arch) => {+ // On macOS the packaged app is universal, so both architectures must be+ // staged regardless of which arch this build pass is for.+ const arches = archesForTarget(platform, arch);++ const stageDir = path.join(appDir, "build", "napi");+ await fsp.mkdir(stageDir, { recursive: true });++ const wanted = [];+ for (const a of arches) {+ const addonPath = await buildNapiAddon(appDir, platform, a);+ wanted.push(path.basename(addonPath));+ await fsp.copyFile(+ addonPath,+ path.join(stageDir, path.basename(addonPath)),+ );+ }++ // Remove addons staged by earlier passes for other platforms or+ // architectures so that they don't get copied into this package.+ for (const entry of await fsp.readdir(stageDir))+ if (!wanted.includes(entry))+ await fsp.rm(path.join(stageDir, entry), { recursive: true });+};++/** Render the napi macro output into the declaration file used by TypeScript. */+const renderTypeDefinitions = async (typeDefDir, outPath) => {+ const { generateTypeDef } = require("@napi-rs/cli");+ const { dts } = await generateTypeDef({ typeDefDir, cwd: process.cwd() });+ await fsp.writeFile(outPath, dts);+};++const main = async () => {+ const [command, ...args] = process.argv.slice(2);+ const appDir = path.resolve(__dirname, "..");++ switch (command) {+ case "build":+ if (args.length)+ throw new Error("usage: node scripts/napi.js build");+ await buildNapiAddon(appDir, process.platform, process.arch);+ return;+ case "dts": {+ const [typeDefDir, outPath] = args;+ if (!typeDefDir || !outPath || args.length != 2)+ throw new Error(+ "usage: node scripts/napi.js dts <typeDefDir> <outPath>",+ );+ await renderTypeDefinitions(typeDefDir, outPath);+ return;+ }+ default:+ throw new Error(+ "usage: node scripts/napi.js <build|dts <typeDefDir> <outPath>>",+ );+ }+};++module.exports = { napiTriple, stageNapiAddons };++if (require.main === module)+ main().catch((e) => {+ console.error(e);+ process.exit(1);+ });diff --git a/desktop/scripts/ort.js b/desktop/scripts/ort.jsnew file mode 100644index 0000000000..791120ca2c--- /dev/null+++ b/desktop/scripts/ort.js@@ -0,0 +1,162 @@+/**+ * [Note: ONNX Runtime binaries]+ *+ * The Rust ML addon (see [Note: ML with Rust]) loads ONNX Runtime dynamically+ * at runtime. We use Ente's pinned custom builds β CoreML-enabled on macOS and+ * WebGPU-capable elsewhere β published at+ * https://github.com/laurens-pilot/ort-packaging/releases+ *+ * This script is the only place that downloads those builds. Postinstall runs+ * it once to acquire both architectures for the host platform. The extracted+ * libraries are kept under `node_modules/.cache/ente-onnxruntime/`, separate+ * from compilation and packaging. It also stages the host libraries into+ * `build/onnxruntime/` for development.+ *+ * During packaging, `beforeBuild.js` calls {@link stageONNXRuntime} to copy the+ * already-installed libraries needed by the current target into+ * `build/onnxruntime/`. That step performs no network access.+ */++const crypto = require("node:crypto");+const fsp = require("node:fs/promises");+const path = require("node:path");+const { execFileSync } = require("node:child_process");++const ortVersion = "1.27.0-r4";++const ortReleaseURL = `https://github.com/laurens-pilot/ort-packaging/releases/download/ort-${ortVersion}`;++/**+ * SHA-256 checksums of the release assets, pinned here so that the library we+ * package cannot change without a corresponding (reviewed) change to this+ * file. When bumping {@link ortVersion}, update these with the values from the+ * release's `.sha256` sidecar files after verifying them against the built+ * archives.+ */+const ortAssetSHA256s = {+ "darwin-arm64":+ "dbb243c3b43963fda6c9475a978d0954b69490d386bc3ecb54a471b2a13ba043",+ "darwin-x64":+ "d655686380c1976a9067a48d82645a381e4bc4bdfcc44c88f300befc24c4ae8e",+ "linux-arm64":+ "0b0367bddd96fb0263781022f715ec531a9856e14d8ed91beea80da8d9bb6a2e",+ "linux-x64":+ "d417b8d92498f3e754e23fbeac0751f13375061c6c31cc83e869f33a37fca8ad",+ "win32-arm64":+ "9e3179a985e08700b37f15af7049bca4fc007cb4d5a7e2ff1d81df10e330544e",+ "win32-x64":+ "0534f35981fe3174379c7e80f93977edd990ce67c40f6cbdde1e6890eb798911",+};++const ortAssetName = (platform, arch) => {+ switch (platform) {+ case "darwin":+ return `onnxruntime-coreml-macos-${arch}-${ortVersion}.tar.gz`;+ case "linux":+ return `onnxruntime-webgpu-linux-${arch}-${ortVersion}.tar.gz`;+ case "win32":+ return `onnxruntime-webgpu-windows-${arch}-${ortVersion}.zip`;+ default:+ throw new Error(`Unsupported platform: ${platform}`);+ }+};++const archesForTarget = (platform, arch) =>+ platform == "darwin" ? ["arm64", "x64"] : [arch];++const installDir = (appDir) =>+ path.join(appDir, "node_modules", ".cache", "ente-onnxruntime");++/** Download and extract one pinned ONNX Runtime build, if it is not current. */+const downloadONNXRuntimeIfNeeded = async (platform, arch, appDir) => {+ const asset = ortAssetName(platform, arch);+ const outDir = path.join(installDir(appDir), arch);+ const stampPath = path.join(outDir, ".ente-ort-stamp");++ try {+ if ((await fsp.readFile(stampPath, "utf8")) == asset) return;+ } catch {}++ const expected = ortAssetSHA256s[`${platform}-${arch}`];+ if (!expected) throw new Error(`No pinned SHA-256 for ${platform}-${arch}`);++ console.log(`Downloading ${asset}`);+ const res = await fetch(`${ortReleaseURL}/${asset}`);+ if (!res.ok)+ throw new Error(`Failed to download ${asset}: HTTP ${res.status}`);+ const archive = Buffer.from(await res.arrayBuffer());++ const actual = crypto.createHash("sha256").update(archive).digest("hex");+ if (actual != expected)+ throw new Error(+ `SHA-256 mismatch for ${asset}: expected ${expected}, got ${actual}`,+ );++ await fsp.rm(outDir, { recursive: true, force: true });+ await fsp.mkdir(outDir, { recursive: true });+ const archivePath = path.join(outDir, asset);+ await fsp.writeFile(archivePath, archive);+ // tar is available on all the platforms we build on (bsdtar on macOS and+ // Windows, GNU tar on Linux), and handles both .tar.gz and .zip.+ execFileSync("tar", ["-xf", asset], { cwd: outDir });+ await fsp.rm(archivePath);+ await fsp.writeFile(stampPath, asset);+};++/**+ * Copy the already-installed ONNX Runtime libraries required by a build target+ * into the resources staging directory. This deliberately never downloads.+ */+const stageONNXRuntime = async (platform, arch, appDir) => {+ const arches = archesForTarget(platform, arch);+ const sourceRoot = installDir(appDir);+ const stageRoot = path.join(appDir, "build", "onnxruntime");++ for (const a of arches) {+ const expected = ortAssetName(platform, a);+ const stampPath = path.join(sourceRoot, a, ".ente-ort-stamp");+ let actual;+ try {+ actual = await fsp.readFile(stampPath, "utf8");+ } catch {+ throw new Error(+ `ONNX Runtime for ${platform}-${a} is not installed (run "npm run postinstall" first)`,+ );+ }+ if (actual != expected)+ throw new Error(+ `Installed ONNX Runtime for ${platform}-${a} is stale (run "npm run postinstall" again)`,+ );+ }++ await fsp.rm(stageRoot, { recursive: true, force: true });+ await fsp.mkdir(stageRoot, { recursive: true });+ for (const a of arches)+ await fsp.cp(path.join(sourceRoot, a), path.join(stageRoot, a), {+ recursive: true,+ verbatimSymlinks: true,+ });+};++/** Acquire every architecture needed by desktop packaging on this platform. */+const installONNXRuntime = async (platform, appDir) => {+ for (const arch of ["arm64", "x64"])+ await downloadONNXRuntimeIfNeeded(platform, arch, appDir);++ // Make the host runtime available to the development app. macOS needs+ // both libraries because its packaged application is universal.+ await stageONNXRuntime(platform, process.arch, appDir);+};++const main = () => {+ const appDir = path.resolve(__dirname, "..");+ return installONNXRuntime(process.platform, appDir);+};++module.exports = { stageONNXRuntime };++if (require.main === module)+ main().catch((e) => {+ console.error(e);+ process.exit(1);+ });diff --git a/desktop/scripts/postinstall.mjs b/desktop/scripts/postinstall.mjsindex a93b81a920..3fea964cb7 100644--- a/desktop/scripts/postinstall.mjs+++ b/desktop/scripts/postinstall.mjs@@ -40,6 +40,6 @@ const run = (cmd) => {run("npm rebuild --ignore-scripts=false ffmpeg-static electron-winstaller");run("npm exec -- electron-builder install-app-deps");run("node scripts/vips.js");-run("node scripts/ml-native.js");+run("node scripts/ort.js");writeFileSync(stamp, treeHash());diff --git a/desktop/src/main/services/ml-native.ts b/desktop/src/main/services/ml-native.tsindex 9ad41dff88..4856bbb5a6 100644--- a/desktop/src/main/services/ml-native.ts+++ b/desktop/src/main/services/ml-native.ts@@ -11,13 +11,14 @@* (rust/crates/photos). The desktop app talks to it through a Node native* addon (rust/bindings/napi/photos), which loads ONNX Runtime dynamically at* runtime from Ente's pinned custom builds (see [Note: ONNX Runtime binaries]- * in scripts/ml-native.js).+ * in scripts/ort.js).*- * During development, `cargo codegen napi` (invoked by our postinstall)- * builds the addon into the gitignored `rust-bindings/` directory. When- * packaging, `beforeBuild.js` stages the addon for the architectures being- * built into `build/ml-native/`, which electron-builder copies (along with- * `build/onnxruntime/`) into the app's resources directory.+ * During development, `scripts/napi.js` (invoked by `npm run dev`) builds the+ * addon into the gitignored `rust-bindings/` directory. When packaging,+ * `beforeBuild.js` builds and stages the addon for the target architectures+ * into `build/napi/`, which electron-builder copies (along with the ONNX+ * Runtime libraries staged in `build/onnxruntime/`) into the app's resources+ * directory.*/import path from "node:path";@@ -75,7 +76,7 @@ export const mlNativePaths = (): MLNativePaths => {const addonName = `index.${napiTriple()}.node`;const addon = isDev? path.resolve("rust-bindings", addonName)- : path.join(process.resourcesPath, "ml-native", addonName);+ : path.join(process.resourcesPath, "napi", addonName);const onnxRuntimeLibrary = path.join(isDev ? path.resolve("build") : process.resourcesPath,"onnxruntime",diff --git a/infra/ml/test/README.md b/infra/ml/test/README.mdindex c517d1baae..b4f96b44dc 100644--- a/infra/ml/test/README.md+++ b/infra/ml/test/README.md@@ -13,7 +13,7 @@ This directory contains the ML indexing parity framework for desktop, Android, i## Prerequisites- Python + `uv`-- Node + the desktop build artifacts (`npm ci && npm run postinstall` in `desktop/`) when running desktop parity+- Node + the desktop build artifacts (`npm ci && npm run postinstall && node scripts/napi.js build` in `desktop/`) when running desktop parity- Flutter SDK (Android/iOS parity)- Android emulator/device and iOS simulator/device when running mobile paritydiff --git a/infra/ml/test/run_ml_parity_tests.sh b/infra/ml/test/run_ml_parity_tests.shindex 5951ddd4f3..d7261e7062 100755--- a/infra/ml/test/run_ml_parity_tests.sh+++ b/infra/ml/test/run_ml_parity_tests.sh@@ -577,10 +577,10 @@ run_preflight_checks() {preflight_errors+=("node is required for desktop parity")fiif ! compgen -G "$ROOT_DIR/desktop/rust-bindings/index.*.node" >/dev/null; then- preflight_errors+=("desktop ML addon not found under desktop/rust-bindings (run 'npm ci && npm run postinstall' in desktop/, or 'cargo codegen napi' from rust/)")+ preflight_errors+=("desktop ML addon not found under desktop/rust-bindings (run 'npm ci && npm run postinstall && node scripts/napi.js build' in desktop/)")fiif [[ ! -d "$ROOT_DIR/desktop/build/onnxruntime" ]]; then- preflight_errors+=("ONNX Runtime library not found under desktop/build/onnxruntime (run 'node scripts/ml-native.js' in desktop/)")+ preflight_errors+=("ONNX Runtime library not found under desktop/build/onnxruntime (run 'npm run postinstall' in desktop/)")fi;;android|ios)diff --git a/infra/ml/test/tools/desktop_parity_runner.js b/infra/ml/test/tools/desktop_parity_runner.jsindex 0b10fc8d5e..06020e0b89 100755--- a/infra/ml/test/tools/desktop_parity_runner.js+++ b/infra/ml/test/tools/desktop_parity_runner.js@@ -13,10 +13,10 @@* the shared Rust pipeline itself lives in the ente-photos crate's* ml_indexing integration test.*- * The desktop build artifacts it needs are produced by the desktop- * postinstall: the addon in `desktop/rust-bindings/` (via `cargo codegen- * napi`) and the ONNX Runtime library in `desktop/build/onnxruntime/` (via- * `node scripts/ml-native.js`).+ * The desktop build artifacts it needs are the addon in+ * `desktop/rust-bindings/` (built by `node scripts/napi.js build`) and the+ * ONNX Runtime library in `desktop/build/onnxruntime/` (downloaded by the+ * desktop postinstall).*/const fsp = require("node:fs/promises");@@ -28,7 +28,7 @@ const desktopDir = path.join(repoRoot, "desktop");// The desktop build scripts own the addon naming scheme.const { napiTriple } = require(- path.join(desktopDir, "scripts", "ml-native.js"),+ path.join(desktopDir, "scripts", "napi.js"),);const requiredFlags = [@@ -51,7 +51,7 @@ const parseCLIArgs = () => {return values;};-/** Load the Rust ML addon built by `cargo codegen napi`. */+/** Load the Rust ML addon built by desktop's `scripts/napi.js`. */const loadMLAddon = () => {const addonPath = path.join(desktopDir,@@ -62,7 +62,7 @@ const loadMLAddon = () => {};/**- * The ONNX Runtime dynamic library downloaded by desktop's `scripts/ml-native.js`+ * The ONNX Runtime dynamic library downloaded by desktop's `scripts/ort.js`* for the current architecture, located by scanning so that this script does* not duplicate the pinned library version.*/@@ -82,7 +82,7 @@ const onnxRuntimeLibraryPath = async () => {);if (!library)throw new Error(- `No ONNX Runtime library found in ${libraryDir} (run "node scripts/ml-native.js" in desktop/)`,+ `No ONNX Runtime library found in ${libraryDir} (run "npm run postinstall" in desktop/)`,);return path.join(libraryDir, library.name);};diff --git a/rust/README.md b/rust/README.mdindex 1fbaf4ce03..6a0e8344da 100644--- a/rust/README.md+++ b/rust/README.md@@ -20,7 +20,7 @@ Other useful commands:```shcargo codegen native # Regenerate bindings used by native appscargo codegen frb # Regenerate bindings used by Flutter apps-cargo codegen napi # Rebuild the desktop (Electron) ML addon and its bindings+cargo codegen napi # Regenerate bindings used by Desktop (Electron) apps# Run integration tests against a local Museumcargo test --features museum,ente-photos/ml-assetsdiff --git a/rust/apps/codegen/src/main.rs b/rust/apps/codegen/src/main.rsindex c811d7c706..095fdef922 100644--- a/rust/apps/codegen/src/main.rs+++ b/rust/apps/codegen/src/main.rs@@ -4,6 +4,7 @@ use std::ffi::OsStr;use std::fs;use std::path::{Path, PathBuf};use std::process::Command;+use std::time::SystemTime;use camino::Utf8PathBuf;use lib_flutter_rust_bridge_codegen::codegen::{@@ -207,44 +208,48 @@ fn generate_frb_package(package_dir: &Path) -> Result<(), DynError> {Ok(())}+/// Regenerates `desktop/rust-bindings/index.d.ts` without building or linking+/// the addon: `cargo check` expands the `#[napi]` macros, which emit type+/// definitions when `NAPI_TYPE_DEF_TMP_FOLDER` is set, and napi-cli's+/// `generateTypeDef` API renders those into the declaration file. The addon+/// itself is built by the desktop build scripts (see [Note: Packaging the+/// N-API addon] in `desktop/scripts/napi.js`).fn generate_napi() -> Result<(), DynError> {let rust_root = rust_root()?;let desktop_dir = repo_root()?.join("desktop");let out_dir = desktop_dir.join("rust-bindings");write_generated_gitignore(&out_dir)?;- let npm = if cfg!(windows) { "npm.cmd" } else { "npm" };- let linux_cross_args = if cfg!(target_os = "linux") {- let target = match env::consts::ARCH {- "x86_64" => "x86_64-unknown-linux-gnu",- "aarch64" => "aarch64-unknown-linux-gnu",- arch => return Err(format!("unsupported Linux architecture for NAPI: {arch}").into()),- };- vec!["--target", target, "--use-napi-cross"]- } else {- Vec::new()- };+ // The macros only run when the crate is recompiled, so force that.+ fs::File::options()+ .write(true)+ .open(rust_root.join("bindings/napi/photos/src/lib.rs"))?+ .set_modified(SystemTime::now())?;++ let type_def_dir = target_dir()?.join("napi-type-defs");+ fs::create_dir_all(&type_def_dir)?;run_command(- Command::new(npm)- .arg("exec")- .arg("--")- .arg("napi")- .arg("build")- .arg("--manifest-path")- .arg(rust_root.join("bindings/napi/photos/Cargo.toml"))+ Command::new("cargo")+ .arg("check")+ .arg("--locked")+ .arg("-p")+ .arg("ente_photos_napi").arg("--target-dir").arg(target_dir()?)- .arg("--release")- .arg("--strip")- .arg("--platform")- .arg("--no-js")- .arg("--dts")- .arg("index.d.ts")- .arg("--output-dir")- .arg(&out_dir)- .args(linux_cross_args)+ .env("NAPI_TYPE_DEF_TMP_FOLDER", &type_def_dir)+ .current_dir(&rust_root),+ "failed to type-check the desktop Node bindings".to_owned(),+ )?;++ run_command(+ Command::new("node")+ .arg("scripts/napi.js")+ .arg("dts")+ .arg(&type_def_dir)+ .arg(out_dir.join("index.d.ts")).current_dir(&desktop_dir),- "failed to build the desktop Node addon (run npm install in desktop/ first)".to_owned(),+ "failed to render the TypeScript declarations (run npm install in desktop/ first)"+ .to_owned(),)}