YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
Rust agentsagiaiai-memoryaudio-recordingcomputer-visionhermeshermes-agentllmlocal-ailocal-firstmachine-learningmcpmultimodalopenclawprivacyrewindscreen-recordingspeech-to-textycombinator
fix(dev): queue native app tests (#6554)
Ezra Ellette9af05ad36f1e
diff --git a/AGENTS.md b/AGENTS.mdindex 41a1e803c..6b848b9f8 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -39,16 +39,21 @@ Scope test runs; the workspace is ~490k lines. `cargo test -p <crate>`, or`cd apps/screenpipe-app-tauri && bun run test`.`src-tauri` is excluded from the workspace and has no CI test job, so root-`cargo test` never compiles it. Test it explicitly with `--manifest-path`, after-`bun scripts/pre_build.js` (its `build.rs` panics without the sidecars). That-build also rewrites tracked `src-tauri/gen/schemas/`; `git checkout --` it.+`cargo test` never compiles it. From `apps/screenpipe-app-tauri`, test it with+`bun run test:tauri <cargo-test-args>`. This command runs `pre_build.js`, uses+the `debug-dev` profile, and holds the machine-wide native build queue for the+entire test. It can rewrite tracked `src-tauri/gen/schemas/`; restore only that+generated noise afterward.For native app development, use only the scripts in-`apps/screenpipe-app-tauri`: `bun run dev:tauri` for the normal live loop and-`bun run build:tauri:dev` for a one-shot test binary. Both select the-`debug-dev` Cargo profile through Tauri and use the machine-wide native build-queue/cache automatically. Do not bypass them with raw Tauri/Cargo commands,-`cargo clean`, target-directory overrides, or ad hoc profile/cache settings.+`apps/screenpipe-app-tauri`: `bun run dev:tauri` for the normal live loop,+`bun run build:tauri:dev` for a one-shot test binary, and `bun run test:tauri`+for native tests. They select the `debug-dev` Cargo profile and use the+machine-wide native build queue/cache automatically. Never run raw+Tauri/Cargo commands for `src-tauri`, even for one focused test. If the queue+or sccache is unavailable, stop and report the native check as blocked; never+accept or continue a local-compilation fallback. Do not use `cargo clean`,+target-directory overrides, or ad hoc profile/cache settings.See `docs/macos-dev-builds.md` for the exact commands and for the separatesigned `.app` path used only when persistent macOS TCC identity is required.diff --git a/apps/screenpipe-app-tauri/package.json b/apps/screenpipe-app-tauri/package.jsonindex cd25acf02..d6fe9a89e 100644--- a/apps/screenpipe-app-tauri/package.json+++ b/apps/screenpipe-app-tauri/package.json@@ -59,8 +59,8 @@"test:e2e:recording-health-return-race": "SCREENPIPE_E2E_SEED=onboarding,no-recording,recording-health-return-race SCREENPIPE_PORT=3044 bun run wdio run e2e/wdio.conf.ts --spec e2e/specs/recording-health-return-race.spec.ts","test:e2e:search-bugs": "SCREENPIPE_E2E_SEED=onboarding,no-recording,search-fixture SCREENPIPE_E2E_TIMELINE_FRAME_LIMIT=2 SCREENPIPE_PORT=3041 bun run wdio run e2e/wdio.conf.ts --spec e2e/specs/search/search-bugs-4645.spec.ts","typecheck": "bun x tsc --noEmit",- "bindings:check": "cargo test -p screenpipe-app tauri_bindings_are_current --manifest-path src-tauri/Cargo.toml -- --nocapture",- "bindings:generate": "UPDATE_TAURI_BINDINGS=1 cargo test -p screenpipe-app export_typescript_bindings --manifest-path src-tauri/Cargo.toml -- --nocapture",+ "bindings:check": "bun scripts/native-build-queue.ts test tauri_bindings_are_current -- --nocapture",+ "bindings:generate": "UPDATE_TAURI_BINDINGS=1 bun scripts/native-build-queue.ts test export_typescript_bindings -- --nocapture","skills:generate": "bun scripts/gen-skill-content.js","build": "bun scripts/build-frontend.js","predev": "bun scripts/gen-skill-content.js && bun scripts/pre_build.js",@@ -69,6 +69,7 @@"build:tauri:dev": "bun scripts/native-build-queue.ts build","build:tauri:e2e": "bun scripts/native-build-queue.ts e2e","build:tauri:status": "bun scripts/native-build-queue.ts status",+ "test:tauri": "bun scripts/native-build-queue.ts test","tauri:build": "bun scripts/native-build-queue.ts signed","clean": "rm -rf out && cargo clean --manifest-path src-tauri/Cargo.toml && rm -rf src-tauri/{screenpipe-*,tesseract-*,ffmpeg*,openblas*,bun*,ollama*,ui_monitor*} node_modules"},diff --git a/apps/screenpipe-app-tauri/scripts/native-build-queue.test.ts b/apps/screenpipe-app-tauri/scripts/native-build-queue.test.tsindex 9756bb1a6..bc9229872 100644--- a/apps/screenpipe-app-tauri/scripts/native-build-queue.test.ts+++ b/apps/screenpipe-app-tauri/scripts/native-build-queue.test.ts@@ -5,6 +5,7 @@import { describe, expect, test } from "bun:test";import {formatDuration,+ nativeTestCommand,parseWorktreeList,sccacheHasBaseDirectories,} from "./native-build-queue";@@ -37,4 +38,13 @@ describe("native build queue helpers", () => {expect(sccacheHasBaseDirectories(stats, ["/code/a", "/code/c"])).toBe(false);expect(sccacheHasBaseDirectories("Base directories (none)\n", ["/code/a"])).toBe(false);});++ test("routes native tests through the debug-dev app manifest", () => {+ expect(nativeTestCommand(["--features", "e2e", "staged_update::"])).toEqual([+ "cargo", "test", "-p", "screenpipe-app",+ "--manifest-path", "src-tauri/Cargo.toml",+ "--profile", "debug-dev",+ "--features", "e2e", "staged_update::",+ ]);+ });});diff --git a/apps/screenpipe-app-tauri/scripts/native-build-queue.ts b/apps/screenpipe-app-tauri/scripts/native-build-queue.tsindex 2a2d7b92e..8865dbb21 100644--- a/apps/screenpipe-app-tauri/scripts/native-build-queue.ts+++ b/apps/screenpipe-app-tauri/scripts/native-build-queue.ts@@ -28,7 +28,7 @@ const SCCACHE_DEFAULT_PORT = "4226";const LEGACY_SCCACHE_PORT = "4227";const WAIT_UPDATE_MS = 10_000;-type BuildMode = "build" | "e2e" | "signed" | "warmup" | "test-hold";+type BuildMode = "build" | "e2e" | "signed" | "test" | "warmup" | "test-hold";type QueueOwner = {requestId: string;@@ -61,6 +61,15 @@ export function sccacheHasBaseDirectories(output: string, worktrees: string[]):return worktrees.every((worktree) => line.includes(`${worktree}/`));}+export function nativeTestCommand(args: string[]): string[] {+ return [+ "cargo", "test", "-p", "screenpipe-app",+ "--manifest-path", "src-tauri/Cargo.toml",+ "--profile", "debug-dev",+ ...args,+ ];+}+function decode(value: Uint8Array): string {return new TextDecoder().decode(value).trim();}@@ -110,7 +119,12 @@ function localSccacheEnvironment(): Record<string, string> {const env = { ...process.env } as Record<string, string>;const sccache = findExecutable("sccache");if (!sccache) {- console.warn("[native-build-queue] sccache not found; continuing without the shared compile cache");+ if (process.platform === "darwin") {+ throw new Error(+ "[native-build-queue] machine-wide sccache is required on macOS; refusing local compilation",+ );+ }+ console.warn("[native-build-queue] sccache not found; continuing without a compile cache");env.RUSTC_WRAPPER = "";return env;}@@ -185,14 +199,17 @@ function localSccacheEnvironment(): Record<string, string> {|| verify.exitCode !== 0|| !sccacheHasBaseDirectories(decode(verify.stdout), worktrees)) {- console.warn(- "[native-build-queue] machine-wide sccache did not start with all worktree bases; compile-cache reuse may be reduced",- );+ const message =+ "[native-build-queue] machine-wide sccache did not start with all worktree bases";+ if (process.platform === "darwin") {+ throw new Error(`${message}; refusing local compilation`);+ }+ console.warn(`${message}; compile-cache reuse may be reduced`);}}// Starting through the configured wrapper preserves the machine-wide- // backend, cache size, and fallback policy.+ // backend and cache size.return env;}@@ -230,6 +247,7 @@ function modeLabel(mode: BuildMode): string {case "build": return "one-shot debug-dev build";case "e2e": return "debug-dev E2E build";case "signed": return "signed debug-dev app build";+ case "test": return "queued native app tests";case "warmup": return "Tauri dev compile warm-up";case "test-hold": return "queue self-test";}@@ -253,6 +271,11 @@ async function perform(mode: BuildMode, args: string[]): Promise<number> {], env);case "signed":return run(["bash", "scripts/build_macos.sh", "--queue-held"], env);+ case "test": {+ const exitCode = await run(["bun", "scripts/pre_build.js"], env);+ if (exitCode !== 0) return exitCode;+ return run(nativeTestCommand(args), env);+ }case "warmup": {let exitCode = await run(["bun", "scripts/pre_build.js"], env);if (exitCode !== 0) return exitCode;@@ -363,11 +386,11 @@ async function main(): Promise<number> {env.SCREENPIPE_NATIVE_PREBUILD_COMPLETE = "1";return run(["bun", "tauri", "dev", "--", "--profile", "debug-dev"], env);}- if (["build", "e2e", "signed", "test-hold"].includes(mode)) {+ if (["build", "e2e", "signed", "test", "test-hold"].includes(mode)) {return queue(mode as BuildMode, args);}- console.error("usage: bun scripts/native-build-queue.ts <dev|build|e2e|signed|status>");+ console.error("usage: bun scripts/native-build-queue.ts <dev|build|e2e|signed|test|status>");return 2;}diff --git a/docs/macos-dev-builds.md b/docs/macos-dev-builds.mdindex 4deb9e45d..bbbb38d09 100644--- a/docs/macos-dev-builds.md+++ b/docs/macos-dev-builds.md@@ -2,7 +2,7 @@<!-- doc-covers: none -->-There are exactly three normal native-development commands. Run them from+There are exactly four normal native-development commands. Run them from`apps/screenpipe-app-tauri`:```bash@@ -14,10 +14,14 @@ bun run build:tauri:dev# E2E-capable one-shot test binary.bun run build:tauri:e2e++# Native app tests; append ordinary cargo-test filters and flags.+bun run test:tauri activity_history::tests```-All package scripts pass the named profile as Cargo runner arguments:-`-- --profile debug-dev`. The space-separated form matters. It makes Tauri+The Tauri build scripts pass the named profile as Cargo runner arguments:+`-- --profile debug-dev`; the native-test command passes the same profile+directly to Cargo. The space-separated Tauri form matters. It makes Tauri2.11.2 select `src-tauri/target/debug-dev`; `--debug` instead selects Cargo'sbuilt-in `dev` profile.@@ -28,7 +32,7 @@ high parallel codegen, and no per-worktree incremental state.## System-wide build queue and cache-The three commands above and the signed build script all use one per-user build+The four commands above and the signed build script all use one per-user buildslot on macOS. A second worktree waits instead of starting another cold nativecompile on the same CPU. Queue output names the current build, PID, worktree,and wait time; inspect it directly with:@@ -45,6 +49,11 @@ worktrees without sharing target directories. The worktree list is refreshedbefore every queued native build. Do not point a second sccache server at thesame local cache directory; sccache local storage supports only one server.+Never run raw `cargo` or Tauri commands against `src-tauri`. They bypass the+system lock; a later queued build may legitimately restart sccache while+refreshing its worktree bases, causing the unqueued build to compile locally.+If the queue or cache is unavailable, stop instead of accepting that fallback.+`bun run dev:tauri` queues only its initial Cargo warm-up. It releases the buildslot before starting the long-running Tauri dev process, so an open app does notblock every later build. Subsequent hot-reload compiles belong to that live dev