YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
Rust agentsagiaiai-memoryaudio-recordingcomputer-visionhermeshermes-agentllmlocal-ailocal-firstmachine-learningmcpmultimodalopenclawprivacyrewindscreen-recordingspeech-to-textycombinator
test(evals): add ACP permission boundary regression
Louis Beaumontfb603db17971
diff --git a/evals/coding-agent/README.md b/evals/coding-agent/README.mdindex 4c2fe67d5..1c0d27a3f 100644--- a/evals/coding-agent/README.md+++ b/evals/coding-agent/README.md@@ -9,7 +9,7 @@ This is an agent eval suite, not a unit-test suite. Every case contains:- saved prompt, transcript, candidate patch, grader output, runtime fingerprint, and result;- repeated-trial reporting with success rate, `pass@k`, and `pass^k`.-The current app corpus contains 20 git-mined regressions. See+The current app corpus contains 21 git-mined regressions. See[DESIGN.md](./DESIGN.md) for the Anthropic guidance, source contract, andhistory-mining workflow. The companion website manifest contributes 20 more.diff --git a/evals/coding-agent/cases.json b/evals/coding-agent/cases.jsonindex b0b2e08aa..a6e95fd69 100644--- a/evals/coding-agent/cases.json+++ b/evals/coding-agent/cases.json@@ -1,7 +1,7 @@{"schema_version": 1,"suite": "screenpipe-app-coding-agent-regressions",- "dataset_version": "2026-08-19.3",+ "dataset_version": "2026-08-23.1","suite_type": "regression","owner": "screenpipe engineering","policy": {@@ -500,6 +500,39 @@],"command": "cd apps/screenpipe-app-tauri && bun x vitest run --config vitest.config.ts lib/settings/__tests__/capture-categories.test.ts"}+ },+ {+ "id": "app-acp-explicit-full-access-boundary",+ "name": "Keep ACP full access explicit, reversible, and adapter-safe",+ "source": {+ "kind": "git_regression",+ "fix_commit": "e83222db608f55b3931204e061262469411de94a",+ "pull_request": "https://github.com/screenpipe/screenpipe/pull/6514"+ },+ "base_ref": "989a53b86af113e0e408b4ee6c5f7dd14d886d3d",+ "oracle_ref": "e83222db608f55b3931204e061262469411de94a",+ "gate": "advisory",+ "trigger_paths": [+ "apps/screenpipe-app-tauri/components/chat/standalone/**acp**",+ "apps/screenpipe-app-tauri/lib/stores/acp-session-config.ts",+ "apps/screenpipe-app-tauri/lib/utils/tauri.ts",+ "apps/screenpipe-app-tauri/src-tauri/src/pi.rs",+ "crates/screenpipe-core/src/agents/acp/**"+ ],+ "tags": ["regression", "escaped-failure", "chat", "acp", "permissions", "safety-boundary"],+ "prompt": "An ACP adapter can advertise working modes without an unrestricted one, leaving users unable to grant full access explicitly. Add a clearly warned opt-in full-access choice that restores a tool-capable mode and enables client-side automatic approval. Keep the default and unknown states approval-based, return to approval prompts when a prompted or read-only choice is selected, persist and reapply the policy for live sessions, and preserve adapter-owned unrestricted controls without duplicates.",+ "agent_timeout_seconds": 1200,+ "grader_dependency_links": [+ { "source_path": "apps/screenpipe-app-tauri/node_modules", "destination_path": "apps/screenpipe-app-tauri/node_modules" }+ ],+ "grader": {+ "timeout_seconds": 420,+ "fixtures": [+ { "source_path": "apps/screenpipe-app-tauri/components/chat/standalone/acp-permission-selector.test.tsx" },+ { "source_path": "apps/screenpipe-app-tauri/lib/stores/acp-session-config.test.ts" }+ ],+ "command": "cd apps/screenpipe-app-tauri && NODE_OPTIONS=--localstorage-file=.eval-localstorage bun x vitest run --config vitest.config.ts components/chat/standalone/acp-permission-selector.test.tsx lib/stores/acp-session-config.test.ts"+ }}]}