YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
Rust agentsagiaiai-memoryaudio-recordingcomputer-visionhermeshermes-agentllmlocal-ailocal-firstmachine-learningmcpmultimodalopenclawprivacyrewindscreen-recordingspeech-to-textycombinator
fix(pipes): run event-triggered pipes once per event
Anshgrover23ad6760368779
diff --git a/TESTING.md b/TESTING.mdindex 012912fb5..4c218f95e 100644--- a/TESTING.md+++ b/TESTING.md@@ -699,6 +699,8 @@ commits: `fa887407`, `815f52e6`, `60840155`, `e66c3ff8`, `c905ffbf`, `01147096`,- [ ] **Pi agent & search timeouts** — Run a long-running search or Pi agent task. Verify it doesn't timeout prematurely at 60s (should allow up to 120s for search). (`f01213cf5`)- [ ] **allow schedule + triggers together in pipe config UI** — Verify that both schedule and triggers can be configured together in the pipe config UI without conflicts. (`f32bf9230`)- [ ] **local event triggers for pipes** — Verify that pipes can be triggered by local events such as meeting start/end and pipe chaining. (`508b68ff7`, `776587aa7`)+- [ ] **one event-triggered run per meeting** — With meeting-summary enabled, join a call, leave, rejoin the same app within 120s (this reopens the same meeting row), then leave again. `meeting_ended` fires twice for the same id; the pipe must run once. verify: `grep "suppressed duplicate 'meeting_ended'" ~/.screenpipe/screenpipe-app.*.log` and `sqlite3 ~/.screenpipe/db.sqlite "SELECT id, trigger_event, trigger_key FROM pipe_executions WHERE pipe_name='meeting-summary' ORDER BY id DESC LIMIT 5"` shows one row per meeting id. (#5481)+- [ ] **failed event run retries** — Force an event-triggered run to fail (e.g. sign out of the preset's provider mid-run). Its claim must be released so the next delivery of that event runs. verify: `sqlite3 ~/.screenpipe/db.sqlite "SELECT * FROM pipe_event_runs"` no longer holds the failed key. (#5481)- [ ] **tool call indicators in pipe run output** — Verify that tool call indicators are displayed in the pipe run output. (`dda33a6a7`)- [ ] **align copy and chat icons in pipe run history** — Verify that copy and chat icons are properly aligned in the pipe run history. (`f8fe5cff4`)- [ ] **Live pipe output streaming** — Open the "Runs" tab for a running pipe. Verify that the output streams live as the pipe executes. (`4c1fab276`)diff --git a/crates/screenpipe-core/assets/pipes/meeting-summary/pipe.md b/crates/screenpipe-core/assets/pipes/meeting-summary/pipe.mdindex ff7cf63eb..37fbb3328 100644--- a/crates/screenpipe-core/assets/pipes/meeting-summary/pipe.md+++ b/crates/screenpipe-core/assets/pipes/meeting-summary/pipe.md@@ -30,12 +30,19 @@ keep the wording of this prompt in sync with `buildMeetingSummarizeInstructions`read the screenpipe skill first so you know the meetings + search endpoints.-step 1 — find the meeting that just ended:+step 1 — find the meeting that just ended. when the scheduler woke you for an event it wrote `./.trigger-context.json` in this pipe's folder; read it first and use the meeting id it names:++ cat ./.trigger-context.json # {"event": "meeting_ended", "key": "<MEETING_ID>", ...}++ curl -s -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \+ "http://localhost:3030/meetings/<MEETING_ID>"++only if that file is missing (a manual run) fall back to the most recent row:curl -s -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \"http://localhost:3030/meetings?limit=1"-the most recent row is the one that just ended. capture its `id`, `meeting_start`, `meeting_end`, `title`, `note`, `meeting_app`, and `attendees`.+either way, capture the meeting's `id`, `meeting_start`, `meeting_end`, `title`, `note`, `meeting_app`, and `attendees`. prefer the id from the trigger file — "most recent" picks the wrong meeting when two end close together.step 2 — search screenpipe for what happened during this meeting and summarize it: key topics, decisions, action items. scope your searches to the meeting's `meeting_start`/`meeting_end` window. prefer `content_type=audio` for transcripts.diff --git a/crates/screenpipe-core/src/pipes/mod.rs b/crates/screenpipe-core/src/pipes/mod.rsindex 49b0c8d37..fd34f5ac6 100644--- a/crates/screenpipe-core/src/pipes/mod.rs+++ b/crates/screenpipe-core/src/pipes/mod.rs@@ -125,6 +125,16 @@ pub fn install_bundled_pipe(pipes_dir: &Path, name: &str) -> Result<bool> {/// reported on `pipe_scheduled_run` so the limit's behavior is observable.const EVENT_TRIGGERED_CONCURRENCY_LIMIT: usize = 4;+/// How long an event-run claim is remembered. Claims exist to suppress duplicate+/// deliveries of one event, so this only has to outlive the delivery window —+/// a week is generous and keeps the table small.+const EVENT_CLAIM_RETENTION_DAYS: i64 = 7;++/// Cap on triggers held back for a later tick. Reached only if a pipe stays busy+/// while its events keep arriving; beyond this the oldest are dropped rather than+/// growing without bound.+const MAX_CARRYOVER_EVENTS: usize = 64;+// ---------------------------------------------------------------------------// Config & log types// ---------------------------------------------------------------------------@@ -1161,6 +1171,10 @@ pub struct PipeExecution {pub error_message: Option<String>,pub duration_ms: Option<i64>,pub session_path: Option<String>,+ /// Event that triggered this run, and that event's identity (e.g. a meeting+ /// id). Both None for scheduled and manual runs.+ pub trigger_event: Option<String>,+ pub trigger_key: Option<String>,}/// Compact activity row for sidebar/history inventory.@@ -1201,6 +1215,51 @@ pub trait PipeStore: Send + Sync {provider: Option<&str>,) -> Result<i64>;+ /// Same, but records which event instance started the run so duplicate runs+ /// can be traced back to the events that caused them.+ async fn create_execution_with_trigger(+ &self,+ pipe_name: &str,+ trigger_type: &str,+ model: &str,+ provider: Option<&str>,+ _trigger_event: Option<&str>,+ _trigger_key: Option<&str>,+ ) -> Result<i64> {+ self.create_execution(pipe_name, trigger_type, model, provider)+ .await+ }++ /// Claim `(pipe, event, key)` for an event-triggered run. Returns false if+ /// the claim is already held, meaning this is a duplicate delivery.+ ///+ /// Default is "always won" so stores without persistence (CLI mode) keep the+ /// previous best-effort behaviour rather than blocking every run.+ async fn claim_event_run(+ &self,+ _pipe_name: &str,+ _event_name: &str,+ _event_key: &str,+ ) -> Result<bool> {+ Ok(true)+ }++ /// Release a claim so the event can trigger the pipe again. Called when the+ /// run fails, so a failure isn't remembered as a completed run.+ async fn release_event_run(+ &self,+ _pipe_name: &str,+ _event_name: &str,+ _event_key: &str,+ ) -> Result<()> {+ Ok(())+ }++ /// Drop claims older than `before`.+ async fn prune_event_runs(&self, _before: DateTime<Utc>) -> Result<()> {+ Ok(())+ }+/// Transition an execution to 'running' and record its PID.async fn set_execution_running(&self, id: i64, pid: Option<u32>) -> Result<()>;@@ -4754,6 +4813,19 @@ impl PipeManager {std::collections::HashMap::new();const CHAIN_COOLDOWN: std::time::Duration = std::time::Duration::from_secs(60);+ // Triggers that matched a pipe which was already busy. They used to be+ // dropped on the floor; instead they wait for the next tick, where the+ // idempotency claim decides whether they still need to run.+ let mut carryover: Vec<(String, serde_json::Value)> = Vec::new();++ // Claims are pruned once per process rather than every tick.+ if let Some(ref store) = store {+ let cutoff = Utc::now() - chrono::Duration::days(EVENT_CLAIM_RETENTION_DAYS);+ if let Err(e) = store.prune_event_runs(cutoff).await {+ warn!("scheduler: failed to prune event claims: {}", e);+ }+ }+loop {// Check for shutdownif *rx.borrow() {@@ -4775,23 +4847,33 @@ impl PipeManager {// Check each pipelet pipe_snapshot: Vec<(String, PipeConfig, String)> = {let p = pipes.lock().await;- p.iter()+ let mut snapshot: Vec<(String, PipeConfig, String)> = p+ .iter().map(|(n, (c, b, _))| (n.clone(), c.clone(), b.clone()))- .collect()+ .collect();+ // HashMap order is randomized per process, which made the start+ // order of pipes sharing a trigger unpredictable. Sorting makes+ // it reproducible; runs still overlap (see event_semaphore).+ snapshot.sort_by(|a, b| a.0.cmp(&b.0));+ snapshot};// Drain pending events and mark matching pipes for immediate execution.// Collect events from targeted subscriptions to avoid processing// high-frequency system events (ui_frame, window_ocr, etc.).- let mut event_triggered: std::collections::HashSet<String> =- std::collections::HashSet::new();+ let mut event_triggered: std::collections::HashMap<String, EventTrigger> =+ std::collections::HashMap::new();let mut connection_triggered: std::collections::HashSet<String> =std::collections::HashSet::new();+ // Triggers held back this tick, carried into the next one.+ let mut deferred: Vec<(String, serde_json::Value)> = Vec::new();{use futures::FutureExt;- // Collect events from all targeted subscriptions into one vec- let mut pending_events: Vec<(String, serde_json::Value)> = Vec::new();+ // Collect events from all targeted subscriptions into one vec,+ // starting with anything held back from the previous tick.+ let mut pending_events: Vec<(String, serde_json::Value)> =+ std::mem::take(&mut carryover);while let Some(e) = meeting_start_rx.next().now_or_never().flatten() {pending_events.push((e.name, e.data));}@@ -4825,7 +4907,18 @@ impl PipeManager {if name == target && config.enabled {info!("scheduler: connection trigger fired pipe '{}'", name);last_run.remove(name);- event_triggered.insert(name.clone());+ // No key: connection triggers already have their+ // own committed-cursor dedupe in+ // connection_triggers.rs, so they must not be+ // suppressed a second time here.+ event_triggered.insert(+ name.clone(),+ EventTrigger {+ name: e.name.clone(),+ key: None,+ data: e.data.clone(),+ },+ );connection_triggered.insert(name.clone());}}@@ -4835,7 +4928,7 @@ impl PipeManager {// Expire old chain cooldownsrecent_chain.retain(|_, ts| ts.elapsed() < CHAIN_COOLDOWN);- for (event_name, _data) in &pending_events {+ for (event_name, data) in &pending_events {for (name, config, _body) in &pipe_snapshot {if !config.enabled {continue;@@ -4871,11 +4964,15 @@ impl PipeManager {let r = running.lock().await;r.contains_key(name)};- if already_running {+ // Busy, or another event already claimed this pipe's+ // slot this tick. Either way the trigger waits for+ // the next tick instead of being lost.+ if already_running || event_triggered.contains_key(name) {debug!(- "scheduler: event '{}' skipped pipe '{}' (already running)",+ "scheduler: event '{}' deferred for pipe '{}' (busy)",event_name, name);+ deferred.push((event_name.clone(), data.clone()));continue;}info!(@@ -4883,7 +4980,14 @@ impl PipeManager {event_name, name);last_run.remove(name);- event_triggered.insert(name.clone());+ event_triggered.insert(+ name.clone(),+ EventTrigger {+ name: event_name.clone(),+ key: event_identity_key(data),+ data: data.clone(),+ },+ );}}}@@ -4894,7 +4998,7 @@ impl PipeManager {continue;}- let triggered_by_event = event_triggered.contains(name);+ let triggered_by_event = event_triggered.contains_key(name);let trigger = if connection_triggered.contains(name) {"connection"} else if triggered_by_event {@@ -4947,6 +5051,17 @@ impl PipeManager {{let qr = queued_or_running.lock().await;if qr.contains(name) {+ // Hold an event trigger back rather than dropping it —+ // the run in flight may be for a different event.+ // Connection triggers are excluded: they are addressed+ // to a pipe by name, so replaying them through event+ // matching wouldn't fire anything, and their watcher+ // already retries uncommitted fires.+ if !connection_triggered.contains(name) {+ if let Some(t) = event_triggered.get(name) {+ deferred.push((t.name.clone(), t.data.clone()));+ }+ }continue;}}@@ -5097,6 +5212,37 @@ impl PipeManager {}}+ // Claim this (pipe, event, key) before committing to a run. A+ // second delivery of the same logical event — `meeting_ended`+ // fires again for a meeting the user rejoined — loses the claim+ // and is suppressed here instead of starting a duplicate run+ // (#5481).+ //+ // Claimed after the guards above, not when the event was+ // matched, so a trigger that lost one of those races is still+ // free to retry on a later tick. Must stay ahead of the queue+ // mark below: suppressing after that would leave the pipe+ // marked as queued forever.+ let mut event_claim: Option<(String, String)> = None;+ if let (Some(t), Some(store)) = (event_triggered.get(name), store.as_ref()) {+ if let Some(key) = t.key.clone() {+ match store.claim_event_run(name, &t.name, &key).await {+ Ok(true) => event_claim = Some((t.name.clone(), key)),+ Ok(false) => {+ info!(+ "scheduler: suppressed duplicate '{}' for pipe '{}' (key {}), already ran",+ t.name, name, key+ );+ continue;+ }+ // Fail open: a store error must not stop the run.+ Err(e) => {+ warn!("scheduler: could not claim event for '{}': {}", name, e)+ }+ }+ }+ }+// Mark as queued so the next tick doesn't double-queue{let mut qr = queued_or_running.lock().await;@@ -5186,6 +5332,19 @@ impl PipeManager {Some("ai_preset_unavailable"),);}+ // The run never happened, so don't let the claim+ // suppress the next delivery of this event.+ if let (Some((event, key)), Some(store)) =+ (&event_claim, store.as_ref())+ {+ if let Err(e) = store.release_event_run(name, event, key).await+ {+ warn!(+ "scheduler: could not release event claim for '{}': {}",+ name, e+ );+ }+ }continue;}None => (@@ -5293,6 +5452,22 @@ impl PipeManager {let queued_ref = queued_or_running.clone();let mcp_server_allowlist = selected_mcp_server_ids(config);+ // Tell the pipe which event fired it, so it acts on that+ // meeting instead of guessing at the most recent one.+ //+ // Skipped for connection triggers: their watcher already wrote+ // the same file with the items it detected, and overwriting it+ // would take that payload away from the pipe.+ if !connection_triggered.contains(name) {+ if let Some(t) = event_triggered.get(name) {+ write_event_trigger_context(&pipe_dir, t);+ }+ }++ let claim_for_release = event_claim.clone();+ let trigger_event_name = event_claim.as_ref().map(|(e, _)| e.clone());+ let trigger_event_key = event_claim.as_ref().map(|(_, k)| k.clone());+tokio::spawn(async move {// Scheduled pipes wait for the previous one to finish// (semaphore of 1). Event-triggered pipes skip that queue@@ -5336,7 +5511,14 @@ impl PipeManager {// Create DB execution rowlet exec_id = if let Some(ref store) = store_ref {match store- .create_execution(&pipe_name, &trigger, &model, provider.as_deref())+ .create_execution_with_trigger(+ &pipe_name,+ &trigger,+ &model,+ provider.as_deref(),+ trigger_event_name.as_deref(),+ trigger_event_key.as_deref(),+ ).await{Ok(id) => {@@ -5637,6 +5819,24 @@ impl PipeManager {let duration_secs =(finished_at - started_at).num_milliseconds() as f64 / 1000.0;let success = log.success;++ // A failed run shouldn't be remembered as one that happened —+ // drop the claim so the event can trigger the pipe again.+ if !success {+ if let (Some((event, key)), Some(ref store)) =+ (&claim_for_release, &store_ref)+ {+ if let Err(e) =+ store.release_event_run(&pipe_name, event, key).await+ {+ warn!(+ "scheduler: could not release event claim for '{}': {}",+ pipe_name, e+ );+ }+ }+ }+let name_for_cb = log.pipe_name.clone();let mut l = logs_ref.lock().await;let entry = l.entry(log.pipe_name.clone()).or_insert_with(VecDeque::new);@@ -5715,6 +5915,20 @@ impl PipeManager {last_cleanup = Instant::now();}+ // Retry deferred triggers next tick, newest last so the oldest are+ // the ones dropped if a pipe stays busy indefinitely.+ if !deferred.is_empty() {+ carryover.extend(deferred);+ if carryover.len() > MAX_CARRYOVER_EVENTS {+ let dropped = carryover.len() - MAX_CARRYOVER_EVENTS;+ carryover.drain(..dropped);+ warn!(+ "scheduler: dropped {} deferred trigger(s), backlog over {}",+ dropped, MAX_CARRYOVER_EVENTS+ );+ }+ }+// Sleep 30s between checkstokio::select! {_ = tokio::time::sleep(std::time::Duration::from_secs(30)) => {},@@ -6275,6 +6489,58 @@ fn validate_one_off_freshness(schedule: &str) -> Result<()> {Ok(())}+/// Name of the file a pipe reads to find out what triggered it. One file per run,+/// written by whichever trigger fired: connection triggers write their detected+/// items here, event triggers write the event below.+const EVENT_TRIGGER_CONTEXT_FILE: &str = ".trigger-context.json";++/// Record what fired the pipe, next to the pipe itself.+///+/// Without this a `meeting_ended` pipe has to guess which meeting it was woken+/// for, which goes wrong exactly when two meetings end close together.+fn write_event_trigger_context(pipe_dir: &Path, trigger: &EventTrigger) {+ if !pipe_dir.is_dir() {+ return;+ }+ let ctx = serde_json::json!({+ "event": trigger.name,+ "key": trigger.key,+ "data": trigger.data,+ "triggered_at": Utc::now().to_rfc3339(),+ });+ if let Ok(s) = serde_json::to_string_pretty(&ctx) {+ let _ = atomic_write(&pipe_dir.join(EVENT_TRIGGER_CONTEXT_FILE), &s);+ }+}++/// One event delivery that matched a pipe's `trigger.events`.+///+/// The payload is kept so the trigger can be held back for a later tick if the+/// pipe is busy, and so the pipe can be told what fired it.+#[derive(Clone, Debug)]+struct EventTrigger {+ /// Event name as matched against `trigger.events`.+ name: String,+ /// The event's own identity, if it has one — a meeting id for `meeting_*`.+ /// Events without one can only be deduplicated within a single tick.+ key: Option<String>,+ data: serde_json::Value,+}++/// Read the identity of an event out of its payload.+///+/// Events that describe a specific thing carry its id (`meeting_ended` carries+/// `meeting_id`), which is what makes "already ran for this one" answerable.+fn event_identity_key(data: &serde_json::Value) -> Option<String> {+ ["meeting_id", "event_id", "id"]+ .iter()+ .find_map(|field| match data.get(field) {+ Some(serde_json::Value::String(s)) if !s.is_empty() => Some(s.clone()),+ Some(serde_json::Value::Number(n)) => Some(n.to_string()),+ _ => None,+ })+}+fn pipe_completed_source(event_name: &str) -> Option<&str> {event_name.strip_prefix(PIPE_COMPLETED_EVENT_PREFIX)}@@ -7144,6 +7410,71 @@ mod tests {PipeManager::new(dir, HashMap::new(), None, 0)}+ // -- event trigger identity (#5481) -------------------------------------++ /// Both deliveries of a rejoined meeting's end must produce the same key,+ /// otherwise the claim can't recognize the second one as a duplicate.+ #[test]+ fn duplicate_meeting_ended_yields_same_key() {+ let first = serde_json::json!({ "meeting_id": 42 });+ let second = serde_json::json!({ "meeting_id": 42 });+ assert_eq!(event_identity_key(&first).as_deref(), Some("42"));+ assert_eq!(event_identity_key(&first), event_identity_key(&second));+ }++ #[test]+ fn different_meetings_yield_different_keys() {+ let a = serde_json::json!({ "meeting_id": 42 });+ let b = serde_json::json!({ "meeting_id": 43 });+ assert_ne!(event_identity_key(&a), event_identity_key(&b));+ }++ #[test]+ fn event_identity_key_accepts_string_ids() {+ let data = serde_json::json!({ "event_id": "abc-123" });+ assert_eq!(event_identity_key(&data).as_deref(), Some("abc-123"));+ }++ /// Events with no identity fall back to the per-tick guard, so they must not+ /// produce a key that would collide across unrelated deliveries.+ #[test]+ fn keyless_events_have_no_identity() {+ assert_eq!(event_identity_key(&serde_json::json!({})), None);+ assert_eq!(+ event_identity_key(&serde_json::json!({ "meeting_id": null })),+ None+ );+ assert_eq!(+ event_identity_key(&serde_json::json!({ "meeting_id": "" })),+ None+ );+ }++ #[test]+ fn event_trigger_context_records_what_fired_the_pipe() {+ let dir = std::env::temp_dir().join(format!(+ "screenpipe-trigger-ctx-{}-{}",+ std::process::id(),+ line!()+ ));+ std::fs::create_dir_all(&dir).unwrap();++ let trigger = EventTrigger {+ name: "meeting_ended".to_string(),+ key: Some("42".to_string()),+ data: serde_json::json!({ "meeting_id": 42 }),+ };+ write_event_trigger_context(&dir, &trigger);++ let written = std::fs::read_to_string(dir.join(EVENT_TRIGGER_CONTEXT_FILE)).unwrap();+ let ctx: serde_json::Value = serde_json::from_str(&written).unwrap();+ assert_eq!(ctx["event"], "meeting_ended");+ assert_eq!(ctx["key"], "42");+ assert_eq!(ctx["data"]["meeting_id"], 42);++ let _ = std::fs::remove_dir_all(&dir);+ }+fn pipe_source(template: bool, body: &str) -> String {format!("---\nschedule: manual\nenabled: true\ntemplate: {}\n---\n\n{}\n",@@ -9186,6 +9517,8 @@ mod tests {error_message: None,duration_ms: Some(60000),session_path: None,+ trigger_event: Some("meeting_ended".to_string()),+ trigger_key: Some("42".to_string()),};let json = serde_json::to_string(&exec).unwrap();let parsed: PipeExecution = serde_json::from_str(&json).unwrap();@@ -9193,6 +9526,8 @@ mod tests {assert_eq!(parsed.status, "completed");assert_eq!(parsed.pid, Some(1234));assert_eq!(parsed.duration_ms, Some(60000));+ assert_eq!(parsed.trigger_event.as_deref(), Some("meeting_ended"));+ assert_eq!(parsed.trigger_key.as_deref(), Some("42"));}#[test]diff --git a/crates/screenpipe-db/src/db/write_ops.rs b/crates/screenpipe-db/src/db/write_ops.rsindex ebb072423..90a4e0f10 100644--- a/crates/screenpipe-db/src/db/write_ops.rs+++ b/crates/screenpipe-db/src/db/write_ops.rs@@ -237,12 +237,17 @@ impl DatabaseManager {}/// Create a pipe execution via the write queue. Returns the new row ID.+ ///+ /// `trigger_event` / `trigger_key` record which event instance started the+ /// run (e.g. `meeting_ended` / meeting id); both None for scheduled runs.pub async fn pipe_create_execution_queued(&self,pipe_name: &str,trigger_type: &str,model: &str,provider: Option<&str>,+ trigger_event: Option<&str>,+ trigger_key: Option<&str>,) -> Result<i64, sqlx::Error> {use crate::write_queue::{WriteOp, WriteResult};let result = self@@ -253,6 +258,8 @@ impl DatabaseManager {model: model.to_string(),provider: provider.map(|s| s.to_string()),started_at: chrono::Utc::now().to_rfc3339(),+ trigger_event: trigger_event.map(|s| s.to_string()),+ trigger_key: trigger_key.map(|s| s.to_string()),}).await?;match result {@@ -261,6 +268,64 @@ impl DatabaseManager {}}+ /// Claim `(pipe, event, key)` for an event-triggered run.+ ///+ /// Returns true if this caller won the claim, false if it was already held —+ /// i.e. a duplicate delivery of the same logical event.+ pub async fn pipe_claim_event_run_queued(+ &self,+ pipe_name: &str,+ event_name: &str,+ event_key: &str,+ ) -> Result<bool, sqlx::Error> {+ use crate::write_queue::{WriteOp, WriteResult};+ let result = self+ .write_queue+ .submit(WriteOp::PipeClaimEventRun {+ pipe_name: pipe_name.to_string(),+ event_name: event_name.to_string(),+ event_key: event_key.to_string(),+ claimed_at: chrono::Utc::now().to_rfc3339(),+ })+ .await?;+ match result {+ WriteResult::Id(rows) => Ok(rows > 0),+ _ => unreachable!(),+ }+ }++ /// Release a claim so the event can trigger the pipe again.+ pub async fn pipe_release_event_run_queued(+ &self,+ pipe_name: &str,+ event_name: &str,+ event_key: &str,+ ) -> Result<(), sqlx::Error> {+ use crate::write_queue::WriteOp;+ self.write_queue+ .submit(WriteOp::PipeReleaseEventRun {+ pipe_name: pipe_name.to_string(),+ event_name: event_name.to_string(),+ event_key: event_key.to_string(),+ })+ .await?;+ Ok(())+ }++ /// Drop claims older than `before`.+ pub async fn pipe_prune_event_runs_queued(+ &self,+ before: chrono::DateTime<chrono::Utc>,+ ) -> Result<(), sqlx::Error> {+ use crate::write_queue::WriteOp;+ self.write_queue+ .submit(WriteOp::PipePruneEventRuns {+ before: before.to_rfc3339(),+ })+ .await?;+ Ok(())+ }+/// Execute a pipe UPDATE/INSERT/DELETE via the write queue.pub async fn pipe_execute_write_queued(&self,diff --git a/crates/screenpipe-db/src/migrations/20260728000000_create_pipe_event_runs.sql b/crates/screenpipe-db/src/migrations/20260728000000_create_pipe_event_runs.sqlnew file mode 100644index 000000000..8ff3f7002--- /dev/null+++ b/crates/screenpipe-db/src/migrations/20260728000000_create_pipe_event_runs.sql@@ -0,0 +1,24 @@+-- Idempotency for event-triggered pipe runs (#5481).+--+-- `meeting_ended` is not once-per-meeting: the 120s rejoin window reopens the+-- same meetings row, so ending a rejoined call emits the same meeting_id twice.+-- The scheduler's in-memory guard is keyed on pipe slug and resets every tick,+-- so two ends landing in different ticks each started a run.+--+-- One row per (pipe, event, key) the scheduler has claimed. Claims are inserted+-- with INSERT OR IGNORE, so the primary key is what makes the claim atomic.+CREATE TABLE IF NOT EXISTS pipe_event_runs (+ pipe_name TEXT NOT NULL,+ event_name TEXT NOT NULL,+ event_key TEXT NOT NULL,+ claimed_at TEXT NOT NULL,+ PRIMARY KEY (pipe_name, event_name, event_key)+);++-- Supports pruning by age.+CREATE INDEX IF NOT EXISTS idx_pipe_event_runs_claimed_at ON pipe_event_runs(claimed_at);++-- Which event instance produced a run, so duplicates are diagnosable after the+-- fact. trigger_type only says "event"; these say which one.+ALTER TABLE pipe_executions ADD COLUMN trigger_event TEXT;+ALTER TABLE pipe_executions ADD COLUMN trigger_key TEXT;diff --git a/crates/screenpipe-db/src/write_queue.rs b/crates/screenpipe-db/src/write_queue.rsindex 5b04dc601..e0b35fe74 100644--- a/crates/screenpipe-db/src/write_queue.rs+++ b/crates/screenpipe-db/src/write_queue.rs@@ -517,6 +517,10 @@ pub(crate) enum WriteOp {model: String,provider: Option<String>,started_at: String,+ /// Event that triggered the run, and the event's identity (e.g. a+ /// meeting id). Both None for scheduled and manual runs.+ trigger_event: Option<String>,+ trigger_key: Option<String>,},PipeUpdateExecution {sql: String,@@ -525,6 +529,24 @@ pub(crate) enum WriteOp {PipeDeleteOldExecutions {keep_per_pipe: i32,},+ /// Claim a (pipe, event, key) tuple for an event-triggered run. Returns the+ /// number of rows inserted: 1 on a fresh claim, 0 if already claimed.+ PipeClaimEventRun {+ pipe_name: String,+ event_name: String,+ event_key: String,+ claimed_at: String,+ },+ /// Drop a claim so the event can be retried (used when the run fails).+ PipeReleaseEventRun {+ pipe_name: String,+ event_name: String,+ event_key: String,+ },+ /// Prune claims older than `before`, bounding the table's growth.+ PipePruneEventRuns {+ before: String,+ },/// Insert a synced frame (video_chunk + frame row) from cloud sync import.SyncInsertFrame {sync_id: String,@@ -2000,10 +2022,12 @@ async fn execute_single_write(model,provider,started_at,+ trigger_event,+ trigger_key,} => {let row = sqlx::query_scalar::<_, i64>(- r#"INSERT INTO pipe_executions (pipe_name, status, trigger_type, model, provider, started_at)- VALUES (?, 'queued', ?, ?, ?, ?)+ r#"INSERT INTO pipe_executions (pipe_name, status, trigger_type, model, provider, started_at, trigger_event, trigger_key)+ VALUES (?, 'queued', ?, ?, ?, ?, ?, ?)RETURNING id"#,).bind(pipe_name)@@ -2011,11 +2035,60 @@ async fn execute_single_write(.bind(model).bind(provider).bind(started_at)+ .bind(trigger_event)+ .bind(trigger_key).fetch_one(&mut **conn).await?;Ok(WriteResult::Id(row))}+ WriteOp::PipeClaimEventRun {+ pipe_name,+ event_name,+ event_key,+ claimed_at,+ } => {+ // OR IGNORE + the primary key is the whole idempotency guarantee:+ // a duplicate claim inserts 0 rows instead of erroring.+ let result = sqlx::query(+ r#"INSERT OR IGNORE INTO pipe_event_runs+ (pipe_name, event_name, event_key, claimed_at)+ VALUES (?, ?, ?, ?)"#,+ )+ .bind(pipe_name)+ .bind(event_name)+ .bind(event_key)+ .bind(claimed_at)+ .execute(&mut **conn)+ .await?;+ Ok(WriteResult::Id(result.rows_affected() as i64))+ }++ WriteOp::PipeReleaseEventRun {+ pipe_name,+ event_name,+ event_key,+ } => {+ sqlx::query(+ r#"DELETE FROM pipe_event_runs+ WHERE pipe_name = ? AND event_name = ? AND event_key = ?"#,+ )+ .bind(pipe_name)+ .bind(event_name)+ .bind(event_key)+ .execute(&mut **conn)+ .await?;+ Ok(WriteResult::Unit)+ }++ WriteOp::PipePruneEventRuns { before } => {+ sqlx::query("DELETE FROM pipe_event_runs WHERE claimed_at < ?")+ .bind(before)+ .execute(&mut **conn)+ .await?;+ Ok(WriteResult::Unit)+ }+WriteOp::PipeUpdateExecution { sql, binds } => {// Pipe update SQL is assembled internally with bind placeholders;// runtime values remain separate in `binds` below.diff --git a/crates/screenpipe-engine/src/pipe_store.rs b/crates/screenpipe-engine/src/pipe_store.rsindex 71904eb8d..c8fde2169 100644--- a/crates/screenpipe-engine/src/pipe_store.rs+++ b/crates/screenpipe-engine/src/pipe_store.rs@@ -35,10 +35,61 @@ impl PipeStore for SqlitePipeStore {) -> Result<i64> {Ok(self.db- .pipe_create_execution_queued(pipe_name, trigger_type, model, provider)+ .pipe_create_execution_queued(pipe_name, trigger_type, model, provider, None, None).await?)}+ async fn create_execution_with_trigger(+ &self,+ pipe_name: &str,+ trigger_type: &str,+ model: &str,+ provider: Option<&str>,+ trigger_event: Option<&str>,+ trigger_key: Option<&str>,+ ) -> Result<i64> {+ Ok(self+ .db+ .pipe_create_execution_queued(+ pipe_name,+ trigger_type,+ model,+ provider,+ trigger_event,+ trigger_key,+ )+ .await?)+ }++ async fn claim_event_run(+ &self,+ pipe_name: &str,+ event_name: &str,+ event_key: &str,+ ) -> Result<bool> {+ Ok(self+ .db+ .pipe_claim_event_run_queued(pipe_name, event_name, event_key)+ .await?)+ }++ async fn release_event_run(+ &self,+ pipe_name: &str,+ event_name: &str,+ event_key: &str,+ ) -> Result<()> {+ self.db+ .pipe_release_event_run_queued(pipe_name, event_name, event_key)+ .await?;+ Ok(())+ }++ async fn prune_event_runs(&self, before: DateTime<Utc>) -> Result<()> {+ self.db.pipe_prune_event_runs_queued(before).await?;+ Ok(())+ }+async fn set_execution_running(&self, id: i64, pid: Option<u32>) -> Result<()> {use screenpipe_db::write_queue::PipeBindValue;let now = Utc::now().to_rfc3339();@@ -116,7 +167,8 @@ impl PipeStore for SqlitePipeStore {sqlx::query_as::<_, PipeExecutionRow>(r#"SELECT id, pipe_name, status, trigger_type, pid, model, provider,started_at, finished_at, stdout, stderr, exit_code,- error_type, error_message, duration_ms, session_path+ error_type, error_message, duration_ms, session_path,+ trigger_event, trigger_keyFROM pipe_executionsWHERE pipe_name = ? AND id < ?ORDER BY id DESC@@ -131,7 +183,8 @@ impl PipeStore for SqlitePipeStore {sqlx::query_as::<_, PipeExecutionRow>(r#"SELECT id, pipe_name, status, trigger_type, pid, model, provider,started_at, finished_at, stdout, stderr, exit_code,- error_type, error_message, duration_ms, session_path+ error_type, error_message, duration_ms, session_path,+ trigger_event, trigger_keyFROM pipe_executionsWHERE pipe_name = ?ORDER BY id DESC@@ -157,7 +210,8 @@ impl PipeStore for SqlitePipeStore {r#"SELECT id, pipe_name, status, trigger_type, pid, model, provider,started_at, finished_at,'' AS stdout, '' AS stderr,- exit_code, error_type, error_message, duration_ms, session_path+ exit_code, error_type, error_message, duration_ms, session_path,+ trigger_event, trigger_keyFROM pipe_executionsWHERE pipe_name = ? AND id < ?ORDER BY id DESC@@ -173,7 +227,8 @@ impl PipeStore for SqlitePipeStore {r#"SELECT id, pipe_name, status, trigger_type, pid, model, provider,started_at, finished_at,'' AS stdout, '' AS stderr,- exit_code, error_type, error_message, duration_ms, session_path+ exit_code, error_type, error_message, duration_ms, session_path,+ trigger_event, trigger_keyFROM pipe_executionsWHERE pipe_name = ?ORDER BY id DESC@@ -338,7 +393,8 @@ impl PipeStore for SqlitePipeStore {r#"SELECT id, pipe_name, status, trigger_type, pid, model, provider,started_at, finished_at,'' AS stdout, '' AS stderr,- exit_code, error_type, error_message, duration_ms, session_path+ exit_code, error_type, error_message, duration_ms, session_path,+ trigger_event, trigger_keyFROM pipe_executionsWHERE pipe_name = ?ORDER BY id DESC@@ -447,6 +503,8 @@ struct PipeExecutionRow {error_message: Option<String>,duration_ms: Option<i64>,session_path: Option<String>,+ trigger_event: Option<String>,+ trigger_key: Option<String>,}impl From<PipeExecutionRow> for PipeExecution {@@ -468,6 +526,8 @@ impl From<PipeExecutionRow> for PipeExecution {error_message: r.error_message,duration_ms: r.duration_ms,session_path: r.session_path,+ trigger_event: r.trigger_event,+ trigger_key: r.trigger_key,}}}@@ -600,6 +660,116 @@ mod tests {.unwrap();}+ /// #5481: `meeting_ended` fires twice for one meeting when the user rejoins+ /// within the 120s merge window. The second delivery must not win a claim.+ #[tokio::test(flavor = "multi_thread", worker_threads = 2)]+ async fn test_duplicate_meeting_ended_claims_once() {+ let (store, _tmp) = setup_test_store().await;++ assert!(store+ .claim_event_run("meeting-summary", "meeting_ended", "42")+ .await+ .unwrap());+ assert!(+ !store+ .claim_event_run("meeting-summary", "meeting_ended", "42")+ .await+ .unwrap(),+ "duplicate meeting_ended for meeting 42 should be suppressed"+ );++ // A different meeting is unrelated work.+ assert!(store+ .claim_event_run("meeting-summary", "meeting_ended", "43")+ .await+ .unwrap());+ // So is the same meeting seen by a different pipe.+ assert!(store+ .claim_event_run("meeting-notes", "meeting_ended", "42")+ .await+ .unwrap());+ }++ #[tokio::test(flavor = "multi_thread", worker_threads = 2)]+ async fn test_released_claim_can_run_again() {+ let (store, _tmp) = setup_test_store().await;++ assert!(store+ .claim_event_run("p", "meeting_ended", "7")+ .await+ .unwrap());+ store+ .release_event_run("p", "meeting_ended", "7")+ .await+ .unwrap();+ assert!(+ store+ .claim_event_run("p", "meeting_ended", "7")+ .await+ .unwrap(),+ "a failed run releases its claim so the event can be retried"+ );+ }++ #[tokio::test(flavor = "multi_thread", worker_threads = 2)]+ async fn test_prune_event_runs_drops_old_claims() {+ let (store, _tmp) = setup_test_store().await;++ assert!(store+ .claim_event_run("p", "meeting_ended", "1")+ .await+ .unwrap());+ store+ .prune_event_runs(Utc::now() + chrono::Duration::days(1))+ .await+ .unwrap();+ assert!(+ store+ .claim_event_run("p", "meeting_ended", "1")+ .await+ .unwrap(),+ "pruned claims no longer suppress"+ );+ }++ #[tokio::test(flavor = "multi_thread", worker_threads = 2)]+ async fn test_execution_records_triggering_event() {+ let (store, _tmp) = setup_test_store().await;++ store+ .create_execution_with_trigger(+ "meeting-summary",+ "event",+ "opus",+ None,+ Some("meeting_ended"),+ Some("42"),+ )+ .await+ .unwrap();++ let execs = store+ .get_executions("meeting-summary", 10, None)+ .await+ .unwrap();+ assert_eq!(execs[0].trigger_event.as_deref(), Some("meeting_ended"));+ assert_eq!(execs[0].trigger_key.as_deref(), Some("42"));+ }++ #[tokio::test(flavor = "multi_thread", worker_threads = 2)]+ async fn test_scheduled_execution_has_no_trigger_event() {+ let (store, _tmp) = setup_test_store().await;++ store+ .create_execution("p", "scheduled", "m", None)+ .await+ .unwrap();++ let execs = store.get_executions("p", 10, None).await.unwrap();+ assert_eq!(execs[0].trigger_event, None);+ assert_eq!(execs[0].trigger_key, None);+ }+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]async fn test_create_execution() {let (store, _tmp) = setup_test_store().await;