YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
Rust agentsagiaiai-memoryaudio-recordingcomputer-visionhermeshermes-agentllmlocal-ailocal-firstmachine-learningmcpmultimodalopenclawprivacyrewindscreen-recordingspeech-to-textycombinator
[pr] skills: move the failing contracts into the examples agents copy (#6281)
Louis Beaumontdff7793b349b
diff --git a/crates/screenpipe-core/assets/skills/screenpipe-api/SKILL.md b/crates/screenpipe-core/assets/skills/screenpipe-api/SKILL.mdindex ed91fbe1c..1777e2f6f 100644--- a/crates/screenpipe-core/assets/skills/screenpipe-api/SKILL.md+++ b/crates/screenpipe-core/assets/skills/screenpipe-api/SKILL.md@@ -33,9 +33,20 @@ No-auth endpoints: `/health`, `/ws/health`, `/audio/device/status`, `/connection## Context Window Protection-Responses can be large. Write curl output to a file (`-o /tmp/sp.json`), check size (`wc -c`), and if over ~5KB read only the first 50-100 lines / extract with `jq`. Never dump full large responses into context.+Responses can be large. Write curl output to a file (`-o /tmp/sp.json`), check size (`wc -c`), and if over ~5KB read only the first 50-100 lines. Never dump full large responses into context.-Cut tokens at the source on list endpoints (`/search`, `/elements`): add `&format=csv` (or `tsv`) for a columnar table (column names written once instead of per-row keys — ~70% cheaper on uniform rows like elements), and `&fields=a,b,c` for only the columns you need (dotted paths like `content.text`). Text-heavy `ocr`/`audio` barely benefit — use `fields` + `max_content_length` there instead.+**Only assume `curl`, `wc`, `head`, `grep`, `sed` and `bun` exist.** `jq` is *not* installed on every machine — stock macOS and the bundled Windows bash both lack it. To pull fields out of JSON, either ask the API for flat rows (`format=csv`, below) and read them with `head`, or use bun, which always ships with screenpipe:++```bash+bun -e 'const d=await Bun.file("/tmp/sp.json").json(); for (const r of d.data.slice(0,20)) console.log(r.type, r.content.app_name??"", (r.content.text??r.content.transcription??"").slice(0,120))'+```++Use `jq` only after confirming it exists (`command -v jq`).++Cut tokens at the source on list endpoints (`/search`, `/elements`). Two independent knobs, both shown in the examples below — copy them:++- **`&fields=a,b,c`** — always set it. Dotted paths (`content.text`, `content.app_name`). Applies to every content type, including text-heavy `ocr`/`audio`, where you should also set `max_content_length`.+- **`&format=csv`** (or `tsv`) — columnar table, column names written once instead of per-row keys. ~70% cheaper on *uniform* rows, so use it on `/elements` and on single-`content_type` `/search` calls. Skip it on mixed `content_type=all`, where rows have different shapes and CSV gains little.---@@ -64,14 +75,16 @@ Use when `/activity-summary` says `ok` but you need verbatim quotes, media paths```bashcurl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \-H "X-Screenpipe-Client: api" \- "http://localhost:3030/search?q=QUERY&content_type=all&limit=10&start_time=1h%20ago"+ -o /tmp/sp.json \+ "http://localhost:3030/search?q=QUERY&content_type=all&limit=10&start_time=1h%20ago&fields=type,content.app_name,content.text,content.transcription,content.timestamp"+wc -c /tmp/sp.json && head -c 2000 /tmp/sp.json```| Parameter | Required | Description ||-----------|----------|-------------|| `q` | No | Keywords. Avoid for audio — transcriptions are noisy, `q` over-filters. || `content_type` | No | `all` (default), `accessibility`, `audio`, `input`, `ocr`, `memory`, `parsed`. Use `parsed` for compact app-specific messages, emails, tasks, documents, and code review. Parsed capture is experimental, may be empty when disabled/unsupported, and is not included in `all`. Screen text is primarily the accessibility tree; OCR is the fallback for apps without it (videos, games, remote desktops). |-| `limit` | No | Default 20. Keep ≤20 to protect context. |+| `limit` | No | Default 20. Must be 1-20 — never pass a larger value; page with `offset` instead. || `offset` | No | Pagination. Default 0. || `start_time` | **Yes** | ISO 8601 or relative (`16h ago`, `2d ago`, `30m ago`). || `end_time` | No | Defaults to now (`now`, `1h ago`). |@@ -87,7 +100,17 @@ curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \| `format` | No | `json` (default), `csv`, `tsv`/`table`. CSV is lossless; TSV collapses newlines. || `fields` | No | Column allowlist of dotted paths, e.g. `type,content.app_name,content.text`. |-**Critical rules:** always include `start_time` (unbounded queries timeout) · "recent" = 30 min, "today" = since midnight, "yesterday" = yesterday's range · if `/search` is empty, fall back to `/activity-summary` and check `data_status` before saying "no data" · on timeout, narrow the range.+**Critical rules:** always include `start_time` (unbounded queries timeout) · always pass `fields=` with only the columns you need · always keep `limit` between 1 and 20 · always write the response to a file with `-o` and read it with `head`, never straight to stdout · "recent" = 30 min, "today" = since midnight, "yesterday" = yesterday's range · if `/search` is empty, fall back to `/activity-summary` and check `data_status` before saying "no data" · on timeout, narrow the range.++Single `content_type` means uniform rows, so add `format=csv` too:++```bash+curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \+ -H "X-Screenpipe-Client: api" \+ -o /tmp/sp.csv \+ "http://localhost:3030/search?content_type=ocr&limit=20&start_time=2h%20ago&format=csv&fields=content.timestamp,content.app_name,content.text"+head -20 /tmp/sp.csv+```**Tags** link people/projects/topics across screen, audio, and memories under one namespace (`person:ada`, `project:atlas`, `topic:pricing`). Add to a frame/audio: `POST /tags/vision/{frame_id}` or `POST /tags/audio/{chunk_id}` body `{"tags":["person:ada"]}`; to a memory: `tags` in `POST /memories`. Retrieve: `GET /search?tags=person:ada&start_time=30d%20ago` (add `content_type=memory` for memories). Frames are pruned by retention — tag a **memory** for durable links (memories carry `created_at` + a `frame_id` back to the moment). `include_related=true` returns co-occurring tags grouped by namespace, replacing 2-3 follow-up calls.diff --git a/crates/screenpipe-core/assets/skills/screenpipe-cli/SKILL.md b/crates/screenpipe-core/assets/skills/screenpipe-cli/SKILL.mdindex f0f92ec7c..17f5db508 100644--- a/crates/screenpipe-core/assets/skills/screenpipe-cli/SKILL.md+++ b/crates/screenpipe-core/assets/skills/screenpipe-cli/SKILL.md@@ -12,13 +12,17 @@ Use `bun x screenpipe@latest` to run CLI commands. No separate install needed.cd "$(mktemp -d)" && bun x screenpipe@latest <command>```+**Rules:** every invocation is `cd "$(mktemp -d)" && bun x screenpipe@latest …` · never drop the `cd` prefix · never drop `@latest` · copy the examples below verbatim rather than shortening them · because the `cd` changes your working directory, **any path you pass must be absolute** (`~/...` or `/...`), never relative (`./my-pipe`).++Works on macOS, Linux, and Windows: the CLI always runs under bash, and `mktemp` is present on all three (on Windows via the bundled git-portable `usr/bin`).+> **Sandboxed shells:** some agents (e.g. Codex) block all shell network access, so `bun x` cannot fetch the package and CLI calls to `localhost:3030` fail instantly. If that happens, use the screenpipe MCP tools instead of the CLI.## Shell- **All platforms** → `bash` (on Windows, the bundled git-portable bash is used automatically)-> **Note:** the bash tool truncates output around ~50 KB. Long listings (`connection list`, `pipe list`, etc.) are sorted with connected/enabled rows first, but if you need a specific row, pipe through `grep` or `head` rather than scanning the full output — e.g. `bun x screenpipe@latest connection list | grep -E 'browser|connected'`.+> **Note:** the bash tool truncates output around ~50 KB. Long listings (`connection list`, `pipe list`, etc.) are sorted with connected/enabled rows first, but if you need a specific row, pipe through `grep` or `head` rather than scanning the full output — e.g. `cd "$(mktemp -d)" && bun x screenpipe@latest connection list | grep -E 'browser|connected'`.---@@ -29,14 +33,14 @@ Pipes are markdown-based AI automations that run on schedule. Each pipe lives at### Commands```bash-bun x screenpipe@latest pipe list # List all pipes (compact table)-bun x screenpipe@latest pipe enable <name> # Enable a pipe-bun x screenpipe@latest pipe disable <name> # Disable a pipe-bun x screenpipe@latest pipe run <name> # Run once immediately (for testing)-bun x screenpipe@latest pipe logs <name> # View execution logs-bun x screenpipe@latest pipe install <url-or-path> # Install from GitHub or local path-bun x screenpipe@latest pipe delete <name> # Delete a pipe-bun x screenpipe@latest pipe models list # View AI model presets+cd "$(mktemp -d)" && bun x screenpipe@latest pipe list # List all pipes (compact table)+cd "$(mktemp -d)" && bun x screenpipe@latest pipe enable <name> # Enable a pipe+cd "$(mktemp -d)" && bun x screenpipe@latest pipe disable <name> # Disable a pipe+cd "$(mktemp -d)" && bun x screenpipe@latest pipe run <name> # Run once immediately (for testing)+cd "$(mktemp -d)" && bun x screenpipe@latest pipe logs <name> # View execution logs+cd "$(mktemp -d)" && bun x screenpipe@latest pipe install <url-or-abs-path> # Install from GitHub or an absolute local path+cd "$(mktemp -d)" && bun x screenpipe@latest pipe delete <name> # Delete a pipe+cd "$(mktemp -d)" && bun x screenpipe@latest pipe models list # View AI model presets```### Creating a Pipe@@ -87,9 +91,9 @@ Screenpipe prepends a context header with time range, timezone, OS, and API URLAfter creating:```bash-bun x screenpipe@latest pipe install ~/.screenpipe/pipes/my-pipe-bun x screenpipe@latest pipe enable my-pipe-bun x screenpipe@latest pipe run my-pipe # terminal-only; in-app chat uses the workflow below+cd "$(mktemp -d)" && bun x screenpipe@latest pipe install ~/.screenpipe/pipes/my-pipe+cd "$(mktemp -d)" && bun x screenpipe@latest pipe enable my-pipe+cd "$(mktemp -d)" && bun x screenpipe@latest pipe run my-pipe # terminal-only; in-app chat uses the workflow below```### Testing from in-app chat@@ -153,29 +157,29 @@ Manage integrations (Telegram, Slack, Discord, Email, Todoist, Teams) from the C### Commands```bash-bun x screenpipe@latest connection list # List all connections + status-bun x screenpipe@latest connection list --json # JSON output-bun x screenpipe@latest connection get <id> # Show status + non-secret settings-bun x screenpipe@latest connection get <id> --json # JSON output-bun x screenpipe@latest connection set <id> key=val # Save credentials-bun x screenpipe@latest connection test <id> # Test a connection-bun x screenpipe@latest connection remove <id> # Remove credentials+cd "$(mktemp -d)" && bun x screenpipe@latest connection list # List all connections + status+cd "$(mktemp -d)" && bun x screenpipe@latest connection list --json # JSON output+cd "$(mktemp -d)" && bun x screenpipe@latest connection get <id> # Show status + non-secret settings+cd "$(mktemp -d)" && bun x screenpipe@latest connection get <id> --json # JSON output+cd "$(mktemp -d)" && bun x screenpipe@latest connection set <id> key=val # Save credentials+cd "$(mktemp -d)" && bun x screenpipe@latest connection test <id> # Test a connection+cd "$(mktemp -d)" && bun x screenpipe@latest connection remove <id> # Remove credentials```### Examples```bash# Set up Telegram-bun x screenpipe@latest connection set telegram bot_token=123456:ABC-DEF chat_id=5776185278+cd "$(mktemp -d)" && bun x screenpipe@latest connection set telegram bot_token=123456:ABC-DEF chat_id=5776185278# Set up Slack webhook-bun x screenpipe@latest connection set slack webhook_url=https://hooks.slack.com/services/...+cd "$(mktemp -d)" && bun x screenpipe@latest connection set slack webhook_url=https://hooks.slack.com/services/...# Verify it works-bun x screenpipe@latest connection test telegram+cd "$(mktemp -d)" && bun x screenpipe@latest connection test telegram# Check what's connected-bun x screenpipe@latest connection list+cd "$(mktemp -d)" && bun x screenpipe@latest connection list```Connection IDs: `telegram`, `slack`, `discord`, `email`, `todoist`, `teams`, `google-calendar`, `openclaw`