High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
fix: reduce skill and tab automation in the benchmark
Mario501a4d0c8a8f
diff --git a/skills/pinchtab/SKILL.md b/skills/pinchtab/SKILL.mdindex 45bd27cc..e983231e 100644--- a/skills/pinchtab/SKILL.md+++ b/skills/pinchtab/SKILL.md@@ -31,29 +31,6 @@ Preferred tool surface:- Use `curl` for profile-management routes or non-shell/API fallback flows.- Use `jq` only when you need structured parsing from JSON responses.-## Agent Identity And Attribution--When multiple agents share one PinchTab server, always give each agent a stable ID.--- CLI flows: prefer `pinchtab --agent-id <agent-id> ...`-- long-running shells: set `PINCHTAB_AGENT_ID=<agent-id>`-- raw HTTP flows: send `X-Agent-Id: <agent-id>` on requests that should be attributed to that agent--That identity is recorded as `agentId` in activity events and powers:--- scheduler task attribution when work is dispatched on behalf of an agent--If you are switching between unrelated browser tasks, do not reuse the same agent ID unless you intentionally want one combined activity trail.--## Safety Defaults--- Default to `http://localhost` targets. Only use a remote PinchTab server when the user explicitly provides it and, if needed, a token.-- Prefer read-only operations first: `text`, `snap -i -c`, `snap -d`, `find`, `click`, `fill`, `type`, `press`, `select`, `hover`, `scroll`.-- Do not evaluate arbitrary JavaScript unless a simpler PinchTab command cannot answer the question.-- Do not upload local files unless the user explicitly names the file to upload and the destination flow requires it.-- Do not save screenshots, PDFs, or downloads to arbitrary paths. Use a user-specified path or a safe temporary/workspace path.-- Never use PinchTab to inspect unrelated local files, browser secrets, stored credentials, or system configuration outside the task.-## Core WorkflowEvery PinchTab automation follows this pattern:@@ -95,15 +72,7 @@ pinchtab fill "#email" "[email protected]" # CSSpinchtab fill e3 "[email protected]" # ref```-The same syntax works in the HTTP API via the `selector` field:--```json-{"kind": "click", "selector": "text:Sign In"}-{"kind": "fill", "selector": "#email", "text": "[email protected]"}-{"kind": "click", "selector": "e5"}-```--Legacy `ref` field is still accepted for backward compatibility.+Same syntax in HTTP API via `selector` field. Legacy `ref` field still accepted.## Command Chaining@@ -144,29 +113,21 @@ pinchtab nav <url> --print-tab-id # Print only the new tabId opinchtab back # Navigate back in historypinchtab forward # Navigate forwardpinchtab reload # Reload current page-pinchtab tab # List tabs (bare form; there is no `tab list` subcommand — `pinchtab tab list` returns 404)+pinchtab tab # List tabs (no subcommand - just `tab`)pinchtab tab <tab-id> # Focus an existing tabpinchtab tab new <url> # Open a new tabpinchtab tab close <tab-id> # Close a tab — use this to clean up stale tabs between runspinchtab instance navigate <instance-id> <url>```-When stdout is a pipe (e.g. inside `$(...)`), `nav` automatically switches to-`--print-tab-id` mode so agents can capture the tab ID with a single line,-then export it once via `PINCHTAB_TAB` so every subsequent tab-scoped command-picks it up without threading `--tab "$TAB"` through each call:-+**Tab workflow** - Capture tab ID once, reuse for all commands:```bash-# Capture once, reuse across every following command.export PINCHTAB_TAB=$(pinchtab nav http://example.com)-pinchtab snap -i -c-pinchtab eval "document.title"-pinchtab click '#submit'-pinchtab drag '#piece' '#zone-a'+pinchtab snap -i -c # Uses PINCHTAB_TAB automatically+pinchtab click e5 # Same tab+pinchtab text # Same tab```--An explicit `--tab <id>` on any command still overrides `PINCHTAB_TAB`.-See `references/env.md` for the full list of supported env vars.+Priority: `--tab <id>` flag > `PINCHTAB_TAB` env var > active tab.### Observation@@ -181,7 +142,7 @@ pinchtab snap --text # Text output formatpinchtab text # Page text content (Readability-filtered; drops nav/repeated headlines)pinchtab text --full # Full page text (document.body.innerText) — use when Readability is dropping content you needpinchtab text --raw # Alias of --full-# `text` has no `--format` / `--plain` flag — `--full` / `--raw` are the only mode switches. The CLI returns JSON `{url, title, text, truncated}`; pipe through `| jq -r .text` if you want just the body.+# CLI returns JSON; use `| jq -r .text` for plain textpinchtab find <query> # Semantic element searchpinchtab find --ref-only <query> # Return refs only```@@ -219,9 +180,9 @@ pinchtab drag <selector> --drag-x <n> --drag-y <n> # Single-step drag by pixelpinchtab type <selector> <text> # Type with keystrokespinchtab fill <selector> <text> # Set value directlypinchtab press <key> # Press key (Enter, Tab, Escape...)-pinchtab hover <selector> # Hover element — dispatches a `mousemove`, which normal event listeners and CSS `:hover` receive. Sites that wire hover via inline `onmouseenter="..."` attributes usually still respond; if they don't, fall back to `pinchtab eval` calling the handler directly (e.g. `pt eval "showInfo(2)"`).+pinchtab hover <selector> # Hover elementpinchtab select <selector> <value|text> # Select dropdown option by value attr, or fall back to visible text-pinchtab scroll <pixels|direction|selector> # Positional only — no `--y`/`--pixels`/`--delta-y` flag. Valid directions: up, down, left, right (800 px step). `top`/`bottom` are NOT direction keywords — scroll to a specific element (e.g. `'#footer'`) or pass an integer like `scroll 99999` to scroll far enough. Examples: `pinchtab scroll 1500`, `pinchtab scroll down`, `pinchtab scroll '#footer'`.+pinchtab scroll <pixels|direction|selector> # e.g. `scroll 1500`, `scroll down`, `scroll '#footer'````Rules:@@ -231,8 +192,8 @@ Rules:- Prefer `click --wait-nav` when a click is expected to navigate.- Prefer low-level `mouse` commands only when normal `click` / `hover` abstractions are insufficient, such as drag handles, canvas widgets, or sites that depend on exact pointer sequences.- Re-snapshot immediately after `click`, `press Enter`, `select`, or `scroll` if the UI can change.-- `select` takes whatever you have: it tries the `<option value="...">` attribute first, then exact (trimmed) visible text, then case-insensitive trimmed text, then case-insensitive substring of visible text (last resort, first match wins). So `pinchtab select '#country' uk`, `pinchtab select '#country' 'United Kingdom'`, and `pinchtab select '#theme' 'Dark'` (matches option "Dark Mode") all work; the form receives the real `value=...` attr in every case. If nothing matches, the server returns a clear error listing available options. Prefer exact forms when multiple options share a prefix.-- If a click opens a JS dialog (`alert`, `confirm`, `prompt`), pass `"dialogAction": "accept"` or `"dialogAction": "dismiss"` on the click action body — or `pinchtab click <selector> --dialog-action accept` from the CLI (use `--dialog-text <str>` to supply a prompt response). The dialog is auto-handled in a single call. Without this, the click hangs until `/tabs/TAB_ID/dialog` is called from a parallel request, and a pending dialog wedges subsequent `/snapshot` and `/text` calls.+- `select` matches by value attr first, then visible text (case-insensitive). Error lists available options if no match.+- For JS dialogs: use `--dialog-action accept` or `--dialog-action dismiss` on click. Add `--dialog-text` for prompt responses.- For the `scroll` action via HTTP, use `"scrollX"` / `"scrollY"` for pixel deltas, or `"selector"` to scroll an element into view. Example: `{"kind":"scroll","scrollY":1500}` or `{"kind":"scroll","selector":"#footer"}`. The `x`/`y` fields are target viewport coordinates, not scroll deltas.- The download HTTP endpoint (`GET /download?url=...` or `GET /tabs/TAB_ID/download?url=...`) returns JSON `{contentType, data (base64), size, url}`, not raw bytes. Decode `data` with base64 to get the file. Only `http`/`https` URLs are allowed. Private/internal hosts are blocked unless listed in `security.downloadAllowedDomains`.@@ -266,108 +227,26 @@ Rules:### HTTP API fallback-```bash-curl -X POST http://localhost:9868/navigate \- -H "Content-Type: application/json" \- -d '{"url":"https://example.com"}'--curl "http://localhost:9868/snapshot?filter=interactive&format=compact"--curl -X POST http://localhost:9868/action \- -H "Content-Type: application/json" \- -d '{"kind":"fill","selector":"e3","text":"[email protected]"}'--curl http://localhost:9868/text--## Instance-scoped solve (instance port, not server port)-curl -X POST http://localhost:9868/solve \- -H "Content-Type: application/json" \- -d '{"maxAttempts": 3}'--curl http://localhost:9868/solvers-```--Use the API when:--- the agent cannot shell out,-- profile creation or mutation is required,-- or you need explicit instance- and tab-scoped routes.+Use curl when CLI unavailable. Key endpoints on instance port (e.g. 9867):+- `POST /navigate` with `{"url":"..."}`+- `GET /snapshot?filter=interactive&format=compact`+- `POST /action` with `{"kind":"fill","selector":"e3","text":"..."}`+- `GET /text`+- `POST /solve` with `{"maxAttempts": 3}`### Tab-scoped HTTP API-**Important:** Each `POST /navigate` creates a new tab by default. The default (non-tab-scoped) endpoints like `/snapshot`, `/action`, `/text` operate on the *active* tab, which may not be the one you just navigated. In multi-tab workflows, always use tab-scoped routes to avoid acting on the wrong page.+Use `/tabs/TAB_ID/...` routes to target specific tabs. Get tab ID from navigate response or `GET /tabs`.-Get the tab ID from the navigate response or from `GET /tabs`.+Pattern: `curl -H "Authorization: Bearer <token>" http://localhost:9867/tabs/TAB_ID/<endpoint>`-```bash-# List all tabs-curl http://localhost:9867/tabs \- -H "Authorization: Bearer <token>"--# Navigate in a specific tab (does not create a new tab)-curl -X POST http://localhost:9867/tabs/TAB_ID/navigate \- -H "Authorization: Bearer <token>" \- -H "Content-Type: application/json" \- -d '{"url":"https://example.com"}'--# Snapshot a specific tab-curl "http://localhost:9867/tabs/TAB_ID/snapshot?filter=interactive&format=compact" \- -H "Authorization: Bearer <token>"--# Get text from a specific tab-curl http://localhost:9867/tabs/TAB_ID/text \- -H "Authorization: Bearer <token>"--# Perform action on a specific tab-curl -X POST http://localhost:9867/tabs/TAB_ID/action \- -H "Authorization: Bearer <token>" \- -H "Content-Type: application/json" \- -d '{"kind":"click","selector":"#submit-btn"}'--# Low-level pointer action on a specific tab-curl -X POST http://localhost:9867/tabs/TAB_ID/action \- -H "Authorization: Bearer <token>" \- -H "Content-Type: application/json" \- -d '{"kind":"mouse-wheel","x":400,"y":320,"deltaY":240}'--# Drag action: selector + pixel OFFSETS (dragX, dragY), not absolute endpoints.-# For low-level absolute coords, sequence mouse-down / mouse-move / mouse-up.-curl -X POST http://localhost:9867/tabs/TAB_ID/action \- -H "Authorization: Bearer <token>" \- -H "Content-Type: application/json" \- -d '{"kind":"drag","selector":"#piece","dragX":12,"dragY":-158}'--# Navigate back/forward in a specific tab-curl -X POST http://localhost:9867/tabs/TAB_ID/back \- -H "Authorization: Bearer <token>"-curl -X POST http://localhost:9867/tabs/TAB_ID/forward \- -H "Authorization: Bearer <token>"--# Screenshot (GET, not POST)-curl http://localhost:9867/tabs/TAB_ID/screenshot \- -H "Authorization: Bearer <token>" \- --output screenshot.png--# PDF export (GET or POST)-curl http://localhost:9867/tabs/TAB_ID/pdf \- -H "Authorization: Bearer <token>" \- --output page.pdf--# Close a tab-curl -X POST http://localhost:9867/tabs/TAB_ID/close \- -H "Authorization: Bearer <token>"--# Handoff for human step: POST /tabs/TAB_ID/handoff with {"reason":"captcha","timeoutMs":120000}-# Resume: POST /tabs/TAB_ID/resume with {"status":"completed"}-```--**Navigation with `waitNav`:** When clicking a link or button that triggers page navigation, include `"waitNav": true` in the action body. Without it, PinchTab returns a `navigation_changed` error to protect against unexpected navigation during form interactions.--```json-{"kind": "click", "selector": "#search-btn", "waitNav": true}-```+Key endpoints: `navigate`, `snapshot`, `text`, `action`, `screenshot`, `pdf`, `back`, `forward`, `close`, `wait`, `download`, `upload`, `handoff`, `resume`.-All tab-scoped routes follow the pattern `/tabs/{TAB_ID}/...` and mirror the default endpoints. The full list includes: `navigate`, `back`, `forward`, `reload`, `snapshot`, `screenshot`, `text`, `pdf`, `action`, `actions`, `dialog`, `wait`, `find`, `lock`, `unlock`, `cookies`, `metrics`, `network`, `solve`, `close`, `storage`, `evaluate`, `download`, `upload`, `handoff`, and `resume`.+Action examples:+- Click: `{"kind":"click","selector":"#btn"}`+- Click with nav: `{"kind":"click","selector":"#link","waitNav":true}`+- Drag: `{"kind":"drag","selector":"#piece","dragX":12,"dragY":-158}`+- Scroll: `{"kind":"scroll","scrollY":1500}` or `{"kind":"scroll","selector":"#footer"}`## Common Patterns@@ -377,13 +256,9 @@ All tab-scoped routes follow the pattern `/tabs/{TAB_ID}/...` and mirror the def**Form submission:** Always click the submit button — never use `press Enter`.-## Security and Token Economy+## Token Economy-- Use a dedicated automation profile, not a daily browsing profile.-- If PinchTab is reachable off-machine, require a token and bind conservatively.-- Prefer `text`, `snap -i -c`, and `snap -d` before screenshots, PDFs, eval, downloads, or uploads.-- Use `--block-images` for read-heavy tasks that do not need visual assets.-- Stop or isolate instances when switching between unrelated accounts or environments.+Prefer low-token commands: `text`, `snap -i -c`, `snap -d`. Use `--block-images` for read-heavy tasks. Reserve screenshots/PDFs for visual verification.## Diffing and Verificationdiff --git a/tests/benchmark/scripts/pt b/tests/benchmark/scripts/ptindex 3703d724..d32959ac 100755--- a/tests/benchmark/scripts/pt+++ b/tests/benchmark/scripts/pt@@ -1,49 +1,8 @@#!/usr/bin/env bash## Thin wrapper around `pinchtab` for the benchmark harness.-#-# Why this exists-# ----------------# In normal use, agents run the `pinchtab` binary directly:-#-# export PINCHTAB_TOKEN=<token>-# export PINCHTAB_TAB=$(pinchtab nav http://example.com)-# pinchtab snap -i -c-# pinchtab click '#submit'-#-# The benchmark runs PinchTab inside a Docker container-# (tests/benchmark/docker-compose.yml) rather than installed on the host, so-# every CLI invocation has to be prefixed with:-#-# docker exec -e PINCHTAB_TOKEN=... -e PINCHTAB_SERVER=... \-# -e PINCHTAB_TAB=... benchmark-pinchtab-1 pinchtab <args...>-#-# That ~140-character preamble on every command bloats the agent's context-# and makes copy-pasted examples unreadable. This script handles the-# preamble so benchmark callers can just write:-#-# TAB=$(./scripts/pt nav http://fixtures/)-# ./scripts/pt snap -i -c # auto-uses $PINCHTAB_TAB or /tmp/pt-tab-# ./scripts/pt click '#submit'-# ./scripts/pt eval --await-promise 'window.fetchPayload()'-#-# The CLI semantics are identical to running `pinchtab` directly.-#-# Tab persistence-# ----------------# Shell functions and env vars don't survive between Bash tool calls in-# Claude Code. To avoid re-typing `PINCHTAB_TAB=$TAB` on every command, the-# wrapper writes the tab ID returned by `pt nav` to `/tmp/pt-tab` and reads-# it back as the default for subsequent calls when $PINCHTAB_TAB isn't set.-# This is purely a benchmark-harness convenience — production `pinchtab`-# installs use the regular PINCHTAB_TAB env var.-#-# Env vars (override at call time if needed)-# -------------------------------------------# PINCHTAB_CONTAINER — container name (default: benchmark-pinchtab-1)-# PINCHTAB_TOKEN — auth token (default: benchmark-token)-# PINCHTAB_SERVER — server URL (default: http://localhost:9867)-# PINCHTAB_TAB — default tab ID; overrides the /tmp/pt-tab sidecar.+# Handles Docker exec prefix only - NO tab management.+# Agent must handle tabs via PINCHTAB_TAB env var per skill docs.set -euo pipefail@@ -51,19 +10,6 @@ set -euo pipefail: "${PINCHTAB_TOKEN:=benchmark-token}": "${PINCHTAB_SERVER:=http://localhost:9867}"-PT_TAB_SIDECAR="/tmp/pt-tab"--# Fall back to the sidecar file when PINCHTAB_TAB isn't set in the caller's-# environment. Trim trailing whitespace so downstream string matches work.-# Record whether the value came from the sidecar so the `nav` path can-# self-heal when the cached tab turns out to be dead (e.g. closed by a-# previous run).-pt_tab_from_sidecar=0-if [[ -z "${PINCHTAB_TAB:-}" && -f "${PT_TAB_SIDECAR}" ]]; then- PINCHTAB_TAB="$(tr -d '[:space:]' < "${PT_TAB_SIDECAR}")"- pt_tab_from_sidecar=1-fi-args=(docker exec-e "PINCHTAB_TOKEN=${PINCHTAB_TOKEN}"-e "PINCHTAB_SERVER=${PINCHTAB_SERVER}")@@ -74,69 +20,4 @@ fiargs+=("${PINCHTAB_CONTAINER}" pinchtab "$@")-# For `pt nav` with stdout piped (the $(pt nav URL) capture idiom), record-# the emitted tab ID into the sidecar so subsequent commands can pick it up-# without the caller having to `export PINCHTAB_TAB`. Keep the stdout-# contract unchanged — the tab ID still goes to stdout, exactly as before.-#-# Only update the sidecar when both (a) the underlying `pinchtab nav` call-# succeeded (exit 0) and (b) it emitted a well-formed 32-character hex tab-# ID. Otherwise the sidecar would point at a ghost tab and every later-# command would silently target nothing.-if [[ "${1:-}" == "nav" && ! -t 1 ]]; then- # Capture stdout and stderr separately so the caller still sees the tab- # ID on stdout, while we can inspect stderr for self-heal signals.- pt_err_file="$(mktemp -t pt-nav-err.XXXXXX)"- set +e- output="$("${args[@]}" 2>"${pt_err_file}")"- rc=$?- set -e- err_output="$(cat "${pt_err_file}" 2>/dev/null || true)"- rm -f "${pt_err_file}"-- # Self-heal: if nav failed AND the tab came from the sidecar (i.e. a stale- # ID from a previous run), clear it and retry once without the env-var so- # the server creates a fresh tab. Only trigger on errors that look like a- # dead-tab signal — keep genuine server errors (401, 403, etc.) unchanged.- if [[ ${rc} -ne 0 && ${pt_tab_from_sidecar} -eq 1 ]] \- && printf '%s\n%s' "${output}" "${err_output}" | grep -qiE 'tab.*not found|no such tab|tab .*does not exist'; then- rm -f "${PT_TAB_SIDECAR}"- # Rebuild the argv without the stale PINCHTAB_TAB env var.- args=(docker exec- -e "PINCHTAB_TOKEN=${PINCHTAB_TOKEN}"- -e "PINCHTAB_SERVER=${PINCHTAB_SERVER}"- "${PINCHTAB_CONTAINER}" pinchtab "$@")- pt_err_file="$(mktemp -t pt-nav-err.XXXXXX)"- set +e- output="$("${args[@]}" 2>"${pt_err_file}")"- rc=$?- set -e- err_output="$(cat "${pt_err_file}" 2>/dev/null || true)"- rm -f "${pt_err_file}"- fi-- if [[ -n "${err_output}" ]]; then- printf '%s\n' "${err_output}" >&2- fi- printf '%s\n' "${output}"- if [[ ${rc} -ne 0 ]]; then- exit ${rc}- fi- tab="$(printf '%s' "${output}" | grep -oE '^[A-F0-9]{32}$' | head -1)"- if [[ -n "${tab}" ]]; then- printf '%s' "${tab}" > "${PT_TAB_SIDECAR}"- fi- exit 0-fi--# When closing the tab the sidecar points at, clear the sidecar so the next-# command can't accidentally target a dead tab. Matches `pt tab close <id>`-# (cobra subcommand form).-if [[ "${1:-}" == "tab" && "${2:-}" == "close" && -n "${3:-}" && -f "${PT_TAB_SIDECAR}" ]]; then- stored="$(tr -d '[:space:]' < "${PT_TAB_SIDECAR}")"- if [[ "${stored}" == "${3}" ]]; then- rm -f "${PT_TAB_SIDECAR}"- fi-fi-exec "${args[@]}"