High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
feat(skill): add commands, agent-optimization, and mcp reference files (#347)
Alice Losassobfa599f2d3ac
diff --git a/skills/pinchtab/references/agent-optimization.md b/skills/pinchtab/references/agent-optimization.mdnew file mode 100644index 00000000..c143a0fd--- /dev/null+++ b/skills/pinchtab/references/agent-optimization.md@@ -0,0 +1,167 @@+# Agent Optimization Playbook++Practical guidance for running token-efficient, resilient PinchTab agent workflows.++---++## Cheapest-Path Decision Tree++Choose the lowest-cost tool that satisfies your goal:++```+Need to check page state?+├─ Know the element ref already? → skip snap, use click/type directly+├─ Need to find interactive elements? → snap -i -c (cheapest)+├─ Need to read text/data only? → pinchtab text (no tree overhead)+├─ Need to find a specific element? → pinchtab find "<text>"+├─ Need full page structure? → snap -c (compact tree)+├─ Need to debug visually? → screenshot (use sparingly, large output)+└─ Need to run a JS check? → eval (precise, zero visual overhead)+```++**Token cost ranking (cheapest → most expensive):**+1. `eval` — single value, no DOM traversal output+2. `find` — targeted element list only+3. `text` — readable text only+4. `snap -i -c` — interactive elements, compact format+5. `snap -c` — full tree, compact+6. `snap -i` — interactive elements, verbose+7. `snap` — full tree, verbose+8. `screenshot` — image payload, highest token cost++**Rule of thumb:** Reach for `snap -i -c` as your default snapshot. Only escalate to `screenshot` when visual layout matters (CAPTCHA, canvas, complex CSS).++---++## Diff Snapshots for Follow-Up Reads++After an interaction, use `snap -d` to see only what changed — not the full tree.++```bash+pinchtab click e5 # trigger action+pinchtab snap -d # only the delta — much smaller+```++**When to use `-d`:**+- After clicks that update part of the UI (e.g. accordion opens, toast appears)+- After form submissions that show inline validation+- During multi-step wizards where only one section changes++**When NOT to use `-d`:**+- After full page navigations (diff will be the entire new page)+- After `nav` — always take a fresh `snap` instead+- First snapshot of a session (no baseline exists)++---++## Lite Engine++Start PinchTab with `--engine lite` for minimal rendering overhead.++```bash+pinchtab start --engine lite+```++**Lite engine capabilities:**+- Faster page loads (no CSS animations, reduced JS execution)+- Lower memory footprint — useful for multi-tab fleet workflows+- Accessibility tree (`snap`) works fully+- `text`, `find`, `eval` all work as normal++**Lite engine limitations:**+- `screenshot` output may not reflect full visual styling+- Pages that depend on CSS transitions for state changes may behave differently+- Some canvas/WebGL content will not render+- Not suitable for visual regression testing++**Best for:** Form automation, data extraction, API-heavy SPAs, scraping workflows where visual fidelity is not required.++---++## Recovery Patterns++### 403 Forbidden+**Cause:** `eval` called without `security.allowEvaluate: true`, or a page blocked the request.++**Recovery:**+```bash+# Option 1: enable eval in config, restart server+# Option 2: switch to snap + find instead of eval+pinchtab find "target text" # avoids eval entirely+```++---++### 401 Unauthorized+**Cause:** Session expired, auth cookie gone, or protected resource.++**Recovery:**+1. `pinchtab screenshot` — confirm login page is showing+2. Re-authenticate: `pinchtab nav <login-url>`, then fill credentials+3. If using a profile: `pinchtab profile use <name>` may restore the session++---++### Connection Refused+**Cause:** PinchTab server is not running or crashed.++**Recovery:**+```bash+pinchtab health # confirm down+pinchtab start # restart+pinchtab health # confirm up before continuing+```++For fleet workflows: check `pinchtab instances` to confirm the right instance is running.++---++### Stale Element Refs+**Cause:** A `snap` was taken, then the page re-rendered (navigation, dynamic update). Old refs (`e5`, `e12`) are no longer valid.++**Symptoms:** Interaction returns "ref not found" or acts on the wrong element.++**Recovery:**+```bash+pinchtab snap -i -c # fresh snapshot → new refs+# Now use the new refs from this response+```++**Prevention:** Never cache refs across navigations. Always re-snap after a page load or major DOM update.++---++### Bot Detection / CAPTCHA+**Cause:** Target site detected automated behavior.++**Recovery options:**+1. `pinchtab screenshot` — confirm what's blocking+2. Slow down: add `pinchtab wait --ms 1500` between interactions+3. Avoid rapid sequential clicks — space them out+4. Switch to a profile with existing session cookies+5. If CAPTCHA: manual intervention required — report to user++---++### Timeout on Navigation+**Cause:** Page load exceeded default timeout (usually 30s).++**Recovery:**+```bash+pinchtab nav <url> --timeout 90 # extend timeout+```++If the page consistently times out, consider `--block-images` to speed up load:+```bash+pinchtab nav <url> --block-images --timeout 60+```++---++## General Efficiency Rules++- **Batch reads before writes.** Snap once, extract all refs, then act. Avoid snap → act → snap → act loops when you can snap → act × N → snap once to verify.+- **Use `text` for extraction tasks.** If you only need to read content (not interact), `text` is cheaper than `snap` + parsing.+- **Scope snapshots.** Use `snap -s <selector>` to target a specific section of the page when you know where the element is.+- **Prefer `fill` over `type` for framework forms.** Saves retries caused by React/Vue not detecting raw keystroke events.+- **Check health before long workflows.** Run `pinchtab health` at the start of a multi-step task to fail fast if the server is down.diff --git a/skills/pinchtab/references/commands.md b/skills/pinchtab/references/commands.mdnew file mode 100644index 00000000..a4082af7--- /dev/null+++ b/skills/pinchtab/references/commands.md@@ -0,0 +1,193 @@+# CLI Commands Reference — Pinchtab 0.8.x++> **Quick tip:** Use `pinchtab help` or `pinchtab <command> --help` for full flag lists.++---++## Control Plane++### `pinchtab start`+Start the PinchTab server (default port 9867).++```bash+pinchtab start+pinchtab start --port 9868+pinchtab start --profile work --headless+```++### `pinchtab stop`+Stop the running server.++### `pinchtab status` / `pinchtab health`+Check if the server is running and healthy.++---++## Browser Commands++### `pinchtab nav <url>`+Navigate the current tab to a URL.++```bash+pinchtab nav https://example.com+pinchtab nav https://example.com --new-tab+pinchtab nav https://example.com --block-images+pinchtab nav https://example.com --timeout 60+```++| Flag | Description |+|------|-------------|+| `--new-tab` | Open in a new tab instead of current |+| `--block-images` | Block image loading (faster, fewer tokens) |+| `--timeout <s>` | Navigation timeout in seconds |+| `--profile <name>` | Target a named profile |++> ⚠️ **Quirk:** Use `--profile`, not `--profileId`. The long-form flag is required.++### `pinchtab tab` (not `tabs`)+Manage browser tabs.++```bash+pinchtab tab list # List all open tabs+pinchtab tab close # Close current tab+pinchtab tab close <tabId> # Close specific tab+```++> ⚠️ **Quirk:** The command is `tab` (singular), not `tabs`.++---++## Interaction Commands++### `pinchtab click <ref>`+Click an element by its accessibility ref (from `snap`).++```bash+pinchtab click e5+pinchtab click e5 --tab <tabId>+```++### `pinchtab type <ref> <text>`+Type text into an input element.++```bash+pinchtab type e12 "hello world"+```++### `pinchtab fill <ref> <value>`+Fill a form field using JS event dispatch. Prefer over `type` for React/Vue/Angular forms.++```bash+pinchtab fill e12 "hello world"+```++### `pinchtab press <key>`+Press a named keyboard key.++```bash+pinchtab press Enter+pinchtab press Tab+pinchtab press Escape+```++### `pinchtab hover <ref>`+Hover over an element to trigger tooltips or hover styles.++### `pinchtab scroll [ref]`+Scroll the page or a specific element.++```bash+pinchtab scroll # scroll page down 300px+pinchtab scroll --pixels -300 # scroll up+pinchtab scroll e20 --pixels 500+```++### `pinchtab select <ref> <value>`+Select an option from a `<select>` dropdown.++```bash+pinchtab select e8 "option-value"+```++---++## Output Commands++### `pinchtab snap` (snapshot)+Get the accessibility tree of the current page. **Primary tool for understanding page state.**++```bash+pinchtab snap # full tree+pinchtab snap -i # interactive elements only (smaller)+pinchtab snap -c # compact format (fewer tokens)+pinchtab snap -i -c # both: cheapest snapshot+pinchtab snap -d # diff: only changes since last snap+pinchtab snap -s main # scoped to CSS selector+pinchtab snap --max-tokens 2000 # token budget cap+```++> ⚠️ **Quirk:** Use `snap`, not `snapshot`. The alias `snap` is the intended short form.++### `pinchtab screenshot`+Capture a screenshot of the current page.++```bash+pinchtab screenshot+pinchtab screenshot --quality 80 # JPEG at 80%+```++> ⚠️ **Quirk:** Use `screenshot` (full word), not `ss` or `shot`.++### `pinchtab text`+Extract readable text from the page.++```bash+pinchtab text+pinchtab text --raw # no formatting cleanup+```++### `pinchtab find <query>`+Find elements by text content or CSS selector.++```bash+pinchtab find "Submit"+pinchtab find ".btn-primary"+```++### `pinchtab eval <expression>`+Run JavaScript in the browser context.++```bash+pinchtab eval "document.title"+pinchtab eval "document.querySelectorAll('a').length"+```++> Requires `security.allowEvaluate: true` in config. Returns 403 by default.++---++## Fleet / Multi-Profile Commands++### `pinchtab profile list`+List all available profiles.++### `pinchtab profile use <name>`+Switch the active profile.++```bash+pinchtab profile use work+```++### `pinchtab instances`+List running PinchTab instances across profiles.++---++## Known Quirks Summary++| Wrong | Right | Note |+|-------|-------|------|+| `pinchtab ss` | `pinchtab screenshot` | No `ss` alias |+| `pinchtab snapshot` | `pinchtab snap` | Use short form |+| `--profileId` | `--profile` | Long-form flag name |+| `pinchtab tabs` | `pinchtab tab` | Singular subcommand |diff --git a/skills/pinchtab/references/mcp.md b/skills/pinchtab/references/mcp.mdnew file mode 100644index 00000000..2bdee7d4--- /dev/null+++ b/skills/pinchtab/references/mcp.md@@ -0,0 +1,131 @@+# MCP Server Reference++PinchTab exposes a Model Context Protocol (MCP) server over **stdio JSON-RPC 2.0** (MCP spec 2025-11-25). This lets AI agents (Claude, GPT-4o, etc.) control a browser directly through their tool-calling interface.++---++## Configuration++Add PinchTab to your MCP client config:++```json+{+ "mcpServers": {+ "pinchtab": {+ "command": "pinchtab",+ "args": ["mcp"]+ }+ }+}+```++For Claude Desktop (`~/Library/Application Support/Claude/claude_desktop_config.json`):++```json+{+ "mcpServers": {+ "pinchtab": {+ "command": "pinchtab",+ "args": ["mcp"],+ "env": {+ "PINCHTAB_PORT": "9867"+ }+ }+ }+}+```++PinchTab must be running (`pinchtab start`) before the MCP server can proxy requests. The MCP server communicates with the PinchTab HTTP API at `localhost:9867` by default.++---++## Available Tools (21 total)++All tool names are prefixed with `pinchtab_`.++### Navigation+| Tool | Description |+|------|-------------|+| `pinchtab_navigate` | Navigate to a URL. Required param: `url`. Optional: `tabId`. |+| `pinchtab_health` | Check server health. No params. |++### Page Reading+| Tool | Description |+|------|-------------|+| `pinchtab_snapshot` | Accessibility tree. Optional: `interactive`, `compact`, `diff`, `selector`, `tabId`. |+| `pinchtab_screenshot` | Capture screenshot. Optional: `quality`, `tabId`. Returns base64 image. |+| `pinchtab_get_text` | Extract readable page text. Optional: `raw`, `tabId`. |+| `pinchtab_find` | Find elements by text or CSS selector. Required: `query`. Optional: `tabId`. |+| `pinchtab_eval` | Execute JavaScript. Required: `expression`. Optional: `tabId`. Needs `security.allowEvaluate: true`. |+| `pinchtab_pdf` | Export page as PDF. Optional: `landscape`, `scale`, `pageRanges`, `tabId`. Returns base64 PDF. |+| `pinchtab_cookies` | Get cookies for current page. Optional: `tabId`. |++### Interaction+| Tool | Description |+|------|-------------|+| `pinchtab_click` | Click element by ref. Required: `ref`. Optional: `tabId`. |+| `pinchtab_type` | Type text keystroke-by-keystroke. Required: `ref`, `text`. Optional: `tabId`. |+| `pinchtab_fill` | Fill input via JS dispatch (React/Vue/Angular safe). Required: `ref`, `value`. Optional: `tabId`. |+| `pinchtab_press` | Press a named key (`Enter`, `Tab`, `Escape`, etc.). Required: `key`. Optional: `tabId`. |+| `pinchtab_hover` | Hover over element. Required: `ref`. Optional: `tabId`. |+| `pinchtab_focus` | Focus an element. Required: `ref`. Optional: `tabId`. |+| `pinchtab_select` | Select dropdown option. Required: `ref`, `value`. Optional: `tabId`. |+| `pinchtab_scroll` | Scroll page or element. Optional: `ref`, `pixels`, `tabId`. |++### Tab Management+| Tool | Description |+|------|-------------|+| `pinchtab_list_tabs` | List all open tabs. No params. |+| `pinchtab_close_tab` | Close a tab. Optional: `tabId` (closes current if omitted). |++### Utility+| Tool | Description |+|------|-------------|+| `pinchtab_wait` | Wait N milliseconds. Required: `ms` (max 30000). |+| `pinchtab_wait_for_selector` | Wait for CSS selector to appear. Required: `selector`. Optional: `timeout`, `tabId`. |++---++## Element Refs++`pinchtab_snapshot` returns an accessibility tree with element refs like `e5`, `e12`. These refs are required by all interaction tools (`click`, `type`, `fill`, `hover`, etc.).++**Important:** Refs are ephemeral. They expire after navigation or significant DOM updates. Always re-call `pinchtab_snapshot` after a page load before using refs in interactions.++---++## What MCP Cannot Do++The MCP surface is intentionally scoped to browser automation. The following are **not available** via MCP tools:++| Capability | Status | Alternative |+|------------|--------|-------------|+| Create/edit/delete profiles | ❌ Not available | Use `pinchtab profile` CLI or HTTP API |+| Configure the scheduler | ❌ Not available | Use `pinchtab schedule` CLI |+| Modify stealth / fingerprint settings | ❌ Not available | Edit config file directly |+| Start or stop the PinchTab server | ❌ Not available | Use `pinchtab start` / `pinchtab stop` CLI |+| Manage fleet instances | ❌ Not available | Use `pinchtab instances` CLI |+| Read/write PinchTab config | ❌ Not available | Edit `~/.pinchtab/config.yaml` directly |++If you need these capabilities in an agent workflow, use the CLI commands alongside the MCP tools, or call the PinchTab HTTP API directly.++---++## Error Handling++MCP tools surface errors as tool errors (not protocol-level errors). Common cases:++| Error | Cause | Fix |+|-------|-------|-----|+| Connection refused | PinchTab not running | `pinchtab start` |+| `ref not found` | Stale element ref | Re-run `pinchtab_snapshot` |+| `evaluate not allowed` (403) | `security.allowEvaluate` is false | Enable in config or use `find`/`snap` instead |+| `invalid URL` | Missing `http://` or `https://` | Include full scheme in URL |++---++## Related++- [MCP Tools Full Parameter Reference](../../docs/reference/mcp-tools.md)+- [API Reference](api.md)+- [Agent Optimization Playbook](agent-optimization.md)