Skip to content

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

docs: update of the main readme and exposed docs to the website

Luigi Agosti8f727ceab56c
diff --git a/README.md b/README.md
index 00cb8c71..2cbaeade 100644
--- a/README.md
+++ b/README.md
@@ -329,8 +329,9 @@ read as "PinchTab is N% cheaper than agent-browser on this metric":
The cost gap widens with workload length (click→snapshot round trips
compound) and is roughly model-invariant at extended scope.
-See the full methodology, per-run tables, and raw logs in the
-[benchmark deep dive](./docs/deep-dive/benchmark.md).
+See the [benchmark summary](./docs/benchmark.md) for an overview, or the
+[benchmark deep dive](./docs/deep-dive/benchmark.md) for full
+methodology, per-run tables, and raw logs.
---
diff --git a/docs/benchmark.md b/docs/benchmark.md
new file mode 100644
index 00000000..28b5ed5a
--- /dev/null
+++ b/docs/benchmark.md
@@ -0,0 +1,102 @@
+# Benchmark
+
+This page summarises how PinchTab compares to `agent-browser` on
+real agent-loop token cost. Full methodology, per-run tables, and raw
+transcripts live in the [benchmark deep dive](./deep-dive/benchmark.md).
+
+## Headline result
+
+PinchTab is cheaper and uses fewer API round trips than agent-browser on
+every scope we measured. Percentages below read as "PinchTab is N%
+cheaper than agent-browser on this metric":
+
+| Scope | n per lane | Cost cheaper | Fewer requests | Fewer tokens |
+|----------------------------|-----------:|-------------:|---------------:|-------------:|
+| Basic Haiku 4.5 (10 steps) | 5 | **9.5%** | 23.0% | 17.9% |
+| Extended Haiku 4.5 (24 steps) | 3 | **19.6%** | 31.1% | 26.2% |
+| Extended Sonnet 4.6 (24 steps) | 2 | **20.3%** | 29.4% | 25.3% |
+
+Absolute per-run costs:
+
+| Scope | PinchTab avg | agent-browser avg |
+|----------------------------|-------------:|------------------:|
+| Basic Haiku 4.5 | $0.1024 | $0.1132 |
+| Extended Haiku 4.5 | $0.3516 | $0.4372 |
+| Extended Sonnet 4.6 | $0.8932 | $1.1204 |
+
+## What's being measured
+
+The number is the **end-to-end token cost of the whole agent loop** —
+system prompt, skill, tool calls, tool outputs, model reasoning, and
+retries — summed over one complete benchmark run. Usage is read directly
+off Anthropic's `usage` object per response; no self-reporting by the
+model.
+
+Both lanes run the same task set inside the same Docker Compose
+environment, against the same benchmark fixture server, driven by the
+same Go runner. The only thing that changes between lanes is the browser
+surface the agent talks to and the matching skill that teaches it the
+command shapes.
+
+## Why PinchTab wins on cost
+
+Two structural differences drive the gap:
+
+1. **Fewer API round trips.** agent-browser follows a click-then-snapshot
+ pattern: every mutation step costs two API calls. PinchTab batches the
+ action and the resulting snapshot into one round trip via
+ `--snap`/`--snap-diff`, so the same step costs one API call.
+2. **Less repeated cache-read.** Those extra round trips on agent-browser
+ don't just cost one turn each — they also re-read the cached system
+ prompt and skill on every turn. Over a 24-step run the extra
+ cache-read tokens dominate the token gap (though not the cost gap,
+ since cache reads are only 10% of uncached input pricing).
+
+## How the gap scales
+
+- **Scope:** the gap widens with step count (9.5% on 10 steps, 19.6% on
+ 24 steps). Every extra step that involves a post-action snapshot adds
+ another round trip on agent-browser.
+- **Model:** the gap is essentially identical on Haiku 4.5 and Sonnet 4.6
+ at extended scope (19.6% vs 20.3%). Stronger reasoning doesn't collapse
+ the click→snapshot pattern — the extra round trips are a property of
+ the tool surface, not a planning failure the model corrects.
+
+## Caveats
+
+- The 10-step task suite was designed alongside PinchTab's development
+ and contains tasks that are awkward-multi-call on agent-browser. A
+ co-designed or much larger task set would reduce task-suite bias.
+- Both lanes run a trimmed subset of their full skills (header + the one
+ reference file the agent actually reaches for) to keep the comparison
+ focused on the tool surface rather than documentation weight. A
+ production re-run with full skills on both sides would give a
+ different number.
+- Variance is ~25–30% of mean at the per-run level; n=5 basic / n=3
+ extended Haiku / n=2 extended Sonnet give a usable central tendency
+ but wide confidence intervals, especially for the Sonnet pair.
+- Agent-browser has one outlier per extended run (lae3); excluding it
+ would narrow the gap meaningfully.
+
+## Reproducing
+
+```bash
+cd tests/benchmark
+docker compose up -d --build
+./scripts/run-optimization.sh
+
+# Baseline (deterministic, ~30s)
+./scripts/baseline.sh
+
+# PinchTab and agent-browser lanes (Anthropic API key required)
+ANTHROPIC_API_KEY=... ./scripts/run-api-benchmark.ts --lane pinchtab --groups 0,1
+ANTHROPIC_API_KEY=... ./scripts/run-api-benchmark.ts --lane agent-browser --groups 0,1
+
+# Inspect usage
+jq '.run_usage' results/pinchtab_benchmark_*.json
+jq '.run_usage' results/agent_browser_benchmark_*.json
+```
+
+See the [benchmark deep dive](./deep-dive/benchmark.md) for per-run
+tables, raw transcripts, token breakdowns, variance discussion, and
+the full list of measurement caveats.
diff --git a/docs/index.json b/docs/index.json
index 0810692f..13dbf555 100644
--- a/docs/index.json
+++ b/docs/index.json
@@ -6,7 +6,8 @@
"core-concepts.md",
"headless-vs-headed.md",
"dashboard.md",
- "mcp.md"
+ "mcp.md",
+ "benchmark.md"
],
"guides": [
"guides/contributing.md",
@@ -27,8 +28,5 @@
],
"reference": [
"*"
- ],
- "implementations": [
- "*"
]
}