High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
dev bench agent-browser --max-turns 100 --verbose --groups 0,1
warning: multiple providers configured, defaulting to anthropic
Force-recreating agent-browser and fixtures...
Compose now can delegate build to bake for better performances
Just set COMPOSE_BAKE=true
[+] Building 0.5s (9/9) FINISHED docker:desktop-linux
=> [agent-browser internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 573B 0.0s
=> [agent-browser internal] load metadata for docker.io/library/node:20-bookworm 0.4s
=> [agent-browser internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [agent-browser 1/4] FROM docker.io/library/node:20-bookworm@sha256:8789e1e0752d81088a085689c04f 0.0s
=> => resolve docker.io/library/node:20-bookworm@sha256:8789e1e0752d81088a085689c04fdb7a5b16e8102e 0.0s
=> CACHED [agent-browser 2/4] RUN apt-get update && apt-get install -y --no-install-recommends 0.0s
=> CACHED [agent-browser 3/4] RUN npm install -g agent-browser 0.0s
=> CACHED [agent-browser 4/4] WORKDIR /workspace/tests/benchmark 0.0s
=> [agent-browser] exporting to image 0.0s
=> => exporting layers 0.0s
=> => exporting manifest sha256:5da4df76bc16d03b53cfdd7e5916ad6f07e39c7567b57d4575b73095e5be8a64 0.0s
=> => exporting config sha256:0005103a0fcf7270a11bc88db271999149c096a465817b178af786c87a3c52b8 0.0s
=> => exporting attestation manifest sha256:ba6b1ac1b6085d8a3e28b878f5eca27292f01e643f5149944ba200 0.0s
=> => exporting manifest list sha256:bf9e36e47806642a7445966ecf756fb3a80f55c15e86dce52c3f5c3a41ce1 0.0s
=> => naming to docker.io/library/tools-agent-browser:latest 0.0s
=> => unpacking to docker.io/library/tools-agent-browser:latest 0.0s
=> [agent-browser] resolving provenance for metadata file 0.0s
[+] Running 3/3
✔ agent-browser Built 0.0s
✔ Container tools-agent-browser-1 Started 3.0s
✔ Container tools-fixtures-1 Healthy 2.9s
Verified: agent-browser up and fixtures reachable
=== PinchTab Benchmark Run #1 ===
Timestamp: 20260420_134029
Report: <repo>/tests/benchmark/results/agent_browser_benchmark_20260420_134029.json
[benchmark-runner] provider=anthropic model=claude-haiku-4-5-20251001 lane=agent-browser groups=0,1 report=<repo>/tests/benchmark/results/agent_browser_benchmark_20260420_134029.json
══ Benchmark start ══
lane: agent-browser
model: claude-haiku-4-5-20251001
report: <repo>/tests/benchmark/results/agent_browser_benchmark_20260420_134029.json
Group 0 — 5 step(s):
0.1 Navigate to page
0.2 Take snapshot
0.3 Read page text
0.4 Click a link
0.5 Extract specific data
Group 1 — 5 step(s):
1.1 Fill a text input
1.2 Click a button
1.3 Use search functionality
1.4 Handle dynamic content
1.5 Navigate back
Total: 10 step(s) across 2 group(s)
─── Turn 1/100 ───
← anthropic/claude-haiku-4-5-20251001: 2 in, 0 cc, 6047 cr, 119 out
▶ ./scripts/ab open http://fixtures/
✓ ✓ PinchTab Benchmark - Home
http://fixtures/ [516ms]
─── Turn 2/100 ───
← anthropic/claude-haiku-4-5-20251001: 174 in, 0 cc, 6047 cr, 100 out
▶ ./scripts/ab snapshot -i -c
✓ - heading "PinchTab Benchmark Suite" [level=1, ref=e1]
- link "Articles" [ref=e2] (+11 lines) [340ms]
─── Turn 3/100 ───
← anthropic/claude-haiku-4-5-20251001: 496 in, 0 cc, 6047 cr, 140 out
▶ ./scripts/runner step-end 0 1 answer "Page loaded successfully with title 'Pi...
✓ Recorded 0.1 answer
Verified: Step 0.1 = pass [1.661s]
─── Turn 4/100 ───
← anthropic/claude-haiku-4-5-20251001: 727 in, 0 cc, 6047 cr, 142 out
▶ ./scripts/runner step-end 0 2 answer "Snapshot captured with 13 interactive e...
✓ Recorded 0.2 answer
Verified: Step 0.2 = pass [1.667s]
─── Turn 5/100 ───
← anthropic/claude-haiku-4-5-20251001: 970 in, 0 cc, 6047 cr, 109 out
▶ ./scripts/ab get text @e7
✓ Welcome to the Benchmark Test Suite [334ms]
─── Turn 6/100 ───
← anthropic/claude-haiku-4-5-20251001: 1122 in, 0 cc, 6047 cr, 123 out
▶ ./scripts/runner step-end 0 3 answer "Text content extracted: 'Welcome to the...
✓ Recorded 0.3 answer
Verified: Step 0.3 = pass [1.68s]
─── Turn 7/100 ───
← anthropic/claude-haiku-4-5-20251001: 1328 in, 0 cc, 6047 cr, 107 out
▶ ./scripts/ab click @e2
✓ ✓ Done [328ms]
─── Turn 8/100 ───
← anthropic/claude-haiku-4-5-20251001: 1218 in, 0 cc, 6047 cr, 97 out
▶ ./scripts/ab snapshot -i -c
✓ - link "Home" [ref=e2]
- link "Articles" [ref=e3] (+16 lines) [329ms]
─── Turn 9/100 ───
← anthropic/claude-haiku-4-5-20251001: 1649 in, 0 cc, 6047 cr, 149 out
▶ ./scripts/runner step-end 0 4 answer "Clicked Articles link and navigated to ...
✓ Recorded 0.4 answer
Verified: Step 0.4 = pass [1.661s]
─── Turn 10/100 ───
← anthropic/claude-haiku-4-5-20251001: 1471 in, 0 cc, 6047 cr, 109 out
▶ ./scripts/ab get text @e4
✓ The Future of Artificial Intelligence [324ms]
─── Turn 11/100 ───
← anthropic/claude-haiku-4-5-20251001: 1622 in, 0 cc, 6047 cr, 126 out
▶ ./scripts/runner step-end 0 5 answer "Extracted text from article heading: 'T...
✓ Recorded 0.5 answer
Verified: Step 0.5 = pass [1.665s]
─── Turn 12/100 ───
← anthropic/claude-haiku-4-5-20251001: 1507 in, 0 cc, 6047 cr, 116 out
▶ ./scripts/ab open http://fixtures/form.html
✓ ✓ Form - PinchTab Benchmark
http://fixtures/form.html [343ms]
─── Turn 13/100 ───
← anthropic/claude-haiku-4-5-20251001: 1682 in, 0 cc, 6047 cr, 96 out
▶ ./scripts/ab snapshot -i -c
✓ - link "Home" [ref=e2]
- link "Form" [ref=e3] (+25 lines) [308ms]
─── Turn 14/100 ───
← anthropic/claude-haiku-4-5-20251001: 1651 in, 0 cc, 6047 cr, 114 out
▶ ./scripts/ab fill @e4 "John Doe"
✓ ✓ Done [322ms]
─── Turn 15/100 ───
← anthropic/claude-haiku-4-5-20251001: 1810 in, 0 cc, 6047 cr, 94 out
▶ ./scripts/ab get value @e4
✓ John Doe [349ms]
─── Turn 16/100 ───
← anthropic/claude-haiku-4-5-20251001: 1542 in, 0 cc, 6047 cr, 121 out
▶ ./scripts/runner step-end 1 1 answer "Filled Full Name field with 'John Doe'"...
✓ Recorded 1.1 answer
Verified: Step 1.1 = pass [1.662s]
─── Turn 17/100 ───
← anthropic/claude-haiku-4-5-20251001: 1745 in, 0 cc, 6047 cr, 105 out
▶ ./scripts/ab click @e14
✓ ✓ Done [327ms]
─── Turn 18/100 ───
← anthropic/claude-haiku-4-5-20251001: 1529 in, 0 cc, 6047 cr, 99 out
▶ ./scripts/ab snapshot -i -c
✓ - link "Home" [ref=e2]
- link "Form" [ref=e3] (+25 lines) [363ms]
─── Turn 19/100 ───
← anthropic/claude-haiku-4-5-20251001: 2081 in, 0 cc, 6047 cr, 109 out
▶ ./scripts/ab get text @e1
✓ Contact Form — VERIFY_FORM_PAGE_22222 [345ms]
─── Turn 20/100 ───
← anthropic/claude-haiku-4-5-20251001: 1534 in, 0 cc, 6047 cr, 161 out
▶ ./scripts/runner step-end 1 2 answer "Clicked Submit Form button successfully...
✓ Recorded 1.2 answer
Verified: Step 1.2 = pass [1.662s]
─── Turn 21/100 ───
← anthropic/claude-haiku-4-5-20251001: 1771 in, 0 cc, 6047 cr, 109 out
▶ ./scripts/ab open http://fixtures/search.html
✓ ✓ Search - PinchTab Benchmark
http://fixtures/search.html [368ms]
─── Turn 22/100 ───
← anthropic/claude-haiku-4-5-20251001: 1624 in, 0 cc, 6047 cr, 100 out
▶ ./scripts/ab snapshot -i -c
✓ - link "Home" [ref=e2]
- link "Search" [ref=e3] (+3 lines) [370ms]
─── Turn 23/100 ───
← anthropic/claude-haiku-4-5-20251001: 1836 in, 0 cc, 6047 cr, 111 out
▶ ./scripts/ab fill @e4 "artificial intelligence"
✓ ✓ Done [368ms]
─── Turn 24/100 ───
← anthropic/claude-haiku-4-5-20251001: 1295 in, 0 cc, 6047 cr, 93 out
▶ ./scripts/ab click @e5
✓ ✓ Done [383ms]
─── Turn 25/100 ───
← anthropic/claude-haiku-4-5-20251001: 1427 in, 0 cc, 6047 cr, 97 out
▶ ./scripts/ab snapshot -i -c
✓ - link "Home" [ref=e2]
- link "Search" [ref=e3] (+5 lines) [359ms]
─── Turn 26/100 ───
← anthropic/claude-haiku-4-5-20251001: 1285 in, 0 cc, 6047 cr, 157 out
▶ ./scripts/runner step-end 1 3 answer "Entered 'artificial intelligence' in se...
✓ Recorded 1.3 answer
Verified: Step 1.3 = pass [1.663s]
─── Turn 27/100 ───
← anthropic/claude-haiku-4-5-20251001: 1553 in, 0 cc, 6047 cr, 117 out
▶ ./scripts/ab open http://fixtures/spa.html
✓ ✓ Task Manager — PinchTab Benchmark
http://fixtures/spa.html [368ms]
─── Turn 28/100 ───
← anthropic/claude-haiku-4-5-20251001: 1405 in, 0 cc, 6047 cr, 97 out
▶ ./scripts/ab snapshot -i -c
✓ - heading "Task Manager — VERIFY_SPA_PAGE_99999" [level=1, ref=e1]
- button "All Tasks" [ref=e2] (+15 lines) [310ms]
─── Turn 29/100 ───
← anthropic/claude-haiku-4-5-20251001: 1790 in, 0 cc, 6047 cr, 115 out
▶ ./scripts/ab fill @e5 "Test dynamic content"
✓ ✓ Done [379ms]
─── Turn 30/100 ───
← anthropic/claude-haiku-4-5-20251001: 1663 in, 0 cc, 6047 cr, 92 out
▶ ./scripts/ab click @e7
✓ ✓ Done [338ms]
─── Turn 31/100 ───
← anthropic/claude-haiku-4-5-20251001: 1794 in, 0 cc, 6047 cr, 101 out
▶ ./scripts/ab snapshot -i -c
✓ - heading "Task Manager — VERIFY_SPA_PAGE_99999" [level=1, ref=e1]
- button "All Tasks" [ref=e2] (+17 lines) [328ms]
─── Turn 32/100 ───
← anthropic/claude-haiku-4-5-20251001: 1688 in, 0 cc, 6047 cr, 171 out
▶ ./scripts/runner step-end 1 4 answer "Added new task 'Test dynamic content' t...
✓ Recorded 1.4 answer
Verified: Step 1.4 = pass [1.69s]
─── Turn 33/100 ───
← anthropic/claude-haiku-4-5-20251001: 1966 in, 0 cc, 6047 cr, 104 out
▶ ./scripts/ab back
✓ http://fixtures/search.html [868ms]
─── Turn 34/100 ───
← anthropic/claude-haiku-4-5-20251001: 1598 in, 0 cc, 6047 cr, 103 out
▶ ./scripts/ab snapshot -i -c
✓ - link "Home" [ref=e2]
- link "Search" [ref=e3] (+5 lines) [339ms]
─── Turn 35/100 ───
← anthropic/claude-haiku-4-5-20251001: 1857 in, 0 cc, 6047 cr, 154 out
▶ ./scripts/runner step-end 1 5 answer "Navigated back to search page (http://f...
✓ Recorded 1.5 answer
Verified: Step 1.5 = pass [1.672s]
─── Turn 36/100 ───
← anthropic/claude-haiku-4-5-20251001: 1864 in, 0 cc, 6047 cr, 251 out
Wrote <repo>/tests/benchmark/results/agent_browser_benchmark_20260420_134029_summary.md
# Benchmark Summary
| Metric | Value |
|--------|-------|
| Type | agent-browser |
| Model | claude-haiku-4-5-20251001 |
| Steps Answered | 10 |
| Execution Failed | 0 |
| Execution Skipped | 0 |
| Answer Rate | 100% |
| Verified Passed | 10 |
| Verified Failed | 0 |
| Verified Skipped | 0 |
| Pending Verification | 0 |
| Verification Pass Rate | 100% |
| Tool Calls | 59 |
## Run Usage
| Metric | Value |
|--------|-------|
| Source | anthropic-messages |
| Provider | anthropic |
| API Requests | 36 |
| Input Tokens (uncached) | 52276 |
| Cache Creation Input Tokens | 6047 |
| Cache Read Input Tokens | 217692 |
| Total Input Tokens | 276015 |
| Output Tokens | 4308 |
| Total Tokens | 280323 |
## Pending Verification
- none
## Failed Steps
- none
## Verification Failures
- none
[run-usage] provider=anthropic requests=36 input=52276 cache_create=6047 cache_read=217692 output=4308 total=280323
══ Benchmark result ══
answered: 10
execution failed: 0
execution skipped: 0
verified passed: 10
verified failed: 0
verified skipped: 0
pending verification: 0
tool calls: 59
verification pass rate: 100.0% (10/10)