High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
dev bench pinchtab --max-turns 100 --verbose --groups 0,1
warning: multiple providers configured, defaulting to anthropic
Rebuilding pinchtab with pinchtab-benchmark.json (wrapContent=false)...
Compose now can delegate build to bake for better performances
Just set COMPOSE_BAKE=true
[+] Building 1.5s (29/29) FINISHED docker:desktop-linux
=> [pinchtab internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.48kB 0.0s
=> [pinchtab internal] load metadata for docker.io/library/alpine:3.21 0.5s
=> [pinchtab internal] load metadata for docker.io/oven/bun:1 0.5s
=> [pinchtab internal] load metadata for docker.io/library/golang:1.26-alpine 0.5s
=> [pinchtab internal] load .dockerignore 0.0s
=> => transferring context: 504B 0.0s
=> [pinchtab dashboard 1/6] FROM docker.io/oven/bun:1@sha256:87416c977a612a204eb54ab9f3927023c2a3c 0.0s
=> => resolve docker.io/oven/bun:1@sha256:87416c977a612a204eb54ab9f3927023c2a3c971f4f345a01da08ea6 0.0s
=> [pinchtab internal] load build context 0.9s
=> => transferring context: 423.26kB 0.9s
=> [pinchtab stage-2 1/6] FROM docker.io/library/alpine:3.21@sha256:48b0309ca019d89d40f670aa1bc06e 0.0s
=> => resolve docker.io/library/alpine:3.21@sha256:48b0309ca019d89d40f670aa1bc06e426dc0931948452e8 0.0s
=> [pinchtab builder 1/9] FROM docker.io/library/golang:1.26-alpine@sha256:f85330846cde1e57ca9ec30 0.0s
=> => resolve docker.io/library/golang:1.26-alpine@sha256:f85330846cde1e57ca9ec309382da3b8e6ae3ab9 0.0s
=> CACHED [pinchtab stage-2 2/6] RUN apk add --no-cache chromium nss freetype harf 0.0s
=> CACHED [pinchtab stage-2 3/6] RUN adduser -D -h /data -g '' pinchtab && mkdir -p /data && 0.0s
=> CACHED [pinchtab builder 2/9] RUN apk add --no-cache git 0.0s
=> CACHED [pinchtab builder 3/9] WORKDIR /build 0.0s
=> CACHED [pinchtab builder 4/9] COPY go.mod go.sum ./ 0.0s
=> CACHED [pinchtab builder 5/9] RUN go mod download 0.0s
=> CACHED [pinchtab builder 6/9] COPY . . 0.0s
=> CACHED [pinchtab dashboard 2/6] WORKDIR /build 0.0s
=> CACHED [pinchtab dashboard 3/6] COPY dashboard/package.json dashboard/bun.lock ./ 0.0s
=> CACHED [pinchtab dashboard 4/6] RUN bun install --frozen-lockfile 0.0s
=> CACHED [pinchtab dashboard 5/6] COPY dashboard/ . 0.0s
=> CACHED [pinchtab dashboard 6/6] RUN bun run build 0.0s
=> CACHED [pinchtab builder 7/9] COPY --from=dashboard /build/dist/ internal/dashboard/dashboard/ 0.0s
=> CACHED [pinchtab builder 8/9] RUN mv internal/dashboard/dashboard/index.html internal/dashboard 0.0s
=> CACHED [pinchtab builder 9/9] RUN go build -ldflags="-s -w" -o pinchtab ./cmd/pinchtab 0.0s
=> CACHED [pinchtab stage-2 4/6] COPY --from=builder /build/pinchtab /usr/local/bin/pinchtab 0.0s
=> CACHED [pinchtab stage-2 5/6] COPY --chmod=0755 docker-entrypoint.sh /usr/local/bin/docker-entr 0.0s
=> CACHED [pinchtab stage-2 6/6] WORKDIR /data 0.0s
=> [pinchtab] exporting to image 0.0s
=> => exporting layers 0.0s
=> => exporting manifest sha256:da7f4403d6e06e90879bf90cca17952d700c1f789dcce834fd0f53899b26ff05 0.0s
=> => exporting config sha256:36a6628c08b8fdba1ceb66bf2b511b5f1586f66649d79c637a5612295ea221de 0.0s
=> => exporting attestation manifest sha256:38b4b66685c66bddd392bf734b5230a5aa560689b66a5c849322e6 0.0s
=> => exporting manifest list sha256:fe0f1f08f56afdcb7b8b7b415633c01d3ca21e98ebfb50010575e01e95710 0.0s
=> => naming to docker.io/library/tools-pinchtab:latest 0.0s
=> => unpacking to docker.io/library/tools-pinchtab:latest 0.0s
=> [pinchtab] resolving provenance for metadata file 0.0s
[+] Running 2/2
✔ pinchtab Built 0.0s
✔ Container tools-pinchtab-1 Started 5.5s
Verified: pinchtab running with wrapContent=false
=== PinchTab Benchmark Run #1 ===
Timestamp: 20260420_140204
Report: <repo>/tests/benchmark/results/pinchtab_benchmark_20260420_140204.json
[benchmark-runner] provider=anthropic model=claude-haiku-4-5-20251001 lane=pinchtab groups=0,1 report=<repo>/tests/benchmark/results/pinchtab_benchmark_20260420_140204.json
══ Benchmark start ══
lane: pinchtab
model: claude-haiku-4-5-20251001
report: <repo>/tests/benchmark/results/pinchtab_benchmark_20260420_140204.json
Group 0 — 5 step(s):
0.1 Navigate to page
0.2 Take snapshot
0.3 Read page text
0.4 Click a link
0.5 Extract specific data
Group 1 — 5 step(s):
1.1 Fill a text input
1.2 Click a button
1.3 Use search functionality
1.4 Handle dynamic content
1.5 Navigate back
Total: 10 step(s) across 2 group(s)
─── Turn 1/100 ───
← anthropic/claude-haiku-4-5-20251001: 2 in, 6232 cc, 0 cr, 122 out
▶ ./scripts/pt nav http://fixtures/ --snap
✓ FD794763775C2B0F29616C6F260B6691
# PinchTab Benchmark - Home | http://fixtures/ | 13 nodes (+13 lines) [343ms]
─── Turn 2/100 ───
← anthropic/claude-haiku-4-5-20251001: 325 in, 0 cc, 6232 cr, 152 out
▶ ./scripts/runner step-end 0 1 answer "Page loaded successfully at http://fixt...
✓ Recorded 0.1 answer
Verified: Step 0.1 = pass [1.668s]
─── Turn 3/100 ───
← anthropic/claude-haiku-4-5-20251001: 577 in, 0 cc, 6232 cr, 95 out
▶ ./scripts/pt snap
✓ # PinchTab Benchmark - Home | http://fixtures/ | 13 nodes
e0:heading "PinchTab Benchmark Suite" (+12 lines) [110ms]
─── Turn 4/100 ───
← anthropic/claude-haiku-4-5-20251001: 847 in, 0 cc, 6232 cr, 138 out
▶ ./scripts/runner step-end 0 2 answer "Snapshot captured with 13 nodes includi...
✓ Recorded 0.2 answer
Verified: Step 0.2 = pass [1.668s]
─── Turn 5/100 ───
← anthropic/claude-haiku-4-5-20251001: 1083 in, 0 cc, 6232 cr, 94 out
▶ ./scripts/pt text
✓ Welcome to the Benchmark Test Suite
This site contains controlled test pages for verifying browse... (+9 lines) [103ms]
─── Turn 6/100 ───
← anthropic/claude-haiku-4-5-20251001: 1305 in, 0 cc, 6232 cr, 121 out
▶ ./scripts/runner step-end 0 3 answer "Page text extracted successfully includ...
✓ Recorded 0.3 answer
Verified: Step 0.3 = pass [1.661s]
─── Turn 7/100 ───
← anthropic/claude-haiku-4-5-20251001: 1508 in, 0 cc, 6232 cr, 112 out
▶ ./scripts/pt click e1 --snap
✓ OK
# Articles - PinchTab Benchmark | http://fixtures/articles.html | 1... (+18 lines) [154ms]
─── Turn 8/100 ───
← anthropic/claude-haiku-4-5-20251001: 1535 in, 0 cc, 6232 cr, 138 out
▶ ./scripts/runner step-end 0 4 answer "Clicked Articles link (e1) and navigate...
✓ Recorded 0.4 answer
Verified: Step 0.4 = pass [1.663s]
─── Turn 9/100 ───
← anthropic/claude-haiku-4-5-20251001: 1765 in, 0 cc, 6232 cr, 110 out
▶ ./scripts/pt text e3
✓ The Future of Artificial Intelligence [105ms]
─── Turn 10/100 ───
← anthropic/claude-haiku-4-5-20251001: 1445 in, 0 cc, 6232 cr, 126 out
▶ ./scripts/runner step-end 0 5 answer "Read heading e3 text: 'The Future of Ar...
✓ Recorded 0.5 answer
Verified: Step 0.5 = pass [1.658s]
─── Turn 11/100 ───
← anthropic/claude-haiku-4-5-20251001: 1657 in, 0 cc, 6232 cr, 132 out
▶ ./scripts/pt click e0 --snap
✗ exit 1 Error 409: unexpected page navigation: http://fixtures/articles.htm... [125ms]
─── Turn 12/100 ───
← anthropic/claude-haiku-4-5-20251001: 1460 in, 0 cc, 6232 cr, 98 out
▶ ./scripts/pt snap
✓ # PinchTab Benchmark - Home | http://fixtures/ | 13 nodes
e0:heading "PinchTab Benchmark Suite" (+12 lines) [98ms]
─── Turn 13/100 ───
← anthropic/claude-haiku-4-5-20251001: 1733 in, 0 cc, 6232 cr, 112 out
▶ ./scripts/pt click e3 --snap
✓ OK
# Form - PinchTab Benchmark | http://fixtures/form.html | 27 nodes (+27 lines) [132ms]
─── Turn 14/100 ───
← anthropic/claude-haiku-4-5-20251001: 1568 in, 0 cc, 6232 cr, 121 out
▶ ./scripts/pt fill e3 "John Doe" --snap-diff
✓ OK
# Form - PinchTab Benchmark | http://fixtures/form.html | 27 nodes ... (+27 lines) [113ms]
─── Turn 15/100 ───
← anthropic/claude-haiku-4-5-20251001: 2042 in, 0 cc, 6232 cr, 135 out
▶ ./scripts/runner step-end 1 1 answer "Filled Full Name field (e3) with 'John ...
✓ Recorded 1.1 answer
Verified: Step 1.1 = pass [1.662s]
─── Turn 16/100 ───
← anthropic/claude-haiku-4-5-20251001: 1936 in, 0 cc, 6232 cr, 123 out
▶ ./scripts/pt fill e4 "[email protected]" --snap-diff
✓ OK
# Form - PinchTab Benchmark | http://fixtures/form.html | 27 nodes ... (+27 lines) [103ms]
─── Turn 17/100 ───
← anthropic/claude-haiku-4-5-20251001: 2425 in, 0 cc, 6232 cr, 101 out
▶ ./scripts/pt click e25 --snap
✓ OK
# Form - PinchTab Benchmark | http://fixtures/form.html | 28 nodes (+28 lines) [121ms]
─── Turn 18/100 ───
← anthropic/claude-haiku-4-5-20251001: 2409 in, 0 cc, 6232 cr, 100 out
▶ ./scripts/pt text
✓ Contact Form — VERIFY_FORM_PAGE_22222
(+73 lines) [104ms]
─── Turn 19/100 ───
← anthropic/claude-haiku-4-5-20251001: 2779 in, 0 cc, 6232 cr, 174 out
▶ ./scripts/runner step-end 1 2 answer "Clicked Submit Form button (e25) and fo...
✓ Recorded 1.2 answer
Verified: Step 1.2 = pass [1.663s]
─── Turn 20/100 ───
← anthropic/claude-haiku-4-5-20251001: 2196 in, 0 cc, 6232 cr, 111 out
▶ ./scripts/pt nav http://fixtures/search.html --snap
✓ FD794763775C2B0F29616C6F260B6691
# Search - PinchTab Benchmark | http://fixtures/search.html | 5 nodes (+5 lines) [341ms]
─── Turn 21/100 ───
← anthropic/claude-haiku-4-5-20251001: 2444 in, 0 cc, 6232 cr, 124 out
▶ ./scripts/pt fill e3 "artificial intelligence" --snap-diff
✓ OK
# Search - PinchTab Benchmark | http://fixtures/search.html | 5 nod... (+5 lines) [103ms]
─── Turn 22/100 ───
← anthropic/claude-haiku-4-5-20251001: 1993 in, 0 cc, 6232 cr, 106 out
▶ ./scripts/pt click e4 --snap
✓ OK
# Search - PinchTab Benchmark | http://fixtures/search.html | 7 nodes (+7 lines) [117ms]
─── Turn 23/100 ───
← anthropic/claude-haiku-4-5-20251001: 2246 in, 0 cc, 6232 cr, 122 out
▶ ./scripts/pt text
✓ Search — VERIFY_SEARCH_PAGE_11111
(+18 lines) [103ms]
─── Turn 24/100 ───
← anthropic/claude-haiku-4-5-20251001: 1663 in, 0 cc, 6232 cr, 198 out
▶ ./scripts/runner step-end 1 3 answer "Entered 'artificial intelligence' in se...
✓ Recorded 1.3 answer
Verified: Step 1.3 = pass [1.668s]
─── Turn 25/100 ───
← anthropic/claude-haiku-4-5-20251001: 1984 in, 0 cc, 6232 cr, 117 out
▶ ./scripts/pt nav http://fixtures/spa.html --snap
✓ FD794763775C2B0F29616C6F260B6691
# Task Manager — PinchTab Benchmark | http://fixtures/spa.html | ... (+17 lines) [330ms]
─── Turn 26/100 ───
← anthropic/claude-haiku-4-5-20251001: 1881 in, 0 cc, 6232 cr, 141 out
▶ ./scripts/pt fill e4 "Test dynamic content" --snap-diff
✓ OK
# Task Manager — PinchTab Benchmark | http://fixtures/spa.html | ... (+17 lines) [117ms]
─── Turn 27/100 ───
← anthropic/claude-haiku-4-5-20251001: 2270 in, 0 cc, 6232 cr, 112 out
▶ ./scripts/pt click e9 --snap
✓ OK
# Task Manager — PinchTab Benchmark | http://fixtures/spa.html | ... (+19 lines) [85ms]
─── Turn 28/100 ───
← anthropic/claude-haiku-4-5-20251001: 2103 in, 0 cc, 6232 cr, 139 out
▶ ./scripts/pt text
✓ Low Priority
Medium Priority (+34 lines) [104ms]
─── Turn 29/100 ───
← anthropic/claude-haiku-4-5-20251001: 2425 in, 0 cc, 6232 cr, 219 out
▶ ./scripts/runner step-end 1 4 answer "Navigated to SPA page with Task Manager...
✓ Recorded 1.4 answer
Verified: Step 1.4 = pass [1.667s]
─── Turn 30/100 ───
← anthropic/claude-haiku-4-5-20251001: 2295 in, 0 cc, 6232 cr, 109 out
▶ ./scripts/pt back --snap
✓ http://fixtures/search.html
# Search - PinchTab Benchmark | http://fixtures/search.html | 7 nodes (+7 lines) [316ms]
─── Turn 31/100 ───
← anthropic/claude-haiku-4-5-20251001: 2553 in, 0 cc, 6232 cr, 185 out
▶ ./scripts/runner step-end 1 5 answer "Used browser back navigation to return ...
✓ Recorded 1.5 answer
Verified: Step 1.5 = pass [1.667s]
─── Turn 32/100 ───
← anthropic/claude-haiku-4-5-20251001: 2149 in, 0 cc, 6232 cr, 230 out
Wrote <repo>/tests/benchmark/results/pinchtab_benchmark_20260420_140204_summary.md
# Benchmark Summary
| Metric | Value |
|--------|-------|
| Type | pinchtab |
| Model | claude-haiku-4-5-20251001 |
| Steps Answered | 10 |
| Execution Failed | 0 |
| Execution Skipped | 0 |
| Answer Rate | 100% |
| Verified Passed | 10 |
| Verified Failed | 0 |
| Verified Skipped | 0 |
| Pending Verification | 0 |
| Verification Pass Rate | 100% |
| Tool Calls | 0 |
## Run Usage
| Metric | Value |
|--------|-------|
| Source | anthropic-messages |
| Provider | anthropic |
| API Requests | 32 |
| Input Tokens (uncached) | 56603 |
| Cache Creation Input Tokens | 6232 |
| Cache Read Input Tokens | 193192 |
| Total Input Tokens | 256027 |
| Output Tokens | 4217 |
| Total Tokens | 260244 |
## Pending Verification
- none
## Failed Steps
- none
## Verification Failures
- none
[run-usage] provider=anthropic requests=32 input=56603 cache_create=6232 cache_read=193192 output=4217 total=260244
══ Benchmark result ══
answered: 10
execution failed: 0
execution skipped: 0
verified passed: 10
verified failed: 0
verified skipped: 0
pending verification: 0
tool calls: 0
verification pass rate: 100.0% (10/10)