Skip to content

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

Add QA report with bugs and performance comparison

Lb8466ec6fc41
diff --git a/QA.md b/QA.md
index 9d0efbab..28f3ab02 100644
--- a/QA.md
+++ b/QA.md
@@ -1,33 +1,130 @@
# Pinchtab QA Report — 2026-02-15
+**Build:** commit 4fc2a3e (latest main)
+**Testers:** Bosch, Mario
+
+---
+
## Bugs
-1. **`newTab:true` broken** — `/navigate` with `newTab:true` silently ignores the flag and navigates the existing tab instead of opening a new one. Tabs get overwritten/lost.
+### 🔴 P0 — Active Tab Not Tracked After Navigate (Bosch)
+
+`/navigate` opens a new tab (or reuses one unpredictably) but `/snapshot` and `/text` don't follow — they return data from a stale tab. This is the main blocker for reliable automation.
+
+**Repro:**
+1. Fresh start — Pinchtab opens with `about:blank`
+2. `/navigate` to `https://example.com` → works (reuses initial tab)
+3. `/navigate` to `https://github.com/luigi-agosti/pinchtab` → opens new tab
+4. `/snapshot` → returns `about:blank` or the previous tab, not GitHub
+
+**Expected:** After `/navigate`, subsequent `/snapshot`/`/text` should return data from the navigated tab.
+
+### 🔴 P0 — Invalid JSON in Snapshot/Text (Bosch)
+
+Both `/snapshot` and `/text` produce invalid JSON on pages with certain content. Unescaped control characters break standard JSON parsers.
+
+**Affected sites:** Yahoo Finance, StackOverflow (confirmed), likely others
+**Error:** `json.JSONDecodeError: Invalid control character at: line 1 column N`
+**Workaround:** `json.loads(raw, strict=False)` in Python
+**Fix:** Escape control characters (U+0000–U+001F) in all string values before JSON serialization.
+
+### 🟡 P1 — `newTab:true` Broken (Mario)
+
+`/navigate` with `newTab:true` silently ignores the flag and navigates the existing tab instead of opening a new one. Tabs get overwritten/lost.
+
+### 🟡 P1 — Tab Close Returns 400 (Bosch)
+
+`POST /tab {"action":"close","tabId":"..."}` returns `{"error":"tabId required"}`.
+Tried both `"id"` and `"tabId"` as field names — neither works.
+
+**Impact:** Can't clean up tabs programmatically, leading to tab accumulation.
+
+### 🟡 P1 — No Tab Switch/Focus API (Bosch)
+
+No way to manually set the active tab. When active tab tracking drifts, there's no recovery path.
+
+**Suggestion:** Add `POST /tab {"action":"switch","tabId":"..."}` or similar.
+
+### 🟡 P1 — `/action` Unhelpful Error Message (Mario)
-2. **`/action` unhelpful error message** — Missing `kind` field returns `{"error":"unknown action: "}` instead of telling the user the field is missing or listing valid values (`click`, `type`, `hover`, `select`, `scroll`).
+Missing `kind` field returns `{"error":"unknown action: "}` instead of telling the user the field is missing or listing valid values (`click`, `type`, `hover`, `select`, `scroll`).
-3. **`/navigate` returns empty title** — Some sites (BBC, x.com) return `"title":""` in the response. Likely a race condition — title isn't set by the time the response is sent.
+### 🟢 P2 — `/navigate` Returns Empty Title (Mario)
-4. **`/text` Google language blob** — Readability extraction on google.com includes the full language picker (all locale names) as content. Should be filtered out.
+Some sites (BBC, x.com) return `"title":""` in the response. Likely a race condition — title isn't set by the time the response is sent.
+
+### 🟢 P2 — `/text` Google Language Blob (Mario)
+
+Readability extraction on google.com includes the full language picker (all locale names) as content. Should be filtered out.
+
+### 🟢 P2 — Chrome Flag Warning (Bosch)
+
+```
+You are using an unsupported command-line flag: --disable-blink-features=AutomationControlled.
+Stability and security will suffer.
+```
+
+Minor — cosmetic warning but could affect stability per Chrome's own messaging.
+
+---
## Improvements
- **Better `/navigate` title** — Wait for `document.title` to be non-empty (with short timeout) before returning response.
- **Actionable error messages** — `/action` should list valid `kind` values in error responses.
- **`/text` content filtering** — Strip known noise patterns (language pickers, cookie banners) from readability output.
-- **Tab management** — Fix `newTab:true`, add `/tab/close` endpoint if not present.
+- **Tab management** — Fix `newTab:true`, fix tab close, add tab switch endpoint.
+- **Compact snapshot format** — Consider an aria-tree-style text output (indented, role + name per line) instead of/alongside verbose JSON. Could cut snapshot tokens by 3–4×.
+
+---
+
+## Performance — Token Usage
+
+### Pinchtab `/snapshot` vs `/text` (Mario)
-## Token Benchmarks
+| Site | /snapshot tokens | /text tokens | Savings |
+|------|-----------------|-------------|---------|
+| Google | ~2K | ~764 | 2.5× |
+| GitHub | ~9.8K | ~1.2K | 7.8× |
+| x.com | ~2K | ~121 | 17× |
+| BBC | ~26.7K | ~3.5K | 7.7× |
+| Wikipedia | ~20.5K | ~3.5K | 5.8× |
+| LinkedIn | ~7.5K | ~6.1K | 1.2× |
-| Site | /snapshot | /text | Savings |
-|------|----------|-------|---------|
-| Google | ~2K | ~764 | 2.5x |
-| GitHub | ~9.8K | ~1.2K | 7.8x |
-| x.com | ~2K | ~121 | 17x |
-| BBC | ~26.7K | ~3.5K | 7.7x |
-| Wikipedia | ~20.5K | ~3.5K | 5.8x |
-| LinkedIn | ~7.5K | ~6.1K | 1.2x |
+### Pinchtab vs OpenClaw Browser (Bosch)
+
+| Site | Pinchtab snapshot | Pinchtab /text | OpenClaw aria tree |
+|------|-------------------|---------------|-------------------|
+| Yahoo Finance | ~16K tokens | ~1.4K tokens | ~3.5K tokens |
+| Google Finance | ~12K tokens | ~1.1K tokens | ~3.7K tokens |
+| Hacker News | ~24K tokens | ~875 tokens | — |
+
+### Key Findings
+
+- **Pinchtab snapshots are 3–4× larger** than OpenClaw aria trees (verbose JSON with `ref`, `role`, `name`, `depth`, `nodeId` per node)
+- **Pinchtab `/text` is the most token-efficient format** (~1K tokens for complex finance pages) — great for content extraction
+- **OpenClaw aria tree** is the best balance for interactive browsing (~3.5K tokens, structured + compact)
+- **`/text` is a real strength** — already very efficient, should be the primary endpoint for read-only tasks
+
+---
## Sites Tested
-Google, GitHub, BBC, Wikipedia, x.com, LinkedIn — all loaded fine, no bot detection, zero crashes.
+**Mario:** Google, GitHub, BBC, Wikipedia, x.com, LinkedIn
+**Bosch:** HN, Example.com, Yahoo Finance, Google Finance, Bloomberg, StackOverflow
+
+All loaded fine, no bot detection, zero crashes. ✅
+
+---
+
+## What Works Well
+
+- ✅ `/navigate` — fast, returns title+URL correctly
+- ✅ `/snapshot` — comprehensive a11y tree when on the right tab
+- ✅ `/snapshot?filter=interactive` — properly filters to actionable elements
+- ✅ `/text` — clean, compact content extraction
+- ✅ `/action` with `click` — reliable (tested cookie consent acceptance)
+- ✅ `/tabs` — accurate tab listing
+- ✅ Startup is fast (~3 seconds to ready)
+- ✅ Headless mode works well
+- ✅ No bot detection on any tested sites