High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
docs(skills): correct the pinchtab-dev, pinchtab-opt and pinchtab-stealth-score skills and the testing guide against the dev script and e2e runner
Luigi Agosti06972866a9b6
diff --git a/TESTING.md b/TESTING.mdindex 43342942..142eb52b 100644--- a/TESTING.md+++ b/TESTING.md@@ -12,7 +12,7 @@ The `dev` developer toolkit is the easiest way to run checks and tests:./dev e2e basic # Basic suite (api + cli + infra)./dev e2e extended # Extended suite./dev e2e smoke # Smoke suite (smoke scenarios + Docker smoke)-./dev e2e smoke-docker # Host Docker smoke only+./dev smoke # Host Docker smoke only./dev e2e api # API basic tests./dev e2e cli # CLI basic tests./dev e2e infra # Infra basic tests@@ -141,7 +141,7 @@ Runs `api-extended`, `cli-extended`, `infra-extended`, and `plugin` in sequence../dev e2e smoke./dev e2e smoke-orchestrator./dev e2e smoke-security-./dev e2e smoke-docker+./dev smoke```Smoke is its own tier: it runs `*-smoke.sh` scenarios plus host-level Docker smoke checks, and it does not include basic or extended scenarios. Use the filtered smoke suites when you only need one smoke lane.diff --git a/skills/pinchtab-dev/SKILL.md b/skills/pinchtab-dev/SKILL.mdindex 7f14bad7..b09946ac 100644--- a/skills/pinchtab-dev/SKILL.md+++ b/skills/pinchtab-dev/SKILL.md@@ -9,9 +9,7 @@ PinchTab is a browser control server for AI agents — Small Go binary with HTTP## Project Location-```bash-cd ~/dev/pinchtab-```+Run everything from the repository root (a clone of `github.com/pinchtab/pinchtab`).## Dev Commands@@ -31,9 +29,9 @@ All development commands run via `./dev`:| `./dev e2e basic` | Basic suite (api + cli + infra) || `./dev e2e extended` | Extended suite (all extended) || `./dev e2e smoke` | Smoke suite |-| `./dev e2e smoke-docker` | Host Docker smoke checks only |+| `./dev smoke` | Host Docker smoke checks (`--browser=chrome\|cloak\|all`) || `./dev e2e test "<name>"` | Run a single E2E test by `start_test` name |-| `./dev all` | check + test + e2e (pre-push gate) |+| `./dev all` | check + test + e2e extended (pre-push gate) || `./dev binaries` | Build the full release binary matrix into dist/ || `./dev doctor` | Setup dev environment |@@ -47,9 +45,10 @@ internal/server/ HTTP serverdashboard/ Embedded React dashboardconfig/ Configuration+ routes/ API route catalogueassets/ Embedded assets (stealth.js)dashboard/ React dashboard source (Vite + TypeScript)-tests/e2e/ E2E test suites+tests/e2e/ E2E runner, fixtures, scenarios/{api,cli,infra,plugin}/```## Workflow: New Feature or Bug Fix@@ -71,7 +70,7 @@ tests/e2e/ E2E test suites./dev e2e basic # E2E tests (Docker required)```-4. **Commit** with conventional commits:+4. **Commit** with conventional commits, usually scoped — `fix(extract): … (PIN-394)`:- `feat:` new feature- `fix:` bug fix- `refactor:` code change without behavior change@@ -110,8 +109,9 @@ tests/e2e/ E2E test suites| `internal/bridge/bridge.go` | Chrome CDP bridge || `internal/handlers/*.go` | HTTP API endpoints || `dashboard/src/` | React dashboard source |-| `tests/e2e/scenarios-api/` | API E2E tests |-| `tests/e2e/scenarios-cli/` | CLI E2E tests |+| `internal/routes/routes.go` | API route catalogue |+| `tests/e2e/scenarios/api/` | API E2E tests |+| `tests/e2e/scenarios/cli/` | CLI E2E tests |## Testing@@ -131,34 +131,34 @@ go test ./internal/handlers # Specific package./dev e2e cli-extended # CLI extended tests./dev e2e infra-extended # Infra extended tests (multi-instance)./dev e2e extended # Full extended suite (all extended tests)-./dev e2e smoke-docker # Host Docker smoke checks only+./dev smoke # Host Docker smoke checks (not an e2e suite)# Run specific test file(s) with filter (second argument)./dev e2e api clipboard # Run only clipboard-basic.sh-./dev e2e api-extended "clipboard|console" # Run clipboard and console tests+./dev e2e api-extended tabs # Run tabs-extended.sh./dev e2e cli browser # Run browser-basic.sh in CLI suite# Run a single test by its start_test name (fastest debug loop)-./dev e2e test "humanClick: click input by ref"+./dev e2e test "click with humanize: click input by ref"./dev e2e test "scroll (down)"./dev e2e test "low-level mouse"```-The scenario filter is a substring matched against scenario filenames. Requires Docker daemon running.+The scenario filter is one plain substring (no regex, no `|`) matched against scenario file names, groups, tiers, helpers and tags. Requires Docker daemon running.#### Single-test mode (`dev e2e test "<name>"`)Use this when iterating on one specific E2E failure. The runner:1. Greps `tests/e2e/scenarios/**/*.sh` for `start_test "...<name substring>..."`.-2. Auto-picks the suite (`api`/`cli`/`infra`/`plugin`) and `-extended` variant from the matching scenario file's path.-3. Builds fresh images (`compose ... up --build`) and runs **only the matching `start_test`...`end_test` block** — the scenario preamble (helper sourcing, `FIXTURES_URL`, etc.) is preserved, every other test in the file is skipped.+2. Auto-picks the suite (`api`/`cli`/`infra`/`plugin`) and `-extended` variant (or `smoke` for `*-smoke.sh`) from the matching scenario file's path.+3. Builds the images (`compose build`, then `up`) and runs, inside that scenario file, **only the `start_test`...`end_test` blocks whose name contains the substring** — the scenario preamble (helper sourcing, `FIXTURES_URL`, etc.) is preserved, every other test in the file is skipped.Notes:- The substring is literal (fgrep), so colons/parens/quotes in test names work without escaping.-- If multiple tests match, the runner uses the first and prints the others — pass a longer/more-specific substring to disambiguate.+- If several tests match, the scenario file of the first match is used and the others are printed — pass a longer/more-specific substring to disambiguate. It is not a way to run a group of tests; for a whole scenario use `go run ./tests/tools/runner e2e --suite <suite> --filter <stem>`.- Logs stream to the terminal by default (unlike full suites which hide logs); helpful for debugging.-- Implemented by `scripts/dev-e2e.sh` + `E2E_TEST_FILTER` plumbing through `scripts/e2e.sh` and `tests/e2e/run.sh`.+- Implemented by `scripts/dev-e2e.sh` → `go run ./tests/tools/runner e2e --suite … --filter <stem> --test "<name>"`, which passes `E2E_TEST_FILTER` to `tests/e2e/run.sh`.### Dashboard Tests```bash@@ -193,16 +193,16 @@ This runs:3. **Verify with pinchtab** — use the pinchtab skill to inspect the dashboard:```bash+ # ./dev dashboard runs the backend with token "dev"# Navigate to the page under development- curl -X POST http://localhost:9867/navigate \+ curl -X POST http://localhost:9867/navigate -H "Authorization: Bearer dev" \-d '{"url":"http://localhost:5173/dashboard/settings"}'- # Take a screenshot to verify the change- curl -X POST http://localhost:9867/screenshot \- -d '{"path":"/tmp/dashboard-check.png"}'+ # Take a screenshot (written under the state dir; response carries "path")+ curl -H "Authorization: Bearer dev" "http://localhost:9867/screenshot?output=file"# Or get a snapshot to inspect elements- curl -s http://localhost:9867/snapshot | jq .+ curl -s -H "Authorization: Bearer dev" http://localhost:9867/snapshot | jq .```4. **Provide evidence** — when reporting changes, include:@@ -214,26 +214,27 @@ This runs:```bash# Navigate to settings-curl -X POST http://localhost:9867/navigate \+curl -X POST http://localhost:9867/navigate -H "Authorization: Bearer dev" \-d '{"url":"http://localhost:5173/dashboard/settings"}'-# Screenshot the result-curl -X POST http://localhost:9867/screenshot \- -d '{"path":"./dashboard-settings.png","fullPage":true}'+# Screenshot the result (beyondViewport=true captures the full page)+curl -H "Authorization: Bearer dev" \+ "http://localhost:9867/screenshot?output=file&beyondViewport=true"-# Find specific element-curl -X POST http://localhost:9867/find \- -d '{"selector":"[data-testid=stealth-level]"}'+# Find an element by description (semantic match, not a CSS selector)+curl -X POST http://localhost:9867/find -H "Authorization: Bearer dev" \+ -d '{"query":"stealth level setting"}'```### Key Dashboard Pages| Page | URL | Purpose ||------|-----|---------|-| Home | `/dashboard/` | Instance overview |-| Settings | `/dashboard/settings` | Configuration |+| Monitoring | `/dashboard/monitoring` | Instance overview (`/dashboard/` redirects here) |+| Activity | `/dashboard/activity` | Activity log || Profiles | `/dashboard/profiles` | Browser profiles |-| Tabs | `/dashboard/tabs` | Active tabs |+| Agents | `/dashboard/agents` | Agents |+| Settings | `/dashboard/settings` | Configuration |### Dashboard Tech Stack@@ -250,8 +251,8 @@ The stealth module (`internal/assets/stealth.js`) has three levels:| Level | Features | Trade-offs ||-------|----------|------------|| `light` | webdriver, CDP markers, plugins, hardware | None — safe |-| `medium` | + userAgentData, chrome.runtime.connect, csi/loadTimes | May affect error monitoring |-| `full` | + WebGL/canvas noise, WebRTC relay | May break WebRTC, canvas apps |+| `medium` | + userAgentData, chrome.runtime.connect, puppeteer/playwright marker cleanup | May affect error monitoring, extension messaging |+| `full` | + screen/window realism, headless-only WebGL vendor/renderer spoofing (canvas/audio/WebRTC stay native) | May break graphics-dependent sites in headless mode |Configure in `~/.pinchtab/config.json`:```json@@ -266,14 +267,14 @@ Configure in `~/.pinchtab/config.json`:### Add new API endpoint1. Create handler in `internal/handlers/`-2. Register route in `internal/server/routes.go`+2. Add the endpoint to `internal/routes/routes.go` and bind it in the `routeBinding` table in `internal/handlers/handlers.go`3. Add tests in same package-4. Add E2E test in `tests/e2e/scenarios-api/`+4. Add E2E test in `tests/e2e/scenarios/api/`### Modify stealth behavior1. Edit `internal/assets/stealth.js`2. Run `./dev build` (embeds via go:embed)-3. Test with `./dev e2e api-fast` (includes stealth tests)+3. Test with `./dev e2e infra stealth` (infra scenarios matching `stealth`; `infra-extended stealth` for the extended ones)### Update dashboard1. Run `./dev dashboard` for hot-reloaddiff --git a/skills/pinchtab-opt/SKILL.md b/skills/pinchtab-opt/SKILL.mdindex 17b091a0..d4d67d7d 100644--- a/skills/pinchtab-opt/SKILL.md+++ b/skills/pinchtab-opt/SKILL.md@@ -109,7 +109,7 @@ trap '' EXIT INT TERM# 3. Misc state-rm -f ~/.local/state/pinchtab/current-tab 2>/dev/null+rm -f ~/.local/state/pinchtab/current-tab-* 2>/dev/nullrm -f "$PROJECT_ROOT/pinchtab" 2>/dev/null# Defensive: clear any stray *config*.json the agent might leave in a real ~/.pinchtab# (no-op because we stashed it above — but kept for runs that skip the stash).@@ -226,9 +226,9 @@ Save the three report file paths — you pass the correct one to each subagent.Use the **Agent** tool with `run_in_background: true`. Split the 47 groups into three batches:-- **Batch A**: groups 0–14 (45 steps)+- **Batch A**: groups 0–14 (43 steps)- **Batch B**: groups 15–29 (30 steps)-- **Batch C**: groups 30–46 (33 steps)+- **Batch C**: groups 30–46 (35 steps)Each subagent receives a provider-aware prompt (replace the placeholders):@@ -275,7 +275,7 @@ While agents run, periodically count step-end recordings:grep -c "step-end" <output_file>```-Expected totals: Batch A ~45, Batch B ~30, Batch C ~33 = 108 total per provider.+Expected totals: Batch A ~43, Batch B ~30, Batch C ~35 = 108 total per provider.### 5. Tear down + collect results@@ -313,7 +313,7 @@ If running `all`, repeat steps 1–5 for each remaining provider, then present c## Reference Numbers-- **Baseline**: 108/108 steps, 272 ops, ~49s total, 2.5 ops/step (stored in `baseline-ref.json`)+- **Baseline**: 108/108 steps, 272 ops, ~49s total, 2.5 ops/step (stored in `baseline-ref.json`; `baseline.sh` currently scripts only groups 0–44, 99 steps)- **Expected agent range**: 250–400 browser ops, 2.5–4 ops/step- **Group count**: 47 groups (`group-00` … `group-46`), 108 total steps@@ -328,7 +328,7 @@ If running `all`, repeat steps 1–5 for each remaining provider, then present c| `tests/optimization/subagent-context.md` | Optimization subagent instructions (env, wrapper, recording) || `tests/optimization/index.md` | Optimization group listing || `tests/optimization/group-00.md` … `group-46.md` | Optimization task descriptions |-| `tests/optimization/up.sh` | Provider-aware setup (chrome vs cloak) |+| `tests/optimization/up.sh` | Provider-aware setup (chrome, cloak, ghost-chrome) || `tests/optimization/down.sh` | Tear down after a provider run || `skills/pinchtab/SKILL.md` | PinchTab command reference (read by subagents) || `skills/pinchtab-dev/SKILL.md` | Build instructions (read by the setup subagent) |diff --git a/skills/pinchtab-stealth-score/SKILL.md b/skills/pinchtab-stealth-score/SKILL.mdindex 836f6dda..c8d0b606 100644--- a/skills/pinchtab-stealth-score/SKILL.md+++ b/skills/pinchtab-stealth-score/SKILL.md@@ -7,8 +7,8 @@ description: "Run the PinchTab stealth-score sweep against 15 bot-detection / fiDrive PinchTab through a list of public bot-detection sites under each browser(`chrome`, `cloak`, or `both`), and collect the metrics that matter-most for analyst comparison. The Docker plumbing rebuilds the PinchTab image-from current source so you're benchmarking the working tree.+most for analyst comparison. `up.sh` reuses cached images, so pass `REBUILD=1`+when you need to benchmark the current working tree.The shape is the same as `/pinchtab-opt`: one container per run, one blindagent that reads English playbooks and drives PinchTab through `./scripts/pt`,@@ -75,7 +75,8 @@ For each requested browser, run the same loop:"$SCORE_DIR/up.sh" "$PROVIDER"```-`up.sh` builds the appropriate image if absent (chrome-smoke or cloakbrowser),+`up.sh` builds the appropriate image if absent (`pinchtab-local:test` ++`pinchtab-chrome-smoke:test`, or `pinchtab-cloakbrowser:test`),writes a browser config with open `allowedDomains`, and starts a containernamed `stealth-score-pinchtab` on host port 9867. It exits non-zero if thecontainer fails to become healthy.diff --git a/tests/optimization/index.md b/tests/optimization/index.mdindex 349521f9..801a9e5e 100644--- a/tests/optimization/index.md+++ b/tests/optimization/index.md@@ -8,7 +8,7 @@ Read this index first, then read the relevant group files.- `setup-pinchtab.md` defines lane-specific setup, wrapper usage, and recording.- This directory defines the shared task groups.-- Each group lives in its own file: `group-00.md` through `group-44.md`.+- Each group lives in its own file: `group-00.md` through `group-46.md`.- When running a subset, read only the selected group files.- When running the full suite, proceed in numeric order.