High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
docs: add Lite Engine changes summary with test results
Yash Jadhav4f19e6d3131d
diff --git a/LITE_ENGINE_CHANGES.md b/LITE_ENGINE_CHANGES.mdnew file mode 100644index 00000000..685ea13a--- /dev/null+++ b/LITE_ENGINE_CHANGES.md@@ -0,0 +1,225 @@+# Lite Engine: Chrome-Free DOM Capture using Gost-DOM++**Branch:** `feat/lite-engine-gostdom`+**Issue:** [#201](https://github.com/pinchtab/pinchtab/issues/201)+**Related Draft PR:** [#200](https://github.com/pinchtab/pinchtab/pull/200)+**Dependency:** [gost-dom/browser v0.11.0](https://github.com/gost-dom/browser) (MIT, ~255 stars, Go 78.4%)++---++## Overview++This implementation adds a **Lite Engine** that can perform DOM capture (navigate, snapshot, text extraction, click, type) without requiring Chrome/Chromium. It uses [Gost-DOM](https://github.com/gost-dom/browser), a headless browser written in pure Go, to parse and traverse HTML documents.++The architecture follows the maintainer's guidance for **"clever routing that is expandable without touching the rest of the code"** — implemented via a strategy-pattern Router with pluggable rules.++## Architecture++### Engine Interface (`internal/engine/engine.go`)++```go+type Engine interface {+ Name() string+ Navigate(ctx context.Context, url string) (*NavigateResult, error)+ Snapshot(ctx context.Context, filter string) ([]SnapshotNode, error)+ Text(ctx context.Context) (string, error)+ Click(ctx context.Context, ref string) error+ Type(ctx context.Context, ref, text string) error+ Capabilities() []Capability+ Close() error+}+```++### Router (`internal/engine/router.go`)++The Router evaluates an ordered chain of `RouteRule` implementations. The first rule to return a non-`Undecided` verdict wins.++```+Request → Router → [Rule 1] → [Rule 2] → ... → [Fallback Rule] → Engine+```++Rules are hot-swappable at runtime via `AddRule()` / `RemoveRule()` — no handler code changes needed.++### Three Modes++| Mode | Behavior | Default Rules |+|------|----------|---------------|+| `chrome` | All requests → Chrome (default, backward compatible) | DefaultChromeRule |+| `lite` | DOM ops → Gost-DOM, screenshots/PDF/evaluate → Chrome | CapabilityRule → DefaultLiteRule |+| `auto` | Per-request routing based on URL patterns | CapabilityRule → ContentHintRule → DefaultChromeRule |++### Built-in Rules (`internal/engine/rules.go`)++| Rule | Purpose |+|------|---------|+| `CapabilityRule` | Routes screenshot/pdf/evaluate/cookies → Chrome (lite can't do these) |+| `ContentHintRule` | Routes `.html/.htm/.xml/.txt/.md` URLs → Lite (for navigate/snapshot/text) |+| `DefaultLiteRule` | Catch-all: routes all DOM ops → Lite |+| `DefaultChromeRule` | Final fallback: routes everything → Chrome |++### Expandability++Adding new routing logic requires only:+1. Implement `RouteRule` interface (2 methods: `Name()`, `Decide()`)+2. Call `router.AddRule(myRule)` — inserted before the fallback rule++No handler, config, or CMD changes needed.++## Files Changed++### New Files (8)+| File | Purpose | Lines |+|------|---------|-------|+| `internal/engine/engine.go` | Engine interface, types, capabilities | ~70 |+| `internal/engine/lite.go` | LiteEngine implementation using Gost-DOM | ~430 |+| `internal/engine/router.go` | Router with AddRule/RemoveRule | ~120 |+| `internal/engine/rules.go` | 4 built-in RouteRule implementations | ~95 |+| `internal/engine/lite_test.go` | LiteEngine unit tests | ~280 |+| `internal/engine/router_test.go` | Router unit tests | ~130 |+| `internal/engine/rules_test.go` | Rule unit tests | ~115 |+| `internal/engine/realworld_test.go` | Real-world website comparison tests | ~570 |++### Modified Files (8)+| File | Change |+|------|--------|+| `internal/config/config.go` | Added `Engine` field to RuntimeConfig + ServerConfig, `PINCHTAB_ENGINE` env var |+| `internal/handlers/handlers.go` | Added `Router *engine.Router` field, `useLite()` helper |+| `internal/handlers/navigation.go` | Lite fast path before ensureChrome |+| `internal/handlers/snapshot.go` | Lite fast path with SnapshotNode → A11yNode conversion |+| `internal/handlers/text.go` | Lite fast path returning plain text |+| `cmd/pinchtab/cmd_bridge.go` | Engine router wiring based on config mode |+| `go.mod` | Added gost-dom/browser v0.11.0, gost-dom/css v0.1.0 |+| `go.sum` | Updated checksums |++## Improvements Over PR #200 Draft++| Area | PR #200 | This Implementation |+|------|---------|-------------------|+| Tab management | Single window | Multi-tab with sequential IDs |+| HTML parsing | `browser.Open()` double-fetches | HTTP fetch → strip scripts → `html.NewWindowReader` |+| Script handling | Panics on `<script>` tags | Pre-parse stripping via `x/net/html` tokenizer |+| Click safety | No panic protection | `defer recover()` in Click method |+| Text output | Raw DOM text | `normalizeWhitespace()` — collapses runs of whitespace |+| Role mapping | Basic (a, button, input, etc.) | Extended: section→region, details→group, summary→button, dialog, article |+| Interactive detection | Basic tags | Adds summary, ARIA roles (tab, menuitem, switch) |+| Routing | None (always lite) | Strategy-pattern Router with pluggable rules |+| Configuration | None | `PINCHTAB_ENGINE` env var, config file support |++## Test Results++### Engine Package Tests (40+ tests, all passing)++```+=== Unit Tests ===+TestLiteEngine_Navigate PASS+TestLiteEngine_Snapshot_All PASS+TestLiteEngine_Snapshot_Interactive PASS+TestLiteEngine_Text PASS+TestLiteEngine_Click PASS+TestLiteEngine_Type PASS+TestLiteEngine_RefNotFound PASS+TestLiteEngine_ScriptStyleSkipped PASS+TestLiteEngine_AriaAttributes PASS+TestLiteEngine_MultiTab PASS+TestLiteEngine_Close PASS+TestLiteEngine_Capabilities PASS+TestLiteEngine_Name PASS+TestNormalizeWhitespace PASS++=== Router Tests ===+TestRouterChromeMode PASS+TestRouterLiteMode PASS+TestRouterAutoModeStaticContent PASS+TestRouterAutoModeLiteNil PASS+TestRouterAddRemoveRule PASS+TestRouterRulesSnapshot PASS++=== Rule Tests ===+TestCapabilityRule (9 cases) PASS+TestContentHintRule (9 cases) PASS+TestDefaultLiteRule (7 cases) PASS+TestDefaultChromeRule (4 cases) PASS+```++### Real-World Website Comparison Tests (16 suites, 63+ subtests)++| Suite | Simulates | Subtests | Result |+|-------|-----------|----------|--------|+| WikipediaStyle | Wikipedia article page | 9 | PASS |+| HackerNewsStyle | HN front page | 4 | PASS |+| EcommerceStyle | Product page with forms | 9 | PASS |+| FormHeavy | Registration form | 7 | PASS |+| AriaHeavy | Dashboard with ARIA roles | 11 | PASS |+| DeeplyNested | 5+ levels of div nesting | 4 | PASS |+| SpecialCharacters | Unicode, HTML entities, CJK | 3 | PASS |+| EmptyPage | Empty HTML body | 1 | PASS |+| NonHTMLContentType | JSON response | 1 | PASS |+| HTTP404 | 404 error page | 1 | PASS |+| LargePagePerformance | 200 sections, 800+ nodes | 1 | PASS |+| MultipleScriptTags | 5 script tags in head+body | 1 | PASS |+| InlineStyles | Style tags in head+body | 1 | PASS |+| ClickWorkflow | Button clicks | 1 | PASS |+| ClickLinkRecovery | Anchor click panic recovery | 1 | PASS |+| TypeWorkflow | Type into all textboxes | 1 | PASS |++### Full Project Test Suite++```+ok cmd/pinchtab 2.8s+ok internal/allocation 2.0s+ok internal/config 1.6s+ok internal/dashboard 3.1s+ok internal/engine 1.4s ← new package+ok internal/handlers 6.8s+ok internal/human 10.7s+ok internal/idpi 2.0s+ok internal/idutil 1.8s+ok internal/instance 2.6s+ok internal/orchestrator 3.2s+ok internal/profiles 2.8s+ok internal/proxy 2.8s+ok internal/scheduler 4.0s+ok internal/semantic 1.6s+ok internal/strategy 1.7s+ok internal/uameta 1.1s+ok internal/web 1.5s+```++## Known Edge Cases & Limitations++| Edge Case | Behavior | Mitigation |+|-----------|----------|------------|+| `<script>` tags in HTML | Gost-DOM panics (nil ScriptHost) | Pre-parse stripping via x/net/html tokenizer |+| Click on `<a href>` | Gost-DOM navigates, may encounter scripts | `defer recover()` in Click, returns error |+| CSS `display:none` | Elements still appear in snapshot | Lite engine has no CSS engine |+| JavaScript-rendered content | Not captured (SPA, dynamic DOM) | Falls back to Chrome in auto mode |+| Screenshots / PDF | Not supported in lite | CapabilityRule routes to Chrome |+| Cookies / Evaluate | Not supported in lite | CapabilityRule routes to Chrome |+| `<noscript>` content | Stripped from snapshot | Consistent with script-disabled behavior |++## Configuration++### Environment Variable+```bash+PINCHTAB_ENGINE=lite # or "chrome" (default) or "auto"+```++### Config File+```json+{+ "server": {+ "engine": "lite"+ }+}+```++### Response Headers+Lite-served responses include `X-Engine: lite` header for observability.++## Dependency Analysis++| Package | Size | License | Purpose |+|---------|------|---------|---------|+| gost-dom/browser v0.11.0 | ~2.5MB source | MIT | Headless browser (HTML parsing, DOM traversal) |+| gost-dom/css v0.1.0 | ~200KB | MIT | CSS selector support |+| golang.org/x/net (existing) | already in go.mod | BSD-3 | HTML tokenizer for script stripping |