High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
docs: add phase 7 & 8 proposals for performance optimization and dashboard enhancements
Mario543dad0ae62d
diff --git a/docs/proposals/README.md b/docs/proposals/README.mdnew file mode 100644index 00000000..1386a969--- /dev/null+++ b/docs/proposals/README.md@@ -0,0 +1,176 @@+# Pinchtab Proposals & Roadmap++This folder contains detailed proposals for future phases of Pinchtab development.++## Current Status++**Phase 6: COMPLETE ✅**+- Multi-instance architecture fully implemented+- Hash-based ID system for profiles, instances, tabs+- Auto-port allocation with cleanup and reuse+- Instance isolation (separate Chrome, cookies, history, profiles)+- Lazy Chrome initialization+- 195+ tests passing+- Production-ready++## Future Phases++### Phase 7: Performance Optimization & Scaling+**File:** `phase-7-performance-optimization.md`++Focus areas:+- **Benchmarking**: Establish baseline metrics for latency, memory, concurrent limits+- **Optimizations**: Connection pooling, caching, batch navigation support+- **Scaling tests**: Validate 100+ concurrent instances, multi-agent load, resource limits++**Effort estimate:** 9-14 hours+**Priority:** High (needed for production deployments with 10+ instances)++#### Key Sections+1. Benchmarking & Metrics+ - Request latency baseline+ - Memory per instance+ - Concurrent instance limits++2. Optimizations & Improvements+ - HTTP connection pooling+ - Cache improvements+ - Batch navigation support++3. Scaling Tests & Validation+ - 100+ instance stress test+ - Multi-agent concurrent load+ - Resource limits testing++---++### Phase 8: Dashboard UI Enhancements+**File:** `phase-8-dashboard-ui-enhancements.md`++Focus areas:+- **Monitoring**: Real-time Chrome status, memory/CPU usage, tab activity+- **Batch operations**: Create multiple instances, terminate all, port range config+- **Navigation**: Search/filter, details modal, live logs viewer++**Effort estimate:** 18-28 hours+**Priority:** Medium-High (improves user experience significantly)++#### Key Sections+1. Instance Monitoring Screen+ - Real-time Chrome status+ - Memory & CPU usage per instance+ - Tab activity visualization++2. Batch Operations+ - Create multiple instances at once+ - Terminate all instances button+ - Port range configuration UI++3. Better Navigation & UX+ - Search & filter instances+ - Instance details modal+ - Live logs viewer++---++## Decision Framework++### When to Pursue Phase 7 vs Phase 8++**Start Phase 7 if:**+- Need to support 10+ concurrent instances+- User feedback indicates performance issues+- Deployment cost is concern (want to optimize resource usage)+- Need monitoring/observability for production++**Start Phase 8 if:**+- Managing 5+ instances is tedious via API/CLI+- Need better visibility into instance state+- Want self-service dashboard without CLI expertise+- Running in environment where dashboard access preferred over CLI++---++## Proposal Structure++Each proposal file contains:++1. **Overview** - Summary of phase goals and scope++2. **Detailed Tasks** - For each feature area:+ - Objectives+ - Tasks with sub-items (checkboxes for progress tracking)+ - Implementation strategy+ - Testing approach+ - Acceptance criteria++3. **Success Metrics** - Measurable targets for quality/performance++4. **Timeline Estimate** - Hours per section and total++5. **Deliverables** - Concrete outputs (files, endpoints, docs)++6. **Notes** - Important considerations and gotchas++---++## How to Use++### For Planning+1. Read the relevant proposal document(s)+2. Estimate effort based on task complexity+3. Adjust timeline based on team capacity+4. Update task lists as work progresses++### For Tracking Progress+- Check off items as completed+- Add dates next to completed items+- Update status at top of file+- Create branch for each phase (e.g., `feat/phase-7-perf`)++### For Documentation+- Link to relevant proposal sections in commit messages+- Reference proposal acceptance criteria in PR reviews+- Use deliverables list as checklist before merging++---++## Future Proposal Ideas++### Phase 9: Advanced Agent Coordination+- Session sharing between agents+- Shared instance resources+- Cross-agent communication+- Advanced routing and load balancing++### Phase 10: Production Hardening+- Docker and Kubernetes manifests+- Monitoring integration (Prometheus, Grafana)+- Deployment guides (AWS, GCP, Azure)+- Security hardening (SSL, auth, RBAC)++### Phase 11: Advanced Features+- Browser profile snapshots/restore+- Debugging integration (DevTools proxy)+- Browser extensions support+- Custom Chrome flags per instance++---++## Related Documentation++- `docs/overview.md` - Architecture overview+- `docs/get-started.md` - Quick start guide+- `docs/references/endpoints.md` - API reference+- `TESTING.md` - Testing guide+- `docs/building.md` - Build from source++---++## Questions?++Refer to:+- **Architecture questions** → `docs/architecture/`+- **API usage** → `docs/references/endpoints.md`+- **Deployment** → `docs/guides/`+- **Testing** → `TESTING.md`diff --git a/docs/proposals/phase-7-performance-optimization.md b/docs/proposals/phase-7-performance-optimization.mdnew file mode 100644index 00000000..8e71ac80--- /dev/null+++ b/docs/proposals/phase-7-performance-optimization.md@@ -0,0 +1,287 @@+# Phase 7: Performance Optimization & Scaling++## Overview++Pinchtab multi-instance architecture is production-ready. Phase 7 focuses on optimizing performance, establishing baseline metrics, and validating system behavior under high load.++## Goals++- Establish performance baselines (latency, memory, throughput)+- Optimize critical paths (connection pooling, caching, batch operations)+- Validate scalability (100+ concurrent instances)+- Document resource requirements and limits++---++## Section 1: Benchmarking & Metrics++### Objectives++Capture baseline performance metrics across single and multi-instance scenarios.++### Tasks++#### 1.1 Request Latency Baseline++- [ ] **Measure endpoint latency (p50, p95, p99)**+ - `/navigate` - first request (Chrome init) vs subsequent+ - `/snapshot` - element retrieval time+ - `/actions` (click, type, etc.) - per action cost+ - `/tabs` aggregate - response time with N instances++- [ ] **Tools & Methods**+ - Use `ab` (Apache Bench) for load testing+ - Use `vegeta` (Go HTTP load testing) for sustained load+ - Capture detailed timing with custom harness+ - Document environment: CPU, RAM, disk speed++- [ ] **Acceptance Criteria**+ - Single instance: `/navigate` < 2s (first request)+ - Single instance: `/navigate` < 200ms (subsequent)+ - 10 instances: `/tabs` aggregate < 500ms+ - All operations p99 < 5s++#### 1.2 Memory per Instance++- [ ] **Measure baseline memory footprint**+ - Idle Chrome process (headless)+ - Idle Chrome process (headed)+ - Memory after 10 navigations+ - Memory after 100 navigations+ - Memory leak detection over 1 hour++- [ ] **Tools & Methods**+ - Use `pmap` or `/proc/[pid]/status` on Linux+ - Use `Instruments.app` or `Activity Monitor` on macOS+ - Track RSS (resident set size) and VSZ (virtual size)+ - Profile heap with `pprof`++- [ ] **Acceptance Criteria**+ - Idle instance: < 150 MB RSS+ - 10 instances: < 2 GB total RSS+ - No memory growth > 5 MB/hour idle+ - No memory growth > 20 MB/100 navigations++#### 1.3 Concurrent Instance Limits++- [ ] **Identify breaking points**+ - How many instances can run concurrently? (CPU bound? File descriptor bound? Memory?)+ - How many concurrent requests per instance?+ - Orchestrator overhead as instance count grows++- [ ] **Tools & Methods**+ - Create instances in loop, monitor system metrics+ - Run concurrent navigation requests across instances+ - Use `ulimit` and `/proc/sys/fs/file-max` to check limits+ - Profile orchestrator CPU usage++- [ ] **Acceptance Criteria**+ - 100 instances should be sustainable with 16GB RAM+ - Orchestrator should handle 1000 concurrent requests+ - Port allocation should handle allocation/release in < 10ms+ - No file descriptor leaks (connections cleaned up properly)++---++## Section 2: Optimizations & Improvements++### Objectives++Implement targeted performance improvements based on bottleneck analysis.++### Tasks++#### 2.1 Connection Pooling++- [ ] **HTTP connection reuse**+ - Orchestrator to instances: Keep-Alive connections+ - Bridge handlers: Connection pools for concurrent requests+ - Measure impact on latency (target: 10-20% improvement)++- [ ] **Implementation Strategy**+ - Update `proxyToInstance()` to use persistent HTTP client+ - Configure http.Client with max idle connections, timeouts+ - Consider connection limits per instance++- [ ] **Testing**+ - Baseline: 100 sequential requests to single endpoint+ - After: 100 sequential requests (measure improvement)+ - Concurrent: 50 concurrent requests, measure connection reuse++- [ ] **Acceptance Criteria**+ - Connection pool should reduce latency by 10-20%+ - No connection leaks (verify with `ss -tna | grep ESTABLISHED`)+ - Graceful connection cleanup on instance shutdown++#### 2.2 Cache Improvements++- [ ] **Query result caching**+ - Cache `/tabs` aggregate results (5-10s TTL)+ - Cache `/instances` listing (2-5s TTL)+ - Cache profile data (1-5m TTL)++- [ ] **Implementation Strategy**+ - Add simple in-memory cache with sync.RWMutex+ - Invalidate cache on instance create/delete/update+ - Log cache hits/misses for monitoring++- [ ] **Testing**+ - Measure reduction in aggregation time+ - Verify cache invalidation on changes+ - Test with 50+ instances++- [ ] **Acceptance Criteria**+ - `/tabs` response time with cache < 50ms (vs 500ms without)+ - Cache eviction doesn't break consistency+ - Stale reads acceptable for 5-10s window++#### 2.3 Batch Navigation Support++- [ ] **Bulk navigation endpoint**+ - `POST /instances/{id}/batch-navigate` - navigate multiple URLs+ - `POST /batch-navigate` - navigate across instances++- [ ] **Implementation Strategy**+ - Accept array of URLs+ - Return array of tab IDs+ - Consider sequential vs parallel (with concurrency limits)++- [ ] **Testing**+ - Navigate 50 URLs on single instance+ - Measure latency improvement vs sequential calls+ - Verify all tabs created correctly++- [ ] **Acceptance Criteria**+ - 50-URL batch < 20s (vs 50 * 200ms = 10s sequential + overhead)+ - All tabs successfully created+ - Error handling for partial failures++---++## Section 3: Scaling Tests & Validation++### Objectives++Validate system behavior under realistic production load scenarios.++### Tasks++#### 3.1 Stress Test: 100+ Concurrent Instances++- [ ] **Test scenario**+ - Create 100 instances in rapid sequence+ - Verify all reach "running" status+ - Verify port allocation doesn't fail+ - Verify Chrome initialization completes for all++- [ ] **Monitoring**+ - Track system metrics: CPU, RAM, file descriptors, disk I/O+ - Monitor orchestrator response times+ - Log any errors or warnings++- [ ] **Implementation**+ - Build test script: Create 100 instances, pause, verify all running+ - Run 3x and average results+ - Document peak resource usage++- [ ] **Acceptance Criteria**+ - All 100 instances successfully created+ - No port allocation conflicts+ - CPU usage < 80% during test+ - RAM usage < 12 GB+ - All instances responsive after test++#### 3.2 Multi-Agent Concurrent Load++- [ ] **Test scenario**+ - Simulate 5-10 agents, each using 2-3 instances+ - Each agent navigates different URLs concurrently+ - Run for 5 minutes+ - Measure latency, errors, resource usage++- [ ] **Monitoring**+ - Track latency distribution (p50, p95, p99)+ - Monitor for race conditions or conflicts+ - Verify instance isolation (no data leakage)++- [ ] **Implementation**+ - Build multi-agent test harness+ - Use goroutines to simulate agents+ - Log actions and timing++- [ ] **Acceptance Criteria**+ - p99 latency < 5s for all operations+ - 0 errors from isolation violations+ - CPU usage stable (not growing over time)+ - No connection leaks++#### 3.3 Resource Limits Testing++- [ ] **Test orchestrator limits**+ - What happens at 1000 instances? (Should gracefully degrade or error)+ - What happens with port range exhaustion? (Clear error, not crash)+ - What happens with memory pressure? (Can't create more instances)++- [ ] **Test instance limits**+ - How many tabs per instance? (Chrome limit, usually 500+)+ - What happens over 500 tabs? (Error or performance degradation)+ - How many concurrent requests per instance?++- [ ] **Implementation**+ - Build test for each limit+ - Document expected behavior and error messages+ - Verify graceful degradation++- [ ] **Acceptance Criteria**+ - All limit conditions handled gracefully (no crashes)+ - Clear error messages to users+ - Documentation of limits+ - Possible auto-scaling recommendations++---++## Success Metrics++| Metric | Current | Target | Priority |+|--------|---------|--------|----------|+| `/navigate` latency (p99) | TBD | < 5s | High |+| Instance memory (idle) | TBD | < 150MB | High |+| 100 instances creation time | TBD | < 2 min | High |+| Cache hit ratio | N/A | > 80% | Medium |+| Connection reuse efficiency | N/A | > 90% | Medium |+| Error rate under load | TBD | < 0.1% | High |++---++## Timeline Estimate++- **Benchmarking (3.1)**: 2-3 hours+- **Connection pooling (2.1)**: 1-2 hours+- **Cache improvements (2.2)**: 1-2 hours+- **Batch navigation (2.3)**: 1-2 hours+- **Stress testing (3.1-3.3)**: 2-3 hours+- **Documentation & analysis**: 1-2 hours++**Total: 9-14 hours**++---++## Deliverables++- [ ] `docs/performance-benchmarks.md` - Baseline metrics and analysis+- [ ] `docs/performance-tuning.md` - Optimization recommendations+- [ ] `scripts/benchmark.sh` - Automated benchmarking harness+- [ ] `scripts/stress-test.sh` - 100+ instance stress test+- [ ] `scripts/multi-agent-load.sh` - Concurrent agent simulation+- [ ] Updated `TESTING.md` with performance test procedures+- [ ] Performance dashboard or metrics export (optional)++---++## Notes++- Focus on realistic production scenarios, not synthetic micro-benchmarks+- Profile actual bottlenecks before optimizing (avoid premature optimization)+- Document all findings, even if results are "good enough"+- Consider infrastructure costs (more instances = more resource usage)+- Plan for monitoring/observability from the startdiff --git a/docs/proposals/phase-8-dashboard-ui-enhancements.md b/docs/proposals/phase-8-dashboard-ui-enhancements.mdnew file mode 100644index 00000000..c05ea266--- /dev/null+++ b/docs/proposals/phase-8-dashboard-ui-enhancements.md@@ -0,0 +1,414 @@+# Phase 8: Dashboard UI Enhancements++## Overview++Phase 8 focuses on improving the dashboard user experience with real-time monitoring, batch operations, and better instance navigation. Current dashboard provides basic instance listing; Phase 8 adds production-grade observability and control.++## Goals++- Real-time visibility into instance health and resource usage+- Batch operations for efficient multi-instance management+- Improved navigation and search across large instance sets+- Live debugging and diagnostics capability++---++## Section 1: Instance Monitoring Screen++### Objectives++Provide comprehensive real-time visibility into instance status, resource consumption, and activity.++### Tasks++#### 1.1 Real-Time Chrome Status++- [ ] **Display per-instance Chrome state**+ - Chrome running / not running / initializing+ - Process ID+ - Window handle (for headed instances)+ - Last activity timestamp++- [ ] **Implementation Strategy**+ - Add `/instances/{id}/health/chrome` endpoint (returns {status, pid, active_window, last_activity})+ - Update dashboard UI to poll every 2-5 seconds+ - Show status indicator (green=running, yellow=initializing, red=crashed)+ - Display "last activity" as relative time (e.g., "2 mins ago")++- [ ] **UI Components**+ - Chrome status badge in instance card+ - "Restart Chrome" button for crashed instances+ - Process info popup (PID, memory, CPU)++- [ ] **Testing**+ - Verify status updates reflect actual state+ - Test with crashed Chrome (simulate process kill)+ - Verify "Restart Chrome" brings instance back online++- [ ] **Acceptance Criteria**+ - Status updates within 5 seconds+ - Accurate reflection of Chrome state+ - Graceful handling of process crashes+ - Clear user feedback++#### 1.2 Memory & CPU Usage per Instance++- [ ] **Display resource consumption**+ - RSS (resident set size) in MB+ - CPU usage as percentage+ - Trend over time (sparkline)+ - Resource threshold alerts (warn if > 80% of available)++- [ ] **Implementation Strategy**+ - Add `/instances/{id}/health/resources` endpoint (returns {memory_mb, cpu_percent, trend})+ - Collect metrics every 10-30 seconds+ - Calculate trend from last 10 samples+ - Store metrics in circular buffer (last 100 samples)++- [ ] **Metric Collection Methods**+ - Linux: Parse `/proc/[pid]/stat` and `/proc/[pid]/status`+ - macOS: Use `ps` command or native APIs+ - Include Chrome process + all child processes++- [ ] **UI Components**+ - Memory gauge (0-1000 MB range, color-coded)+ - CPU percentage with trend sparkline+ - Alert badge if threshold exceeded+ - Click to see detailed metrics over time++- [ ] **Testing**+ - Verify metrics accurate compared to system tools (`ps`, `top`)+ - Test with idle instance vs active navigation+ - Verify trend calculation correct+ - Test threshold alerts++- [ ] **Acceptance Criteria**+ - Memory readings within ±5 MB of actual+ - CPU readings within ±2% of actual+ - Metrics updated every 10-30 seconds+ - Alerts triggered correctly at thresholds++#### 1.3 Tab Activity Visualization++- [ ] **Display per-instance tab activity**+ - Tab list with URL, title+ - Last modified time+ - Activity sparkline (show tab usage over time)+ - Highlight most active tab++- [ ] **Implementation Strategy**+ - Extend `/instances/{id}/tabs` to include: `last_modified`, `activity_count`, `last_action`+ - Track tab actions (navigate, click, etc.) and timestamps+ - Calculate activity trend (e.g., actions/minute)+ - Reset counters hourly++- [ ] **UI Components**+ - Tab list in instance details panel+ - Activity sparkline for each tab+ - Hover to see: URL, title, last action, action count+ - Color indicate activity level (green=active, gray=idle)++- [ ] **Testing**+ - Verify activity tracking accuracy+ - Test with multiple concurrent tabs+ - Verify sparkline rendering+ - Test activity reset on schedule++- [ ] **Acceptance Criteria**+ - Activity metrics accurate+ - Sparklines render correctly+ - Performance impact negligible (< 5% CPU increase)+ - Activity data persists across instance lifecycle++---++## Section 2: Batch Operations++### Objectives++Enable efficient management of multiple instances with bulk actions.++### Tasks++#### 2.1 Create Multiple Instances at Once++- [ ] **Bulk instance creation UI**+ - Input: Count, profile name, headless flag+ - e.g., "Create 10 scraping instances (headless)"+ - Progress indicator during creation+ - Confirmation of all created (IDs, ports, status)++- [ ] **Implementation Strategy**+ - New form on "Instances" tab+ - Add `/instances/batch-create` endpoint (accepts {count, profile, headless})+ - Create instances in sequence (avoid overwhelming system)+ - Return array of created instance IDs and ports++- [ ] **UI Components**+ - "Create Batch" button+ - Modal with count/profile/headless selectors+ - Progress bar during creation+ - Success confirmation with download CSV option (IDs, ports)++- [ ] **Error Handling**+ - Handle port exhaustion (stop at max available)+ - Handle instance creation failures (skip, continue with others)+ - Provide detailed error messages++- [ ] **Testing**+ - Create 5 instances in batch+ - Verify all created with correct config+ - Test port exhaustion scenario+ - Verify error handling++- [ ] **Acceptance Criteria**+ - 5 instances created in < 30 seconds+ - All instances reachable and responsive+ - Clear feedback on success/failure+ - CSV export contains all created instances++#### 2.2 Terminate All Instances Button++- [ ] **Bulk shutdown UI**+ - "Terminate All" button with confirmation+ - Optional: Select subset before terminating+ - Progress indicator during shutdown+ - Cleanup verification (no orphaned processes)++- [ ] **Implementation Strategy**+ - Add `/instances/batch-stop` endpoint (stops all or filtered list)+ - Send stop requests in parallel (with concurrency limit)+ - Track completion+ - Verify ports released++- [ ] **UI Components**+ - "Terminate All" button in Instances toolbar+ - Confirmation dialog: "Terminate all {N} instances?"+ - Progress bar with count (e.g., "Stopped 7/10")+ - Success message: "All instances terminated, ports released"++- [ ] **Error Handling**+ - Handle instances that fail to stop (provide manual options)+ - Log any orphaned processes+ - Clear error reporting++- [ ] **Testing**+ - Create 5 instances, terminate all+ - Verify all stopped+ - Verify ports released (can create new instances)+ - Test partial failure scenario++- [ ] **Acceptance Criteria**+ - All instances stopped in < 30 seconds+ - All ports released and available for reuse+ - No orphaned Chrome processes+ - Clear feedback on results++#### 2.3 Port Range Configuration UI++- [ ] **Configure instance port range**+ - Display current range (e.g., 9868-9968)+ - Allow modification (validate: start < end, reasonable range)+ - Show available ports vs allocated+ - Visual indicator of "capacity" (e.g., "48/100 ports allocated")++- [ ] **Implementation Strategy**+ - Add `/config/port-range` endpoint (GET/POST)+ - GET returns: current_start, current_end, allocated_count, total_count+ - POST to change range (with validation)+ - Update dashboard to call GET on load++- [ ] **UI Components**+ - Settings panel with port range inputs+ - Validation error messages+ - Visual progress bar: "48 of 100 ports in use"+ - Warning: "Some instances may be disconnected if range changed while running"+ - "Apply" button (with confirmation if instances running)++- [ ] **Error Handling**+ - Reject invalid ranges (start >= end, out of system limits)+ - Warn if changing while instances running+ - Prevent change if no available capacity for existing instances++- [ ] **Testing**+ - Change port range from 9800-9900 to 9900-10000+ - Verify new instances use new range+ - Test invalid range inputs (rejected)+ - Test changing range with instances running (warn user)++- [ ] **Acceptance Criteria**+ - Port range configurable from UI+ - Validation prevents invalid configurations+ - User can see allocated vs available ports+ - Changes take effect immediately for new instances++---++## Section 3: Better Navigation & UX++### Objectives++Improve usability when managing large numbers of instances.++### Tasks++#### 3.1 Search & Filter Instances++- [ ] **Search and filter capabilities**+ - Search by instance ID, profile name, port+ - Filter by: status (running/stopped/error), headless (true/false)+ - Filter by resource usage (e.g., "memory > 200MB")+ - Combine multiple filters++- [ ] **Implementation Strategy**+ - Frontend filtering (no backend API change needed initially)+ - Add search box to Instances toolbar+ - Add filter button with dropdown/checkboxes+ - Real-time filtering as user types/selects++- [ ] **UI Components**+ - Search input field (with clear button)+ - "Filters" dropdown showing active filters+ - Filter options: Status, Profile, Headless, Memory, CPU+ - Results count: "Showing 12 of 50 instances"+ - "Clear filters" button++- [ ] **Testing**+ - Search for instance ID (partial match)+ - Search for profile name+ - Filter by status+ - Combine multiple filters+ - Verify result count accurate++- [ ] **Acceptance Criteria**+ - Search results instant (< 100ms) for 100+ instances+ - All filter combinations work+ - Results count accurate+ - Clear indication of active filters++#### 3.2 Instance Details Modal++- [ ] **Detailed view for single instance**+ - Full instance info (ID, profile, port, mode, headless)+ - Resource metrics (memory, CPU, trend)+ - Chrome status (PID, window handle)+ - Tab list with activity metrics+ - Action buttons (restart Chrome, restart instance, stop instance)+ - Log viewer (last 50 lines of instance logs)++- [ ] **Implementation Strategy**+ - Create modal component that opens on instance click+ - Combine data from: `/instances/{id}`, `/health/chrome`, `/health/resources`, `/tabs`+ - Add `/instances/{id}/logs` endpoint (returns last N log lines)+ - Update modal every 5 seconds with fresh data++- [ ] **UI Components**+ - Modal with tabs: Overview, Resources, Tabs, Logs+ - Responsive design (readable on small screens)+ - Copy-to-clipboard for IDs/URLs+ - Action buttons with confirmation dialogs++- [ ] **Testing**+ - Open detail modal for multiple instances+ - Verify all data displayed correctly+ - Verify refresh works (data updates)+ - Verify actions work (restart, stop)+ - Test with many tabs (scroll, performance)++- [ ] **Acceptance Criteria**+ - Modal loads in < 1 second+ - Data refreshes every 5 seconds+ - All instance details visible+ - Actions execute correctly+ - Performance acceptable with 50+ tabs++#### 3.3 Live Logs Viewer++- [ ] **Real-time log visualization**+ - Stream logs from orchestrator and instances+ - Filter by log level (info, warning, error, debug)+ - Filter by source (orchestrator, instance ID)+ - Timestamp on each log+ - Autoscroll (can toggle off)+ - Search within logs++- [ ] **Implementation Strategy**+ - Add `/logs` endpoint with SSE (Server-Sent Events) or WebSocket+ - Return stream of {timestamp, level, source, message}+ - Client-side filtering (no backend filter needed initially)+ - Store last 1000 logs in memory++- [ ] **UI Components**+ - New "Logs" tab in main navigation+ - Timeline view with scrollable log list+ - Filter controls: level, source, search term+ - Clear logs button+ - Export logs button (download as text)++- [ ] **Testing**+ - Verify logs stream in real-time+ - Verify filtering works (level, source)+ - Test with high log volume (1000+ logs/minute)+ - Verify autoscroll works+ - Test lag/latency with updates++- [ ] **Acceptance Criteria**+ - Logs appear in < 500ms after event+ - Filtering instant (no noticeable lag)+ - Performance acceptable with 1000+ logs+ - Search works on displayed logs+ - Clear and export functions work++---++## Success Metrics++| Feature | Metric | Target | Priority |+|---------|--------|--------|----------|+| Chrome status | Update latency | < 5s | High |+| Resource metrics | Accuracy | ±5% of actual | High |+| Batch create | Time for 10 instances | < 30s | Medium |+| Search | Response time | < 100ms | Medium |+| Log viewer | Update latency | < 500ms | Low |++---++## Timeline Estimate++- **Chrome status (1.1)**: 2-3 hours+- **Resource metrics (1.2)**: 2-3 hours+- **Tab activity (1.3)**: 1-2 hours+- **Batch create (2.1)**: 2-3 hours+- **Terminate all (2.2)**: 1-2 hours+- **Port range config (2.3)**: 1-2 hours+- **Search & filter (3.1)**: 2-3 hours+- **Details modal (3.2)**: 3-4 hours+- **Live logs (3.3)**: 2-3 hours+- **Integration & testing**: 2-3 hours++**Total: 18-28 hours**++---++## Deliverables++- [ ] Updated dashboard with monitoring screen+- [ ] Monitoring components: Chrome status, resource metrics, tab activity+- [ ] Batch operations: Create multiple, terminate all+- [ ] Port range configuration UI+- [ ] Search and filter system+- [ ] Instance details modal with logs viewer+- [ ] Updated `dashboard/` frontend files+- [ ] New API endpoints: `/health/*`, `/batch-*`, `/logs`, `/config/*`+- [ ] Updated documentation: Dashboard usage guide++---++## Notes++- Prioritize high-visibility metrics (Chrome status, memory usage)+- Use polling initially, upgrade to WebSocket/SSE if performance issues+- Consider mobile responsiveness for remote dashboards+- Log viewer can be phase 8.5 if time-constrained+- Batch operations enable scripting (e.g., "create 100 instances for load test")+- Details modal is foundation for future alerts and anomaly detectiondiff --git a/memory/2026-02-28.md b/memory/2026-02-28.mdindex 0c1f0bf8..ec518042 100644--- a/memory/2026-02-28.md+++ b/memory/2026-02-28.md@@ -870,3 +870,66 @@ go build -o pinchtab ./cmd/pinchtab### Critical Insight for Future SessionsThe multi-instance architecture is **architecturally complete**. All core functionality works. Remaining phases are cosmetic/operational improvements that can be added incrementally based on user demand, not blockers for deployment.+---++## FINAL SESSION UPDATE - TabId Formatting Refinement (9:07 PM → 9:15 PM)++### TabId Hash-Based Conversion Continuation++**Context:** Work began on converting CDP target IDs to hash-based `tab_XXXXXXXX` format across all API endpoints.++**Changes Made (In Progress):**++1. **Added TabIDFromCDPTarget() to IdManager** (`internal/idutil/idutil.go`)+ - New method: `TabIDFromCDPTarget(cdpTargetID string) string`+ - Implementation: Directly hashes CDP target string using existing `hashID("tab", cdpTargetID)` function+ - Deterministic: Same CDP ID always produces same hash-based ID+ - Pattern: Returns `tab_XXXXXXXX` format (8 hex chars)++2. **Updated Handlers Structure** (`internal/handlers/handlers.go`)+ - Added field: `IdMgr *idutil.Manager`+ - Added import: `idutil`+ - Updated `Handlers.New()` to initialize: `h.IdMgr = idutil.NewManager()`++3. **Navigation Handler Update** (~line 122 in `navigation.go`)+ - After getting targetID from bridge navigation+ - Convert: `tabIdStr := h.IdMgr.TabIDFromCDPTarget(targetID)`+ - Return hash-based format in response JSON++4. **Tab Handler Update** (~line 221 in `handlers.go`)+ - `handleTab` function's `tabActionNew` case+ - Convert: `newTabId := h.IdMgr.TabIDFromCDPTarget(newTargetID)`+ - Return hash-based format in response++**Verification:**+- ✅ Build succeeds: `go build -o pinchtab ./cmd/pinchtab`+- ✅ Tests pass: All 195+ unit tests still passing+- ✅ No regressions detected++**Remaining Concerns Identified:**++Need to check all remaining tabId usage locations:+- `actions.go:78` - May need hash conversion for tabId input parameter+- `cookies.go:81` - May need hash conversion for tabId input parameter+- `lock_handler.go` - May need hash conversion for tabId handling+- `lock_shutdown.go` - May need hash conversion for tabId handling+- `stealth.go` - May need tabId conversion if used++**Key Design Question:**+- Are Bridge methods expecting hash-based tabIds (tab_XXXXX) or raw CDP target IDs?+- If Bridge methods use CDP internally: Need reverse mapping function `CDPTargetFromTabID()`+- If Bridge methods now use hash IDs: All callers updated to convert first++**Status:** Awaiting clarification on whether:+1. All endpoints return hash-based tabIds ✅ (navigation & tab handlers done)+2. Clients sending hash-based tabIds need reverse mapping (investigate remaining handlers)+3. Bridge internal API needs consistent ID format (likely CDP, not hash-based)++**Next Steps When Resuming:**+1. Investigate whether other handlers need TabID conversion+2. Determine if reverse mapping `CDPTargetFromTabID()` is needed for request processing+3. Check if Bridge methods expect hash-based or CDP-based tabIds+4. Add integration test verifying hash-based tabIds work end-to-end+5. Complete remaining handler updates+6. Final build and test verification+