High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
docs: consolidate roadmap - Phase 2.5 (port allocation) complete
Marioddf8fff60c13
diff --git a/memory/2026-02-28.md b/memory/2026-02-28.mdindex dde6e25b..dbbb8aca 100644--- a/memory/2026-02-28.md+++ b/memory/2026-02-28.md@@ -28,6 +28,19 @@- **Cleanup**: Removed from orchestrator.Launch(), handlers.go, profiles.js, main.go- **Result**: Simpler codebase, fewer flags to track+### 5. Configurable Port Ranges for Instance Allocation+- **Decision**: Add environment variables for instance port range management+- **Implementation**:+ - `INSTANCE_PORT_START` (default 9868) - first available instance port+ - `INSTANCE_PORT_END` (default 9968) - last available instance port (100 instances max by default)+ - `BRIDGE_PORT` (default 9867) - dashboard/orchestrator port (existing, already configurable)+- **Port Allocator**: New utility to track and allocate ports within range+ - Thread-safe with mutex+ - Auto-detects available ports and skips in-use ones+ - Supports unlimited instances if port range is large enough+ - Can be configured per deployment (Docker, K8s, etc.)+- **Rationale**: Clean separation of dashboard vs instances, enables multi-deployment scenarios+## Architecture Improvements### Instance Hierarchy@@ -65,12 +78,12 @@ Dashboard (9867) manages:## Branch Status- **Branch**: `feat/make-cli-useful`- **Recent commits**:- - `5a15688` - docs: add comprehensive api endpoints reference- - `54ad6e3` - refactor: remove unused BRIDGE_NO_DASHBOARD- - `912a0c6` - fix: handle array response from /instances- - `6725ca8` - style: center empty state on instances/profiles screens- - `807c7f9` - feat: add instances screen to dashboard web ui- - `c8df419` - feat: add instances screen to dashboard web ui+ - `a0b527e` - feat: add configurable port ranges for instance allocation+ - `ddfa0ed` - docs: update memory with Phase 2 completion and Phase 3-5 planning+ - `a4ec3ee` - feat: update handlers to ensure Chrome initialization (Phase 2 Part 2)+ - `56491d2` - feat: implement Chrome initialization (Phase 2 Part 1)+ - `1130a3f` - docs: update to reflect Chrome initialization on instance creation+ - `bdccfbe` - feat: add lazy browser initialization infrastructure to Bridge## Implementation Progress@@ -85,6 +98,38 @@ Dashboard (9867) manages:### ✅ Phase 2: Browser Initialization at Instance Creation (COMPLETED)+### ✅ Phase 2.5: Port Allocation System (COMPLETED - Commit a0b527e)++**What was added:**+- **internal/config/config.go** - Added InstancePortStart and InstancePortEnd to RuntimeConfig+- **internal/orchestrator/ports.go** - New PortAllocator utility (119 lines)+- **internal/orchestrator/orchestrator.go** - Integrated PortAllocator into Orchestrator+- **cmd/pinchtab/cmd_dashboard.go** - Call SetPortRange() during dashboard init++**PortAllocator features:**+- `AllocatePort()` - Finds next available port in configured range+- `ReleasePort(port)` - Marks port as no longer allocated+- `IsAllocated(port)` - Check allocation status+- `AllocatedPorts()` - List all allocated ports+- Thread-safe with sync.Mutex+- Wraps around if reaches end of range+- Actually checks port availability (doesn't just track allocation)++**Configuration:**+- Env vars: INSTANCE_PORT_START, INSTANCE_PORT_END+- File config support: instancePortStart, instancePortEnd in config.json+- Defaults: 9868-9968 (100 instance slots)+- All env vars override file config++**Use Cases Enabled:**+1. Multiple dashboards on different machines/ports+2. Containerized deployments with port ranges+3. Development: 9867 dashboard, 9868-9900 instances+4. Production: 9867 dashboard, 9868-10000 instances (133 instances)+5. Kubernetes: Different port ranges per deployment++### ✅ Phase 2: Browser Initialization at Instance Creation (COMPLETED)+#### Phase 2 Part 1 (Commit 56491d2) - Chrome Init Functions- ✅ Created internal/bridge/init.go with InitChrome() function- ✅ Extracted setupAllocator() from cmd/pinchtab/browser.go@@ -107,28 +152,34 @@ Dashboard (9867) manages:- ✅ All 195+ tests passing- ✅ All pre-commit checks passing-### 📋 Phase 3: Instance-Scoped Endpoints (NEXT - Estimated 3-4 commits)-1. **Create bridge initialization in orchestrator**- - When instance.Launch() creates a new Pinchtab process via runner- - After process starts listening on port, orchestrator must:- a. Allocate Chrome allocator (setupAllocator from browser.go)- b. Start Chrome browser (startChrome with headless/headed flags)- c. Call bridge.SetBrowserContexts() to register the contexts- - Wait for Chrome initialization to complete before returning from /instances POST--2. **Refactor browser.go initialization code**- - Extract setupAllocator(), buildChromeOpts(), startChrome() to package-level functions in internal/bridge- - Or create InitChrome(cfg *config.RuntimeConfig) (*context.Context, *context.Context, error) helper- - Make it callable from orchestrator--3. **Update orchestrator instance launch flow**- - POST /instances endpoint calls orchestrator.Launch()- - orchestrator.Launch() spawns instance process- - orchestrator waits for instance health check- - orchestrator initializes Chrome using the extracted helper- - orchestrator returns initialized instance with Chrome running--### 📋 Phase 3: Instance-Scoped Endpoints (NEXT - Estimated 3-4 commits)+### 📋 Phase 3: Auto-Port Allocation in Orchestrator (NEXT - Estimated 1-2 commits)++**Goal**: Use PortAllocator when launching instances instead of requiring manual port specification++1. **Update orchestrator.Launch()**+ - If port parameter is empty, call `o.portAllocator.AllocatePort()`+ - Return allocated port to caller+ - On instance shutdown/failure, call `o.portAllocator.ReleasePort(port)`++2. **Update orchestrator handlers** (/instances POST endpoint)+ - Accept optional port in request body+ - If port provided, use it (manual override)+ - If not provided, auto-allocate+ - Return allocated port in response++3. **Wire handler to use allocator**+ - orchestrator/handlers.go CreateInstance() should:+ a. Extract port from request (optional)+ b. Call orchestrator.Launch(name, port, headless)+ c. If Launch succeeds, return instance with port++4. **Testing**+ - POST /instances with no port → should auto-allocate (9868)+ - POST /instances 5 times → should get 9868, 9869, 9870, 9871, 9872+ - Verify PortAllocator tracks them+ - Clean shutdown → ports released and reusable++### 📋 Phase 4: Instance-Scoped Endpoints (After Phase 3 - Estimated 3-4 commits)**Key insight**: Each instance already runs on its own port (9868, 9869, etc.) with its own Bridge.The handler routes on that port don't need `/instances/{id}` prefix - just register at `/navigate`, `/snapshot`, etc.@@ -139,57 +190,84 @@ Requests to port 9868 automatically go to that instance's Bridge.- Each port gets its own Bridge instance ✓- Routes on that Bridge's handlers.go are isolated ✓-2. **Backward-compat routes on dashboard (port 9867)**- - Dashboard port 9867 runs its own handlers- - Could keep old `/navigate`, `/snapshot`, etc. routing to orchestrator's default instance- - Or update CLI to use specific instance port directly+2. **Update CLI to use specific instance port**+ - CLI commands should target instance port directly+ - Or query orchestrator for first running instance+ - Dashboard can proxy to instance ports if needed for UI3. **Testing approach**- Create instance via POST http://localhost:9867/instances → returns port (e.g., 9868)- Navigate via POST http://localhost:9868/navigate- - No need for `/instances/{id}` prefix - port already targets instance- - Dashboard can proxy to instance ports if needed for UI+ - Port already targets instance (no path prefix needed)-### 📋 Phase 4: Orchestrator Integration (Phase 3 continuation - Estimated 2-3 commits)+### 📋 Phase 5: Orchestrator Chrome Initialization (After Phase 4 - Estimated 2-3 commits)-**Goal**: Make POST /instances call Chrome initialization+**Goal**: Initialize Chrome when instances are created (POST /instances)-1. **Update orchestrator.Launch() to initialize Chrome**- - After spawning instance process- - After health check passes- - Call bridge.InitChrome() for that instance- - Return initialized instance with Chrome running+1. **Update orchestrator instance launch flow**+ - After instance process starts and passes health check+ - Call bridge.InitChrome() via HTTP request to instance+ - Instance calls bridge.EnsureChrome() which runs InitChrome()+ - Wait for Chrome to fully initialize+ - Return instance with Chrome ready2. **Test POST /instances endpoint**- Create instance with {profile: "work", headless: false}- Verify Chrome window opens (headless=false)- - Verify returned instance has port and status="running"- - Verify subsequent POST /navigate works immediately+ - Verify returned instance has port, status="running", tabs=[]+ - Verify subsequent POST /navigate works immediately (Chrome already running)3. **Test headless instances**- - Create instance with headless=true- - Verify no Chrome window- - Verify Chrome still responds to navigate/snapshot calls+ - Create instance with {headless: true}+ - Verify no Chrome window visible+ - Verify POST /navigate still works (Chrome running in background)++4. **Test auto-port + Chrome init flow**+ - POST /instances (no port) → auto-allocate 9868, start Chrome+ - POST /instances (no port) → auto-allocate 9869, start Chrome+ - Both instances ready immediately+ - No race conditions or port conflicts-### 📋 Phase 5: Testing & Validation (After Phase 4)+### 📋 Phase 6: Testing & Validation (After Phase 5)1. **Test headed vs headless**- - Launch instance with headless=false → Chrome window should appear- - Launch instance with headless=true → No window+ - Create instance {headless: false} → Chrome window appears+ - Create instance {headless: true} → No window, runs in background+ - Both work identically except visibility-2. **Test tab creation timing**- - Create instance → no Chrome running yet- - POST /instances/{id}/navigate → Chrome starts, navigates- - Subsequent requests reuse browser+2. **Multi-instance coordination**+ - Create 3 instances simultaneously+ - Each gets different port (9868, 9869, 9870)+ - Each has independent Chrome with own cookies/history+ - No state leakage between instances3. **Multi-agent isolation**- - Agent A: instance-a-work (headless=false, profile=work)- - Agent B: instance-b-scrape (headless=true, profile=scraping)- - Verify no state leakage between instances--4. **Test lazy init**- - Create 10 instances → none start Chrome- - POST navigate to 3 of them → only 3 have running Chrome- - POST navigate to remaining 7 → browsers start on first request+ - Agent A: targets http://localhost:9868/navigate (work profile)+ - Agent B: targets http://localhost:9869/navigate (scraping profile)+ - Agent C: targets http://localhost:9870/navigate (another profile)+ - Verify each agent only sees its own instance's data++4. **Dashboard-to-instance communication**+ - Dashboard (9867) lists instances+ - Dashboard proxies requests to instance ports+ - UI shows instance status, tabs, resources+ - Can terminate instances and release ports++### 📋 Phase 7: CLI & Documentation (After Phase 6)++1. **Update CLI to use auto-allocation**+ - `pinchtab instances new --profile work --headless=false`+ - Should create instance, auto-allocate port, start Chrome+ - Print port to stdout for use in scripts++2. **Update documentation**+ - docs/get-started.md: Multi-instance workflow+ - docs/references/endpoints.md: Auto-allocation details+ - docs/architecture/: Instance lifecycle diagram++3. **Add example workflows**+ - Multi-agent scraping with dedicated instances+ - Load distribution across headless instances+ - Clean startup/shutdown with port cleanup## Key Insights