The agent that turns websites into APIs!
cleanup: simplify agent mode providers
Kalil Bouzigues8b94c847a5d9
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 440218c..b96273a 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,11 @@ All notable changes to this project will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [Unreleased]++### Removed+- **`browser-use` and `stagehand` agent providers**: Both have been retired due to upstream churn. Agent mode now offers only `auto` (Playwright MCP) and `chrome-mcp` (Chrome DevTools MCP). Configs with `agent_provider` set to `browser-use` or `stagehand` migrate to `auto`; the `browser_use_model` and `stagehand_model` keys are dropped silently. The `[agent]` optional-dependency extra is removed.+## [0.7.1] - 2026-04-06### Fixeddiff --git a/README.md b/README.mdindex 99db5ea..f475c2a 100644--- a/README.md+++ b/README.md@@ -48,7 +48,7 @@ No more manual reverse engineering—just browse, capture, and get clean API cod## ✨ Features- 🌐 **Browser Automation**: Built on Playwright with stealth mode for realistic browsing-- 🤖 **Autonomous Agent Mode**: Fully automated browser interaction using AI agents (auto mode with MCP, browser-use, stagehand)+- 🤖 **Autonomous Agent Mode**: Fully automated browser interaction using AI agents via MCP (Playwright MCP or Chrome DevTools MCP)- 📊 **HAR Recording**: Captures all network traffic in HTTP Archive format- 🧠 **AI-Powered Generation**: Uses Claude 4.6 to analyze traffic and generate clean Python code- 🔍 **Collector Mode**: Data collection with automatic JSON/CSV export@@ -68,21 +68,12 @@ No more manual reverse engineering—just browse, capture, and get clean API cod### Using uv (recommended)```bash-# Basic installationuv tool install reverse-api-engineer--# With agent mode support (includes browser-use with HAR recording)-uv tool install 'reverse-api-engineer[agent]' --with 'browser-use @ git+https://github.com/browser-use/browser-use.git@49a345fb19e9f12befc5cc1658e0033873892455'```### Using pip```bash-# Basic installationpip install reverse-api-engineer--# With agent mode support-pip install 'reverse-api-engineer[agent]'-pip install git+https://github.com/browser-use/browser-use.git@49a345fb19e9f12befc5cc1658e0033873892455```### Post-installation@@ -174,9 +165,8 @@ Fully automated browser interaction using AI agents:**Agent Provider Options:**-- **auto** (default): Uses MCP-based browser automation with Claude Agent SDK & Opencode. Combines browser control and real-time reverse engineering in a single workflow. No additional installation required beyond the base package.-- **browser-use**: Uses browser-use library for browser automation. Requires installation with `[agent]` extra and browser-use from specific git commit (includes HAR recording support).-- **stagehand**: Uses Stagehand for browser automation with Computer Use models.+- **auto** (default): Uses Playwright MCP browser automation with Claude Agent SDK & Opencode. Combines browser control and real-time reverse engineering in a single workflow.+- **chrome-mcp**: Uses [Chrome DevTools MCP](https://www.npmjs.com/package/chrome-devtools-mcp) to drive your real Chrome browser (with existing sessions, cookies, and auth). Requires Chrome 146+ and Node.js 20.19+.Change agent provider in `/settings` → "agent provider".@@ -252,7 +242,6 @@ Settings stored in `~/.reverse-api/config.json`:```json{"agent_provider": "auto",- "browser_use_model": "bu-llm","claude_code_model": "claude-sonnet-4-6","collector_model": "claude-sonnet-4-6","opencode_model": "claude-sonnet-4-6",@@ -260,8 +249,7 @@ Settings stored in `~/.reverse-api/config.json`:"output_dir": null,"output_language": "python","real_time_sync": true,- "sdk": "claude",- "stagehand_model": "openai/computer-use-preview-2025-03-11"+ "sdk": "claude"}```@@ -284,32 +272,12 @@ If you use Opencode, look at the [models](https://models.dev).Configure AI agents for autonomous browser automation.**Agent Providers:**-- **auto** (default): MCP-based browser automation with real-time reverse engineering. Uses Claude Agent SDK with browser MCP tools. Combines browser control and API reverse engineering in a single unified workflow. Works with Claude SDK (default) or OpenCode SDK.-- **browser-use**: Supports Browser-Use LLM, OpenAI, and Google models. Requires installation with `[agent]` extra.-- **stagehand**: Supports OpenAI and Anthropic Computer Use models--**Agent Models:**--**Browser-Use Provider:**-- `bu-llm` (default) - Requires `BROWSER_USE_API_KEY`-- `openai/gpt-4`, `openai/gpt-3.5-turbo` - Requires `OPENAI_API_KEY`-- `google/gemini-pro`, `google/gemini-1.5-pro` - Requires `GOOGLE_API_KEY`+- **auto** (default): Playwright MCP browser automation with real-time reverse engineering. Uses Claude Agent SDK with browser MCP tools. Combines browser control and API reverse engineering in a single unified workflow. Works with Claude SDK (default) or OpenCode SDK.+- **chrome-mcp**: Drives your real Chrome browser via [Chrome DevTools MCP](https://www.npmjs.com/package/chrome-devtools-mcp). Useful when you need existing sessions, cookies, or auth. Requires Chrome 146+ and Node.js 20.19+; enable auto-connect at `chrome://inspect/#remote-debugging`.-**Stagehand Provider (Computer Use only):**-- `openai/computer-use-preview-2025-03-11` - Requires `OPENAI_API_KEY`-- `anthropic/claude-sonnet-4-6-20260301` - Requires `ANTHROPIC_API_KEY`-- `anthropic/claude-haiku-4-5-20251001` - Requires `ANTHROPIC_API_KEY`-- `anthropic/claude-opus-4-6-20260301` - Requires `ANTHROPIC_API_KEY`--**Setting API Keys:**-```bash-export BROWSER_USE_API_KEY="your-api-key" # For Browser-Use-export OPENAI_API_KEY="your-api-key" # For OpenAI models-export ANTHROPIC_API_KEY="your-api-key" # For Anthropic models-export GOOGLE_API_KEY="your-api-key" # For Google models-```+The agent's reasoning model is the same as the SDK model — see [Model Selection](#model-selection).-Change in `/settings` → "agent provider" and "agent model"+Change in `/settings` → "agent provider"### SDK Selectiondiff --git a/pyproject.toml b/pyproject.tomlindex 7aae590..df67363 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -36,11 +36,6 @@ dependencies = [][project.optional-dependencies]-agent = [- # browser-use must be installed separately from git (PyPI doesn't allow git dependencies)- # See installation instructions in README- "stagehand"-]pricing = ["litellm>=1.83.0",]diff --git a/src/reverse_api/browser.py b/src/reverse_api/browser.pyindex af04dbe..548f895 100644--- a/src/reverse_api/browser.py+++ b/src/reverse_api/browser.py@@ -1,14 +1,10 @@"""Browser management with Playwright for HAR recording."""-import asyncioimport ioimport json-import logging-import osimport randomimport signalimport sys-from contextlib import redirect_stderrfrom pathlib import Pathfrom playwright.sync_api import Browser, BrowserContext, Page, sync_playwright@@ -16,9 +12,7 @@ from playwright_stealth import Stealthfrom rich.console import Consolefrom rich.status import Status-from . import __version__from .action_recorder import ActionRecorder, RecordedAction-from .tui import ERROR_CTAfrom .utils import get_har_dir, get_timestampconsole = Console()@@ -27,15 +21,6 @@ console = Console()_null_stderr = io.StringIO()-def _suppress_stagehand_logs():- """Suppress all Stagehand-related logs. Call before importing stagehand."""- for name in ["stagehand", "agent", "browserbase"]:- logger = logging.getLogger(name)- logger.setLevel(logging.CRITICAL + 100)- logger.propagate = False- logger.handlers = [logging.NullHandler()]--def _null_logger(message: dict) -> None:"""Null logger that discards all messages."""pass@@ -677,567 +662,3 @@ class ManualBrowser:return self.har_path--def parse_agent_model(agent_model: str, agent_provider: str = "browser-use") -> tuple[str, str | None]:- """Parse agent_model string into provider and model name.-- Args:- agent_model: Model string in format "bu-llm" or "{provider}/{model_name}"- agent_provider: Agent provider ("browser-use" or "stagehand")-- Returns:- Tuple of (provider, model_name) where model_name is None for "bu-llm"-- Examples:- "bu-llm" -> ("bu-llm", None)- "openai/gpt-4" -> ("openai", "gpt-4")- "google/gemini-pro" -> ("google", "gemini-pro")- """- if agent_model == "bu-llm":- return ("bu-llm", None)-- if "/" in agent_model:- parts = agent_model.split("/", 1)- if len(parts) == 2:- provider, model_name = parts- provider = provider.lower()-- if agent_provider == "stagehand":- openai_cua_models = [- "computer-use-preview-2025-03-11",- ]-- anthropic_cua_models = [- "claude-sonnet-4-6-20260301",- "claude-haiku-4-5-20251001",- "claude-opus-4-6-20260301",- ]-- if provider == "openai":- if model_name not in openai_cua_models:- raise ValueError(f"Unsupported OpenAI model: {model_name}. Supported OpenAI CUA models: {', '.join(openai_cua_models)}")- elif provider == "anthropic":- if model_name not in anthropic_cua_models:- raise ValueError(- f"Unsupported Anthropic model: {model_name}. Supported Anthropic CUA models: {', '.join(anthropic_cua_models)}"- )- else:- raise ValueError(- f"Stagehand only supports OpenAI and Anthropic Computer Use models. "- f"Got: {agent_model}. Expected format: 'openai/{{model}}' or 'anthropic/{{model}}'"- )-- return (provider, model_name)-- raise ValueError(f"Invalid agent_model format: {agent_model}. Expected 'bu-llm' or '{{provider}}/{{model_name}}' (e.g., 'openai/gpt-4')")---def get_required_api_key(provider: str, agent_provider: str = "browser-use") -> tuple[str, str]:- """Get the required API key environment variable name for a provider.-- Args:- provider: Provider name ("bu-llm", "openai", "google", or "anthropic")- agent_provider: Agent provider ("browser-use" or "stagehand")-- Returns:- Tuple of (env_var_name, display_name)-- Raises:- ValueError: If provider is not supported- """- if agent_provider == "stagehand":- if provider == "anthropic":- return ("ANTHROPIC_API_KEY", "Anthropic API key")- else:- return ("OPENAI_API_KEY", "OpenAI API key")-- provider_map = {- "bu-llm": ("BROWSER_USE_API_KEY", "Browser-Use API key"),- "openai": ("OPENAI_API_KEY", "OpenAI API key"),- "google": ("GOOGLE_API_KEY", "Google API key"),- "anthropic": ("ANTHROPIC_API_KEY", "Anthropic API key"),- }-- if provider not in provider_map:- supported = ", ".join(provider_map.keys())- raise ValueError(f"Unsupported provider: {provider}. Supported providers: {supported}")-- return provider_map[provider]---def validate_api_key(provider: str, agent_provider: str = "browser-use") -> tuple[bool, str | None]:- """Validate that the required API key exists for the provider.-- Args:- provider: Provider name ("bu-llm", "openai", "google", or "anthropic")- agent_provider: Agent provider ("browser-use" or "stagehand")-- Returns:- Tuple of (is_valid, error_message)- """- try:- env_var_name, display_name = get_required_api_key(provider, agent_provider)- api_key = os.getenv(env_var_name)-- if not api_key:- return (- False,- f"Missing {display_name}. Please set the {env_var_name} environment variable.",- )-- return (True, None)- except ValueError as e:- return (False, str(e))---class AgentBrowser:- """Manages autonomous browser session with browser-use or stagehand and HAR recording.-- Supports two agent providers:- - browser-use: Uses browser-use library with multiple LLM providers- - stagehand: Uses Stagehand library with OpenAI Computer Use models only- """-- def __init__(- self,- run_id: str,- prompt: str,- output_dir: str | None = None,- timeout: int = 300,- browser_use_model: str = "bu-llm",- stagehand_model: str = "openai/computer-use-preview-2025-03-11",- agent_provider: str = "browser-use",- start_url: str | None = None,- ):- """Initialize agent browser."""- self.run_id = run_id- self.prompt = prompt- self.output_dir = output_dir- self.timeout = timeout- self.browser_use_model = browser_use_model- self.stagehand_model = stagehand_model- self.agent_provider = agent_provider- self.start_url = start_url-- # Setup paths- self.har_dir = get_har_dir(run_id, output_dir)- self.har_path = self.har_dir / "recording.har"- self.metadata_path = self.har_dir / "metadata.json"-- self._start_time: str | None = None- self._cdp_port = 9222-- def _save_metadata(self, end_time: str, result: dict | None = None) -> None:- """Save run metadata to JSON file."""- # Select the appropriate model based on agent_provider- agent_model = self.stagehand_model if self.agent_provider == "stagehand" else self.browser_use_model-- metadata = {- "run_id": self.run_id,- "prompt": self.prompt,- "mode": "agent",- "agent_provider": self.agent_provider,- "agent_model": agent_model,- "browser_use_model": self.browser_use_model,- "stagehand_model": self.stagehand_model,- "start_time": self._start_time,- "end_time": end_time,- "har_file": str(self.har_path),- "start_url": self.start_url,- "result": result,- }- with open(self.metadata_path, "w") as f:- json.dump(metadata, f, indent=2)-- async def _run_with_har_capture(self, cdp_url: str = None) -> dict:- """Run agent with HAR recording. Routes to browser-use or stagehand based on agent_provider."""- if self.agent_provider == "stagehand":- return await self._run_with_stagehand()- else:- return await self._run_with_browser_use()-- async def _run_with_browser_use(self) -> dict:- """Run agent with HAR recording via browser-use's built-in HAR capture."""- import logging- import os-- # Set before importing to ensure it takes effect- os.environ["BROWSER_USE_LOGGING_LEVEL"] = "WARNING"- os.environ["BROWSER_USE_LOGGING_LEVEL"] = "WARNING"-- # Import browser-use after setting environment variable- try:- from browser_use import Agent, Browser, ChatBrowserUse-- # Suppress all browser-use loggers after import- def suppress_browser_use_logs():- for logger_name in [- "browser_use",- "Agent",- "BrowserSession",- "service",- "tools",- ]:- logger = logging.getLogger(logger_name)- logger.setLevel(logging.CRITICAL)- logger.handlers.clear()- logger.addHandler(logging.NullHandler())- logger.propagate = False-- suppress_browser_use_logs()- except ImportError:- result = {- "success": False,- "message": None,- "error": "browser-use is required for agent mode. Install it with: pip install git+https://github.com/browser-use/browser-use.git@49a345fb19e9f12befc5cc1658e0033873892455",- }- console.print(f" [red]error:[/red] {result['error']}")- console.print(f" [dim]{ERROR_CTA}[/dim]")- return result-- result = {"success": False, "message": None, "error": None}- browser = None-- try:- # Parse agent model and validate API key- try:- provider, model_name = parse_agent_model(self.browser_use_model, self.agent_provider)- except ValueError as e:- result["error"] = str(e)- console.print(f" [red]error:[/red] {e}")- console.print(f" [dim]{ERROR_CTA}[/dim]")- return result-- # Validate API key- is_valid, error_msg = validate_api_key(provider, self.agent_provider)- if not is_valid:- result["error"] = error_msg- console.print(f" [red]error:[/red] {error_msg}")- console.print(f" [dim]{ERROR_CTA}[/dim]")- return result-- # Create appropriate LLM instance- env_var_name, _ = get_required_api_key(provider, self.agent_provider)- api_key = os.getenv(env_var_name)-- if provider == "bu-llm":- llm = ChatBrowserUse(api_key=api_key)- elif provider == "openai":- try:- from browser_use import ChatOpenAI-- llm = ChatOpenAI(model=model_name, api_key=api_key)- except ImportError:- result["error"] = "Failed to initialize OpenAI LLM"- console.print(f" [red]error:[/red] {result['error']}")- console.print(f" [dim]{ERROR_CTA}[/dim]")- return result- elif provider == "google":- try:- from browser_use import ChatGoogle-- llm = ChatGoogle(model=model_name, api_key=api_key)- except ImportError:- result["error"] = "Failed to initialize Google LLM"- console.print(f" [red]error:[/red] {result['error']}")- console.print(f" [dim]{ERROR_CTA}[/dim]")- return result- else:- result["error"] = f"Unsupported provider: {provider}"- console.print(f" [red]error:[/red] {result['error']}")- console.print(f" [dim]{ERROR_CTA}[/dim]")- return result-- suppress_browser_use_logs()-- console.print(" [dim]starting browser with har...[/dim]")- browser = Browser(- record_har_path=str(self.har_path),- record_har_mode="full",- record_har_content="embed",- )-- suppress_browser_use_logs()-- await browser.start()-- suppress_browser_use_logs()-- console.print(" [dim]browser started[/dim]")-- task = self.prompt-- agent = Agent(task=task, llm=llm, browser=browser)-- suppress_browser_use_logs()-- agent_result = await agent.run()-- # Extract final result using browser-use's built-in method- final_message = None- if agent_result and hasattr(agent_result, "final_result"):- final_result_attr = agent_result.final_result- final_message = final_result_attr() if callable(final_result_attr) else final_result_attr-- result["success"] = True- result["message"] = final_message or "Task completed"-- except Exception as e:- result["error"] = str(e)- console.print(f" [yellow]agent error: {e}[/yellow]")- finally:- if browser:- try:- await browser.stop()- # Check if HAR was created- if self.har_path.exists():- har_size = self.har_path.stat().st_size- console.print(f" [dim]har saved: {har_size} bytes[/dim]")- else:- console.print(" [dim]har not saved by browser-use[/dim]")- except Exception as e:- console.print(f" [dim]browser stop error: {e}[/dim]")-- return result-- async def _run_with_stagehand(self) -> dict:- """Run agent with HAR recording via Stagehand's built-in HAR capture."""- _suppress_stagehand_logs()-- try:- from stagehand import Stagehand- except ImportError:- return {- "success": False,- "message": None,- "error": "stagehand is required. Install with: pip install 'reverse-api-engineer[agent]'",- }-- result = {"success": False, "message": None, "error": None}- stagehand = None-- try:- try:- provider, model_name = parse_agent_model(self.stagehand_model, self.agent_provider)- except ValueError as e:- result["error"] = str(e)- console.print(f" [red]error:[/red] {e}")- console.print(f" [dim]{ERROR_CTA}[/dim]")- return result-- is_valid, error_msg = validate_api_key(provider, self.agent_provider)- if not is_valid:- result["error"] = error_msg- console.print(f" [red]error:[/red] {error_msg}")- console.print(f" [dim]{ERROR_CTA}[/dim]")- return result-- console.print(" [dim]starting stagehand with har...[/dim]")-- env_var_name, _ = get_required_api_key(provider, self.agent_provider)- api_key = os.getenv(env_var_name)-- with redirect_stderr(_null_stderr):- stagehand = Stagehand(env="LOCAL", verbose=0, logger=_null_logger)- await stagehand.init()-- # Set up HAR recording using Playwright's route_from_har- # Create empty HAR file first, then use route_from_har with update=True to record- if hasattr(stagehand, "context") and stagehand.context:- self.har_dir.mkdir(parents=True, exist_ok=True)-- import json-- empty_har = {- "log": {- "version": "1.2",- "creator": {- "name": "reverse-api-engineer",- "version": __version__,- },- "pages": [],- "entries": [],- }- }-- with open(self.har_path, "w") as f:- json.dump(empty_har, f)-- # Use route_from_har with update=True to record all network traffic- try:- await stagehand.context.route_from_har(- str(self.har_path),- update=True, # Records new HAR file- not_found="abort", # Abort requests not in HAR (but we have empty HAR)- )- except Exception as e:- console.print(f" [dim]warning: could not set up HAR recording: {e}[/dim]")-- agent_options = {- "api_key": api_key,- }-- with redirect_stderr(_null_stderr):- if provider == "anthropic":- agent = stagehand.agent(- provider="anthropic",- model=model_name,- options=agent_options,- )- else:- # OpenAI (default for stagehand)- agent = stagehand.agent(- provider="openai",- model=model_name,- options=agent_options,- )-- console.print(" [dim]stagehand started[/dim]")-- # Navigate to start URL first if provided (best practice per Stagehand docs)- if self.start_url:- # Stagehand agent has access to the page, navigate before executing- # We'll include the URL in the instruction if agent doesn't expose page directly- task_instruction = f"{self.prompt}\n\nStart URL: {self.start_url}"- else:- task_instruction = self.prompt-- # Execute the task- with redirect_stderr(_null_stderr):- agent_result = await agent.execute(- {- "instruction": task_instruction,- "maxSteps": self.timeout // 10, # Convert timeout seconds to reasonable step limit- }- )-- # Extract result from agent_result- # According to Stagehand v2 docs, when using object form, result has:- # - success: boolean indicating if task completed- # - result/text: the actual result content- final_message = None- task_success = False-- if agent_result:- if isinstance(agent_result, dict):- task_success = agent_result.get("success", False)- final_message = agent_result.get("message") or agent_result.get("result") or agent_result.get("text")- elif hasattr(agent_result, "success"):- task_success = agent_result.success- # Only extract message field, ignore actions, usage, etc.- if hasattr(agent_result, "message") and agent_result.message:- final_message = str(agent_result.message)- elif hasattr(agent_result, "result"):- final_message = str(agent_result.result)- elif hasattr(agent_result, "text"):- final_message = agent_result.text- else:- # Fallback: try to get message if it exists as attribute- final_message = getattr(agent_result, "message", "Task completed")- elif isinstance(agent_result, str):- # String result (simple form)- final_message = agent_result- task_success = True # Assume success if string returned- elif hasattr(agent_result, "message") and agent_result.message:- # Object with message attribute- final_message = str(agent_result.message)- task_success = getattr(- agent_result,- "success",- getattr(agent_result, "completed", True),- )- elif hasattr(agent_result, "result"):- final_message = str(agent_result.result)- task_success = True- elif hasattr(agent_result, "text"):- final_message = agent_result.text- task_success = True- else:- # Last resort: try message attribute, otherwise use default- final_message = getattr(agent_result, "message", "Task completed")- task_success = True-- result["success"] = task_success- result["message"] = final_message or "Task completed"-- except Exception as e:- result["error"] = str(e)- console.print(f" [yellow]stagehand error: {e}[/yellow]")- finally:- if stagehand:- try:- with redirect_stderr(_null_stderr):- await stagehand.close()- except Exception:- pass-- # Check if HAR file was created and has content- if self.har_path.exists():- har_size = self.har_path.stat().st_size- if har_size > 100: # HAR file should have some content- console.print(f" [dim]har saved: {har_size} bytes[/dim]")- else:- console.print(" [dim]har file exists but appears empty[/dim]")- else:- console.print(" [dim]har not saved[/dim]")-- return result-- def start(self) -> Path:- """Start agent execution with HAR recording. Returns HAR path when done."""- self._start_time = get_timestamp()-- console.print(" [dim]agent mode starting...[/dim]")- console.print(f" [dim]━[/dim] [white]{self.run_id}[/white]")- console.print(f" [dim]task[/dim] [white]{self.prompt}[/white]")- console.print()-- result = None-- try:- result = asyncio.run(self._run_with_har_capture())-- if result.get("success"):- console.print(" [green]agent task completed[/green]")- msg = result.get("message", "").strip()- if msg:- if len(msg) > 500:- msg = msg[:500] + "..."- console.print(f" [dim]result:[/dim]\n [white]{msg}[/white]")- else:- error = result.get("error", "unknown error")- console.print(f" [yellow]agent error: {error}[/yellow]")-- except Exception as e:- console.print(f" [red]error: {e}[/red]")- console.print(f" [dim]{ERROR_CTA}[/dim]")- result = {"error": str(e)}-- self._save_metadata(get_timestamp(), result)- console.print(" [dim]metadata synced[/dim]")-- return self.har_path---def run_agent_browser(- run_id: str,- prompt: str,- output_dir: str | None = None,- timeout: int = 300,- browser_use_model: str = "bu-llm",- stagehand_model: str = "openai/computer-use-preview-2025-03-11",- agent_provider: str = "browser-use",- start_url: str | None = None,-) -> Path:- """Run agent browser with HAR recording."""- agent_browser = AgentBrowser(- run_id=run_id,- prompt=prompt,- output_dir=output_dir,- timeout=timeout,- browser_use_model=browser_use_model,- stagehand_model=stagehand_model,- agent_provider=agent_provider,- start_url=start_url,- )- return agent_browser.start()diff --git a/src/reverse_api/cli.py b/src/reverse_api/cli.pyindex 8c6f32d..7716f9d 100644--- a/src/reverse_api/cli.py+++ b/src/reverse_api/cli.py@@ -17,7 +17,7 @@ from questionary import Choicefrom rich.console import Consolefrom . import __version__-from .browser import ManualBrowser, run_agent_browser+from .browser import ManualBrowserfrom .config import ConfigManagerfrom .engineer import run_reverse_engineeringfrom .messages import MessageStore@@ -598,7 +598,6 @@ def handle_settings(mode_color=THEME_PRIMARY):# Settings menu (sorted alphabetically)choices = [Choice(title="Agent Provider", value="agent_provider"),- Choice(title="Browser-Use Model", value="browser_use_model"),Choice(title="Claude Code Model", value="claude_code_model"),Choice(title="Copilot Model", value="copilot_model"),Choice(title="OpenCode Model", value="opencode_model"),@@ -607,7 +606,6 @@ def handle_settings(mode_color=THEME_PRIMARY):Choice(title="Output Language", value="output_language"),Choice(title="Real-time Sync", value="real_time_sync"),Choice(title="SDK", value="sdk"),- Choice(title="Stagehand Model", value="stagehand_model"),Choice(title="Back", value="back"),]@@ -698,8 +696,6 @@ def handle_settings(mode_color=THEME_PRIMARY):provider_choices = [Choice(title="auto (Playwright MCP)", value="auto"),Choice(title="chrome-mcp (Chrome DevTools MCP)", value="chrome-mcp"),- Choice(title="browser-use", value="browser-use"),- Choice(title="stagehand", value="stagehand"),Choice(title="back", value="back"),]provider = questionary.select(@@ -796,83 +792,6 @@ def handle_settings(mode_color=THEME_PRIMARY):config_manager.set("opencode_model", new_model)console.print(f" [dim]updated[/dim] opencode model: {new_model}\n")- elif action == "browser_use_model":- from .browser import parse_agent_model-- current = config_manager.get("browser_use_model", "bu-llm")- instruction = "(Format: 'bu-llm' or 'provider/model', e.g., 'openai/gpt-4')"-- new_model = questionary.text(- " > browser-use model",- default=current or "bu-llm",- instruction=instruction,- qmark="",- style=questionary.Style(- [- ("question", f"fg:{THEME_SECONDARY}"),- ("instruction", f"fg:{THEME_DIM} italic"),- ]- ),- ).ask()- if new_model is not None:- new_model = new_model.strip()- if not new_model:- console.print(" [yellow]error:[/yellow] browser-use model cannot be empty\n")- else:- # Validate format for browser-use- try:- parse_agent_model(new_model, "browser-use")- config_manager.set("browser_use_model", new_model)- console.print(f" [dim]updated[/dim] browser-use model: {new_model}\n")- except ValueError as e:- console.print(f" [yellow]error:[/yellow] {e}\n")- console.print(- " [dim]Valid formats:[/dim]\n"- " [dim] - bu-llm[/dim]\n"- " [dim] - openai/model_name (e.g., openai/gpt-4)[/dim]\n"- " [dim] - google/model_name (e.g., google/gemini-pro)[/dim]\n"- )-- elif action == "stagehand_model":- from .browser import parse_agent_model-- current = config_manager.get("stagehand_model", "openai/computer-use-preview-2025-03-11")- instruction = (- "(Format: 'openai/model' or 'anthropic/model', e.g., 'openai/computer-use-preview-2025-03-11' or 'anthropic/claude-sonnet-4-6-20260301')"- )-- new_model = questionary.text(- " > stagehand model",- default=current or "openai/computer-use-preview-2025-03-11",- instruction=instruction,- qmark="",- style=questionary.Style(- [- ("question", f"fg:{THEME_SECONDARY}"),- ("instruction", f"fg:{THEME_DIM} italic"),- ]- ),- ).ask()- if new_model is not None:- new_model = new_model.strip()- if not new_model:- console.print(" [yellow]error:[/yellow] stagehand model cannot be empty\n")- else:- # Validate format for stagehand- try:- parse_agent_model(new_model, "stagehand")- config_manager.set("stagehand_model", new_model)- console.print(f" [dim]updated[/dim] stagehand model: {new_model}\n")- except ValueError as e:- console.print(f" [yellow]error:[/yellow] {e}\n")- console.print(- " [dim]Valid formats for stagehand:[/dim]\n"- " [dim] - openai/computer-use-preview-2025-03-11[/dim]\n"- " [dim] - anthropic/claude-sonnet-4-6-20260301[/dim]\n"- " [dim] - anthropic/claude-haiku-4-5-20251001[/dim]\n"- " [dim] - anthropic/claude-opus-4-6-20260301[/dim]\n"- )-elif action == "real_time_sync":current = config_manager.get("real_time_sync", True)sync_choices = [@@ -1366,116 +1285,15 @@ def run_agent_capture(prompt=None, url=None, reverse_engineer=False, model=None,console.print(" [dim]tip:[/dim] @record-only <prompt> - e.g., @record-only navigate checkout flow")return- run_id = generate_run_id()- timestamp = get_timestamp()- sdk = config_manager.get("sdk", "claude")-- # Get agent models and provider from config- browser_use_model = config_manager.get("browser_use_model", "bu-llm")- stagehand_model = config_manager.get("stagehand_model", "openai/computer-use-preview-2025-03-11")- agent_provider = config_manager.get("agent_provider", "browser-use")-- if agent_provider in ("auto", "chrome-mcp"):- return run_auto_capture(- prompt=prompt,- url=url,- model=model,- output_dir=output_dir,- agent_provider=agent_provider,- )-- # Record initial session- session_manager.add_run(- run_id=run_id,+ agent_provider = config_manager.get("agent_provider", "auto")+ return run_auto_capture(prompt=prompt,- timestamp=timestamp,url=url,model=model,- mode="agent", # Track mode in history- sdk=sdk,- paths={"har_dir": str(get_har_dir(run_id, output_dir))},+ output_dir=output_dir,+ agent_provider=agent_provider,)- # Run agent browser- try:- har_path = run_agent_browser(- run_id=run_id,- prompt=prompt,- output_dir=output_dir,- browser_use_model=browser_use_model,- stagehand_model=stagehand_model,- agent_provider=agent_provider,- start_url=url,- )-- # Optionally run reverse engineering- if reverse_engineer:- engineer_prompt = prompt- try:- wants_new_prompt = questionary.confirm(- " > new prompt for engineer?",- default=False,- qmark="",- style=questionary.Style(- [- ("question", f"fg:{THEME_SECONDARY}"),- ("instruction", f"fg:{THEME_DIM} italic"),- ]- ),- ).ask()-- if wants_new_prompt is None:- raise KeyboardInterrupt-- if wants_new_prompt:- new_prompt = questionary.text(- " > engineer prompt",- instruction="(Enter to use original)",- default="",- qmark="",- style=questionary.Style(- [- ("question", f"fg:{THEME_SECONDARY}"),- ("instruction", f"fg:{THEME_DIM} italic"),- ]- ),- ).ask()-- if new_prompt is None:- raise KeyboardInterrupt-- if new_prompt and new_prompt.strip():- engineer_prompt = new_prompt.strip()- except KeyboardInterrupt:- pass-- result = run_engineer(- run_id=run_id,- har_path=har_path,- prompt=engineer_prompt,- model=model,- output_dir=output_dir,- )- if result:- sdk = config_manager.get("sdk", "claude")- session_manager.update_run(- run_id=run_id,- sdk=sdk,- usage=result.get("usage", {}),- paths={"script_path": result.get("script_path")},- )- elif is_record_only:- # Show helpful message for record-only mode- console.print(" [dim]>[/dim] [white]recording complete[/white]")- console.print(f" [dim]>[/dim] [white]run_id: {run_id}[/white]")- console.print(f" [dim]>[/dim] [dim]use @id {run_id} <prompt> to engineer later[/dim]\n")- except Exception as e:- console.print(f" [red]agent mode error: {e}[/red]")- console.print(f" [dim]{ERROR_CTA}[/dim]")- import traceback-- traceback.print_exc()-def run_collector(prompt=None, model=None, output_dir=None):"""Run AI-powered data collection with Collector class."""diff --git a/src/reverse_api/config.py b/src/reverse_api/config.pyindex de77b28..366ea78 100644--- a/src/reverse_api/config.py+++ b/src/reverse_api/config.py@@ -5,8 +5,7 @@ from pathlib import Pathfrom typing import AnyDEFAULT_CONFIG = {- "agent_provider": "auto", # "auto" (Playwright MCP), "chrome-mcp" (Chrome DevTools MCP), "browser-use", or "stagehand"- "browser_use_model": "bu-llm", # "bu-llm" or "{provider}/{model_name}" (e.g. "openai/gpt-5-mini")+ "agent_provider": "auto", # "auto" (Playwright MCP) or "chrome-mcp" (Chrome DevTools MCP)"claude_code_model": "claude-sonnet-4-6","collector_model": "claude-sonnet-4-6", # Model for collector mode"copilot_model": "gpt-5", # Model for Copilot SDK sessions@@ -16,7 +15,6 @@ DEFAULT_CONFIG = {"output_language": "python", # "python", "javascript", or "typescript""real_time_sync": True, # Enable real-time file sync during engineering"sdk": "claude", # "claude", "opencode", or "copilot"- "stagehand_model": "openai/computer-use-preview-2025-03-11", # "{provider}/{model_name}" format}@@ -40,15 +38,9 @@ class ConfigManager:if "model" in user_config and "claude_code_model" not in user_config:user_config["claude_code_model"] = user_config["model"]- # Migrate "agent_model" -> "browser_use_model" and "stagehand_model"- if "agent_model" in user_config:- agent_provider = user_config.get("agent_provider", "browser-use")- if agent_provider == "stagehand":- if "stagehand_model" not in user_config:- user_config["stagehand_model"] = user_config["agent_model"]- else:- if "browser_use_model" not in user_config:- user_config["browser_use_model"] = user_config["agent_model"]+ # Reset removed agent providers to default+ if user_config.get("agent_provider") in ("browser-use", "stagehand"):+ user_config["agent_provider"] = "auto"# Only keep valid keysvalid_config = {k: v for k, v in user_config.items() if k in self.config}diff --git a/tests/test_config.py b/tests/test_config.pyindex 42b9f78..2eef2c9 100644--- a/tests/test_config.py+++ b/tests/test_config.py@@ -63,33 +63,19 @@ class TestConfigMigration:cm = ConfigManager(config_path)assert cm.get("claude_code_model") == "new"- def test_migrate_agent_model_to_browser_use_model(self, config_path):- """Old 'agent_model' migrates to 'browser_use_model' for browser-use provider."""+ def test_removed_browser_use_provider_resets_to_auto(self, config_path):+ """Legacy 'browser-use' agent_provider falls back to 'auto'."""config_path.parent.mkdir(parents=True, exist_ok=True)- config_path.write_text(json.dumps({"agent_model": "openai/gpt-4", "agent_provider": "browser-use"}))+ config_path.write_text(json.dumps({"agent_provider": "browser-use"}))cm = ConfigManager(config_path)- assert cm.get("browser_use_model") == "openai/gpt-4"+ assert cm.get("agent_provider") == "auto"- def test_migrate_agent_model_to_stagehand_model(self, config_path):- """Old 'agent_model' migrates to 'stagehand_model' for stagehand provider."""+ def test_removed_stagehand_provider_resets_to_auto(self, config_path):+ """Legacy 'stagehand' agent_provider falls back to 'auto'."""config_path.parent.mkdir(parents=True, exist_ok=True)- config_path.write_text(json.dumps({"agent_model": "openai/cu-preview", "agent_provider": "stagehand"}))+ config_path.write_text(json.dumps({"agent_provider": "stagehand"}))cm = ConfigManager(config_path)- assert cm.get("stagehand_model") == "openai/cu-preview"-- def test_migrate_agent_model_default_provider(self, config_path):- """Old 'agent_model' defaults to browser-use migration."""- config_path.parent.mkdir(parents=True, exist_ok=True)- config_path.write_text(json.dumps({"agent_model": "bu-llm"}))- cm = ConfigManager(config_path)- assert cm.get("browser_use_model") == "bu-llm"-- def test_no_migrate_agent_model_if_target_exists(self, config_path):- """Migration skipped if target key already exists."""- config_path.parent.mkdir(parents=True, exist_ok=True)- config_path.write_text(json.dumps({"agent_model": "old", "browser_use_model": "new"}))- cm = ConfigManager(config_path)- assert cm.get("browser_use_model") == "new"+ assert cm.get("agent_provider") == "auto"class TestConfigManagerOperations:@@ -157,16 +143,15 @@ class TestDefaultConfig:"""DEFAULT_CONFIG contains all expected keys."""expected_keys = {"agent_provider",- "browser_use_model","claude_code_model","collector_model",+ "copilot_model","opencode_model","opencode_provider","output_dir","output_language","real_time_sync","sdk",- "stagehand_model",}assert set(DEFAULT_CONFIG.keys()) == expected_keysdiff --git a/uv.lock b/uv.lockindex 59741df..b9d601f 100644--- a/uv.lock+++ b/uv.lock@@ -2100,9 +2100,6 @@ dependencies = [][package.optional-dependencies]-agent = [- { name = "stagehand" },-]collector = [{ name = "beautifulsoup4" },{ name = "markdownify" },@@ -2140,10 +2137,9 @@ requires-dist = [{ name = "requests", specifier = ">=2.33.0" },{ name = "rich", specifier = ">=13.0.0" },{ name = "setproctitle", specifier = ">=1.3.0" },- { name = "stagehand", marker = "extra == 'agent'" },{ name = "watchdog", specifier = ">=3.0.0" },]-provides-extras = ["agent", "pricing", "collector", "copilot"]+provides-extras = ["pricing", "collector", "copilot"][package.metadata.requires-dev]dev = [@@ -2420,26 +2416,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/61/28/8cb142d3fe80c4a2d8af54ca0b003f47ce0ba920974e7990fa6e016402d1/sse_starlette-3.3.2-py3-none-any.whl", hash = "sha256:5c3ea3dad425c601236726af2f27689b74494643f57017cafcb6f8c9acfbb862", size = 14270, upload-time = "2026-02-28T11:24:32.984Z" },]-[[package]]-name = "stagehand"-version = "3.6.0"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "anyio" },- { name = "distro" },- { name = "httpx" },- { name = "pydantic" },- { name = "sniffio" },- { name = "typing-extensions" },-]-sdist = { url = "https://files.pythonhosted.org/packages/57/d1/40c292a0923c5b9032d03fedc7229acf103a979f3d47f2d8d8e3e586ecdd/stagehand-3.6.0.tar.gz", hash = "sha256:d306de4f168eb5d1fd56fab4a8d01bc8e3fe7c68d5673a9cbd939ed5126dbeda", size = 260927, upload-time = "2026-02-25T20:18:55.4Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/8b/4f/5bdebccb945029821eaa82d4623dd2f0d603e86f1e59c55ce66ba164f833/stagehand-3.6.0-py3-none-macosx_10_9_x86_64.whl", hash = "sha256:bbf9f3f67ada735622584d8bed4ff7d911e9b8d645aa592975b4a0c5a5f8e168", size = 34031436, upload-time = "2026-02-25T20:19:03.819Z" },- { url = "https://files.pythonhosted.org/packages/b6/a5/c8032ad7894f18e8024803fcd98869b22494ec1a918d89ca229f3c2fddac/stagehand-3.6.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:0752ce8a6a9224fb9979df8150275da7ba5dad20514e5c9ce599929c9a9eaee7", size = 32738841, upload-time = "2026-02-25T20:19:08.143Z" },- { url = "https://files.pythonhosted.org/packages/ea/0e/892163c11aa44f0a1facabe73b0909a99e60365ef281172f32613265e30c/stagehand-3.6.0-py3-none-manylinux2014_x86_64.whl", hash = "sha256:136c0c53dc6f73b0e3f1ab06528d52456e4d4cf3ee81562b57b16f98b5b124fb", size = 36827049, upload-time = "2026-02-25T20:18:59.029Z" },- { url = "https://files.pythonhosted.org/packages/62/4d/efffa89a8784e45498d98ee62dec85b48b430f8f50a9a0ae38a977f2895b/stagehand-3.6.0-py3-none-win_amd64.whl", hash = "sha256:5ce228596ab1f41faff49814a77767e9c003f45899f78f2c620b0961a238434e", size = 30314762, upload-time = "2026-02-25T20:19:12.543Z" },-]-[[package]]name = "starlette"version = "0.52.1"