Skip to content

The agent that turns websites into APIs!

release: v0.2.5 - fix OpenCodeEngineer initialization and improve README

kalil0321e6eaf7a65c3b
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a4e8d58..5372d08 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,13 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [0.2.5] - 2025-12-27
+
+### Fixed
+- **OpenCodeEngineer initialization**: Refactored to properly pop specific kwargs (`opencode_provider` and `opencode_model`) before passing to parent class
+ - Ensures only relevant arguments are sent to BaseEngineer
+ - Improves initialization logic clarity and prevents unintended argument passing
+
## [0.2.4] - 2025-12-27
### Fixed
diff --git a/README.md b/README.md
index c9f6ef7..0a9d974 100644
--- a/README.md
+++ b/README.md
@@ -12,39 +12,43 @@ CLI tool that captures browser traffic and automatically generates production-re
![Reverse API Engineer Demo](assets/reverse-api-engineer.gif)
+## Table of Contents
+
+- [Features](#-features)
+- [Installation](#-installation)
+- [Quick Start](#-quick-start)
+- [Usage Modes](#-usage-modes)
+ - [Manual Mode](#manual-mode)
+ - [Engineer Mode](#engineer-mode)
+ - [Agent Mode](#agent-mode)
+- [Configuration](#-configuration)
+ - [Model Selection](#model-selection)
+ - [Agent Configuration](#agent-configuration)
+ - [SDK Selection](#sdk-selection)
+- [CLI Commands](#-cli-commands)
+- [Examples](#-examples)
+- [Development](#-development)
+- [Contributing](#-contributing)
+
## ✨ Features
- 🌐 **Browser Automation**: Built on Playwright with stealth mode for realistic browsing
- πŸ€– **Autonomous Agent Mode**: Fully automated browser interaction using AI agents (browser-use, stagehand)
- πŸ“Š **HAR Recording**: Captures all network traffic in HTTP Archive format
- 🧠 **AI-Powered Generation**: Uses Claude 4.5 to analyze traffic and generate clean Python code
-- πŸ”Œ **OpenCode SDK Support**: Native integration with OpenCode SDK for more flexibility
+- πŸ”Œ **Multi-SDK Support**: Native integration with Claude and OpenCode SDKs
- πŸ’» **Interactive CLI**: Minimalist terminal interface with mode cycling (Shift+Tab)
- πŸ“¦ **Production Ready**: Generated scripts include error handling, type hints, and documentation
- πŸ’Ύ **Session History**: All runs saved locally with full message logs
- πŸ’° **Cost Tracking**: Detailed token usage and cost estimation with cache support
-- πŸ”‘ **Multi-Provider Support**: Supports Browser-Use and Stagehand agent providers with multiple LLM options
-
-## Limitations
-- This tool executes code locally using Claude Code please monitor output.
+### Limitations
-- Some websites employ advanced bot-detection and anti-automation protections, which may limit capture or require additional manual interaction.
+- This tool executes code locally using Claude Codeβ€”please monitor output
+- Some websites employ advanced bot-detection that may limit capture or require manual interaction
## πŸš€ Installation
-### Using pip
-```bash
-# Basic installation (manual and engineer modes)
-pip install reverse-api-engineer
-
-# With agent mode support
-pip install 'reverse-api-engineer[agent]'
-
-# Install browser-use with HAR recording support (required for agent mode)
-pip install git+https://github.com/browser-use/browser-use.git@49a345fb19e9f12befc5cc1658e0033873892455
-```
-
### Using uv (recommended)
```bash
# Basic installation
@@ -54,11 +58,14 @@ uv tool install reverse-api-engineer
uv tool install 'reverse-api-engineer[agent]' --with 'browser-use @ git+https://github.com/browser-use/browser-use.git@49a345fb19e9f12befc5cc1658e0033873892455'
```
-### From source
+### Using pip
```bash
-git clone https://github.com/kalil0321/reverse-api-engineer.git
-cd reverse-api-engineer
-uv sync
+# Basic installation
+pip install reverse-api-engineer
+
+# With agent mode support
+pip install 'reverse-api-engineer[agent]'
+pip install git+https://github.com/browser-use/browser-use.git@49a345fb19e9f12befc5cc1658e0033873892455
```
### Post-installation
@@ -67,9 +74,7 @@ Install Playwright browsers:
playwright install chromium
```
-## πŸ“– Usage
-
-### Quick Start
+## πŸš€ Quick Start
Launch the interactive CLI:
```bash
@@ -77,87 +82,65 @@ reverse-api-engineer
```
The CLI has three modes (cycle with **Shift+Tab**):
-- **manual**: Full pipeline (browser capture + AI generation)
-- **engineer**: Reverse engineer from existing run_id
-- **agent**: Autonomous browser agent + capture (fully automated)
+- **manual**: Browser capture + AI generation
+- **engineer**: Re-process existing captures
+- **agent**: Autonomous AI browser agent
+
+Example workflow:
+```bash
+$ reverse-api-engineer
+> fetch all apple jobs from their careers page
-### Manual Mode (Full Pipeline)
+# Browser opens, navigate and interact
+# Close browser when done
+# AI generates production-ready API client
+
+# Scripts saved to: ./scripts/apple_jobs_api/
+```
+
+## πŸ“– Usage Modes
+
+### Manual Mode
+
+Full pipeline with manual browser interaction:
1. Start the CLI: `reverse-api-engineer`
-2. Enter your task description (e.g., "Fetch Apple job listings")
-3. Optionally provide a starting URL
+2. Enter task description (e.g., "Fetch Apple job listings")
+3. Optionally provide starting URL
4. Browse and interact with the website
-5. Close the browser when done
+5. Close browser when done
6. AI automatically generates the API client
-Generated scripts are saved to:
+**Output locations:**
- `~/.reverse-api/runs/scripts/{run_id}/` (permanent storage)
- `./scripts/{descriptive_name}/` (local copy with readable name)
-### Engineer Mode (Re-process existing captures)
+### Engineer Mode
Re-run AI generation on a previous capture:
```bash
-# In CLI, switch to engineer mode (Shift+Tab) and enter run_id
+# Switch to engineer mode (Shift+Tab) and enter run_id
# Or use command line:
reverse-api-engineer engineer <run_id>
```
-### Agent Mode (Autonomous Browser Agent)
+### Agent Mode
Fully automated browser interaction using AI agents:
-1. Install with agent support (see installation instructions above for pip or uv)
-2. Start the CLI: `reverse-api-engineer`
-3. Switch to agent mode (Shift+Tab until you see `[agent]`)
-4. Enter your task description (e.g., "Click on the first job listing")
-5. Optionally provide a starting URL
-6. The agent automatically navigates and interacts with the website
-7. HAR is captured automatically
-8. Optionally run reverse engineering to generate API client
-
-**Important:** Browser-use must be installed from the specific git commit shown in the installation instructions above. This version includes HAR recording support which is not yet available in the PyPI release.
-
-**Agent Provider Configuration:**
-- **Browser-Use** (default): Supports Browser-Use LLM, OpenAI, and Google models
-- **Stagehand**: Supports OpenAI and Anthropic Computer Use models
-
-**Agent Model Configuration:**
-- **Browser-Use Provider**:
- - **Browser-Use LLM**: Requires `BROWSER_USE_API_KEY`
- - **OpenAI Models**: Format `openai/{model}` (e.g., `openai/gpt-4`), requires `OPENAI_API_KEY`
- - **Google Models**: Format `google/{model}` (e.g., `google/gemini-pro`), requires `GOOGLE_API_KEY`
-- **Stagehand Provider**:
- - **OpenAI Computer Use Models**: Format `openai/{cua-model}` (e.g., `openai/computer-use-preview-2025-03-11`), requires `OPENAI_API_KEY`
- - **Anthropic Computer Use Models**: Format `anthropic/{cua-model}` (e.g., `anthropic/claude-sonnet-4-5-20250929`), requires `ANTHROPIC_API_KEY`
- - Note: Stagehand only supports Computer Use Agents (CUA) models
-
-Configure agent provider and model in `/settings` β†’ "agent provider" and "agent model"
-
-### CLI Commands
-
-While in the CLI, use these slash commands:
-- `/settings` - Configure model, agent model, SDK, and output directory
-- `/history` - View past runs with costs
-- `/messages <run_id>` - View detailed message logs
-- `/help` - Show all commands
-- `/exit` - Quit
-
-### Model Selection
+1. Install with agent support (see [Installation](#-installation))
+2. Start CLI and switch to agent mode (Shift+Tab)
+3. Enter task description (e.g., "Click on the first job listing")
+4. Optionally provide starting URL
+5. Agent automatically navigates and interacts
+6. HAR captured automatically
+7. Optionally generate API client
-Choose from Claude 4.5 models:
-- **Sonnet 4.5** (default): Balanced performance and cost
-- **Opus 4.5**: Maximum capability for complex APIs
-- **Haiku 4.5**: Fastest and most economical
-
-Change model in `/settings` or via CLI:
-```bash
-reverse-api-engineer manual --model claude-sonnet-4-5
-```
+**Note:** Requires browser-use from the specific git commit shown in installation instructions (includes HAR recording support).
## πŸ”§ Configuration
-Settings are stored in `~/.reverse-api/config.json`:
+Settings stored in `~/.reverse-api/config.json`:
```json
{
"model": "claude-sonnet-4-5",
@@ -168,71 +151,64 @@ Settings are stored in `~/.reverse-api/config.json`:
}
```
-### Agent Provider Configuration
+### Model Selection
-The `agent_provider` setting controls which browser automation library is used:
-- **`browser-use`** (default): Browser-Use library with support for multiple LLM providers
-- **`stagehand`**: Stagehand library with OpenAI and Anthropic Computer Use models
+Choose from Claude 4.5 models for API generation:
+- **Sonnet 4.5** (default): Balanced performance and cost
+- **Opus 4.5**: Maximum capability for complex APIs
+- **Haiku 4.5**: Fastest and most economical
-### Agent Model Configuration
+Change in `/settings` or via CLI:
+```bash
+reverse-api-engineer manual --model claude-sonnet-4-5
+```
-The `agent_model` setting controls which LLM is used for autonomous browser agents:
+### Agent Configuration
-**For Browser-Use Provider:**
-- **`bu-llm`** (default): Browser-Use's own LLM
- - Requires: `BROWSER_USE_API_KEY` environment variable
-- **`openai/{model}`**: OpenAI models (e.g., `openai/gpt-4`, `openai/gpt-3.5-turbo`)
- - Requires: `OPENAI_API_KEY` environment variable
- - Optional: Install `langchain-openai` for additional model support
-- **`google/{model}`**: Google models (e.g., `google/gemini-pro`, `google/gemini-1.5-pro`)
- - Requires: `GOOGLE_API_KEY` environment variable
- - Optional: Install `langchain-google-genai` for additional model support
+Configure AI agents for autonomous browser automation.
-**For Stagehand Provider:**
-- **`openai/{cua-model}`**: OpenAI Computer Use models (e.g., `openai/computer-use-preview-2025-03-11`)
- - Requires: `OPENAI_API_KEY` environment variable
- - Supported models: `computer-use-preview-2025-03-11`
-- **`anthropic/{cua-model}`**: Anthropic Computer Use models (e.g., `anthropic/claude-sonnet-4-5-20250929`)
- - Requires: `ANTHROPIC_API_KEY` environment variable
- - Supported models: `claude-sonnet-4-5-20250929`, `claude-haiku-4-5-20251001`, `claude-opus-4-5-20251101`
-- Note: Stagehand's `agent.execute()` mode only supports Computer Use Agents (CUA)
+**Agent Providers:**
+- **browser-use** (default): Supports Browser-Use LLM, OpenAI, and Google models
+- **stagehand**: Supports OpenAI and Anthropic Computer Use models
-**Setting API Keys:**
-```bash
-# Browser-Use (default)
-export BROWSER_USE_API_KEY="your-api-key"
+**Agent Models:**
-# OpenAI (required for OpenAI models and Stagehand)
-export OPENAI_API_KEY="your-api-key"
+**Browser-Use Provider:**
+- `bu-llm` (default) - Requires `BROWSER_USE_API_KEY`
+- `openai/gpt-4`, `openai/gpt-3.5-turbo` - Requires `OPENAI_API_KEY`
+- `google/gemini-pro`, `google/gemini-1.5-pro` - Requires `GOOGLE_API_KEY`
-# Anthropic (required for Anthropic models with Stagehand)
-export ANTHROPIC_API_KEY="your-api-key"
+**Stagehand Provider (Computer Use only):**
+- `openai/computer-use-preview-2025-03-11` - Requires `OPENAI_API_KEY`
+- `anthropic/claude-sonnet-4-5-20250929` - Requires `ANTHROPIC_API_KEY`
+- `anthropic/claude-haiku-4-5-20251001` - Requires `ANTHROPIC_API_KEY`
+- `anthropic/claude-opus-4-5-20251101` - Requires `ANTHROPIC_API_KEY`
-# Google (for Browser-Use only)
-export GOOGLE_API_KEY="your-api-key"
+**Setting API Keys:**
+```bash
+export BROWSER_USE_API_KEY="your-api-key" # For Browser-Use
+export OPENAI_API_KEY="your-api-key" # For OpenAI models
+export ANTHROPIC_API_KEY="your-api-key" # For Anthropic models
+export GOOGLE_API_KEY="your-api-key" # For Google models
```
-Change agent provider and model in `/settings` β†’ "agent provider" and "agent model" or edit `config.json` directly.
+Change in `/settings` β†’ "agent provider" and "agent model"
### SDK Selection
-Choose between two SDKs:
-- **OpenCode**: Uses OpenCode SDK for AI-powered reverse engineering. Requires OpenCode to be running locally.
-- **Claude** (default): Direct integration with Anthropic's Claude API.
+- **Claude** (default): Direct integration with Anthropic's Claude API
+- **OpenCode**: Uses OpenCode SDK (requires OpenCode running locally)
-Change SDK in `/settings` or edit `config.json` directly. When using OpenCode SDK, ensure OpenCode is running (`opencode` command).
+Change in `/settings` or edit `config.json` directly.
-## πŸ“ Project Structure
+## πŸ’» CLI Commands
-```
-~/.reverse-api/
-β”œβ”€β”€ config.json # User settings
-β”œβ”€β”€ history.json # Run history with metadata
-└── runs/
- β”œβ”€β”€ har/{run_id}/ # Captured HAR files
- β”œβ”€β”€ scripts/{run_id}/ # Generated API clients
- └── messages/{run_id}.jsonl # Full message logs
-```
+Use these slash commands while in the CLI:
+- `/settings` - Configure model, agent, SDK, and output directory
+- `/history` - View past runs with costs
+- `/messages <run_id>` - View detailed message logs
+- `/help` - Show all commands
+- `/exit` - Quit
## πŸ’‘ Examples
@@ -260,15 +236,6 @@ Generated `api_client.py` includes:
- Error handling
- Production-ready code
-## πŸ—ΊοΈ Roadmap
-
-### SDK Support
-- βœ… **Claude** - Integration with Claude Code
-- βœ… **OpenCode** - Integration with OpenCode
-- πŸ”„ **Codex** - Codex SDK support
-
-
-
## πŸ› οΈ Development
### Setup
@@ -285,15 +252,21 @@ uv run reverse-api-engineer
### Build
```bash
-uv build
+./scripts/clean_build.sh
```
+## πŸ—ΊοΈ Roadmap
+
+- βœ… **Claude SDK** - Integration with Claude Code
+- βœ… **OpenCode SDK** - Integration with OpenCode
+- πŸ”„ **Codex SDK** - Codex SDK support
+
## πŸ” Requirements
- Python 3.11+
- Claude Code / OpenCode (for reverse engineering)
- Playwright browsers installed
-- API key for agent mode (see [Agent Model Configuration](#agent-model-configuration))
+- API key for agent mode (see [Agent Configuration](#agent-configuration))
## 🀝 Contributing
diff --git a/pyproject.toml b/pyproject.toml
index 2086fd0..4b16a22 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
[project]
name = "reverse-api-engineer"
-version = "0.2.4"
+version = "0.2.5"
description = "A tool to capture browser traffic for API reverse engineering"
readme = "README.md"
requires-python = ">=3.11"
diff --git a/src/reverse_api/__init__.py b/src/reverse_api/__init__.py
index 0eed36f..99b2c4c 100644
--- a/src/reverse_api/__init__.py
+++ b/src/reverse_api/__init__.py
@@ -1,3 +1,3 @@
"""Reverse API - Browser traffic capture for API reverse engineering."""
-__version__ = "0.2.4"
+__version__ = "0.2.5"