Skip to content

The agent that turns websites into APIs!

chore: remove Claude plugin and skills

Claude50a13af84880
diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json
deleted file mode 100644
index 2be999f..0000000
--- a/.claude-plugin/marketplace.json
+++ /dev/null
@@ -1,14 +0,0 @@
-{
- "name": "reverse-api-engineer",
- "owner": {
- "name": "Kalil Sama Bouzigues",
- "email": "[email protected]"
- },
- "plugins": [
- {
- "name": "reverse-api-engineer",
- "source": "./plugins/reverse-api-engineer",
- "description": "A plugin to reverse engineer APIs from websites"
- }
- ]
-}
\ No newline at end of file
diff --git a/plugins/reverse-api-engineer/.claude-plugin/plugin.json b/plugins/reverse-api-engineer/.claude-plugin/plugin.json
deleted file mode 100644
index 93af865..0000000
--- a/plugins/reverse-api-engineer/.claude-plugin/plugin.json
+++ /dev/null
@@ -1,38 +0,0 @@
-{
- "name": "reverse-api-engineer",
- "description": "A plugin to reverse engineer APIs from websites",
- "version": "1.0.0",
- "author": {
- "name": "Kalil Sama Bouzigues",
- "email": "[email protected]",
- "url": "https://github.com/kalil0321"
- },
- "repository": "https://github.com/kalil0321/reverse-api-engineer",
- "license": "MIT",
- "keywords": [
- "api",
- "reverse-engineering",
- "har",
- "web-scraping",
- "automation",
- "browser",
- "playwright",
- "network-analysis",
- "api-client",
- "code-generation",
- "python",
- "traffic-capture",
- "http",
- "authentication",
- "endpoints"
- ],
- "mcpServers": {
- "rae-playwright-mcp": {
- "command": "npx",
- "args": [
- "rae-playwright-mcp@latest",
- "run-mcp-server"
- ]
- }
- }
-}
\ No newline at end of file
diff --git a/plugins/reverse-api-engineer/CHANGELOG.md b/plugins/reverse-api-engineer/CHANGELOG.md
deleted file mode 100644
index 2a2774c..0000000
--- a/plugins/reverse-api-engineer/CHANGELOG.md
+++ /dev/null
@@ -1,23 +0,0 @@
-# Changelog
-
-All notable changes to the Reverse API Engineer plugin will be documented in this file.
-
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
-and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
-
-## [1.0.0] - 2026-01-02
-
-### Added
-- Initial release of Reverse API Engineer plugin
-- Manual mode command for browser-based HAR capture
-- Engineer mode command for re-analyzing existing HAR files
-- Agent mode command for autonomous browser navigation
-- Proactive agent for automatic API reverse engineering suggestions
-- Comprehensive reverse-engineering-api skill with:
- - HAR analysis guidance
- - Authentication pattern detection
- - API client generation templates
- - Testing and refinement protocols
-- Integration with rae-playwright-mcp for browser control
-- UUID generation and run history tracking
-- Support for both run_id and direct HAR path references
diff --git a/plugins/reverse-api-engineer/LICENSE b/plugins/reverse-api-engineer/LICENSE
deleted file mode 100644
index 71a4c89..0000000
--- a/plugins/reverse-api-engineer/LICENSE
+++ /dev/null
@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2025 Kalil
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
diff --git a/plugins/reverse-api-engineer/README.md b/plugins/reverse-api-engineer/README.md
deleted file mode 100644
index 9d70bac..0000000
--- a/plugins/reverse-api-engineer/README.md
+++ /dev/null
@@ -1,211 +0,0 @@
-# Reverse API Engineer Plugin
-
-Claude Code plugin for reverse engineering web APIs by capturing browser traffic and generating production-ready Python API clients.
-
-## Features
-
-- **Manual Mode**: Control browser with HAR recording, then AI generates API client
-- **Engineer Mode**: Re-analyze existing HAR captures to generate new clients
-- **Agent Mode**: Autonomous browser navigation with automatic HAR capture and generation
-- **Proactive Agent**: Automatically suggests reverse engineering when working with APIs
-
-## Installation
-
-### From Marketplace
-
-```bash
-claude # Open REPL
-/plugin marketplace add kalil0321/reverse-api-engineer
-/plugin install reverse-api-engineer@reverse-api-engineer
-```
-
-### Local Development
-
-```bash
-claude
-/plugin marketplace add .
-/plugin install reverse-api-engineer@reverse-api-engineer
-```
-
-## Prerequisites
-
-- **Playwright MCP**: The plugin uses `rae-playwright-mcp` for browser control
-- **Node.js**: Required for running the MCP server
-
-The MCP server will be automatically installed when the plugin loads.
-
-## Usage
-
-### Commands
-
-#### `/reverse-api-engineer:manual [task] [url]`
-
-Launch a browser with HAR recording enabled. Navigate manually to trigger API calls, then close the browser to generate the API client.
-
-**Examples:**
-```
-/reverse-api-engineer:manual "fetch Apple jobs" https://jobs.apple.com
-/reverse-api-engineer:manual "scrape product data"
-```
-
-**Workflow:**
-1. Browser opens with HAR recording
-2. Navigate and interact with the website
-3. Close browser when done
-4. AI analyzes HAR and generates Python API client
-5. Scripts saved to `./scripts/{task_name}/`
-
-#### `/reverse-api-engineer:engineer <run_id|har_path>`
-
-Re-run AI generation on a previous HAR capture.
-
-**Examples:**
-```
-/reverse-api-engineer:engineer abc-123-def
-/reverse-api-engineer:engineer ~/.reverse-api/runs/har/xyz-789/recording.har
-/reverse-api-engineer:engineer ./captures/my-traffic.har
-```
-
-**Use cases:**
-- Regenerate client with different approach
-- Extract additional endpoints from same capture
-- Improve existing client implementation
-
-#### `/reverse-api-engineer:agent [task] [url]`
-
-Fully autonomous browser navigation using Playwright MCP. The agent automatically navigates, captures traffic, and generates the API client.
-
-**Examples:**
-```
-/reverse-api-engineer:agent "find all job listings" https://jobs.apple.com
-/reverse-api-engineer:agent "search for products"
-```
-
-**Workflow:**
-1. Agent autonomously navigates the website
-2. HAR automatically captured
-3. AI generates API client
-4. Scripts saved to `./scripts/{task_name}/`
-
-### Proactive Agent
-
-The plugin includes a proactive agent that automatically triggers when you mention:
-- "reverse engineer an API"
-- "create API client for..."
-- "automate interactions with..."
-- "scrape data from..."
-- "HAR file analysis"
-
-**Example:**
-```
-You: "I need to automate fetching data from this website's API"
-Agent: "I can help reverse engineer that API! I'll launch a browser with HAR recording..."
-```
-
-### Skill
-
-The `reverse-engineering-api` skill provides guidance for:
-- Browser capture with HAR recording
-- HAR file analysis and filtering
-- Authentication pattern detection
-- API client code generation
-- Testing and refinement
-
-This skill automatically activates when you mention API reverse engineering tasks.
-
-## Output Structure
-
-### HAR Files
-
-Captured traffic is saved to:
-```
-~/.reverse-api/runs/har/{run_id}/recording.har
-```
-
-### Generated Scripts
-
-API clients are saved to:
-```
-./scripts/{task_name}/
-├── api_client.py # Main API client class
-└── README.md # Usage documentation
-```
-
-### Run History
-
-All runs are tracked in:
-```
-~/.reverse-api/history.json
-```
-
-## How It Works
-
-1. **Browser Capture**: Playwright launches with HAR recording enabled
-2. **Traffic Analysis**: HAR file is analyzed to identify API endpoints, authentication, and patterns
-3. **Code Generation**: AI generates production-ready Python code with:
- - Type hints
- - Error handling
- - Authentication support
- - Comprehensive documentation
-4. **Testing**: Generated client is tested and refined
-
-## Generated Code Quality
-
-All generated API clients include:
-- ✅ Type hints for all parameters and return values
-- ✅ Docstrings for all public methods
-- ✅ Error handling with proper exceptions
-- ✅ Logging for debugging
-- ✅ Session management for connection reuse
-- ✅ Authentication handling (Bearer tokens, API keys, OAuth, etc.)
-- ✅ Clean function interfaces
-- ✅ Production-ready code
-
-## Tips
-
-- **Manual Mode**: Best when you need precise control over interactions
-- **Agent Mode**: Best for simple, automatable tasks
-- **Engineer Mode**: Best for iterating on existing captures
-- **Multiple Captures**: Capture different user flows to discover all endpoints
-- **Authentication**: Make sure to log in during capture to record auth patterns
-
-## Examples
-
-### Example 1: Job Board API
-
-```
-You: /reverse-api-engineer:manual "fetch all Apple jobs" https://jobs.apple.com
-
-[Browser opens]
-1. Navigate to jobs.apple.com
-2. Search for jobs, browse listings
-3. Close browser
-
-[AI generates]
-Scripts saved to: ./scripts/apple_jobs_api/
-- api_client.py (with search_jobs(), get_job_details() methods)
-- README.md (usage documentation)
-```
-
-### Example 2: Re-engineer Existing Capture
-
-```
-You: /reverse-api-engineer:engineer ~/.reverse-api/runs/abc-123/har/recording.har
-
-[AI analyzes HAR]
-Found endpoints:
-- GET /api/v1/jobs/search
-- GET /api/v1/jobs/{id}
-
-[AI generates improved client]
-Scripts saved to: ./scripts/jobs_api/
-```
-
-## Support
-
-For issues, questions, or contributions, visit:
-https://github.com/kalil0321/reverse-api-engineer
-
-## License
-
-MIT License - See LICENSE file for details
diff --git a/plugins/reverse-api-engineer/agents/api-reverse-engineer.md b/plugins/reverse-api-engineer/agents/api-reverse-engineer.md
deleted file mode 100644
index c5143fa..0000000
--- a/plugins/reverse-api-engineer/agents/api-reverse-engineer.md
+++ /dev/null
@@ -1,338 +0,0 @@
----
-name: api-reverse-engineer
-description: Use this agent when the user mentions APIs, web scraping, automating websites, reverse engineering, capturing network traffic, HAR files, or creating API clients. This agent proactively offers to help with API reverse engineering and guides users through the process. Examples: <example>
-Context: User mentions wanting to automate interactions with a website
-user: "I need to automate logging into this website and grabbing some data"
-assistant: "That sounds like an API reverse engineering task."
-<commentary>
-User is describing a classic use case for capturing and analyzing network traffic to build an API client. The agent should trigger to offer the reverse-api-engineer tool.
-</commentary>
-assistant: "I can help you reverse engineer that website's API! I'll use the api-reverse-engineer agent to guide you through the process."
-</example> <example>
-Context: User asks about scraping or extracting data from a web service
-user: "How do I scrape data from this dashboard that requires authentication?"
-assistant: "Authentication usually means there's an API underneath."
-<commentary>
-User wants to extract data from an authenticated service. This is a perfect case for capturing HAR files to understand the authentication flow and API endpoints. Agent should trigger proactively.
-</commentary>
-assistant: "I'll use the api-reverse-engineer agent to help you capture the authentication flow and generate an API client."
-</example> <example>
-Context: User explicitly asks about API reverse engineering
-user: "Can you help me reverse engineer this API?"
-assistant: "I'll use the api-reverse-engineer agent to help you capture and analyze the API traffic."
-</example> <example>
-Context: User has existing HAR files and wants to analyze them
-user: "I have a HAR file from my browser's network tab, can you analyze it?"
-assistant: "Yes! I can analyze that HAR file and generate a Python API client from it."
-<commentary>
-User already has captured traffic and needs analysis. The agent should help them use the engineer mode to re-engineer from existing HAR files.
-</commentary>
-assistant: "I'll use the api-reverse-engineer agent to analyze your HAR file and generate the API client."
-</example>
-model: inherit
-color: cyan
----
-
-You are an expert API reverse engineering specialist with deep knowledge of HTTP protocols, authentication patterns, web scraping, and automated API client generation. You serve as a proactive guide for the reverse-api-engineer plugin, helping users capture, analyze, and reverse engineer web APIs efficiently.
-
-# Core Expertise
-
-You possess advanced knowledge in:
-- HTTP/HTTPS protocol analysis and network traffic interpretation
-- Authentication mechanisms (OAuth2, JWT, session tokens, API keys, cookies)
-- Browser automation and network capture techniques (HAR files, Playwright)
-- API endpoint detection and request pattern analysis
-- Python API client architecture and code generation
-- Anti-scraping measures and mitigation strategies
-- Rate limiting patterns and best practices
-
-# Primary Responsibilities
-
-1. **Proactive Detection**: Automatically recognize when users are discussing:
- - Website automation or data extraction
- - API integration or reverse engineering
- - Web scraping or traffic capture
- - Authentication flows or session management
- - Creating API clients or wrappers
-
-2. **Mode Selection Guidance**: Help users choose the optimal capture mode:
- - **Manual Mode** (`/reverse-api-engineer:manual`): When users want control over browser interactions
- - **Engineer Mode** (`/reverse-api-engineer:engineer`): When users have existing HAR files to analyze
- - **Agent Mode** (`/reverse-api-engineer:agent`): When users want autonomous browser automation
-
-3. **Workflow Assistance**: Guide users through the complete reverse engineering process:
- - Planning the capture strategy
- - Executing the appropriate mode
- - Analyzing generated API clients
- - Troubleshooting authentication issues
- - Optimizing API usage patterns
-
-4. **Technical Advisory**: Provide expertise on:
- - Identifying API endpoints from HAR files
- - Understanding authentication flows
- - Detecting rate limiting and handling strategies
- - Recognizing anti-bot measures
- - Best practices for API client design
-
-# Detailed Process
-
-## Step 1: Detect API-Related Context
-Monitor user messages for keywords and phrases indicating API work:
-- Direct mentions: "API", "reverse engineer", "scrape", "automate"
-- Indirect indicators: "capture traffic", "HAR file", "authentication flow"
-- Use case descriptions: "extract data from website", "automate login", "build wrapper"
-
-When detected, proactively offer assistance even if not explicitly requested.
-
-## Step 2: Assess User Situation
-Determine what the user has and what they need:
-
-**If user has no HAR file yet:**
-- Ask about their goal (what website/service, what data they need)
-- Ask about authentication requirements
-- Recommend manual mode for initial exploration or agent mode for automation
-
-**If user has existing HAR file:**
-- Get the file path
-- Recommend engineer mode to analyze and generate API client
-
-**If user is unsure:**
-- Explain the three modes and their use cases
-- Ask clarifying questions about their technical comfort level
-- Suggest starting with manual mode for learning
-
-## Step 3: Guide Mode Selection
-
-### Manual Mode (`/reverse-api-engineer:manual`)
-**Best for:**
-- Users who want hands-on control
-- Complex authentication flows requiring human interaction
-- Exploratory analysis of unfamiliar APIs
-- Cases requiring CAPTCHA solving or multi-step verification
-
-**Guidance:**
-```
-I recommend manual mode for your use case. This will:
-1. Open a browser window where you can perform the actions manually
-2. Capture all network traffic as a HAR file
-3. Analyze the traffic with AI to generate a Python API client
-
-Use this command: /reverse-api-engineer:manual "<task description>"
-
-Example: /reverse-api-engineer:manual "Log into example.com and capture the user profile API"
-```
-
-### Engineer Mode (`/reverse-api-engineer:engineer`)
-**Best for:**
-- Users with existing HAR files
-- Re-analyzing previous captures
-- Improving or regenerating API clients
-
-**Guidance:**
-```
-Since you have a HAR file, use engineer mode to analyze it:
-
-/reverse-api-engineer:engineer <path-to-har-file>
-
-Example: /reverse-api-engineer:engineer ~/.reverse-api/runs/abc123/har/recording.har
-
-The AI will analyze the traffic and generate a production-ready Python API client.
-```
-
-### Agent Mode (`/reverse-api-engineer:agent`)
-**Best for:**
-- Fully autonomous capture
-- Simple, predictable authentication flows
-- Batch processing of similar sites
-- Users comfortable with autonomous agents
-
-**Guidance:**
-```
-For fully autonomous capture, use agent mode:
-
-/reverse-api-engineer:agent "<task description>"
-
-Example: /reverse-api-engineer:agent "Navigate to jobs.apple.com and capture job listing APIs"
-
-The agent will autonomously navigate, capture traffic, and generate the API client.
-```
-
-## Step 4: Execute and Monitor
-After user runs the appropriate command:
-1. **Monitor for completion**: Watch for output indicating HAR capture or script generation
-2. **Identify output location**: Note where scripts are saved (usually `./scripts/{task_name}/`)
-3. **Prepare for analysis**: Be ready to review generated code or troubleshoot issues
-
-## Step 5: Post-Capture Analysis
-When scripts are generated:
-
-**Review the API client:**
-- Examine authentication methods
-- Identify key endpoints and parameters
-- Check for rate limiting handling
-- Look for session management
-
-**Suggest improvements:**
-- Additional error handling
-- Better type hints
-- Documentation additions
-- Testing strategies
-
-**Answer user questions:**
-- How to use the generated client
-- How to handle authentication in production
-- How to modify the client for specific needs
-- How to troubleshoot API issues
-
-# Output Format Guidelines
-
-## Proactive Suggestions
-When you detect API-related context, use this format:
-
-```
-I can help you reverse engineer that API!
-
-Based on your needs, I recommend [mode] because [reason].
-
-To get started:
-/reverse-api-engineer:[mode] "[specific task]"
-
-Would you like me to guide you through this process?
-```
-
-## Mode Recommendations
-When recommending a mode, always include:
-1. **Why** this mode fits their situation
-2. **What** will happen when they run it
-3. **How** to execute the command with a concrete example
-4. **Where** to find the output
-5. **What** to expect next
-
-## Technical Explanations
-When explaining API concepts:
-- Start with the high-level concept
-- Provide concrete examples from common patterns (OAuth, JWT, etc.)
-- Relate back to what reverse-api-engineer will capture/generate
-- Offer to dive deeper if user needs more details
-
-## Troubleshooting Guidance
-When issues arise:
-1. **Diagnose**: Ask targeted questions about the error/issue
-2. **Explain**: Clarify what's happening and why
-3. **Resolve**: Provide step-by-step solutions
-4. **Prevent**: Suggest ways to avoid the issue in future captures
-
-# Authentication Pattern Recognition
-
-You should recognize and explain common authentication patterns:
-
-**Session-based (Cookies)**
-- Look for Set-Cookie headers
-- Identify session IDs or tokens
-- Explain cookie persistence needs
-
-**Token-based (JWT, Bearer)**
-- Identify Authorization headers
-- Explain token refresh flows
-- Suggest token storage strategies
-
-**API Key**
-- Spot API keys in headers or query params
-- Discuss API key security
-- Recommend environment variable usage
-
-**OAuth2**
-- Recognize OAuth flows (authorization code, client credentials)
-- Identify token endpoints and scopes
-- Explain refresh token handling
-
-**Custom/Proprietary**
-- Identify non-standard authentication
-- Analyze request signatures or custom headers
-- Suggest how to replicate in API client
-
-# Edge Cases and Special Situations
-
-**Rate Limiting Detected**
-- Identify rate limit headers (X-RateLimit-*, Retry-After)
-- Suggest implementing backoff strategies
-- Recommend respectful API usage practices
-
-**Anti-Bot Measures**
-- Recognize Cloudflare, reCAPTCHA, or similar
-- Explain limitations of automated capture
-- Suggest manual mode with human intervention
-
-**Complex Multi-Step Flows**
-- Break down the flow into stages
-- Suggest multiple captures for different stages
-- Help sequence the API client logic
-
-**WebSocket or Streaming APIs**
-- Clarify that HAR capture works best for HTTP/HTTPS
-- Suggest alternative approaches for real-time protocols
-- Offer to help with manual client implementation
-
-**GraphQL APIs**
-- Recognize GraphQL patterns in HAR
-- Explain query/mutation structure
-- Suggest how to build GraphQL client
-
-# Quality Standards
-
-**Always:**
-- Use the MCP tools (rae-playwright-mcp, Read, Write, Bash) when examining captures or files
-- Provide absolute file paths when referencing HAR files or generated scripts
-- Include concrete examples with actual commands
-- Explain the "why" behind recommendations, not just the "how"
-- Anticipate follow-up questions and address them proactively
-
-**Never:**
-- Assume user knows how to use the tool without explanation
-- Recommend a mode without explaining why it fits their situation
-- Leave users without next steps after a command completes
-- Ignore authentication or rate limiting concerns
-- Suggest approaches that violate terms of service
-
-# Communication Style
-
-- **Proactive**: Offer help when you detect API-related work, don't wait to be asked
-- **Clear**: Use concrete examples and commands, avoid vague instructions
-- **Educational**: Explain concepts so users learn, not just copy commands
-- **Encouraging**: Validate user's goals and help them achieve success
-- **Realistic**: Set appropriate expectations about what can be automated
-
-# Integration with Reverse-API-Engineer Plugin
-
-You have deep knowledge of the plugin's architecture:
-
-**Run Structure (`~/.reverse-api/runs/`)**
-- HAR files in `har/{run_id}` subdirectory
-- Run history tracked in `~/.reverse-api/history.json`
-- Local script copies in `./scripts/{descriptive_name}/`
-
-**Command Structure**
-- Manual: `/reverse-api-engineer:manual "<task>" [url]`
-- Engineer: `/reverse-api-engineer:engineer <run_id|har_path>`
-- Agent: `/reverse-api-engineer:agent "<task>" [url]`
-
-**Output Expectations**
-- Python scripts with requests library
-- Authentication classes and methods
-- Endpoint wrappers with type hints
-- Example usage code
-- README with setup instructions
-
-When guiding users, reference these specifics to help them understand where to find files and what to expect from the tool.
-
-# Success Criteria
-
-You've succeeded when:
-1. User successfully captures API traffic (manual or agent mode)
-2. User understands which mode to use for their situation
-3. Generated API client works for user's use case
-4. User can explain the authentication pattern of their target API
-5. User knows how to modify or extend the generated client
-6. User follows best practices (rate limiting, authentication security)
-
-Remember: You are not just a command reference—you are a knowledgeable guide helping users master API reverse engineering. Be proactive, educational, and thorough in your assistance.
diff --git a/plugins/reverse-api-engineer/commands/agent.md b/plugins/reverse-api-engineer/commands/agent.md
deleted file mode 100644
index 1461671..0000000
--- a/plugins/reverse-api-engineer/commands/agent.md
+++ /dev/null
@@ -1,546 +0,0 @@
----
-description: Autonomous browser navigation with HAR capture and API client generation
-argument-hint: "[task] [url]"
-allowed-tools: ["MCP:rae-playwright-mcp", "Read", "Write", "Bash"]
----
-
-# Agent Mode - Autonomous Browser Navigation
-
-Fully autonomous browser navigation using Playwright MCP. The agent automatically navigates the website, captures HAR traffic in real-time, and generates a production-ready Python API client.
-
-## Prerequisites
-
-Ensure the `reverse-engineering-api` skill is loaded for guidance on HAR analysis and code generation.
-
-## Command Arguments
-
-- `task` (required): Description of what to automate (e.g., "find all job listings", "search for products")
-- `url` (optional): Starting URL for the browser
-
-If arguments are not provided, prompt the user interactively.
-
-## Task Tracking
-
-Before starting the workflow, create a todo list to track progress:
-
-```python
-TodoWrite([
- {"content": "Launch browser with HAR recording", "status": "pending", "activeForm": "Launching browser"},
- {"content": "Navigate autonomously to complete task", "status": "pending", "activeForm": "Navigating autonomously"},
- {"content": "Verify HAR capture", "status": "pending", "activeForm": "Verifying HAR file"},
- {"content": "Filter HAR using har_filter.py", "status": "pending", "activeForm": "Filtering HAR"},
- {"content": "Analyze HAR using har_analyze.py", "status": "pending", "activeForm": "Analyzing endpoints"},
- {"content": "Generate API client code", "status": "pending", "activeForm": "Generating code"},
- {"content": "Validate code using har_validate.py", "status": "pending", "activeForm": "Validating code"},
- {"content": "Test API client implementation", "status": "pending", "activeForm": "Testing implementation"},
- {"content": "Generate documentation", "status": "pending", "activeForm": "Writing README"}
-])
-```
-
-**CRITICAL:** Mark each task as `in_progress` when starting, `completed` when done. NEVER skip tasks - complete all 9 steps.
-
-## Workflow
-
-### Step 1: Gather Task Information
-
-If `task` argument is not provided:
-```
-Ask: "What would you like the agent to do? (e.g., 'navigate to jobs page and list all postings')"
-```
-
-If `url` argument is not provided:
-```
-Ask: "What URL should I start from? (optional, press Enter to skip)"
-```
-
-### Step 2: Generate Run ID and Setup Paths
-
-Generate a unique run ID using UUID format:
-```
-run_id = uuid4() (e.g., "abc-123-def-456")
-```
-
-Setup paths:
-```
-har_dir = ~/.reverse-api/runs/har/{run_id}
-har_path = {har_dir}/recording.har
-```
-
-Ensure directory exists:
-```bash
-mkdir -p ~/.reverse-api/runs/har/{run_id}
-```
-
-### Step 3: Launch Browser with HAR Recording
-
-Use the Playwright MCP `rae-playwright-mcp` to launch a browser with HAR recording enabled:
-
-```
-Call MCP tool: playwright_navigate
-Parameters:
- - url: {url} or "about:blank" if no URL provided
- - options:
- - record_har: true
- - har_path: ~/.reverse-api/runs/har/{run_id}/recording.har
-```
-
-Inform the user:
-```
-Agent mode started with HAR recording enabled.
-Task: {task}
-Starting URL: {url}
-
-I'll navigate autonomously and capture all API traffic.
-HAR will be saved to: ~/.reverse-api/runs/har/{run_id}/recording.har
-```
-
-### Step 4: Autonomous Navigation
-
-Use Playwright MCP tools to autonomously navigate and complete the task:
-
-**Available Playwright MCP actions:**
-- `playwright_navigate(url)` - Navigate to URL
-- `playwright_click(selector)` - Click element
-- `playwright_fill(selector, text)` - Fill form field
-- `playwright_get_text(selector)` - Extract text
-- `playwright_wait_for_selector(selector)` - Wait for element
-- `playwright_screenshot()` - Take screenshot
-- `playwright_evaluate(script)` - Execute JavaScript
-- `playwright_get_har_entries()` - Get captured HAR entries in real-time
-
-**Navigation strategy:**
-
-1. **Analyze the page**: Use `playwright_get_text()` to understand page structure
-2. **Plan actions**: Based on task, determine what elements to interact with
-3. **Execute actions**: Use `playwright_click()`, `playwright_fill()` to navigate
-4. **Monitor traffic**: Periodically check `playwright_get_har_entries()` to see captured API calls
-5. **Iterate**: Continue until task is complete or sufficient API traffic captured
-
-**Real-time monitoring:**
-```
-Progress update:
-- Navigated to: {current_url}
-- Actions taken: {count}
-- API calls captured: {har_entry_count}
-- Current step: {description}
-```
-
-**Example navigation for "find all job listings":**
-```
-1. Navigate to homepage
-2. Look for "Careers" or "Jobs" link
-3. Click to navigate to jobs page
-4. Wait for job listings to load
-5. Capture API calls for job data
-6. Check for pagination
-7. If pagination exists, load next pages to capture all API patterns
-8. Close browser when done
-```
-
-### Step 5: Verify HAR Capture
-
-When navigation is complete, verify HAR file exists and contains data:
-
-```bash
-if [ -f ~/.reverse-api/runs/har/{run_id}/recording.har ]; then
- size=$(ls -lh ~/.reverse-api/runs/har/{run_id}/recording.har | awk '{print $5}')
- echo "HAR file captured: $size"
-else
- echo "Error: HAR file not found"
- exit 1
-fi
-```
-
-Read HAR file to count entries:
-```python
-import json
-with open("~/.reverse-api/runs/har/{run_id}/recording.har", "r") as f:
- har_data = json.load(f)
- entry_count = len(har_data["log"]["entries"])
-```
-
-Display:
-```
-Browser session completed.
-Captured {entry_count} network requests in HAR file.
-Analyzing for API endpoints...
-```
-
-### Step 6: Analyze HAR File
-
-**Mark todo in_progress: "Filter HAR using har_filter.py"**
-
-Filter the HAR file to remove static assets, analytics, and CDN resources:
-
-```bash
-python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_filter.py ~/.reverse-api/runs/har/{run_id}/recording.har --output {output_dir}/filtered.har --stats
-```
-
-Read and display the filtering statistics:
-```
-Filtering complete!
-Total entries: {total}
-API endpoints found: {filtered_entries}
-Removed:
-- Static assets: {removed_static}
-- Analytics/tracking: {removed_analytics}
-- CDN resources: {removed_cdn}
-
-API patterns detected: {api_patterns_found}
-```
-
-**Mark todo completed**
-
-**Mark todo in_progress: "Analyze HAR using har_analyze.py"**
-
-Extract structured endpoint information from filtered HAR:
-
-```bash
-python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_analyze.py {output_dir}/filtered.har --output {output_dir}/analysis.json
-```
-
-Read analysis.json and summarize for the user:
-
-```bash
-cat {output_dir}/analysis.json
-```
-
-Display summary:
-```
-Analysis complete!
-
-Base URL: {base_url}
-Authentication: {authentication.type} ({authentication.description})
-Unique endpoints: {unique_endpoints}
-Pagination: {pagination.type if pagination.detected else "Not detected"}
-
-Endpoints found:
-{for each endpoint in endpoints:}
-- {endpoint.methods} {endpoint.pattern}
- Calls observed: {endpoint.calls_observed}
- Auth required: {endpoint.requires_auth}
- Query params: {endpoint.query_params.required + endpoint.query_params.optional}
-```
-
-**Mark todo completed**
-
-### Step 7: Generate API Client
-
-Generate a descriptive name for the script based on the task:
-```
-task_name = sanitize(task) # e.g., "job_listings_api"
-output_dir = ./scripts/{task_name}/
-```
-
-Create output directory:
-```bash
-mkdir -p ./scripts/{task_name}/
-```
-
-Generate `api_client.py` with the following structure:
-
-```python
-"""
-Auto-generated API client for {domain}
-Generated from autonomous agent capture on {date}
-
-Task: {task}
-Run ID: {run_id}
-HAR file: ~/.reverse-api/runs/har/{run_id}/recording.har
-"""
-
-import requests
-from typing import Optional, Dict, Any, List
-import logging
-
-logging.basicConfig(level=logging.INFO)
-logger = logging.getLogger(__name__)
-
-
-class {ClassName}Client:
- """API client for {domain}."""
-
- def __init__(
- self,
- base_url: str = "{base_url}",
- session: Optional[requests.Session] = None,
- ):
- self.base_url = base_url.rstrip("/")
- self.session = session or requests.Session()
- self._setup_session()
-
- def _setup_session(self):
- """Configure session with default headers."""
- self.session.headers.update({
- "User-Agent": "Mozilla/5.0 (compatible)",
- "Accept": "application/json",
- # Add detected required headers
- })
-
- def _request(
- self,
- method: str,
- endpoint: str,
- **kwargs,
- ) -> requests.Response:
- """Make an HTTP request with error handling."""
- url = f"{self.base_url}{endpoint}"
- try:
- response = self.session.request(method, url, **kwargs)
- response.raise_for_status()
- return response
- except requests.exceptions.RequestException as e:
- logger.error(f"Request failed: {e}")
- raise
-
- # Generated endpoint methods
- # For each detected endpoint, create a method like:
-
- def get_example(self, param: str) -> Dict[str, Any]:
- """
- {Method description based on endpoint}.
-
- Args:
- param: {Parameter description}
-
- Returns:
- JSON response data
-
- Raises:
- requests.exceptions.RequestException: If request fails
- """
- response = self._request("GET", f"/api/example/{param}")
- return response.json()
-
-
-# Example usage
-if __name__ == "__main__":
- client = {ClassName}Client()
- # Add example calls based on captured endpoints
-```
-
-Generate `README.md`:
-```markdown
-# {Task Name} API Client
-
-Auto-generated Python API client from autonomous browser agent.
-
-## Generated From
-
-- **Run ID**: {run_id}
-- **HAR File**: ~/.reverse-api/runs/har/{run_id}/recording.har
-- **Date**: {date}
-- **Task**: {task}
-- **Base URL**: {base_url}
-- **Mode**: Agent (autonomous navigation)
-
-## Installation
-
-```bash
-pip install requests
-```
-
-## Usage
-
-```python
-from api_client import {ClassName}Client
-
-client = {ClassName}Client()
-
-# Example: {endpoint_1}
-result = client.{method_1}({params})
-print(result)
-```
-
-## Available Methods
-
-{List all generated methods with descriptions}
-
-## Authentication
-
-{Describe detected authentication mechanism and how to configure}
-
-## Agent Navigation Path
-
-The agent autonomously performed these actions:
-{List navigation steps taken by agent}
-
-## Notes
-
-- Generated from autonomous agent capture
-- Agent task: {task}
-- Endpoints based on observed API calls during navigation
-- May require authentication tokens/API keys
-- Test thoroughly before production use
-```
-
-### Step 7.5: Validate Generated Code
-
-**Mark todo in_progress: "Validate code using har_validate.py"**
-
-Validate the generated API client against the HAR analysis:
-
-```bash
-python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_validate.py {output_dir}/api_client.py {output_dir}/analysis.json
-```
-
-Check the validation score and issues:
-
-**If validation score < 90:**
-1. Read the validation issues carefully
-2. Fix each issue in api_client.py:
- - Missing endpoints: Add methods for each missing endpoint
- - Missing auth: Implement authentication from analysis.json
- - Missing error handling: Add try-except blocks and custom exceptions
- - Missing type hints: Add type hints to all methods
-3. Save the updated api_client.py
-4. Run validation again
-5. Repeat until score >= 90
-
-**Example validation loop:**
-```
-Attempt 1:
-- Score: 75
-- Issues: 2 missing endpoints, no auth implementation
-- Fix: Add missing endpoints, implement bearer token auth
-- Regenerate api_client.py
-
-Attempt 2:
-- Score: 92
-- Issues: 1 info (minor type hint improvement)
-- Result: PASS (score >= 90)
-```
-
-Display validation result:
-```
-Validation complete!
-Score: {score}/100
-Coverage: {coverage.percentage}% ({coverage.endpoints_covered}/{coverage.endpoints_total} endpoints)
-Issues: {summary.errors} errors, {summary.warnings} warnings, {summary.info} info
-
-{if score >= 90:}
-✓ Code validation passed!
-{else:}
-✗ Code validation failed - fixing issues and regenerating...
-```
-
-**Mark todo completed** (only when score >= 90)
-
-### Step 8: Save Run History
-
-Update `~/.reverse-api/history.json` with run metadata:
-
-```json
-{
- "run_id": "{run_id}",
- "mode": "agent",
- "task": "{task}",
- "url": "{url}",
- "timestamp": "{ISO timestamp}",
- "har_path": "~/.reverse-api/runs/har/{run_id}/recording.har",
- "output_dir": "./scripts/{task_name}/",
- "navigation_steps": {count},
- "status": "completed"
-}
-```
-
-### Step 9: Summary
-
-Present final summary to user:
-```
-Agent completed task successfully!
-
-Task: {task}
-Run ID: {run_id}
-HAR file: ~/.reverse-api/runs/har/{run_id}/recording.har
-
-Agent navigation:
-- Pages visited: {page_count}
-- Actions taken: {action_count}
-- API calls captured: {entry_count}
-
-Generated files:
-- ./scripts/{task_name}/api_client.py
-- ./scripts/{task_name}/README.md
-
-Detected endpoints: {count}
-Authentication: {auth_type}
-
-Next steps:
-1. Review the generated code
-2. Test with: python ./scripts/{task_name}/api_client.py
-3. Integrate into your project
-```
-
-## Error Handling
-
-- **Navigation fails**: Agent couldn't complete task (suggest manual mode or retry with clearer instructions)
-- **HAR file empty**: No network traffic captured (check if site has API endpoints)
-- **No API endpoints found**: All entries were filtered out (adjust filters or try different navigation)
-- **Agent stuck**: Navigation loop detected (suggest manual mode for complex interactions)
-
-## Tips for Users
-
-- **Clear tasks**: Provide specific, actionable task descriptions
-- **Starting URL**: Provide URL when possible to speed up navigation
-- **Complex sites**: Use manual mode for sites with heavy JavaScript or complex auth flows
-- **Simple tasks**: Agent works best for straightforward navigation patterns
-- **Re-engineer**: Use `/reverse-api-engineer:engineer {run_id}` to regenerate if needed
-
-## Real-time Monitoring Benefits
-
-Agent mode with Playwright MCP offers unique advantages:
-
-1. **Live HAR access**: Can check captured API calls during navigation via `playwright_get_har_entries()`
-2. **Adaptive navigation**: Can adjust strategy based on observed API patterns
-3. **Early termination**: Can stop once sufficient endpoints are captured
-4. **Smart pagination**: Can detect when all data patterns are captured
-
-## Examples
-
-```
-# Full specification
-/reverse-api-engineer:agent "find all job listings" https://jobs.apple.com
-
-# Task only (agent will search for starting page)
-/reverse-api-engineer:agent "scrape product catalog"
-
-# Interactive mode
-/reverse-api-engineer:agent
-> "navigate to pricing page and capture API calls"
-> "https://example.com"
-```
-
-## Agent vs Manual Mode
-
-**Use Agent Mode when:**
-- Task is straightforward and automatable
-- Site structure is simple
-- You want hands-off operation
-- Real-time HAR monitoring is beneficial
-
-**Use Manual Mode when:**
-- Site requires complex interactions
-- Custom authentication flows
-- Need precise control over captured endpoints
-- Site has bot detection
-- Task involves manual judgment
-
-## Advanced: Real-time HAR Monitoring
-
-During navigation, you can periodically check captured HAR entries:
-
-```
-# Check every few actions
-entries = playwright_get_har_entries()
-
-# Analyze on-the-fly
-api_calls = filter_api_endpoints(entries)
-
-# Decide if done
-if len(api_calls) >= 5 and pagination_pattern_detected:
- stop_navigation()
- proceed_to_analysis()
-```
-
-This allows intelligent navigation that adapts based on captured API patterns.
diff --git a/plugins/reverse-api-engineer/commands/capture.md b/plugins/reverse-api-engineer/commands/capture.md
deleted file mode 100644
index 7160215..0000000
--- a/plugins/reverse-api-engineer/commands/capture.md
+++ /dev/null
@@ -1,191 +0,0 @@
----
-description: Launch browser with HAR recording for manual traffic capture
-argument-hint: "[url]"
-allowed-tools: ["MCP:rae-playwright-mcp", "Bash", "Read"]
----
-
-# Capture Mode - Browser HAR Recording Only
-
-Launch a browser with HAR (HTTP Archive) recording enabled. Navigate manually, then close the browser to save the HAR file. This command does NOT generate code - use `/reverse-api-engineer:engineer <run_id>` afterward to analyze the captured traffic and generate an API client.
-
-## Purpose
-
-Capture mode is useful when you want to:
-- Record multiple browsing sessions before analysis
-- Separate capture from code generation
-- Share HAR files with others for analysis
-- Capture traffic first, decide on implementation later
-
-## Workflow
-
-### Step 1: Generate Run ID
-
-Create a unique run ID for this capture session:
-
-```bash
-run_id=$(uuidgen | tr '[:upper:]' '[:lower:]' | tr -d '-' | cut -c1-12)
-echo "Starting capture session: ${run_id}"
-```
-
-Setup HAR file path:
-```bash
-har_dir="${HOME}/.reverse-api/runs/har/${run_id}"
-mkdir -p "${har_dir}"
-har_path="${har_dir}/recording.har"
-```
-
-Display to user:
-```
-Capture session started!
-Run ID: {run_id}
-HAR file: ~/.reverse-api/runs/har/{run_id}/recording.har
-```
-
-### Step 2: Launch Browser
-
-Launch browser with HAR recording enabled using Playwright MCP:
-
-```
-Call MCP tool: playwright_navigate
-Parameters:
- - url: {url} (if provided) or "about:blank"
- - options:
- - record_har: true
- - har_path: ~/.reverse-api/runs/har/{run_id}/recording.har
- - headless: false
-```
-
-Display instructions to user:
-```
-Browser launched with HAR recording enabled.
-
-Instructions:
-1. Navigate to the website you want to capture
-2. Perform the actions you want to reverse engineer:
- - Log in (to capture authentication)
- - Browse pages (to capture GET requests)
- - Submit forms (to capture POST/PUT requests)
- - Trigger any API calls you need
-3. Close the browser when done
-
-The HAR file will be automatically saved when you close the browser.
-```
-
-### Step 3: Wait for Browser Close
-
-Monitor the browser session. When the browser window closes, proceed to verification.
-
-### Step 4: Verify HAR Capture
-
-Check that the HAR file was created and contains data:
-
-```bash
-if [ -f ~/.reverse-api/runs/har/{run_id}/recording.har ]; then
- size=$(ls -lh ~/.reverse-api/runs/har/{run_id}/recording.har | awk '{print $5}')
- echo "✓ HAR file saved: $size"
-else
- echo "✗ Error: HAR file not found"
- exit 1
-fi
-```
-
-Count network requests in the HAR file:
-
-```python
-import json
-from pathlib import Path
-
-har_path = Path.home() / ".reverse-api" / "runs" / "har" / "{run_id}" / "recording.har"
-
-with open(har_path, 'r') as f:
- har_data = json.load(f)
- entry_count = len(har_data['log']['entries'])
-
-print(f"Captured {entry_count} network requests")
-```
-
-### Step 5: Save Capture Metadata
-
-Save metadata about this capture session:
-
-```bash
-cat > ~/.reverse-api/runs/har/{run_id}/metadata.json <<EOF
-{
- "run_id": "{run_id}",
- "mode": "capture",
- "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
- "har_path": "~/.reverse-api/runs/har/{run_id}/recording.har",
- "url": "{url or 'manual navigation'}",
- "entry_count": {entry_count}
-}
-EOF
-```
-
-### Step 6: Display Summary
-
-Present completion summary to user:
-
-```
-Capture complete!
-
-Run ID: {run_id}
-HAR file: ~/.reverse-api/runs/har/{run_id}/recording.har
-File size: {size}
-Network requests captured: {entry_count}
-
-Next steps:
-━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-
-Option 1: Analyze and generate API client
- /reverse-api-engineer:engineer {run_id}
-
-Option 2: View captured HAR file
- cat ~/.reverse-api/runs/har/{run_id}/recording.har | jq
-
-Option 3: Filter HAR to see API calls only
- python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_filter.py \\
- ~/.reverse-api/runs/har/{run_id}/recording.har --stats
-
-Tip: You can capture multiple times and analyze later!
-```
-
-## Advantages Over /manual Mode
-
-- **Faster**: No analysis step during capture
-- **Flexible**: Capture once, analyze multiple times with different approaches
-- **Shareable**: HAR files can be shared with team members
-- **Clean separation**: Capture and analysis are independent steps
-
-## Example Usage
-
-```bash
-# Capture with starting URL
-/reverse-api-engineer:capture https://api.example.com
-
-# Capture with manual navigation (no starting URL)
-/reverse-api-engineer:capture
-
-# After capture, analyze whenever ready
-/reverse-api-engineer:engineer abc123def456
-```
-
-## Tips
-
-1. **Capture thoroughly**: Include all API calls you want to reverse engineer
-2. **Authentication**: If the site requires login, make sure to log in during capture
-3. **Pagination**: If listing data, navigate through pages to capture pagination patterns
-4. **Multiple captures**: You can run multiple capture sessions and analyze them separately
-
-## Error Handling
-
-**Browser doesn't launch:**
-- Ensure Playwright MCP is properly configured
-- Check that the browser binary is installed
-
-**HAR file is empty:**
-- Make sure you performed actions in the browser
-- Some sites may have very few network requests
-
-**HAR file is too large:**
-- This is normal for sites with many static assets
-- Use har_filter.py to reduce it before analysis
diff --git a/plugins/reverse-api-engineer/commands/engineer.md b/plugins/reverse-api-engineer/commands/engineer.md
deleted file mode 100644
index 305e4c7..0000000
--- a/plugins/reverse-api-engineer/commands/engineer.md
+++ /dev/null
@@ -1,490 +0,0 @@
----
-description: Re-analyze existing HAR capture to generate new API client
-argument-hint: "<run_id|har_path>"
-allowed-tools: ["Read", "Write", "Bash"]
----
-
-# Engineer Mode - Re-engineer from Existing HAR
-
-Re-run AI generation on an existing HAR capture. This is useful when you want to regenerate the client with a different approach, extract additional endpoints, or improve the implementation without re-capturing traffic.
-
-## Prerequisites
-
-Ensure the `reverse-engineering-api` skill is loaded for guidance on HAR analysis and code generation.
-
-## Command Arguments
-
-- `run_id_or_path` (required): Either a run ID (e.g., "abc-123-def") or direct path to HAR file
-
-## Task Tracking
-
-Before starting the workflow, create a todo list to track progress:
-
-```python
-TodoWrite([
- {"content": "Verify HAR file exists", "status": "pending", "activeForm": "Verifying HAR file"},
- {"content": "Filter HAR using har_filter.py", "status": "pending", "activeForm": "Filtering HAR"},
- {"content": "Analyze HAR using har_analyze.py", "status": "pending", "activeForm": "Analyzing endpoints"},
- {"content": "Generate API client code", "status": "pending", "activeForm": "Generating code"},
- {"content": "Validate code using har_validate.py", "status": "pending", "activeForm": "Validating code"},
- {"content": "Test API client implementation", "status": "pending", "activeForm": "Testing implementation"},
- {"content": "Generate documentation", "status": "pending", "activeForm": "Writing README"}
-])
-```
-
-**CRITICAL:** Mark each task as `in_progress` when starting, `completed` when done. NEVER skip tasks - complete all 7 steps.
-
-## Workflow
-
-### Step 1: Parse Input Argument
-
-Determine if the argument is a run_id or a file path:
-
-```python
-import os
-
-arg = "{argument}"
-
-if os.path.exists(arg) and arg.endswith('.har'):
- # Direct HAR file path
- har_path = arg
- run_id = None
-elif '/' in arg or '\\' in arg:
- # Looks like a path but doesn't exist
- Error: "HAR file not found at: {arg}"
-elif arg:
- # Looks like a run_id
- run_id = arg
- har_path = f"~/.reverse-api/runs/har/{run_id}/recording.har"
-else:
- # No argument provided
- Error: "Please provide either a run_id or path to HAR file"
-```
-
-### Step 2: Verify HAR File Exists
-
-Check if the HAR file exists and is readable:
-
-```bash
-if [ -f {har_path} ]; then
- echo "HAR file found: {har_path}"
- ls -lh {har_path}
-else
- echo "Error: HAR file not found at {har_path}"
- exit 1
-fi
-```
-
-If using run_id but HAR file doesn't exist:
-```
-Error: "No HAR file found for run_id: {run_id}
-
-Expected location: ~/.reverse-api/runs/har/{run_id}/recording.har
-
-Available runs:
-{list runs in ~/.reverse-api/runs/}"
-```
-
-### Step 3: List Available Runs (if needed)
-
-If the HAR file is not found and a run_id was provided, list available runs to help the user:
-
-```bash
-ls -1 ~/.reverse-api/runs/ | head -10
-```
-
-Display:
-```
-Available run IDs:
-- abc-123-def (2025-01-01)
-- xyz-789-ghi (2025-01-02)
-...
-
-Use: /reverse-api-engineer:engineer <run_id>
-Or provide direct path: /reverse-api-engineer:engineer /path/to/capture.har
-```
-
-### Step 4: Read Existing Run Metadata (if available)
-
-If using run_id, try to read metadata from history:
-
-```bash
-if [ -f ~/.reverse-api/history.json ]; then
- # Read history.json and find entry for this run_id
- # Extract: task, url, timestamp, original mode
-fi
-```
-
-Display context:
-```
-Re-engineering HAR capture:
-- Run ID: {run_id}
-- Original task: {task}
-- Captured on: {timestamp}
-- Original mode: {mode}
-```
-
-### Step 5: Prompt for New Task Description
-
-Ask the user what to focus on for this regeneration:
-
-```
-The HAR file contains captured API traffic from a previous session.
-
-What would you like to generate? (e.g., "focus on authentication endpoints", "create a simpler client", "add pagination support")
-
-Press Enter to use original task: "{original_task}"
-```
-
-If user provides new description, use it. Otherwise, use original task.
-
-### Step 6: Analyze HAR File
-
-**Mark todo in_progress: "Filter HAR using har_filter.py"**
-
-Filter the HAR file to remove static assets, analytics, and CDN resources:
-
-```bash
-python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_filter.py {har_path} --output {output_dir}/filtered.har --stats
-```
-
-Read and display the filtering statistics:
-```
-Filtering HAR file: {har_path}
-Total entries: {total}
-API endpoints found: {filtered_entries}
-Removed:
-- Static assets: {removed_static}
-- Analytics/tracking: {removed_analytics}
-- CDN resources: {removed_cdn}
-
-API patterns detected: {api_patterns_found}
-```
-
-**Mark todo completed**
-
-**Mark todo in_progress: "Analyze HAR using har_analyze.py"**
-
-Extract structured endpoint information from filtered HAR:
-
-```bash
-python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_analyze.py {output_dir}/filtered.har --output {output_dir}/analysis.json
-```
-
-Read analysis.json and summarize for the user:
-
-```bash
-cat {output_dir}/analysis.json
-```
-
-Display summary:
-```
-Analysis complete!
-
-Base URL: {base_url}
-Authentication: {authentication.type} ({authentication.description})
-Unique endpoints: {unique_endpoints}
-Pagination: {pagination.type if pagination.detected else "Not detected"}
-
-Endpoints found:
-{for each endpoint in endpoints:}
-- {endpoint.methods} {endpoint.pattern}
- Calls observed: {endpoint.calls_observed}
- Auth required: {endpoint.requires_auth}
- Query params: {endpoint.query_params.required + endpoint.query_params.optional}
-```
-
-**Mark todo completed**
-
-### Step 7: Generate New Run ID
-
-Generate a new run ID for this regeneration:
-
-```python
-import uuid
-new_run_id = str(uuid.uuid4())
-```
-
-This keeps the original capture separate from the new generation.
-
-### Step 8: Generate API Client
-
-Generate a descriptive name for the script based on the task:
-
-```
-task_name = sanitize(task) # e.g., "apple_jobs_api"
-output_dir = ./scripts/{task_name}/
-```
-
-Create output directory:
-```bash
-mkdir -p ./scripts/{task_name}/
-```
-
-Generate `api_client.py` with the following structure:
-
-```python
-"""
-Auto-generated API client for {domain}
-Generated from HAR capture on {date}
-
-Original Run ID: {original_run_id}
-Original HAR file: {har_path}
-Regenerated Run ID: {new_run_id}
-"""
-
-import requests
-from typing import Optional, Dict, Any, List
-import logging
-
-logging.basicConfig(level=logging.INFO)
-logger = logging.getLogger(__name__)
-
-
-class {ClassName}Client:
- """API client for {domain}."""
-
- def __init__(
- self,
- base_url: str = "{base_url}",
- session: Optional[requests.Session] = None,
- ):
- self.base_url = base_url.rstrip("/")
- self.session = session or requests.Session()
- self._setup_session()
-
- def _setup_session(self):
- """Configure session with default headers."""
- self.session.headers.update({
- "User-Agent": "Mozilla/5.0 (compatible)",
- "Accept": "application/json",
- # Add detected required headers
- })
-
- def _request(
- self,
- method: str,
- endpoint: str,
- **kwargs,
- ) -> requests.Response:
- """Make an HTTP request with error handling."""
- url = f"{self.base_url}{endpoint}"
- try:
- response = self.session.request(method, url, **kwargs)
- response.raise_for_status()
- return response
- except requests.exceptions.RequestException as e:
- logger.error(f"Request failed: {e}")
- raise
-
- # Generated endpoint methods
- # For each detected endpoint, create a method like:
-
- def get_example(self, param: str) -> Dict[str, Any]:
- """
- {Method description based on endpoint}.
-
- Args:
- param: {Parameter description}
-
- Returns:
- JSON response data
-
- Raises:
- requests.exceptions.RequestException: If request fails
- """
- response = self._request("GET", f"/api/example/{param}")
- return response.json()
-
-
-# Example usage
-if __name__ == "__main__":
- client = {ClassName}Client()
- # Add example calls based on captured endpoints
-```
-
-Generate `README.md`:
-```markdown
-# {Task Name} API Client
-
-Auto-generated Python API client from browser traffic capture.
-
-## Generated From
-
-- **Original Run ID**: {original_run_id}
-- **HAR File**: {har_path}
-- **Regenerated**: {date}
-- **Regeneration Run ID**: {new_run_id}
-- **Base URL**: {base_url}
-
-## Installation
-
-```bash
-pip install requests
-```
-
-## Usage
-
-```python
-from api_client import {ClassName}Client
-
-client = {ClassName}Client()
-
-# Example: {endpoint_1}
-result = client.{method_1}({params})
-print(result)
-```
-
-## Available Methods
-
-{List all generated methods with descriptions}
-
-## Authentication
-
-{Describe detected authentication mechanism and how to configure}
-
-## Notes
-
-- Regenerated from previous HAR capture
-- Original capture: {original_date}
-- Endpoints based on observed API calls
-- May require authentication tokens/API keys
-- Test thoroughly before production use
-```
-
-### Step 8.5: Validate Generated Code
-
-**Mark todo in_progress: "Validate code using har_validate.py"**
-
-Validate the generated API client against the HAR analysis:
-
-```bash
-python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_validate.py {output_dir}/api_client.py {output_dir}/analysis.json
-```
-
-Check the validation score and issues:
-
-**If validation score < 90:**
-1. Read the validation issues carefully
-2. Fix each issue in api_client.py:
- - Missing endpoints: Add methods for each missing endpoint
- - Missing auth: Implement authentication from analysis.json
- - Missing error handling: Add try-except blocks and custom exceptions
- - Missing type hints: Add type hints to all methods
-3. Save the updated api_client.py
-4. Run validation again
-5. Repeat until score >= 90
-
-**Example validation loop:**
-```
-Attempt 1:
-- Score: 75
-- Issues: 2 missing endpoints, no auth implementation
-- Fix: Add missing endpoints, implement bearer token auth
-- Regenerate api_client.py
-
-Attempt 2:
-- Score: 92
-- Issues: 1 info (minor type hint improvement)
-- Result: PASS (score >= 90)
-```
-
-Display validation result:
-```
-Validation complete!
-Score: {score}/100
-Coverage: {coverage.percentage}% ({coverage.endpoints_covered}/{coverage.endpoints_total} endpoints)
-Issues: {summary.errors} errors, {summary.warnings} warnings, {summary.info} info
-
-{if score >= 90:}
-✓ Code validation passed!
-{else:}
-✗ Code validation failed - fixing issues and regenerating...
-```
-
-**Mark todo completed** (only when score >= 90)
-
-### Step 9: Save Run History
-
-Update `~/.reverse-api/history.json` with run metadata:
-
-```json
-{
- "run_id": "{new_run_id}",
- "mode": "engineer",
- "task": "{task}",
- "source_run_id": "{original_run_id}",
- "har_path": "{har_path}",
- "timestamp": "{ISO timestamp}",
- "output_dir": "./scripts/{task_name}/",
- "status": "completed"
-}
-```
-
-### Step 10: Summary
-
-Present final summary to user:
-
-```
-API client re-generated successfully!
-
-Original HAR: {har_path}
-Original Run ID: {original_run_id}
-New Run ID: {new_run_id}
-
-Generated files:
-- ./scripts/{task_name}/api_client.py
-- ./scripts/{task_name}/README.md
-
-Detected endpoints: {count}
-Authentication: {auth_type}
-
-Next steps:
-1. Review the generated code
-2. Test with: python ./scripts/{task_name}/api_client.py
-3. Compare with previous version (if any)
-4. Integrate into your project
-
-To regenerate with different focus:
-/reverse-api-engineer:engineer {original_run_id}
-```
-
-## Error Handling
-
-- **HAR file not found**: Provide list of available runs and suggest correct usage
-- **Invalid HAR format**: Check file is valid JSON and follows HAR specification
-- **No API endpoints found**: All entries were filtered out (suggest adjusting filters or recapturing)
-- **Generation fails**: Retry with different approach or analyze HAR manually
-
-## Tips for Users
-
-- **Different approaches**: Try regenerating with different focus areas
-- **Refinement**: Use engineer mode to improve previous generations
-- **Testing**: Compare multiple generated versions to find the best implementation
-- **Documentation**: Each regeneration creates a new run_id for tracking
-- **Original HAR**: The original HAR file is never modified
-
-## Examples
-
-```
-# Using run_id
-/reverse-api-engineer:engineer abc-123-def
-
-# Using direct path
-/reverse-api-engineer:engineer ~/.reverse-api/runs/har/xyz-789/recording.har
-
-# Using relative path
-/reverse-api-engineer:engineer ./captures/myapp.har
-
-# Re-engineer with focus
-/reverse-api-engineer:engineer abc-123-def
-> "focus on authentication endpoints only"
-```
-
-## Use Cases
-
-1. **Improve previous generation**: Regenerate with better endpoint detection
-2. **Extract subset**: Focus on specific endpoints from large capture
-3. **Different implementation**: Generate alternative client architecture
-4. **Update documentation**: Regenerate with better docstrings and examples
-5. **Fix issues**: Address problems in previous generation without re-capturing
diff --git a/plugins/reverse-api-engineer/commands/manual.md b/plugins/reverse-api-engineer/commands/manual.md
deleted file mode 100644
index 6a99d97..0000000
--- a/plugins/reverse-api-engineer/commands/manual.md
+++ /dev/null
@@ -1,422 +0,0 @@
----
-description: Launch browser with HAR recording, capture traffic manually, then generate Python API client
-argument-hint: "[task] [url]"
-allowed-tools: ["MCP:rae-playwright-mcp", "Read", "Write", "Bash"]
----
-
-# Manual Mode - Browser Capture and API Generation
-
-Launch a browser with HAR recording enabled. The user will navigate manually to trigger the desired API calls. When the browser closes, analyze the captured HAR file and generate a production-ready Python API client.
-
-## Prerequisites
-
-Ensure the `reverse-engineering-api` skill is loaded for guidance on HAR analysis and code generation.
-
-## Command Arguments
-
-- `task` (optional): Description of what to reverse engineer (e.g., "fetch Apple jobs", "scrape product data")
-- `url` (optional): Starting URL for the browser
-
-If arguments are not provided, prompt the user interactively.
-
-## Task Tracking
-
-Before starting the workflow, create a todo list to track progress:
-
-```python
-TodoWrite([
- {"content": "Launch browser with HAR recording", "status": "pending", "activeForm": "Launching browser"},
- {"content": "Wait for user to complete manual navigation", "status": "pending", "activeForm": "Recording traffic"},
- {"content": "Verify HAR capture", "status": "pending", "activeForm": "Verifying HAR file"},
- {"content": "Filter HAR using har_filter.py", "status": "pending", "activeForm": "Filtering HAR"},
- {"content": "Analyze HAR using har_analyze.py", "status": "pending", "activeForm": "Analyzing endpoints"},
- {"content": "Generate API client", "status": "pending", "activeForm": "Generating code"},
- {"content": "Validate code using har_validate.py", "status": "pending", "activeForm": "Validating code"},
- {"content": "Test implementation", "status": "pending", "activeForm": "Testing API client"},
- {"content": "Generate documentation", "status": "pending", "activeForm": "Writing README"}
-])
-```
-
-**CRITICAL:** Mark each task as `in_progress` when starting, `completed` when done. NEVER skip tasks - complete all 9 steps.
-
-## Workflow
-
-### Step 1: Gather Task Information
-
-If `task` argument is not provided:
-```
-Ask: "What would you like to reverse engineer? (e.g., 'fetch job listings from Company X')"
-```
-
-If `url` argument is not provided:
-```
-Ask: "What URL should I open? (optional, press Enter to skip)"
-```
-
-### Step 2: Generate Run ID and Setup Paths
-
-Generate a unique run ID using UUID format:
-```
-run_id = uuid4() (e.g., "abc-123-def-456")
-```
-
-Setup paths:
-```
-har_dir = ~/.reverse-api/runs/har/{run_id}/
-har_path = {har_dir}/recording.har
-```
-
-Ensure directory exists:
-```bash
-mkdir -p ~/.reverse-api/runs/har/{run_id}/
-```
-
-### Step 3: Launch Browser with HAR Recording
-
-Use the Playwright MCP `rae-playwright-mcp` to launch a browser with HAR recording:
-
-```
-Call MCP tool: playwright_navigate
-Parameters:
- - url: {url} or "about:blank" if no URL provided
- - options:
- - record_har: true
- - har_path: ~/.reverse-api/runs/har/{run_id}/recording.har
-```
-
-Inform the user:
-```
-Browser launched with HAR recording enabled.
-Navigate to the website and interact with it to trigger API calls.
-Close the browser when you're done to proceed with analysis.
-
-HAR will be saved to: ~/.reverse-api/runs/har/{run_id}/recording.har
-```
-
-### Step 4: Wait for Browser to Close
-
-Monitor the browser session. When it closes, the HAR file will be saved automatically.
-
-Verify HAR file exists:
-```bash
-ls -lh ~/.reverse-api/runs/har/{run_id}/recording.har
-```
-
-If file doesn't exist or is empty:
-```
-Error: "HAR file not found or empty. Please ensure you interacted with the website before closing the browser."
-Exit with error
-```
-
-### Step 5: Analyze HAR File
-
-**Mark todo in_progress: "Filter HAR using har_filter.py"**
-
-Filter the HAR file to remove static assets, analytics, and CDN resources:
-
-```bash
-python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_filter.py ~/.reverse-api/runs/har/{run_id}/recording.har --output {output_dir}/filtered.har --stats
-```
-
-Read and display the filtering statistics:
-```
-Filtering complete!
-Total entries: {total}
-API endpoints found: {filtered_entries}
-Removed:
-- Static assets: {removed_static}
-- Analytics/tracking: {removed_analytics}
-- CDN resources: {removed_cdn}
-
-API patterns detected: {api_patterns_found}
-```
-
-**Mark todo completed**
-
-**Mark todo in_progress: "Analyze HAR using har_analyze.py"**
-
-Extract structured endpoint information from filtered HAR:
-
-```bash
-python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_analyze.py {output_dir}/filtered.har --output {output_dir}/analysis.json
-```
-
-Read analysis.json and summarize for the user:
-
-```bash
-cat {output_dir}/analysis.json
-```
-
-Display summary:
-```
-Analysis complete!
-
-Base URL: {base_url}
-Authentication: {authentication.type} ({authentication.description})
-Unique endpoints: {unique_endpoints}
-Pagination: {pagination.type if pagination.detected else "Not detected"}
-
-Endpoints found:
-{for each endpoint in endpoints:}
-- {endpoint.methods} {endpoint.pattern}
- Calls observed: {endpoint.calls_observed}
- Auth required: {endpoint.requires_auth}
- Query params: {endpoint.query_params.required + endpoint.query_params.optional}
-```
-
-**Mark todo completed**
-
-### Step 6: Generate API Client
-
-Generate a descriptive name for the script based on the task:
-```
-task_name = sanitize(task) # e.g., "apple_jobs_api"
-output_dir = ./scripts/{task_name}/
-```
-
-Create output directory:
-```bash
-mkdir -p ./scripts/{task_name}/
-```
-
-Generate `api_client.py` with the following structure:
-
-```python
-"""
-Auto-generated API client for {domain}
-Generated from HAR capture on {date}
-
-Run ID: {run_id}
-HAR file: ~/.reverse-api/runs/har/{run_id}/recording.har
-"""
-
-import requests
-from typing import Optional, Dict, Any, List
-import logging
-
-logging.basicConfig(level=logging.INFO)
-logger = logging.getLogger(__name__)
-
-
-class {ClassName}Client:
- """API client for {domain}."""
-
- def __init__(
- self,
- base_url: str = "{base_url}",
- session: Optional[requests.Session] = None,
- ):
- self.base_url = base_url.rstrip("/")
- self.session = session or requests.Session()
- self._setup_session()
-
- def _setup_session(self):
- """Configure session with default headers."""
- self.session.headers.update({
- "User-Agent": "Mozilla/5.0 (compatible)",
- "Accept": "application/json",
- # Add detected required headers
- })
-
- def _request(
- self,
- method: str,
- endpoint: str,
- **kwargs,
- ) -> requests.Response:
- """Make an HTTP request with error handling."""
- url = f"{self.base_url}{endpoint}"
- try:
- response = self.session.request(method, url, **kwargs)
- response.raise_for_status()
- return response
- except requests.exceptions.RequestException as e:
- logger.error(f"Request failed: {e}")
- raise
-
- # Generated endpoint methods
- # For each detected endpoint, create a method like:
-
- def get_example(self, param: str) -> Dict[str, Any]:
- """
- {Method description based on endpoint}.
-
- Args:
- param: {Parameter description}
-
- Returns:
- JSON response data
-
- Raises:
- requests.exceptions.RequestException: If request fails
- """
- response = self._request("GET", f"/api/example/{param}")
- return response.json()
-
-
-# Example usage
-if __name__ == "__main__":
- client = {ClassName}Client()
- # Add example calls based on captured endpoints
-```
-
-Generate `README.md`:
-```markdown
-# {Task Name} API Client
-
-Auto-generated Python API client from browser traffic capture.
-
-## Generated From
-
-- **Run ID**: {run_id}
-- **HAR File**: ~/.reverse-api/runs/har/{run_id}/recording.har
-- **Date**: {date}
-- **Base URL**: {base_url}
-
-## Installation
-
-```bash
-pip install requests
-```
-
-## Usage
-
-```python
-from api_client import {ClassName}Client
-
-client = {ClassName}Client()
-
-# Example: {endpoint_1}
-result = client.{method_1}({params})
-print(result)
-```
-
-## Available Methods
-
-{List all generated methods with descriptions}
-
-## Authentication
-
-{Describe detected authentication mechanism and how to configure}
-
-## Notes
-
-- Generated from captured browser traffic
-- Endpoints based on observed API calls
-- May require authentication tokens/API keys
-- Test thoroughly before production use
-```
-
-### Step 6.5: Validate Generated Code
-
-**Mark todo in_progress: "Validate code using har_validate.py"**
-
-Validate the generated API client against the HAR analysis:
-
-```bash
-python plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_validate.py {output_dir}/api_client.py {output_dir}/analysis.json
-```
-
-Check the validation score and issues:
-
-**If validation score < 90:**
-1. Read the validation issues carefully
-2. Fix each issue in api_client.py:
- - Missing endpoints: Add methods for each missing endpoint
- - Missing auth: Implement authentication from analysis.json
- - Missing error handling: Add try-except blocks and custom exceptions
- - Missing type hints: Add type hints to all methods
-3. Save the updated api_client.py
-4. Run validation again
-5. Repeat until score >= 90
-
-**Example validation loop:**
-```
-Attempt 1:
-- Score: 75
-- Issues: 2 missing endpoints, no auth implementation
-- Fix: Add missing endpoints, implement bearer token auth
-- Regenerate api_client.py
-
-Attempt 2:
-- Score: 92
-- Issues: 1 info (minor type hint improvement)
-- Result: PASS (score >= 90)
-```
-
-Display validation result:
-```
-Validation complete!
-Score: {score}/100
-Coverage: {coverage.percentage}% ({coverage.endpoints_covered}/{coverage.endpoints_total} endpoints)
-Issues: {summary.errors} errors, {summary.warnings} warnings, {summary.info} info
-
-{if score >= 90:}
-✓ Code validation passed!
-{else:}
-✗ Code validation failed - fixing issues and regenerating...
-```
-
-**Mark todo completed** (only when score >= 90)
-
-### Step 7: Save Run History
-
-Update `~/.reverse-api/history.json` with run metadata:
-
-```json
-{
- "run_id": "{run_id}",
- "mode": "manual",
- "task": "{task}",
- "url": "{url}",
- "timestamp": "{ISO timestamp}",
- "har_path": "~/.reverse-api/runs/har/{run_id}/recording.har",
- "output_dir": "./scripts/{task_name}/",
- "status": "completed"
-}
-```
-
-### Step 8: Summary
-
-Present final summary to user:
-```
-API client generated successfully!
-
-Run ID: {run_id}
-HAR file: ~/.reverse-api/runs/har/{run_id}/recording.har
-
-Generated files:
-- ./scripts/{task_name}/api_client.py
-- ./scripts/{task_name}/README.md
-
-Detected endpoints: {count}
-Authentication: {auth_type}
-
-Next steps:
-1. Review the generated code
-2. Test with: python ./scripts/{task_name}/api_client.py
-3. Integrate into your project
-```
-
-## Error Handling
-
-- **Browser launch fails**: Check Playwright MCP is running and accessible
-- **HAR file empty**: User didn't interact with website, or no network traffic captured
-- **No API endpoints found**: All entries were filtered out (adjust filters or capture more traffic)
-- **Generation fails**: Use `/reverse-api-engineer:engineer {run_id}` to retry with different approach
-
-## Tips for Users
-
-- **Login flows**: Log in during capture to record authentication
-- **Pagination**: Navigate through pages to capture query parameter patterns
-- **Different actions**: Create, read, update, delete to capture all CRUD endpoints
-- **Multiple captures**: You can run manual mode multiple times for different flows
-- **Re-engineer**: Use `/reverse-api-engineer:engineer {run_id}` to regenerate from same capture
-
-## Examples
-
-```
-/reverse-api-engineer:manual "fetch Apple jobs" https://jobs.apple.com
-/reverse-api-engineer:manual "scrape product data"
-/reverse-api-engineer:manual
-```
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/CHANGELOG.md b/plugins/reverse-api-engineer/skills/reverse-engineering-api/CHANGELOG.md
deleted file mode 100644
index dabebd6..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/CHANGELOG.md
+++ /dev/null
@@ -1,33 +0,0 @@
-# Changelog
-
-All notable changes to the Reverse Engineering API skill will be documented in this file.
-
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
-and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
-
-## [1.0.0] - 2025-12-31
-
-### Added
-
-- Initial release of the Reverse Engineering API skill
-- Core SKILL.md with 4-phase workflow:
- - Phase 1: Browser Capture with HAR Recording
- - Phase 2: HAR Analysis
- - Phase 3: API Client Generation
- - Phase 4: Testing & Refinement
-- Reference documentation:
- - `references/AUTH_PATTERNS.md` - 8 authentication patterns with code examples
- - `references/HAR_ANALYSIS.md` - HAR parsing and endpoint extraction guide
-- Playwright MCP integration for browser control
-- Support for HAR files at `~/.reverse-api/runs/har/{run_id}/`
-- Decision tree for workflow routing
-- Python API client generation templates
-- Iteration protocol with up to 5 retry attempts
-- Bot detection handling with Playwright fallback
-
-### Dependencies
-
-Scripts require:
-- `aiohttp` - Async HTTP client (required for all scripts)
-- `playwright` - Browser automation (required for mapper.py)
-- `beautifulsoup4` - HTML parsing (optional for crawler.py, falls back to regex)
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/LICENSE b/plugins/reverse-api-engineer/skills/reverse-engineering-api/LICENSE
deleted file mode 100644
index 0b4233a..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/LICENSE
+++ /dev/null
@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2025 kalil0321
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/SKILL.md b/plugins/reverse-api-engineer/skills/reverse-engineering-api/SKILL.md
deleted file mode 100644
index be80b96..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/SKILL.md
+++ /dev/null
@@ -1,377 +0,0 @@
----
-name: reverse-engineering-api
-description: Reverse engineer web APIs by capturing browser traffic (HAR files) and generating production-ready Python API clients. Use when the user wants to create an API client for a website, automate web interactions, or understand undocumented APIs. Activate on tasks mentioning "reverse engineer", "API client", "HAR file", "capture traffic", or "automate website".
-compatibility: Requires Playwright MCP for browser control. HAR files saved to ~/.reverse-api/runs/har/{run_id}
-metadata:
- author: reverse-api-engineer
- version: "1.0"
-allowed-tools: Bash(python:*) Bash(uv:*) Read Write Edit
----
-
-# Reverse Engineering API Skill
-
-This skill enables you to reverse engineer web APIs by:
-1. Controlling a browser with HAR recording enabled
-2. Analyzing captured network traffic
-3. Generating production-ready Python API clients
-
-## Prerequisites
-
-- **Playwright MCP**: You must have access to Playwright MCP tools for browser control
-- **HAR Recording**: The browser must be configured to record HAR files
-- **Python**: For running analysis scripts and generated clients
-
-## Workflow Overview
-
-```
-[User Task] -> [Browser Capture] -> [HAR Analysis] -> [API Client Generation] -> [Testing & Refinement]
-```
-
-## Phase 0: Preparation (Using HAR Helper Scripts)
-
-### Available Helper Scripts
-
-This skill provides Python utilities for HAR analysis located at:
-
-**Script Directory:** `plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/`
-
-**Available Scripts:**
-- `har_filter.py` - Filter HAR files to API endpoints only
-- `har_analyze.py` - Extract structured endpoint information
-- `har_validate.py` - Validate generated code against HAR analysis
-- `har_utils.py` - Shared utility functions
-
-### Script Usage Pattern
-
-Use these scripts in sequence for optimal code generation:
-
-```bash
-# 1. Filter HAR to remove noise (static assets, analytics, CDN)
-python {SKILL_DIR}/scripts/har_filter.py {har_path} --output filtered.har --stats
-
-# 2. Analyze endpoints and extract patterns
-python {SKILL_DIR}/scripts/har_analyze.py filtered.har --output analysis.json
-
-# 3. Read analysis for code generation guidance
-cat analysis.json
-
-# 4. Generate API client code based on analysis
-
-# 5. Validate generated code
-python {SKILL_DIR}/scripts/har_validate.py api_client.py analysis.json
-```
-
-### Why Use These Scripts?
-
-**har_filter.py benefits:**
-- Reduces HAR file size by 80-90% (removes noise)
-- Focuses analysis on actual API calls
-- Significantly improves code generation quality
-- Outputs statistics showing what was filtered
-
-**har_analyze.py benefits:**
-- Provides structured endpoint information
-- Detects authentication patterns automatically
-- Identifies pagination mechanisms
-- Extracts request/response schemas
-- Groups endpoints by pattern
-
-**har_validate.py benefits:**
-- Ensures all endpoints are implemented
-- Validates authentication handling
-- Checks for proper error handling
-- Calculates coverage score (must be >= 90)
-- Identifies missing features
-
-### Task Tracking
-
-Use TodoWrite to track workflow progress:
-- Mark tasks as `pending`, `in_progress`, or `completed`
-- Only ONE task should be `in_progress` at a time
-- Complete ALL tasks - never stop early
-
-**Example TodoWrite usage:**
-```python
-TodoWrite([
- {"content": "Filter HAR using har_filter.py", "status": "in_progress", "activeForm": "Filtering HAR"},
- {"content": "Analyze HAR using har_analyze.py", "status": "pending", "activeForm": "Analyzing endpoints"},
- {"content": "Generate API client", "status": "pending", "activeForm": "Generating code"},
- {"content": "Validate using har_validate.py", "status": "pending", "activeForm": "Validating code"},
- {"content": "Test implementation", "status": "pending", "activeForm": "Testing API client"}
-])
-```
-
-**CRITICAL:** Task tracking ensures complete workflow execution. Never skip tasks or stop early.
-
-## Phase 1: Browser Capture with HAR Recording
-
-### Starting the Browser
-
-When starting a browser session for API capture:
-
-1. Launch browser with HAR recording enabled via Playwright MCP
-2. Generate a unique run ID: `{run_id}`
-3. Configure HAR output path: `~/.reverse-api/runs/har/{run_id}/recording.har`
-
-### During Capture
-
-Navigate autonomously to trigger the API calls needed:
-- Login flows (capture authentication)
-- Data fetching (capture GET endpoints)
-- Form submissions (capture POST/PUT endpoints)
-- Pagination (capture query parameter patterns)
-
-### On Browser Close
-
-When the browser closes, note the HAR file location:
-```
-HAR file saved to: ~/.reverse-api/runs/har/{run_id}/recording.har
-```
-
-## Phase 2: HAR Analysis
-
-### Reading the HAR File
-
-HAR files are JSON with this structure:
-```json
-{
- "log": {
- "entries": [
- {
- "request": {
- "method": "GET|POST|PUT|DELETE",
- "url": "https://api.example.com/endpoint",
- "headers": [...],
- "postData": {...}
- },
- "response": {
- "status": 200,
- "headers": [...],
- "content": {...}
- }
- }
- ]
- }
-}
-```
-
-### Filtering Relevant Entries
-
-Filter out noise by excluding:
-- Static assets: `.js`, `.css`, `.png`, `.jpg`, `.svg`, `.woff`, `.ico`
-- Analytics: `google-analytics`, `segment`, `mixpanel`, `hotjar`
-- Ads: `doubleclick`, `adsense`, `facebook.com/tr`
-- CDN resources: `cloudflare`, `cdn.`, `static.`
-
-Focus on:
-- API endpoints: `/api/`, `/v1/`, `/v2/`, `/graphql`
-- XHR/Fetch requests with JSON responses
-- Requests with authentication headers
-
-### Extracting Patterns
-
-For each relevant endpoint, extract:
-
-1. **URL Pattern**: Base URL, path, query parameters
-2. **Method**: GET, POST, PUT, DELETE, PATCH
-3. **Headers**:
- - Required headers (Authorization, Content-Type, custom headers)
- - Optional headers (User-Agent, Accept)
-4. **Request Body**: JSON schema, form data structure
-5. **Response Schema**: JSON structure, status codes
-6. **Authentication**: See [references/AUTH_PATTERNS.md](references/AUTH_PATTERNS.md)
-
-## Phase 3: API Client Generation
-
-### Code Structure
-
-Generate a Python module with:
-
-```
-{output_dir}/
- api_client.py # Main API client class
- README.md # Usage documentation
-```
-
-### api_client.py Template
-
-```python
-"""
-Auto-generated API client for {domain}
-Generated from HAR capture on {date}
-"""
-
-import requests
-from typing import Optional, Dict, Any, List
-import logging
-
-logging.basicConfig(level=logging.INFO)
-logger = logging.getLogger(__name__)
-
-
-class {ClassName}Client:
- """API client for {domain}."""
-
- def __init__(
- self,
- base_url: str = "{base_url}",
- session: Optional[requests.Session] = None,
- ):
- self.base_url = base_url.rstrip("/")
- self.session = session or requests.Session()
- self._setup_session()
-
- def _setup_session(self):
- """Configure session with default headers."""
- self.session.headers.update({
- "User-Agent": "Mozilla/5.0 (compatible)",
- "Accept": "application/json",
- # Add other required headers
- })
-
- def _request(
- self,
- method: str,
- endpoint: str,
- **kwargs,
- ) -> requests.Response:
- """Make an HTTP request with error handling."""
- url = f"{self.base_url}{endpoint}"
- try:
- response = self.session.request(method, url, **kwargs)
- response.raise_for_status()
- return response
- except requests.exceptions.RequestException as e:
- logger.error(f"Request failed: {e}")
- raise
-
- # Generated endpoint methods go here
- def get_example(self, param: str) -> Dict[str, Any]:
- """
- Fetch example data.
-
- Args:
- param: Description of parameter
-
- Returns:
- JSON response data
- """
- response = self._request("GET", f"/api/example/{param}")
- return response.json()
-
-
-# Example usage
-if __name__ == "__main__":
- client = {ClassName}Client()
- # Example calls
-```
-
-### Code Quality Requirements
-
-All generated code must include:
-
-1. **Type hints** for all parameters and return values
-2. **Docstrings** for all public methods
-3. **Error handling** with try-except blocks
-4. **Logging** for debugging
-5. **Session management** for connection reuse
-6. **Authentication handling** based on detected patterns
-
-## Phase 4: Testing & Refinement
-
-### Testing the Generated Client
-
-After generating the client:
-
-1. Run the example usage section
-2. Verify responses match expected structure
-3. Handle any errors encountered
-
-### Iteration Protocol
-
-You have up to 5 attempts to fix issues:
-
-```
-Attempt 1: Initial implementation
- - What was tried
- - What failed (if anything)
- - What was changed
-
-Attempt 2: Refinement
- ...
-```
-
-### Common Issues
-
-| Issue | Solution |
-|-------|----------|
-| 403 Forbidden | Add missing headers, check authentication |
-| Bot detection | Switch to Playwright with stealth mode |
-| Rate limiting | Add delays, respect Retry-After headers |
-| Session expiry | Implement token refresh logic |
-| CORS errors | Use server-side requests (not applicable to Python) |
-
-## Domain Discovery (Optional)
-
-Before capture, you may want to map the domain to understand its structure.
-
-### Using the Mapper Script
-
-Run `scripts/mapper.py` to quickly discover:
-- All pages on the domain or subdomains
-- Subdomains
-
-It is useful for generalizing your scripts on multitenants websites.
-
-For example, for Ashby ATS or Workday it's useful to find other companies using this ATS when trying to generalize your script.
-
-```bash
-python scripts/mapper.py https://example.com
-```
-
-### Using the Sitemap Parser
-
-Run `scripts/sitemap.py` to extract URLs from sitemaps:
-
-```bash
-python scripts/sitemap.py https://example.com
-```
-
-## Output Locations
-
-- **HAR files**: `~/.reverse-api/runs/har/{run_id}/`
-- **Generated scripts**: `./{task_name}`
-
-## Example Session
-
-```
-User: "Create an API client for the Apple Jobs website"
-
-
-1. [Browser Capture]
- Launch browser with HAR recording
- Navigate to jobs.apple.com
- Perform search, browse listings
- Close browser
- HAR saved to: ~/.reverse-api/runs/har/{run_id}/recording.har
-
- Note: you can monitor browser requests with the Playwright MCP
-
-2. [HAR Analysis]
- Found endpoints:
- - GET /api/role/search?query=...
- - GET /api/role/{id}
- Authentication: None required (public API)
-
-3. [Generate Client]
- Create : {task_name}/api_client.py
-
-4. [Test]
- Ran example usage - Success!
-
-5. [Summary]
- Generated Apple Jobs API client with:
- - search_roles(query, location, page)
- - get_role(role_id)
- Files: ./{task_name}/
-```
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/references/AUTH_PATTERNS.md b/plugins/reverse-api-engineer/skills/reverse-engineering-api/references/AUTH_PATTERNS.md
deleted file mode 100644
index 7554b4f..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/references/AUTH_PATTERNS.md
+++ /dev/null
@@ -1,575 +0,0 @@
-# Authentication Patterns Reference
-
-This document covers common authentication patterns found in web APIs and how to handle them when generating API clients. These patterns align with the reverse-api-engineer codebase conventions.
-
-## Overview
-
-When analyzing HAR files, identify authentication by checking:
-- Request headers: `Authorization`, `X-API-Key`, `X-Token`, `Cookie`
-- Response headers: `Set-Cookie`, `X-CSRF-Token`
-- Request body: `username`, `password`, `token`, `api_key`
-- Response body: `access_token`, `refresh_token`, `session_id`
-- Query parameters: `key`, `api_key`, `token`, `access_token`
-- Login endpoints: `/login`, `/auth`, `/oauth/token`, `/api/auth`
-
----
-
-## 1. Bearer Token (JWT)
-
-### Detection in HAR
-```json
-{
- "name": "Authorization",
- "value": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
-}
-```
-
-### Generated Client Pattern
-```python
-"""
-API client with Bearer token authentication.
-Generated by reverse-api-engineer.
-"""
-
-import requests
-from typing import Optional, Dict, Any
-import logging
-
-logging.basicConfig(level=logging.INFO)
-logger = logging.getLogger(__name__)
-
-
-class APIClient:
- """API client with Bearer token authentication."""
-
- def __init__(
- self,
- base_url: str,
- token: str,
- session: Optional[requests.Session] = None,
- ):
- self.base_url = base_url.rstrip("/")
- self.token = token
- self.session = session or requests.Session()
- self._setup_session()
-
- def _setup_session(self):
- """Configure session with authentication headers."""
- self.session.headers.update({
- "Authorization": f"Bearer {self.token}",
- "Accept": "application/json",
- "Content-Type": "application/json",
- })
-
- def _request(
- self,
- method: str,
- endpoint: str,
- **kwargs,
- ) -> requests.Response:
- """Make an HTTP request with error handling."""
- url = f"{self.base_url}{endpoint}"
- try:
- response = self.session.request(method, url, **kwargs)
- response.raise_for_status()
- return response
- except requests.exceptions.RequestException as e:
- logger.error(f"Request failed: {e}")
- raise
-```
-
-### Token Refresh Pattern
-If you detect a `/refresh` or `/token/refresh` endpoint in the HAR:
-```python
-def refresh_token(self, refresh_token: str) -> str:
- """
- Refresh the access token.
-
- Args:
- refresh_token: The refresh token from initial auth
-
- Returns:
- New access token
- """
- response = self._request("POST", "/auth/refresh", json={
- "refresh_token": refresh_token
- })
- data = response.json()
- self.token = data["access_token"]
- self.session.headers["Authorization"] = f"Bearer {self.token}"
- return self.token
-```
-
----
-
-## 2. API Key
-
-### Detection in HAR
-Header-based:
-```json
-{"name": "X-API-Key", "value": "sk_live_xxxxx"}
-```
-
-Query parameter:
-```json
-{"name": "api_key", "value": "xxxxx"}
-```
-
-### Generated Client Pattern (Header)
-```python
-class APIClient:
- """API client with API key authentication."""
-
- def __init__(
- self,
- base_url: str,
- api_key: str,
- session: Optional[requests.Session] = None,
- ):
- self.base_url = base_url.rstrip("/")
- self.api_key = api_key
- self.session = session or requests.Session()
- self._setup_session()
-
- def _setup_session(self):
- """Configure session with API key header."""
- self.session.headers.update({
- "X-API-Key": self.api_key,
- "Accept": "application/json",
- })
-```
-
-### Generated Client Pattern (Query Parameter)
-```python
-class APIClient:
- """API client with API key in query parameters."""
-
- def __init__(self, base_url: str, api_key: str):
- self.base_url = base_url.rstrip("/")
- self.api_key = api_key
- self.session = requests.Session()
-
- def _request(
- self,
- method: str,
- endpoint: str,
- **kwargs,
- ) -> requests.Response:
- """Make request with API key in query params."""
- url = f"{self.base_url}{endpoint}"
- params = kwargs.pop("params", {})
- params["api_key"] = self.api_key
- try:
- response = self.session.request(method, url, params=params, **kwargs)
- response.raise_for_status()
- return response
- except requests.exceptions.RequestException as e:
- logger.error(f"Request failed: {e}")
- raise
-```
-
----
-
-## 3. Session Cookies
-
-### Detection in HAR
-Look for `Set-Cookie` in response headers after login:
-```json
-{"name": "Set-Cookie", "value": "sessionid=abc123; Path=/; HttpOnly"}
-```
-
-Common cookie names: `sessionid`, `PHPSESSID`, `connect.sid`, `_session`
-
-### Generated Client Pattern
-```python
-class APIClient:
- """API client with session-based authentication."""
-
- def __init__(self, base_url: str):
- self.base_url = base_url.rstrip("/")
- self.session = requests.Session() # Automatically handles cookies
-
- def login(self, username: str, password: str) -> bool:
- """
- Authenticate with username and password.
-
- Args:
- username: User's username
- password: User's password
-
- Returns:
- True if login successful
- """
- try:
- response = self.session.post(
- f"{self.base_url}/login",
- data={"username": username, "password": password}
- )
- response.raise_for_status()
- return True
- except requests.exceptions.RequestException as e:
- logger.error(f"Login failed: {e}")
- return False
-
- def _request(
- self,
- method: str,
- endpoint: str,
- **kwargs,
- ) -> requests.Response:
- """Make request with session cookies."""
- url = f"{self.base_url}{endpoint}"
- try:
- response = self.session.request(method, url, **kwargs)
- response.raise_for_status()
- return response
- except requests.exceptions.RequestException as e:
- logger.error(f"Request failed: {e}")
- raise
-```
-
----
-
-## 4. CSRF Tokens
-
-### Detection in HAR
-Hidden form field in HTML:
-```html
-<input type="hidden" name="csrf_token" value="...">
-```
-
-Header in requests:
-```json
-{"name": "X-CSRF-Token", "value": "..."}
-```
-
-Cookie:
-```json
-{"name": "csrftoken", "value": "..."}
-```
-
-### Generated Client Pattern
-```python
-import re
-
-
-class APIClient:
- """API client with CSRF token handling."""
-
- def __init__(self, base_url: str):
- self.base_url = base_url.rstrip("/")
- self.session = requests.Session()
- self.csrf_token: Optional[str] = None
-
- def _get_csrf_token(self) -> str:
- """Fetch CSRF token from cookie or page."""
- # Option 1: From cookie
- if "csrftoken" in self.session.cookies:
- return self.session.cookies["csrftoken"]
-
- # Option 2: From page content
- response = self.session.get(f"{self.base_url}/")
- match = re.search(r'csrf_token["\']:\s*["\']([^"\']+)', response.text)
- if match:
- return match.group(1)
-
- # Option 3: From meta tag
- match = re.search(r'<meta name="csrf-token" content="([^"]+)"', response.text)
- if match:
- return match.group(1)
-
- return ""
-
- def _request(
- self,
- method: str,
- endpoint: str,
- **kwargs,
- ) -> requests.Response:
- """Make request with CSRF token for mutating operations."""
- url = f"{self.base_url}{endpoint}"
-
- # Add CSRF token for POST, PUT, DELETE, PATCH
- if method.upper() in ("POST", "PUT", "DELETE", "PATCH"):
- if not self.csrf_token:
- self.csrf_token = self._get_csrf_token()
- headers = kwargs.pop("headers", {})
- headers["X-CSRF-Token"] = self.csrf_token
- kwargs["headers"] = headers
-
- try:
- response = self.session.request(method, url, **kwargs)
- response.raise_for_status()
- return response
- except requests.exceptions.RequestException as e:
- logger.error(f"Request failed: {e}")
- raise
-```
-
----
-
-## 5. OAuth 2.0
-
-### Detection in HAR
-Look for:
-- Token endpoint: `/oauth/token`, `/auth/token`
-- `grant_type` in request body
-- `access_token` and `refresh_token` in response
-
-### Generated Client Pattern (Client Credentials)
-```python
-class APIClient:
- """API client with OAuth 2.0 Client Credentials flow."""
-
- def __init__(
- self,
- base_url: str,
- client_id: str,
- client_secret: str,
- ):
- self.base_url = base_url.rstrip("/")
- self.client_id = client_id
- self.client_secret = client_secret
- self.access_token: Optional[str] = None
- self.refresh_token: Optional[str] = None
- self.session = requests.Session()
-
- def authenticate(self) -> bool:
- """
- Authenticate using Client Credentials flow.
-
- Returns:
- True if authentication successful
- """
- try:
- response = requests.post(
- f"{self.base_url}/oauth/token",
- data={
- "grant_type": "client_credentials",
- "client_id": self.client_id,
- "client_secret": self.client_secret,
- }
- )
- response.raise_for_status()
- data = response.json()
- self.access_token = data["access_token"]
- self.refresh_token = data.get("refresh_token")
- self.session.headers["Authorization"] = f"Bearer {self.access_token}"
- return True
- except requests.exceptions.RequestException as e:
- logger.error(f"OAuth authentication failed: {e}")
- return False
-```
-
----
-
-## 6. Basic Authentication
-
-### Detection in HAR
-```json
-{"name": "Authorization", "value": "Basic dXNlcm5hbWU6cGFzc3dvcmQ="}
-```
-
-The value is base64 encoded `username:password`.
-
-### Generated Client Pattern
-```python
-from requests.auth import HTTPBasicAuth
-
-
-class APIClient:
- """API client with HTTP Basic authentication."""
-
- def __init__(
- self,
- base_url: str,
- username: str,
- password: str,
- ):
- self.base_url = base_url.rstrip("/")
- self.session = requests.Session()
- self.session.auth = HTTPBasicAuth(username, password)
- self.session.headers.update({
- "Accept": "application/json",
- })
-```
-
----
-
-## 7. Custom Headers
-
-### Detection in HAR
-Non-standard headers like:
-```json
-{"name": "X-App-Token", "value": "..."}
-{"name": "X-Client-Id", "value": "..."}
-{"name": "X-Request-Id", "value": "..."}
-```
-
-### Generated Client Pattern
-```python
-class APIClient:
- """API client with custom header authentication."""
-
- def __init__(
- self,
- base_url: str,
- app_token: str,
- client_id: str,
- ):
- self.base_url = base_url.rstrip("/")
- self.session = requests.Session()
- self._setup_session(app_token, client_id)
-
- def _setup_session(self, app_token: str, client_id: str):
- """Configure session with custom headers."""
- self.session.headers.update({
- "X-App-Token": app_token,
- "X-Client-Id": client_id,
- "Accept": "application/json",
- })
-```
-
----
-
-## 8. Signed Requests (HMAC)
-
-### Detection in HAR
-Look for:
-- Signature headers: `X-Signature`, `X-Hmac-Signature`
-- Timestamp headers: `X-Timestamp`, `X-Request-Timestamp`
-- Nonce headers: `X-Nonce`
-
-### Generated Client Pattern
-```python
-import hmac
-import hashlib
-import time
-import uuid
-
-
-class APIClient:
- """API client with HMAC request signing."""
-
- def __init__(
- self,
- base_url: str,
- api_key: str,
- secret: str,
- ):
- self.base_url = base_url.rstrip("/")
- self.api_key = api_key
- self.secret = secret
- self.session = requests.Session()
-
- def _sign_request(
- self,
- method: str,
- path: str,
- body: str = "",
- ) -> Dict[str, str]:
- """
- Generate signature headers for request.
-
- Args:
- method: HTTP method
- path: Request path
- body: Request body as string
-
- Returns:
- Dictionary of signature headers
- """
- timestamp = str(int(time.time()))
- nonce = str(uuid.uuid4())
- message = f"{method}{path}{timestamp}{nonce}{body}"
-
- signature = hmac.new(
- self.secret.encode(),
- message.encode(),
- hashlib.sha256
- ).hexdigest()
-
- return {
- "X-API-Key": self.api_key,
- "X-Timestamp": timestamp,
- "X-Nonce": nonce,
- "X-Signature": signature,
- }
-
- def _request(
- self,
- method: str,
- endpoint: str,
- **kwargs,
- ) -> requests.Response:
- """Make signed request."""
- url = f"{self.base_url}{endpoint}"
- body = kwargs.get("json", kwargs.get("data", ""))
- if isinstance(body, dict):
- import json
- body = json.dumps(body)
-
- headers = kwargs.pop("headers", {})
- headers.update(self._sign_request(method, endpoint, body))
- kwargs["headers"] = headers
-
- try:
- response = self.session.request(method, url, **kwargs)
- response.raise_for_status()
- return response
- except requests.exceptions.RequestException as e:
- logger.error(f"Request failed: {e}")
- raise
-```
-
----
-
-## Priority Order
-
-When multiple auth methods are detected in HAR, use this priority:
-
-1. **OAuth 2.0** - Most robust for production
-2. **Bearer Token** - Simple and stateless
-3. **Session Cookies** - Required for some sites
-4. **API Key** - Simple but less secure
-5. **Basic Auth** - Only if no other option
-
----
-
-## Bot Detection Fallback
-
-If `requests` fails with 403/captcha, switch to Playwright:
-
-```python
-from playwright.sync_api import sync_playwright
-
-
-class PlaywrightClient:
- """Playwright-based client for sites with bot detection."""
-
- def __init__(self, base_url: str):
- self.base_url = base_url.rstrip("/")
- self.playwright = None
- self.browser = None
- self.context = None
- self.page = None
-
- def __enter__(self):
- self.playwright = sync_playwright().start()
- self.browser = self.playwright.chromium.launch(headless=True)
- self.context = self.browser.new_context()
- self.page = self.context.new_page()
- return self
-
- def __exit__(self, exc_type, exc_val, exc_tb):
- if self.browser:
- self.browser.close()
- if self.playwright:
- self.playwright.stop()
-
- def get(self, endpoint: str) -> dict:
- """Make GET request via browser."""
- url = f"{self.base_url}{endpoint}"
- response = self.page.goto(url)
- return self.page.evaluate("() => JSON.parse(document.body.innerText)")
-```
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/references/HAR_ANALYSIS.md b/plugins/reverse-api-engineer/skills/reverse-engineering-api/references/HAR_ANALYSIS.md
deleted file mode 100644
index 2c78e7b..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/references/HAR_ANALYSIS.md
+++ /dev/null
@@ -1,381 +0,0 @@
-# HAR File Analysis Reference
-
-This document covers how to analyze HAR (HTTP Archive) files to extract API endpoints and patterns.
-
-## HAR File Structure
-
-```json
-{
- "log": {
- "version": "1.2",
- "creator": { "name": "Browser", "version": "1.0" },
- "entries": [
- {
- "startedDateTime": "2025-01-01T12:00:00.000Z",
- "time": 150,
- "request": { ... },
- "response": { ... },
- "cache": {},
- "timings": { ... }
- }
- ]
- }
-}
-```
-
-## Entry Structure
-
-### Request Object
-```json
-{
- "method": "POST",
- "url": "https://api.example.com/v1/users",
- "httpVersion": "HTTP/2.0",
- "headers": [
- { "name": "Content-Type", "value": "application/json" },
- { "name": "Authorization", "value": "Bearer xxx" }
- ],
- "queryString": [
- { "name": "page", "value": "1" }
- ],
- "postData": {
- "mimeType": "application/json",
- "text": "{\"name\": \"John\"}"
- },
- "cookies": [
- { "name": "session", "value": "abc123" }
- ]
-}
-```
-
-### Response Object
-```json
-{
- "status": 200,
- "statusText": "OK",
- "headers": [
- { "name": "Content-Type", "value": "application/json" }
- ],
- "content": {
- "size": 1024,
- "mimeType": "application/json",
- "text": "{\"id\": 1, \"name\": \"John\"}"
- }
-}
-```
-
-## Filtering Entries
-
-### URLs to Exclude
-
-**Static Assets:**
-- `.js`, `.css`, `.map`
-- `.png`, `.jpg`, `.jpeg`, `.gif`, `.svg`, `.webp`, `.ico`
-- `.woff`, `.woff2`, `.ttf`, `.eot`
-- `.mp4`, `.webm`, `.mp3`
-
-**Analytics & Tracking:**
-- `google-analytics.com`
-- `googletagmanager.com`
-- `segment.io`, `segment.com`
-- `mixpanel.com`
-- `hotjar.com`
-- `facebook.com/tr`
-- `doubleclick.net`
-- `adsense`
-
-**CDN Resources:**
-- `cdn.`, `static.`, `assets.`
-- `cloudflare.com`
-- `cloudfront.net`
-- `akamai`
-
-### URLs to Focus On
-
-**API Patterns:**
-- `/api/`
-- `/v1/`, `/v2/`, `/v3/`
-- `/graphql`
-- `/rest/`
-- `/data/`
-
-**Content Types:**
-- `application/json`
-- `application/xml`
-- `text/plain` (sometimes API responses)
-
-## Extracting Endpoints
-
-### Step 1: Parse HAR
-```python
-import json
-from urllib.parse import urlparse, parse_qs
-
-def load_har(path: str) -> dict:
- with open(path) as f:
- return json.load(f)
-
-har = load_har("capture.har")
-entries = har["log"]["entries"]
-```
-
-### Step 2: Filter Relevant Entries
-```python
-SKIP_EXTENSIONS = {'.js', '.css', '.png', '.jpg', '.svg', '.woff', '.ico'}
-SKIP_DOMAINS = {'google-analytics.com', 'facebook.com', 'segment.io'}
-
-def is_relevant(entry: dict) -> bool:
- url = entry["request"]["url"]
- parsed = urlparse(url)
-
- # Skip static assets
- if any(parsed.path.endswith(ext) for ext in SKIP_EXTENSIONS):
- return False
-
- # Skip tracking
- if any(d in parsed.netloc for d in SKIP_DOMAINS):
- return False
-
- # Check for API patterns
- if any(p in parsed.path for p in ['/api/', '/v1/', '/graphql']):
- return True
-
- # Check content type
- response = entry["response"]
- content_type = next(
- (h["value"] for h in response["headers"]
- if h["name"].lower() == "content-type"),
- ""
- )
- if "application/json" in content_type:
- return True
-
- return False
-
-api_entries = [e for e in entries if is_relevant(e)]
-```
-
-### Step 3: Group by Endpoint
-```python
-from collections import defaultdict
-
-endpoints = defaultdict(list)
-
-for entry in api_entries:
- request = entry["request"]
- url = request["url"]
- parsed = urlparse(url)
-
- # Create endpoint key (method + path without query)
- method = request["method"]
- path = parsed.path
- key = f"{method} {path}"
-
- endpoints[key].append(entry)
-
-# Analyze each endpoint
-for key, entries in endpoints.items():
- print(f"\n{key}")
- print(f" Calls: {len(entries)}")
-
- # Get unique query parameters
- all_params = set()
- for e in entries:
- for param in e["request"].get("queryString", []):
- all_params.add(param["name"])
- if all_params:
- print(f" Query params: {', '.join(all_params)}")
-```
-
-### Step 4: Extract Request Schema
-```python
-def extract_request_schema(entries: list) -> dict:
- """Infer request body schema from multiple calls."""
- schemas = []
-
- for entry in entries:
- post_data = entry["request"].get("postData", {})
- if post_data.get("mimeType") == "application/json":
- try:
- body = json.loads(post_data.get("text", "{}"))
- schemas.append(body)
- except:
- pass
-
- if not schemas:
- return {}
-
- # Merge schemas to find common fields
- all_keys = set()
- for s in schemas:
- if isinstance(s, dict):
- all_keys.update(s.keys())
-
- return {
- "type": "object",
- "properties": {k: {"type": "unknown"} for k in all_keys}
- }
-```
-
-### Step 5: Extract Response Schema
-```python
-def extract_response_schema(entries: list) -> dict:
- """Infer response schema from multiple calls."""
- for entry in entries:
- content = entry["response"].get("content", {})
- if content.get("mimeType") == "application/json":
- try:
- text = content.get("text", "")
- if text:
- data = json.loads(text)
- return infer_schema(data)
- except:
- pass
- return {}
-
-def infer_schema(data) -> dict:
- """Infer JSON schema from data."""
- if isinstance(data, dict):
- return {
- "type": "object",
- "properties": {k: infer_schema(v) for k, v in data.items()}
- }
- elif isinstance(data, list):
- if data:
- return {"type": "array", "items": infer_schema(data[0])}
- return {"type": "array", "items": {}}
- elif isinstance(data, bool):
- return {"type": "boolean"}
- elif isinstance(data, int):
- return {"type": "integer"}
- elif isinstance(data, float):
- return {"type": "number"}
- elif isinstance(data, str):
- return {"type": "string"}
- else:
- return {"type": "null"}
-```
-
-## Identifying Path Parameters
-
-Look for patterns in paths:
-```python
-import re
-
-def find_path_params(paths: list[str]) -> str:
- """Find path parameters by looking at variations."""
- # Example: ["/users/1", "/users/2", "/users/3"] -> "/users/{id}"
-
- # Split paths into segments
- segments_list = [p.strip("/").split("/") for p in paths]
-
- if not segments_list:
- return ""
-
- result = []
- num_segments = len(segments_list[0])
-
- for i in range(num_segments):
- values = set(s[i] for s in segments_list if len(s) > i)
-
- # If all values are the same, it's a static segment
- if len(values) == 1:
- result.append(values.pop())
- # If values look like IDs (numbers, UUIDs), it's a parameter
- elif all(v.isdigit() or is_uuid(v) for v in values):
- result.append("{id}")
- else:
- result.append("{param}")
-
- return "/" + "/".join(result)
-
-def is_uuid(s: str) -> bool:
- pattern = r'^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
- return bool(re.match(pattern, s, re.IGNORECASE))
-```
-
-## Extracting Headers
-
-### Required vs Optional Headers
-```python
-def analyze_headers(entries: list) -> tuple[set, set]:
- """Determine required vs optional headers."""
- all_headers = []
-
- for entry in entries:
- headers = {h["name"].lower(): h["value"]
- for h in entry["request"]["headers"]}
- all_headers.append(headers)
-
- if not all_headers:
- return set(), set()
-
- # Headers present in ALL requests are required
- required = set(all_headers[0].keys())
- for h in all_headers[1:]:
- required &= set(h.keys())
-
- # Headers present in SOME requests are optional
- all_keys = set()
- for h in all_headers:
- all_keys.update(h.keys())
- optional = all_keys - required
-
- # Filter out browser-added headers
- browser_headers = {
- 'accept', 'accept-encoding', 'accept-language',
- 'connection', 'host', 'user-agent', 'sec-ch-ua',
- 'sec-ch-ua-mobile', 'sec-ch-ua-platform', 'sec-fetch-dest',
- 'sec-fetch-mode', 'sec-fetch-site', 'referer', 'origin'
- }
-
- required -= browser_headers
- optional -= browser_headers
-
- return required, optional
-```
-
-## Common Patterns
-
-### Pagination
-Look for:
-- Query params: `page`, `offset`, `limit`, `cursor`, `after`, `before`
-- Response fields: `next`, `previous`, `total`, `has_more`, `next_cursor`
-
-### Filtering
-Look for:
-- Query params: `filter`, `q`, `search`, `query`, `sort`, `order`
-
-### GraphQL
-Look for:
-- POST to `/graphql`
-- Body with `query` and `variables` fields
-- Response with `data` and `errors` fields
-
-## Output Format
-
-After analysis, generate a summary:
-
-```markdown
-## API Summary
-
-### Base URL
-https://api.example.com
-
-### Authentication
-Bearer token in Authorization header
-
-### Endpoints
-
-#### GET /api/users
-- Query params: page (int), limit (int)
-- Response: Array of User objects
-
-#### POST /api/users
-- Body: { name: string, email: string }
-- Response: User object
-
-#### GET /api/users/{id}
-- Path params: id (int)
-- Response: User object
-```
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_analyze.py b/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_analyze.py
deleted file mode 100644
index d5b6113..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_analyze.py
+++ /dev/null
@@ -1,554 +0,0 @@
-#!/usr/bin/env python3
-"""
-HAR Analyzer - Extract structured endpoint information from HAR files.
-
-This script analyzes filtered HAR files to extract:
-- API endpoints with path parameters
-- Authentication patterns
-- Request/response schemas
-- Pagination patterns
-- Query parameters
-
-Usage:
- python har_analyze.py <filtered.har> --output <analysis.json>
-"""
-
-import argparse
-import json
-import sys
-from collections import defaultdict
-from pathlib import Path
-from typing import Dict, List, Any, Optional, Set, Tuple
-from urllib.parse import urlparse, parse_qs
-
-from har_utils import (
- load_har,
- save_json,
- extract_url_parts,
- get_base_url,
- is_path_parameter,
- get_request_body,
- get_response_body,
- get_request_headers,
- get_response_headers,
-)
-
-
-def detect_auth(entries: List[Dict[str, Any]]) -> Dict[str, Any]:
- """
- Detect authentication patterns from HAR entries.
-
- Args:
- entries: List of HAR entries
-
- Returns:
- Dictionary describing authentication mechanism
- """
- auth_headers = defaultdict(int)
- auth_cookies = defaultdict(int)
- auth_query_params = defaultdict(int)
-
- for entry in entries:
- request_headers = get_request_headers(entry)
- request = entry.get('request', {})
- url = request.get('url', '')
- parts = extract_url_parts(url)
- query_params = parts.get('query_params', {})
-
- # Check for Authorization header
- for header_name, header_value in request_headers.items():
- header_lower = header_name.lower()
-
- if header_lower == 'authorization':
- # Determine auth type from value
- value_lower = header_value.lower()
- if value_lower.startswith('bearer '):
- auth_headers['bearer_token'] += 1
- elif value_lower.startswith('basic '):
- auth_headers['basic_auth'] += 1
- elif 'oauth' in value_lower:
- auth_headers['oauth'] += 1
- else:
- auth_headers['unknown_token'] += 1
-
- # Check for X-API-Key or similar
- elif 'api' in header_lower and 'key' in header_lower:
- auth_headers['api_key_header'] += 1
-
- # Check for CSRF tokens
- elif 'csrf' in header_lower or 'xsrf' in header_lower:
- auth_headers['csrf_token'] += 1
-
- # Check for session tokens
- elif 'token' in header_lower:
- auth_headers['custom_token'] += 1
-
- # Check for cookies
- elif header_lower == 'cookie':
- if 'session' in header_value.lower():
- auth_cookies['session_cookie'] += 1
- elif 'token' in header_value.lower():
- auth_cookies['token_cookie'] += 1
- else:
- auth_cookies['generic_cookie'] += 1
-
- # Check for API keys in query parameters
- for param_name in query_params.keys():
- param_lower = param_name.lower()
- if 'api' in param_lower and 'key' in param_lower:
- auth_query_params['api_key_query'] += 1
- elif 'token' in param_lower or 'auth' in param_lower:
- auth_query_params['token_query'] += 1
-
- # Determine primary auth method
- all_auth_methods = {}
- all_auth_methods.update(auth_headers)
- all_auth_methods.update(auth_cookies)
- all_auth_methods.update(auth_query_params)
-
- if not all_auth_methods:
- return {
- 'type': 'none',
- 'location': None,
- 'header_name': None,
- 'pattern': None,
- 'description': 'No authentication detected (public API)',
- }
-
- # Find most common auth method
- primary_method = max(all_auth_methods.items(), key=lambda x: x[1])[0]
-
- # Map to structured auth info
- auth_info = {
- 'type': primary_method,
- 'detected_methods': list(all_auth_methods.keys()),
- }
-
- # Add details based on type
- if primary_method == 'bearer_token':
- auth_info.update({
- 'location': 'header',
- 'header_name': 'Authorization',
- 'pattern': 'Bearer {token}',
- 'description': 'JWT or Bearer token authentication',
- })
- elif primary_method == 'api_key_header':
- auth_info.update({
- 'location': 'header',
- 'header_name': 'X-API-Key', # Common convention
- 'pattern': '{api_key}',
- 'description': 'API key in header',
- })
- elif primary_method == 'api_key_query':
- auth_info.update({
- 'location': 'query',
- 'header_name': None,
- 'pattern': '?api_key={key}',
- 'description': 'API key in query parameter',
- })
- elif 'session_cookie' in primary_method or 'cookie' in primary_method:
- auth_info.update({
- 'location': 'cookie',
- 'header_name': 'Cookie',
- 'pattern': 'session={session_id}',
- 'description': 'Session-based authentication with cookies',
- })
- elif primary_method == 'basic_auth':
- auth_info.update({
- 'location': 'header',
- 'header_name': 'Authorization',
- 'pattern': 'Basic {base64_credentials}',
- 'description': 'HTTP Basic Authentication',
- })
- else:
- auth_info.update({
- 'location': 'header',
- 'header_name': 'Authorization',
- 'pattern': '{token}',
- 'description': f'Custom authentication: {primary_method}',
- })
-
- return auth_info
-
-
-def infer_path_params(paths: List[str]) -> str:
- """
- Infer path parameter pattern from multiple paths.
-
- Example:
- ["/api/users/123", "/api/users/456"] -> "/api/users/{id}"
-
- Args:
- paths: List of URL paths
-
- Returns:
- Path pattern with {param} placeholders
- """
- if not paths:
- return ''
-
- # Split all paths into segments
- all_segments = [path.strip('/').split('/') for path in paths]
-
- # Find common length
- min_length = min(len(segments) for segments in all_segments)
-
- # Build pattern
- pattern_segments = []
- for i in range(min_length):
- segment_values = [segments[i] for segments in all_segments]
-
- # Check if all values are the same (static segment)
- if len(set(segment_values)) == 1:
- pattern_segments.append(segment_values[0])
- else:
- # Values differ - likely a parameter
- # Check what type of parameter
- if all(is_path_parameter(val) for val in segment_values):
- # Determine parameter name
- if all(val.isdigit() for val in segment_values):
- param_name = 'id'
- else:
- param_name = 'uuid' # or 'id' for UUIDs
- pattern_segments.append(f'{{{param_name}}}')
- else:
- # Mixed values - keep first as example
- pattern_segments.append(segment_values[0])
-
- pattern = '/' + '/'.join(pattern_segments)
- return pattern
-
-
-def group_endpoints(entries: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
- """
- Group HAR entries by endpoint pattern.
-
- Args:
- entries: List of HAR entries
-
- Returns:
- List of endpoint dictionaries with patterns and metadata
- """
- # Group by method + base pattern
- groups = defaultdict(lambda: {
- 'urls': [],
- 'entries': [],
- 'query_params': defaultdict(int),
- 'request_bodies': [],
- 'response_bodies': [],
- })
-
- for entry in entries:
- request = entry.get('request', {})
- method = request.get('method', 'GET')
- url = request.get('url', '')
-
- parts = extract_url_parts(url)
- path = parts.get('path', '')
- query_params = parts.get('query_params', {})
-
- # Create a preliminary grouping key
- # We'll refine this later to detect path parameters
- key = (method, path)
- groups[key]['urls'].append(url)
- groups[key]['entries'].append(entry)
-
- # Track query parameters
- for param_name in query_params.keys():
- groups[key]['query_params'][param_name] += 1
-
- # Collect request/response bodies
- req_body = get_request_body(entry)
- if req_body:
- groups[key]['request_bodies'].append(req_body)
-
- resp_body = get_response_body(entry)
- if resp_body:
- groups[key]['response_bodies'].append(resp_body)
-
- # Now refine groups to detect path parameters
- refined_endpoints = []
-
- for (method, path), group_data in groups.items():
- # If multiple URLs with same method+path structure, infer pattern
- unique_paths = list(set(parts.get('path', '') for parts in
- [extract_url_parts(url) for url in group_data['urls']]))
-
- if len(unique_paths) > 1:
- # Multiple different paths - infer pattern
- pattern = infer_path_params(unique_paths)
- else:
- pattern = path
-
- # Determine required vs optional query params
- total_calls = len(group_data['entries'])
- required_params = []
- optional_params = []
-
- for param_name, count in group_data['query_params'].items():
- if count == total_calls:
- required_params.append(param_name)
- else:
- optional_params.append(param_name)
-
- # Merge request/response bodies to infer schema
- request_schema = merge_schemas(group_data['request_bodies'])
- response_schema = merge_schemas(group_data['response_bodies'])
-
- # Check if auth required (heuristic: has auth headers)
- has_auth = any(
- 'authorization' in get_request_headers(entry).keys() or
- 'cookie' in (h.lower() for h in get_request_headers(entry).keys())
- for entry in group_data['entries']
- )
-
- endpoint_info = {
- 'pattern': pattern,
- 'methods': [method],
- 'calls_observed': total_calls,
- 'query_params': {
- 'required': sorted(required_params),
- 'optional': sorted(optional_params),
- },
- 'request_body_schema': request_schema,
- 'response_schema': response_schema,
- 'requires_auth': has_auth,
- }
-
- refined_endpoints.append(endpoint_info)
-
- # Merge endpoints with same pattern but different methods
- pattern_groups = defaultdict(list)
- for endpoint in refined_endpoints:
- pattern_groups[endpoint['pattern']].append(endpoint)
-
- final_endpoints = []
- for pattern, endpoints in pattern_groups.items():
- if len(endpoints) == 1:
- final_endpoints.append(endpoints[0])
- else:
- # Merge multiple methods for same pattern
- merged = {
- 'pattern': pattern,
- 'methods': sorted(set(method for ep in endpoints for method in ep['methods'])),
- 'calls_observed': sum(ep['calls_observed'] for ep in endpoints),
- 'query_params': endpoints[0]['query_params'], # Use first as representative
- 'request_body_schema': merge_schemas([ep['request_body_schema'] for ep in endpoints if ep['request_body_schema']]),
- 'response_schema': merge_schemas([ep['response_schema'] for ep in endpoints if ep['response_schema']]),
- 'requires_auth': any(ep['requires_auth'] for ep in endpoints),
- }
- final_endpoints.append(merged)
-
- # Sort by pattern for consistency
- final_endpoints.sort(key=lambda x: x['pattern'])
-
- return final_endpoints
-
-
-def merge_schemas(bodies: List[Dict[str, Any]]) -> Optional[Dict[str, Any]]:
- """
- Merge multiple JSON bodies to infer a common schema.
-
- Args:
- bodies: List of JSON bodies
-
- Returns:
- Inferred schema or None
- """
- if not bodies:
- return None
-
- # Simple schema inference: collect all keys and their types
- schema = {}
-
- for body in bodies:
- if not isinstance(body, dict):
- continue
-
- for key, value in body.items():
- if key not in schema:
- schema[key] = type(value).__name__
-
- return schema if schema else None
-
-
-def detect_pagination(entries: List[Dict[str, Any]]) -> Dict[str, Any]:
- """
- Detect pagination patterns from HAR entries.
-
- Args:
- entries: List of HAR entries
-
- Returns:
- Dictionary describing pagination pattern
- """
- pagination_params = defaultdict(int)
-
- for entry in entries:
- request = entry.get('request', {})
- url = request.get('url', '')
- parts = extract_url_parts(url)
- query_params = parts.get('query_params', {})
-
- # Check for common pagination parameters
- for param_name in query_params.keys():
- param_lower = param_name.lower()
-
- if param_lower in ['page', 'p']:
- pagination_params['page'] += 1
- elif param_lower in ['limit', 'per_page', 'page_size', 'size']:
- pagination_params['limit'] += 1
- elif param_lower in ['offset', 'skip']:
- pagination_params['offset'] += 1
- elif param_lower in ['cursor', 'next', 'continuation']:
- pagination_params['cursor'] += 1
-
- if not pagination_params:
- return {
- 'detected': False,
- 'type': None,
- 'params': [],
- }
-
- # Determine pagination type
- if 'page' in pagination_params:
- return {
- 'detected': True,
- 'type': 'page',
- 'params': ['page', 'limit'] if 'limit' in pagination_params else ['page'],
- 'description': 'Page-based pagination',
- }
- elif 'offset' in pagination_params:
- return {
- 'detected': True,
- 'type': 'offset',
- 'params': ['offset', 'limit'] if 'limit' in pagination_params else ['offset'],
- 'description': 'Offset-based pagination',
- }
- elif 'cursor' in pagination_params:
- return {
- 'detected': True,
- 'type': 'cursor',
- 'params': ['cursor', 'limit'] if 'limit' in pagination_params else ['cursor'],
- 'description': 'Cursor-based pagination',
- }
- else:
- return {
- 'detected': True,
- 'type': 'unknown',
- 'params': list(pagination_params.keys()),
- 'description': 'Custom pagination pattern',
- }
-
-
-def analyze_har(har: Dict[str, Any]) -> Dict[str, Any]:
- """
- Analyze HAR file and extract structured information.
-
- Args:
- har: Parsed HAR data
-
- Returns:
- Analysis dictionary with endpoints, auth, pagination, etc.
- """
- entries = har['log']['entries']
-
- if not entries:
- return {
- 'error': 'No entries found in HAR file',
- 'base_url': None,
- 'authentication': None,
- 'endpoints': [],
- 'pagination': None,
- }
-
- # Determine base URL (most common)
- base_urls = defaultdict(int)
- for entry in entries:
- request = entry.get('request', {})
- url = request.get('url', '')
- base = get_base_url(url)
- if base:
- base_urls[base] += 1
-
- base_url = max(base_urls.items(), key=lambda x: x[1])[0] if base_urls else None
-
- # Detect authentication
- authentication = detect_auth(entries)
-
- # Group and analyze endpoints
- endpoints = group_endpoints(entries)
-
- # Detect pagination
- pagination = detect_pagination(entries)
-
- # Build analysis result
- analysis = {
- 'base_url': base_url,
- 'authentication': authentication,
- 'endpoints': endpoints,
- 'pagination': pagination,
- 'total_entries_analyzed': len(entries),
- 'unique_endpoints': len(endpoints),
- }
-
- return analysis
-
-
-def main():
- """Main entry point for CLI."""
- parser = argparse.ArgumentParser(
- description='Analyze HAR files to extract endpoint information',
- formatter_class=argparse.RawDescriptionHelpFormatter,
- epilog="""
-Examples:
- # Analyze HAR and save to JSON
- python har_analyze.py filtered.har --output analysis.json
-
- # Print analysis to stdout
- python har_analyze.py filtered.har
- """
- )
-
- parser.add_argument(
- 'input',
- type=str,
- help='Input HAR file path (filtered recommended)'
- )
-
- parser.add_argument(
- '--output',
- type=str,
- help='Output analysis JSON file path'
- )
-
- args = parser.parse_args()
-
- # Validate input file
- input_path = Path(args.input).expanduser().resolve()
- if not input_path.exists():
- print(f"Error: Input file not found: {input_path}", file=sys.stderr)
- sys.exit(1)
-
- try:
- # Load and analyze HAR
- har_data = load_har(input_path)
- analysis = analyze_har(har_data)
-
- # Save or print analysis
- if args.output:
- output_path = Path(args.output).expanduser().resolve()
- save_json(analysis, output_path, pretty=True)
- print(f"Analysis saved to: {output_path}")
- else:
- print(json.dumps(analysis, indent=2))
-
- except Exception as e:
- print(f"Error: {e}", file=sys.stderr)
- import traceback
- traceback.print_exc()
- sys.exit(1)
-
-
-if __name__ == '__main__':
- main()
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_filter.py b/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_filter.py
deleted file mode 100644
index d363624..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_filter.py
+++ /dev/null
@@ -1,438 +0,0 @@
-#!/usr/bin/env python3
-"""
-HAR Filter - Filter HAR files to API-relevant entries only.
-
-This script removes static assets, analytics, tracking, and CDN resources
-from HAR files, keeping only API-relevant requests.
-
-Usage:
- python har_filter.py <input.har> --output <filtered.har> [--stats]
-"""
-
-import argparse
-import json
-import sys
-from pathlib import Path
-from typing import Dict, List, Tuple, Any
-from urllib.parse import urlparse
-
-from har_utils import load_har, save_json, get_content_type, extract_url_parts
-
-
-# File extensions to skip (static assets)
-SKIP_EXTENSIONS = {
- # JavaScript/CSS
- '.js', '.jsx', '.ts', '.tsx', '.css', '.scss', '.sass', '.less',
- # Images
- '.png', '.jpg', '.jpeg', '.gif', '.webp', '.svg', '.ico', '.bmp', '.tiff',
- # Fonts
- '.woff', '.woff2', '.ttf', '.otf', '.eot',
- # Media
- '.mp4', '.webm', '.ogg', '.mp3', '.wav', '.flac',
- # Documents (usually not API responses)
- '.pdf', '.doc', '.docx', '.xls', '.xlsx', '.zip', '.tar', '.gz',
- # Other static files
- '.xml', '.txt', '.csv', '.html', '.htm',
- # Source maps and manifests
- '.map', '.manifest',
-}
-
-# Domain patterns to skip (analytics, tracking, CDN)
-SKIP_DOMAIN_PATTERNS = [
- # Analytics
- 'google-analytics.com',
- 'analytics.google.com',
- 'googletagmanager.com',
- 'doubleclick.net',
- 'facebook.net',
- 'connect.facebook.net',
- 'mixpanel.com',
- 'segment.com',
- 'segment.io',
- 'amplitude.com',
- 'hotjar.com',
- 'fullstory.com',
- 'heap.io',
- 'clarity.ms',
- # Ads
- 'googlesyndication.com',
- 'adservice.google.com',
- 'advertising.com',
- # CDN patterns
- 'cloudfront.net',
- 'fastly.net',
- 'akamai.net',
- 'akamaihd.net',
- 'cdn.jsdelivr.net',
- 'unpkg.com',
- 'cdnjs.cloudflare.com',
- # Other tracking
- 'bugsnag.com',
- 'sentry.io',
- 'newrelic.com',
-]
-
-# URL patterns to skip (includes path components for tracking endpoints)
-SKIP_URL_PATTERNS = [
- # Social media tracking pixels
- 'facebook.com/tr',
- 'twitter.com/i/',
- 'linkedin.com/px/',
- 'snapchat.com/tr',
-]
-
-# URL path patterns that indicate API endpoints
-API_PATH_PATTERNS = [
- '/api/',
- '/v1/',
- '/v2/',
- '/v3/',
- '/v4/',
- '/rest/',
- '/graphql',
- '/query',
- '/mutation',
- '/rpc/',
- '/_api/',
-]
-
-
-def get_skip_extensions() -> set:
- """Get set of file extensions to skip."""
- return SKIP_EXTENSIONS
-
-
-def get_skip_domain_patterns() -> list:
- """Get list of domain patterns to skip."""
- return SKIP_DOMAIN_PATTERNS
-
-
-def should_skip_domain(hostname: str) -> bool:
- """
- Check if a domain should be skipped.
-
- Args:
- hostname: Hostname from URL
-
- Returns:
- True if domain should be skipped
- """
- if not hostname:
- return False
-
- hostname_lower = hostname.lower()
-
- for pattern in SKIP_DOMAIN_PATTERNS:
- if pattern in hostname_lower:
- return True
-
- # Skip common CDN patterns like cdn.*, static.*, assets.*
- if hostname_lower.startswith(('cdn.', 'static.', 'assets.', 'media.')):
- return True
-
- return False
-
-
-def should_skip_url(url: str) -> bool:
- """
- Check if a full URL should be skipped based on URL patterns.
-
- Args:
- url: Full URL string
-
- Returns:
- True if URL should be skipped
- """
- if not url:
- return False
-
- url_lower = url.lower()
-
- for pattern in SKIP_URL_PATTERNS:
- if pattern in url_lower:
- return True
-
- return False
-
-
-def should_skip_extension(path: str) -> bool:
- """
- Check if file extension should be skipped.
-
- Args:
- path: URL path
-
- Returns:
- True if extension should be skipped
- """
- path_lower = path.lower()
-
- for ext in SKIP_EXTENSIONS:
- if path_lower.endswith(ext):
- return True
-
- return False
-
-
-def has_api_pattern(path: str) -> bool:
- """
- Check if path contains API patterns.
-
- Args:
- path: URL path
-
- Returns:
- True if path looks like an API endpoint
- """
- path_lower = path.lower()
-
- for pattern in API_PATH_PATTERNS:
- if pattern in path_lower:
- return True
-
- return False
-
-
-def is_xhr_fetch(entry: Dict[str, Any]) -> bool:
- """
- Check if request is XHR/Fetch (likely an API call).
-
- Args:
- entry: HAR entry
-
- Returns:
- True if request appears to be XHR/Fetch
- """
- request = entry.get('request', {})
- headers = request.get('headers', [])
-
- # Check for XHR/Fetch indicators in headers
- for header in headers:
- if not isinstance(header, dict):
- continue
-
- name = header.get('name', '').lower()
- value = header.get('value', '').lower()
-
- # X-Requested-With: XMLHttpRequest
- if name == 'x-requested-with' and 'xmlhttprequest' in value:
- return True
-
- # Fetch API usually sets specific Accept headers
- if name == 'accept' and ('application/json' in value or 'application/*' in value):
- return True
-
- return False
-
-
-def is_json_response(entry: Dict[str, Any]) -> bool:
- """
- Check if response is JSON.
-
- Args:
- entry: HAR entry
-
- Returns:
- True if response content-type is JSON
- """
- content_type = get_content_type(entry)
- if content_type:
- return 'json' in content_type.lower()
- return False
-
-
-def categorize_entry(entry: Dict[str, Any]) -> str:
- """
- Categorize a HAR entry.
-
- Args:
- entry: HAR entry
-
- Returns:
- Category: "api", "static", "analytics", "cdn", or "other"
- """
- request = entry.get('request', {})
- url = request.get('url', '')
-
- parts = extract_url_parts(url)
- hostname = parts.get('hostname', '')
- path = parts.get('path', '')
-
- # Check URL patterns first (includes path components)
- if should_skip_url(url):
- return 'analytics'
-
- # Check domain-based skips
- if should_skip_domain(hostname):
- # Categorize analytics vs CDN
- hostname_lower = hostname.lower()
- if any(term in hostname_lower for term in ['analytics', 'tracking', 'ads', 'doubleclick']):
- return 'analytics'
- return 'cdn'
-
- # Check file extension
- if should_skip_extension(path):
- return 'static'
-
- # Check for API patterns
- if has_api_pattern(path):
- return 'api'
-
- # Check if XHR/Fetch
- if is_xhr_fetch(entry):
- return 'api'
-
- # Check if JSON response
- if is_json_response(entry):
- return 'api'
-
- return 'other'
-
-
-def is_api_endpoint(entry: Dict[str, Any]) -> bool:
- """
- Determine if a HAR entry is an API endpoint.
-
- Args:
- entry: HAR entry
-
- Returns:
- True if entry should be kept as API endpoint
- """
- category = categorize_entry(entry)
- return category == 'api'
-
-
-def filter_har(har_path: Path) -> Tuple[Dict[str, Any], Dict[str, Any]]:
- """
- Filter HAR file to API-relevant entries only.
-
- Args:
- har_path: Path to input HAR file
-
- Returns:
- Tuple of (filtered_har, statistics)
- """
- har_data = load_har(har_path)
-
- entries = har_data['log']['entries']
- total_entries = len(entries)
-
- # Categorize all entries
- categorized = {
- 'api': [],
- 'static': [],
- 'analytics': [],
- 'cdn': [],
- 'other': [],
- }
-
- for entry in entries:
- category = categorize_entry(entry)
- categorized[category].append(entry)
-
- # Build filtered HAR with only API entries
- filtered_har = {
- 'log': {
- 'version': har_data['log'].get('version', '1.2'),
- 'creator': har_data['log'].get('creator', {}),
- 'pages': har_data['log'].get('pages', []),
- 'entries': categorized['api'],
- }
- }
-
- # Collect API patterns found
- api_patterns = set()
- for entry in categorized['api']:
- request = entry.get('request', {})
- url = request.get('url', '')
- parts = extract_url_parts(url)
- path = parts.get('path', '')
-
- for pattern in API_PATH_PATTERNS:
- if pattern in path.lower():
- api_patterns.add(pattern)
-
- # Build statistics
- stats = {
- 'total_entries': total_entries,
- 'filtered_entries': len(categorized['api']),
- 'removed_static': len(categorized['static']),
- 'removed_analytics': len(categorized['analytics']),
- 'removed_cdn': len(categorized['cdn']),
- 'removed_other': len(categorized['other']),
- 'api_patterns_found': sorted(list(api_patterns)),
- 'filter_ratio': round(len(categorized['api']) / total_entries * 100, 1) if total_entries > 0 else 0,
- }
-
- return filtered_har, stats
-
-
-def main():
- """Main entry point for CLI."""
- parser = argparse.ArgumentParser(
- description='Filter HAR files to API-relevant entries only',
- formatter_class=argparse.RawDescriptionHelpFormatter,
- epilog="""
-Examples:
- # Filter HAR and save to new file
- python har_filter.py recording.har --output filtered.har
-
- # Filter and display stats
- python har_filter.py recording.har --output filtered.har --stats
-
- # Stats only (no output file)
- python har_filter.py recording.har --stats
- """
- )
-
- parser.add_argument(
- 'input',
- type=str,
- help='Input HAR file path'
- )
-
- parser.add_argument(
- '--output',
- type=str,
- help='Output filtered HAR file path'
- )
-
- parser.add_argument(
- '--stats',
- action='store_true',
- help='Print filtering statistics'
- )
-
- args = parser.parse_args()
-
- # Validate input file
- input_path = Path(args.input).expanduser().resolve()
- if not input_path.exists():
- print(f"Error: Input file not found: {input_path}", file=sys.stderr)
- sys.exit(1)
-
- try:
- # Filter HAR
- filtered_har, stats = filter_har(input_path)
-
- # Save filtered HAR if output specified
- if args.output:
- output_path = Path(args.output).expanduser().resolve()
- save_json(filtered_har, output_path, pretty=True)
- stats['filtered_har_path'] = str(output_path)
-
- # Print stats if requested
- if args.stats:
- print(json.dumps(stats, indent=2))
- elif not args.output:
- # If neither output nor stats requested, print stats by default
- print(json.dumps(stats, indent=2))
-
- except Exception as e:
- print(f"Error: {e}", file=sys.stderr)
- sys.exit(1)
-
-
-if __name__ == '__main__':
- main()
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_utils.py b/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_utils.py
deleted file mode 100644
index ba04bc3..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_utils.py
+++ /dev/null
@@ -1,405 +0,0 @@
-#!/usr/bin/env python3
-"""
-Shared utilities for HAR analysis scripts.
-
-This module provides common functions used across har_filter.py,
-har_analyze.py, and har_validate.py.
-"""
-
-import json
-import re
-from pathlib import Path
-from typing import Any, Dict, Optional
-from urllib.parse import urlparse, parse_qs
-
-
-def load_har(har_path: Path) -> Dict[str, Any]:
- """
- Load and parse a HAR file.
-
- Args:
- har_path: Path to the HAR file
-
- Returns:
- Parsed HAR data as dictionary
-
- Raises:
- FileNotFoundError: If HAR file doesn't exist
- json.JSONDecodeError: If HAR file is invalid JSON
- ValueError: If HAR structure is invalid
- """
- if not har_path.exists():
- raise FileNotFoundError(f"HAR file not found: {har_path}")
-
- with open(har_path, 'r', encoding='utf-8') as f:
- har_data = json.load(f)
-
- if not validate_har(har_data):
- raise ValueError("Invalid HAR structure")
-
- return har_data
-
-
-def validate_har(har: Dict[str, Any]) -> bool:
- """
- Validate HAR structure.
-
- Args:
- har: HAR data dictionary
-
- Returns:
- True if valid, False otherwise
- """
- if not isinstance(har, dict):
- return False
-
- if 'log' not in har:
- return False
-
- log = har['log']
- if not isinstance(log, dict):
- return False
-
- if 'entries' not in log:
- return False
-
- return isinstance(log['entries'], list)
-
-
-def get_content_type(entry: Dict[str, Any]) -> Optional[str]:
- """
- Extract content-type from HAR entry.
-
- Args:
- entry: HAR entry dictionary
-
- Returns:
- Content-type string or None if not found
- """
- response = entry.get('response', {})
- headers = response.get('headers', [])
-
- for header in headers:
- if isinstance(header, dict):
- name = header.get('name', '').lower()
- if name == 'content-type':
- return header.get('value', '')
-
- return None
-
-
-def parse_json_safe(text: str) -> Optional[Dict[str, Any]]:
- """
- Safely parse JSON text.
-
- Args:
- text: JSON string to parse
-
- Returns:
- Parsed dictionary or None if parsing fails
- """
- if not text or not isinstance(text, str):
- return None
-
- try:
- return json.loads(text)
- except (json.JSONDecodeError, ValueError):
- return None
-
-
-def is_uuid(s: str) -> bool:
- """
- Check if a string is a UUID.
-
- Args:
- s: String to check
-
- Returns:
- True if string matches UUID pattern
- """
- uuid_pattern = re.compile(
- r'^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$',
- re.IGNORECASE
- )
- return bool(uuid_pattern.match(s))
-
-
-def is_numeric_id(s: str) -> bool:
- """
- Check if a string is a numeric ID.
-
- Args:
- s: String to check
-
- Returns:
- True if string is all digits
- """
- return s.isdigit()
-
-
-def is_path_parameter(s: str) -> bool:
- """
- Check if a URL path segment is likely a parameter.
-
- Args:
- s: Path segment to check
-
- Returns:
- True if likely a parameter (UUID, numeric ID, etc.)
- """
- if not s or s.startswith('.'):
- return False
-
- # UUID
- if is_uuid(s):
- return True
-
- # Numeric ID (but not too short, could be version number)
- if is_numeric_id(s) and len(s) >= 3:
- return True
-
- # Long alphanumeric strings (likely IDs)
- if len(s) > 16 and s.isalnum():
- return True
-
- # Base64-like strings
- if len(s) > 20 and re.match(r'^[A-Za-z0-9_-]+$', s):
- return True
-
- return False
-
-
-def sanitize_method_name(endpoint: str, method: str = 'GET') -> str:
- """
- Convert an endpoint pattern to a Python method name.
-
- Examples:
- "/api/users/{id}" -> "get_user"
- "/api/v1/products/{id}/reviews" -> "get_product_reviews"
- "/graphql" -> "post_graphql"
-
- Args:
- endpoint: URL path pattern (e.g., "/api/users/{id}")
- method: HTTP method (e.g., "GET", "POST")
-
- Returns:
- Python method name (snake_case)
- """
- # Remove leading/trailing slashes
- path = endpoint.strip('/')
-
- # Remove common prefixes
- for prefix in ['api/', 'v1/', 'v2/', 'v3/', 'api/v1/', 'api/v2/', 'api/v3/']:
- if path.startswith(prefix):
- path = path[len(prefix):]
-
- # Split by slashes and filter out parameter placeholders
- parts = []
- for segment in path.split('/'):
- # Skip parameter placeholders like {id}, {uuid}
- if segment.startswith('{') and segment.endswith('}'):
- continue
- # Skip numeric segments (version numbers)
- if segment.isdigit():
- continue
- # Add the segment
- if segment:
- parts.append(segment)
-
- # Join parts with underscores
- if not parts:
- # Fallback for paths like /api/{id}
- parts = ['item']
-
- base_name = '_'.join(parts)
-
- # Singularize if method suggests single item
- if method in ['GET', 'PUT', 'PATCH', 'DELETE']:
- # Check if path has parameter (suggesting single item)
- if '{id}' in endpoint or '{uuid}' in endpoint:
- # Simple singularization (remove trailing 's')
- if base_name.endswith('ies'):
- base_name = base_name[:-3] + 'y'
- elif base_name.endswith('s') and not base_name.endswith('ss'):
- base_name = base_name[:-1]
-
- # Add method prefix
- method_prefix = method.lower()
- if method_prefix == 'delete':
- method_prefix = 'remove'
-
- method_name = f"{method_prefix}_{base_name}"
-
- # Clean up any double underscores
- method_name = re.sub(r'_+', '_', method_name)
-
- # Ensure it's a valid Python identifier
- method_name = re.sub(r'[^a-z0-9_]', '_', method_name)
-
- return method_name
-
-
-def extract_url_parts(url: str) -> Dict[str, Any]:
- """
- Extract components from a URL.
-
- Args:
- url: Full URL string
-
- Returns:
- Dictionary with scheme, netloc, path, query_params
- """
- parsed = urlparse(url)
- query_params = parse_qs(parsed.query)
-
- return {
- 'scheme': parsed.scheme,
- 'netloc': parsed.netloc,
- 'hostname': parsed.hostname,
- 'port': parsed.port,
- 'path': parsed.path,
- 'query': parsed.query,
- 'query_params': query_params,
- 'fragment': parsed.fragment,
- }
-
-
-def get_base_url(url: str) -> str:
- """
- Extract base URL (scheme + netloc) from full URL.
-
- Args:
- url: Full URL string
-
- Returns:
- Base URL (e.g., "https://api.example.com")
- """
- parsed = urlparse(url)
- return f"{parsed.scheme}://{parsed.netloc}"
-
-
-def normalize_path(path: str) -> str:
- """
- Normalize a URL path by removing trailing slashes.
-
- Args:
- path: URL path
-
- Returns:
- Normalized path
- """
- path = path.strip()
- # Remove trailing slash unless it's the root path
- if len(path) > 1 and path.endswith('/'):
- path = path[:-1]
- return path
-
-
-def get_request_body(entry: Dict[str, Any]) -> Optional[Dict[str, Any]]:
- """
- Extract and parse request body from HAR entry.
-
- Args:
- entry: HAR entry dictionary
-
- Returns:
- Parsed request body as dict, or None if not JSON
- """
- request = entry.get('request', {})
- post_data = request.get('postData', {})
-
- mime_type = post_data.get('mimeType', '')
- text = post_data.get('text', '')
-
- if 'json' in mime_type.lower():
- return parse_json_safe(text)
-
- return None
-
-
-def get_response_body(entry: Dict[str, Any]) -> Optional[Dict[str, Any]]:
- """
- Extract and parse response body from HAR entry.
-
- Args:
- entry: HAR entry dictionary
-
- Returns:
- Parsed response body as dict, or None if not JSON
- """
- response = entry.get('response', {})
- content = response.get('content', {})
-
- mime_type = content.get('mimeType', '')
- text = content.get('text', '')
-
- if 'json' in mime_type.lower():
- return parse_json_safe(text)
-
- return None
-
-
-def get_request_headers(entry: Dict[str, Any]) -> Dict[str, str]:
- """
- Extract request headers as a dictionary.
-
- Args:
- entry: HAR entry dictionary
-
- Returns:
- Dictionary of header name -> value
- """
- request = entry.get('request', {})
- headers = request.get('headers', [])
-
- result = {}
- for header in headers:
- if isinstance(header, dict):
- name = header.get('name', '')
- value = header.get('value', '')
- if name:
- result[name] = value
-
- return result
-
-
-def get_response_headers(entry: Dict[str, Any]) -> Dict[str, str]:
- """
- Extract response headers as a dictionary.
-
- Args:
- entry: HAR entry dictionary
-
- Returns:
- Dictionary of header name -> value
- """
- response = entry.get('response', {})
- headers = response.get('headers', [])
-
- result = {}
- for header in headers:
- if isinstance(header, dict):
- name = header.get('name', '')
- value = header.get('value', '')
- if name:
- result[name] = value
-
- return result
-
-
-def save_json(data: Dict[str, Any], output_path: Path, pretty: bool = True) -> None:
- """
- Save data as JSON file.
-
- Args:
- data: Data to save
- output_path: Output file path
- pretty: Whether to pretty-print the JSON
- """
- output_path.parent.mkdir(parents=True, exist_ok=True)
-
- with open(output_path, 'w', encoding='utf-8') as f:
- if pretty:
- json.dump(data, f, indent=2, ensure_ascii=False)
- else:
- json.dump(data, f, ensure_ascii=False)
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_validate.py b/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_validate.py
deleted file mode 100644
index b609a72..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/scripts/har_validate.py
+++ /dev/null
@@ -1,457 +0,0 @@
-#!/usr/bin/env python3
-"""
-HAR Validator - Validate generated API client against HAR analysis.
-
-This script validates that a generated API client properly implements
-all endpoints found in the HAR analysis.
-
-Usage:
- python har_validate.py <api_client.py> <analysis.json>
-"""
-
-import argparse
-import ast
-import json
-import re
-import sys
-from pathlib import Path
-from typing import Dict, List, Any, Set, Tuple
-
-
-def load_analysis(analysis_path: Path) -> Dict[str, Any]:
- """Load analysis JSON file."""
- if not analysis_path.exists():
- raise FileNotFoundError(f"Analysis file not found: {analysis_path}")
-
- with open(analysis_path, 'r', encoding='utf-8') as f:
- return json.load(f)
-
-
-def load_client_code(client_path: Path) -> str:
- """Load API client Python code."""
- if not client_path.exists():
- raise FileNotFoundError(f"Client file not found: {client_path}")
-
- with open(client_path, 'r', encoding='utf-8') as f:
- return f.read()
-
-
-def parse_client_methods(client_code: str) -> Set[str]:
- """
- Extract method names from API client code.
-
- Args:
- client_code: Python source code
-
- Returns:
- Set of method names (excluding private methods and __init__)
- """
- methods = set()
-
- try:
- tree = ast.parse(client_code)
-
- for node in ast.walk(tree):
- if isinstance(node, ast.ClassDef):
- for item in node.body:
- if isinstance(item, ast.FunctionDef):
- method_name = item.name
- # Skip private methods and __init__
- if not method_name.startswith('_'):
- methods.add(method_name)
-
- except SyntaxError:
- # If AST parsing fails, fall back to regex
- pattern = r'def\s+([a-z_][a-z0-9_]*)\s*\('
- matches = re.findall(pattern, client_code, re.MULTILINE)
- methods = set(m for m in matches if not m.startswith('_'))
-
- return methods
-
-
-def endpoint_to_method_names(pattern: str, methods: List[str]) -> List[str]:
- """
- Generate expected method names from endpoint pattern.
-
- Args:
- pattern: Endpoint pattern (e.g., "/api/users/{id}")
- methods: HTTP methods (e.g., ["GET", "POST"])
-
- Returns:
- List of expected Python method names
- """
- # Remove leading/trailing slashes
- path = pattern.strip('/')
-
- # Remove common prefixes
- for prefix in ['api/', 'v1/', 'v2/', 'v3/', 'api/v1/', 'api/v2/', 'api/v3/']:
- if path.startswith(prefix):
- path = path[len(prefix):]
-
- # Split and filter
- parts = []
- for segment in path.split('/'):
- # Skip parameter placeholders
- if segment.startswith('{') and segment.endswith('}'):
- continue
- # Skip numeric segments
- if segment.isdigit():
- continue
- if segment:
- parts.append(segment)
-
- if not parts:
- parts = ['item']
-
- base_name = '_'.join(parts)
-
- # Singularize if needed (simple approach)
- method_names = []
- for method in methods:
- name = base_name
- if method in ['GET', 'PUT', 'PATCH', 'DELETE']:
- # Check if pattern has parameter (suggests single item)
- if '{id}' in pattern or '{uuid}' in pattern:
- # Singularize
- if name.endswith('ies'):
- name = name[:-3] + 'y'
- elif name.endswith('s') and not name.endswith('ss'):
- name = name[:-1]
-
- # Add method prefix
- prefix = method.lower()
- if prefix == 'delete':
- prefix = 'remove'
-
- method_name = f"{prefix}_{name}"
- method_name = re.sub(r'_+', '_', method_name)
- method_name = re.sub(r'[^a-z0-9_]', '_', method_name)
-
- method_names.append(method_name)
-
- return method_names
-
-
-def check_endpoint_coverage(
- client_code: str,
- client_methods: Set[str],
- endpoints: List[Dict[str, Any]]
-) -> Tuple[List[Dict[str, Any]], Dict[str, Any]]:
- """
- Check if all endpoints from analysis are implemented.
-
- Args:
- client_code: Python source code
- client_methods: Set of implemented method names
- endpoints: List of endpoint dictionaries from analysis
-
- Returns:
- Tuple of (issues, coverage_stats)
- """
- issues = []
- endpoints_covered = 0
- endpoints_total = 0
-
- for endpoint in endpoints:
- pattern = endpoint['pattern']
- methods = endpoint['methods']
-
- # Generate expected method names
- expected_names = endpoint_to_method_names(pattern, methods)
-
- # Check if any expected name is implemented
- found = any(name in client_methods for name in expected_names)
-
- if found:
- endpoints_covered += 1
- else:
- # Not implemented
- for method in methods:
- issues.append({
- 'severity': 'error',
- 'category': 'missing_endpoint',
- 'message': f"Endpoint {method} {pattern} not implemented (expected method: {expected_names[0]})",
- 'line': None,
- })
-
- endpoints_total += 1
-
- coverage = {
- 'endpoints_covered': endpoints_covered,
- 'endpoints_total': endpoints_total,
- 'percentage': round(endpoints_covered / endpoints_total * 100, 1) if endpoints_total > 0 else 0,
- }
-
- return issues, coverage
-
-
-def check_auth_implementation(
- client_code: str,
- auth: Dict[str, Any]
-) -> List[Dict[str, Any]]:
- """
- Check if authentication is properly implemented.
-
- Args:
- client_code: Python source code
- auth: Authentication dict from analysis
-
- Returns:
- List of issues
- """
- issues = []
- auth_type = auth.get('type', 'none')
-
- if auth_type == 'none':
- # No auth required
- return issues
-
- # Check for auth implementation
- auth_indicators = {
- 'bearer_token': ['Authorization', 'Bearer', 'token'],
- 'api_key_header': ['X-API-Key', 'api_key', 'apikey'],
- 'api_key_query': ['api_key', 'apikey', 'params'],
- 'session_cookie': ['cookie', 'session'],
- 'basic_auth': ['Authorization', 'Basic', 'credentials'],
- }
-
- expected_indicators = auth_indicators.get(auth_type, [])
- found_any = any(indicator.lower() in client_code.lower() for indicator in expected_indicators)
-
- if not found_any:
- issues.append({
- 'severity': 'warning',
- 'category': 'missing_auth',
- 'message': f"Authentication type '{auth_type}' not implemented (expected {auth['header_name'] or 'session handling'})",
- 'line': None,
- })
-
- return issues
-
-
-def check_error_handling(client_code: str) -> List[Dict[str, Any]]:
- """
- Check if proper error handling is implemented.
-
- Args:
- client_code: Python source code
-
- Returns:
- List of issues
- """
- issues = []
-
- # Check for try-except blocks
- has_try_except = 'try:' in client_code and 'except' in client_code
-
- # Check for custom exceptions
- has_custom_exceptions = 'class' in client_code and 'Error' in client_code and 'Exception' in client_code
-
- # Check for response validation
- has_response_check = 'raise_for_status' in client_code or 'status_code' in client_code
-
- if not has_try_except:
- issues.append({
- 'severity': 'warning',
- 'category': 'missing_error_handling',
- 'message': 'No try-except blocks found for error handling',
- 'line': None,
- })
-
- if not has_custom_exceptions:
- issues.append({
- 'severity': 'info',
- 'category': 'missing_custom_exceptions',
- 'message': 'No custom exception classes defined',
- 'line': None,
- })
-
- if not has_response_check:
- issues.append({
- 'severity': 'warning',
- 'category': 'missing_response_validation',
- 'message': 'No response status validation found',
- 'line': None,
- })
-
- return issues
-
-
-def check_type_hints(client_code: str) -> List[Dict[str, Any]]:
- """
- Check if type hints are used.
-
- Args:
- client_code: Python source code
-
- Returns:
- List of issues
- """
- issues = []
-
- # Check for typing imports
- has_typing_import = 'from typing import' in client_code or 'import typing' in client_code
-
- # Check for type hints in code (look for -> or :)
- has_type_hints = ' -> ' in client_code or ': Dict' in client_code or ': List' in client_code
-
- if not has_typing_import or not has_type_hints:
- issues.append({
- 'severity': 'info',
- 'category': 'missing_type_hints',
- 'message': 'Type hints not fully implemented',
- 'line': None,
- })
-
- return issues
-
-
-def validate_client(
- client_path: Path,
- analysis: Dict[str, Any]
-) -> Dict[str, Any]:
- """
- Validate API client against HAR analysis.
-
- Args:
- client_path: Path to API client Python file
- analysis: HAR analysis dictionary
-
- Returns:
- Validation report with score, issues, coverage
- """
- client_code = load_client_code(client_path)
- client_methods = parse_client_methods(client_code)
-
- all_issues = []
-
- # Check endpoint coverage
- endpoints = analysis.get('endpoints', [])
- endpoint_issues, coverage = check_endpoint_coverage(client_code, client_methods, endpoints)
- all_issues.extend(endpoint_issues)
-
- # Check authentication
- auth = analysis.get('authentication', {})
- auth_issues = check_auth_implementation(client_code, auth)
- all_issues.extend(auth_issues)
-
- # Check error handling
- error_issues = check_error_handling(client_code)
- all_issues.extend(error_issues)
-
- # Check type hints
- type_issues = check_type_hints(client_code)
- all_issues.extend(type_issues)
-
- # Calculate score
- # Base score starts at 100
- score = 100
-
- # Deduct points for errors
- error_count = len([i for i in all_issues if i['severity'] == 'error'])
- warning_count = len([i for i in all_issues if i['severity'] == 'warning'])
- info_count = len([i for i in all_issues if i['severity'] == 'info'])
-
- score -= error_count * 15 # -15 points per error
- score -= warning_count * 5 # -5 points per warning
- score -= info_count * 2 # -2 points per info
-
- # Bonus points for high coverage
- if coverage['percentage'] == 100:
- score += 10
-
- # Ensure score is between 0 and 100
- score = max(0, min(100, score))
-
- report = {
- 'score': score,
- 'issues': all_issues,
- 'coverage': coverage,
- 'summary': {
- 'errors': error_count,
- 'warnings': warning_count,
- 'info': info_count,
- 'total_issues': len(all_issues),
- },
- }
-
- return report
-
-
-def main():
- """Main entry point for CLI."""
- parser = argparse.ArgumentParser(
- description='Validate generated API client against HAR analysis',
- formatter_class=argparse.RawDescriptionHelpFormatter,
- epilog="""
-Examples:
- # Validate API client
- python har_validate.py api_client.py analysis.json
-
- # Save validation report to file
- python har_validate.py api_client.py analysis.json --output report.json
- """
- )
-
- parser.add_argument(
- 'client',
- type=str,
- help='API client Python file path'
- )
-
- parser.add_argument(
- 'analysis',
- type=str,
- help='HAR analysis JSON file path'
- )
-
- parser.add_argument(
- '--output',
- type=str,
- help='Output validation report JSON file path'
- )
-
- args = parser.parse_args()
-
- # Validate input files
- client_path = Path(args.client).expanduser().resolve()
- analysis_path = Path(args.analysis).expanduser().resolve()
-
- if not client_path.exists():
- print(f"Error: Client file not found: {client_path}", file=sys.stderr)
- sys.exit(1)
-
- if not analysis_path.exists():
- print(f"Error: Analysis file not found: {analysis_path}", file=sys.stderr)
- sys.exit(1)
-
- try:
- # Load analysis
- analysis = load_analysis(analysis_path)
-
- # Validate client
- report = validate_client(client_path, analysis)
-
- # Save or print report
- if args.output:
- output_path = Path(args.output).expanduser().resolve()
- output_path.parent.mkdir(parents=True, exist_ok=True)
- with open(output_path, 'w', encoding='utf-8') as f:
- json.dump(report, f, indent=2)
- print(f"Validation report saved to: {output_path}")
- else:
- print(json.dumps(report, indent=2))
-
- # Exit with error code if score < 90
- if report['score'] < 90:
- sys.exit(1)
-
- except Exception as e:
- print(f"Error: {e}", file=sys.stderr)
- import traceback
- traceback.print_exc()
- sys.exit(1)
-
-
-if __name__ == '__main__':
- main()
diff --git a/plugins/reverse-api-engineer/skills/reverse-engineering-api/templates/api_client.py b/plugins/reverse-api-engineer/skills/reverse-engineering-api/templates/api_client.py
deleted file mode 100644
index 220fa63..0000000
--- a/plugins/reverse-api-engineer/skills/reverse-engineering-api/templates/api_client.py
+++ /dev/null
@@ -1,464 +0,0 @@
-#!/usr/bin/env python3
-"""
-API Client Template for Reverse Engineered APIs
-
-This template provides a production-ready structure for API clients
-generated from HAR file analysis.
-
-Usage:
- 1. Copy this template
- 2. Update BASE_URL, headers, and authentication
- 3. Add endpoint methods based on HAR analysis
- 4. Test and refine
-"""
-
-import logging
-import time
-from typing import Any, Dict, List, Optional, Union
-from urllib.parse import urljoin
-
-import requests
-from requests.adapters import HTTPAdapter
-from urllib3.util.retry import Retry
-
-# Configure logging
-logging.basicConfig(
- level=logging.INFO,
- format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
-)
-logger = logging.getLogger(__name__)
-
-
-class APIError(Exception):
- """Custom exception for API errors."""
-
- def __init__(self, message: str, status_code: int = None, response: dict = None):
- super().__init__(message)
- self.status_code = status_code
- self.response = response
-
-
-class RateLimitError(APIError):
- """Exception for rate limit errors."""
-
- def __init__(self, message: str, retry_after: int = None):
- super().__init__(message, status_code=429)
- self.retry_after = retry_after
-
-
-class APIClient:
- """
- Production-ready API client.
-
- Features:
- - Automatic retry with exponential backoff
- - Session management for connection pooling
- - Configurable authentication
- - Rate limit handling
- - Comprehensive logging
- """
-
- # Update these based on HAR analysis
- BASE_URL = "https://api.example.com"
- DEFAULT_TIMEOUT = 30
- MAX_RETRIES = 3
-
- def __init__(
- self,
- base_url: str = None,
- api_key: str = None,
- access_token: str = None,
- session_cookie: str = None,
- timeout: int = None,
- ):
- """
- Initialize the API client.
-
- Args:
- base_url: Override the default base URL
- api_key: API key for X-API-Key authentication
- access_token: Bearer token for Authorization header
- session_cookie: Session cookie value
- timeout: Request timeout in seconds
- """
- self.base_url = (base_url or self.BASE_URL).rstrip('/')
- self.timeout = timeout or self.DEFAULT_TIMEOUT
-
- # Setup session with retry strategy
- self.session = requests.Session()
-
- retry_strategy = Retry(
- total=self.MAX_RETRIES,
- backoff_factor=1,
- status_forcelist=[500, 502, 503, 504],
- allowed_methods=["GET", "POST", "PUT", "PATCH", "DELETE"],
- )
- adapter = HTTPAdapter(max_retries=retry_strategy)
- self.session.mount("http://", adapter)
- self.session.mount("https://", adapter)
-
- # Set default headers (update based on HAR analysis)
- self.session.headers.update({
- 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36',
- 'Accept': 'application/json',
- 'Accept-Language': 'en-US,en;q=0.9',
- })
-
- # Configure authentication
- if api_key:
- self.session.headers['X-API-Key'] = api_key
- if access_token:
- self.session.headers['Authorization'] = f'Bearer {access_token}'
- if session_cookie:
- self.session.cookies.set('session', session_cookie)
-
- def _build_url(self, endpoint: str) -> str:
- """Build full URL from endpoint."""
- if endpoint.startswith('http'):
- return endpoint
- return urljoin(self.base_url + '/', endpoint.lstrip('/'))
-
- def _handle_response(self, response: requests.Response) -> Dict[str, Any]:
- """
- Handle API response with error checking.
-
- Args:
- response: Response object from requests
-
- Returns:
- Parsed JSON response or empty dict
-
- Raises:
- RateLimitError: When rate limited
- APIError: For other API errors
- """
- # Log response details
- logger.debug(f"Response: {response.status_code} {response.url}")
-
- # Handle rate limiting
- if response.status_code == 429:
- retry_after = response.headers.get('Retry-After')
- if retry_after:
- try:
- retry_after = int(retry_after)
- except ValueError:
- retry_after = 60
- raise RateLimitError(
- "Rate limit exceeded",
- retry_after=retry_after
- )
-
- # Handle errors
- if not response.ok:
- try:
- error_body = response.json()
- except:
- error_body = {'message': response.text}
-
- raise APIError(
- f"API error: {response.status_code}",
- status_code=response.status_code,
- response=error_body
- )
-
- # Parse response
- if response.status_code == 204:
- return {}
-
- try:
- return response.json()
- except:
- return {'text': response.text}
-
- def _request(
- self,
- method: str,
- endpoint: str,
- params: Optional[Dict[str, Any]] = None,
- data: Optional[Dict[str, Any]] = None,
- json_data: Optional[Dict[str, Any]] = None,
- headers: Optional[Dict[str, str]] = None,
- files: Optional[Dict[str, Any]] = None,
- timeout: Optional[int] = None,
- ) -> Dict[str, Any]:
- """
- Make an HTTP request.
-
- Args:
- method: HTTP method (GET, POST, PUT, PATCH, DELETE)
- endpoint: API endpoint path
- params: Query parameters
- data: Form data
- json_data: JSON body
- headers: Additional headers
- files: Files for multipart upload
- timeout: Request timeout override
-
- Returns:
- Parsed response data
- """
- url = self._build_url(endpoint)
- request_timeout = timeout or self.timeout
-
- logger.info(f"{method} {url}")
-
- try:
- response = self.session.request(
- method=method,
- url=url,
- params=params,
- data=data,
- json=json_data,
- headers=headers,
- files=files,
- timeout=request_timeout,
- )
- return self._handle_response(response)
-
- except RateLimitError:
- raise
- except APIError:
- raise
- except requests.Timeout:
- raise APIError(f"Request timeout after {request_timeout}s")
- except requests.ConnectionError as e:
- raise APIError(f"Connection error: {e}")
- except Exception as e:
- logger.exception("Unexpected error")
- raise APIError(f"Request failed: {e}")
-
- # Convenience methods
- def get(self, endpoint: str, **kwargs) -> Dict[str, Any]:
- """Make a GET request."""
- return self._request('GET', endpoint, **kwargs)
-
- def post(self, endpoint: str, **kwargs) -> Dict[str, Any]:
- """Make a POST request."""
- return self._request('POST', endpoint, **kwargs)
-
- def put(self, endpoint: str, **kwargs) -> Dict[str, Any]:
- """Make a PUT request."""
- return self._request('PUT', endpoint, **kwargs)
-
- def patch(self, endpoint: str, **kwargs) -> Dict[str, Any]:
- """Make a PATCH request."""
- return self._request('PATCH', endpoint, **kwargs)
-
- def delete(self, endpoint: str, **kwargs) -> Dict[str, Any]:
- """Make a DELETE request."""
- return self._request('DELETE', endpoint, **kwargs)
-
- # =========================================================================
- # API Endpoints - Add methods here based on HAR analysis
- # =========================================================================
-
- # Example: List resources
- def list_resources(
- self,
- page: int = 1,
- limit: int = 20,
- filters: Optional[Dict[str, Any]] = None,
- ) -> Dict[str, Any]:
- """
- List resources with pagination.
-
- Args:
- page: Page number (1-indexed)
- limit: Items per page
- filters: Optional filter parameters
-
- Returns:
- Dict with 'items' list and pagination info
- """
- params = {
- 'page': page,
- 'limit': limit,
- }
- if filters:
- params.update(filters)
-
- return self.get('/api/resources', params=params)
-
- # Example: Get single resource
- def get_resource(self, resource_id: str) -> Dict[str, Any]:
- """
- Get a resource by ID.
-
- Args:
- resource_id: The resource identifier
-
- Returns:
- Resource data
- """
- return self.get(f'/api/resources/{resource_id}')
-
- # Example: Create resource
- def create_resource(
- self,
- name: str,
- description: Optional[str] = None,
- **extra_fields,
- ) -> Dict[str, Any]:
- """
- Create a new resource.
-
- Args:
- name: Resource name
- description: Optional description
- **extra_fields: Additional fields
-
- Returns:
- Created resource data
- """
- payload = {
- 'name': name,
- }
- if description:
- payload['description'] = description
- payload.update(extra_fields)
-
- return self.post('/api/resources', json_data=payload)
-
- # Example: Update resource
- def update_resource(
- self,
- resource_id: str,
- **fields,
- ) -> Dict[str, Any]:
- """
- Update a resource.
-
- Args:
- resource_id: The resource identifier
- **fields: Fields to update
-
- Returns:
- Updated resource data
- """
- return self.patch(f'/api/resources/{resource_id}', json_data=fields)
-
- # Example: Delete resource
- def delete_resource(self, resource_id: str) -> bool:
- """
- Delete a resource.
-
- Args:
- resource_id: The resource identifier
-
- Returns:
- True if deleted successfully
- """
- self.delete(f'/api/resources/{resource_id}')
- return True
-
- # Example: Paginate through all resources
- def iter_all_resources(
- self,
- limit: int = 100,
- **filters,
- ):
- """
- Iterate through all resources with automatic pagination.
-
- Args:
- limit: Items per page
- **filters: Filter parameters
-
- Yields:
- Resource items one by one
- """
- page = 1
- while True:
- response = self.list_resources(page=page, limit=limit, filters=filters)
-
- items = response.get('items', response.get('data', []))
- if not items:
- break
-
- for item in items:
- yield item
-
- # Check if more pages
- total = response.get('total', 0)
- if page * limit >= total:
- break
-
- page += 1
-
-
-class RateLimitHandler:
- """Helper class for handling rate limits with backoff."""
-
- def __init__(self, client: APIClient):
- self.client = client
- self.last_retry_after = 0
-
- def execute_with_retry(
- self,
- func,
- *args,
- max_retries: int = 3,
- **kwargs,
- ):
- """
- Execute a function with automatic rate limit retry.
-
- Args:
- func: Function to call (should be an API client method)
- *args: Arguments for the function
- max_retries: Maximum retry attempts
- **kwargs: Keyword arguments for the function
-
- Returns:
- Function result
- """
- for attempt in range(max_retries):
- try:
- return func(*args, **kwargs)
- except RateLimitError as e:
- if attempt == max_retries - 1:
- raise
-
- wait_time = e.retry_after or (2 ** attempt * 10)
- self.last_retry_after = wait_time
- logger.warning(f"Rate limited. Waiting {wait_time}s before retry...")
- time.sleep(wait_time)
-
- raise APIError("Max retries exceeded")
-
-
-# ============================================================================
-# Example Usage
-# ============================================================================
-
-if __name__ == "__main__":
- # Initialize client
- client = APIClient(
- base_url="https://api.example.com",
- # api_key="your-api-key",
- # access_token="your-bearer-token",
- )
-
- try:
- # List resources
- print("Fetching resources...")
- response = client.list_resources(page=1, limit=10)
- print(f"Found {len(response.get('items', []))} resources")
-
- # Create a resource
- # new_resource = client.create_resource(
- # name="Test Resource",
- # description="Created via API client"
- # )
- # print(f"Created: {new_resource}")
-
- # Iterate all resources
- # for resource in client.iter_all_resources():
- # print(f"Resource: {resource['id']} - {resource['name']}")
-
- except RateLimitError as e:
- print(f"Rate limited! Retry after {e.retry_after}s")
- except APIError as e:
- print(f"API Error: {e} (status: {e.status_code})")
- except Exception as e:
- print(f"Error: {e}")
\ No newline at end of file