Skip to content

100+ AI Agents, Agent Skills and RAG Apps - Free and Open Source.

feat: YAML Based Multi-Agent Web Research Agent with Firecrawl MCP (low-code)

Shubhamsaboob4e2fdd489f8
diff --git a/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/README.md b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/README.md
new file mode 100644
index 00000000..3d7283ce
--- /dev/null
+++ b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/README.md
@@ -0,0 +1,198 @@
+# Multi-Agent Web Research System
+
+A sophisticated multi-agent system built with Google ADK that uses Firecrawl MCP tools for web scraping and coordinates between specialized research and summary agents.
+
+> **Location**: This example is part of the Google ADK Crash Course examples, located in `ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/`
+
+## Architecture
+
+This system consists of:
+
+1. **Main Coordinator Agent** (`root_agent.yaml`) - Orchestrates the entire workflow
+2. **Research Agent** (`research_agent.yaml`) - Uses Firecrawl MCP tools for web scraping and content analysis
+3. **Summary Agent** (`summary_agent.yaml`) - Creates comprehensive reports and summaries
+4. **Firecrawl MCP Integration** - Advanced web scraping with proper sub-agent configuration
+
+## Features
+
+- πŸ” **Advanced Web Scraping**: Uses Firecrawl MCP tools for reliable content extraction
+- πŸ”¬ **Intelligent Content Analysis**: Research agent extracts insights, patterns, and key data
+- πŸ“ **Comprehensive Report Generation**: Summary agent creates structured reports and recommendations
+- πŸ€– **Multi-Agent Coordination**: Main agent orchestrates the entire workflow seamlessly
+- πŸ” **Secure API Management**: Firecrawl API key managed via environment variables
+- ⚑ **Sub-Agent MCP Support**: Properly configured MCP tools in sub-agents
+
+## Setup
+
+### Prerequisites
+
+1. Install Google ADK:
+ ```bash
+ pip install google-adk
+ ```
+
+2. Get Firecrawl API key from [firecrawl.dev](https://firecrawl.dev)
+
+3. Set environment variables in `.env` file:
+
+ **Option A: Google AI Studio (Recommended for development)**
+ ```bash
+ GOOGLE_GENAI_USE_VERTEXAI=0
+ GOOGLE_API_KEY=<your-google-gemini-api-key>
+ FIRECRAWL_API_KEY=<your-firecrawl-api-key>
+ ```
+
+ **Option B: Vertex AI (Recommended for production)**
+ ```bash
+ GOOGLE_GENAI_USE_VERTEXAI=1
+ GOOGLE_CLOUD_PROJECT=<your-gcp-project-id>
+ GOOGLE_CLOUD_LOCATION=us-central1
+ FIRECRAWL_API_KEY=<your-firecrawl-api-key>
+ ```
+
+ **Getting API Keys:**
+ - **Google AI Studio**: Get your API key from [Google AI Studio](https://aistudio.google.com/app/apikey)
+ - **Vertex AI**: Set up authentication using [Google Cloud Authentication](https://cloud.google.com/vertex-ai/generative-ai/docs/start/api-keys)
+ - **Firecrawl**: Get your API key from [Firecrawl](https://firecrawl.dev/app/api-keys)
+
+### Installation
+
+1. Navigate to the agent directory:
+ ```bash
+ cd ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher
+ ```
+
+2. Verify ADK installation:
+ ```bash
+ adk --version
+ ```
+
+## Usage
+
+### Running the Agent
+
+Choose one of these methods to run your agent:
+
+1. **Web Interface** (Recommended for testing):
+ ```bash
+ adk web
+ ```
+
+2. **Command Line**:
+ ```bash
+ adk run
+ ```
+
+3. **API Server** (For integration):
+ ```bash
+ adk api_server
+ ```
+
+## Agent Configuration
+
+### Main Agent (`root_agent.yaml`)
+
+The coordinator agent that:
+- Delegates tasks to specialized sub-agents
+- Coordinates between research and summary agents
+- Synthesizes final comprehensive reports
+- Provides clear instructions to sub-agents
+
+### Research Agent (`research_agent.yaml`)
+
+Specialized for web scraping and content analysis:
+- **Firecrawl MCP Tools**: Uses `firecrawl_scrape` and `firecrawl_search`
+- **Content Analysis**: Extracts key findings and insights
+- **Pattern Recognition**: Identifies trends and relationships
+- **Data Extraction**: Highlights important quotes and data points
+- **Research Suggestions**: Suggests areas for further investigation
+
+**Available Firecrawl Tools:**
+- `firecrawl_scrape`: Scrape content from single URLs
+- `firecrawl_search`: Search the web for relevant content
+- `firecrawl_batch_scrape`: Scrape multiple URLs efficiently
+- `firecrawl_map`: Discover URLs on websites
+- `firecrawl_crawl`: Comprehensive website crawling
+
+### Summary Agent (`summary_agent.yaml`)
+
+Specialized for report generation:
+- Creates executive summaries
+- Organizes information by topic
+- Generates key takeaways
+- Provides actionable recommendations
+
+## Workflow
+
+1. **Input**: User provides URL or research topic
+2. **Delegation**: Main agent passes clear instructions to research_agent
+3. **Web Scraping**: Research agent uses Firecrawl MCP tools to extract content
+4. **Analysis**: Research agent analyzes scraped content for insights
+5. **Summarization**: Summary agent creates comprehensive report
+6. **Synthesis**: Main agent combines findings into final report
+
+## Environment Variables
+
+### Google AI Studio Configuration
+| Variable | Description | Required |
+|----------|-------------|----------|
+| `GOOGLE_GENAI_USE_VERTEXAI` | Set to 0 for Google AI Studio | Yes |
+| `GOOGLE_API_KEY` | Google Gemini API key from AI Studio | Yes |
+| `FIRECRAWL_API_KEY` | Firecrawl API key for web scraping | Yes |
+
+### Vertex AI Configuration
+| Variable | Description | Required |
+|----------|-------------|----------|
+| `GOOGLE_GENAI_USE_VERTEXAI` | Set to 1 for Vertex AI | Yes |
+| `GOOGLE_CLOUD_PROJECT` | Your Google Cloud Project ID | Yes |
+| `GOOGLE_CLOUD_LOCATION` | GCP region (e.g., us-central1) | Yes |
+| `FIRECRAWL_API_KEY` | Firecrawl API key for web scraping | Yes |
+
+### Authentication Methods
+
+**Google AI Studio:**
+- Simple API key authentication
+- Best for development and testing
+- No Google Cloud setup required
+
+**Vertex AI:**
+- Enterprise-grade authentication
+- Best for production deployments
+- Requires Google Cloud Project setup
+- Supports advanced features like grounding and safety settings
+
+
+## Example Usage
+
+### Web Interface
+1. Run `adk web`
+2. Open browser to the provided URL
+3. Enter a URL or research topic (e.g., "Scrape and analyze https://example.com" or "Research AI trends")
+4. Watch the multi-agent system process your request
+
+### Command Line
+```bash
+adk run
+# Enter your research query when prompted
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **API Key Errors**: Ensure all required API keys are set in `.env`
+2. **ADK Not Found**: Make sure ADK is installed and Python environment is activated
+3. **Firecrawl Errors**: Verify your Firecrawl API key is valid and has sufficient credits
+4. **MCP Connection Issues**: Check that Node.js and npm are properly installed
+5. **Authentication Issues**:
+ - **Google AI Studio**: Verify your API key is valid and has proper permissions
+ - **Vertex AI**: Ensure Google Cloud authentication is set up correctly (`gcloud auth application-default login`)
+ - **Project ID**: Verify your Google Cloud Project ID is correct for Vertex AI
+
+
+## References
+
+- [Google ADK Documentation](https://google.github.io/adk-docs/)
+- [Agent Config Reference](https://google.github.io/adk-docs/agents/config/#build-an-agent)
+- [Firecrawl Documentation](https://docs.firecrawl.dev/)
+- [MCP Tools](https://modelcontextprotocol.io/)
diff --git a/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/.env.example b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/.env.example
new file mode 100644
index 00000000..2db1a762
--- /dev/null
+++ b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/.env.example
@@ -0,0 +1,12 @@
+### Uncomment either of the two sections based on your prefrence.
+
+## Using Google AI Studio
+#GOOGLE_GENAI_USE_VERTEXAI=0
+#GOOGLE_API_KEY=<your-google-gemini-api-key>
+#FIRECRAWL_API_KEY=<your-firecrawl-api-key>
+
+## Using Vertex AI
+#GOOGLE_GENAI_USE_VERTEXAI=1
+#GOOGLE_CLOUD_PROJECT=<your-gcp-project-id>
+#GOOGLE_CLOUD_LOCATION=us-central1
+#FIRECRAWL_API_KEY=<your-firecrawl-api-key>
diff --git a/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/__init__.py b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/__init__.py
new file mode 100644
index 00000000..8b137891
--- /dev/null
+++ b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/__init__.py
@@ -0,0 +1 @@
+
diff --git a/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/research_agent.yaml b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/research_agent.yaml
new file mode 100644
index 00000000..b8c22651
--- /dev/null
+++ b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/research_agent.yaml
@@ -0,0 +1,33 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/google/adk-python/refs/heads/main/src/google/adk/agents/config_schemas/AgentConfig.json
+name: research_agent
+model: gemini-2.5-flash
+description: 'Specialized agent for analyzing web content and extracting insights, patterns, and key information.'
+instruction: |
+ You are a research analysis agent with access to Firecrawl web scraping tools. Your job is to:
+ 1. Use Firecrawl tools to scrape and search web content
+ 2. Analyze scraped content for key insights and patterns
+ 3. Identify important facts, trends, and relationships
+ 4. Extract relevant quotes and data points
+ 5. Provide structured analysis of the content
+ 6. Highlight any inconsistencies or gaps in information
+ Firecrawl Tool Usage:
+ - For URLs: Use `firecrawl_scrape` with parameter: {"url": "https://example.com"}
+ - For search queries: Use `firecrawl_search` with parameter: {"query": "search term"}
+ - Always use simple object parameters, not arrays or complex structures
+
+ Always provide your analysis in a structured format with clear sections for:
+ - Key Findings
+ - Important Data Points
+ - Trends and Patterns
+ - Notable Quotes
+ - Areas for Further Investigation
+tools:
+ - name: MCPToolset
+ args:
+ stdio_server_params:
+ command: "npx"
+ args:
+ - "-y"
+ - "firecrawl-mcp"
+ env:
+ FIRECRAWL_API_KEY: "${FIRECRAWL_API_KEY}"
\ No newline at end of file
diff --git a/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/root_agent.yaml b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/root_agent.yaml
new file mode 100644
index 00000000..3277aa1d
--- /dev/null
+++ b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/root_agent.yaml
@@ -0,0 +1,23 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/google/adk-python/refs/heads/main/src/google/adk/agents/config_schemas/AgentConfig.json
+name: web_research_coordinator
+model: gemini-2.5-flash
+description: 'A coordinator agent that manages web research using Firecrawl for scraping and two specialized sub-agents for research and summarization.'
+instruction: |
+ You are a web research coordinator agent. Your job is to:
+ 1. Coordinate web research tasks using two sub-agents:
+ - research_agent: Handles web search and scraping using the Firecrawl MCP tool, and analyzes content for insights and patterns
+ - summary_agent: Creates comprehensive summaries and reports
+ 2. Synthesize findings from both agents into actionable insights
+ Important: When delegating to research_agent, provide clear, specific instructions:
+ For URLs: "Please scrape and analyze the content from [URL]"
+ For research topics: "Please search for and analyze information about [TOPIC]"
+ Do NOT pass complex objects or arrays to the research_agent. Use simple, clear text instructions.
+ When given a URL or research topic:
+ - Pass a clear, simple instruction to the research_agent (e.g., "Scrape and analyze https://example.com" or "Research AI trends")
+ - The research_agent will use appropriate Firecrawl tools with correct parameters
+ - The research_agent will analyze the content and return key findings
+ - Delegate summarization of the research_agent's analysis to the summary_agent
+ - Combine outputs from both agents into a final comprehensive report
+sub_agents:
+ - config_path: research_agent.yaml
+ - config_path: summary_agent.yaml
\ No newline at end of file
diff --git a/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/summary_agent.yaml b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/summary_agent.yaml
new file mode 100644
index 00000000..067e7fdc
--- /dev/null
+++ b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/multi_agent_web_researcher/summary_agent.yaml
@@ -0,0 +1,22 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/google/adk-python/refs/heads/main/src/google/adk/agents/config_schemas/AgentConfig.json
+name: summary_agent
+model: gemini-2.5-flash
+description: 'Specialized agent for creating comprehensive summaries and reports from research findings.'
+instruction: |
+ You are a summarization agent. Your job is to:
+ 1. Create clear, concise summaries of research findings
+ 2. Organize information into logical sections
+ 3. Generate executive summaries for quick understanding
+ 4. Create detailed reports with proper formatting
+ 5. Ensure all important information is captured and presented clearly
+ Always structure your output with:
+ - Executive Summary (2-3 sentences)
+ - Detailed Summary (organized by topic)
+ - Key Takeaways (bullet points)
+ - Recommendations (if applicable)
+ When creating summaries, ensure:
+ - Information is accurate and well-organized
+ - Key points are highlighted and easy to find
+ - Complex information is simplified without losing meaning
+ - Recommendations are actionable and specific
+ - The summary is comprehensive yet concise
diff --git a/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/requirements.txt b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/requirements.txt
new file mode 100644
index 00000000..74aa2660
--- /dev/null
+++ b/ai_agent_framework_crash_course/google_adk_crash_course/adk_yaml_examples/multi_agent_web_research_team/requirements.txt
@@ -0,0 +1,2 @@
+google-adk
+firecrawl-py
\ No newline at end of file