🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org
Python agentai-societiesartificial-intelligencecommunicative-aicooperative-aideep-learninglarge-language-modelsmulti-agent-systemsnatural-language-processing
Doc_test (#2539)
Tao Sun045d82040ff2
diff --git a/docs/cookbooks/data_processing/summarisation_agent_with_mistral_ocr.ipynb b/docs/cookbooks/data_processing/summarisation_agent_with_mistral_ocr.ipynbindex c4ff0f4b3..ec943dd4f 100644--- a/docs/cookbooks/data_processing/summarisation_agent_with_mistral_ocr.ipynb+++ b/docs/cookbooks/data_processing/summarisation_agent_with_mistral_ocr.ipynb@@ -2,6 +2,9 @@"cells": [{"cell_type": "markdown",+ "metadata": {+ "id": "zbmL45h7fWRT"+ },"source": ["# Create Document Summarization Agents with Mistral OCR & CAMEL-AI 🐫\n","\n",@@ -27,22 +30,22 @@"⭐ <i>Star us on [*Github*](https://github.com/camel-ai/camel), join our [*Discord*](https://discord.camel-ai.org) or follow our [*X*](https://x.com/camelaiorg)</i>\n","</div>\n","\n"- ],- "metadata": {- "id": "zbmL45h7fWRT"- }+ ]},{"cell_type": "markdown",- "source": [- ""- ],"metadata": {"id": "wbUMBBIB3LM3"- }+ },+ "source": [+ ""+ ]},{"cell_type": "markdown",+ "metadata": {+ "id": "Bl90urdzuQLu"+ },"source": ["## **Introduction to Mistral OCR**\n","\n",@@ -66,10 +69,7 @@" - Available via `mistral-ocr-latest` on Mistral’s developer suite, cloud partners, and on-premises self-hosting for sensitive data.\n","\n","Ready to unlock your documents? Let’s dive into the extraction guide.\n"- ],- "metadata": {- "id": "Bl90urdzuQLu"- }+ ]},{"cell_type": "markdown",@@ -84,8 +84,8 @@"cell_type": "code","execution_count": null,"metadata": {- "id": "I2X5A0LBc92C",- "collapsed": true+ "collapsed": true,+ "id": "I2X5A0LBc92C"},"outputs": [],"source": [@@ -94,12 +94,12 @@},{"cell_type": "markdown",- "source": [- "## ⚡️ Step-by-step Guide: Mistral OCR Loader"- ],"metadata": {"id": "WCfZ6vA0iFQv"- }+ },+ "source": [+ "## ⚡️ Step-by-step Guide: Mistral OCR Loader"+ ]},{"cell_type": "markdown",@@ -135,17 +135,22 @@},{"cell_type": "markdown",+ "metadata": {+ "id": "LUpFBEB9vrIz"+ },"source": ["**Step 2: Upload your PDF or image file for OCR**\n","\n","In a Colab or Jupyter environment, you can upload any PDF file directly:\n"- ],- "metadata": {- "id": "LUpFBEB9vrIz"- }+ ]},{"cell_type": "code",+ "execution_count": null,+ "metadata": {+ "id": "dUVE7z9hwEV7"+ },+ "outputs": [],"source": ["# Colab file upload\n","from google.colab import files\n",@@ -153,24 +158,24 @@"uploaded = files.upload()\n","# Grab the first uploaded filename\n","file_path = next(iter(uploaded))"- ],- "metadata": {- "id": "dUVE7z9hwEV7"- },- "execution_count": null,- "outputs": []+ ]},{"cell_type": "markdown",- "source": [- "**Step 3: Import and initialize the Mistral OCR loader**"- ],"metadata": {"id": "fwlB09epyeWK"- }+ },+ "source": [+ "**Step 3: Import and initialize the Mistral OCR loader**"+ ]},{"cell_type": "code",+ "execution_count": null,+ "metadata": {+ "id": "SgIsQpOX82Bo"+ },+ "outputs": [],"source": ["# Importing the MistralReader class from the camel.loaders module\n","# This class handles document processing using Mistral OCR capabilities\n",@@ -179,39 +184,34 @@"# Initializing an instance of MistralReader\n","# This object will be used to submit tasks and manage OCR processing\n","mistral_reader = MistralReader()"- ],- "metadata": {- "id": "SgIsQpOX82Bo"- },- "execution_count": null,- "outputs": []+ ]},{"cell_type": "markdown",+ "metadata": {+ "id": "ecWmxX4ZyoS8"+ },"source": ["## Step 4: Obtain OCR output from Mistral\n","\n","Once the task completes, retrieve its output using the returned `task.id`.\n","\n","The output of **Mistral OCR** is a structured object:\n"- ],- "metadata": {- "id": "ecWmxX4ZyoS8"- }+ ]},{"cell_type": "code",+ "execution_count": null,+ "metadata": {+ "id": "rTKuFUXuyoS8"+ },+ "outputs": [],"source": ["# Retrieve the OCR output\n","# CORRECT: Just use extract_text for local files or URLs\n","ocr_response = mistral_reader.extract_text(file_path)\n","print(ocr_response)"- ],- "metadata": {- "id": "rTKuFUXuyoS8"- },- "execution_count": null,- "outputs": []+ ]},{"cell_type": "markdown",@@ -226,6 +226,9 @@},{"cell_type": "markdown",+ "metadata": {+ "id": "AtaiHz437A2_"+ },"source": ["If you don't have a Mistral API key, you can obtain one by following these steps:\n","\n",@@ -236,13 +239,15 @@"3. Choose your plan\n","\n","For more details, you can also check the Mistral documentation: https://docs.mistral.ai/getting-started/quickstart/"- ],- "metadata": {- "id": "AtaiHz437A2_"- }+ ]},{"cell_type": "code",+ "execution_count": null,+ "metadata": {+ "id": "1mKu0Dgj5TV9"+ },+ "outputs": [],"source": ["from camel.configs import MistralConfig\n","from camel.models import ModelFactory\n",@@ -256,12 +261,7 @@"\n","# Use Mistral model\n","model = mistral_model"- ],- "metadata": {- "id": "1mKu0Dgj5TV9"- },- "execution_count": null,- "outputs": []+ ]},{"cell_type": "code",@@ -288,26 +288,29 @@},{"cell_type": "markdown",- "source": [- "**For advanced usage of RAG capabilities with large files, please refer to our [RAG cookbook](https://docs.camel-ai.org/cookbooks/advanced_features/agents_with_rag#rag-cookbook).**"- ],"metadata": {"id": "p3VfU-hdX4Ti"- }+ },+ "source": [+ "**For advanced usage of RAG capabilities with large files, please refer to our [RAG cookbook](https://docs.camel-ai.org/cookbooks/advanced_features/agents_with_rag#rag-cookbook).**"+ ]},{"cell_type": "markdown",+ "metadata": {+ "id": "sR4BU5Z_oanP"+ },"source": ["## 🧑🏻💻 Conclusion\n","\n","In conclusion, integrating **Mistral OCR** within CAMEL-AI revolutionizes the process of document data extraction and preparation, enhancing your capabilities for AI-driven applications. With Mistral OCR’s robust features—state-of-the-art complex document understanding, natively multilingual & multimodal parsing, and doc-as-prompt structured Markdown output—you can seamlessly process complex PDFs and images into machine-readable formats optimized for LLMs, directly feeding into CAMEL-AI’s multi-agent workflows. This integration not only simplifies data preparation but also empowers intelligent and accurate analytics at scale. With these tools at your disposal, you’re equipped to transform raw document data into actionable insights, unlocking new possibilities in automation and AI-powered decision-making.\n"- ],- "metadata": {- "id": "sR4BU5Z_oanP"- }+ ]},{"cell_type": "markdown",+ "metadata": {+ "id": "oA81T8-ToaWz"+ },"source": ["That's everything: Got questions about 🐫 CAMEL-AI? Join us on [Discord](https://discord.camel-ai.org)! Whether you want to share feedback, explore the latest in multi-agent systems, get support, or connect with others on exciting projects, we’d love to have you in the community! 🤝\n","\n",@@ -332,19 +335,7 @@" \n","⭐ <i>Star us on <a href=\"https://github.com/camel-ai/camel\">Github</a> </i>, join our [*Discord*](https://discord.camel-ai.org) or follow our [*X*](https://x.com/camelaiorg) ⭐\n","</div>\n"- ],- "metadata": {- "id": "oA81T8-ToaWz"- }- },- {- "cell_type": "code",- "source": [],- "metadata": {- "id": "k5hq3Y4Z6UBC"- },- "execution_count": null,- "outputs": []+ ]}],"metadata": {@@ -361,4 +352,4 @@},"nbformat": 4,"nbformat_minor": 0-}\ No newline at end of file+}diff --git a/docs/mintlify/cookbooks/data_processing/images/summarisation_agent_with_mistral_ocr_1.png b/docs/mintlify/cookbooks/data_processing/images/summarisation_agent_with_mistral_ocr_1.pngnew file mode 100644index 000000000..208382bf3Binary files /dev/null and b/docs/mintlify/cookbooks/data_processing/images/summarisation_agent_with_mistral_ocr_1.png differdiff --git a/docs/mintlify/cookbooks/data_processing/summarisation_agent_with_mistral_ocr.mdx b/docs/mintlify/cookbooks/data_processing/summarisation_agent_with_mistral_ocr.mdxnew file mode 100644index 000000000..ba140412b--- /dev/null+++ b/docs/mintlify/cookbooks/data_processing/summarisation_agent_with_mistral_ocr.mdx@@ -0,0 +1,213 @@+---+title: "Create Document Summarization Agents with Mistral OCR & CAMEL-AI 🐫"+---++You can also check this cookbook in Colab [here](https://colab.research.google.com/drive/1ZwVmqa5vjpZ0C3H7k1XIseFfbCR4mq17?usp=sharing)++In this cookbook, we’ll explore [**Mistral OCR**](https://mistral.ai/news/mistral-ocr)—a state-of-the-art Optical Character Recognition API that understands complex document layouts and extracts text, tables, images, and equations with unprecedented accuracy. We’ll show you how to:++- Use the Mistral OCR API to convert scanned or image-based PDFs into structured Markdown+- Leverage a Mistral LLM agent within CAMEL to summarize and analyze the extracted content+- Build a seamless, end-to-end pipeline for retrieval-augmented generation (RAG), research, or business automation++## Table of Contents++1. 🧑🏻💻 Introduction+2. ⚡️ Step-by-step Guide: Mistral OCR Extraction+3. 💫 Quick Demo with Mistral Agent+4. 🧑🏻💻 Conclusion++<div class="align-center">+ <a href="https://www.camel-ai.org/"><img src="https://i.postimg.cc/KzQ5rfBC/button.png" width="150"></a>+ <a href="https://discord.camel-ai.org"><img src="https://i.postimg.cc/L4wPdG9N/join-2.png" width="150"></a>+++⭐ *Star us on [GitHub](https://github.com/camel-ai/camel), join our [Discord](https://discord.camel-ai.org), or follow us on [X](https://x.com/camelaiorg)*++---++++++## **Introduction to Mistral OCR**++Throughout history, advancements in information abstraction and retrieval have driven human progress—from hieroglyphs to digitization. Today, over 90% of organizational data lives in documents, often locked in complex layouts and multiple languages. **Mistral OCR** ushers in the next leap in document understanding: a multimodal API that comprehends every element—text, images, tables, equations—and outputs ordered, structured Markdown with embedded media references.++#### **Key Features of Mistral OCR:**++1. **State-of-the-art complex document understanding**+ - Extracts interleaved text, figures, tables, and mathematical expressions with high fidelity.++2. **Natively multilingual & multimodal**+ - Parses scripts and fonts from across the globe, handling right-to-left layouts and non-Latin characters seamlessly.++3. **Doc-as-prompt, structured output**+ - Returns ordered Markdown, embedding images and bounding-box metadata ready for RAG and downstream AI workflows.++4. **Top-tier benchmarks & speed**+ - Outperforms leading OCR systems in accuracy—especially in math, tables, and multilingual tests—while delivering fast batch inference (∼2000 pages/min).++5. **Scalable & flexible deployment**+ - Available via `mistral-ocr-latest` on Mistral’s developer suite, cloud partners, and on-premises self-hosting for sensitive data.++Ready to unlock your documents? Let’s dive into the extraction guide.+++First, install the CAMEL package with all its dependencies.+++```python+!pip install "camel-ai[all]==0.2.61"+```++## ⚡️ Step-by-step Guide: Mistral OCR Loader++**Step 1: Set up your Mistral API key**++If you don’t have a Mistral API key, you can obtain one by following these steps:++1. **Create an account:**+ Go to [Mistral Console](https://console.mistral.ai/home) and sign up for an organization account.++2. **Get your API key:**+ Once logged in, navigate to **Organization** → **API Keys**, generate a new key, copy it, and store it securely.++++```python+import os+from getpass import getpass++mistral_api_key = getpass('Enter your Mistral API key: ')+os.environ['MISTRAL_API_KEY'] = mistral_api_key++```++**Step 2: Upload your PDF or image file for OCR**++In a Colab or Jupyter environment, you can upload any PDF file directly:++++```python+# Colab file upload+from google.colab import files++uploaded = files.upload()+# Grab the first uploaded filename+file_path = next(iter(uploaded))+```++**Step 3: Import and initialize the Mistral OCR loader**+++```python+# Importing the MistralReader class from the camel.loaders module+# This class handles document processing using Mistral OCR capabilities+from camel.loaders import MistralReader++# Initializing an instance of MistralReader+# This object will be used to submit tasks and manage OCR processing+mistral_reader = MistralReader()+```++## Step 4: Obtain OCR output from Mistral++Once the task completes, retrieve its output using the returned `task.id`.++The output of **Mistral OCR** is a structured object:++++```python+# Retrieve the OCR output+# CORRECT: Just use extract_text for local files or URLs+ocr_response = mistral_reader.extract_text(file_path)+print(ocr_response)+```++## 💫 Quick Demo with CAMEL Agent++Here we choose Mistral model for our demo. If you'd like to explore different models or tools to suit your needs, feel free to visit the [CAMEL documentation page](https://docs.camel-ai.org/), where you'll find guides and tutorials.+++If you don't have a Mistral API key, you can obtain one by following these steps:++1. Visit the Mistral Console (https://console.mistral.ai/)++2. In the left panel, click on API keys under API section++3. Choose your plan++For more details, you can also check the Mistral documentation: https://docs.mistral.ai/getting-started/quickstart/+++```python+from camel.configs import MistralConfig+from camel.models import ModelFactory+from camel.types import ModelPlatformType, ModelType++mistral_model = ModelFactory.create(+ model_platform=ModelPlatformType.MISTRAL,+ model_type=ModelType.MISTRAL_LARGE,+ model_config_dict=MistralConfig(temperature=0.0).as_dict(),+)++# Use Mistral model+model = mistral_model+```+++```python+from camel.agents import ChatAgent++# Initialize a ChatAgent+agent = ChatAgent(+ system_message="You are a helpful document assistant.", # Define the agent's role+ model=mistral_model+)++# Use the ChatAgent to generate insights based on the OCR output+response = agent.step(+ f"Based on the following OCR-extracted content, give me a concise conclusion of the document:\n{ocr_response}"+)+print(response.msgs[0].content)+```++**For advanced usage of RAG capabilities with large files, please refer to our [RAG cookbook](https://docs.camel-ai.org/cookbooks/advanced_features/agents_with_rag#rag-cookbook).**++## 🧑🏻💻 Conclusion++In conclusion, integrating **Mistral OCR** within CAMEL-AI revolutionizes the process of document data extraction and preparation, enhancing your capabilities for AI-driven applications. With Mistral OCR’s robust features—state-of-the-art complex document understanding, natively multilingual & multimodal parsing, and doc-as-prompt structured Markdown output—you can seamlessly process complex PDFs and images into machine-readable formats optimized for LLMs, directly feeding into CAMEL-AI’s multi-agent workflows. This integration not only simplifies data preparation but also empowers intelligent and accurate analytics at scale. With these tools at your disposal, you’re equipped to transform raw document data into actionable insights, unlocking new possibilities in automation and AI-powered decision-making.+++That's everything: Got questions about 🐫 CAMEL-AI? Join us on [Discord](https://discord.camel-ai.org)! Whether you want to share feedback, explore the latest in multi-agent systems, get support, or connect with others on exciting projects, we’d love to have you in the community! 🤝++Check out some of our other work:++1. 🐫 Creating Your First CAMEL Agent [free Colab](https://colab.research.google.com/drive/1cmWPxXEsyMbmjPhD2bWfHuhd_Uz6FaJQ?usp=sharing)++2. Graph RAG Cookbook [free Colab](https://colab.research.google.com/drive/1uZKQSuu0qW6ukkuSv9TukLB9bVaS1H0U?usp=sharing)++3. 🧑⚖️ Create A Hackathon Judge Committee with Workforce [free Colab](https://colab.research.google.com/drive/18ajYUMfwDx3WyrjHow3EvUMpKQDcrLtr?usp=sharing)++4. 🔥 3 ways to ingest data from websites with Firecrawl & CAMEL [free Colab](https://colab.research.google.com/drive/1lOmM3VmgR1hLwDKdeLGFve_75RFW0R9I?usp=sharing)++5. 🦥 Agentic SFT Data Generation with CAMEL and Mistral Models, Fine-Tuned with Unsloth [free Colab](https://colab.research.google.com/drive/1lYgArBw7ARVPSpdwgKLYnp_NEXiNDOd-?usp=sharingg)++Thanks from everyone at 🐫 CAMEL-AI+++<div style={{ display: "flex", justifyContent: "center", alignItems: "center", gap: "1rem", marginBottom: "2rem" }}>+ <a href="https://www.camel-ai.org/">+ <img src="https://i.postimg.cc/KzQ5rfBC/button.png" width="150" alt="CAMEL Homepage"/>+ </a>+ <a href="https://discord.camel-ai.org">+ <img src="https://i.postimg.cc/L4wPdG9N/join-2.png" width="150" alt="Join Discord"/>+ </a>+</div>++⭐ *Star us on [GitHub](https://github.com/camel-ai/camel), join our [Discord](https://discord.camel-ai.org), or follow us on [X](https://x.com/camelaiorg)*++diff --git a/docs/mintlify/docs.json b/docs/mintlify/docs.jsonindex 1a7b4c324..66957d68d 100644--- a/docs/mintlify/docs.json+++ b/docs/mintlify/docs.json@@ -134,6 +134,7 @@"pages": ["cookbooks/data_processing/agent_with_chunkr_for_pdf_parsing","cookbooks/data_processing/ingest_data_from_websites_with_Firecrawl",+ "cookbooks/data_processing/summarisation_agent_with_mistral_ocr","cookbooks/data_processing/video_analysis"]},diff --git a/docs/mintlify/key_modules/interpreters.mdx b/docs/mintlify/key_modules/interpreters.mdxindex 583c598ad..b2e9654ab 100644--- a/docs/mintlify/key_modules/interpreters.mdx+++ b/docs/mintlify/key_modules/interpreters.mdx@@ -3,23 +3,94 @@ title: "Interpreters"---## 1. Concept-Interpreters are tools that empower the agents to execute given code snippets-on local machine.+Interpreters are tools that empower agents to execute given code snippets on the local machine or in isolated environments. CAMEL supports several types of interpreters to cater to different needs and security considerations.### 1.1. Internal Python Interpreter+The Internal Python Interpreter executes Python code directly within the same process as the agent. This is the simplest interpreter and is suitable for trusted code execution. In its default (safe) mode, it evaluates expressions and executes statements in a controlled environment; `print()` is not directly available as a side effect, so code should evaluate to a string for output++**Example Usage:**+```python+from camel.interpreters import InternalPythonInterpreter++# Initialize the interpreter+interpreter = InternalPythonInterpreter()++# Code to execute (must be an expression that evaluates to a string in safe mode)+python_code = "'Hello from InternalPythonInterpreter!'"++# Execute the code+result_str = interpreter.run(code=python_code, code_type="python")+print(f"Result: {result_str}")+```### 1.2. Subprocess Interpreter+The Subprocess Interpreter runs code in a separate subprocess. This provides a degree of isolation from the main agent process. It can execute various shell commands and scripts in addition to Python code.++**Example Usage:**+```python+from camel.interpreters import SubprocessInterpreter++# Initialize the interpreter+interpreter = SubprocessInterpreter()++# Command to execute (e.g., bash)+shell_command = "echo 'Hello from SubprocessInterpreter!'"++# Execute the command+result_str = interpreter.run(code=shell_command, code_type="bash")+print(f"Result: {result_str.strip()}")+```### 1.3. Docker Interpreter-Yes, we know how disturbing it is to run any code snippets directly on local-system. To avoid this, we have introduced Docker Interpreter. This interpreter-runs the code snippets in a Docker container, ensuring that all the code is-executed in an isolated environment.--**To utilize this interpreter, you need to have Docker installed on your-system.** For more information on Docker installation, visit-[Docker's official website](https://docs.docker.com/get-docker/).--### 1.4. E2B Interpreter-You could leverage the E2B sandbox to run the code.-[E2B official documentation](https://e2b.dev/docs)\ No newline at end of file+The Docker Interpreter executes code snippets within a Docker container. This offers a high level of isolation, ensuring that code runs in a controlled and sandboxed environment. This is particularly useful for running untrusted code or code with specific dependencies.++**To utilize this interpreter, you need to have Docker installed on your system.** For more information on Docker installation, visit [Docker's official website](https://docs.docker.com/get-docker/).++**Example Usage:**+```python+from camel.interpreters import DockerInterpreter++# Initialize the interpreter (assumes a basic python image is available)+interpreter = DockerInterpreter()++# Code to execute in Docker+python_code_in_docker = "print('Hello from DockerInterpreter!')"++# Execute the code+result_str = interpreter.run(code=python_code_in_docker, code_type="python")+print(f"Result: {result_str.strip()}")+```++### 1.4. IPython Interpreter+The IPython Interpreter (JupyterKernelInterpreter) leverages an IPython kernel to execute code. This allows for a rich, interactive execution environment, similar to using a Jupyter notebook. It supports features like state persistence across code executions within a session.++**Example Usage:**+```python+from camel.interpreters import JupyterKernelInterpreter++interpreter = JupyterKernelInterpreter(+ require_confirm=False, print_stdout=True, print_stderr=True+)+++python_code_in_jupyter_kernel = "print('Hello from JupyterKernelInterpreter!')"+result = interpreter.run(code=python_code_in_jupyter_kernel, code_type="python")+print(result)+```++### 1.5. E2B Interpreter+The E2B Interpreter utilizes the E2B cloud-based sandboxed environments for code execution. This provides a secure and managed environment for running code without requiring local setup of complex dependencies or risking local system integrity.++For more information, visit the [E2B official documentation](https://e2b.dev/docs).++**Example Usage:**+```python+from camel.interpreters import E2BInterpreter++interpreter = E2BInterpreter()+++python_code_in_e2b = "print('Hello from E2BInterpreter!')"+result = interpreter.run(code=python_code_in_e2b, code_type="python")+print(result)+```\ No newline at end of file