Skip to content

armgit / camel

Public Imported from GitHub
0

๐Ÿซ CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org

12.3 KB
Raw Blame
{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ssX_map8c6mx"
      },
      "source": [
        "# Video Analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9S0pePjZkwUz"
      },
      "source": [
        "You can also check this cookbook in colab [here](https://colab.research.google.com/drive/1egJ-14pRtBbM9lkqhHcZp75mAki1lgzS?usp=sharing)\n",
        "\n",
        "<div class=\"align-center\">\n",
        "  <a href=\"https://www.camel-ai.org/\"><img src=\"https://i.postimg.cc/KzQ5rfBC/button.png\"width=\"150\"></a>\n",
        "  <a href=\"https://discord.camel-ai.org\"><img src=\"https://i.postimg.cc/L4wPdG9N/join-2.png\"  width=\"150\"></a></a>\n",
        "  \n",
        "โญ <i>Star us on [*Github*](https://github.com/camel-ai/camel), join our [*Discord*](https://discord.camel-ai.org) or follow our [*X*](https://x.com/camelaiorg)\n",
        "</div>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dIJRO_u9ZNqe"
      },
      "source": [
        "This notebook demonstrates how to set up and leverage CAMEL's ability to do video analysis.\n",
        "\n",
        "In this notebook, you'll explore:\n",
        "\n",
        "* **CAMEL**: A powerful multi-agent framework that enables Retrieval-Augmented Generation and multi-agent role-playing scenarios, allowing for sophisticated AI-driven tasks.\n",
        "* **Video Analysis**: How to use CAMEL to read and generate descriptions of uploaded videos."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x2rTlUddZNqf"
      },
      "source": [
        "## ๐Ÿ“ฆ Installation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "collapsed": true,
        "id": "AkvGo5umOwxM",
        "outputId": "5bb6d523-3c52-4803-c923-b68a7bf4fe90"
      },
      "outputs": [],
      "source": [
        "%pip install \"camel-ai[all]==0.2.66\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_JiOqhOIZNqg"
      },
      "source": [
        "## ๐Ÿ”‘ Setting Up API Keys"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_-8_5_srcpC_",
        "outputId": "18ae5802-9593-4bd7-9d8d-4a8b76fc96bf"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from getpass import getpass\n",
        "\n",
        "# Prompt for the API key securely\n",
        "openai_api_key = getpass('Enter your API key: ')\n",
        "os.environ[\"OPENAI_API_KEY\"] = openai_api_key"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KqiVn6o2ZNqh"
      },
      "source": [
        "Alternatively, if running on Colab, you could save your API keys and tokens as **Colab Secrets**, and use them across notebooks.\n",
        "\n",
        "To do so, **comment out** the above **manual** API key prompt code block(s), and **uncomment** the following codeblock.\n",
        "\n",
        "โš ๏ธ Don't forget granting access to the API key you would be using to the current notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "QYFuUz8oZNqh"
      },
      "outputs": [],
      "source": [
        "# import os\n",
        "# from google.colab import userdata\n",
        "\n",
        "# os.environ[\"OPENAI_API_KEY\"] = userdata.get(\"OPENAI_API_KEY\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UL43-gPZZNqh"
      },
      "source": [
        "## Set up an agent for video analysis task"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CkTI25jpZVQI",
        "outputId": "8aeb49fe-9d9c-44d9-8fd8-0ee64fdb8b67"
      },
      "outputs": [],
      "source": [
        "from camel.toolkits import VideoDownloaderToolkit\n",
        "#Download video to be analyse\n",
        "urls=[\"https://www.youtube.com/watch?v=n3u5EcjRmdY\",\"https://youtu.be/PNqrHNQlU6I?si=8s2Zvh_F1-bIkeBr\"]\n",
        "video_downloader = VideoDownloaderToolkit(\"./download\")\n",
        "for url in urls:\n",
        "  video_downloader.download_video(url)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "J97N3GtSexwu"
      },
      "outputs": [],
      "source": [
        "from camel.agents import ChatAgent\n",
        "from camel.configs.openai_config import ChatGPTConfig\n",
        "from camel.messages import BaseMessage\n",
        "from camel.prompts.prompt_templates import PromptTemplateGenerator\n",
        "from camel.types import ModelType, ModelPlatformType\n",
        "from camel.types.enums import RoleType, TaskType\n",
        "from camel.models import ModelFactory\n",
        "\n",
        "# Define system message\n",
        "sys_msg_prompt = PromptTemplateGenerator().get_prompt_from_key(\n",
        "    TaskType.VIDEO_DESCRIPTION, RoleType.ASSISTANT\n",
        ")\n",
        "sys_msg = BaseMessage.make_assistant_message(\n",
        "    role_name=\"Assistant\",\n",
        "    content=sys_msg_prompt,\n",
        ")\n",
        "\n",
        "# Set model\n",
        "model=ModelFactory.create(\n",
        "    model_platform=ModelPlatformType.OPENAI,\n",
        "    model_type=ModelType.GPT_4O,\n",
        "    model_config_dict=ChatGPTConfig().as_dict(),\n",
        ")\n",
        "\n",
        "# Set agent\n",
        "camel_agent = ChatAgent(\n",
        "    sys_msg,\n",
        "    model=model\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7nikG1slZNqi"
      },
      "source": [
        "## Providing video and set user message"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cQIJBZh_YXRY",
        "outputId": "c0f69b1a-b5ab-46b7-8af3-3ff3e50fe571"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Step into the future of video content with our latest demo, where we showcase the groundbreaking capabilities of Camel AI. Discover how to seamlessly integrate AI to process and narrate videos using GPT's visual prowess and TTS API. From content moderation to accessibility enhancements, watch as we transform and analyze videos in real-time. Whether you're a creator, developer, or tech enthusiast, this video will inspire you to unlock the full potential of AI in multimedia. Join us on this exciting journey and see how the next generation of video technology is shaping up!\n"
          ]
        }
      ],
      "source": [
        "# Provide your video path\n",
        "video_cctv = \"/content/download/This AI Agent Analyses Videos Brilliantly (5 Easy Examples You Can Try).mp4\"\n",
        "with open(video_cctv, \"rb\") as video_cctv:\n",
        "    video_bytes_cctv = video_cctv.read()\n",
        "\n",
        "# Set user message\n",
        "user_msg_cctv = BaseMessage.make_user_message(\n",
        "    role_name=\"User\",\n",
        "    content=\"These are frames from a video that I want to upload. Generate a\"\n",
        "    \"compelling description that I can upload along with the video.\",\n",
        "    video_bytes=video_bytes_cctv,\n",
        ")\n",
        "\n",
        "# Get response information\n",
        "response_cctv = camel_agent.step(user_msg_cctv)\n",
        "print(response_cctv.msgs[0].content)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "q7NIRGNT3bWx",
        "outputId": "25b52670-b166-4fd6-f251-c3fd95f61f6f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Introducing CRAB, the revolutionary AI-powered solution that connects and controls all your devices seamlessly. Imagine a world where a single prompt can unlock boundless capabilities across your tech ecosystem. From playing music with a voice command to syncing tasks across platforms, CRAB does it all with unparalleled efficiency. Dive into a new era of human-like multitasking, where technology works harmoniously to enhance your digital life. Explore this cutting-edge innovation now and see the future of integrated device management unfold before your eyes!\n"
          ]
        }
      ],
      "source": [
        "# Provide your video path\n",
        "video_help = \"/content/download/Introducing CRAB.mp4\"\n",
        "with open(video_help, \"rb\") as video_help:\n",
        "    video_bytes_help = video_help.read()\n",
        "\n",
        "# Set user message\n",
        "user_msg_help = BaseMessage.make_user_message(\n",
        "    role_name=\"User\",\n",
        "    content=\"These are frames from a video that I want to upload. Generate a\"\n",
        "    \"compelling description that I can upload along with the video.\",\n",
        "    video_bytes=video_bytes_help,\n",
        ")\n",
        "\n",
        "# Get response information\n",
        "response_help = camel_agent.step(user_msg_help)\n",
        "print(response_help.msgs[0].content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "viFQeSvoZNqj"
      },
      "source": [
        "## ๐ŸŒŸ Highlights"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "34kfhNBdZNqj"
      },
      "source": [
        "This notebook has guided you through setting up an agent and analyzing videos using CAMEL.\n",
        "\n",
        "Now, you know how to generate description for uploaded videos."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OZooeak3ZNqj"
      },
      "source": [
        "That's everything: Got questions about ๐Ÿซ CAMEL-AI? Join us on [Discord](https://discord.camel-ai.org)! Whether you want to share feedback, explore the latest in multi-agent systems, get support, or connect with others on exciting projects, weโ€™d love to have you in the community! ๐Ÿค\n",
        "\n",
        "Check out some of our other work:\n",
        "\n",
        "1. ๐Ÿซ Creating Your First CAMEL Agent [free Colab](https://docs.camel-ai.org/cookbooks/create_your_first_agent.html)\n",
        "\n",
        "2.  Graph RAG Cookbook [free Colab](https://colab.research.google.com/drive/1uZKQSuu0qW6ukkuSv9TukLB9bVaS1H0U?usp=sharing)\n",
        "\n",
        "3. ๐Ÿง‘โ€โš–๏ธ Create A Hackathon Judge Committee with Workforce [free Colab](https://colab.research.google.com/drive/18ajYUMfwDx3WyrjHow3EvUMpKQDcrLtr?usp=sharing)\n",
        "\n",
        "4. ๐Ÿ”ฅ 3 ways to ingest data from websites with Firecrawl & CAMEL [free Colab](https://colab.research.google.com/drive/1lOmM3VmgR1hLwDKdeLGFve_75RFW0R9I?usp=sharing)\n",
        "\n",
        "5. ๐Ÿฆฅ Agentic SFT Data Generation with CAMEL and Mistral Models, Fine-Tuned with Unsloth [free Colab](https://colab.research.google.com/drive/1lYgArBw7ARVPSpdwgKLYnp_NEXiNDOd-?usp=sharingg)\n",
        "\n",
        "Thanks from everyone at ๐Ÿซ CAMEL-AI\n",
        "\n",
        "\n",
        "<div class=\"align-center\">\n",
        "  <a href=\"https://www.camel-ai.org/\"><img src=\"https://i.postimg.cc/KzQ5rfBC/button.png\"width=\"150\"></a>\n",
        "  <a href=\"https://discord.camel-ai.org\"><img src=\"https://i.postimg.cc/L4wPdG9N/join-2.png\"  width=\"150\"></a></a>\n",
        "  \n",
        "โญ <i>Star us on [*Github*](https://github.com/camel-ai/camel), join our [*Discord*](https://discord.camel-ai.org) or follow our [*X*](https://x.com/camelaiorg)\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JB4qquEUZNqj"
      },
      "source": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}