๐ซ CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org
Python agentai-societiesartificial-intelligencecommunicative-aicooperative-aideep-learninglarge-language-modelsmulti-agent-systemsnatural-language-processing
usecase: youtube video comprehension (#2503)
Bipul70701498fc822a084
diff --git a/examples/usecases/chat_with_youtube/README.md b/examples/usecases/chat_with_youtube/README.mdnew file mode 100644index 000000000..3a4188de4--- /dev/null+++ b/examples/usecases/chat_with_youtube/README.md@@ -0,0 +1,146 @@+# ๐ฅ Video Content Q&A with CAMEL-AI++An intelligent Streamlit-based application that uses the [CAMEL-AI](https://www.camel-ai.org/) framework to allow users to extract, understand, and query the contents of YouTube videos. It leverages CAMEL toolkits and OpenAI models for transcription, summarization, and natural language Q&A.++---++## ๐ Features++- **YouTube Video Downloader**: Fetches video content directly from YouTube using CAMEL's `VideoDownloaderToolkit`.+- **Audio Transcription**: Extracts and transcribes audio with `AudioAnalysisToolkit`.+- **Summarization Agent**: Uses `ChatAgent` with GPT-4o-mini to generate concise summaries of the transcript.+- **Question Answering**: Ask any natural language questions based on the transcript โ answers are grounded in the video's content.++---++## ๐งฐ Toolkits & Architecture++This app utilizes the following CAMEL-AI components:++- `VideoDownloaderToolkit`: For downloading YouTube videos.+- `AudioAnalysisToolkit`: For audio-to-text transcription.+- `ChatAgent`: To summarize transcripts and answer questions using OpenAI models.++---++## ๐ ๏ธ Installation++1. **Clone the Repository**:++ ```bash+ git clone https://github.com/camel-ai/camel.git+ cd examples/usecases/chat_with_youtube+ ```++2. **Set Up Virtual Environment**:++ ```bash+ python -m venv venv+ source venv/bin/activate # On Windows: venv\Scripts\activate+ ```++3. **Install Dependencies**:++ ```bash+ pip install -r requirements.txt+ ```++4. **Configure Environment Variables**:++ Create a `.env` file and add your OpenAI API key:++ ```env+ OPENAI_API_KEY=your_openai_api_key+ ```++5. **Install ffmpeg** (for audio extraction):++ - macOS: `brew install ffmpeg`+ - Ubuntu: `sudo apt install ffmpeg`+ - Windows: [Download from ffmpeg.org](https://ffmpeg.org/download.html)++---+## ๐ช Using Authentication Cookies with yt-dlp++YouTube may require authentication for certain videos. To allow yt-dlp to access such content, you need to provide your own `cookies.txt` file.++### Exporting Cookies from Your Browser++1. **Using a Browser Extension**:++ - **Chrome**: Install [Get cookies.txt LOCALLY](https://chromewebstore.google.com/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpdbgdldlbecc)+ - **Firefox**: Install [cookies.txt](https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/)++ After installing the extension:++ - Navigate to [YouTube](https://www.youtube.com) and log in.+ - Click the extension icon and export the cookies.+ - Save the file as `cookies.txt` in the `examples/usecases/chat_with_youtube` directory.+++### Updating the Application to Use Your Cookies++In your code, specify the path to your `cookies.txt`:++```python+video_toolkit = VideoAnalysisToolkit(cookies_path="cookies.txt")+```++Ensure that this path points to your exported `cookies.txt` file.++---++## โถ๏ธ Running the Application++Start the Streamlit server:++```bash+streamlit run app.py+```++Then, paste a YouTube link, and the app will download the video, transcribe it, summarize it, and let you chat with it.++---++## ๐ Project Structure++```+video-qa-camel/+โโโ app.py # Main Streamlit application+โโโ downloads/ # Temporary video/audio files+โโโ .env # Environment variables (not committed)+โโโ requirements.txt # Python dependencies+โโโ README.md # Project documentation+```++---++## ๐ก Example Usage++1. Launch the app using Streamlit.+2. Paste a YouTube video URL.+3. Wait for the download, audio extraction, and transcription to complete.+4. View the summary.+5. Ask any question based on the video's content and get an AI-generated answer.++---++## ๐ Notes++- This app does not yet support chunked retrieval-based Q&A for long transcripts. For scalable Q&A over long-form content, consider integrating an external vector database (e.g., Milvus, Pinecone) with chunking and embedding.+- All answers are context-aware, based on the entire transcript, but may be truncated for long videos due to model token limits.+++---++## ๐ค Contributing++Feel free to open pull requests, issues, or discussions for improvements, new features, or bug fixes.++---++## ๐ง Powered by++- [CAMEL-AI](https://github.com/camel-ai/camel)+- [OpenAI GPT Models](https://openai.com/)+- [Streamlit](https://streamlit.io/)diff --git a/examples/usecases/chat_with_youtube/app.py b/examples/usecases/chat_with_youtube/app.pynew file mode 100644index 000000000..0332f974d--- /dev/null+++ b/examples/usecases/chat_with_youtube/app.py@@ -0,0 +1,76 @@+# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========++import streamlit as st+import ffmpeg+from camel.toolkits.video_download_toolkit import VideoDownloaderToolkit+from camel.toolkits.audio_analysis_toolkit import AudioAnalysisToolkit+from camel.agents import ChatAgent+import os+from dotenv import load_dotenv++load_dotenv()++# Initialize toolkits+video_downloader = VideoDownloaderToolkit(download_directory="downloads/")+audio_toolkit = AudioAnalysisToolkit(cache_dir="downloads/")+summarizer_agent = ChatAgent(system_message="You are a helpful assistant that summarizes transcripts.", model="gpt-4o-mini")+qa_agent = ChatAgent(system_message="Answer the question based on the context provided.", model="gpt-4o-mini")++st.title("๐ฅ Video Content Q&A with CAMEL")++# Input URL+url = st.text_input("Enter a YouTube video URL:",value="https://www.youtube.com/watch?v=hT_nvWreIhg")++if st.button("Process Video"):+ if url:+ with st.spinner("Downloading and processing video..."):+ # Download video+ video_path = video_downloader.download_video(url)+ st.video(video_path)++ # Extract audio+ audio_path = os.path.splitext(video_path)[0] + ".wav"+ ffmpeg.input(video_path).output(audio_path, ac=1, ar=16000).run(overwrite_output=True)++ # Transcribe+ transcript = audio_toolkit.audio2text(audio_path)++ # Summarize+ summary = summarizer_agent.step(transcript)+ st.subheader("๐ Summary")+ st.write(summary)++ # Store transcript for Q&A+ st.session_state.transcript = transcript+ st.session_state.qa_agent = qa_agent++if "transcript" in st.session_state:+ st.subheader("๐ฌ Ask Questions about the Video")++ if "chat_history" not in st.session_state:+ st.session_state.chat_history = []++ user_q = st.text_input("Your question:")++ if user_q:+ transcript = st.session_state.transcript+ prompt = f"Context: {transcript}\n\nQuestion: {user_q}"+ answer = st.session_state.qa_agent.step(prompt).msgs[0].content++ st.session_state.chat_history.append((user_q, answer))++ for q, a in st.session_state.chat_history:+ st.markdown(f"**User:** {q}")+ st.markdown(f"**AI:** {a}")diff --git a/examples/usecases/chat_with_youtube/requirements.txt b/examples/usecases/chat_with_youtube/requirements.txtnew file mode 100644index 000000000..eac962344--- /dev/null+++ b/examples/usecases/chat_with_youtube/requirements.txt@@ -0,0 +1,3 @@+streamlit==1.45.1+camel-ai[all]==0.2.61+docx2markdown==0.1.1\ No newline at end of file