The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.
Python agentic-memoryagentic-memory-systemagentsaiai-agentschatgptgenaillmlong-term-memorymemorymemory-managementpythonragstate-management
feat: changed doc_file to docx and update readme (#157)
Sahil Kumar Yadav0bb3d0afe99f
diff --git a/README.md b/README.mdindex d9d7c7da3..6317ebf90 100644--- a/README.md+++ b/README.md@@ -1,15 +1,17 @@# embedchain-[](https://discord.gg/6PzXDgEjG5)[](https://pypi.org/project/embedchain/)+[](https://discord.gg/6PzXDgEjG5)+[](https://twitter.com/embedchain)+[](https://embedchain.substack.com/)embedchain is a framework to easily create LLM powered bots over any dataset. If you want a javascript version, check out [embedchain-js](https://github.com/embedchain/embedchainjs)# Latest Updates-* Introduce a new interface called `chat`. It remembers the history (last 5 messages) and can be used to powerful stateful bots. You can use it by calling `.chat` on any app instance. Works for both OpenAI and OpenSourceApp.+- Introduce a new interface called `chat`. It remembers the history (last 5 messages) and can be used to powerful stateful bots. You can use it by calling `.chat` on any app instance. Works for both OpenAI and OpenSourceApp.-* Introduce a new app type called `OpenSourceApp`. It uses `gpt4all` as the LLM and `sentence transformers` all-MiniLM-L6-v2 as the embedding model. If you use this app, you dont have to pay for anything.+- Introduce a new app type called `OpenSourceApp`. It uses `gpt4all` as the LLM and `sentence transformers` all-MiniLM-L6-v2 as the embedding model. If you use this app, you dont have to pay for anything.# What is embedchain?@@ -130,9 +132,9 @@ from embedchain import OpenSourceApp as ECOSApp### Query Interface-* This interface is like a question answering bot. It takes a question and gets the answer. It does not maintain context about the previous chats.+- This interface is like a question answering bot. It takes a question and gets the answer. It does not maintain context about the previous chats.-* To use this, call `.query` function to get the answer for any query.+- To use this, call `.query` function to get the answer for any query.```pythonprint(naval_chat_bot.query("What unique capacity does Naval argue humans possess when it comes to understanding explanations or concepts?"))@@ -141,9 +143,9 @@ print(naval_chat_bot.query("What unique capacity does Naval argue humans possess### Chat Interface-* This interface is chat interface where it remembers previous conversation. Right now it remembers 5 conversation by default.+- This interface is chat interface where it remembers previous conversation. Right now it remembers 5 conversation by default.-* To use this, call `.chat` function to get the answer for any query.+- To use this, call `.chat` function to get the answer for any query.```pythonprint(naval_chat_bot.chat("How to be happy in life?"))@@ -188,10 +190,10 @@ app.add('web_page', 'a_valid_web_page_url')### Doc File-To add any doc/docx file, use the data_type as `doc_file`. Eg:+To add any doc/docx file, use the data_type as `docx`. Eg:```python-app.add('doc_file', 'a_local_doc_file_path')+app.add('docx', 'a_local_docx_file_path')```### Textdiff --git a/embedchain/chunkers/doc_file.py b/embedchain/chunkers/docx_file.pysimilarity index 85%rename from embedchain/chunkers/doc_file.pyrename to embedchain/chunkers/docx_file.pyindex 49ee64034..03db7bea8 100644--- a/embedchain/chunkers/doc_file.py+++ b/embedchain/chunkers/docx_file.py@@ -4,13 +4,13 @@ from langchain.text_splitter import RecursiveCharacterTextSplitterTEXT_SPLITTER_CHUNK_PARAMS = {- "chunk_size": 500,+ "chunk_size": 1000,"chunk_overlap": 0,"length_function": len,}-class DocFileChunker(BaseChunker):+class DocxFileChunker(BaseChunker):def __init__(self):text_splitter = RecursiveCharacterTextSplitter(**TEXT_SPLITTER_CHUNK_PARAMS)super().__init__(text_splitter)diff --git a/embedchain/embedchain.py b/embedchain/embedchain.pyindex 66cc1e563..ea8892ac0 100644--- a/embedchain/embedchain.py+++ b/embedchain/embedchain.py@@ -13,13 +13,13 @@ from embedchain.loaders.pdf_file import PdfFileLoaderfrom embedchain.loaders.web_page import WebPageLoaderfrom embedchain.loaders.local_qna_pair import LocalQnaPairLoaderfrom embedchain.loaders.local_text import LocalTextLoader-from embedchain.loaders.doc_file import DocFileLoader+from embedchain.loaders.docx_file import DocxFileLoaderfrom embedchain.chunkers.youtube_video import YoutubeVideoChunkerfrom embedchain.chunkers.pdf_file import PdfFileChunkerfrom embedchain.chunkers.web_page import WebPageChunkerfrom embedchain.chunkers.qna_pair import QnaPairChunkerfrom embedchain.chunkers.text import TextChunker-from embedchain.chunkers.doc_file import DocFileChunker+from embedchain.chunkers.docx_file import DocxFileChunkerfrom embedchain.vectordb.chroma_db import ChromaDB@@ -61,6 +61,7 @@ class EmbedChain:'web_page': WebPageLoader(),'qna_pair': LocalQnaPairLoader(),'text': LocalTextLoader(),+ 'docx': DocxFileLoader(),}if data_type in loaders:return loaders[data_type]@@ -81,6 +82,7 @@ class EmbedChain:'web_page': WebPageChunker(),'qna_pair': QnaPairChunker(),'text': TextChunker(),+ 'docx': DocxFileChunker(),}if data_type in chunkers:return chunkers[data_type]diff --git a/embedchain/loaders/doc_file.py b/embedchain/loaders/docx_file.pysimilarity index 65%rename from embedchain/loaders/doc_file.pyrename to embedchain/loaders/docx_file.pyindex e7351010b..d9872e231 100644--- a/embedchain/loaders/doc_file.py+++ b/embedchain/loaders/docx_file.py@@ -1,9 +1,8 @@-from langchain.document_loaders import UnstructuredWordDocumentLoader+from langchain.document_loaders import Docx2txtLoader--class DocFileLoader:+class DocxFileLoader:def load_data(self, url):- loader = UnstructuredWordDocumentLoader(url)+ loader = Docx2txtLoader(url)output = []data = loader.load()content = data[0].page_contentdiff --git a/setup.py b/setup.pyindex c6aa8e32a..a200bbf5c 100644--- a/setup.py+++ b/setup.py@@ -31,6 +31,6 @@ setuptools.setup("pytube","gpt4all","sentence_transformers",- "unstructured",+ "docx2txt",],)