Convert any text to a graph of knowledge. This can be used for Graph Augmented Generation or Knowledge Graph based QnA
HTML
minor readme updates
Nayak136d336a609e
diff --git a/README.md b/README.mdindex 729a01b..5830ca4 100644--- a/README.md+++ b/README.md@@ -70,6 +70,12 @@ I am using the [Mistral 7B Openorca](https://huggingface.co/Open-Orca/Mistral-7BOllama makes it easy to host any model locally. Mistral 7B OpenOrca version is already available with Ollama to use out of the box.+To set up this project, you must install Ollama on your local machine.++Step 1: Install Ollama https://ollama.ai++Step 2: run `ollama run zephyr` in your terminal. This will pull the zephyr model to your local machine and start the Ollama server.+### Pandasdataframes for graph schema (can use a graphdb at a later stage).diff --git a/old_notebooks/zefyr.ipynb b/old_notebooks/zefyr.ipynbindex c697bec..d8392e9 100644--- a/old_notebooks/zefyr.ipynb+++ b/old_notebooks/zefyr.ipynb@@ -2,7 +2,7 @@"cells": [{"cell_type": "code",- "execution_count": 4,+ "execution_count": 2,"metadata": {},"outputs": [{@@ -13,7 +13,7 @@" 'template': '{{- if .System }}\\n<|system|>\\n{{ .System }}\\n</s>\\n{{- end }}\\n<|user|>\\n{{ .Prompt }}\\n</s>\\n<|assistant|>\\n'}"]},- "execution_count": 4,+ "execution_count": 2,"metadata": {},"output_type": "execute_result"}@@ -33,7 +33,7 @@},{"cell_type": "code",- "execution_count": 5,+ "execution_count": 3,"metadata": {},"outputs": [],"source": [@@ -45,17 +45,17 @@" # print( chalk.blue(model_info))\n","\n"," SYS_PROMPT = (\n",- " \"You are a network graph maker who extracts terms and their relations from a given context\"\n",+ " \"You are a network graph maker who extracts terms and their relations from a given context. \"\n"," \"You are provided with a context chunk (delimited by ```) Your task is to extract the ontology \"\n"," \"of terms mentioned in the given context. These terms should represent the key concepts as per the context. \\n\"\n",- " \"Thought 1: While travelling through every sentence, Think about the key terms mentioned in it.\\n\"\n",- " \"\\tTerms may include object, entity, location, organization, person, \\n\"\n",- " \"\\tcondition, acronym, documents, service, concept, etc.\\n\"\n",- " \"\\tTerms should be as atomistic as possible\\n\\n\"\n",- " \"Thought 3: Think about how these terms can have one on one relation with other terms.\\n\"\n",- " \"\\tTerms that are mentioned in same sentence or the same paragraph are typically related to each other.\\n\"\n",- " \"\\tTerms can be related to many other terms\\n\\n\"\n",- " \"Thought 4: Find out the relation between each such related pair of terms. \\n\\n\"\n",+ " \"Thought 1: While traversing through each sentence, Think about the key terms mentioned in it.\\n\"\n",+ " \"\\tTerms may include object, entity, location, organization, person, \\n\"\n",+ " \"\\tcondition, acronym, documents, service, concept, etc.\\n\"\n",+ " \"\\tTerms should be as atomistic as possible\\n\\n\"\n",+ " \"Thought 2: Think about how these terms can have one on one relation with other terms.\\n\"\n",+ " \"\\tTerms that are mentioned in the same sentence or the same paragraph are typically related to each other.\\n\"\n",+ " \"\\tTerms can be related to many other terms\\n\\n\"\n",+ " \"Thought 3: Find out the relation between each such related pair of terms. \\n\\n\"\n"," \"Format your output as a list of json. Each element of the list contains a pair of terms\"\n"," \"and the relation between them, like the follwing: \\n\"\n"," \"[\\n\"\n",@@ -82,7 +82,7 @@},{"cell_type": "code",- "execution_count": 6,+ "execution_count": 5,"metadata": {},"outputs": [{@@ -90,47 +90,73 @@"output_type": "stream","text": ["[\n",- " {\n",- " \"node_1\": \"Mary\",\n",- " \"node_2\": \"lamb\",\n",- " \"edge\": \"owned by\"\n",- " },\n",- " {\n",- " \"node_1\": \"Mary\",\n",- " \"node_2\": \"plate\",\n",- " \"edge\": \"passed to consume more food\"\n",- " },\n",- " {\n",- " \"node_1\": \"plate\",\n",- " \"node_2\": \"food\",\n",- " \"edge\": \"contained\"\n",- " }\n",+ " {\n",+ " \"node_1\": \"Mary\",\n",+ " \"node_2\": \"person\",\n",+ " \"edge\": \"Mary is a person.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"little lamb\",\n",+ " \"node_2\": \"animal\",\n",+ " \"edge\": \"The text mentions a little lamb, which is an animal.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"had\",\n",+ " \"node_2\": \"action\",\n",+ " \"edge\": \"Mary had something, which is an action.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"passed\",\n",+ " \"node_2\": \"action\",\n",+ " \"edge\": \"Mary passed something, which is an action.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"plate\",\n",+ " \"node_2\": \"object\",\n",+ " \"edge\": \"The text mentions a plate, which is an object.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"more\",\n",+ " \"node_2\": \"quantity\",\n",+ " \"edge\": \"Mary had more of something, which is a quantity.\"\n",+ " }\n","]\n",- "``` \n",- "\n",- "In this example, we identified the key concepts (nodes) and their relationships (edges). We represented them as a list of JSON objects. The first node is \"Mary\", who owned the second node (\"lamb\"). The third node (\"plate\") was passed to Mary, containing the fourth node (\"food\").\n",+ "```\n","\n","ERROR ### Here is the buggy response: [\n",- " {\n",- " \"node_1\": \"Mary\",\n",- " \"node_2\": \"lamb\",\n",- " \"edge\": \"owned by\"\n",- " },\n",- " {\n",- " \"node_1\": \"Mary\",\n",- " \"node_2\": \"plate\",\n",- " \"edge\": \"passed to consume more food\"\n",- " },\n",- " {\n",- " \"node_1\": \"plate\",\n",- " \"node_2\": \"food\",\n",- " \"edge\": \"contained\"\n",- " }\n",+ " {\n",+ " \"node_1\": \"Mary\",\n",+ " \"node_2\": \"person\",\n",+ " \"edge\": \"Mary is a person.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"little lamb\",\n",+ " \"node_2\": \"animal\",\n",+ " \"edge\": \"The text mentions a little lamb, which is an animal.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"had\",\n",+ " \"node_2\": \"action\",\n",+ " \"edge\": \"Mary had something, which is an action.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"passed\",\n",+ " \"node_2\": \"action\",\n",+ " \"edge\": \"Mary passed something, which is an action.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"plate\",\n",+ " \"node_2\": \"object\",\n",+ " \"edge\": \"The text mentions a plate, which is an object.\"\n",+ " },\n",+ " {\n",+ " \"node_1\": \"more\",\n",+ " \"node_2\": \"quantity\",\n",+ " \"edge\": \"Mary had more of something, which is a quantity.\"\n",+ " }\n","]\n","``` \n","\n",- "In this example, we identified the key concepts (nodes) and their relationships (edges). We represented them as a list of JSON objects. The first node is \"Mary\", who owned the second node (\"lamb\"). The third node (\"plate\") was passed to Mary, containing the fourth node (\"food\"). \n",- "\n","\n"]}@@ -173,7 +199,7 @@"name": "python","nbconvert_exporter": "python","pygments_lexer": "ipython3",- "version": "3.11.1"+ "version": "3.11.4"}},"nbformat": 4,