🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org
Python agentai-societiesartificial-intelligencecommunicative-aicooperative-aideep-learninglarge-language-modelsmulti-agent-systemsnatural-language-processing
docs: add OceanBase Cookbook documentation (#3766)
pi-dal2fcdb1457621
diff --git a/docs/cookbooks/advanced_features/agents_with_oceanbase_advanced_usage.ipynb b/docs/cookbooks/advanced_features/agents_with_oceanbase_advanced_usage.ipynbnew file mode 100644index 000000000..a24bb00f0--- /dev/null+++ b/docs/cookbooks/advanced_features/agents_with_oceanbase_advanced_usage.ipynb@@ -0,0 +1,310 @@+{+ "cells": [+ {+ "cell_type": "markdown",+ "metadata": {+ "id": "title"+ },+ "source": [+ "# Agents with OceanBase (Advanced Usage)\n"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {+ "id": "meta"+ },+ "source": [+ "Star us on GitHub: https://github.com/camel-ai/camel\n",+ "\n",+ "This cookbook shows how to:\n",+ "- Run OceanBase Community Edition (CE) locally with Docker (4.3.5+).\n",+ "- Use `OceanBaseStorage` as a persistent vector store in CAMEL.\n",+ "- Tune vector index settings and query strategy for different data scales.\n"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {+ "id": "overview"+ },+ "source": [+ "## Overview\n",+ "\n",+ "OceanBase is a MySQL-compatible distributed database that can also serve as a vector database via OceanBase Vector. CAMEL provides `OceanBaseStorage` (backed by `pyobvector`) so you can store embeddings + metadata and run ANN similarity search.\n",+ "\n",+ "This notebook focuses on a repeatable developer setup and the CAMEL-side API.\n"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {+ "id": "installation"+ },+ "source": [+ "## Installation\n",+ "\n",+ "You need the RAG/vector dependencies (this pulls in `pyobvector` on supported Python versions).\n",+ "\n",+ "```python\n",+ "!pip install \"camel-ai[rag]==0.2.85\"\n",+ "```\n",+ "\n",+ "If you already have a local checkout installed from source, you can skip this step.\n"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {+ "id": "docker"+ },+ "source": [+ "## Recipe 1: Install OceanBase CE with Docker (4.3.5+)\n",+ "\n",+ "OceanBase provides an official Docker image: `oceanbase/oceanbase-ce`.\n",+ "\n",+ "- Port: `2881` (MySQL-compatible endpoint) and `2886` (OceanBase dashboard / obshell)\n",+ "- Recommended environment variable: `OB_TENANT_PASSWORD`\n",+ "\n",+ "Official image docs:\n",+ "- https://github.com/oceanbase/docker-images/tree/main/oceanbase-ce\n",+ "- https://hub.docker.com/r/oceanbase/oceanbase-ce\n",+ "\n",+ "### Option A: Quick start (`docker run`)\n",+ "\n",+ "```bash\n",+ "docker pull oceanbase/oceanbase-ce\n",+ "\n",+ "docker run -d --name oceanbase-ce \\\n",+ " -p 2881:2881 \\\n",+ " -p 2886:2886 \\\n",+ " -e MODE=SLIM \\\n",+ " -e OB_TENANT_PASSWORD=TenantPassw0rd! \\\n",+ " oceanbase/oceanbase-ce\n",+ "\n",+ "# Watch logs until the service prints \"boot success!\"\n",+ "docker logs -f oceanbase-ce\n",+ "```\n",+ "\n",+ "### Option B: With persistence (recommended for iterative dev)\n",+ "\n",+ "```bash\n",+ "mkdir -p ob ob-cluster\n",+ "\n",+ "docker run -d --name oceanbase-ce \\\n",+ " -p 2881:2881 \\\n",+ " -p 2886:2886 \\\n",+ " -e MODE=SLIM \\\n",+ " -e OB_TENANT_PASSWORD=TenantPassw0rd! \\\n",+ " -v \"$PWD/ob:/root/ob\" \\\n",+ " -v \"$PWD/ob-cluster:/root/.obd/cluster\" \\\n",+ " oceanbase/oceanbase-ce\n",+ "```\n",+ "\n",+ "### Sanity check (optional)\n",+ "\n",+ "If you have a MySQL client installed, try connecting (usernames can be tenant-qualified, e.g. `root@test`).\n",+ "\n",+ "```bash\n",+ "mysql -h 127.0.0.1 -P 2881 -u root@test -p\n",+ "```\n",+ "\n",+ "Notes:\n",+ "- The exact default tenants/users can vary by image version and your configuration.\n",+ "- For local dev, `MODE=SLIM` starts a lightweight single-node setup quickly.\n",+ "- Wait until `docker logs -f oceanbase-ce` shows `boot success!` before running the CAMEL storage cells below.\n"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {+ "id": "camelside"+ },+ "source": [+ "## Recipe 2: Use OceanBaseStorage in CAMEL\n",+ "\n",+ "`OceanBaseStorage` stores vectors and metadata in an OceanBase table and creates an HNSW vector index if the table does not already exist.\n",+ "\n",+ "Key parameters (see `camel/storages/vectordb_storages/oceanbase.py`):\n",+ "- `vector_dim`: embedding dimension\n",+ "- `table_name`: table name in OceanBase\n",+ "- `uri`: `host:port` (for local Docker: `127.0.0.1:2881`)\n",+ "- `user`: typically tenant-qualified (e.g. `root@test`)\n",+ "- `password`: password for the user\n",+ "- `db_name`: database name\n",+ "- `distance`: `\"l2\"` or `\"cosine\"`\n",+ "\n",+ "IMPORTANT: In the current CAMEL implementation, the HNSW index build parameters are fixed in code:\n",+ "- `m = 16`\n",+ "- `ef_construction = 256`\n",+ "If you need different HNSW parameters, you must modify/extend the implementation.\n"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "metadata": {+ "collapsed": true,+ "id": "imports"+ },+ "outputs": [],+ "source": [+ "import os\n",+ "import random\n",+ "\n",+ "from camel.storages import OceanBaseStorage, VectorDBQuery, VectorRecord\n"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "metadata": {+ "id": "config"+ },+ "outputs": [],+ "source": [+ "# Connection settings for local Docker\n",+ "OB_URI = os.environ.get(\"OB_URI\", \"127.0.0.1:2881\")\n",+ "OB_USER = os.environ.get(\"OB_USER\", \"root@test\")\n",+ "OB_PASSWORD = os.environ.get(\"OB_PASSWORD\", \"TenantPassw0rd!\")\n",+ "OB_DB_NAME = os.environ.get(\"OB_DB_NAME\", \"test\")\n",+ "\n",+ "TABLE_NAME = os.environ.get(\"OB_TABLE_NAME\", \"camel_oceanbase_vectors\")\n",+ "VECTOR_DIM = int(os.environ.get(\"VECTOR_DIM\", \"4\"))\n"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "metadata": {+ "id": "init"+ },+ "outputs": [],+ "source": [+ "storage = OceanBaseStorage(\n",+ " vector_dim=VECTOR_DIM,\n",+ " table_name=TABLE_NAME,\n",+ " uri=OB_URI,\n",+ " user=OB_USER,\n",+ " password=OB_PASSWORD,\n",+ " db_name=OB_DB_NAME,\n",+ " distance=\"cosine\",\n",+ ")\n",+ "\n",+ "print(storage.status())\n"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {+ "id": "basicops"+ },+ "source": [+ "### Basic operations: add/query/delete/clear\n",+ "\n",+ "This quick sanity check does not require any embedding model. We insert random vectors, query with a random vector, then clean up.\n"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "metadata": {+ "id": "add"+ },+ "outputs": [],+ "source": [+ "random.seed(20260131)\n",+ "\n",+ "records = []\n",+ "for i in range(200):\n",+ " records.append(\n",+ " VectorRecord(\n",+ " vector=[random.uniform(-1, 1) for _ in range(VECTOR_DIM)],\n",+ " payload={\"doc_id\": f\"doc-{i}\", \"source\": \"sanity-check\"},\n",+ " )\n",+ " )\n",+ "\n",+ "storage.add(records=records, batch_size=100)\n",+ "print(storage.status())\n"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "metadata": {+ "id": "query"+ },+ "outputs": [],+ "source": [+ "query_vector = [random.uniform(-1, 1) for _ in range(VECTOR_DIM)]\n",+ "results = storage.query(VectorDBQuery(query_vector=query_vector, top_k=5))\n",+ "\n",+ "for r in results:\n",+ " print({\"id\": r.record.id, \"similarity\": r.similarity, \"payload\": r.record.payload})\n"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "metadata": {+ "id": "cleanup"+ },+ "outputs": [],+ "source": [+ "# Clean up the table contents (destructive)\n",+ "storage.clear()\n",+ "print(storage.status())\n"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {+ "id": "tuning"+ },+ "source": [+ "## Recipe 3: Index tuning best practices by scale\n",+ "\n",+ "OceanBase uses an HNSW-style ANN index for vector search. HNSW tuning is always a tradeoff among recall, latency, build time, and memory.\n",+ "\n",+ "Terminology (common across HNSW implementations):\n",+ "- `m`: graph connectivity (higher improves recall but increases memory and build time)\n",+ "- `ef_construction`: build-time search width (higher improves recall but increases build time)\n",+ "- `ef_search` (or similar): query-time search width (higher improves recall but increases query latency)\n",+ "\n",+ "### Practical guidance\n",+ "\n",+ "Small collections (<= 100k vectors):\n",+ "- Favor fast iteration: keep build settings moderate and increase query-time effort when needed.\n",+ "\n",+ "Medium collections (~100k to 5M vectors):\n",+ "- Measure P95/P99 latency and recall. If you cannot reach recall targets without excessive query latency, increase build settings and rebuild.\n",+ "\n",+ "Large collections (5M+ vectors):\n",+ "- Plan for memory and rebuild time. Consider sharding/partitioning by tenant/corpus/time to reduce the search space.\n",+ "\n",+ "### CAMEL-specific note\n",+ "\n",+ "In the current CAMEL `OceanBaseStorage` implementation, the HNSW index is created with fixed values (`m=16`, `ef_construction=256`) when the table is first created. To tune these, you need to update the implementation so the index creation parameters are configurable.\n",+ "\n",+ "### Metadata and relational index tips\n",+ "\n",+ "Vector similarity search is only part of most RAG workloads. For real applications you often filter by tenant, document id, time range, or tags.\n",+ "If you add structured columns (or store frequently-filtered values in dedicated columns instead of only JSON), you can use normal relational indexing to speed up filters before ANN search.\n"+ ]+ }+ ],+ "metadata": {+ "colab": {+ "provenance": []+ },+ "kernelspec": {+ "display_name": "Python 3",+ "name": "python3"+ },+ "language_info": {+ "name": "python"+ }+ },+ "nbformat": 4,+ "nbformat_minor": 0+}diff --git a/docs/cookbooks/advanced_features/index.rst b/docs/cookbooks/advanced_features/index.rstindex 9f930719c..838c95cdd 100644--- a/docs/cookbooks/advanced_features/index.rst+++ b/docs/cookbooks/advanced_features/index.rst@@ -19,6 +19,7 @@ Advanced Featuresagents_with_ragagents_with_graph_ragagents_with_MCP+ agents_with_oceanbase_advanced_usageagents_trackingcritic_agents_and_tree_searchembodied_agentsdiff --git a/docs/mintlify/cookbooks/advanced_features/agents_with_oceanbase_advanced_usage.mdx b/docs/mintlify/cookbooks/advanced_features/agents_with_oceanbase_advanced_usage.mdxnew file mode 100644index 000000000..4cc2fd55b--- /dev/null+++ b/docs/mintlify/cookbooks/advanced_features/agents_with_oceanbase_advanced_usage.mdx@@ -0,0 +1,134 @@+---+title: "Agents with OceanBase (Advanced Usage)"+---++Star us on GitHub: https://github.com/camel-ai/camel++## Overview++OceanBase is a MySQL-compatible distributed database that can also serve as a vector database via OceanBase Vector. CAMEL provides `OceanBaseStorage` (backed by `pyobvector`) so you can store embeddings + metadata and run ANN similarity search.++This cookbook shows how to:++- Run OceanBase Community Edition (CE) locally with Docker (4.3.5+).+- Use `OceanBaseStorage` as a persistent vector store in CAMEL.+- Tune vector index settings and query strategy for different data scales.++## Installation++You need the RAG/vector dependencies (this pulls in `pyobvector` on supported Python versions):++```python+!pip install "camel-ai[rag]==0.2.85"+```++If you already have a local checkout installed from source, you can skip this step.++## Recipe 1: Install OceanBase CE with Docker (4.3.5+)++OceanBase provides an official Docker image: `oceanbase/oceanbase-ce`.++- Port: `2881` (MySQL-compatible endpoint) and `2886` (OceanBase dashboard / obshell)+- Recommended environment variable: `OB_TENANT_PASSWORD`++Official image docs:++- https://github.com/oceanbase/docker-images/tree/main/oceanbase-ce+- https://hub.docker.com/r/oceanbase/oceanbase-ce++### Option A: Quick start (docker run)++```bash+docker pull oceanbase/oceanbase-ce++docker run -d --name oceanbase-ce \+ -p 2881:2881 \+ -p 2886:2886 \+ -e MODE=SLIM \+ -e OB_TENANT_PASSWORD=TenantPassw0rd! \+ oceanbase/oceanbase-ce++# Watch logs until the service prints "boot success!"+docker logs -f oceanbase-ce+```++### Option B: With persistence++```bash+mkdir -p ob ob-cluster++docker run -d --name oceanbase-ce \+ -p 2881:2881 \+ -p 2886:2886 \+ -e MODE=SLIM \+ -e OB_TENANT_PASSWORD=TenantPassw0rd! \+ -v "$PWD/ob:/root/ob" \+ -v "$PWD/ob-cluster:/root/.obd/cluster" \+ oceanbase/oceanbase-ce+```++Optional sanity check (if you have a MySQL client installed):++```bash+mysql -h 127.0.0.1 -P 2881 -u root@test -p+```++Notes:++- Default tenants/users can vary by image version and your configuration.+- For local dev, `MODE=SLIM` starts a lightweight single-node setup quickly.+- Wait until `docker logs -f oceanbase-ce` shows `boot success!` before running the CAMEL storage cells below.++## Recipe 2: Use OceanBaseStorage in CAMEL++`OceanBaseStorage` stores vectors and metadata in an OceanBase table and creates an HNSW vector index if the table does not already exist.++Key parameters (see `camel/storages/vectordb_storages/oceanbase.py`):++- `vector_dim`: embedding dimension+- `table_name`: table name in OceanBase+- `uri`: `host:port` (for local Docker: `127.0.0.1:2881`)+- `user`: typically tenant-qualified (e.g. `root@test`)+- `password`: password for the user+- `db_name`: database name+- `distance`: `"l2"` or `"cosine"`++IMPORTANT: In the current CAMEL implementation, the HNSW index build parameters are fixed in code:++- `m = 16`+- `ef_construction = 256`++If you need different HNSW parameters, you must modify/extend the implementation.++## Recipe 3: Index tuning best practices by scale++OceanBase uses an HNSW-style ANN index for vector search. HNSW tuning is always a tradeoff among recall, latency, build time, and memory.++Terminology (common across HNSW implementations):++- `m`: graph connectivity (higher improves recall but increases memory and build time)+- `ef_construction`: build-time search width (higher improves recall but increases build time)+- `ef_search` (or similar): query-time search width (higher improves recall but increases query latency)++Practical guidance:++Small collections (<= 100k vectors):++- Favor fast iteration: keep build settings moderate and increase query-time effort when needed.++Medium collections (~100k to 5M vectors):++- Measure P95/P99 latency and recall. If you cannot reach recall targets without excessive query latency, increase build settings and rebuild.++Large collections (5M+ vectors):++- Plan for memory and rebuild time. Consider sharding/partitioning by tenant/corpus/time to reduce the search space.++CAMEL-specific note:++- `OceanBaseStorage` creates the index with fixed values (`m=16`, `ef_construction=256`) when the table is first created.++Metadata and relational index tips:++- Vector similarity search is only part of most RAG workloads.+- If you filter by tenant, document id, time range, or tags, consider adding structured columns and using relational indexes to speed up filters.diff --git a/docs/mintlify/docs.json b/docs/mintlify/docs.jsonindex 0d6087d9b..62874c53c 100644--- a/docs/mintlify/docs.json+++ b/docs/mintlify/docs.json@@ -112,6 +112,7 @@"cookbooks/advanced_features/agents_with_human_in_loop_and_tool_approval","cookbooks/advanced_features/agents_with_memory","cookbooks/advanced_features/agents_with_rag",+ "cookbooks/advanced_features/agents_with_oceanbase_advanced_usage","cookbooks/advanced_features/agents_with_tools","cookbooks/advanced_features/agents_with_tools_from_ACI","cookbooks/advanced_features/agents_with_tools_from_Composio",@@ -849,4 +850,4 @@"article:tag": "CAMEL, AI agents, multi-agent systems, LLM, open source, developers, MCP"}}-}\ No newline at end of file+}