Skip to content

OpenRAG is a comprehensive, single package Retrieval-Augmented Generation platform built on Langflow, Docling, and Opensearch.

Python

remove not relevant docs

Edwin Jose4fb52d35e5ca
diff --git a/sdks/mcp/src/openrag_mcp/mcp.md b/sdks/mcp/src/openrag_mcp/mcp.md
deleted file mode 100644
index 15f888f9a..000000000
--- a/sdks/mcp/src/openrag_mcp/mcp.md
+++ /dev/null
@@ -1,205 +0,0 @@
-# OpenRAG MCP
-
-OpenRAG MCP is a [Model Context Protocol](https://modelcontextprotocol.io/) server that exposes your OpenRAG knowledge base to AI assistants. It enables seamless integration between OpenRAG's RAG capabilities and MCP-compatible applications like Cursor, Claude Desktop, and IBM Watson Orchestrate.
-
----
-
-## Quick Start
-
-Run the MCP server using `uvx` (no installation required):
-
-```bash
-uvx openrag-mcp
-```
-
-For a specific version:
-
-```bash
-uvx --from openrag-mcp==0.1.1 openrag-mcp
-```
-
-### Prerequisites
-
-- Python 3.10+
-- A running OpenRAG instance
-- An OpenRAG API key (create one in **Settings > API Keys**)
-
----
-
-## Environment Variables
-
-| Variable | Description | Required | Default |
-|:---------|:------------|:--------:|:--------|
-| `OPENRAG_API_KEY` | Your OpenRAG API key | Yes | - |
-| `OPENRAG_URL` | URL of your OpenRAG instance | No | `http://localhost:3000` |
-
----
-
-## Features
-
-| Feature | Tool | Status | Description |
-|:--------|:-----|:------:|:------------|
-| **Chat** | `openrag_chat` | GA | RAG-enhanced conversations with your knowledge base |
-| **Search** | `openrag_search` | WIP | Semantic search over documents with relevance scoring |
-| **File Ingestion** | `openrag_ingest_file` | WIP | Ingest local files (PDF, DOCX, TXT, MD, HTML) |
-| **URL Ingestion** | `openrag_ingest_url` | WIP | Ingest content from web URLs |
-| **Delete Document** | `openrag_delete_document` | WIP | Remove documents from knowledge base |
-| **Task Status** | `openrag_get_task_status` | WIP | Check async ingestion task progress |
-| **Wait for Task** | `openrag_wait_for_task` | WIP | Poll until ingestion completes |
-
----
-
-## Use Cases
-
-### Why OpenRAG MCP?
-
-OpenRAG MCP acts as a **universal connectivity layer** between your knowledge base and AI-powered applications. Instead of building custom integrations for each platform, MCP provides a standardized protocol that works everywhere.
-
-**Key Benefits:**
-
-- **Zero Custom Integration** - No need to build platform-specific APIs or plugins. One MCP server works across all compatible applications.
-
-- **Agentic AI Enablement** - Empowers AI agents to autonomously search, retrieve, and reason over your organizational knowledge without human intervention.
-
-- **Future-Proof Architecture** - MCP is an open standard backed by Anthropic and adopted by major AI platforms. Your integration investment carries forward as the ecosystem grows.
-
-- **Centralized Knowledge Access** - Single point of entry to your knowledge base for all AI applications, ensuring consistent answers and reducing knowledge silos.
-
-- **Secure by Design** - API key authentication ensures only authorized applications access your knowledge. No data leaves your infrastructure without explicit permission.
-
-- **Multi-Agent Workflows** - Enable complex workflows where multiple AI agents collaborate using shared knowledge, such as a coding agent consulting documentation while a review agent validates compliance.
-
-- **Reduced Hallucination** - RAG-enhanced responses are grounded in your actual documents, reducing AI hallucinations with verifiable, sourced answers.
-
-- **Enterprise Knowledge Democratization** - Make institutional knowledge accessible to every team member through their preferred AI tool, from developers in Cursor to analysts in Claude.
-
-- **Real-Time Knowledge Sync** - As you ingest new documents into OpenRAG, all connected AI applications immediately gain access to the updated knowledge.
-
-- **Minimal Infrastructure Overhead** - Runs as a lightweight subprocess spawned by the AI application. No separate servers, containers, or complex deployment required.
-
-### Agentic Application Integration
-
-| Platform | Use Case |
-|:---------|:---------|
-| **Cursor IDE** | Context-aware code assistance powered by your documentation, design specs, and internal wikis |
-| **Claude Desktop** | Enhanced conversations that leverage your organization's knowledge base for accurate, sourced answers |
-| **IBM Watson Orchestrate** | Workflow automation with RAG capabilities for enterprise knowledge retrieval |
-| **Custom MCP Clients** | Build your own integrations using the open MCP protocol |
-
-### Example Scenarios
-
-- **Developer Productivity**: Query internal API docs, architecture decisions, and runbooks directly from your IDE
-- **Customer Support**: AI assistants that reference your product documentation and knowledge base
-- **Research & Analysis**: Search and synthesize information across multiple document sources
-- **Workflow Automation**: Trigger document ingestion and knowledge queries as part of automated pipelines
-
----
-
-## Platform Configuration
-
-### Cursor IDE
-
-Add to `~/.cursor/mcp.json`:
-
-```json
-{
- "mcpServers": {
- "openrag": {
- "command": "uvx",
- "args": ["openrag-mcp"],
- "env": {
- "OPENRAG_URL": "https://your-openrag-instance.com",
- "OPENRAG_API_KEY": "orag_your_api_key_here"
- }
- }
- }
-}
-```
-
-### Claude Desktop
-
-**macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`
-**Windows**: `%APPDATA%\Claude\claude_desktop_config.json`
-
-```json
-{
- "mcpServers": {
- "openrag": {
- "command": "uvx",
- "args": ["openrag-mcp"],
- "env": {
- "OPENRAG_URL": "https://your-openrag-instance.com",
- "OPENRAG_API_KEY": "orag_your_api_key_here"
- }
- }
- }
-}
-```
-
----
-
-## Architecture
-
-```mermaid
-flowchart LR
- subgraph clients [AI Applications]
- Cursor[Cursor IDE]
- Claude[Claude Desktop]
- Watson[IBM Watson Orchestrate]
- Other[Other MCP Clients]
- end
-
- subgraph mcp [MCP Server]
- Server[openrag-mcp]
- end
-
- subgraph backend [Backend]
- OpenRAG[OpenRAG Instance]
- KB[(Knowledge Base)]
- end
-
- Cursor <-->|stdio / JSON-RPC| Server
- Claude <-->|stdio / JSON-RPC| Server
- Watson <-->|stdio / JSON-RPC| Server
- Other <-->|stdio / JSON-RPC| Server
-
- Server <-->|HTTPS / X-API-Key| OpenRAG
- OpenRAG <--> KB
-```
-
----
-
-## Example Prompts
-
-Once configured, interact with OpenRAG through natural language:
-
-- *"Search my knowledge base for authentication best practices"*
-- *"What documents do I have about deployment procedures?"*
-- *"Chat with OpenRAG about the Q4 roadmap"*
-- *"Ingest this PDF: /path/to/architecture-overview.pdf"*
-- *"Delete the outdated spec document from my knowledge base"*
-
----
-
-## Troubleshooting
-
-| Issue | Solution |
-|:------|:---------|
-| "OPENRAG_API_KEY environment variable is required" | Ensure `OPENRAG_API_KEY` is set in the `env` section of your MCP config |
-| "Connection refused" errors | Verify OpenRAG is running and `OPENRAG_URL` is correct |
-| Tools not appearing | Restart your AI application after config changes |
-
----
-
-## Resources
-
-- [OpenRAG Documentation](https://docs.openr.ag)
-- [MCP SDK README](sdks/mcp/README.md)
-- [Model Context Protocol Specification](https://modelcontextprotocol.io/)
-
----
-
-## License
-
-Apache 2.0 - See [LICENSE](LICENSE) for details.
-
diff --git a/sdks/mcp/src/openrag_mcp/sequence.md b/sdks/mcp/src/openrag_mcp/sequence.md
deleted file mode 100644
index c75a67a35..000000000
--- a/sdks/mcp/src/openrag_mcp/sequence.md
+++ /dev/null
@@ -1,501 +0,0 @@
-# OpenRAG MCP Sequence Diagrams
-
-This document illustrates the interaction flows between AI applications and OpenRAG through the Model Context Protocol (MCP) server.
-
-## Participants
-
-| Participant | Description |
-|:------------|:------------|
-| **AI App** | MCP-compatible application (Cursor IDE, Claude Desktop, Watson Orchestrate) |
-| **MCP Server** | OpenRAG MCP server (`openrag-mcp`) running as subprocess |
-| **OpenRAG SDK** | Python SDK client within MCP server |
-| **OpenRAG API** | OpenRAG FastAPI backend server |
-| **Knowledge Base** | OpenSearch vector database storing document embeddings |
-
----
-
-## 1. Server Initialization
-
-```mermaid
-sequenceDiagram
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
-
- Note over AIApp,MCP: Startup Phase
-
- AIApp->>MCP: Spawn subprocess (uvx openrag-mcp)
- activate MCP
- MCP->>MCP: Load OPENRAG_API_KEY from env
- MCP->>MCP: Load OPENRAG_URL from env
- MCP->>SDK: Initialize client with credentials
- activate SDK
- SDK-->>MCP: Client ready
- deactivate SDK
- MCP->>MCP: Register tools (chat, search, ingest, delete)
- MCP-->>AIApp: Server initialized (stdio ready)
-
- Note over AIApp,MCP: Tool Discovery
-
- AIApp->>MCP: JSON-RPC: tools/list
- MCP-->>AIApp: Available tools array
-```
-
----
-
-## 2. Chat Flow (openrag_chat)
-
-RAG-enhanced conversation using the knowledge base.
-
-```mermaid
-sequenceDiagram
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
- participant KB as Knowledge Base
-
- AIApp->>MCP: JSON-RPC: tools/call<br/>name: openrag_chat<br/>args: {message, chat_id?, filter_id?, limit?, score_threshold?}
- activate MCP
-
- MCP->>MCP: Validate message not empty
- MCP->>SDK: client.chat.create(...)
- activate SDK
-
- SDK->>API: POST /api/chat<br/>X-API-Key: orag_xxx
- activate API
-
- Note over API,KB: RAG Pipeline
-
- API->>KB: Semantic search for relevant chunks
- activate KB
- KB-->>API: Matching document chunks with scores
- deactivate KB
-
- API->>API: Build context from chunks
- API->>API: Generate LLM response with context
- API-->>SDK: {response, sources[], chat_id}
- deactivate API
-
- SDK-->>MCP: ChatResponse object
- deactivate SDK
-
- MCP->>MCP: Format response with sources
- MCP-->>AIApp: TextContent with answer + sources + chat_id
- deactivate MCP
-
- Note over AIApp,MCP: Conversation Continuation
-
- AIApp->>MCP: JSON-RPC: tools/call<br/>name: openrag_chat<br/>args: {message, chat_id: "prev_id"}
- MCP->>SDK: client.chat.create(chat_id=prev_id)
- SDK->>API: POST /api/chat (with chat_id)
- API->>API: Load conversation history
- API->>KB: Search with conversation context
- KB-->>API: Relevant chunks
- API-->>SDK: Contextual response
- SDK-->>MCP: ChatResponse
- MCP-->>AIApp: Continued conversation response
-```
-
----
-
-## 3. Search Flow (openrag_search)
-
-Semantic search over the knowledge base.
-
-```mermaid
-sequenceDiagram
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
- participant KB as Knowledge Base
-
- AIApp->>MCP: JSON-RPC: tools/call<br/>name: openrag_search<br/>args: {query, limit?, score_threshold?, filter_id?, data_sources?, document_types?}
- activate MCP
-
- MCP->>MCP: Validate query not empty
- MCP->>MCP: Build SearchFilters if data_sources or document_types provided
- MCP->>SDK: client.search.query(...)
- activate SDK
-
- SDK->>API: POST /api/search<br/>X-API-Key: orag_xxx
- activate API
-
- API->>API: Generate query embedding
- API->>KB: k-NN vector search with filters
- activate KB
- KB-->>API: Scored document chunks
- deactivate KB
-
- API->>API: Apply score_threshold filter
- API-->>SDK: {results[{filename, text, score, page}]}
- deactivate API
-
- SDK-->>MCP: SearchResponse object
- deactivate SDK
-
- alt Results found
- MCP->>MCP: Format results with scores<br/>Truncate content > 500 chars
- MCP-->>AIApp: TextContent with formatted results
- else No results
- MCP-->>AIApp: "No results found."
- end
- deactivate MCP
-```
-
----
-
-## 4. File Ingestion Flow (openrag_ingest_file)
-
-Ingest local files into the knowledge base with sync or async modes.
-
-```mermaid
-sequenceDiagram
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
- participant KB as Knowledge Base
-
- Note over AIApp,KB: Synchronous Ingestion (wait=true, default)
-
- AIApp->>MCP: JSON-RPC: tools/call<br/>name: openrag_ingest_file<br/>args: {file_path, wait: true}
- activate MCP
-
- MCP->>MCP: Validate file exists and is a file
- MCP->>SDK: client.documents.ingest(file_path, wait=true)
- activate SDK
-
- SDK->>API: POST /api/documents/upload<br/>multipart/form-data
- activate API
-
- API->>API: Parse document (Docling)
- API->>API: Chunk document
- API->>API: Generate embeddings
- API->>KB: Index chunks with vectors
- activate KB
- KB-->>API: Indexing complete
- deactivate KB
-
- API-->>SDK: {status: "completed", successful_files, failed_files}
- deactivate API
-
- SDK-->>MCP: IngestResponse
- deactivate SDK
-
- MCP-->>AIApp: "Successfully ingested 'filename'"
- deactivate MCP
-
- Note over AIApp,KB: Asynchronous Ingestion (wait=false)
-
- AIApp->>MCP: JSON-RPC: tools/call<br/>name: openrag_ingest_file<br/>args: {file_path, wait: false}
- activate MCP
-
- MCP->>MCP: Validate file exists
- MCP->>SDK: client.documents.ingest(file_path, wait=false)
- activate SDK
-
- SDK->>API: POST /api/documents/upload
- activate API
- API-->>SDK: {task_id: "abc123", filename}
- deactivate API
-
- SDK-->>MCP: IngestResponse with task_id
- deactivate SDK
-
- MCP-->>AIApp: "Queued for ingestion. Task ID: abc123"
- deactivate MCP
-
- Note over API,KB: Background Processing
- API--)KB: Async: Parse, chunk, embed, index
-```
-
----
-
-## 5. URL Ingestion Flow (openrag_ingest_url)
-
-Ingest content from web URLs.
-
-```mermaid
-sequenceDiagram
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
- participant KB as Knowledge Base
-
- AIApp->>MCP: JSON-RPC: tools/call<br/>name: openrag_ingest_url<br/>args: {url}
- activate MCP
-
- MCP->>MCP: Validate URL starts with http:// or https://
- MCP->>SDK: client.chat.create(message with URL)
- activate SDK
-
- SDK->>API: POST /api/chat<br/>X-API-Key: orag_xxx
- activate API
-
- Note over API: Agent processes URL ingestion request
-
- API->>API: Fetch URL content
- API->>API: Parse HTML/content
- API->>API: Chunk and embed
- API->>KB: Index content
- activate KB
- KB-->>API: Indexed
- deactivate KB
-
- API-->>SDK: {response: "URL ingested successfully"}
- deactivate API
-
- SDK-->>MCP: ChatResponse
- deactivate SDK
-
- MCP-->>AIApp: "URL ingestion requested. [response]"
- deactivate MCP
-```
-
----
-
-## 6. Task Status Flow (openrag_get_task_status)
-
-Check the status of an async ingestion task.
-
-```mermaid
-sequenceDiagram
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
-
- AIApp->>MCP: JSON-RPC: tools/call<br/>name: openrag_get_task_status<br/>args: {task_id}
- activate MCP
-
- MCP->>MCP: Validate task_id not empty
- MCP->>SDK: client.documents.get_task_status(task_id)
- activate SDK
-
- SDK->>API: GET /api/documents/tasks/{task_id}<br/>X-API-Key: orag_xxx
- activate API
-
- API-->>SDK: {status, task_id, total_files, processed_files, successful_files, failed_files, files}
- deactivate API
-
- SDK-->>MCP: TaskStatus object
- deactivate SDK
-
- MCP->>MCP: Format status report
- MCP-->>AIApp: TextContent with task status details
- deactivate MCP
-```
-
----
-
-## 7. Wait for Task Flow (openrag_wait_for_task)
-
-Poll until an ingestion task completes.
-
-```mermaid
-sequenceDiagram
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
-
- AIApp->>MCP: JSON-RPC: tools/call<br/>name: openrag_wait_for_task<br/>args: {task_id, timeout?}
- activate MCP
-
- MCP->>MCP: Validate task_id not empty
- MCP->>SDK: client.documents.wait_for_task(task_id, timeout)
- activate SDK
-
- loop Poll until complete or timeout
- SDK->>API: GET /api/documents/tasks/{task_id}
- activate API
- API-->>SDK: {status: "processing", ...}
- deactivate API
- SDK->>SDK: Sleep interval
- end
-
- SDK->>API: GET /api/documents/tasks/{task_id}
- activate API
- API-->>SDK: {status: "completed", ...}
- deactivate API
-
- SDK-->>MCP: TaskStatus (final)
- deactivate SDK
-
- alt Task completed
- MCP-->>AIApp: "Task Completed: [status details]"
- else Timeout
- MCP-->>AIApp: "Task did not complete within X seconds"
- end
- deactivate MCP
-```
-
----
-
-## 8. Delete Document Flow (openrag_delete_document)
-
-Remove a document from the knowledge base.
-
-```mermaid
-sequenceDiagram
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
- participant KB as Knowledge Base
-
- AIApp->>MCP: JSON-RPC: tools/call<br/>name: openrag_delete_document<br/>args: {filename}
- activate MCP
-
- MCP->>MCP: Validate filename not empty
- MCP->>SDK: client.documents.delete(filename)
- activate SDK
-
- SDK->>API: DELETE /api/documents/{filename}<br/>X-API-Key: orag_xxx
- activate API
-
- API->>KB: Delete all chunks for filename
- activate KB
- KB-->>API: Deleted chunk count
- deactivate KB
-
- API-->>SDK: {success: true, deleted_chunks: N}
- deactivate API
-
- SDK-->>MCP: DeleteResponse
- deactivate SDK
-
- alt Success
- MCP-->>AIApp: "Successfully deleted 'filename' (N chunks removed)"
- else Not found
- MCP-->>AIApp: "Document not found: [message]"
- end
- deactivate MCP
-```
-
----
-
-## 9. Error Handling
-
-All tools implement consistent error handling.
-
-```mermaid
-sequenceDiagram
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
-
- AIApp->>MCP: JSON-RPC: tools/call
- activate MCP
- MCP->>SDK: API call
- activate SDK
- SDK->>API: HTTPS request
- activate API
-
- alt AuthenticationError (401)
- API-->>SDK: 401 Unauthorized
- SDK-->>MCP: AuthenticationError
- MCP-->>AIApp: "Authentication error: [message]"
- else ValidationError (400)
- API-->>SDK: 400 Bad Request
- SDK-->>MCP: ValidationError
- MCP-->>AIApp: "Invalid request: [message]"
- else RateLimitError (429)
- API-->>SDK: 429 Too Many Requests
- SDK-->>MCP: RateLimitError
- MCP-->>AIApp: "Rate limited: [message]"
- else ServerError (5xx)
- API-->>SDK: 500 Internal Server Error
- SDK-->>MCP: ServerError
- MCP-->>AIApp: "Server error: [message]"
- else NotFoundError (404)
- API-->>SDK: 404 Not Found
- SDK-->>MCP: NotFoundError
- MCP-->>AIApp: "Not found: [message]"
- else Success
- API-->>SDK: 200 OK with data
- SDK-->>MCP: Response object
- MCP-->>AIApp: Formatted success response
- end
-
- deactivate API
- deactivate SDK
- deactivate MCP
-```
-
----
-
-## Complete Architecture Overview
-
-```mermaid
-sequenceDiagram
- participant User as User/AI Agent
- participant AIApp as AI Application
- participant MCP as MCP Server
- participant SDK as OpenRAG SDK
- participant API as OpenRAG API
- participant KB as Knowledge Base
-
- Note over User,KB: End-to-End Knowledge Workflow
-
- User->>AIApp: "Ingest this PDF"
- AIApp->>MCP: openrag_ingest_file
- MCP->>SDK: documents.ingest()
- SDK->>API: POST /api/documents/upload
- API->>KB: Index embeddings
- KB-->>API: Done
- API-->>SDK: Success
- SDK-->>MCP: IngestResponse
- MCP-->>AIApp: "Ingested successfully"
- AIApp-->>User: Document added
-
- User->>AIApp: "What does it say about X?"
- AIApp->>MCP: openrag_chat
- MCP->>SDK: chat.create()
- SDK->>API: POST /api/chat
- API->>KB: Semantic search
- KB-->>API: Relevant chunks
- API->>API: LLM generates response
- API-->>SDK: Response + sources
- SDK-->>MCP: ChatResponse
- MCP-->>AIApp: Answer with citations
- AIApp-->>User: RAG-enhanced answer
-
- User->>AIApp: "Find all references to Y"
- AIApp->>MCP: openrag_search
- MCP->>SDK: search.query()
- SDK->>API: POST /api/search
- API->>KB: Vector search
- KB-->>API: Scored results
- API-->>SDK: SearchResponse
- SDK-->>MCP: Results
- MCP-->>AIApp: Formatted results
- AIApp-->>User: Search results with scores
-```
-
----
-
-## Protocol Details
-
-| Layer | Protocol | Format |
-|:------|:---------|:-------|
-| AI App ↔ MCP Server | stdio | JSON-RPC 2.0 |
-| MCP Server ↔ OpenRAG API | HTTPS | REST + JSON |
-| Authentication | Header | `X-API-Key: orag_xxx` |
-
-## Tool Summary
-
-| Tool | Purpose | Key Parameters |
-|:-----|:--------|:---------------|
-| `openrag_chat` | RAG conversation | message, chat_id, filter_id, limit, score_threshold |
-| `openrag_search` | Semantic search | query, limit, score_threshold, filter_id, data_sources, document_types |
-| `openrag_ingest_file` | Ingest local file | file_path, wait |
-| `openrag_ingest_url` | Ingest from URL | url |
-| `openrag_get_task_status` | Check task status | task_id |
-| `openrag_wait_for_task` | Wait for completion | task_id, timeout |
-| `openrag_delete_document` | Remove document | filename |