Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

docs: faq

qin-ctx33d5d3a5e118
diff --git a/README.md b/README.md
index 620ea05fb..23cda0b8e 100644
--- a/README.md
+++ b/README.md
@@ -266,7 +266,7 @@ After running the first example, let's dive into the design philosophy of OpenVi
We no longer view context as flat text slices but unify them into an abstract virtual filesystem. Whether it's memories, resources, or capabilities, they are mapped to virtual directories under the `viking://` protocol, each with a unique URI.
-This paradigm gives Agents unprecedented context manipulation capabilities, enabling them to locate, browse, and manipulate information precisely and deterministically through standard commands like `ls` and `find`, just like a developer. This transforms context management from vague semantic matching into intuitive, traceable "file operations".
+This paradigm gives Agents unprecedented context manipulation capabilities, enabling them to locate, browse, and manipulate information precisely and deterministically through standard commands like `ls` and `find`, just like a developer. This transforms context management from vague semantic matching into intuitive, traceable "file operations". Learn more: [Viking URI](./docs/en/concepts/03-viking-uri.md) | [Context Types](./docs/en/concepts/02-context-types.md)
```
viking://
@@ -299,6 +299,8 @@ Stuffing massive amounts of context into a prompt all at once is not only expens
- **L1 (Overview)**: Contains core information and usage scenarios for Agent decision-making during the planning phase.
- **L2 (Details)**: The full original data, for deep reading by the Agent when absolutely necessary.
+Learn more: [Context Layers](./docs/en/concepts/04-context-layers.md)
+
```
viking://resources/my_project/
├── .abstract # L0 Layer: Abstract (~100 tokens) - Quick relevance check
@@ -326,13 +328,13 @@ Single vector retrieval struggles with complex query intents. OpenViking has des
4. **Recursive Drill-down**: If subdirectories exist, recursively repeat the secondary retrieval steps layer by layer.
5. **Result Aggregation**: Finally, obtain the most relevant context to return.
-This "lock high-score directory first, then refine content exploration" strategy not only finds the semantically best-matching fragments but also understands the full context where the information resides, thereby improving the globality and accuracy of retrieval.
+This "lock high-score directory first, then refine content exploration" strategy not only finds the semantically best-matching fragments but also understands the full context where the information resides, thereby improving the globality and accuracy of retrieval. Learn more: [Retrieval Mechanism](./docs/en/concepts/06-retrieval.md)
### 4. Visualized Retrieval Trajectory → Observable Context
OpenViking's organization uses a hierarchical virtual filesystem structure. All context is integrated in a unified format, and each entry corresponds to a unique URI (like a `viking://` path), breaking the traditional flat black-box management mode with a clear hierarchy that is easy to understand.
-The retrieval process adopts a directory recursive strategy. The trajectory of directory browsing and file positioning for each retrieval is fully preserved, allowing users to clearly observe the root cause of problems and guide the optimization of retrieval logic.
+The retrieval process adopts a directory recursive strategy. The trajectory of directory browsing and file positioning for each retrieval is fully preserved, allowing users to clearly observe the root cause of problems and guide the optimization of retrieval logic. Learn more: [Retrieval Mechanism](./docs/en/concepts/06-retrieval.md)
### 5. Automatic Session Management → Context Self-Iteration
@@ -341,7 +343,7 @@ OpenViking has a built-in memory self-iteration loop. At the end of each session
- **User Memory Update**: Update memories related to user preferences, making Agent responses better fit user needs.
- **Agent Experience Accumulation**: Extract core content such as operational tips and tool usage experience from task execution experience, aiding efficient decision-making in subsequent tasks.
-This allows the Agent to get "smarter with use" through interactions with the world, achieving self-evolution.
+This allows the Agent to get "smarter with use" through interactions with the world, achieving self-evolution. Learn more: [Session Management](./docs/en/concepts/08-session.md)
---
diff --git a/README_CN.md b/README_CN.md
index cde252589..40e6b0ea2 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -265,7 +265,7 @@ Search results:
我们不再将上下文视为扁平的文本切片,而是将其统一抽象并组织于一个虚拟文件系统中。无论是记忆、资源还是能力,都会被映射到 `viking://` 协议下的虚拟目录,拥有唯一的 URI。
-这种范式赋予了 Agent 前所未有的上下文操控能力,使其能像开发者一样,通过 `ls`、`find` 等标准指令来精确、确定性地定位、浏览和操作信息,让上下文的管理从模糊的语义匹配演变为直观、可追溯的"文件操作"。
+这种范式赋予了 Agent 前所未有的上下文操控能力,使其能像开发者一样,通过 `ls`、`find` 等标准指令来精确、确定性地定位、浏览和操作信息,让上下文的管理从模糊的语义匹配演变为直观、可追溯的"文件操作"。了解更多:[Viking URI](./docs/zh/concepts/03-viking-uri.md) | [上下文类型](./docs/zh/concepts/02-context-types.md)
```
viking://
@@ -298,6 +298,8 @@ viking://
- **L1 (概述)**:包含核心信息和使用场景,供 Agent 在规划阶段进行决策
- **L2 (详情)**:完整的原始数据,供 Agent 在确有必要时深入读取
+了解更多:[上下文分层](./docs/zh/concepts/04-context-layers.md)
+
```
viking://resources/my_project/
├── .abstract # L0 层:摘要(~100 tokens)- 快速判断相关性
@@ -325,13 +327,13 @@ viking://resources/my_project/
4. **递归下探**:若目录下仍存在子目录,则逐层递归重复上述二次检索步骤
5. **结果汇总**:最终拿到最相关上下文返回
-这种"先锁定高分目录、再精细探索内容"的策略,不仅能找到语义最匹配的片段,更能理解信息所在的完整语境,从而提升检索的全局性与准确性。
+这种"先锁定高分目录、再精细探索内容"的策略,不仅能找到语义最匹配的片段,更能理解信息所在的完整语境,从而提升检索的全局性与准确性。了解更多:[检索机制](./docs/zh/concepts/06-retrieval.md)
### 4. 可视化检索轨迹 → 上下文可观测
OpenViking 的组织方式采用层次化虚拟文件系统结构,所有上下文均以统一格式整合且每个条目对应唯一 URI(如 `viking://` 路径),打破传统扁平黑箱式管理模式,层次分明易于理解。
-检索过程采用目录递归策略,每次检索的目录浏览、文件定位轨迹均被完整留存,能够清晰观测问题根源并指导检索逻辑优化。
+检索过程采用目录递归策略,每次检索的目录浏览、文件定位轨迹均被完整留存,能够清晰观测问题根源并指导检索逻辑优化。了解更多:[检索机制](./docs/zh/concepts/06-retrieval.md)
### 5. 会话自动管理 → 上下文自迭代
@@ -340,7 +342,7 @@ OpenViking 内置了记忆自迭代闭环。在每次会话结束时,开发者
- **用户记忆更新**:更新用户偏好相关记忆,使 Agent 回应更贴合用户需求
- **Agent 经验积累**:从任务执行经验中提取操作技巧、工具使用经验等核心内容,助力后续任务高效决策
-让 Agent 在与世界的交互中"越用越聪明",实现自我进化。
+让 Agent 在与世界的交互中"越用越聪明",实现自我进化。了解更多:[会话管理](./docs/zh/concepts/08-session.md)
---
diff --git a/docs/en/about/roadmap.md b/docs/en/about/roadmap.md
index 8922027fd..b09b5da3f 100644
--- a/docs/en/about/roadmap.md
+++ b/docs/en/about/roadmap.md
@@ -43,35 +43,31 @@ This document outlines the development roadmap for OpenViking.
## Future Plans
+### Service Deployment
+- Service mode deployment
+- Distributed storage backend
+
### Multi-modal Support
- Intelligent parsing and access for multi-modal resources (images, video, audio, etc.)
+- Directory storage structure for multi-modal resources
+
+### Context Management
+- Propagation updates when context is modified
+- Version management and rollback for context (git-like)
### Resource Node Access Control
- Multi-Agent / Multi-User support
- Role-based isolation design
- Access control and permission design for resource directory nodes
-### Context Version Control
-- Version management and rollback for context
-
-### Integration
+### Ecosystem Integration
- Popular Agent framework adapters
+- Plugin system for custom components
-### Distributed Architecture
-- Distributed storage backend
-- Horizontal scaling
-- Multi-node deployment
-- Cloud-native support
+We welcome suggestions and feedback in issues.
---
## Contributing
We welcome contributions to help achieve these goals. See [Contributing](contributing.md) for guidelines.
-
-Priority areas for contribution:
-- Performance optimization
-- New format parsers
-- Integration adapters
-- Documentation improvements
-- Test coverage
diff --git a/docs/en/faq/faq.md b/docs/en/faq/faq.md
index 16b172275..390f0eac0 100644
--- a/docs/en/faq/faq.md
+++ b/docs/en/faq/faq.md
@@ -1,32 +1,65 @@
# FAQ
-Frequently asked questions about OpenViking.
+## Basic Concepts
-## General
+### What is OpenViking? What problems does it solve?
-### What is OpenViking?
+OpenViking is an open-source context database designed specifically for AI Agents. It solves core pain points when building AI Agents:
-OpenViking is a context database for AI Agents. It follows the "Data in, Context out" philosophy - you add various data sources, and OpenViking provides optimized context for your AI applications.
+- **Fragmented Context**: Memories, resources, and skills are scattered everywhere, difficult to manage uniformly
+- **Poor Retrieval Effectiveness**: Traditional RAG's flat storage lacks global view, making it hard to understand complete context
+- **Unobservable Context**: Implicit retrieval chains are like black boxes, difficult to debug when errors occur
+- **Limited Memory Iteration**: Lacks Agent-related task memory and self-evolution capabilities
-### How is OpenViking different from vector databases?
+OpenViking unifies all context management through a filesystem paradigm, enabling tiered delivery and self-iteration.
-| Feature | Vector Database | OpenViking |
-|---------|-----------------|------------|
-| Storage | Vectors only | Content + Vectors |
-| Retrieval | Similarity search | Multi-layer context |
-| Output | Raw chunks | Structured context (L0/L1/L2) |
-| Memory | Not supported | Built-in memory management |
-| Sessions | Not supported | Conversation tracking |
+### What's the fundamental difference between OpenViking and traditional vector databases?
-### Is OpenViking open source?
+| Dimension | Traditional Vector DB | OpenViking |
+|-----------|----------------------|------------|
+| **Storage Model** | Flat vector storage | Hierarchical filesystem (AGFS) |
+| **Retrieval Method** | Single vector similarity search | Directory recursive retrieval + Intent analysis + Rerank |
+| **Output Format** | Raw chunks | Structured context (L0 Abstract/L1 Overview/L2 Details) |
+| **Memory Capability** | Not supported | Built-in 6 memory categories with auto-extraction and iteration |
+| **Observability** | Black box | Fully traceable retrieval trajectory |
+| **Context Types** | Documents only | Resource + Memory + Skill three types |
-Yes, OpenViking is fully open source under the Apache 2.0 license.
+### What is the L0/L1/L2 layered model? Why is it needed?
+
+L0/L1/L2 is OpenViking's progressive content loading mechanism, solving the problem of "stuffing massive context into prompts all at once":
+
+| Layer | Name | Token Limit | Purpose |
+|-------|------|-------------|---------|
+| **L0** | Abstract | ~100 tokens | Vector search recall, quick filtering, list display |
+| **L1** | Overview | ~2000 tokens | Rerank refinement, content navigation, decision reference |
+| **L2** | Details | Unlimited | Complete original content, on-demand deep loading |
-## Installation
+This design allows Agents to browse abstracts for quick positioning, then load details on demand, significantly saving token consumption.
-### What Python version is required?
+### What is Viking URI? What's its purpose?
+
+Viking URI is OpenViking's unified resource identifier, formatted as `viking://{scope}/{path}`. It enables precise location of any context:
+
+```
+viking://
+├── resources/ # Knowledge base: documents, code, web pages, etc.
+│ └── my_project/
+├── user/ # User context
+│ └── memories/ # User memories (preferences, entities, events)
+└── agent/ # Agent context
+ ├── skills/ # Callable skills
+ └── memories/ # Agent memories (cases, patterns)
+```
-Python 3.10 or higher is required.
+## Installation & Configuration
+
+### What are the environment requirements?
+
+- **Python Version**: 3.10 or higher
+- **Required Dependencies**: Embedding model (Volcengine Doubao recommended)
+- **Optional Dependencies**:
+ - VLM (Vision Language Model): For multimodal content processing and semantic extraction
+ - Rerank model: For improved retrieval precision
### How do I install OpenViking?
@@ -34,120 +67,317 @@ Python 3.10 or higher is required.
pip install openviking
```
-### What are the required dependencies?
+### How do I configure OpenViking?
+
+Create an `ov.conf` configuration file in your project directory:
+
+```json
+{
+ "user": "default_user",
+ "embedding": {
+ "dense": {
+ "backend": "volcengine",
+ "api_key": "your-api-key",
+ "model": "doubao-embedding-vision-250615",
+ "dimension": 1024,
+ "input": "multimodal"
+ }
+ },
+ "vlm": {
+ "backend": "volcengine",
+ "api_key": "your-api-key",
+ "model": "doubao-seed-1-8-251228",
+ "api_base": "https://ark.cn-beijing.volces.com/api/v3"
+ },
+ "rerank": {
+ "backend": "volcengine",
+ "api_key": "your-api-key",
+ "model": "doubao-rerank-250615"
+ },
+ "storage": {
+ "agfs": { "backend": "local", "path": "./data" },
+ "vectordb": { "backend": "local", "path": "./data" }
+ }
+}
+```
+
+Configuration priority: Constructor parameters > Config object > Config file > Environment variables > Default values
-OpenViking requires:
-- An embedding model (e.g., Volcengine Doubao)
-- Optional: VLM for multimodal content
-- Optional: Rerank model for improved retrieval
+### What Embedding backends are supported?
-## Usage
+| Backend | Description |
+|---------|-------------|
+| `volcengine` | Volcengine Embedding API (Recommended) |
+| `openai` | OpenAI Embedding API |
+| `vikingdb` | VikingDB Embedding API |
-### How do I initialize OpenViking?
+Supports Dense, Sparse, and Hybrid embedding modes.
+
+## Usage Guide
+
+### How do I initialize the client?
```python
import openviking as ov
+# Async client (recommended)
client = ov.AsyncOpenViking(path="./my_data")
await client.initialize()
+
+# Or use a config object
+from openviking.utils.config import OpenVikingConfig
+config = OpenVikingConfig(...)
+client = ov.AsyncOpenViking(config=config)
+await client.initialize()
```
### What file formats are supported?
-- **Text**: `.txt`, `.md`, `.json`, `.yaml`
-- **Code**: `.py`, `.js`, `.ts`, `.go`, `.java`, etc.
-- **Documents**: `.pdf`, `.docx`
-- **Images**: `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`
-- **Video**: `.mp4`, `.mov`, `.avi`
-- **Audio**: `.mp3`, `.wav`, `.m4a`
+| Type | Supported Formats |
+|------|-------------------|
+| **Text** | `.txt`, `.md`, `.json`, `.yaml` |
+| **Code** | `.py`, `.js`, `.ts`, `.go`, `.java`, `.cpp`, etc. |
+| **Documents** | `.pdf`, `.docx` |
+| **Images** | `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp` |
+| **Video** | `.mp4`, `.mov`, `.avi` |
+| **Audio** | `.mp3`, `.wav`, `.m4a` |
### How do I add resources?
```python
-# Single file
-await client.add_resource("./document.pdf")
+# Add single file
+await client.add_resource(
+ "./document.pdf",
+ reason="Project technical documentation", # Describe resource purpose to improve retrieval quality
+ target="viking://resources/docs/" # Specify storage location
+)
+
+# Add web page
+await client.add_resource(
+ "https://example.com/api-docs",
+ reason="API reference documentation"
+)
+
+# Wait for processing to complete
+await client.wait_processed()
+```
-# Directory
-await client.add_resource("./docs/", recursive=True)
+### What's the difference between `find()` and `search()`? Which should I use?
-# URL
-await client.add_resource("https://example.com/page")
+| Feature | `find()` | `search()` |
+|---------|----------|------------|
+| **Session Context** | Not required | Required |
+| **Intent Analysis** | Not used | Uses LLM to analyze and generate 0-5 queries |
+| **Latency** | Low | Higher |
+| **Use Case** | Simple semantic search | Complex tasks requiring context understanding |
+
+```python
+# find(): Simple direct semantic search
+results = await client.find(
+ "OAuth authentication flow",
+ target_uri="viking://resources/"
+)
+
+# search(): Complex tasks requiring intent analysis
+results = await client.search(
+ "Help me implement user login functionality",
+ session_info=session
+)
```
-### What's the difference between `find` and `search`?
+**Selection Guide**:
+- Know exactly what you're looking for → Use `find()`
+- Complex tasks needing multiple context types → Use `search()`
-- `find`: Simple semantic search, returns matching contexts
-- `search`: Session-aware search with intent analysis and query rewriting
+### How do I use session management?
-### How do I use sessions?
+Session management is a core capability of OpenViking, supporting conversation tracking and memory extraction:
```python
+# Create session
session = client.session()
-await session.add_message("user", [{"type": "text", "text": "Hello"}])
-await session.add_message("assistant", [{"type": "text", "text": "Hi!"}])
-await session.commit() # Extract memories
-```
-## Architecture
+# Add conversation messages
+await session.add_message("user", [{"type": "text", "text": "Help me analyze performance issues in this code"}])
+await session.add_message("assistant", [{"type": "text", "text": "Let me analyze..."}])
-### What is the L0/L1/L2 model?
+# Mark used context (for tracking)
+await session.used(["viking://resources/code/main.py"])
-- **L0 (Abstract)**: ~100 tokens, brief summary
-- **L1 (Overview)**: ~2000 tokens, detailed overview
-- **L2 (Content)**: Full original content
+# Commit session to trigger memory extraction
+await session.commit()
+```
-### What is a Viking URI?
+### What memory types does OpenViking support?
-Viking URI is a unified resource identifier: `viking://{scope}/{path}`
+OpenViking has 6 built-in memory categories, automatically extracted during session commit:
-Scopes:
-- `resources/` - Knowledge base
-- `user/memories/` - User memories
-- `agent/memories/` - Agent memories
-- `skills/` - Available skills
+| Category | Belongs To | Description |
+|----------|------------|-------------|
+| **profile** | user | User basic info (name, role, etc.) |
+| **preferences** | user | User preferences (code style, tool choices, etc.) |
+| **entities** | user | Entity memories (people, projects, organizations, etc.) |
+| **events** | user | Event records (decisions, milestones, etc.) |
+| **cases** | agent | Cases learned by Agent |
+| **patterns** | agent | Patterns learned by Agent |
-### What is AGFS?
+### How do I use Unix-like filesystem APIs?
-AGFS (Agent File System) is OpenViking's content storage layer that organizes data in a hierarchical structure with metadata.
+```python
+# List directory contents
+items = await client.ls("viking://resources/")
-## Performance
+# Read full content (L2)
+content = await client.read("viking://resources/doc.md")
-### How much data can OpenViking handle?
+# Get abstract (L0)
+abstract = await client.abstract("viking://resources")
-OpenViking is designed to handle large knowledge bases. Performance depends on:
-- Hardware resources
-- Embedding model speed
-- Vector index configuration
+# Get overview (L1)
+overview = await client.overview("viking://resources")
+```
+
+## Retrieval Optimization
### How do I improve retrieval quality?
-1. Use a rerank model
-2. Provide meaningful `reason` when adding resources
-3. Organize resources with appropriate `target` URIs
-4. Use sessions for context-aware search
+1. **Use Rerank model**: Configuring Rerank significantly improves ranking effectiveness
+2. **Provide meaningful `reason`**: Describe purpose when adding resources to help system understand resource value
+3. **Organize directory structure properly**: Use `target` parameter to group related resources together
+4. **Use session context**: `search()` leverages session history for intent analysis
+5. **Choose appropriate Embedding mode**: Use `multimodal` input for multimodal content
+
+### How is the retrieval result score calculated?
+
+OpenViking uses a score propagation mechanism:
+
+```
+Final Score = 0.5 × Embedding Similarity + 0.5 × Parent Directory Score
+```
+
+This design gives content under high-scoring directories a boost, reflecting the importance of "contextual environment".
+
+### What is directory recursive retrieval?
+
+Directory recursive retrieval is OpenViking's innovative retrieval strategy:
+
+1. **Intent Analysis**: Analyze query to generate multiple retrieval conditions
+2. **Initial Positioning**: Vector retrieval to locate high-scoring directories
+3. **Refined Exploration**: Secondary retrieval within high-scoring directories
+4. **Recursive Drill-down**: Layer-by-layer recursion until convergence
+5. **Result Aggregation**: Return the most relevant context
+
+This strategy finds semantically matching fragments while understanding the complete context of the information.
## Troubleshooting
-### Resources are not being indexed
+### Resources not being indexed after adding
+
+**Possible causes and solutions**:
+
+1. **Didn't wait for processing to complete**
+ ```python
+ await client.add_resource("./doc.pdf")
+ await client.wait_processed() # Must wait
+ ```
+
+2. **Embedding model configuration error**
+ - Check if `api_key` in `ov.conf` is correct
+ - Confirm model name and endpoint are configured correctly
+
+3. **Unsupported file format**
+ - Check if file extension is in the supported list
+ - Confirm file content is valid and not corrupted
-1. Check if `wait_processed()` was called
-2. Verify embedding model configuration
-3. Check logs for processing errors
+4. **View processing logs**
+ ```python
+ import logging
+ logging.basicConfig(level=logging.DEBUG)
+ ```
-### Search returns no results
+### Search not returning expected results
-1. Verify resources were added and processed
-2. Check the `target_uri` filter
-3. Try broader search terms
+**Troubleshooting steps**:
+
+1. **Confirm resources have been processed**
+ ```python
+ # Check if resources exist
+ items = await client.ls("viking://resources/")
+ ```
+
+2. **Check `target_uri` filter condition**
+ - Ensure search scope includes target resources
+ - Try expanding search scope
+
+3. **Try different query approaches**
+ - Use more specific or broader keywords
+ - Compare effects of `find()` and `search()`
+
+4. **Check L0 abstract quality**
+ ```python
+ abstract = await client.abstract("viking://resources/your-doc")
+ print(abstract) # Confirm abstract accurately reflects content
+ ```
### Memory extraction not working
-1. Ensure `session.commit()` was called
-2. Check LLM configuration for memory extraction
-3. Verify conversation has meaningful content
+**Troubleshooting steps**:
+
+1. **Ensure `commit()` was called**
+ ```python
+ await session.commit() # Triggers memory extraction
+ ```
+
+2. **Check VLM configuration**
+ - Memory extraction requires VLM model
+ - Confirm `vlm` configuration is correct
+
+3. **Confirm conversation content is meaningful**
+ - Casual chat may not produce memories
+ - Needs to contain extractable information (preferences, entities, events, etc.)
+
+4. **View extracted memories**
+ ```python
+ memories = await client.find("", target_uri="viking://user/memories/")
+ ```
+
+### Performance issues
+
+**Optimization suggestions**:
+
+1. **Batch processing**: Adding multiple resources at once is more efficient than one by one
+2. **Set appropriate `batch_size`**: Adjust batch processing size in Embedding configuration
+3. **Use local storage**: Use `local` backend during development to reduce network latency
+4. **Async operations**: Fully utilize `AsyncOpenViking`'s async capabilities
+
+## Deployment
+
+### What's the difference between embedded mode and service mode?
+
+| Mode | Use Case | Characteristics |
+|------|----------|-----------------|
+| **Embedded** | Local development, single-process apps | Auto-starts AGFS subprocess, uses local vector index |
+| **Service Mode** | Production, distributed deployment | Connects to remote services, supports multi-instance concurrency, independently scalable |
+
+```python
+# Embedded mode
+client = ov.AsyncOpenViking(path="./data")
+
+# Service mode
+client = ov.AsyncOpenViking(
+ vectordb_url="http://vectordb:5000",
+ agfs_url="http://agfs:8080"
+)
+```
+
+### Is OpenViking open source?
+
+Yes, OpenViking is fully open source under the Apache 2.0 license.
## Related Documentation
-- [Introduction](getting-started/instruction.md)
-- [Quick Start](getting-started/quickstart.md)
-- [Architecture](concepts/01-architecture.md)
+- [Introduction](../getting-started/introduction.md) - Understand OpenViking's design philosophy
+- [Quick Start](../getting-started/quickstart.md) - 5-minute tutorial
+- [Architecture Overview](../concepts/01-architecture.md) - Deep dive into system design
+- [Retrieval Mechanism](../concepts/06-retrieval.md) - Detailed retrieval process
+- [Configuration Guide](../configuration/configuration.md) - Complete configuration reference
diff --git a/docs/images/wechat-group-qrcode.jpg b/docs/images/wechat-group-qrcode.jpg
index b4e28698a..992a9aad1 100644
Binary files a/docs/images/wechat-group-qrcode.jpg and b/docs/images/wechat-group-qrcode.jpg differ
diff --git a/docs/zh/about/roadmap.md b/docs/zh/about/roadmap.md
index bfb9fb672..0312e3304 100644
--- a/docs/zh/about/roadmap.md
+++ b/docs/zh/about/roadmap.md
@@ -43,35 +43,30 @@
## 未来计划
+### 服务部署
+- 服务模式部署
+- 分布式存储后端
+
### 多模态支持
- 支持图像、视频、音频等多模态资源的智能解析和访问
+- 多模态资源的目录存储结构
+
+### 上下文管理
+- 上下文修改对上层的传导更新
+- 支持对上下文的版本管理和回滚 (参考 git)
### 资源节点的权限控制
- 支持多Agent / 多User 使用
- 不同角色的隔离设计
- 资源目录节点的访问控制和权限设计
-### 上下文版本控制
-- 支持对上下文的版本管理和回滚
-
-### 集成
+### 生态集成
- 流行 Agent 框架适配
+- 插件系统,支持自定义组件
-### 分布式架构
-- 分布式存储后端
-- 水平扩展
-- 多节点部署
-- 云原生支持
-
+欢迎在 issue 中提出建议和反馈。
---
## 贡献
我们欢迎贡献以帮助实现这些目标。请参阅 [贡献指南](contributing.md)。
-
-优先贡献领域:
-- 性能优化
-- 新格式解析器
-- 集成适配器
-- 文档改进
-- 测试覆盖率
diff --git a/docs/zh/faq/faq.md b/docs/zh/faq/faq.md
index d0b000cfe..a98aae77f 100644
--- a/docs/zh/faq/faq.md
+++ b/docs/zh/faq/faq.md
@@ -1,32 +1,65 @@
# 常见问题
-关于 OpenViking 的常见问题解答。
+## 基础概念
-## 基础问题
+### OpenViking 是什么?解决什么问题?
-### OpenViking 是什么?
+OpenViking 是一个专为 AI Agent 设计的开源上下文数据库。它解决了构建 AI Agent 时的核心痛点:
-OpenViking 是一个面向 AI Agent 的上下文数据库。它遵循 "Data in, Context out" 理念 - 你添加各种数据源,OpenViking 为你的 AI 应用提供优化的上下文。
+- **上下文碎片化**:记忆、资源、技能散落各处,难以统一管理
+- **检索效果不佳**:传统 RAG 平铺式存储缺乏全局视野,难以理解完整语境
+- **上下文不可观测**:隐式检索链路如同黑箱,出错时难以调试
+- **记忆迭代有限**:缺乏 Agent 相关的任务记忆和自我进化能力
-### OpenViking 和向量数据库有什么区别?
+OpenViking 通过文件系统范式统一管理所有上下文,实现分层供给与自我迭代。
-| 特性 | 向量数据库 | OpenViking |
-|------|-----------|------------|
-| 存储 | 仅向量 | 内容 + 向量 |
-| 检索 | 相似度搜索 | 多层上下文 |
-| 输出 | 原始分块 | 结构化上下文 (L0/L1/L2) |
-| 记忆 | 不支持 | 内置记忆管理 |
-| 会话 | 不支持 | 对话追踪 |
+### OpenViking 和传统向量数据库有什么本质区别?
-### OpenViking 是开源的吗?
+| 维度 | 传统向量数据库 | OpenViking |
+|------|---------------|------------|
+| **存储模型** | 扁平化向量存储 | 层级化文件系统(AGFS) |
+| **检索方式** | 单一向量相似度搜索 | 目录递归检索 + 意图分析 + Rerank |
+| **输出形式** | 原始分块 | 结构化上下文(L0 摘要/L1 概览/L2 详情) |
+| **记忆能力** | 不支持 | 内置 6 种记忆分类,支持自动提取和迭代 |
+| **可观测性** | 黑箱 | 检索轨迹完整可追溯 |
+| **上下文类型** | 仅文档 | Resource + Memory + Skill 三种类型 |
-是的,OpenViking 完全开源,采用 Apache 2.0 许可证。
+### 什么是 L0/L1/L2 分层模型?为什么需要它?
+
+L0/L1/L2 是 OpenViking 的渐进式内容加载机制,解决了"海量上下文一次性塞入提示词"的问题:
+
+| 层级 | 名称 | Token 限制 | 用途 |
+|------|------|-----------|------|
+| **L0** | 摘要 | ~100 tokens | 向量搜索召回、快速过滤、列表展示 |
+| **L1** | 概览 | ~2000 tokens | Rerank 精排、内容导航、决策参考 |
+| **L2** | 详情 | 无限制 | 完整原始内容、按需深度加载 |
-## 安装问题
+这种设计让 Agent 可以先浏览摘要快速定位,再按需加载详情,显著节省 Token 消耗。
-### 需要什么 Python 版本?
+### Viking URI 是什么?有什么作用?
+
+Viking URI 是 OpenViking 的统一资源标识符,格式为 `viking://{scope}/{path}`。它让系统能精准定位任何上下文:
+
+```
+viking://
+├── resources/ # 知识库:文档、代码、网页等
+│ └── my_project/
+├── user/ # 用户上下文
+│ └── memories/ # 用户记忆(偏好、实体、事件)
+└── agent/ # Agent 上下文
+ ├── skills/ # 可调用技能
+ └── memories/ # Agent 记忆(案例、模式)
+```
-需要 Python 3.10 或更高版本。
+## 安装与配置
+
+### 环境要求是什么?
+
+- **Python 版本**:3.10 或更高
+- **必需依赖**:Embedding 模型(推荐火山引擎 Doubao)
+- **可选依赖**:
+ - VLM(视觉语言模型):用于多模态内容处理和语义提取
+ - Rerank 模型:用于提升检索精度
### 如何安装 OpenViking?
@@ -34,120 +67,317 @@ OpenViking 是一个面向 AI Agent 的上下文数据库。它遵循 "Data in,
pip install openviking
```
-### 有哪些必需的依赖?
+### 如何配置 OpenViking?
+
+在项目目录创建 `ov.conf` 配置文件:
+
+```json
+{
+ "user": "default_user",
+ "embedding": {
+ "dense": {
+ "backend": "volcengine",
+ "api_key": "your-api-key",
+ "model": "doubao-embedding-vision-250615",
+ "dimension": 1024,
+ "input": "multimodal"
+ }
+ },
+ "vlm": {
+ "backend": "volcengine",
+ "api_key": "your-api-key",
+ "model": "doubao-seed-1-8-251228",
+ "api_base": "https://ark.cn-beijing.volces.com/api/v3"
+ },
+ "rerank": {
+ "backend": "volcengine",
+ "api_key": "your-api-key",
+ "model": "doubao-rerank-250615"
+ },
+ "storage": {
+ "agfs": { "backend": "local", "path": "./data" },
+ "vectordb": { "backend": "local", "path": "./data" }
+ }
+}
+```
+
+配置优先级:构造函数参数 > Config 对象 > 配置文件 > 环境变量 > 默认值
-OpenViking 需要:
-- Embedding 模型(如火山引擎 Doubao)
-- 可选:VLM 用于多模态内容
-- 可选:Rerank 模型用于提升检索质量
+### 支持哪些 Embedding 后端?
-## 使用问题
+| 后端 | 说明 |
+|------|------|
+| `volcengine` | 火山引擎 Embedding API(推荐) |
+| `openai` | OpenAI Embedding API |
+| `vikingdb` | VikingDB Embedding API |
-### 如何初始化 OpenViking?
+支持 Dense、Sparse 和 Hybrid 三种 Embedding 模式。
+
+## 使用指南
+
+### 如何初始化客户端?
```python
import openviking as ov
+# 异步客户端(推荐)
client = ov.AsyncOpenViking(path="./my_data")
await client.initialize()
+
+# 或使用配置对象
+from openviking.utils.config import OpenVikingConfig
+config = OpenVikingConfig(...)
+client = ov.AsyncOpenViking(config=config)
+await client.initialize()
```
### 支持哪些文件格式?
-- **文本**:`.txt`、`.md`、`.json`、`.yaml`
-- **代码**:`.py`、`.js`、`.ts`、`.go`、`.java` 等
-- **文档**:`.pdf`、`.docx`
-- **图片**:`.png`、`.jpg`、`.jpeg`、`.gif`、`.webp`
-- **视频**:`.mp4`、`.mov`、`.avi`
-- **音频**:`.mp3`、`.wav`、`.m4a`
+| 类型 | 支持格式 |
+|------|----------|
+| **文本** | `.txt`、`.md`、`.json`、`.yaml` |
+| **代码** | `.py`、`.js`、`.ts`、`.go`、`.java`、`.cpp` 等 |
+| **文档** | `.pdf`、`.docx` |
+| **图片** | `.png`、`.jpg`、`.jpeg`、`.gif`、`.webp` |
+| **视频** | `.mp4`、`.mov`、`.avi` |
+| **音频** | `.mp3`、`.wav`、`.m4a` |
### 如何添加资源?
```python
-# 单个文件
-await client.add_resource("./document.pdf")
+# 添加单个文件
+await client.add_resource(
+ "./document.pdf",
+ reason="项目技术文档", # 描述资源用途,提升检索质量
+ target="viking://resources/docs/" # 指定存储位置
+)
+
+# 添加网页
+await client.add_resource(
+ "https://example.com/api-docs",
+ reason="API 参考文档"
+)
+
+# 等待处理完成
+await client.wait_processed()
+```
-# 目录
-await client.add_resource("./docs/", recursive=True)
+### `find()` 和 `search()` 有什么区别?应该用哪个?
-# URL
-await client.add_resource("https://example.com/page")
+| 特性 | `find()` | `search()` |
+|------|----------|------------|
+| **会话上下文** | 不需要 | 需要 |
+| **意图分析** | 不使用 | 使用 LLM 分析生成 0-5 个查询 |
+| **延迟** | 低 | 较高 |
+| **适用场景** | 简单语义搜索 | 复杂任务、需要理解上下文 |
+
+```python
+# find(): 简单直接的语义搜索
+results = await client.find(
+ "OAuth 认证流程",
+ target_uri="viking://resources/"
+)
+
+# search(): 复杂任务,需要意图分析
+results = await client.search(
+ "帮我实现用户登录功能",
+ session_info=session
+)
```
-### `find` 和 `search` 有什么区别?
+**选择建议**:
+- 明确知道要找什么 → 用 `find()`
+- 复杂任务需要多种上下文 → 用 `search()`
-- `find`:简单语义搜索,返回匹配的上下文
-- `search`:会话感知搜索,包含意图分析和查询重写
+### 如何使用会话管理?
-### 如何使用会话?
+会话管理是 OpenViking 的核心能力,支持对话追踪和记忆提取:
```python
+# 创建会话
session = client.session()
-await session.add_message("user", [{"type": "text", "text": "你好"}])
-await session.add_message("assistant", [{"type": "text", "text": "你好!"}])
-await session.commit() # 提取记忆
-```
-## 架构问题
+# 添加对话消息
+await session.add_message("user", [{"type": "text", "text": "帮我分析这段代码的性能问题"}])
+await session.add_message("assistant", [{"type": "text", "text": "我来分析一下..."}])
-### 什么是 L0/L1/L2 模型?
+# 标记使用的上下文(用于追踪)
+await session.used(["viking://resources/code/main.py"])
-- **L0(摘要)**:约 100 tokens,简要摘要
-- **L1(概览)**:约 2000 tokens,详细概览
-- **L2(内容)**:完整原始内容
+# 提交会话,触发记忆提取
+await session.commit()
+```
-### 什么是 Viking URI?
+### OpenViking 支持哪些记忆类型?
-Viking URI 是统一资源标识符:`viking://{scope}/{path}`
+OpenViking 内置 6 种记忆分类,在会话提交时自动提取:
-作用域:
-- `resources/` - 知识库
-- `user/memories/` - 用户记忆
-- `agent/memories/` - Agent 记忆
-- `skills/` - 可用技能
+| 分类 | 归属 | 说明 |
+|------|------|------|
+| **profile** | user | 用户基本信息(姓名、角色等) |
+| **preferences** | user | 用户偏好(代码风格、工具选择等) |
+| **entities** | user | 实体记忆(人物、项目、组织等) |
+| **events** | user | 事件记录(决策、里程碑等) |
+| **cases** | agent | Agent 学习的案例 |
+| **patterns** | agent | Agent 学习的模式 |
-### 什么是 AGFS?
+### 如何使用类 Unix 的文件系统 API?
-AGFS(Agent File System)是 OpenViking 的内容存储层,以层级结构组织数据和元数据。
+```python
+# 列出目录内容
+items = await client.ls("viking://resources/")
-## 性能问题
+# 读取完整内容(L2)
+content = await client.read("viking://resources/doc.md")
-### OpenViking 能处理多少数据?
+# 获取摘要(L0)
+abstract = await client.abstract("viking://resources")
-OpenViking 设计用于处理大型知识库。性能取决于:
-- 硬件资源
-- Embedding 模型速度
-- 向量索引配置
+# 获取概览(L1)
+overview = await client.overview("viking://resources")
+```
+
+## 检索优化
### 如何提升检索质量?
-1. 使用 Rerank 模型
-2. 添加资源时提供有意义的 `reason`
-3. 使用适当的 `target` URI 组织资源
-4. 使用会话进行上下文感知搜索
+1. **使用 Rerank 模型**:配置 Rerank 可显著提升精排效果
+2. **提供有意义的 `reason`**:添加资源时描述用途,帮助系统理解资源价值
+3. **合理组织目录结构**:使用 `target` 参数将相关资源放在一起
+4. **使用会话上下文**:`search()` 会利用会话历史进行意图分析
+5. **选择合适的 Embedding 模式**:多模态内容使用 `multimodal` 输入
+
+### 检索结果的分数是如何计算的?
+
+OpenViking 使用分数传播机制:
+
+```
+最终分数 = 0.5 × Embedding 相似度 + 0.5 × 父目录分数
+```
+
+这种设计让高分目录下的内容获得加成,体现了"上下文语境"的重要性。
+
+### 什么是目录递归检索?
+
+目录递归检索是 OpenViking 的创新检索策略:
+
+1. **意图分析**:分析查询生成多个检索条件
+2. **初始定位**:向量检索定位高分目录
+3. **精细探索**:在高分目录下进行二次检索
+4. **递归下探**:逐层递归直到收敛
+5. **结果汇总**:返回最相关的上下文
+
+这种策略能找到语义匹配的片段,同时理解信息的完整语境。
## 故障排除
-### 资源没有被索引
+### 资源添加后没有被索引
+
+**可能原因及解决方案**:
+
+1. **未等待处理完成**
+ ```python
+ await client.add_resource("./doc.pdf")
+ await client.wait_processed() # 必须等待
+ ```
+
+2. **Embedding 模型配置错误**
+ - 检查 `ov.conf` 中的 `api_key` 是否正确
+ - 确认模型名称和 endpoint 配置正确
+
+3. **文件格式不支持**
+ - 检查文件扩展名是否在支持列表中
+ - 确认文件内容有效且未损坏
-1. 检查是否调用了 `wait_processed()`
-2. 验证 Embedding 模型配置
-3. 检查日志中的处理错误
+4. **查看处理日志**
+ ```python
+ import logging
+ logging.basicConfig(level=logging.DEBUG)
+ ```
-### 搜索没有返回结果
+### 搜索没有返回预期结果
-1. 验证资源已添加并处理完成
-2. 检查 `target_uri` 过滤条件
-3. 尝试更宽泛的搜索词
+**排查步骤**:
+
+1. **确认资源已处理完成**
+ ```python
+ # 检查资源是否存在
+ items = await client.ls("viking://resources/")
+ ```
+
+2. **检查 `target_uri` 过滤条件**
+ - 确保搜索范围包含目标资源
+ - 尝试扩大搜索范围
+
+3. **尝试不同的查询方式**
+ - 使用更具体或更宽泛的关键词
+ - 尝试 `find()` 和 `search()` 对比效果
+
+4. **检查 L0 摘要质量**
+ ```python
+ abstract = await client.abstract("viking://resources/your-doc")
+ print(abstract) # 确认摘要是否准确反映内容
+ ```
### 记忆提取不工作
-1. 确保调用了 `session.commit()`
-2. 检查用于记忆提取的 LLM 配置
-3. 验证对话包含有意义的内容
+**排查步骤**:
+
+1. **确保调用了 `commit()`**
+ ```python
+ await session.commit() # 触发记忆提取
+ ```
+
+2. **检查 VLM 配置**
+ - 记忆提取需要 VLM 模型
+ - 确认 `vlm` 配置正确
+
+3. **确认对话内容有意义**
+ - 闲聊内容可能不会产生记忆
+ - 需要包含可提取的信息(偏好、实体、事件等)
+
+4. **查看提取的记忆**
+ ```python
+ memories = await client.find("", target_uri="viking://user/memories/")
+ ```
+
+### 性能问题
+
+**优化建议**:
+
+1. **批量处理**:一次添加多个资源比逐个添加更高效
+2. **合理设置 `batch_size`**:Embedding 配置中调整批处理大小
+3. **使用本地存储**:开发阶段使用 `local` 后端减少网络延迟
+4. **异步操作**:充分利用 `AsyncOpenViking` 的异步特性
+
+## 部署相关
+
+### 嵌入式模式和服务模式有什么区别?
+
+| 模式 | 适用场景 | 特点 |
+|------|----------|------|
+| **嵌入式** | 本地开发、单进程应用 | 自动启动 AGFS 子进程,使用本地向量索引 |
+| **服务模式** | 生产环境、分布式部署 | 连接远程服务,支持多实例并发,可独立扩展 |
+
+```python
+# 嵌入式模式
+client = ov.AsyncOpenViking(path="./data")
+
+# 服务模式
+client = ov.AsyncOpenViking(
+ vectordb_url="http://vectordb:5000",
+ agfs_url="http://agfs:8080"
+)
+```
+
+### OpenViking 是开源的吗?
+
+是的,OpenViking 完全开源,采用 Apache 2.0 许可证。
## 相关文档
-- [简介](getting-started/instruction.md)
-- [快速开始](getting-started/quickstart.md)
-- [架构概述](concepts/01-architecture.md)
+- [简介](../getting-started/introduction.md) - 了解 OpenViking 的设计理念
+- [快速开始](../getting-started/quickstart.md) - 5 分钟上手教程
+- [架构概述](../concepts/01-architecture.md) - 深入理解系统设计
+- [检索机制](../concepts/06-retrieval.md) - 检索流程详解
+- [配置指南](../configuration/configuration.md) - 完整配置参考