Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
eat(opencode): add opencode plugin and update docs (#351)
yangxinxin-7c02b4e8b1afd
diff --git a/docs/en/concepts/06-extraction.md b/docs/en/concepts/06-extraction.mdindex 68a3133ce..cf387e372 100644--- a/docs/en/concepts/06-extraction.md+++ b/docs/en/concepts/06-extraction.md@@ -131,6 +131,62 @@ Leaf directories → Parent directories → Root| `max_images_per_call` | 10 | Max images per VLM call || `max_sections_per_call` | 20 | Max sections per VLM call |+## Code Skeleton Extraction (AST Mode)++For code files, OpenViking supports AST-based skeleton extraction via tree-sitter as a lightweight alternative to LLM summarization, significantly reducing processing cost.++### Modes++Controlled by `code_summary_mode` in `ov.conf` (see [Configuration](../guides/01-configuration.md#code)):++| Mode | Description |+|------|-------------|+| `"ast"` | Extract structural skeleton for files ≥100 lines, skip LLM calls (**default**) |+| `"llm"` | Always use LLM for summarization (original behavior) |+| `"ast_llm"` | Extract AST skeleton first, then pass it as context to LLM for summarization |++### What AST Extracts++The skeleton includes:++- Module-level docstring (first line)+- Import statement list+- Class names, base classes, and method signatures (`ast` mode: first-line docstrings only; `ast_llm` mode: full docstrings)+- Top-level function signatures++### Supported Languages++The following languages have dedicated extractors built on tree-sitter:++| Language | Status |+|----------|--------|+| Python | Supported |+| JavaScript / TypeScript | Supported |+| Rust | Supported |+| Go | Supported |+| Java | Supported |+| C / C++ | Supported |++Other languages automatically fall back to LLM.++### Fallback Behavior++The following conditions trigger automatic fallback to LLM, with the reason logged. The overall pipeline is unaffected:++- Language not in the supported list+- File has fewer than 100 lines+- AST parse error+- Extraction produces an empty skeleton++### File Structure++```+openviking/parse/parsers/code/ast/+├── extractor.py # Language detection and dispatch+├── skeleton.py # CodeSkeleton / FunctionSig / ClassSkeleton data structures+└── languages/ # Per-language extractors+```+## Three Context Types Extraction### Flow Comparisondiff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.mdindex 48125c94d..0444650c9 100644--- a/docs/en/guides/01-configuration.md+++ b/docs/en/guides/01-configuration.md@@ -282,6 +282,40 @@ When resources are added, VLM generates:If VLM is not configured, L0/L1 will be generated from content directly (less semantic), and multimodal resources may have limited descriptions.+### code++Controls how code files are summarized via `code_summary_mode`. Both config formats are equivalent:++```json+{+ "code": {+ "code_summary_mode": "ast"+ }+}+```++```json+{+ "parsers": {+ "code": {+ "code_summary_mode": "ast"+ }+ }+}+```++Set `code_summary_mode` to one of:++| Value | Description | Default |+|-------|-------------|---------|+| `"ast"` | Extract AST skeleton (class names, method signatures, first-line docstrings, imports) for files ≥100 lines, skip LLM calls. **Recommended for large-scale code indexing** | ✓ |+| `"llm"` | Always use LLM for summarization (higher cost) | |+| `"ast_llm"` | Extract AST skeleton (with full docstrings) first, then pass it as context to LLM (highest quality, moderate cost) | |++AST extraction supports: Python, JavaScript/TypeScript, Rust, Go, Java, C/C++. Other languages, extraction failures, or empty skeletons automatically fall back to LLM.++See [Code Skeleton Extraction](../concepts/06-extraction.md#code-skeleton-extraction-ast-mode) for details.+### rerankReranking model for search result refinement.@@ -609,6 +643,9 @@ For startup and deployment details see [Deployment](./03-deployment.md), for aut"project": "string"}},+ "code": {+ "code_summary_mode": "ast"+ },"server": {"host": "0.0.0.0","port": 1933,diff --git a/docs/zh/concepts/06-extraction.md b/docs/zh/concepts/06-extraction.mdindex a7f079223..69d379be7 100644--- a/docs/zh/concepts/06-extraction.md+++ b/docs/zh/concepts/06-extraction.md@@ -130,6 +130,62 @@ SemanticMsg(| `max_images_per_call` | 10 | 单次 VLM 最大图片数 || `max_sections_per_call` | 20 | 单次 VLM 最大章节数 |+## 代码骨架提取(AST 模式)++对于代码文件,OpenViking 支持基于 tree-sitter 的 AST 骨架提取,作为 LLM 摘要的轻量替代方案,可显著降低处理成本。++### 工作模式++在 `ov.conf` 中通过 `code_summary_mode` 字段控制(参见[配置文档](../guides/01-configuration.md#code)),支持三种模式:++| 模式 | 说明 |+|------|------|+| `"ast"` | 对 ≥100 行的代码文件提取结构骨架,跳过 LLM 调用(**默认**) |+| `"llm"` | 全部走 LLM 生成摘要(原有行为) |+| `"ast_llm"` | 先提取 AST 骨架,再将骨架作为上下文辅助 LLM 生成摘要 |++### AST 提取内容++提取的骨架包含:++- 模块级 docstring(首行)+- import 语句列表+- 类名、继承关系及方法签名(`ast` 模式仅保留 docstring 首行,`ast_llm` 模式保留完整 docstring)+- 顶层函数签名++### 支持语言++以下语言有专属 extractor,基于 tree-sitter 实现精确提取:++| 语言 | 说明 |+|------|------|+| Python | 完整支持 |+| JavaScript / TypeScript | 完整支持 |+| Rust | 完整支持 |+| Go | 完整支持 |+| Java | 完整支持 |+| C / C++ | 完整支持 |++其他语言不在支持列表内,自动 fallback 到 LLM。++### Fallback 机制++以下情况自动回退到 LLM,并在日志中记录原因,整体流程不受影响:++- 语言不在支持列表中+- 文件行数 < 100+- AST 解析报错+- 提取结果为空骨架++### 文件结构++```+openviking/parse/parsers/code/ast/+├── extractor.py # 语言检测 + 分发入口+├── skeleton.py # CodeSkeleton / FunctionSig / ClassSkeleton 数据结构+└── languages/ # 各语言专属 extractor+```+## 三种上下文提取### 流程对比diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.mdindex e525951bf..c8db6cbf9 100644--- a/docs/zh/guides/01-configuration.md+++ b/docs/zh/guides/01-configuration.md@@ -289,6 +289,40 @@ OpenViking 使用 JSON 配置文件(`ov.conf`)进行设置。配置文件支如果未配置 VLM,L0/L1 将直接从内容生成(语义性较弱),多模态资源的描述可能有限。+### code++通过 `code_summary_mode` 控制代码文件的摘要生成方式。以下两种写法等价:++```json+{+ "code": {+ "code_summary_mode": "ast"+ }+}+```++```json+{+ "parsers": {+ "code": {+ "code_summary_mode": "ast"+ }+ }+}+```++将 `code_summary_mode` 设置为以下三个值之一:++| 值 | 说明 | 默认 |+|----|------|------|+| `"ast"` | 对 ≥100 行的代码文件提取 AST 骨架(类名、方法签名、首行注释、import),跳过 LLM 调用。**推荐用于大规模代码索引** | ✓ |+| `"llm"` | 全部走 LLM 生成摘要(成本较高) | |+| `"ast_llm"` | 先提取 AST 骨架(含完整注释),再将骨架作为上下文辅助 LLM 生成摘要(质量最高,成本居中) | |++AST 提取支持:Python、JavaScript/TypeScript、Rust、Go、Java、C/C++。其他语言、提取失败或骨架为空时自动 fallback 到 LLM。++详见 [代码骨架提取](../concepts/06-extraction.md#代码骨架提取ast-模式)。+### rerank用于搜索结果精排的 Rerank 模型。@@ -618,6 +652,9 @@ HTTP 客户端(`SyncHTTPClient` / `AsyncHTTPClient`)和 CLI 工具连接远"project": "string"}},+ "code": {+ "code_summary_mode": "ast"+ },"server": {"host": "string","port": 1933,diff --git a/examples/opencode/plugin/README.md b/examples/opencode/plugin/README.mdnew file mode 100644index 000000000..f87e11fd3--- /dev/null+++ b/examples/opencode/plugin/README.md@@ -0,0 +1,67 @@+# openviking-opencode++OpenViking plugin for [OpenCode](https://opencode.ai). Injects your indexed code repos into the AI's context and auto-starts the OpenViking server when needed.++## Prerequisites++Install the latest OpenViking and configure `~/.openviking/ov.conf`:++```bash+pip install openviking --upgrade+```++```json+{+ "storage": {+ "workspace": "/path/to/your/workspace"+ },+ "embedding": {+ "dense": {+ "provider": "openai",+ "model": "your-embedding-model",+ "api_key": "your-api-key",+ "api_base": "https://your-provider/v1",+ "dimension": 1024+ },+ "max_concurrent": 100+ },+ "vlm": {+ "provider": "openai",+ "model": "your-vlm-model",+ "api_key": "your-api-key",+ "api_base": "https://your-provider/v1"+ }+}+```++For other providers (Volcengine, Anthropic, DeepSeek, Ollama, etc.) see the [OpenViking docs](https://www.openviking.ai/docs).++Before starting OpenCode, make sure the OpenViking server is running. If it's not already started:++```bash+openviking-server --config ~/.openviking/ov.conf > /tmp/openviking.log 2>&1 &+```++## Usage in OpenCode++Add the plugin to `~/.config/opencode/opencode.json`:++```json+{+ "plugin": ["openviking-opencode"]+}+```++Restart OpenCode — the skill is installed automatically.++**Index a repo** (just ask in chat):+```+"Add https://github.com/tiangolo/fastapi to OpenViking"+```++**Search** — once repos are indexed, the AI searches them automatically when relevant. You can also trigger it explicitly:+```+"How does fastapi handle dependency injection?"+"Use openviking to find how JWT tokens are verified"+```+diff --git a/examples/opencode/plugin/index.mjs b/examples/opencode/plugin/index.mjsnew file mode 100644index 000000000..a91efe1fa--- /dev/null+++ b/examples/opencode/plugin/index.mjs@@ -0,0 +1,151 @@+import { exec } from "child_process"+import { promisify } from "util"+import { readFileSync, mkdirSync, writeFileSync, existsSync } from "fs"+import { homedir } from "os"+import { join, dirname } from "path"+import { fileURLToPath } from "url"++const execAsync = promisify(exec)+const __dirname = dirname(fileURLToPath(import.meta.url))+const OV_CONF = join(homedir(), ".openviking", "ov.conf")++// ── Helpers ───────────────────────────────────────────────────────────────────++async function run(cmd, opts = {}) {+ return execAsync(cmd, { timeout: 10000, ...opts })+}++async function isHealthy() {+ try {+ await run("ov health", { timeout: 3000 })+ return true+ } catch {+ return false+ }+}++async function startServer() {+ // Start in background, wait up to 30s for healthy+ await run("openviking-server --config " + OV_CONF + " > /tmp/openviking.log 2>&1 &")+ for (let i = 0; i < 10; i++) {+ await new Promise((r) => setTimeout(r, 3000))+ if (await isHealthy()) return true+ }+ return false+}++// ── Skill auto-install ────────────────────────────────────────────────────────++function installSkill() {+ const src = join(__dirname, "skills", "openviking", "SKILL.md")+ const dest = join(homedir(), ".config", "opencode", "skills", "openviking", "SKILL.md")+ try {+ if (!existsSync(dirname(dest))) mkdirSync(dirname(dest), { recursive: true })+ const content = readFileSync(src, "utf8")+ if (!existsSync(dest) || readFileSync(dest, "utf8") !== content) {+ writeFileSync(dest, content, "utf8")+ }+ } catch {}+}++// ── Repo context cache ────────────────────────────────────────────────────────++let cachedRepos = null+let lastFetchTime = 0+const CACHE_TTL_MS = 60 * 1000++async function loadRepos() {+ const now = Date.now()+ if (cachedRepos !== null && now - lastFetchTime < CACHE_TTL_MS) return++ try {+ const { stdout } = await run(+ "ov --output json ls viking://resources/ --abs-limit 2000"+ )+ const items = JSON.parse(stdout)?.result ?? []+ const repos = items+ .filter((item) => item.uri?.startsWith("viking://resources/"))+ .map((item) => {+ const name = item.uri.replace("viking://resources/", "").replace(/\/$/, "")+ return item.abstract+ ? `- **${name}** (${item.uri})\n ${item.abstract}`+ : `- **${name}** (${item.uri})`+ })+ if (repos.length > 0) {+ cachedRepos = repos.join("\n")+ lastFetchTime = now+ }+ } catch {}+}++// ── Init: check deps, start server if needed ─────────────────────────────────++async function init(client) {+ const toast = (message, variant = "warning") =>+ client.tui.showToast({+ body: { title: "OpenViking", message, variant, duration: 8000 },+ }).catch(() => {})++ // 服务已在跑,直接返回+ if (await isHealthy()) return true++ // 没跑,先看是哪种情况+ try {+ await run("command -v ov", { timeout: 2000 })+ } catch {+ // command not found → 没装+ await toast("openviking 未安装,请运行: pip install openviking", "error")+ return false+ }++ // 装了但没有配置文件 → 无法启动+ if (!existsSync(OV_CONF)) {+ await toast("未找到 ~/.openviking/ov.conf,请先配置 API keys 后再启动服务", "warning")+ return false+ }++ // 装了有配置,服务没跑 → 静默自动启动+ const started = await startServer()+ if (!started) {+ await toast("openviking 服务启动失败,查看日志: /tmp/openviking.log", "error")+ return false+ }++ return true+}++// ── Plugin export ─────────────────────────────────────────────────────────────++/**+ * @type {import('@opencode-ai/plugin').Plugin}+ */+export async function OpenVikingPlugin({ client }) {+ installSkill()++ // 后台初始化,不阻塞 opencode 启动+ Promise.resolve().then(async () => {+ const ready = await init(client)+ if (ready) await loadRepos()+ })++ return {+ "experimental.chat.system.transform": (_input, output) => {+ if (!cachedRepos) return+ output.system.push(+ `## OpenViking — Indexed Code Repositories\n\n` ++ `The following repos are semantically indexed and searchable.\n` ++ `When the user asks about any of these projects or their internals, ` ++ `you MUST proactively load skill("openviking") and use the correct ov commands to search and retrieve content before answering.\n\n` ++ cachedRepos+ )+ },++ "session.created": async () => {+ const ready = await init(client)+ if (ready) {+ cachedRepos = null+ await loadRepos()+ }+ },+ }+}diff --git a/examples/opencode/plugin/openviking-context.ts b/examples/opencode/plugin/openviking-context.tsnew file mode 100644index 000000000..1b728ea78--- /dev/null+++ b/examples/opencode/plugin/openviking-context.ts@@ -0,0 +1,60 @@+import { exec } from "child_process"+import { promisify } from "util"++const execAsync = promisify(exec)++let cachedRepos: string | null = null++async function loadRepos(): Promise<void> {+ try {+ const { stdout } = await execAsync(+ "ov --output json ls viking://resources/ --abs-limit 2000",+ { timeout: 8000 }+ )+ const parsed = JSON.parse(stdout)+ const items: Array<{ uri: string; abstract?: string }> = parsed?.result ?? []+ const repos = items+ .filter((item) => item.uri?.startsWith("viking://resources/"))+ .map((item) => {+ const name = item.uri.replace("viking://resources/", "").replace(/\/$/, "")+ return item.abstract+ ? `- **${name}** (${item.uri})\n ${item.abstract}`+ : `- **${name}** (${item.uri})`+ })+ if (repos.length > 0) {+ cachedRepos = repos.join("\n")+ }+ } catch {+ // openviking not available — plugin is a no-op+ }+}++/**+ * @type {import('@opencode-ai/plugin').Plugin}+ */+export async function OpenVikingContextPlugin() {+ // Fetch repos at startup so the cache is ready before any request+ await loadRepos()++ return {+ // Inject repo list into every LLM request's system prompt (sync — no await)+ "experimental.chat.system.transform": (+ _input: unknown,+ output: { system: string[] }+ ) => {+ if (!cachedRepos) return+ output.system.push(+ `## OpenViking — Indexed Code Repositories\n\n` ++ `The following repos are semantically indexed and searchable.\n` ++ `When the user asks about any of these projects or their internals, ` ++ `you MUST proactively call skill("openviking") before answering.\n\n` ++ cachedRepos+ )+ },++ // Refresh repo list when a new session starts+ "session.created": async () => {+ await loadRepos()+ },+ }+}diff --git a/examples/opencode/plugin/package.json b/examples/opencode/plugin/package.jsonnew file mode 100644index 000000000..8b9705c3b--- /dev/null+++ b/examples/opencode/plugin/package.json@@ -0,0 +1,22 @@+{+ "name": "openviking-opencode",+ "version": "0.3.0",+ "description": "OpenCode plugin for OpenViking — injects indexed repo context into the AI assistant and auto-installs the openviking skill",+ "type": "module",+ "main": "index.mjs",+ "exports": {+ ".": "./index.mjs"+ },+ "keywords": [+ "opencode",+ "opencode-plugin",+ "openviking",+ "rag",+ "code-search"+ ],+ "license": "Apache-2.0",+ "repository": {+ "type": "git",+ "url": "https://github.com/volcengine/OpenViking"+ }+}diff --git a/examples/opencode/plugin/skills/openviking/SKILL.md b/examples/opencode/plugin/skills/openviking/SKILL.mdnew file mode 100644index 000000000..c9a0b6ec8--- /dev/null+++ b/examples/opencode/plugin/skills/openviking/SKILL.md@@ -0,0 +1,130 @@+---+name: openviking+description: "Activate when the user asks about any repository listed in the system prompt under 'OpenViking — Indexed Code Repositories', or when they ask about an external library, framework, or project that may have been indexed. Also activate when the user wants to add, remove, or manage repos. Always search the local codebase first before using this skill."+license: MIT+compatibility: opencode+---++# OpenViking Code Repository Search++**IMPORTANT: All `ov` commands are terminal (shell) commands — run them via the `bash` tool. Execute directly — no pre-checks, no test commands. Handle errors when they occur.**++## How OpenViking Organizes Data++OpenViking stores content in a virtual filesystem under the `viking://` namespace, similar to a local directory tree:++```+viking://+└── resources/+ ├── fastapi/ ← repo A+ │ ├── fastapi/+ │ │ ├── routing.py+ │ │ └── dependencies/+ │ └── tests/+ └── requests/ ← repo B+ ├── requests/+ └── tests/+```++Each directory has AI-generated summaries (`abstract` / `overview`). **The key principle: narrow the URI scope to improve retrieval efficiency.** Instead of searching all repos, lock to a specific repo or subdirectory — this reduces noise and speeds up results significantly.++## Search Commands++Choose the right command based on what you're looking for:++| Command | Use when | Example |+|---------|----------|---------|+| `ov find` | You know the **concept** but not the exact code | "dependency injection", "rate limiting logic" |+| `ov grep` | You know the **exact keyword or symbol** | function name, class name, error string |+| `ov glob` | You want to **enumerate files** by pattern | all `*.py` files, all test files |++```bash+# Semantic search — concept/intent based+ov find "dependency injection" --uri viking://resources/fastapi --limit 10+ov find "how tokens are refreshed" --uri viking://resources/fastapi/fastapi/security+ov find "JWT authentication" --limit 10 # across all repos++# Keyword search — exact match or regex+ov grep "verify_token" --uri viking://resources/fastapi+ov grep "class.*Session" --uri viking://resources/requests/requests++# File enumeration — by name pattern (--uri is required)+ov glob "**/*.py" --uri viking://resources/fastapi+ov glob "**/test_*.py" --uri viking://resources/fastapi/tests+ov glob "**/*.py" --uri viking://resources/ # across all repos+```++**Narrowing scope:** once you identify a relevant directory, pass it as `--uri` to restrict subsequent searches to that subtree — this is faster and more precise than searching the whole repo.++## Read Content++```bash+# Directories: AI-generated summaries+ov abstract viking://resources/fastapi/fastapi/dependencies/ # one-line summary+ov overview viking://resources/fastapi/fastapi/dependencies/ # detailed breakdown++# Files: raw content+ov read viking://resources/fastapi/fastapi/dependencies/utils.py+ov read viking://resources/fastapi/fastapi/dependencies/utils.py --offset 100 --limit 50+```++`abstract` / `overview` only work on directories. `read` only works on files.++## Browse++```bash+ov ls viking://resources/ # list all indexed repos+ov ls viking://resources/fastapi # list repo top-level contents+ov tree viking://resources/fastapi # full directory tree+```++## Add a Repository++```bash+ov add-resource https://github.com/owner/repo --to viking://resources/ --timeout 300+ov add-resource /path/to/project --to viking://resources/ --timeout 300+```++`--timeout` is required (seconds). Use 300 (5 min) for small repos, increase for larger ones.++After submitting, run `ov observer queue` once and report status to user. Indexing runs in background — do not poll or wait.++| Repo Size | Files | Est. Time |+|-----------|-------|-----------|+| Small | < 100 | 2–5 min |+| Medium | 100–500 | 5–20 min |+| Large | 500+ | 20–60+ min |++## Remove a Repository++```bash+ov rm viking://resources/fastapi --recursive+```++This permanently deletes the repo and all its indexed content. Confirm with the user before running.++## Error Handling++**`command not found: ov`** → Tell user: `pip install openviking --upgrade`. Stop.++**`url is required` / `CLI_CONFIG` error** → Auto-create config and retry:+```bash+mkdir -p ~/.openviking && echo '{"url": "http://localhost:1933"}' > ~/.openviking/ovcli.conf+```++**`CONNECTION_ERROR` / failed to connect:**+- `~/.openviking/ov.conf` **exists** → auto-start server, wait until healthy, retry:+ ```bash+ openviking-server --config ~/.openviking/ov.conf > /tmp/openviking.log 2>&1 &+ for i in $(seq 1 10); do ov health 2>/dev/null && break; sleep 3; done+ ```+- **Does not exist** → Tell user to configure `~/.openviking/ov.conf` first. Stop.++## More Help++For other issues or command details, run:++```bash+ov help+ov <command> --help # e.g. ov find --help+```