Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
docs(guides): review encryption, backup, OAuth and storage guides (#5509)
Zayn Jarvis1beeb0e10215
diff --git a/docs/.vitepress/theme/components/EncryptionFlowDiagram.vue b/docs/.vitepress/theme/components/EncryptionFlowDiagram.vuenew file mode 100644index 000000000..d5809b2ea--- /dev/null+++ b/docs/.vitepress/theme/components/EncryptionFlowDiagram.vue@@ -0,0 +1,206 @@+<script setup>+import { computed, useId } from 'vue'++// Sequence diagram for the at-rest encryption write and read paths.+// Lanes are fixed; `mode` picks the message list.+const props = defineProps({+ mode: { type: String, default: 'write' }+})++const W = 600+const LANES = [+ { key: 'client', x: 64, w: 108, name: 'Client' },+ { key: 'wrapper', x: 238, w: 164, name: 'Encryption wrapper', sub: 'RAGFS' },+ { key: 'key', x: 404, w: 132, name: 'Account key', sub: 'from root key' },+ { key: 'backend', x: 534, w: 108, name: 'Backend', sub: 'local / S3' }+]+const laneX = Object.fromEntries(LANES.map((l) => [l.key, l.x]))++const FLOWS = {+ write: {+ title: 'Encrypted write path',+ desc:+ 'The client writes plaintext. The RAGFS encryption wrapper derives the account key, generates a random file key and nonces, encrypts the content with AES-256-GCM, wraps the file key with the account key, and writes the OVE1 envelope to the backend.',+ items: [+ { type: 'msg', from: 'client', to: 'wrapper', label: 'write(uri, plaintext)', y: 86 },+ { type: 'msg', from: 'wrapper', to: 'key', label: 'derive (HKDF, cached)', y: 120 },+ { type: 'msg', from: 'key', to: 'wrapper', label: 'account key', y: 150, dashed: true },+ {+ type: 'note',+ lane: 'wrapper',+ y: 172,+ lines: [+ '1. random File Key + nonces',+ '2. AES-256-GCM encrypt content',+ '3. wrap File Key with account key',+ '4. build OVE1 envelope'+ ]+ },+ { type: 'msg', from: 'wrapper', to: 'backend', label: 'write envelope (ciphertext)', y: 290 },+ { type: 'msg', from: 'backend', to: 'wrapper', label: 'stored', y: 320, dashed: true },+ { type: 'msg', from: 'wrapper', to: 'client', label: 'ok', y: 350, dashed: true }+ ],+ height: 372+ },+ read: {+ title: 'Encrypted read path',+ desc:+ 'The client reads a URI. The wrapper fetches the stored bytes; files without the OVE1 magic are returned as plaintext. Otherwise it derives the account key, unwraps the file key, authenticates and decrypts the content, and returns plaintext or an error.',+ items: [+ { type: 'msg', from: 'client', to: 'wrapper', label: 'read(uri)', y: 86 },+ { type: 'msg', from: 'wrapper', to: 'backend', label: 'read bytes', y: 116 },+ { type: 'msg', from: 'backend', to: 'wrapper', label: 'stored bytes', y: 146, dashed: true },+ { type: 'note', lane: 'wrapper', y: 166, lines: ['no OVE1 magic → return as plaintext'] },+ { type: 'msg', from: 'wrapper', to: 'key', label: 'derive (HKDF, cached)', y: 226 },+ { type: 'msg', from: 'key', to: 'wrapper', label: 'account key', y: 256, dashed: true },+ {+ type: 'note',+ lane: 'wrapper',+ y: 278,+ lines: [+ '1. unwrap File Key with account key',+ '2. authenticate + decrypt (AES-GCM)',+ 'wrong key or bad tag → error'+ ]+ },+ { type: 'msg', from: 'wrapper', to: 'client', label: 'plaintext', y: 378, dashed: true }+ ],+ height: 400+ }+}++const id = useId()+const arrowId = `enc-arrow-${id}`+const flow = computed(() => FLOWS[props.mode] ?? FLOWS.write)++const NOTE_W = 250+const NOTE_LINE = 18+function noteHeight(n) {+ return n.lines.length * NOTE_LINE + 12+}+function msgPath(m) {+ const x1 = laneX[m.from]+ const x2 = laneX[m.to]+ const dir = x2 > x1 ? -1 : 1+ return `M${x1} ${m.y}H${x2 + dir * 2}`+}+</script>++<template>+ <figure class="enc-diagram">+ <svg :viewBox="`0 0 ${W} ${flow.height}`" role="img" :aria-labelledby="`${id}-t ${id}-d`">+ <title :id="`${id}-t`">{{ flow.title }}</title>+ <desc :id="`${id}-d`">{{ flow.desc }}</desc>+ <defs>+ <marker :id="arrowId" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">+ <path class="arrow-head" d="M1 1.5 9 5 1 8.5z" />+ </marker>+ </defs>++ <g v-for="lane in LANES" :key="lane.key">+ <line class="lifeline" :x1="lane.x" :x2="lane.x" y1="56" :y2="flow.height - 8" />+ <rect class="box" :x="lane.x - lane.w / 2" y="8" :width="lane.w" height="48" rx="8" />+ <text class="name" :x="lane.x" :y="lane.sub ? 25 : 32">{{ lane.name }}</text>+ <text v-if="lane.sub" class="sub" :x="lane.x" y="43">{{ lane.sub }}</text>+ </g>++ <template v-for="(item, i) in flow.items" :key="i">+ <g v-if="item.type === 'msg'">+ <path class="msg" :class="{ dashed: item.dashed }" :d="msgPath(item)" :marker-end="`url(#${arrowId})`" />+ <text class="label" :x="(laneX[item.from] + laneX[item.to]) / 2" :y="item.y - 8">{{ item.label }}</text>+ </g>+ <g v-else>+ <rect+ class="note"+ :x="laneX[item.lane] - NOTE_W / 2"+ :y="item.y"+ :width="NOTE_W"+ :height="noteHeight(item)"+ rx="6"+ />+ <text+ v-for="(line, j) in item.lines"+ :key="j"+ class="note-text"+ :x="laneX[item.lane]"+ :y="item.y + 6 + j * NOTE_LINE + NOTE_LINE / 2"+ >{{ line }}</text>+ </g>+ </template>+ </svg>+ </figure>+</template>++<style scoped>+.enc-diagram {+ max-width: 640px;+ margin: 16px auto;+ overflow-x: auto;+}++svg {+ display: block;+ width: 100%;+ min-width: 540px;+ height: auto;+ font-family: inherit;+}++text {+ text-anchor: middle;+ dominant-baseline: middle;+}++.name {+ font-size: 14px;+ font-weight: 600;+ fill: var(--vp-c-text-1);+}++.sub {+ font-size: 12px;+ fill: var(--vp-c-text-2);+}++.label {+ font-size: 12.5px;+ fill: var(--vp-c-text-1);+}++.note-text {+ font-size: 12.5px;+ fill: var(--vp-c-text-1);+}++.box {+ fill: var(--vp-c-bg-soft);+ stroke: var(--vp-c-divider);+ stroke-width: 1;+}++.note {+ fill: var(--vp-c-brand-soft);+ stroke: var(--vp-c-brand-1);+ stroke-width: 1;+}++.lifeline {+ stroke: var(--vp-c-divider);+ stroke-width: 1;+ stroke-dasharray: 4 4;+}++.msg {+ fill: none;+ stroke: var(--vp-c-text-2);+ stroke-width: 1.25;+}++.msg.dashed {+ stroke-dasharray: 5 4;+}++.arrow-head {+ fill: var(--vp-c-text-2);+}+</style>diff --git a/docs/.vitepress/theme/components/FlowDiagram.vue b/docs/.vitepress/theme/components/FlowDiagram.vuenew file mode 100644index 000000000..4c556aba3--- /dev/null+++ b/docs/.vitepress/theme/components/FlowDiagram.vue@@ -0,0 +1,135 @@+<script setup>+import { computed, useId } from 'vue'++// Vertical flow: one box per step, arrows between them. `notes` lines are+// wrapped by the caller so widths stay predictable at phone size; `edge`+// labels the arrow to the next step.+const props = defineProps({+ title: { type: String, required: true },+ desc: { type: String, required: true },+ steps: { type: Array, required: true },+ // index of the step drawn as the core node (brand border); -1 for none+ core: { type: Number, default: -1 }+})++const W = 400+const BOX_X = 20+const BOX_W = 360+const PAD = 8+const GAP = 30+const NAME_H = 22+const NOTE_H = 18++const id = useId()+const arrowId = `flow-arrow-${id}`++const layout = computed(() => {+ let y = PAD+ const boxes = props.steps.map((step, i) => {+ const notes = step.notes ?? []+ const h = 14 + NAME_H + notes.length * NOTE_H+ const gap = step.edge ? GAP + 10 : GAP+ const box = { ...step, notes, i, y, h, gap }+ y += h + gap+ return box+ })+ const last = boxes[boxes.length - 1]+ return { boxes, height: last.y + last.h + PAD }+})+</script>++<template>+ <figure class="flow-diagram">+ <svg :viewBox="`0 0 ${W} ${layout.height}`" role="img" :aria-labelledby="`${id}-t ${id}-d`">+ <title :id="`${id}-t`">{{ title }}</title>+ <desc :id="`${id}-d`">{{ desc }}</desc>+ <defs>+ <marker :id="arrowId" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">+ <path class="arrow-head" d="M1 1.5 9 5 1 8.5z" />+ </marker>+ </defs>++ <g v-for="box in layout.boxes" :key="box.i">+ <path+ v-if="box.i < layout.boxes.length - 1"+ class="arrow"+ :d="`M${W / 2} ${box.y + box.h}v${box.gap - 2}`"+ :marker-end="`url(#${arrowId})`"+ />+ <text+ v-if="box.edge && box.i < layout.boxes.length - 1"+ class="edge"+ :x="W / 2 + 10"+ :y="box.y + box.h + box.gap / 2"+ >{{ box.edge }}</text>+ <rect class="box" :class="{ core: box.i === core }" :x="BOX_X" :y="box.y" :width="BOX_W" :height="box.h" rx="8" />+ <text class="name" :x="W / 2" :y="box.y + 7 + NAME_H / 2">{{ box.name }}</text>+ <text+ v-for="(line, j) in box.notes"+ :key="j"+ class="note"+ :x="W / 2"+ :y="box.y + 7 + NAME_H + j * NOTE_H + NOTE_H / 2"+ >{{ line }}</text>+ </g>+ </svg>+ </figure>+</template>++<style scoped>+.flow-diagram {+ max-width: 460px;+ margin: 16px auto;+}++svg {+ display: block;+ width: 100%;+ height: auto;+ font-family: inherit;+}++text {+ text-anchor: middle;+ dominant-baseline: middle;+}++.name {+ font-size: 15.5px;+ font-weight: 600;+ fill: var(--vp-c-text-1);+}++.note {+ font-size: 13.5px;+ fill: var(--vp-c-text-2);+}++.edge {+ font-size: 13px;+ text-anchor: start;+ fill: var(--vp-c-text-2);+}++.box {+ fill: var(--vp-c-bg-soft);+ stroke: var(--vp-c-divider);+ stroke-width: 1;+}++.box.core {+ fill: var(--vp-c-brand-soft);+ stroke: var(--vp-c-brand-1);+ stroke-width: 1.5;+}++.arrow {+ fill: none;+ stroke: var(--vp-c-text-2);+ stroke-width: 1.25;+}++.arrow-head {+ fill: var(--vp-c-text-2);+}+</style>diff --git a/docs/.vitepress/theme/components/KeyHierarchyDiagram.vue b/docs/.vitepress/theme/components/KeyHierarchyDiagram.vuenew file mode 100644index 000000000..492668b20--- /dev/null+++ b/docs/.vitepress/theme/components/KeyHierarchyDiagram.vue@@ -0,0 +1,18 @@+<script setup>+import FlowDiagram from './FlowDiagram.vue'++const steps = [+ { name: 'Root Key', notes: ['one per OpenViking instance', 'local file, or KMS / Vault-wrapped'], edge: 'HKDF-SHA256 per account' },+ { name: 'Account Key (KEK)', notes: ['one per account, derived at runtime,', 'never stored'], edge: 'AES-256-GCM key wrap' },+ { name: 'File Key (DEK)', notes: ['random per write, stored wrapped', 'in the file envelope; encrypts content'] }+]+</script>++<template>+ <FlowDiagram+ title="Three-layer key hierarchy"+ desc="The root key, unique per instance and stored locally or wrapped by KMS or Vault, derives one account key per account with HKDF-SHA256. The account key wraps a random per-write file key with AES-256-GCM; the wrapped file key is stored in the file envelope and encrypts the content."+ :steps="steps"+ :core="0"+ />+</template>diff --git a/docs/en/concepts/10-encryption.md b/docs/en/concepts/10-encryption.mdindex 6b49a3388..0aceae26a 100644--- a/docs/en/concepts/10-encryption.md+++ b/docs/en/concepts/10-encryption.md@@ -1,53 +1,35 @@# Data Encryption-OpenViking provides transparent at-rest data encryption to ensure data security and isolation in multi-tenant environments.+<script setup>+import EncryptionFlowDiagram from '../../.vitepress/theme/components/EncryptionFlowDiagram.vue'+import KeyHierarchyDiagram from '../../.vitepress/theme/components/KeyHierarchyDiagram.vue'+</script>++OpenViking supports at-rest encryption: it encrypts files before storage and decrypts them for authorized reads. Each account uses a separate account key.## Overview### Why Encryption-In a multi-tenant architecture, resources, memories, and skills from different customers (accounts) are stored in a shared AGFS instance. Encryption ensures:+Multiple accounts can share an AGFS instance. When encryption is enabled:-- Even if an attacker gains AGFS disk access, they cannot read any customer's plaintext data+- Encrypted files require the corresponding keys to decrypt; protect keys separately from data- Different accounts' data is encrypted with independent keys for tenant isolation-- All encryption/decryption operations are centralized at the VikingFS layer; AGFS and external object stores only see ciphertext+- The RAGFS encryption wrapper handles file reads and writes at runtime; coverage depends on each backend configuration### Transparency-Encryption is completely transparent to users and developers:+Enabling encryption preserves the client API:- **No client API changes**: Existing code works without modification-- **Application layer unaware**: Read/write operations behave exactly like unencrypted-- **Backward compatible**: Unencrypted old files can still be read normally+- **Plaintext responses**: Authorized reads still return decrypted content; unavailable keys or failed ciphertext authentication cause errors+- **Compatible with existing files**: Old plaintext files remain readable; enabling encryption does not encrypt them automatically## Three-Layer Key ArchitectureOpenViking uses an Envelope Encryption architecture with a three-layer key system:-```-┌─────────────────────────────────────────────────────────┐-│ Layer 1: Root Key │-│ • Global unique per OpenViking instance │-│ • Storage: KMS service / ~/.openviking/master.key │-│ • Purpose: Derive all account keys │-└────────────────────┬────────────────────────────────────┘- │ HKDF derivation- ▼-┌─────────────────────────────────────────────────────────┐-│ Layer 2: Account Key (KEK) │-│ • One independent key per account │-│ • Not stored, derived at runtime │-│ • Purpose: Encrypt all file keys for this account │-└────────────────────┬────────────────────────────────────┘- │ AES-256-GCM encryption- ▼-┌─────────────────────────────────────────────────────────┐-│ Layer 3: File Key (DEK) │-│ • New random key generated per write operation │-│ • Stored encrypted in file header (envelope) │-│ • Purpose: Encrypt actual file content │-└─────────────────────────────────────────────────────────┘-```+<KeyHierarchyDiagram />### Key Hierarchy Summary@@ -64,8 +46,8 @@ OpenViking supports three key providers for different deployment scenarios:| Provider | Use Case | Root Key Storage | Features ||----------|----------|-----------------|----------|| **Local** | Dev environments, single-node deployments | Local file `~/.openviking/master.key` | Simple, no external services |-| **Vault** | Production, multi-cloud | HashiCorp Vault Transit Engine | Enterprise-grade key management, version control |-| **Volcengine KMS** | Volcengine cloud deployments | Volcengine KMS | Cloud-native KMS service |+| **Vault** | Production, multi-cloud | Transit-encrypted value stored in Vault KV | Vault protects the root key |+| **Volcengine KMS** | Volcengine cloud deployments | KMS-encrypted value stored in local `key_file` | KMS protects the root key |### Local (File)@@ -133,81 +115,49 @@ Suitable for Volcengine cloud deployments:## How It Works+Startup resolves the root key, then RAGFS `EncryptionWrappedFS` handles file content for protected backends. Python key providers load or unwrap the root key; individual file operations do not call KMS to derive account keys.+### Write Flow-```-Client VikingFS FileEncryptor KeyManager AGFS- │ │ │ │ │- │ write(uri, data) │ │ │ │- │──────────────────>│ │ │ │- │ │ encrypt(account_id, │ │ │- │ │ plaintext) │ │ │- │ │──────────────────────>│ │ │- │ │ │ derive_account_key()│ │- │ │ │────────────────────>│ │- │ │ │<────────────────────│ │- │ │ │ account_key │ │- │ │ 1. Generate random File Key │- │ │ 2. Encrypt content with File Key │- │ │ 3. Encrypt File Key with Account Key │- │ │ 4. Build envelope format │- │ │<──────────────────────│ │ │- │ │ ciphertext │ │ │- │ │──────────────────────────────────────────────────────────>│- │ │ │ │ Write │- │<──────────────────│ │ │ │- │ success │ │ │ │-```+<EncryptionFlowDiagram mode="write" />++1. Derive the account key from the root key and `account_id` with HKDF-SHA256; it may be cached at runtime.+2. Generate a random File Key and nonces for this encryption.+3. Encrypt content with the File Key using AES-256-GCM, then wrap the File Key with the account key.+4. Persist the envelope header, wrapped File Key, nonces, and content ciphertext.### Read Flow-```-Client VikingFS FileEncryptor KeyManager AGFS- │ │ │ │ │- │ read(uri) │ │ │ │- │──────────────────>│ │ │ │- │ │──────────────────────────────────────────────────────────>│- │ │ │ │ Read │- │ │<──────────────────────────────────────────────────────────│- │ │ raw_bytes │ │ │- │ │ Check magic == "OVE1"?│ │ │- │ │ Yes → decrypt() │ │ │- │ │──────────────────────>│ │ │- │ │ │ derive_account_key()│ │- │ │ │────────────────────>│ │- │ │ │<────────────────────│ │- │ │ │ account_key │ │- │ │ 1. Parse envelope format │- │ │ 2. Decrypt File Key with Account Key │- │ │ 3. Decrypt content with File Key │- │ │<──────────────────────│ │ │- │ │ plaintext │ │ │- │<──────────────────│ │ │ │- │ content │ │ │ │-```+<EncryptionFlowDiagram mode="read" />++For encrypted files, the account key unwraps the File Key, which authenticates and decrypts the content. Old files without `OVE1` are read as plaintext. An incorrect key or failed ciphertext authentication returns an error, not decrypted content.### Envelope FormatEncrypted files use a unified envelope format starting with the magic number `OVE1` (OpenViking Encryption v1):-```-┌─────────────────────────────────────────────────────────────┐-│ Magic │ Version │ Provider │ Encrypted File Key │ ... │-│ 4 bytes │ 1 byte │ 1 byte │ Variable length │ ... │-│ "OVE1" │ 0x01 │ 0x01=local│ │ ... │-└─────────────────────────────────────────────────────────────┘-```+| Field | Size | Value |+| --- | --- | --- |+| Magic | 4 bytes | `OVE1` |+| Version | 1 byte | `0x01` |+| Provider | 1 byte | `0x01` = local |+| Encrypted File Key | variable | File Key wrapped with the account key |+| … | | nonces and content ciphertext |- If a file doesn't start with `OVE1`, it's treated as unencrypted and plaintext is returned directly-- Backward compatible, old files don't need migration+- Old files remain readable; protecting existing plaintext requires a separate migration or rewrite## Multi-Tenant IsolationDifferent accounts' data is encrypted with independent Account Keys:- Account A's key cannot decrypt Account B's files-- Even with full AGFS access, data can't be read without the corresponding key-- Tenant isolation is implemented at the key layer, not relying on storage permissions+- For files already encrypted, obtaining backend ciphertext still requires the corresponding key to decrypt it+- Separate keys supplement tenant access controls; they do not replace authentication or storage permissions++## Encryption Coverage++This protects file backends where encryption is enabled. It does not automatically encrypt a separate vector database, logs, or exported OVPack files; vector records can contain abstracts and memory text. Multi-write backups can disable encryption individually, so check each backend configuration.## Configuration Examplediff --git a/docs/en/concepts/14-multi-write-storage.md b/docs/en/concepts/14-multi-write-storage.mdindex 231acd0cb..b5ffd735b 100644--- a/docs/en/concepts/14-multi-write-storage.md+++ b/docs/en/concepts/14-multi-write-storage.md@@ -1,6 +1,6 @@# Multi-Write Storage-Multi-write storage lets OpenViking use one primary storage backend together with multiple backup backends under a unified filesystem abstraction. It is suitable for high availability, cross-region replicas, read acceleration, and storage migration.+Multi-write storage lets OpenViking use one primary storage backend together with multiple backup backends under a unified filesystem abstraction. It supports replicas (including cross-region replicas), configurable read routing, and storage migration; it does not automatically promote a backup to primary.From the API user's point of view, interfaces such as `read()`, `write()`, `ls()`, and `stat()` do not change. Multi-write logic lives inside RAGFS, so callers do not need to care which underlying backend ultimately stores a file.@@ -38,9 +38,9 @@ Multi-write supports two consistency modes.| Async multi-write | `async` | Return as soon as the primary write succeeds; backups sync in the background | Low-latency writes, eventual consistency || Sync multi-write | `sync` | Wait for backup acknowledgements after the primary write succeeds | Stronger write confirmation when extra latency is acceptable |-In async mode, backups may lag behind the primary for a short time. In sync mode, `write_ack_count` and `write_ack_timeout_ms` control how many backup acknowledgements are required and how long the system waits.+In async mode, backups may lag behind the primary; recovery depends on backend availability and retry outcomes. In sync mode, `write_ack_count` and `write_ack_timeout_ms` control how many backup acknowledgements are required and how long the system waits.-Even in sync mode, backups that timed out or did not confirm are still retried in the background.+If sync mode fails to reach the required acknowledgement count, the call returns an error even though the primary may already have committed; that write is not automatically rolled back. Backups that did not acknowledge are still retried in the background; to inspect sync state, see the [Multi-Write Storage Guide](../guides/13-multi-write-storage.md).## Read Path@@ -113,14 +113,14 @@ Multi-write only handles new writes after it is enabled. It does not automaticalRecommended migration flow:-1. Use OVPack or another controlled process for a full data migration.+1. Pause writes or arrange a verifiable incremental catch-up window, then migrate existing data with OVPack or another controlled process.2. Validate the target backend data.3. Enable the multi-write configuration.-4. Let future new writes and updates continue to replicate through multi-write.+4. Catch up changes made during migration and verify sync state before resuming writes; multi-write replicates subsequent additions and updates.## Limitations-- In async mode, backups may lag temporarily.+- In async mode, backups may lag; reads from them cannot assume the latest write is visible.- Historical files that existed before multi-write was enabled need a separate migration or backfill process.- Redirected files rely on internal metadata to reconstruct the directory view.- Concurrent writes from multiple processes to the same primary backend still need future distributed metadata locking.diff --git a/docs/en/guides/08-encryption.md b/docs/en/guides/08-encryption.mdindex 885ea3cf7..947f93a21 100644--- a/docs/en/guides/08-encryption.md+++ b/docs/en/guides/08-encryption.md@@ -4,12 +4,12 @@ This guide describes how to enable and use at-rest data encryption in OpenViking## Overview-OpenViking provides transparent at-rest data encryption to ensure data security and isolation in multi-tenant environments:+OpenViking can encrypt newly written files with account-specific keys. The storage layer handles encryption and decryption through the existing APIs:-- ✅ **Transparent encryption**: No API changes, application layer unaware-- ✅ **Multi-tenant isolation**: Different accounts use independent keys-- ✅ **Three key providers**: Local, Vault, Volcengine KMS-- ✅ **Backward compatible**: Unencrypted old files still readable+- **Transparent encryption**: No API changes, application layer unaware+- **Multi-tenant isolation**: Different accounts use independent keys+- **Three key providers**: Local, Vault, Volcengine KMS+- **Backward compatible**: Unencrypted old files still readableSee [Data Encryption](../concepts/10-encryption.md) for conceptual explanations.@@ -36,7 +36,7 @@ ov system crypto init-key --output-file ~/.openviking/master.key### 2. Configure Encryption-Edit `~/.openviking/ov.conf`:+Merge the following encryption settings into `~/.openviking/ov.conf`; retain your model and server settings:```json{@@ -90,23 +90,23 @@ It waits for import processing and checks retrieval; successful retrieval alone## API Key Hashing Configuration-OpenViking provides two layers of encryption protection:+File encryption and API key hashing are independent controls:| Encryption Layer | Config | Algorithm | Reversible | Description ||------------------|--------|-----------|------------|-------------|-| **File Layer** | `encryption.enabled` | AES-GCM | ✅ Yes | Protects entire storage files |-| **API Key Field Layer** | `encryption.api_key_hashing.enabled` | Argon2id | ❌ No | Protects API keys themselves |+| **File Layer** | `encryption.enabled` | AES-GCM | Yes | Protects entire storage files |+| **API Key Field Layer** | `encryption.api_key_hashing.enabled` | Argon2id | No | Protects API keys themselves |-### ⚠️ Breaking Change Notice+### Upgrading from implicit API key hashing-**Version Change**: OpenViking v0.3.12 → later versions+Applies when upgrading from v0.3.12 or earlier to v0.3.13 or later.**Behavior Change**:- **Before**: `encryption.enabled = true` implicitly enabled API key Argon2id hashing- **Now**: You must explicitly configure `encryption.api_key_hashing.enabled`**Impact**:-- After upgrade, if `encryption.enabled = true` but `encryption.api_key_hashing.enabled` is not explicitly set to `true`, you will see the following warning log on startup:+- After upgrade, if `encryption.enabled = true` but `encryption.api_key_hashing.enabled` is not explicitly set to `true`, you will see the following informational log excerpt on startup:```API key hashing is disabled while file encryption is enabled.Previously, encryption.enabled=true implicitly enabled API key Argon2id hashing.@@ -119,7 +119,7 @@ OpenViking provides two layers of encryption protection:| Option | Config | Behavior ||--------|--------|----------|| **Maintain Previous Behavior** | `api_key_hashing.enabled = true` | API keys stored using Argon2id hashing |-| **Recommended New Behavior** | `api_key_hashing.enabled = false` (default) | API keys stored in plaintext (file layer still encrypted) |+| **Recoverable key storage (default)** | `api_key_hashing.enabled = false` (default) | API keys stored in plaintext (file layer still encrypted) |### Default Behavior@@ -130,7 +130,7 @@ OpenViking provides two layers of encryption protection:### Enabling Argon2id Hashing-For maximum API key protection, you can enable Argon2id one-way hashing:+Enable Argon2id one-way hashing to stop storing recoverable API key values:```json{@@ -170,7 +170,7 @@ For maximum API key protection, you can enable Argon2id one-way hashing:| Provider | Use Case | Pros | Cons ||----------|----------|------|------|-| **Local** | Dev environments, single-node | Simple, no external services | Key stored locally, less secure |+| **Local** | Dev environments, single-node | Simple, no external services | Requires local file permissions and separate key backups || **Vault** | Production, multi-cloud | Enterprise-grade KMS, version control | Requires deploying and maintaining Vault || **Volcengine KMS** | Volcengine cloud | Cloud-native KMS service | Volcengine-only |@@ -221,7 +221,7 @@ ov system crypto init-key -f ~/.openviking/master.key### Prerequisites-1. HashiCorp Vault service deployed+1. HashiCorp Vault service deployed; install the Python `hvac` dependency in the server environment2. Transit engine enabled3. Vault Token with sufficient permissions@@ -237,10 +237,16 @@ vault secrets enable transit```bash# KV v2 (recommended)-vault secrets enable -version=2 kv+vault secrets enable -path=secret -version=2 kv# Or KV v1-vault secrets enable kv+vault secrets enable -path=secret -version=1 kv+```++Use one KV command matching your deployment, not both. The example below uses the `secret` mount with KV v2. Pre-create the Transit key with an administrative token:++```bash+vault write -f transit/keys/openviking-root-key type=aes256-gcm96```3. Configure OpenViking:@@ -255,7 +261,7 @@ vault secrets enable kv"token": "hvs.xxxxxxxxxxxxxxxxxxxxx","mount_point": "transit","kv_mount_point": "secret",- "kv_version": 1,+ "kv_version": 2,"root_key_name": "openviking-root-key","encrypted_root_key_key": "openviking-encrypted-root-key"}@@ -277,18 +283,32 @@ vault secrets enable kv### Vault Permission Recommendations-Configure minimal permissions for the Token:+For the configuration above, the service token needs Transit key metadata access, encrypt/decrypt access, and KV access to store the wrapped root key. With the engines and Transit key pre-created, a policy example is:```hcl-path "transit/encrypt/openviking-root" {+path "auth/token/lookup-self" {+ capabilities = ["read"]+}++path "transit/keys/openviking-root-key" {+ capabilities = ["read"]+}++path "secret/data/openviking-encrypted-root-key" {+ capabilities = ["read", "create", "update"]+}++path "transit/encrypt/openviking-root-key" {capabilities = ["update"]}-path "transit/decrypt/openviking-root" {+path "transit/decrypt/openviking-root-key" {capabilities = ["update"]}```+For KV v1, replace `secret/data/openviking-encrypted-root-key` with `secret/openviking-encrypted-root-key`. The provider also probes `sys/mounts`; denial is logged as a warning and does not stop startup when Transit is already enabled.+---## Volcengine KMS Mode Detailed Guide@@ -363,7 +383,7 @@ Configure minimal permissions for the Access Key:### Method 1: Check File Content-Encrypted files start with magic number `OVE1`:+Inspect the physical backend file, not the API response, which is decrypted. Substitute its actual path below. Encrypted files start with magic number `OVE1`:```bash# View first 4 bytes@@ -381,21 +401,11 @@ hexdump -C ./data/agfs/your-file | head -100000000 7b 22 63 6f 6e 74 65 6e 74 73 22 3a 5b 7b 22 70 |{"contents":[{"p|```-### Method 2: Cross-Provider Verification+### Method 2: Restart and Read Back-Try decrypting with different providers — it should fail (this is normal security behavior):+Read the same imported file through the API before and after restarting the server. Verify its content matches and inspect the physical backend file header as above. This checks that the deployment reloads the expected key and can read its ciphertext.-```python-# Encrypt with Provider A-encrypted = await provider_a.encrypt_file_key(plaintext, "test-account")--# Try decrypting with Provider B (should fail)-try:- await provider_b.decrypt_file_key(encrypted, "test-account")- print("❌ Security vulnerability: Cross-provider decryption succeeded!")-except Exception as e:- print("✓ Secure: Cross-provider decryption failed as expected")-```+Do not treat an arbitrary exception in an internal provider call as an encryption test: argument errors, missing dependencies, and network errors can all fail before decryption.---@@ -426,12 +436,7 @@ See [OVPack Import and Export](09-ovpack.md#full-backup-and-restore) for support### Switching Key Providers-1. Back up existing data and keys-2. Decrypt all data with old provider-3. Configure new provider-4. Re-encrypt all data--**Note**: This is a destructive operation, recommend testing first in a staging environment.+Keep the original provider and key available while exporting a logical OVPack backup. Restore it into a separate empty deployment configured with the new provider, following the migration steps above. Verify contents, access permissions, and rebuilt indexes before switching traffic. Changing the provider setting in place does not re-encrypt existing files.---@@ -471,19 +476,11 @@ Error: Invalid credentials2. Confirm key has sufficient permissions3. Verify region configuration is correct-### Cross-Provider Decryption Failed (This is Normal)--```-Error: KeyMismatchError-```--**Explanation**: This is expected security behavior. Different providers use different root keys and cannot decrypt each other's data.+### Existing ciphertext cannot be read-### Partial Read Returns Ciphertext+Check that the deployment still has the original root key, provider settings, account identity, and intact ciphertext. For Vault, retain both the Transit key and the KV entry containing the wrapped root key; for KMS, retain the KMS key and local wrapped-key file. Do not generate a replacement root key to repair access to existing ciphertext.-If using encrypted files created with an older OpenViking version, partial reads may return ciphertext.--**Solution**: Upgrade to the latest OpenViking version.+Partial reads load and authenticate the full encrypted file, then return the requested plaintext slice. If a partial read returns ciphertext, record the server version and compare full and partial reads on a copied test file. Check the storage encryption configuration and relevant fixes before upgrading, and keep the original data and keys available for rollback.---@@ -492,3 +489,5 @@ If using encrypted files created with an older OpenViking version, partial reads- [Data Encryption](../concepts/10-encryption.md) - Encryption concepts- [Configuration Guide](./01-configuration.md) - Complete configuration reference- [Multi-Tenant](../concepts/11-multi-tenant.md) - Account, user, and agent isolation model++Vault references: [KV v2](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2), [Transit API](https://developer.hashicorp.com/vault/api-docs/secret/transit).diff --git a/docs/en/guides/09-ovpack.md b/docs/en/guides/09-ovpack.mdindex 3502ed261..a7fae25d4 100644--- a/docs/en/guides/09-ovpack.md+++ b/docs/en/guides/09-ovpack.md@@ -1,7 +1,7 @@# OVPack Import and ExportOVPack is OpenViking's recoverable content package format for migrating or-backing up public content trees under `viking://`. It stores file content,+backing up supported content trees under `viking://`. It stores file content,semantic sidecar files, portable index scalar fields, and optional dense vectorsnapshots.@@ -20,14 +20,14 @@ Regular `export/import` handles one package root:- `viking://user/...`Full migration uses the separate `backup/restore` flow. It packages public-scope roots together:+scope roots together (these are namespace roots, not a statement that their content is publicly readable):- `viking://resources`- all `viking://user/{user_id}` content in the current accountOnly ROOT or ADMIN can call `backup/restore`, and these operations traverse alluser content within the current account. Backup packages do not contain user-accounts, API keys, or other authentication data.+accounts or the server API-key registry. User files, including privacy configuration containing external credentials, are still part of the content tree; treat the archive accordingly.Sessions are included through the user namespace at`viking://user/{user_id}/sessions/{session_id}`. The@@ -77,7 +77,7 @@ root is `my-project`, the imported URI is:viking://resources/imported/my-project```-Overwrite an existing root:+To replace an existing root, first back up that target and confirm its entire subtree may be removed:```bashov import ./exports/my-project.ovpack viking://resources/imported/ --on-conflict overwrite@@ -163,7 +163,6 @@ Do not use `export viking://` for full migration. Use a backup package:```bashov backup ./backups/openviking.ovpack-ov restore ./backups/openviking.ovpack --on-conflict overwrite```Backup packages can only be restored with `restore`; regular `import` rejects@@ -204,6 +203,8 @@ After restoring content, register the remaining users with the same `user_id` va## Python SDK+Use source credentials for export/backup and a separately configured target client for import/restore across deployments. The full restore examples below assume you have completed the target-account and overwrite checks above. Queue completion alone does not prove a successful index rebuild; inspect errors and run a consistency check.+```pythonfrom openviking_sdk import AsyncHTTPClient@@ -221,11 +222,12 @@ async def migrate_project():imported_uri = await client.import_ovpack(file_path="./exports/my-project.ovpack",parent="viking://resources/imported/",- on_conflict="overwrite",+ on_conflict="fail",vector_mode="auto",)print(imported_uri)- await client.wait_processed()+ print(await client.wait_processed(timeout=120))+ print(await client.check_consistency(uri=imported_uri))finally:await client.close()```@@ -262,7 +264,7 @@ importedURI, err := client.ImportOVPack(outPath,"viking://resources/imported/",&openviking.ImportPackOptions{- OnConflict: "overwrite",+ OnConflict: "fail",VectorMode: "auto",},)@@ -306,7 +308,7 @@ the local `.ovpack`, then call the pack endpoint with `temp_file_id`.Export:```bash-curl -X POST http://localhost:1933/api/v1/pack/export \+curl -f -X POST http://localhost:1933/api/v1/pack/export \-H "Content-Type: application/json" \-H "X-API-Key: your-admin-key" \-d '{"uri":"viking://resources/my-project","include_vectors":false}' \@@ -317,19 +319,19 @@ Import:```bashTEMP_FILE_ID=$(- curl -sS -X POST http://localhost:1933/api/v1/resources/temp_upload \+ curl -fsS -X POST http://localhost:1933/api/v1/resources/temp_upload \-H "X-API-Key: your-admin-key" \-F "file=@./exports/my-project.ovpack" \- | jq -r ".result.temp_file_id"+ | jq -er ".result.temp_file_id")-curl -X POST http://localhost:1933/api/v1/pack/import \+curl -f -X POST http://localhost:1933/api/v1/pack/import \-H "Content-Type: application/json" \-H "X-API-Key: your-admin-key" \-d "{\"temp_file_id\": \"$TEMP_FILE_ID\",\"parent\": \"viking://resources/imported/\",- \"on_conflict\": \"overwrite\",+ \"on_conflict\": \"fail\",\"vector_mode\": \"auto\"}"```@@ -337,7 +339,7 @@ curl -X POST http://localhost:1933/api/v1/pack/import \Full backup:```bash-curl -X POST http://localhost:1933/api/v1/pack/backup \+curl -f -X POST http://localhost:1933/api/v1/pack/backup \-H "Content-Type: application/json" \-H "X-API-Key: your-admin-key" \-d '{"include_vectors":true}' \@@ -388,7 +390,7 @@ my-project/_ovpack/manifest.jsonDotfiles are no longer escaped with `_._`. `_ovpack/` stores OVPack internalmetadata and is not imported as user content.-The manifest stores package structure, file checksums, and checksums for+The following manifest is illustrative (placeholder checksums do not form a valid package). It stores package structure, file checksums, and checksums forinternal index files. It does not inline per-file index records:```json@@ -597,7 +599,7 @@ re-export from an environment that can read that version.| `source path is incompatible with target path` | Structured scope root depth would change | Import into the correct system parent directory. || `Top-level scope ovpack packages must be imported to viking://` | A top-level scope package was imported to a non-root parent | Import to `viking://`. || `Backup ovpack packages must be restored` | A backup package was imported with regular import | Use `ov restore`. |-| `Resource already exists` | Target root already exists | Use `--on-conflict overwrite` or `--on-conflict skip`. |+| `Resource already exists` | Target root already exists | Choose a new target parent, use `--on-conflict skip` to keep the existing root, or back it up and use `--on-conflict overwrite` to replace it. || `incomplete OpenViking vector index snapshot` | `--include-vectors` found missing index records in the export range | Run `ov system consistency <uri>` to locate the issue, then wait for processing or reindex. || `dense vector snapshot is incompatible` | Package embedding metadata does not match current config | Use `--vector-mode recompute`, or switch to a compatible config. |@@ -619,6 +621,6 @@ migration and know the environments are compatible.**What if large package imports are slow?**-Default import rebuilds target semantic and vector state. For large migrations,+Import preserves packaged semantic sidecars and enqueues vector recomputation when a compatible dense snapshot is not restored. For large migrations,use `--include-vectors` to reduce recomputation, or split content into smallerOVPack files and import them in batches.diff --git a/docs/en/guides/11-oauth.md b/docs/en/guides/11-oauth.mdindex 7a95c0eaf..dd65dfec5 100644--- a/docs/en/guides/11-oauth.md+++ b/docs/en/guides/11-oauth.md@@ -1,15 +1,11 @@# OAuth 2.1 Guide-OpenViking server ships a native OAuth 2.1 implementation. Any client that-needs OAuth — including MCP clients (Claude.ai, Claude Desktop, ChatGPT,-Cursor) and any future first-party browser app — can authorize against the-server directly, without a third-party proxy. The protocol surface (DCR,-authorize, token, metadata) is provided by the official `mcp.server.auth`-SDK and is otherwise standards-compliant OAuth 2.1, so non-MCP OAuth clients-work just as well.+OpenViking provides native OAuth authorization for clients that support its discovery, dynamic registration, and PKCE flow, including MCP clients such as Claude.ai, Claude Desktop, ChatGPT, and Cursor. The protocol endpoints (registration, authorize, token, metadata) come from the official `mcp.server.auth` SDK, so non-MCP OAuth clients can use the same flow. Users authorize access through Studio, and clients receive opaque access and refresh tokens. API Key authentication remains available.## Recommended setup+Configure API Key mode and create the account and user/admin key first, as described in [Authentication](04-authentication.md). OAuth consent needs this registered identity; enabling OAuth alone does not create it. Merge the excerpts below into the existing configuration.+> **Prerequisite**: public HTTPS. OAuth 2.1 (and the MCP SDK) **requires> HTTPS** for any non-localhost issuer. See the> [Public Access Guide](12-public-access.md) for how to set up HTTPS with@@ -34,24 +30,13 @@ work just as well.dialog; then click **Authorize** on the consent card. Claude.ai gets thetoken and the connector is live.-That's the whole production path. The rest of this guide explains why each-piece exists, how to test locally without HTTPS, and how to verify with curl-when something doesn't work.+After connecting, confirm the client can discover tools and read a permitted URI. The sections below cover local testing, token behavior, and troubleshooting.---## Why native OAuth-Some MCP clients only accept OAuth 2.1, not API keys. Until now the only path-was to deploy the community [MCP-Key2OAuth](https://github.com/t0saki/MCP-Key2OAuth)-Cloudflare Worker proxy that translates OAuth into an API-Key bearer. Native-support removes:--- The extra deployment unit (CF Worker + KV namespaces)-- The third-party trust boundary (the proxy operator can decrypt the upstream API key)-- The copy-paste UX where users paste the API key into a browser textbox--API-Key auth still works as before — OAuth layers on top.+Use OAuth when a client expects browser authorization and token refresh. The server handles the flow itself; no proxy sits between the client and the API key. The first Studio sign-in still needs a registered user/admin API key; later approvals can reuse the Studio identity.---@@ -106,6 +91,8 @@ visiting the authorize page.## Quick start (HTTP, local only)+Use the same API Key mode and registered user/admin key described above. Do not use unauthenticated dev mode as the consent identity.+The fastest way to verify OAuth is wired correctly is on `127.0.0.1`. The MCPSDK accepts `http://127.0.0.1` and `http://localhost` as issuer URLs withoutHTTPS — but Claude.ai and Claude Desktop themselves require **public HTTPS**@@ -156,11 +143,10 @@ OAuth 2.1 **requires HTTPS** for any non-localhost issuer. Thenginx, docker compose, CDN — in detail. The short version:1. Follow [Public Access Guide § Adding HTTPS](12-public-access.md#adding-https-for-public-access)- to get `https://your-domain.com` serving port 1934 over TLS.+ to expose `https://your-domain.com` on 443 with the proxy forwarding to the OpenViking port (1933 by default).2. Enable OAuth: `{ "oauth": { "enabled": true } }` in `ov.conf`.-3. Restart: `docker compose restart openviking`.-4. Set `OPENVIKING_PUBLIC_BASE_URL=https://your-domain.com` in `.env` (the- server uses this as the issuer in OAuth metadata and `WWW-Authenticate`).+3. Set `OPENVIKING_PUBLIC_BASE_URL=https://your-domain.com` in the Compose `.env` file.+4. Run `docker compose up -d` to apply changed container environment variables. A plain `restart` does not apply `.env` changes.Once HTTPS + OAuth are both up, connect clients as described below.@@ -200,9 +186,7 @@ Claude.ai's once configured.### ChatGPT (Codex, Plus, Enterprise)-Connector setup is via Settings → Beta features → Custom Connectors. Enter-the MCP URL; ChatGPT discovers the OAuth endpoints from the-`/.well-known/...` documents and walks the same authorize → token flow.+Create a custom App in ChatGPT developer mode, enter the MCP URL, and complete OAuth authorization in the browser. Availability and admin controls depend on the plan and workspace settings; follow the [official OpenAI instructions](https://help.openai.com/en/articles/12584461-developer-mode-and-mcp-apps-in-chatgpt). For Codex plugins and MCP configuration, see the [Codex integration](../agent-integrations/04-codex.md).### Cursor@@ -216,39 +200,53 @@ URL via Cursor's MCP settings.You can drive the entire flow without a real MCP client:+Set `OV_OAUTH_ORIGIN` to your HTTPS origin and `API_KEY` to a registered user/admin key. This example needs `curl`, `jq`, OpenSSL, and Python 3. Copy the pending ID and returned authorization code from the browser where indicated.+```bash-# 1. Register a client-curl -X POST -H "Content-Type: application/json" \- -d '{"redirect_uris":["http://127.0.0.1:9999/cb"],"client_name":"test","token_endpoint_auth_method":"none"}' \- https://my.ov/register-# → {"client_id":"...", ...}--# 2. PKCE pair-VERIFIER=$(openssl rand -base64 64 | tr -d '=+/' | head -c 64)-CHALLENGE=$(printf "%s" "$VERIFIER" | openssl dgst -sha256 -binary | basenc --base64url | tr -d '=')--# 3. Open the authorize URL in a browser. The page shows a 6-char code.-echo "https://my.ov/authorize?response_type=code&client_id=$CID&redirect_uri=http://127.0.0.1:9999/cb&code_challenge=$CHALLENGE&code_challenge_method=S256&state=xyz"--# 4. Approve from the Studio consent page (or via API).-# - Studio path uses pending_id (?pending=... from the authorize URL).-# - Cross-device path uses the 6-char display_code.-curl -X POST -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \- -d '{"pending_id":"<pending-id-from-authorize-url>","decision":"approve"}' \- https://my.ov/api/v1/auth/oauth-verify--# 5. The browser auto-redirects to /cb?code=ovac_...&state=xyz. Copy the code.--# 6. Exchange the auth code for tokens.-curl -X POST \- -d "grant_type=authorization_code&code=ovac_...&client_id=$CID&code_verifier=$VERIFIER&redirect_uri=http://127.0.0.1:9999/cb" \- https://my.ov/token-# → {"access_token":"ovat_...","refresh_token":"ovrt_...","expires_in":3600}--# 7. Call MCP with the access token.-curl -X POST -H "Authorization: Bearer ovat_..." \- -d '{"jsonrpc":"2.0","method":"tools/list","id":1}' \- https://my.ov/mcp+OV_OAUTH_ORIGIN=https://my.ov+CID=$(curl -fsS "$OV_OAUTH_ORIGIN/register" \+ -H "Content-Type: application/json" \+ -d '{"redirect_uris":["http://127.0.0.1:9999/cb"],"client_name":"test","token_endpoint_auth_method":"none"}' \+ | jq -er '.client_id')++VERIFIER=$(python3 -c 'import secrets; print(secrets.token_urlsafe(48))')+CHALLENGE=$(printf "%s" "$VERIFIER" | openssl dgst -sha256 -binary | openssl base64 -A | tr '+/' '-_' | tr -d '=')+STATE=$(python3 -c 'import secrets; print(secrets.token_urlsafe(16))')+printf '%s\n' "$OV_OAUTH_ORIGIN/authorize?response_type=code&client_id=$CID&redirect_uri=http://127.0.0.1:9999/cb&code_challenge=$CHALLENGE&code_challenge_method=S256&state=$STATE"++# Open the URL in a browser; approve in Studio or supply its pending ID here.+curl -fsS "$OV_OAUTH_ORIGIN/api/v1/auth/oauth-verify" \+ -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \+ -d '{"pending_id":"<pending-id>","decision":"approve"}'++# From the redirect URL, verify state matches $STATE and copy the code.+# The callback page need not load for this manual test.+AUTH_CODE='<code-from-callback-url>'+TOKEN_RESPONSE=$(curl -fsS "$OV_OAUTH_ORIGIN/token" \+ --data-urlencode "grant_type=authorization_code" \+ --data-urlencode "code=$AUTH_CODE" \+ --data-urlencode "client_id=$CID" \+ --data-urlencode "code_verifier=$VERIFIER" \+ --data-urlencode "redirect_uri=http://127.0.0.1:9999/cb")+ACCESS_TOKEN=$(printf '%s' "$TOKEN_RESPONSE" | jq -er '.access_token')++curl -fsS "$OV_OAUTH_ORIGIN/mcp" \+ -H "Authorization: Bearer $ACCESS_TOKEN" \+ -H "Content-Type: application/json" \+ -H "Accept: application/json, text/event-stream" \+ -d '{"jsonrpc":"2.0","method":"initialize","id":1,"params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"manual-test","version":"1"}}}'++curl -fsS "$OV_OAUTH_ORIGIN/mcp" \+ -H "Authorization: Bearer $ACCESS_TOKEN" \+ -H "Content-Type: application/json" \+ -H "Accept: application/json, text/event-stream" \+ -d '{"jsonrpc":"2.0","method":"notifications/initialized"}'++curl -fsS "$OV_OAUTH_ORIGIN/mcp" \+ -H "Authorization: Bearer $ACCESS_TOKEN" \+ -H "Content-Type: application/json" \+ -H "Accept: application/json, text/event-stream" \+ -d '{"jsonrpc":"2.0","method":"tools/list","id":2}'```---@@ -288,15 +286,13 @@ Environment variables:| Authorization code | `secrets.token_urlsafe(40)` | `ovac_` | 5 minutes | SQLite (SHA-256 indexed) || Display code (page) | 6-char alphanumeric (no O/0/I/1) | — | 10 minutes | SQLite (`oauth_pending_authorizations`) |-All tokens are opaque; OpenViking does **not** issue JWTs. There is no-cryptographic key to manage on the server side. Token claims are looked up+All tokens are opaque; OpenViking does **not** issue JWTs. OAuth does not require a JWT signing key. The deployment still needs API credentials, protected token storage, and any configured storage-encryption keys. Token claims are looked upfrom SQLite on every request, so revoking a token is a single `UPDATE`.### Token = identityEach issued token is bound to a single `(account_id, user_id, role)` triple-recorded at authorization time. An OAuth token grants the same permissions-as the API key that produced it — *not* more, *not* less.+recorded at authorization time. The token carries that identity and remains subject to current resource ACLs and key validity. A later role promotion does not upgrade an already issued token; role downgrade checks can reject it. OAuth-issued tokens cannot approve new OAuth clients.### OAuth lifetime ≤ authorizing key lifetime@@ -380,3 +376,5 @@ curl -i https://my.ov/mcp -d '{}' -H 'Content-Type: application/json' | grep -i- [RFC 7591 — Dynamic Client Registration](https://datatracker.ietf.org/doc/html/rfc7591)- [RFC 7636 — PKCE](https://datatracker.ietf.org/doc/html/rfc7636)- [OpenViking MCP Integration Guide](06-mcp-integration.md)++[Compose restart behavior](https://docs.docker.com/reference/cli/docker/compose/restart/).diff --git a/docs/en/guides/12-public-access.md b/docs/en/guides/12-public-access.mdindex e14999934..9249c306b 100644--- a/docs/en/guides/12-public-access.md+++ b/docs/en/guides/12-public-access.md@@ -96,14 +96,15 @@ volumes:caddy_config:```+For a proxy-only public entrypoint, remove the OpenViking host port mapping, or bind it to `127.0.0.1`. Also remove the legacy `1934:1934` mapping or bind it to localhost. Caddy can still reach `openviking:1933` through the Compose network.+### 4. Launch```bashdocker compose up -d```-The first HTTPS request triggers ACME certificate issuance. Subsequent-requests use the cached cert. Caddy handles renewal automatically.+Caddy obtains and renews certificates for the configured domain. Check its logs if issuance fails; on-demand issuance is a separate configuration. See [Caddy automatic HTTPS](https://caddyserver.com/docs/automatic-https).### 5. Verify@@ -125,9 +126,12 @@ point the upstream straight at OV's 1933.### nginx+This example assumes nginx and OpenViking share a host. The upstream timeout is an example; size it for your longest request. Disabling response buffering lets streamed MCP responses reach the client promptly; see [nginx proxy buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering).+```nginxserver {- listen 443 ssl http2;+ listen 443 ssl;+ http2 on; # nginx < 1.25.1: remove this line and use `listen 443 ssl http2;`server_name ov.your-domain.com;ssl_certificate /etc/letsencrypt/live/ov.your-domain.com/fullchain.pem;@@ -135,6 +139,9 @@ server {location / {proxy_pass http://127.0.0.1:1933;+ proxy_http_version 1.1;+ proxy_buffering off;+ proxy_read_timeout 300s;proxy_set_header Host $host;proxy_set_header X-Forwarded-Proto $scheme;proxy_set_header X-Forwarded-Host $host;@@ -158,28 +165,28 @@ ov.your-domain.com {### Cloudflare / CDN-Point the CDN origin at `http://your-server-ip:1933`. Set-`OPENVIKING_PUBLIC_BASE_URL=https://ov.your-domain.com` so the server knows-its public address. Make sure the CDN forwards `Host`, `X-Forwarded-Proto`,-and `X-Forwarded-Host`.+Configure an HTTPS origin with certificate validation, or a private origin tunnel. Keep direct origin access restricted to the proxy. Set `OPENVIKING_PUBLIC_BASE_URL=https://ov.your-domain.com`, forward the public host/protocol headers, and check that the proxy preserves streaming and allows your request duration and upload sizes.## Telling the server its public URL-OAuth metadata, `WWW-Authenticate` headers, and resource URLs need to embed-the public origin. Resolution order (**highest to lowest**):+OAuth metadata and `WWW-Authenticate` headers need the public origin. The OAuth origin helper resolves it in this order (**highest to lowest**):1. `OPENVIKING_PUBLIC_BASE_URL` environment variable2. `oauth.issuer` in `ov.conf`3. `X-Forwarded-Proto` + `X-Forwarded-Host` request headers4. The request's `Host` header-Behind any reverse proxy, set option 1 explicitly:+The authorization-server issuer is fixed at startup: environment variable → `oauth.issuer` → `http://127.0.0.1:1933`; forwarded headers do not change it. Protected-resource metadata uses an explicit `oauth.issuer` for `authorization_servers`, even when the environment variable is set.++MCP upload URLs use: environment variable → `server.public_base_url` → forwarded host/protocol → `Host` → listen-address fallback. Setting the environment variable keeps both aligned; if you also set `oauth.issuer`, use the same origin.++Behind a reverse proxy, set option 1 in the server process environment:```bashexport OPENVIKING_PUBLIC_BASE_URL="https://ov.your-domain.com"```-or in `ov.conf`:+For Compose, set it in `.env` and run `docker compose up -d`; an `export` in an unrelated shell does not update an existing container. If configuring the OAuth issuer in `ov.conf` instead:```jsonc{@@ -195,7 +202,7 @@ or in `ov.conf`:`docker compose up` also ships a Caddy reverse proxy on port 1934, simply`reverse_proxy openviking:{$OPENVIKING_SERVER_PORT:1933}` — **kept only for compatibility withdeployments that already bookmarked 1934**. New deployments can connect to-1933 directly; there is no routing value here. Remove the caddy service and+1933 on the private network; public clients should use the HTTPS entrypoint above. Remove the caddy service andthe 1934 port mapping in `docker-compose.yml` if you don't need it.## Relateddiff --git a/docs/en/guides/13-multi-write-storage.md b/docs/en/guides/13-multi-write-storage.mdindex b83c3776e..2c6c8f318 100644--- a/docs/en/guides/13-multi-write-storage.md+++ b/docs/en/guides/13-multi-write-storage.md@@ -1,6 +1,6 @@# Multi-Write Storage Guide-This guide explains how to configure multi-write storage in OpenViking. Multi-write storage lets one primary backend replicate writes to multiple backup backends for high availability, cross-region replicas, read acceleration, and storage migration.+This guide explains how to configure multi-write storage in OpenViking. Multi-write storage lets one primary backend replicate writes to multiple backup backends for replica storage (including cross-region replicas), optional read routing, and migration. It does not automatically promote a backup when the primary fails.Multi-write lives inside RAGFS. The Python SDK, HTTP API, and CLI usage remain unchanged.@@ -13,7 +13,7 @@ Multi-write lives inside RAGFS. The Python SDK, HTTP API, and CLI usage remain u## Minimal Configuration-The following example uses a local directory as the primary backend and replicates writes to another local directory.+Merge these excerpts into your existing model/server configuration. The following example uses a local directory as the primary backend and replicates writes to another local directory.```json{@@ -94,12 +94,12 @@ Recommendations:### S3-Compatible Storage Notes-When using S3-compatible services (MinIO, RustFS, Ceph, etc.), the `s3` section requires additional fields:+Match these `s3` fields to the endpoint and object-store configuration:| Field | Required? | Description || --- | --- | --- |-| `use_path_style` | Yes for most S3-compatible | Set to `true` for path-style URLs (`http://host/bucket/key`). Most S3-compatible services require this. |-| `directory_marker_mode` | Yes for S3-compatible | **Must be explicitly set to `"none"`**. Without this, the RAGFS Rust binding panics with `AGFSConfigError: invalid directory_marker_mode: null` during startup, causing a silent crash loop. |+| `use_path_style` | Optional; default `true` | Use path-style URLs (`http://host/bucket/key`); select the style your endpoint supports. |+| `directory_marker_mode` | Yes for backup items | `none` creates no marker, `empty` creates a zero-byte marker, and `nonempty` creates a marker with content. Set it explicitly in each S3 backup item; if omitted, startup fails with `AGFSConfigError: configuration error: invalid directory_marker_mode: null (valid: none, empty, nonempty)`. || `use_ssl` | Optional | Set to `false` for HTTP endpoints (e.g. `http://localhost:9000`). |**Minimal S3-compatible example (RustFS/MinIO):**@@ -121,16 +121,14 @@ When using S3-compatible services (MinIO, RustFS, Ceph, etc.), the `s3` section}```-> **Why is `directory_marker_mode` required?**->-> S3-compatible storage services handle "directories" differently from AWS S3. The RAGFS Rust binding must know whether to write directory marker objects when creating directories. Valid values are `"none"`, `"empty"`, and `"nonempty"`. For S3-compatible services that don't use directory markers (RustFS, MinIO, Ceph, etc.), set this to `"none"`. If omitted, the Rust binding defaults to `null` which is rejected, causing the server to crash silently during startup with `AGFSConfigError: invalid directory_marker_mode: null`.+The example uses `none`, which suits S3-compatible services that do not use directory markers (RustFS, MinIO, Ceph, etc.). Backup items do not get a default for this field. The top-level `storage` S3 configuration defaults to `empty`.### Docker Networking for S3 BackupWhen running OpenViking in Docker with an S3 backup on the same host:- **Linux Docker**: Use `--network host` or the host's LAN IP. Docker bridge network can reach the host's LAN via gateway IP (e.g. `172.17.0.1:9000`).-- **macOS/Windows Docker Desktop**: `--network host` is **not supported** on Docker Desktop. Use `host.docker.internal` as the S3 endpoint (maps to the host's localhost). Alternatively, use the host's LAN IP.+- **macOS/Windows Docker Desktop**: Use `host.docker.internal` to reach the host from the container. Host networking is available as an opt-in feature in Docker Desktop 4.34 and later; see [Docker host networking](https://docs.docker.com/engine/network/drivers/host/).If the server crashes silently on startup with S3 backup enabled, check the Docker networking first. The RAGFS Rust binding will produce `dispatch failure` if the S3 endpoint is unreachable from inside the container.@@ -154,7 +152,7 @@ Characteristics:- Returns immediately after the primary write succeeds.- Backup writes run in the background.- Low write latency.-- Backups may lag temporarily.+- Lag has no fixed bound while a backend is unavailable; monitor synchronization and recovery.Suitable for:@@ -172,7 +170,7 @@ Sync mode waits for backup acknowledgements."sync_type": "sync","write_ack_count": 1,"write_ack_timeout_ms": 5000,- "items": []+ "items": [{"name": "local-backup", "backend": "local", "local": {"workspace": "./backup-data"}}]}}```@@ -206,6 +204,7 @@ Backups do not participate in reads by default. To let a backup serve reads, exp"name": "cache-backend","backend": "memfs","operations": [+ {"operation": "write", "priority": 0},{"operation": "read","priority": 10@@ -378,7 +377,7 @@ ov write viking://resources/multiwrite-check.txt \ov read viking://resources/multiwrite-check.txt```-If you use a local backup, you can also inspect the backup directory directly. In production, system health checks and sync-status commands are preferable.+If you use a local backup, you can also inspect the backup directory directly. Also run `ov system backend sync-status viking://resources` and check every intended replica. A successful ordinary read may come only from the primary and does not prove replication.## FAQ@@ -389,6 +388,7 @@ Backups are write-only by default. To make a backup serve reads, configure:```json{"operations": [+ {"operation": "write", "priority": 0},{"operation": "read","priority": 10@@ -403,7 +403,7 @@ Multi-write only handles new writes after it is enabled. For historical data, fo### Can async mode guarantee that the newest data is immediately readable from a backup?-No. Async mode provides eventual consistency only. If you need stronger read consistency, let reads fall back to the primary backend or avoid routing reads to backups that may lag.+No. Background retries need a functioning backend to catch up. A stale but successful backup read does not fall back to the primary; route reads to the primary when current data is required.### Will internal metadata files appear in normal user listings?diff --git a/docs/en/guides/14-ragfs-cache.md b/docs/en/guides/14-ragfs-cache.mdindex d4352b919..0da4bfc31 100644--- a/docs/en/guides/14-ragfs-cache.md+++ b/docs/en/guides/14-ragfs-cache.md@@ -60,6 +60,11 @@ Start Redis and OpenViking:```bashredis-server+```++In a second terminal, start OpenViking:++```bashopenviking-server --config ~/.openviking/ov.conf```@@ -83,7 +88,7 @@ Available Providers:Set `storage.agfs.pathlock.provider` to `cache` to coordinate path locksbetween OpenViking processes through the shared Redis CacheRuntime.`pathlock.namespace` is required and must be the same for every process in-one OpenViking deployment. Cache-backed PathLock only supports the built-in+one OpenViking deployment; use distinct namespaces for independent deployments. Distributed path locking does not remove the single-writer requirement of CachedFileSystem. Cache-backed PathLock only supports the built-inRedis Provider.Redis HASH keys are partitioned by logical path scope:@@ -160,7 +165,7 @@ The top-level `cache` section is a sibling of `storage`:| Option | Type | Default | Description ||--------|------|---------|-------------|| `provider` | str | `"filesystem"` | `filesystem`, `memory`, or `cache` |-| `namespace` | str or null | `null` | Required OpenViking instance name when `provider=cache` |+| `namespace` | str or null | `null` | Required deployment namespace shared by collaborating processes when `provider=cache` || `lock_expire_secs` | float | `30.0` | Lock stale timeout; must be at least `1.0` |Redis configuration:@@ -328,4 +333,6 @@ Recommended signals to watch:4. Cache summary files and raw `read_dir` first, then expand to more regular small files.5. Add lock, control-plane, and permission-sensitive paths to `bypass_prefixes`.-In short: RAGFS cache is responsible for correct invalidation according to filesystem semantics, while the Provider is responsible for where cache objects live. As long as the backend remains the source of truth, every cache hit must pass envelope and generation validation before it is returned.+RAGFS cache handles invalidation according to filesystem semantics; the Provider decides where cache objects live. The backend remains the source of truth, and every cache hit must pass envelope and generation validation before it is returned.++After enabling the cache, compare repeated reads and invalidation after a file update against the baseline. A higher hit rate is useful only if the returned contents remain current.diff --git a/docs/en/guides/15-snapshot.md b/docs/en/guides/15-snapshot.mdindex 38733dc08..450d6efc9 100644--- a/docs/en/guides/15-snapshot.md+++ b/docs/en/guides/15-snapshot.md@@ -1,6 +1,6 @@# Snapshots (Multi-Version Management) Guide-This guide explains how to enable and use OpenViking's multi-version management (snapshots). On top of VikingFS, it provides Git-based `commit`/`log`/`show`/`restore` primitives, letting you save an account's resource tree as a series of immutable snapshots, walk history, compare versions, and restore the workspace to any past state.+Snapshots save immutable versions of a selected file tree. Use `commit` to save a version, `log` to browse history, `show` to read an older file, and `restore` to recover saved content. Snapshots do not save uncommitted changes or files excluded at commit time; ACLs and vector indexes are not versioned. Restore compares the source snapshot with HEAD, so affected paths can overwrite uncommitted changes.Multi-version management is powered by [gitoxide](https://github.com/Byron/gitoxide) embedded in the Rust RAGFS layer, maintaining one logical Git repository per `account_id`. It is fully transparent to callers — you never run any `git` command yourself.@@ -126,7 +126,7 @@ Configuration reference:| `git.s3.use_path_style` | `true` | `true` uses path-style addressing (MinIO, etc.); `false` uses virtual-host style (TOS, etc.) || `git.s3.cas_mode` | `native` | Ref CAS mode. `native` uses S3 conditional writes (If-Match) |-After editing the config, restart the OpenViking service (or re-initialize the SDK client) for it to take effect.+After editing the server config, restart the OpenViking service. Reinitializing an HTTP SDK client does not reload server configuration.> The repository ships ready-to-use examples: [ov.conf.git-local.example](https://github.com/volcengine/OpenViking/blob/main/examples/snapshot/ov.conf.git-local.example) and [ov.conf.git-s3-tos.example](https://github.com/volcengine/OpenViking/blob/main/examples/snapshot/ov.conf.git-s3-tos.example).@@ -151,7 +151,7 @@ Key points:- `.ovgit` is an internal data directory. It is **not** exposed through `viking://` — users cannot see or modify it through the filesystem APIs (`ls`/`read`, etc.).- Its layout matches a standard Git object store (content-addressed `objects/`, loose `refs/`), but it is managed automatically by OpenViking. You should **not** run `git` commands against it.-- When backing up or migrating the workspace, copy `.ovgit` along with it to preserve the full version history.+- For a physical backup or migration, pause writes and copy the workspace and `.ovgit` at the same checkpoint. An [OVPack export](09-ovpack.md) saves current content, not snapshot history.- With the `s3` backend, no local `.ovgit` directory is created; data lives under the bucket's `{prefix}/{account}/...` keys instead.## Usage@@ -163,12 +163,13 @@ Once enabled, all three surfaces expose snapshot commands. The examples below shSnapshot methods live under the `client.snapshot.*` namespace.```python+from uuid import uuid4from openviking_sdk import SyncHTTPClientclient = SyncHTTPClient(url="http://localhost:1933", api_key="your-key")client.initialize()-root = "viking://resources/my_project"+root = f"viking://resources/snapshot-demo-{uuid4().hex[:8]}"# 1. Write initial content and commit v1client.write(@@ -176,7 +177,10 @@ client.write(content="# Guide\n\nv1 content\n",mode="create",)+print(client.wait_processed(timeout=120))v1 = client.snapshot.commit(message="v1 initial import", paths=[root])+if not v1.get("commit_oid"):+ raise RuntimeError(f"No snapshot created: {v1}")print("v1:", v1["commit_oid"])# 2. Modify and commit v2@@ -185,6 +189,7 @@ client.write(content="# Guide\n\nv2 content\n",mode="replace",)+print(client.wait_processed(timeout=120))v2 = client.snapshot.commit(message="v2 update", paths=[root])# 3. Walk history@@ -194,12 +199,14 @@ for c in client.snapshot.log(limit=10, paths=[root]):# 4. Read historical file contentprint(client.snapshot.show(v1["commit_oid"], path=f"{root}/guide.md"))-# 5. Restore the workspace to v1 (creates a new "forward" commit on top of v2)-client.snapshot.restore(project_dir=root, source_commit=v1["commit_oid"], message="restore to v1")+# 5. Preview the restore before changing files+print(client.snapshot.restore(project_dir=root, source_commit=v1["commit_oid"], dry_run=True))client.close()```+The example uses a new directory on each run, waits for background processing before each commit, and ends with a preview. Inspect the processing results for errors; queue completion alone does not prove index consistency. After reviewing the plan, reconnect and call `restore` with the same `project_dir` and `source_commit`, with `dry_run=False`, to apply it.+### CLIThe CLI subcommands live under `ov snapshot`:@@ -211,17 +218,23 @@ ov snapshot commit -m "v1 initial import" --paths viking://resources/my_project# Walk history (newest first)ov snapshot log --paths viking://resources/my_project --limit 10 -o json+# Set this to the commit_oid returned above+COMMIT_OID="replace-with-commit-oid"+# Read historical file content-ov snapshot show <commit_oid> --path viking://resources/my_project/guide.md+ov snapshot show "$COMMIT_OID" --path viking://resources/my_project/guide.md# Read a file's content from a commit (defaults to stdout; use --out-file to write a local file)-ov snapshot show <commit_oid> --path viking://resources/my_project/guide.md --out-file ./guide.md+ov snapshot show "$COMMIT_OID" --path viking://resources/my_project/guide.md --out-file ./guide.md++# Preview which files would change+ov snapshot restore "$COMMIT_OID" viking://resources/my_project --dry-run -o json+```-# Restore a directory to a past snapshot (positional args are <source_commit> then <project_dir>)-ov snapshot restore <commit_oid> viking://resources/my_project -m "restore to v1" -o json+After reviewing the preview, apply the restore:-# Preview which files would change first-ov snapshot restore <commit_oid> viking://resources/my_project --dry-run -o json+```bash+ov snapshot restore "$COMMIT_OID" viking://resources/my_project -m "restore to v1" -o json```### HTTP API@@ -241,19 +254,21 @@ curl -X GET "http://localhost:1933/api/v1/snapshot/log?branch=main&limit=10&pathcurl -X GET "http://localhost:1933/api/v1/snapshot/show?target_ref=<commit_oid>&path=viking://resources/my_project/guide.md" \-H "X-API-Key: your-key"-# Restore+# Preview restorecurl -X POST "http://localhost:1933/api/v1/snapshot/restore" \-H "Content-Type: application/json" \-H "X-API-Key: your-key" \- -d '{"project_dir": "viking://resources/my_project", "source_commit": "<commit_oid>", "message": "restore to v1"}'+ -d '{"project_dir": "viking://resources/my_project", "source_commit": "<commit_oid>", "message": "restore to v1", "dry_run": true}'```+Replace `<commit_oid>` with the saved commit ID. The restore request above is a preview; set `dry_run` to `false` only when you intend to apply it.+## Key Semantics: Forward-Commit Restore`restore` uses **forward-commit** semantics: it reads the content at `source_commit`, writes the diff back into the workspace, and creates a **new commit on top of the current HEAD**. Therefore:- The new commit's parent is the HEAD that existed before the restore — **not** `source_commit`.-- HEAD always advances monotonically, and **history is never rewritten or lost** — going back to an older version is itself a new commit.+- When the source and HEAD trees differ in the selected scope, restore adds a commit without rewriting history. If those trees match, it returns `result: "noop"` without a new commit, even if the working files differ. Snapshot storage still needs backups.- `restore` only affects files within `project_dir` (the whole account tree when omitted); files outside that scope are left untouched.## Excluding Files with `.ovgitignore`@@ -297,7 +312,7 @@ On subsequent commits, files matching the rules are excluded, and the response's```pythonv = client.snapshot.commit(message="with ignore", paths=["viking://resources/my_project"])-print(v["result"], v.get("ignored")) # created, 1+print(v["result"], v.get("ignored"))```### CLI@@ -334,7 +349,7 @@ curl -X DELETE "http://localhost:1933/api/v1/snapshot/ignore" \## Notes-- After editing the `git` config, restart the service / re-initialize the client for it to take effect.+- Restart the service after editing its `git` configuration.- With the `s3` backend, `git.s3.bucket` and `git.s3.region` are required; missing them causes initialization to fail.- If a restore has vector side effects (files written/deleted), the response carries a `task_id` you can poll via `GET /api/v1/tasks/{task_id}` to track the background vector rebuild (see the [Observability guide](05-observability.md) and [API Overview](../api/01-overview.md)).- If `.ovgitignore` is too large (over 64 KiB) or contains unsupported syntax (`!` negation, backslash escaping), `commit` fails with an `invalid operation` error; `set_gitignore` validates the size up front.diff --git a/docs/zh/concepts/10-encryption.md b/docs/zh/concepts/10-encryption.mdindex 34566e6b9..506ef1edd 100644--- a/docs/zh/concepts/10-encryption.md+++ b/docs/zh/concepts/10-encryption.md@@ -1,53 +1,34 @@# 数据加密-OpenViking 提供透明的静态数据加密,确保多租户环境下的数据安全与隔离。+<script setup>+import EncryptionFlowDiagram from '../../.vitepress/theme/components/EncryptionFlowDiagram.vue'+</script>++OpenViking 支持静态数据加密,在写入存储前加密文件,并在授权读取时解密。不同 account 使用独立的账户密钥。## 概述### 为什么需要加密-在多租户架构中,不同客户(账户)的资源文件、记忆和技能都存储在共享的 AGFS 实例中。加密确保:+多个 account 可以共用 AGFS。启用加密后:-- 即使攻击者获得 AGFS 磁盘访问权限,也无法读取任何客户的明文数据+- 已加密文件需要对应密钥才能解密;密钥应与数据分开保护- 不同账户的数据使用独立密钥加密,实现租户隔离-- 所有加密解密操作集中在 VikingFS 层,AGFS 和外部对象存储只看到密文+- 运行时由 RAGFS 的加密包装层处理文件读写;每个后端的加密范围取决于配置### 对谁透明-加密功能对用户和开发者完全透明:+启用加密不改变客户端 API:- **客户端 API 无变化**:现有代码无需修改-- **应用层无感知**:读写操作与未加密时完全相同-- **向后兼容**:未加密的旧文件仍可正常读取+- **返回明文**:授权读取仍返回解密后的内容;密钥不可用或密文校验失败时,读写会报错+- **兼容旧文件**:未加密的旧文件仍可读取;开启加密不会自动加密已有文件## 三层密钥架构OpenViking 采用信封加密(Envelope Encryption)架构,使用三层密钥体系:-```-┌─────────────────────────────────────────────────────────┐-│ Layer 1: Root Key(根密钥) │-│ • 整个 OpenViking 实例全局唯一 │-│ • 存储:KMS 服务 / ~/.openviking/master.key │-│ • 用途:派生所有账户密钥 │-└────────────────────┬────────────────────────────────────┘- │ HKDF 派生- ▼-┌─────────────────────────────────────────────────────────┐-│ Layer 2: Account Key(账户密钥,KEK) │-│ • 每个账户一个独立密钥 │-│ • 不存储,运行时派生 │-│ • 用途:加密该账户下的所有文件密钥 │-└────────────────────┬────────────────────────────────────┘- │ AES-256-GCM 加密- ▼-┌─────────────────────────────────────────────────────────┐-│ Layer 3: File Key(文件密钥,DEK) │-│ • 每次写操作生成新的随机密钥 │-│ • 加密后存储在文件头(信封)中 │-│ • 用途:加密实际文件内容 │-└─────────────────────────────────────────────────────────┘-```+<KeyHierarchyDiagram />### 密钥层次说明@@ -64,8 +45,8 @@ OpenViking 支持三种密钥提供程序,适应不同的部署场景:| 提供程序 | 适用场景 | Root Key 存储 | 特点 ||---------|---------|--------------|------|| **Local** | 开发环境、单节点部署 | 本地文件 `~/.openviking/master.key` | 简单,无需外部服务 |-| **Vault** | 生产环境、多云部署 | HashiCorp Vault Transit Engine | 企业级密钥管理,支持密钥版本控制 |-| **Volcengine KMS** | 火山引擎云部署 | 火山引擎 KMS | 云原生密钥管理服务 |+| **Vault** | 生产环境、多云部署 | 经 Transit 加密后存入 Vault KV | 由 Vault 保护根密钥 |+| **Volcengine KMS** | 火山引擎云部署 | 经 KMS 加密后存入本地 `key_file` | 由 KMS 保护根密钥 |### Local(本地文件)@@ -133,81 +114,49 @@ ov system crypto init-key --output-file ~/.openviking/master.key## 工作原理+运行时在启动阶段取得根密钥,再由 RAGFS `EncryptionWrappedFS` 处理受保护后端的文件内容。Python 的密钥提供程序负责加载或解封根密钥;每次文件读写不需要到 KMS 派生账户密钥。+### 写流程-```-客户端 VikingFS FileEncryptor KeyManager AGFS- │ │ │ │ │- │ write(uri, data) │ │ │ │- │──────────────────>│ │ │ │- │ │ encrypt(account_id, │ │ │- │ │ plaintext) │ │ │- │ │──────────────────────>│ │ │- │ │ │derive_account_key()│ │- │ │ │───────────────────>│ │- │ │ │<───────────────────│ │- │ │ │ account_key │ │- │ │ 1. 生成随机 File Key │ │- │ │ 2. 用 File Key 加密内容 │ │- │ │ 3. 用 Account Key 加密 File Key │ │- │ │ 4. 构建信封格式 │ │- │ │<──────────────────────│ │ │- │ │ ciphertext │ │ │- │ │─────────────────────────────────────────────────────────>│- │ │ │ │ Write │- │<──────────────────│ │ │ │- │ success │ │ │ │-```+<EncryptionFlowDiagram mode="write" />++1. 用根密钥和 `account_id` 经 HKDF-SHA256 派生账户密钥(可在运行时缓存)。+2. 为这次加密生成随机 File Key 和 nonce。+3. 用 File Key 对内容执行 AES-256-GCM 加密,再用账户密钥加密 File Key。+4. 将信封头、加密后的 File Key、nonce 和内容密文写入后端。### 读流程-```-客户端 VikingFS FileEncryptor KeyManager AGFS- │ │ │ │ │- │ read(uri) │ │ │ │- │──────────────────>│ │ │ │- │ │──────────────────────────────────────────────────────────>│- │ │ │ │ Read │- │ │<──────────────────────────────────────────────────────────│- │ │ raw_bytes │ │ │- │ │ 检查魔术数 == "OVE1"? │ │ │- │ │ 是 → decrypt() │ │ │- │ │──────────────────────>│ │ │- │ │ │ derive_account_key()│ │- │ │ │────────────────────>│ │- │ │ │<────────────────────│ │- │ │ │ account_key │ │- │ │ 1. 解析信封格式 │ │- │ │ 2. 用 Account Key 解密 File Key │ │- │ │ 3. 用 File Key 解密内容 │ │- │ │<──────────────────────│ │ │- │ │ plaintext │ │ │- │<──────────────────│ │ │ │- │ content │ │ │ │-```+<EncryptionFlowDiagram mode="read" />++读取加密文件时,先用账户密钥解封 File Key,再校验并解密内容。没有 `OVE1` 标记的旧文件按明文读取。密钥错误或密文认证失败会报错,不会返回解密结果。### 信封格式加密文件使用统一的信封格式,以魔术数 `OVE1`(OpenViking Encryption v1)开头:-```-┌─────────────────────────────────────────────────────────────┐-│ 魔术数 │ 版本 │ Provider │ 加密的 File Key │ .. │-│ 4 字节 │ 1 字节 │ 1 字节 │ 可变长度 │ ... │-│ "OVE1" │ 0x01 │ 0x01=local │ │ ... │-└─────────────────────────────────────────────────────────────┘-```+| 字段 | 长度 | 取值 |+| --- | --- | --- |+| Magic | 4 字节 | `OVE1` |+| Version | 1 字节 | `0x01` |+| Provider | 1 字节 | `0x01` = local |+| Encrypted File Key | 可变 | 用账户密钥封装的 File Key |+| … | | nonce 和内容密文 |- 如果文件不以 `OVE1` 开头,视为未加密文件,直接返回明文-- 支持向后兼容,旧文件无需迁移+- 旧文件可继续读取;需要保护存量明文时,必须另行迁移或重写## 多租户隔离不同账户的数据使用独立的 Account Key 加密:- 账户 A 的密钥无法解密账户 B 的文件-- 即使 AGFS 被完全访问,没有对应密钥也无法读取数据-- 租户隔离在密钥层面实现,不依赖存储层权限+- 对已经加密的文件,仅取得后端密文仍需要对应密钥才能解密+- 独立密钥补充租户访问控制,不能代替身份认证和存储权限管理++## 加密范围++这里保护的是启用了加密的文件后端。它不自动加密独立向量库、日志或导出的 OVPack;向量记录可能包含摘要和记忆正文。多写 backup 可单独关闭加密,部署时需要逐个检查后端配置。## 配置示例diff --git a/docs/zh/concepts/14-multi-write-storage.md b/docs/zh/concepts/14-multi-write-storage.mdindex 6cd24d574..29f2cd709 100644--- a/docs/zh/concepts/14-multi-write-storage.md+++ b/docs/zh/concepts/14-multi-write-storage.md@@ -1,6 +1,6 @@# 多写存储-多写存储让 OpenViking 在一个统一的文件系统抽象下,同时使用一个主存储和多个备份存储。它适合数据高可用、跨区域副本、读加速、存储迁移等场景。+多写存储让 OpenViking 在一个统一的文件系统抽象下,同时使用一个主存储和多个备份存储。它可用于保存副本(包括跨地域副本)、配置读路由和迁移存储;不会自动把 backup 提升为 primary。从 API 使用者视角看,`read()`、`write()`、`ls()`、`stat()` 等接口不变。多写逻辑位于 RAGFS 内部,调用方不需要关心文件最终落在哪个底层后端。@@ -38,9 +38,9 @@ backup 未配置 `operations` 时默认参与写入。这样可以用最少配| 异步多写 | `async` | primary 写成功后立即返回,backup 后台同步 | 低延迟写入、最终一致 || 同步多写 | `sync` | primary 写成功后等待 backup 确认 | 更强写入确认、可接受额外延迟 |-异步模式下,backup 可能在短时间内落后于 primary。同步模式下,可以通过 `write_ack_count` 和 `write_ack_timeout_ms` 控制需要等待多少 backup 确认,以及等待多久。+异步模式下,backup 可能落后于 primary,恢复时间取决于后端可用性和重试结果。同步模式下,可以通过 `write_ack_count` 和 `write_ack_timeout_ms` 控制需要等待多少 backup 确认,以及等待多久。-即使使用同步模式,未确认或超时的 backup 仍会由后台重试修复。+同步模式未达到所需确认数时会返回错误,但 primary 可能已经写入成功,不会因此自动回滚。未确认的 backup 仍会在后台重试;同步状态的检查方法见[多写存储指南](../guides/13-multi-write-storage.md)。## 读取路径@@ -113,14 +113,14 @@ Exclude 策略配置在 backup 上,只影响该 backup 是否接收写入。如果要迁移存量数据,推荐流程是:-1. 使用 OVPack 或其他受控方式完成全量迁移。+1. 暂停写入或安排可核对的增量补齐窗口,再使用 OVPack 或其他受控方式迁移存量。2. 校验目标 backend 数据。3. 启用多写配置。-4. 后续新增和修改的数据由多写持续复制。+4. 补齐迁移期间的变更并验证同步状态后恢复写入;后续新增和修改由多写复制。## 限制-- 异步模式下 backup 可能短暂落后。+- 异步模式下 backup 可能落后;读取 backup 时不能假设已经看到最新写入。- 启用多写前的历史文件需要单独迁移或回填。- redirect 文件依赖内部元数据恢复目录视图。- 多进程同时写同一 primary 时,需要未来的分布式元数据锁能力。diff --git a/docs/zh/guides/08-encryption.md b/docs/zh/guides/08-encryption.mdindex 0375db75f..a2c3e5201 100644--- a/docs/zh/guides/08-encryption.md+++ b/docs/zh/guides/08-encryption.md@@ -4,12 +4,12 @@## 概述-OpenViking 提供透明的静态数据加密,确保多租户环境下的数据安全与隔离:+OpenViking 可以按账户加密存储中新写入的文件。加解密由存储层处理,客户端沿用现有 API:-- ✅ **透明加密**:API 无变化,应用层无感知-- ✅ **多租户隔离**:不同账户使用独立密钥-- ✅ **三种密钥提供程序**:Local、Vault、火山引擎 KMS-- ✅ **向后兼容**:未加密的旧文件仍可正常读取+- **透明加密**:API 无变化,应用层无感知+- **多租户隔离**:不同账户使用独立密钥+- **三种密钥提供程序**:Local、Vault、火山引擎 KMS+- **向后兼容**:未加密的旧文件仍可正常读取加密功能的概念说明见 [数据加密](../concepts/10-encryption.md)。@@ -36,7 +36,7 @@ ov system crypto init-key --output-file ~/.openviking/master.key### 2. 配置加密-编辑 `~/.openviking/ov.conf`:+将下面的加密配置合并到 `~/.openviking/ov.conf`,保留已有模型和服务端配置:```json{@@ -90,23 +90,23 @@ asyncio.run(test())## API Key 哈希配置-OpenViking 提供两层加密保护:+文件加密和 API Key 哈希是两项独立配置:| 加密层 | 配置项 | 算法 | 可逆性 | 说明 ||--------|--------|------|--------|------|-| **文件层** | `encryption.enabled` | AES-GCM | ✅ 可逆 | 保护整个存储文件 |-| **API key 字段层** | `encryption.api_key_hashing.enabled` | Argon2id | ❌ 不可逆 | 保护 API key 本身 |+| **文件层** | `encryption.enabled` | AES-GCM | 可逆 | 保护整个存储文件 |+| **API key 字段层** | `encryption.api_key_hashing.enabled` | Argon2id | 不可逆 | 保护 API key 本身 |-### ⚠️ Breaking Change 说明+### 从隐式 API Key 哈希配置升级-**版本变更**:OpenViking v0.3.12 → later versions+适用于从 v0.3.12 及更早版本升级到 v0.3.13 及之后版本。**行为变化**:- **之前**:`encryption.enabled = true` 隐式启用 API key Argon2id 哈希- **现在**:需要显式配置 `encryption.api_key_hashing.enabled`**影响**:-- 升级后,如果 `encryption.enabled = true` 但 `encryption.api_key_hashing.enabled` 未显式配置为 `true`,会在启动时看到以下警告日志:+- 升级后,如果 `encryption.enabled = true` 但 `encryption.api_key_hashing.enabled` 未显式配置为 `true`,会在启动时看到以下 INFO 日志片段:```API key hashing is disabled while file encryption is enabled.Previously, encryption.enabled=true implicitly enabled API key Argon2id hashing.@@ -119,7 +119,7 @@ OpenViking 提供两层加密保护:| 选项 | 配置 | 行为 ||------|------|------|| **保持原有行为** | `api_key_hashing.enabled = true` | API key 使用 Argon2id 哈希存储 |-| **推荐新行为** | `api_key_hashing.enabled = false`(默认) | API key 明文存储(文件层仍加密) |+| **可恢复的 key 存储(默认)** | `api_key_hashing.enabled = false`(默认) | API key 明文存储(文件层仍加密) |### 默认行为@@ -130,7 +130,7 @@ OpenViking 提供两层加密保护:### 启用 Argon2id 哈希-如果需要最高级别的 API key 保护,可以启用 Argon2id 单向哈希:+启用 Argon2id 单向哈希后,服务端不再保存可还原的 API Key 明文:```json{@@ -170,7 +170,7 @@ OpenViking 提供两层加密保护:| 提供程序 | 适用场景 | 优点 | 缺点 ||---------|---------|------|------|-| **Local** | 开发环境、单节点部署 | 简单,无需外部服务 | 密钥存储在本地,安全性较低 |+| **Local** | 开发环境、单节点部署 | 简单,无需外部服务 | 需要管理本地文件权限和独立密钥备份 || **Vault** | 生产环境、多云部署 | 企业级密钥管理,支持版本控制 | 需要部署和维护 Vault || **Volcengine KMS** | 火山引擎云部署 | 云原生密钥管理服务 | 仅限火山引擎环境 |@@ -221,7 +221,7 @@ ov system crypto init-key -f ~/.openviking/master.key### 前置条件-1. 已部署 HashiCorp Vault 服务+1. 已部署 HashiCorp Vault 服务,并在服务端 Python 环境安装 `hvac`2. 已启用 Transit 引擎3. 有足够权限的 Vault Token@@ -237,10 +237,16 @@ vault secrets enable transit```bash# KV v2(推荐)-vault secrets enable -version=2 kv+vault secrets enable -path=secret -version=2 kv# 或 KV v1-vault secrets enable kv+vault secrets enable -path=secret -version=1 kv+```++按实际部署选择一条 KV 命令,不要两条都执行。下面使用 `secret` 挂载点和 KV v2。先由管理员创建 Transit key:++```bash+vault write -f transit/keys/openviking-root-key type=aes256-gcm96```3. 配置 OpenViking:@@ -255,7 +261,7 @@ vault secrets enable kv"token": "hvs.xxxxxxxxxxxxxxxxxxxxx","mount_point": "transit","kv_mount_point": "secret",- "kv_version": 1,+ "kv_version": 2,"root_key_name": "openviking-root-key","encrypted_root_key_key": "openviking-encrypted-root-key"}@@ -277,18 +283,32 @@ vault secrets enable kv### Vault 权限建议-为 Token 配置最小权限:+对于上面的配置,服务 token 需要读取 Transit key 元数据、加解密权限,以及保存封装根密钥的 KV 读写权限。提前创建引擎和 Transit key 后,可参考:```hcl-path "transit/encrypt/openviking-root" {+path "auth/token/lookup-self" {+ capabilities = ["read"]+}++path "transit/keys/openviking-root-key" {+ capabilities = ["read"]+}++path "secret/data/openviking-encrypted-root-key" {+ capabilities = ["read", "create", "update"]+}++path "transit/encrypt/openviking-root-key" {capabilities = ["update"]}-path "transit/decrypt/openviking-root" {+path "transit/decrypt/openviking-root-key" {capabilities = ["update"]}```+使用 KV v1 时,将 `secret/data/openviking-encrypted-root-key` 改为 `secret/openviking-encrypted-root-key`。provider 还会检查 `sys/mounts`;权限不足会记录警告,Transit 已启用时不会因此停止启动。+---## Volcengine KMS 模式详细指南@@ -363,7 +383,7 @@ path "transit/decrypt/openviking-root" {### 方法一:检查文件内容-加密文件以魔术数 `OVE1` 开头:+检查物理后端文件,API 返回的内容已经解密。下面的路径需换成实际文件路径。加密文件以魔术数 `OVE1` 开头:```bash# 查看文件前 4 字节@@ -381,21 +401,11 @@ hexdump -C ./data/agfs/your-file | head -100000000 7b 22 63 6f 6e 74 65 6e 74 73 22 3a 5b 7b 22 70 |{"contents":[{"p|```-### 方法二:跨提供程序验证+### 方法二:重启后读回-尝试用不同提供程序解密彼此的数据,应该会失败(这是正常的安全行为):+重启前后,通过 API 读取同一份已导入文件,确认内容一致,再按上面的方法核对物理存储文件头。这能检查部署是否重新加载了原密钥,并能读取已有密文。-```python-# 用 Provider A 加密-encrypted = await provider_a.encrypt_file_key(plaintext, "test-account")--# 尝试用 Provider B 解密(应该失败)-try:- await provider_b.decrypt_file_key(encrypted, "test-account")- print("❌ 安全漏洞:跨提供程序解密成功!")-except Exception as e:- print("✓ 安全:跨提供程序解密失败,符合预期")-```+内部 provider 调用抛出异常不能直接当作加密验证通过。参数错误、依赖缺失和网络错误都可能在解密前失败。---@@ -426,12 +436,7 @@ ov restore ./backups/before-encryption.ovpack --on-conflict overwrite### 切换密钥提供程序-1. 备份现有数据和密钥-2. 使用旧提供程序解密所有数据-3. 配置新提供程序-4. 重新加密所有数据--**注意**:这是一个破坏性操作,建议在测试环境先验证。+保持旧 provider 和密钥可用,导出逻辑 OVPack 备份。按上面的迁移步骤,在使用新 provider 的独立空环境恢复,核对内容、权限和重建后的索引,再切换流量。直接修改原环境的 provider 配置不会重新加密已有文件。---@@ -471,19 +476,11 @@ Error: Invalid credentials2. 确认密钥有足够权限3. 验证区域配置正确-### 跨提供程序解密失败(这是正常的)--```-Error: KeyMismatchError-```--**说明**:这是预期的安全行为。不同提供程序使用不同的根密钥,无法相互解密。+### 已有密文无法读取-### 部分读取返回密文+检查原根密钥、provider 配置、account 身份和密文是否仍然完整。Vault 需要保留 Transit key 和存放封装根密钥的 KV 条目;KMS 需要保留 KMS key 和本地封装密钥文件。不要通过生成新根密钥修复旧密文的读取问题。-如果使用旧版本 OpenViking 创建的加密文件,部分读取可能返回密文。--**解决方案**:升级到最新版本的 OpenViking。+部分读取会先加载并校验整个加密文件,再返回所需的明文切片。部分读取返回密文时,记录服务版本,在测试副本上比较完整读取与部分读取,核对存储加密配置和相关修复后再升级。保留原数据及密钥,以便回退。---@@ -492,3 +489,5 @@ Error: KeyMismatchError- [数据加密](../concepts/10-encryption.md) - 加密概念说明- [配置指南](./01-configuration.md) - 完整配置参考- [多租户](../concepts/11-multi-tenant.md) - 账号、用户与 Agent 的隔离模型++Vault 参考:[KV v2](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2)、[Transit API](https://developer.hashicorp.com/vault/api-docs/secret/transit)。diff --git a/docs/zh/guides/09-ovpack.md b/docs/zh/guides/09-ovpack.mdindex eb187c342..a1aa74ff5 100644--- a/docs/zh/guides/09-ovpack.md+++ b/docs/zh/guides/09-ovpack.md@@ -1,6 +1,6 @@# OVPack 导入导出-OVPack 是 OpenViking 的可恢复内容包格式,用来迁移或备份 `viking://` 下的公开内容树。+OVPack 是 OpenViking 的可恢复内容包格式,用来迁移或备份 `viking://` 下受支持的内容树。它保存文件内容、语义侧边文件、可迁移的索引标量,以及可选的 dense 向量快照。OVPack 不是裸 ZIP 拷贝,也不是可信发布格式。导入会校验 manifest、文件列表、目录列表和@@ -16,13 +16,13 @@ checksum,保证包内容没有偏离 manifest;如果攻击者能同时篡改- `viking://resources/...`- `viking://user/...`-全量迁移使用单独的 `backup/restore`,它会把公开 scope root 一起打进备份包:+全量迁移使用单独的 `backup/restore`,它会把支持的 scope root 一起打进备份包。这里指命名空间根,不表示内容对所有人公开:- `viking://resources`- 当前账号下所有 `viking://user/{user_id}` 内容`backup/restore` 仅允许 ROOT 或 ADMIN 调用,并以账号为边界访问所有用户内容。备份包不包含-用户账号、API Key 或其他鉴权数据。+用户账号和服务端 API Key 注册表。用户文件仍属于内容树,包括可能存放外部凭证的 privacy 配置,因此需要保护归档。Session 通过 user 命名空间一起迁移,路径为`viking://user/{user_id}/sessions/{session_id}`。`viking://session/...`@@ -67,7 +67,7 @@ ov import ./exports/my-project.ovpack viking://resources/imported/viking://resources/imported/my-project```-覆盖已有 root:+需要替换已有 root 时,先备份目标,并确认允许删除其整个子树:```bashov import ./exports/my-project.ovpack viking://resources/imported/ --on-conflict overwrite@@ -145,7 +145,6 @@ curl -X POST http://localhost:1933/api/v1/system/consistency \```bashov backup ./backups/openviking.ovpack-ov restore ./backups/openviking.ovpack --on-conflict overwrite```备份包只能通过 `restore` 恢复,不能通过普通 `import` 导入到任意父目录。@@ -184,6 +183,8 @@ OPENVIKING_CLI_CONFIG_FILE=./restore.ovcli.conf \## Python SDK+跨部署迁移时,导出和备份使用源端身份,导入和恢复使用单独配置的目标客户端。下方全量恢复示例以已完成上述目标 account 初始化和覆盖检查为前提。队列结束不证明索引重建成功,仍需检查错误和一致性结果。+```pythonfrom openviking_sdk import AsyncHTTPClient@@ -201,11 +202,12 @@ async def migrate_project():imported_uri = await client.import_ovpack(file_path="./exports/my-project.ovpack",parent="viking://resources/imported/",- on_conflict="overwrite",+ on_conflict="fail",vector_mode="auto",)print(imported_uri)- await client.wait_processed()+ print(await client.wait_processed(timeout=120))+ print(await client.check_consistency(uri=imported_uri))finally:await client.close()```@@ -242,7 +244,7 @@ importedURI, err := client.ImportOVPack(outPath,"viking://resources/imported/",&openviking.ImportPackOptions{- OnConflict: "overwrite",+ OnConflict: "fail",VectorMode: "auto",},)@@ -286,7 +288,7 @@ HTTP 导出接口直接返回文件流;HTTP 导入和恢复必须先上传本导出:```bash-curl -X POST http://localhost:1933/api/v1/pack/export \+curl -f -X POST http://localhost:1933/api/v1/pack/export \-H "Content-Type: application/json" \-H "X-API-Key: your-admin-key" \-d '{"uri":"viking://resources/my-project","include_vectors":false}' \@@ -297,19 +299,19 @@ curl -X POST http://localhost:1933/api/v1/pack/export \```bashTEMP_FILE_ID=$(- curl -sS -X POST http://localhost:1933/api/v1/resources/temp_upload \+ curl -fsS -X POST http://localhost:1933/api/v1/resources/temp_upload \-H "X-API-Key: your-admin-key" \-F "file=@./exports/my-project.ovpack" \- | jq -r ".result.temp_file_id"+ | jq -er ".result.temp_file_id")-curl -X POST http://localhost:1933/api/v1/pack/import \+curl -f -X POST http://localhost:1933/api/v1/pack/import \-H "Content-Type: application/json" \-H "X-API-Key: your-admin-key" \-d "{\"temp_file_id\": \"$TEMP_FILE_ID\",\"parent\": \"viking://resources/imported/\",- \"on_conflict\": \"overwrite\",+ \"on_conflict\": \"fail\",\"vector_mode\": \"auto\"}"```@@ -317,7 +319,7 @@ curl -X POST http://localhost:1933/api/v1/pack/import \全量备份:```bash-curl -X POST http://localhost:1933/api/v1/pack/backup \+curl -f -X POST http://localhost:1933/api/v1/pack/backup \-H "Content-Type: application/json" \-H "X-API-Key: your-admin-key" \-d '{"include_vectors":true}' \@@ -365,7 +367,7 @@ my-project/_ovpack/manifest.json`files/` 下保存用户内容,路径与 OpenViking 中的相对路径完全一致,不再对点文件做 `_._` 转义。`_ovpack/` 下保存 OVPack 内部文件,不参与用户内容导入。-manifest 只保存包结构、文件 checksum 和内部索引文件的 checksum,不直接内嵌每个文件的索引记录:+下方 manifest 仅示意格式,含占位 checksum,不能作为有效包导入。manifest 保存包结构、文件 checksum 和内部索引文件的 checksum,不直接内嵌每个文件的索引记录:```json{@@ -561,7 +563,7 @@ OVPack v2 包也会被当前 OpenViking 拒绝。导入旧包前,需要先用| `source path is incompatible with target path` | 结构化 scope 的 root 层级会改变 | 导入到正确系统父目录。 || `Top-level scope ovpack packages must be imported to viking://` | 将顶级 scope 包导入了非根父目录 | 改为导入 `viking://`。 || `Backup ovpack packages must be restored` | 用普通 import 导入 backup 包 | 使用 `ov restore`。 |-| `Resource already exists` | 目标 root 已存在 | 使用 `--on-conflict overwrite` 或 `--on-conflict skip`。 |+| `Resource already exists` | 目标 root 已存在 | 换用新的目标父目录;用 `--on-conflict skip` 保留已有 root;或备份目标后用 `--on-conflict overwrite` 替换。 || `incomplete OpenViking vector index snapshot` | 使用 `--include-vectors` 时,导出范围内应索引内容缺少索引记录 | 先执行 `ov system consistency <uri>` 定位问题,再等待处理完成或重新 reindex。 || `dense vector snapshot is incompatible` | 包内 embedding 元数据和当前配置不一致 | 用 `--vector-mode recompute`,或换到兼容配置。 |@@ -579,5 +581,5 @@ manifest 校验;如果同时修改 manifest 和内容,则需要依赖外部**大包导入很慢怎么办?**-默认导入会重建目标环境的语义和向量。大包迁移可以使用 `--include-vectors` 减少重算,或按目录+导入保留包内语义侧边文件;未恢复兼容 dense 快照时,会提交向量重算。大包迁移可以使用 `--include-vectors` 减少重算,或按目录拆成多个 OVPack 分批导入。diff --git a/docs/zh/guides/11-oauth.md b/docs/zh/guides/11-oauth.mdindex deb4bb699..b5bb48b6c 100644--- a/docs/zh/guides/11-oauth.md+++ b/docs/zh/guides/11-oauth.md@@ -1,13 +1,11 @@# OAuth 2.1 接入指南-OpenViking 服务端原生实现 OAuth 2.1。任何需要 OAuth 的客户端 — 包括 MCP-客户端(Claude.ai / Claude Desktop / ChatGPT / Cursor)以及其他浏览器应用-— 都可以直接对服务器授权,无需任何第三方代理。协议层(DCR、authorize、-token、metadata)由官方 `mcp.server.auth` SDK 提供,整体遵循 OAuth 2.1-规范,非 MCP 的 OAuth 客户端也能正常工作。+OpenViking 提供原生 OAuth 授权,供支持其发现、动态注册和 PKCE 流程的客户端接入,包括 Claude.ai、Claude Desktop、ChatGPT、Cursor 等 MCP 客户端。协议端点(注册、authorize、token、metadata)由官方 `mcp.server.auth` SDK 提供,非 MCP 的 OAuth 客户端也可以走同一流程。用户在 Studio 确认身份和授权,客户端取得 access token 和 refresh token。API Key 认证仍可使用。## 推荐配置+先按[认证指南](04-authentication.md)配置 API Key 模式,创建 account 和 user/admin key。OAuth consent 需要这个已注册身份,开启 OAuth 不会自动创建。下面的片段应合并到已有配置。+> **前提**:公网 HTTPS。OAuth 2.1(以及 MCP SDK)对非 localhost 的 issuer> **强制要求 HTTPS**。请参阅[公网访问指南](12-public-access.md)了解如何配置> Caddy 或 nginx 的 HTTPS。@@ -29,22 +27,13 @@ token、metadata)由官方 `mcp.server.auth` SDK 提供,整体遵循 OAuth 2先在弹出的"连接与身份"对话框里粘入 API Key;然后在 consent 卡片上点**Authorize**。浏览器自动跳回 Claude.ai,连接器就位。-线上路径就这四步。后续章节解释每一块为什么这样设计、本地怎么不走 HTTPS 做-联调、出问题时怎么用 curl 排查。+连接后,确认客户端能发现工具并读取有权限的 URI。后续章节介绍本地联调、token 行为和故障排查。---## 为什么需要原生 OAuth-部分 MCP 客户端只接受 OAuth 2.1,不接受 API Key。在此之前唯一的方案是部署社区的-[MCP-Key2OAuth](https://github.com/t0saki/MCP-Key2OAuth) Cloudflare Worker-代理,把 OAuth 翻译成 API Key bearer。原生支持解决了:--- 额外部署单元(CF Worker + 2 个 KV namespace)-- 第三方信任面(代理运营方有解密上游 API Key 的能力)-- 用户在浏览器里手动粘贴 API Key 的体验--API Key 认证仍按原方式工作,OAuth 只是叠加层。+客户端需要浏览器授权和 token 刷新时,使用 OAuth。授权流程由服务端自己完成,客户端和 API Key 之间没有中间代理。首次登录 Studio 仍需已注册的 user/admin API Key,后续授权可以复用 Studio 身份。---@@ -91,6 +80,8 @@ Studio 侧边栏底部的"**OAuth 验证**"入口会直接打开这个跨设备## 快速验证(HTTP,仅本地)+沿用上面的 API Key 模式和已注册 user/admin key,不要用无认证 Dev 身份完成 consent。+最快确认 OAuth 装配正确的方式是在 `127.0.0.1` 跑一遍。MCP SDK 接受`http://127.0.0.1` 与 `http://localhost` 作为 issuer URL 而无需 HTTPS — 但Claude.ai / Claude Desktop 等线上客户端**只接受公网 HTTPS**,所以这个模式只@@ -139,11 +130,10 @@ OAuth 2.1 对非 localhost 的 issuer **强制要求 HTTPS**。CDN 的配置方法。简要步骤:1. 按[公网访问指南 § 添加 HTTPS](12-public-access.md#添加-https公网访问)- 配置好 `https://your-domain.com`,使 1934 端口走 TLS。+ 让 `https://your-domain.com` 通过 443 提供 HTTPS,代理转发到 OpenViking 端口(默认 1933)。2. 启用 OAuth:`ov.conf` 里 `{ "oauth": { "enabled": true } }`。-3. 重启:`docker compose restart openviking`。-4. 在 `.env` 设置 `OPENVIKING_PUBLIC_BASE_URL=https://your-domain.com`- (服务端用它作为 OAuth 元数据和 `WWW-Authenticate` 的 issuer)。+3. 在 Compose 的 `.env` 设置 `OPENVIKING_PUBLIC_BASE_URL=https://your-domain.com`。+4. 运行 `docker compose up -d` 应用容器环境变量变化。仅执行 `restart` 不会加载 `.env` 的修改。HTTPS + OAuth 就绪后,按下面的方式接入客户端。@@ -180,8 +170,7 @@ claude mcp add --transport http openviking https://my.ov/mcp \### ChatGPT (Codex / Plus / Enterprise)-Settings → Beta features → Custom Connectors。输入 MCP URL,ChatGPT 通过-`/.well-known/...` 文档自动发现 OAuth 端点,走相同的 authorize → token 流程。+在 ChatGPT 中通过开发者模式创建自定义 App,填入服务的 MCP URL,并完成浏览器中的 OAuth 授权。可用入口和管理权限因套餐与工作区设置而异,按 [OpenAI 官方接入说明](https://help.openai.com/en/articles/12584461-developer-mode-and-mcp-apps-in-chatgpt)操作。Codex 的插件与 MCP 配置见 [Codex 集成](../agent-integrations/04-codex.md)。### Cursor@@ -194,39 +183,53 @@ Cursor 看到 401 + `WWW-Authenticate: Bearer resource_metadata=...` 后会自不需要真实 MCP 客户端:+将 `OV_OAUTH_ORIGIN` 设为服务 HTTPS origin,`API_KEY` 设为已注册用户的 user/admin key。示例需要 `curl`、`jq`、OpenSSL 和 Python 3;按注释从浏览器复制 pending ID 和授权码。+```bash-# 1. 注册客户端-curl -X POST -H "Content-Type: application/json" \- -d '{"redirect_uris":["http://127.0.0.1:9999/cb"],"client_name":"test","token_endpoint_auth_method":"none"}' \- https://my.ov/register-# → {"client_id":"...", ...}--# 2. PKCE 对-VERIFIER=$(openssl rand -base64 64 | tr -d '=+/' | head -c 64)-CHALLENGE=$(printf "%s" "$VERIFIER" | openssl dgst -sha256 -binary | basenc --base64url | tr -d '=')--# 3. 浏览器访问 authorize URL,页面会显示 6 字符码-echo "https://my.ov/authorize?response_type=code&client_id=$CID&redirect_uri=http://127.0.0.1:9999/cb&code_challenge=$CHALLENGE&code_challenge_method=S256&state=xyz"--# 4. 在 Studio consent 页确认(或直接 curl)-# - Studio 路径用 pending_id(authorize 页的 ?pending=... 参数)-# - 跨设备路径用 6 字符 display_code-curl -X POST -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \- -d '{"pending_id":"<pending-id-from-authorize-url>","decision":"approve"}' \- https://my.ov/api/v1/auth/oauth-verify--# 5. 浏览器自动 302 到 /cb?code=ovac_...&state=xyz,记下 code--# 6. 用 auth code 换 token-curl -X POST \- -d "grant_type=authorization_code&code=ovac_...&client_id=$CID&code_verifier=$VERIFIER&redirect_uri=http://127.0.0.1:9999/cb" \- https://my.ov/token-# → {"access_token":"ovat_...","refresh_token":"ovrt_...","expires_in":3600}--# 7. 用 access token 调 MCP-curl -X POST -H "Authorization: Bearer ovat_..." \- -d '{"jsonrpc":"2.0","method":"tools/list","id":1}' \- https://my.ov/mcp+OV_OAUTH_ORIGIN=https://my.ov+CID=$(curl -fsS "$OV_OAUTH_ORIGIN/register" \+ -H "Content-Type: application/json" \+ -d '{"redirect_uris":["http://127.0.0.1:9999/cb"],"client_name":"test","token_endpoint_auth_method":"none"}' \+ | jq -er '.client_id')++VERIFIER=$(python3 -c 'import secrets; print(secrets.token_urlsafe(48))')+CHALLENGE=$(printf "%s" "$VERIFIER" | openssl dgst -sha256 -binary | openssl base64 -A | tr '+/' '-_' | tr -d '=')+STATE=$(python3 -c 'import secrets; print(secrets.token_urlsafe(16))')+printf '%s\n' "$OV_OAUTH_ORIGIN/authorize?response_type=code&client_id=$CID&redirect_uri=http://127.0.0.1:9999/cb&code_challenge=$CHALLENGE&code_challenge_method=S256&state=$STATE"++# Open the URL in a browser; approve in Studio or supply its pending ID here.+curl -fsS "$OV_OAUTH_ORIGIN/api/v1/auth/oauth-verify" \+ -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \+ -d '{"pending_id":"<pending-id>","decision":"approve"}'++# From the redirect URL, verify state matches $STATE and copy the code.+# The callback page need not load for this manual test.+AUTH_CODE='<code-from-callback-url>'+TOKEN_RESPONSE=$(curl -fsS "$OV_OAUTH_ORIGIN/token" \+ --data-urlencode "grant_type=authorization_code" \+ --data-urlencode "code=$AUTH_CODE" \+ --data-urlencode "client_id=$CID" \+ --data-urlencode "code_verifier=$VERIFIER" \+ --data-urlencode "redirect_uri=http://127.0.0.1:9999/cb")+ACCESS_TOKEN=$(printf '%s' "$TOKEN_RESPONSE" | jq -er '.access_token')++curl -fsS "$OV_OAUTH_ORIGIN/mcp" \+ -H "Authorization: Bearer $ACCESS_TOKEN" \+ -H "Content-Type: application/json" \+ -H "Accept: application/json, text/event-stream" \+ -d '{"jsonrpc":"2.0","method":"initialize","id":1,"params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"manual-test","version":"1"}}}'++curl -fsS "$OV_OAUTH_ORIGIN/mcp" \+ -H "Authorization: Bearer $ACCESS_TOKEN" \+ -H "Content-Type: application/json" \+ -H "Accept: application/json, text/event-stream" \+ -d '{"jsonrpc":"2.0","method":"notifications/initialized"}'++curl -fsS "$OV_OAUTH_ORIGIN/mcp" \+ -H "Authorization: Bearer $ACCESS_TOKEN" \+ -H "Content-Type: application/json" \+ -H "Accept: application/json, text/event-stream" \+ -d '{"jsonrpc":"2.0","method":"tools/list","id":2}'```---@@ -266,13 +269,12 @@ curl -X POST -H "Authorization: Bearer ovat_..." \| Authorization code | `secrets.token_urlsafe(40)` | `ovac_` | 5 分钟 | SQLite (SHA-256 索引) || Display code(页面) | 6 字符(去 O/0/I/1) | — | 10 分钟 | SQLite (`oauth_pending_authorizations`) |-所有 token 都是 opaque(不签发 JWT),服务端**没有任何加密密钥需要管理**。+所有 token 都是 opaque(不签发 JWT),OAuth 不需要 JWT 签名密钥。部署仍需管理 API 凭证、保护 token 数据库,以及保留已配置的存储加密密钥。每次请求按 SHA-256 哈希查 SQLite,撤销 token 是一次 `UPDATE`。### Token 与身份-每个 token 在签发时绑定一个 `(account_id, user_id, role)` 三元组。OAuth-token 拥有的权限 = 颁发它时所用 API Key 的权限,**不更多也不更少**。+每个 token 在签发时绑定一个 `(account_id, user_id, role)` 三元组。token 使用该身份,访问时仍受当前资源 ACL 和 key 有效性约束。用户后续提升角色不会升级已签发 token,降权检查可能拒绝旧 token。OAuth token 不能用于批准新的 OAuth 客户端。### OAuth 生命周期 ≤ 授权 Key 生命周期@@ -318,8 +320,7 @@ MCP SDK 拒绝非 `127.0.0.1` / `localhost` 的 `http://` issuer。三选一:### 跨设备 fallback 页有码,但 `/studio/oauth/verify` 报 "Invalid code"码是 6 字符**全大写**,传输时区分大小写。`/studio/oauth/verify` 的输入框会-自动转大写。如果手-动输入,注意字母与数字的混淆字符(字母表已经排除了 `O`、`0`、`I`、`1`)。+自动转大写。如果手动输入,注意字母与数字的混淆字符(字母表已经排除了 `O`、`0`、`I`、`1`)。### Refresh 一次后再用旧 token 被拒@@ -347,3 +348,5 @@ curl -i https://my.ov/mcp -d '{}' -H 'Content-Type: application/json' | grep -i- [RFC 7591 — Dynamic Client Registration](https://datatracker.ietf.org/doc/html/rfc7591)- [RFC 7636 — PKCE](https://datatracker.ietf.org/doc/html/rfc7636)- [OpenViking MCP 集成指南](06-mcp-integration.md)++[Compose restart 的配置加载行为](https://docs.docker.com/reference/cli/docker/compose/restart/)。diff --git a/docs/zh/guides/12-public-access.md b/docs/zh/guides/12-public-access.mdindex 5f607715c..e94443137 100644--- a/docs/zh/guides/12-public-access.md+++ b/docs/zh/guides/12-public-access.md@@ -92,13 +92,15 @@ volumes:caddy_config:```+只通过代理提供公网入口时,删除 OpenViking 的宿主机端口映射,或绑定到 `127.0.0.1`。旧的 `1934:1934` 映射也应删除或仅绑定 localhost。Caddy 仍可通过 Compose 网络访问 `openviking:1933`。+### 4. 启动```bashdocker compose up -d```-首次 HTTPS 请求触发 ACME 证书签发,后续使用缓存。Caddy 自动续期。+Caddy 会为配置的域名申请并续期证书,签发失败时查看 Caddy 日志。按请求触发签发需要单独配置,见 [Caddy 自动 HTTPS](https://caddyserver.com/docs/automatic-https)。### 5. 验证@@ -120,9 +122,12 @@ OV 服务的 1933 端口。### nginx+示例假设 nginx 与 OpenViking 在同一宿主机。超时值需按最长请求调整;关闭响应缓冲可及时传递 MCP 流式响应,见 [nginx 代理缓冲](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering)。+```nginxserver {- listen 443 ssl http2;+ listen 443 ssl;+ http2 on; # nginx < 1.25.1:删除此行,改用 `listen 443 ssl http2;`server_name ov.your-domain.com;ssl_certificate /etc/letsencrypt/live/ov.your-domain.com/fullchain.pem;@@ -130,6 +135,9 @@ server {location / {proxy_pass http://127.0.0.1:1933;+ proxy_http_version 1.1;+ proxy_buffering off;+ proxy_read_timeout 300s;proxy_set_header Host $host;proxy_set_header X-Forwarded-Proto $scheme;proxy_set_header X-Forwarded-Host $host;@@ -153,27 +161,28 @@ ov.your-domain.com {### Cloudflare / CDN-CDN 源站指向 `http://your-server-ip:1933`。设置-`OPENVIKING_PUBLIC_BASE_URL=https://ov.your-domain.com` 让服务端知道自己的-公网地址。确保 CDN 转发 `Host`、`X-Forwarded-Proto`、`X-Forwarded-Host` 头。+配置验证证书的 HTTPS 源站或私有源站隧道,限制源站只接受代理访问。设置 `OPENVIKING_PUBLIC_BASE_URL=https://ov.your-domain.com`,转发公网 host/protocol 请求头,并核对代理能否保留流式响应、允许所需请求时长和上传大小。## 告诉服务端公网 URL-OAuth 元数据、`WWW-Authenticate` 头、资源 URL 都需要包含公网 origin。-解析顺序(**优先级从高到低**):+OAuth 元数据和 `WWW-Authenticate` 头需要包含公网 origin。OAuth 地址辅助函数的解析顺序(**优先级从高到低**):1. `OPENVIKING_PUBLIC_BASE_URL` 环境变量2. `ov.conf` 里的 `oauth.issuer`3. `X-Forwarded-Proto` + `X-Forwarded-Host` 请求头4. 请求的 `Host` 头-在反代后面,务必显式设置选项 1:+授权服务器的 issuer 在启动时确定:环境变量 → `oauth.issuer` → `http://127.0.0.1:1933`,转发请求头不会改变它。受保护资源元数据的 `authorization_servers` 优先使用显式配置的 `oauth.issuer`,即使已设置环境变量。++MCP 上传 URL 的顺序是:环境变量 → `server.public_base_url` → 转发 host/protocol → `Host` → 监听地址回退。统一设置环境变量可让两者使用同一地址;若同时设置 `oauth.issuer`,保持 origin 一致。++使用反向代理时,在服务端进程环境中设置选项 1:```bashexport OPENVIKING_PUBLIC_BASE_URL="https://ov.your-domain.com"```-或者 `ov.conf`:+Compose 部署应修改 `.env` 并运行 `docker compose up -d`,在其他 shell 中执行 `export` 不会更新已有容器。只配置 OAuth issuer 时也可以使用 `ov.conf`:```jsonc{@@ -188,7 +197,7 @@ export OPENVIKING_PUBLIC_BASE_URL="https://ov.your-domain.com"`docker compose up` 默认在 1934 端口启一个 Caddy 反代,单纯 `reverse_proxyopenviking:1933`,**仅为兼容已经书签到 1934 的旧部署保留**。新部署直接连-1933 即可,没有任何路由价值;不需要这个入口可以从 `docker-compose.yml` 注释+私网中的 1933,公网客户端使用上面的 HTTPS 入口;不需要这个入口可以从 `docker-compose.yml` 注释掉 caddy 服务和 1934 端口映射。## 相关文档diff --git a/docs/zh/guides/13-multi-write-storage.md b/docs/zh/guides/13-multi-write-storage.mdindex 3bdead0ff..5f95c0de3 100644--- a/docs/zh/guides/13-multi-write-storage.md+++ b/docs/zh/guides/13-multi-write-storage.md@@ -1,6 +1,6 @@# 多写存储指南-本指南介绍如何配置 OpenViking 的多写存储能力。多写存储允许一个 primary 后端同时复制写入多个 backup 后端,用于高可用、跨区域副本、读加速和存储迁移。+本指南介绍如何配置 OpenViking 的多写存储能力。多写存储允许一个 primary 后端同时复制写入多个 backup 后端,用于保存副本(包括跨地域副本)、配置读路由和存储迁移。primary 故障时不会自动提升 backup。多写能力位于 RAGFS 内部。OpenViking 的 Python SDK、HTTP API 和 CLI 使用方式保持不变。@@ -13,7 +13,7 @@## 最小配置-下面示例使用本地目录作为 primary,并把写入复制到另一个本地目录。+将下列片段合并到现有模型和服务端配置。下面示例使用本地目录作为 primary,并把写入复制到另一个本地目录。```json{@@ -94,12 +94,12 @@### S3 兼容存储注意事项-使用 S3 兼容服务(MinIO、RustFS、Ceph 等)时,`s3` 段需要额外配置以下字段:+按实际 endpoint 和对象存储配置选择以下 `s3` 字段:| 字段 | 是否必填 | 说明 || --- | --- | --- |-| `use_path_style` | 大多数 S3 兼容服务必填 | 设置为 `true` 使用路径风格 URL(`http://host/bucket/key`)。大多数 S3 兼容服务需要此配置。 |-| `directory_marker_mode` | S3 兼容服务必填 | **必须显式设置为 `"none"`**。如果不配置,RAGFS Rust binding 启动时会报 `AGFSConfigError: invalid directory_marker_mode: null` 并静默崩溃。 |+| `use_path_style` | 可选,默认 `true` | 使用路径风格 URL(`http://host/bucket/key`),按 endpoint 支持的方式选择。 |+| `directory_marker_mode` | backup 项必填 | `none` 不创建标记,`empty` 创建零字节标记,`nonempty` 创建带内容的标记。每个 S3 backup 项都要显式设置;省略时启动会报 `AGFSConfigError: configuration error: invalid directory_marker_mode: null (valid: none, empty, nonempty)`。 || `use_ssl` | 可选 | HTTP 端点(如 `http://localhost:9000`)需要设置为 `false`。 |**S3 兼容存储最小示例(RustFS/MinIO):**@@ -121,16 +121,14 @@}```-> **为什么需要 `directory_marker_mode`?**->-> S3 兼容存储服务对"目录"的处理方式与 AWS S3 不同。RAGFS Rust binding 必须知道创建目录时是否需要写入目录标记对象。合法取值为 `"none"`、`"empty"` 和 `"nonempty"`。对于不使用目录标记的 S3 兼容服务(RustFS、MinIO、Ceph 等),设置为 `"none"`。如果省略,Rust binding 默认值为 `null`(不合法),导致服务端在启动时静默崩溃,报错 `AGFSConfigError: invalid directory_marker_mode: null`。+上面的示例使用 `none`,适用于不使用目录标记的 S3 兼容服务(RustFS、MinIO、Ceph 等)。backup 项的这个字段没有默认值;顶层 `storage` 的 S3 配置默认使用 `empty`。### Docker 网络配置在 Docker 中运行 OpenViking 并配置同主机的 S3 备份时,需要注意:- **Linux Docker**:使用 `--network host` 或宿主机局域网 IP。Docker bridge 网络可通过网关 IP(如 `172.17.0.1:9000`)访问宿主机局域网。-- **macOS/Windows Docker Desktop**:`--network host` **不支持**。S3 端点使用 `host.docker.internal`(映射为宿主机的 localhost),或使用宿主机局域网 IP。+- **macOS/Windows Docker Desktop**:从容器访问宿主机时可使用 `host.docker.internal`。Docker Desktop 4.34 及之后的版本也提供可选的 host networking,启用方式见 [Docker 官方说明](https://docs.docker.com/engine/network/drivers/host/)。如果启用 S3 备份后服务静默崩溃,请优先排查 Docker 网络。RAGFS Rust binding 在容器内无法访问 S3 端点时会报 `dispatch failure` 错误。@@ -154,7 +152,7 @@- primary 写入成功后立即返回。- backup 写入在后台执行。- 写入延迟低。-- backup 可能短暂落后。+- 后端不可用期间,落后时间没有固定上限,需观察同步和恢复状态。适合:@@ -172,7 +170,7 @@"sync_type": "sync","write_ack_count": 1,"write_ack_timeout_ms": 5000,- "items": []+ "items": [{"name": "local-backup", "backend": "local", "local": {"workspace": "./backup-data"}}]}}```@@ -206,6 +204,7 @@ backup 默认不参与读取。要让 backup 服务读取,需要显式配置 `"name": "cache-backend","backend": "memfs","operations": [+ {"operation": "write", "priority": 0},{"operation": "read","priority": 10@@ -378,7 +377,7 @@ ov write viking://resources/multiwrite-check.txt \ov read viking://resources/multiwrite-check.txt```-如果使用本地 backup,可以直接检查 backup 目录中是否出现对应文件。生产环境更推荐使用系统健康检查和同步状态命令。+如果使用本地 backup,可以直接检查 backup 目录中是否出现对应文件。还应运行 `ov system backend sync-status viking://resources`,逐个核对预期副本。普通读取成功可能只读到了 primary,不能证明复制完成。## 常见问题@@ -389,6 +388,7 @@ backup 默认只参与写入,不参与读取。需要在 backup 上显式配```json{"operations": [+ {"operation": "write", "priority": 0},{"operation": "read","priority": 10@@ -403,7 +403,7 @@ backup 默认只参与写入,不参与读取。需要在 backup 上显式配### 异步模式下能否保证立即读到 backup 的最新数据?-不能。异步模式只保证最终一致。需要强读一致时,应让读取回退到 primary,或避免让可能滞后的 backup 参与读路由。+不能。后台重试需要后端恢复可用才能追上进度。backup 返回旧数据但读取成功时,不会自动回退 primary;需要最新数据时,应让读路由使用 primary。### 内部元数据文件会出现在用户列表里吗?diff --git a/docs/zh/guides/14-ragfs-cache.md b/docs/zh/guides/14-ragfs-cache.mdindex e3724e7d1..318ffd301 100644--- a/docs/zh/guides/14-ragfs-cache.md+++ b/docs/zh/guides/14-ragfs-cache.md@@ -59,6 +59,11 @@ openviking-server doctor```bashredis-server+```++在另一个终端启动 OpenViking:++```bashopenviking-server --config ~/.openviking/ov.conf```@@ -82,7 +87,7 @@ openviking-server将 `storage.agfs.pathlock.provider` 设为 `cache`,即可通过共享 RedisCacheRuntime 协调多个 OpenViking 进程的路径锁。`pathlock.namespace`为必填项,同一 OpenViking 部署的所有进程必须使用相同值。Cache PathLock-只支持内置 Redis Provider。+只支持内置 Redis Provider。独立部署应使用不同 namespace;分布式路径锁不会解除 CachedFileSystem 的单写者前提。Redis HASH key 按逻辑路径 scope 拆分:@@ -155,7 +160,7 @@ Provider 发布物应注明 ABI 版本、目标 OS/CPU、最低运行时版本| 参数 | 类型 | 默认值 | 说明 ||------|------|--------|------|| `provider` | str | `"filesystem"` | `filesystem`、`memory` 或 `cache` |-| `namespace` | str 或 null | `null` | `provider=cache` 时必填的 OpenViking 实例名 |+| `namespace` | str 或 null | `null` | `provider=cache` 时必填,同一部署的协作进程共享此命名空间 || `lock_expire_secs` | float | `30.0` | 锁 stale 超时;不得小于 `1.0` |Redis 配置:@@ -322,4 +327,6 @@ RAGFS 会自动绕过不适合缓存的路径:4. 先缓存摘要文件和 raw `read_dir`,再扩展到更多普通小文件。5. 将锁、控制面和权限敏感路径加入 `bypass_prefixes`。-一句话总结:RAGFS 缓存负责“按文件系统语义正确失效”,Provider 负责“把缓存对象放在哪里”。只要 backend 是事实来源,缓存命中就必须先通过 envelope 和 generation 校验。+RAGFS 缓存负责按文件系统语义失效,Provider 负责决定缓存对象放在哪里。backend 仍是事实来源,缓存命中必须先通过 envelope 和 generation 校验才会返回。++启用后,将重复读取和文件更新后的失效行为与基线比较。确认返回内容保持最新,再判断命中率提升是否带来收益。diff --git a/docs/zh/guides/15-snapshot.md b/docs/zh/guides/15-snapshot.mdindex d6df37a2f..d08456fd1 100644--- a/docs/zh/guides/15-snapshot.md+++ b/docs/zh/guides/15-snapshot.md@@ -1,6 +1,6 @@# 多版本管理(快照)指南-本指南介绍如何启用并使用 OpenViking 的多版本管理(快照)能力。多版本管理在 VikingFS 之上提供基于 Git 的 `commit`/`log`/`show`/`restore` 原语,让你把账号下的资源树保存成一系列不可变快照,随时回溯历史、对比版本,并把工作区恢复到任意历史状态。+快照将指定范围内的文件树保存为不可变版本。使用 `commit` 保存、`log` 查看历史、`show` 读取旧版文件、`restore` 恢复已保存的内容。快照不保存未提交的改动或提交时被排除的文件,ACL 和向量索引也不保存历史版本。恢复会比较来源快照与 HEAD,因此受影响路径上的未提交改动可能被覆盖。多版本管理由内嵌在 Rust RAGFS 层的 [gitoxide](https://github.com/Byron/gitoxide) 驱动,以 `account_id` 为粒度维护一个逻辑 Git 仓库(每个账号一个仓库),对调用方完全透明——你无需手动执行任何 `git` 命令。@@ -126,7 +126,7 @@ Filesystem 对缺失目标跳过锁,避免锁文件创建目标目录或缺失| `git.s3.use_path_style` | `true` | `true` 用 path-style 寻址(MinIO 等);`false` 用 virtual-host 寻址(TOS 等) || `git.s3.cas_mode` | `native` | 引用 CAS 模式。`native` 使用 S3 条件写(If-Match) |-修改配置后,重启 OpenViking 服务(或重新初始化 SDK 客户端)使其生效。+修改服务端配置后,重启 OpenViking 服务。重新初始化 HTTP SDK 客户端不会重新加载服务端配置。> 仓库中提供了可直接参考的完整示例:[ov.conf.git-local.example](https://github.com/volcengine/OpenViking/blob/main/examples/snapshot/ov.conf.git-local.example) 与 [ov.conf.git-s3-tos.example](https://github.com/volcengine/OpenViking/blob/main/examples/snapshot/ov.conf.git-s3-tos.example)。@@ -151,7 +151,7 @@ data/ # storage.workspace- `.ovgit` 是内部数据目录,**不会**通过 `viking://` 暴露,用户在文件系统 API(`ls`/`read` 等)中看不到也无法修改它。- 它与 Git 的标准对象库布局一致(内容寻址的 `objects/`、loose 引用的 `refs/`),但由 OpenViking 自动管理,**无需也不应**手动运行 `git` 命令去操作它。-- 备份或迁移工作区时,把 `.ovgit` 一并复制即可保留完整的版本历史。+- 物理备份或迁移前先暂停写入,在同一检查点复制工作区和 `.ovgit`。[OVPack 导出](09-ovpack.md) 保存当前内容,不包含快照历史。- 选择 `s3` 后端时,不会创建本地 `.ovgit` 目录,数据改为存放在 bucket 的 `{prefix}/{account}/...` 键下。## 使用方法@@ -163,12 +163,13 @@ data/ # storage.workspace快照方法挂在 `client.snapshot.*` 命名空间下。```python+from uuid import uuid4from openviking_sdk import SyncHTTPClientclient = SyncHTTPClient(url="http://localhost:1933", api_key="your-key")client.initialize()-root = "viking://resources/my_project"+root = f"viking://resources/snapshot-demo-{uuid4().hex[:8]}"# 1. 写入初始内容并提交 v1client.write(@@ -176,7 +177,10 @@ client.write(content="# Guide\n\nv1 content\n",mode="create",)+print(client.wait_processed(timeout=120))v1 = client.snapshot.commit(message="v1 initial import", paths=[root])+if not v1.get("commit_oid"):+ raise RuntimeError(f"No snapshot created: {v1}")print("v1:", v1["commit_oid"])# 2. 修改后再提交 v2@@ -185,6 +189,7 @@ client.write(content="# Guide\n\nv2 content\n",mode="replace",)+print(client.wait_processed(timeout=120))v2 = client.snapshot.commit(message="v2 update", paths=[root])# 3. 查看历史@@ -194,12 +199,14 @@ for c in client.snapshot.log(limit=10, paths=[root]):# 4. 读取历史文件内容print(client.snapshot.show(v1["commit_oid"], path=f"{root}/guide.md"))-# 5. 把工作区恢复到 v1(会在 v2 之上生成一个新的“正向”提交)-client.snapshot.restore(project_dir=root, source_commit=v1["commit_oid"], message="restore to v1")+# 5. 预览恢复计划,暂不修改文件+print(client.snapshot.restore(project_dir=root, source_commit=v1["commit_oid"], dry_run=True))client.close()```+示例每次创建新目录,在每次提交前等待后台处理结束,最后只预览恢复计划。检查处理结果中的错误;队列结束不代表索引一致。检查计划后,重新连接客户端,使用相同的 `project_dir` 和 `source_commit`,并设置 `dry_run=False` 才会执行恢复。+### CLICLI 子命令位于 `ov snapshot` 下:@@ -211,17 +218,23 @@ ov snapshot commit -m "v1 initial import" --paths viking://resources/my_project# 回溯历史(最新在前)ov snapshot log --paths viking://resources/my_project --limit 10 -o json+# 填入上面返回的 commit_oid+COMMIT_OID="replace-with-commit-oid"+# 读取历史文件内容-ov snapshot show <commit_oid> --path viking://resources/my_project/guide.md+ov snapshot show "$COMMIT_OID" --path viking://resources/my_project/guide.md# 读取某个提交中的文件内容(默认输出到 stdout,可用 --out-file 写入本地文件)-ov snapshot show <commit_oid> --path viking://resources/my_project/guide.md --out-file ./guide.md+ov snapshot show "$COMMIT_OID" --path viking://resources/my_project/guide.md --out-file ./guide.md++# 预览会改动的文件+ov snapshot restore "$COMMIT_OID" viking://resources/my_project --dry-run -o json+```-# 把目录恢复到某个历史快照(位置参数依次为 <source_commit> <project_dir>)-ov snapshot restore <commit_oid> viking://resources/my_project -m "restore to v1" -o json+确认预览后,执行恢复:-# 先预演,确认会改动哪些文件-ov snapshot restore <commit_oid> viking://resources/my_project --dry-run -o json+```bash+ov snapshot restore "$COMMIT_OID" viking://resources/my_project -m "restore to v1" -o json```### HTTP API@@ -241,19 +254,21 @@ curl -X GET "http://localhost:1933/api/v1/snapshot/log?branch=main&limit=10&pathcurl -X GET "http://localhost:1933/api/v1/snapshot/show?target_ref=<commit_oid>&path=viking://resources/my_project/guide.md" \-H "X-API-Key: your-key"-# 恢复+# 预览恢复curl -X POST "http://localhost:1933/api/v1/snapshot/restore" \-H "Content-Type: application/json" \-H "X-API-Key: your-key" \- -d '{"project_dir": "viking://resources/my_project", "source_commit": "<commit_oid>", "message": "restore to v1"}'+ -d '{"project_dir": "viking://resources/my_project", "source_commit": "<commit_oid>", "message": "restore to v1", "dry_run": true}'```+将 `<commit_oid>` 替换为保存的提交 ID。上面的恢复请求只预览;确认执行时再把 `dry_run` 改为 `false`。+## 重要语义:正向恢复`restore` 采用**正向恢复(forward-commit)**:它读取 `source_commit` 的内容,把差异写回工作区,并在**当前 HEAD 之上生成一个新的提交**。因此:- 新提交的父提交是恢复操作发生前的 HEAD,**不是** `source_commit`。-- HEAD 始终单调向前推进,**历史永远不会被改写或丢失**——回到旧版本本身也是一次新的提交。+- 所选范围内的来源快照与 HEAD 文件树存在差异时,恢复会追加提交,不改写历史。两棵树相同时返回 `result: "noop"`,不创建提交,即使工作区文件已有变化。快照存储本身仍需要备份。- `restore` 只影响 `project_dir`(省略时为整棵账号树)范围内的文件,范围之外的文件保持不变。## 使用 `.ovgitignore` 排除文件@@ -297,7 +312,7 @@ client.snapshot.delete_gitignore()```pythonv = client.snapshot.commit(message="with ignore", paths=["viking://resources/my_project"])-print(v["result"], v.get("ignored")) # created, 1+print(v["result"], v.get("ignored"))```### CLI@@ -334,7 +349,7 @@ curl -X DELETE "http://localhost:1933/api/v1/snapshot/ignore" \## 注意事项-- 修改 `git` 配置后必须重启服务 / 重新初始化客户端才能生效。+- 修改服务端 `git` 配置后,重启服务使其生效。- 启用 `s3` 后端时,`git.s3.bucket` 与 `git.s3.region` 为必填项,缺失会导致初始化失败。- 恢复操作如涉及向量副作用(写入/删除文件),响应会返回一个 `task_id`,可通过 `GET /api/v1/tasks/{task_id}` 轮询后台向量重建进度(参见 [系统指南](05-observability.md) 与 [API 概览](../api/01-overview.md))。- `.ovgitignore` 内容过大(超过 64 KiB)或包含 `!` 取反、反斜杠转义等不支持语法时,`commit` 会失败并报 `invalid operation` 错误;写入时(`set_gitignore`)会预先校验大小。