Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

docs: clarify OpenClaw assemble, PathLock and snapshot semantics (#5092)

Zayn Jarvis35d76c8fd305
diff --git a/docs/en/agent-integrations/03-openclaw.md b/docs/en/agent-integrations/03-openclaw.md
index cd269e020..fb8fc046e 100644
--- a/docs/en/agent-integrations/03-openclaw.md
+++ b/docs/en/agent-integrations/03-openclaw.md
@@ -82,6 +82,42 @@ openclaw openviking setup --base-url http://your-server:1933 --api-key sk-xxx --
New configuration should use `sender`; existing `peer_role=person` configurations remain compatible and are treated as `sender`. OpenViking initializes the managed `peers/` container for every user, so `none` means that no concrete `peers/<peer_id>/memories` subtree is used. Actor-peer recall includes shared user memory plus the current peer memory, and changing the scope does not move existing memories.
+## How assemble builds context
+
+The plugin occupies OpenClaw's `contextEngine` slot. It handles session history, long-term memory recall, and the pending user input separately; `assemble()` returns context for the current model request. It does not persist assembled summaries or recalled context to the OpenClaw session transcript, nor does it append messages to the OV session through this call. The host may update the current turn's in-memory messages with the result; that is separate from writing persistent conversation history.
+
+Main assemble prepares history at the start of each new turn. The plugin identifies this call by the presence of at least one of `prompt`, `availableTools`, or `citationsMode`.
+
+### Main assemble: history and pending input
+
+The main branch calls `getSessionContext(tokenBudget)` and builds:
+
+```text
+summaryMessage = { role: "user", content: "[Session History Summary]\n" + latest_archive_overview }
+messages = [summaryMessage] + OV active messages
+systemPromptAddition = Session Context Guide (when archives exist) + recalled context (when available)
+```
+
+`latest_archive_overview` is the summary text returned by the server; `[Session History Summary]` is the literal heading prepended by the plugin. This synthetic user message is inserted only when the overview is nonempty. Active messages provide recent uncompressed conversation. The host adds the pending `prompt` to the turn. The plugin uses it for recall without appending a second copy to the returned history. Recalled context belongs to this request and is not directly captured as new conversation in OV.
+
+OV generates the overview through its server-side working-memory flow; the plugin reads the result. The server budgets active messages first and omits the overview if the remaining space is insufficient. `pre_archive_abstracts` is currently an empty array, so the response is not a complete archive index. Use `ov_archive_search` for original details.
+
+The plugin reserves output headroom, subtracts estimated guide and summary tokens, trims active messages from the oldest end, and normalizes provider message formats such as tool calls and results. It does not hard-truncate the summary to the calculated archive budget, so the partitions are not strict per-layer limits. A new recall block is omitted if it would push the total estimate above `tokenBudget`.
+
+The history branch falls back to host messages when OV has no data, has fewer messages than the host without an archive, produces an empty converted history, or fails to load. Main-branch recall can still run with a valid `prompt` and `autoRecall` enabled even when history passes through. Missing recall results or recall failures do not stop the conversation.
+
+### transformContext
+
+`transformContext` runs before each LLM call, whether the last message is a user message or a tool response. The best use of this hook in the OV integration has not yet been determined.
+
+### Capture and compaction
+
+- `ingest()` / `ingestBatch()` do not write messages. Regular capture uses `afterTurn`. For stable OpenClaw versions from 2026.9.3 onward, the plugin also captures completed turns delivered through `commitTurn`; older hosts and standalone runners use `afterTurn`. If the host version cannot be classified, `commitTurn` rejects acknowledgement to avoid confirming uncaptured data.
+- Capture removes injected context, converts text and tool messages, and writes them to the OV session. When `pending_tokens >= tokenBudget × commitTokenThresholdRatio`, it starts an asynchronous session commit. The default ratio is `0.5`; retention defaults to the most recent `10` messages, with `turn_budget` available as another policy. `pending_tokens` counts messages eligible for archiving under the server retention policy, not the entire model request.
+- `ownsCompaction: true` assigns compaction to the plugin. Normal `compact()` commits the OV session with `wait=true` and retention `0`, then reads the overview as its summary. The next main assemble rebuilds history from that summary and active messages. Bypassed sessions attempt to delegate compaction to the host; if the host bridge is unavailable, the plugin returns a skip result.
+
+A **session commit** archives conversation and processes memory. It is separate from a [snapshot commit](../guides/15-snapshot.md), which versions resource files.
+
## Verify
```bash
diff --git a/docs/en/api/11-snapshot.md b/docs/en/api/11-snapshot.md
index cd0f671bf..53f74a7e2 100644
--- a/docs/en/api/11-snapshot.md
+++ b/docs/en/api/11-snapshot.md
@@ -44,7 +44,7 @@ Existing nodes keep their current ACL after restore. Newly restored nodes inheri
- HTTP routes: [snapshot.py](https://github.com/volcengine/OpenViking/blob/main/openviking/server/routers/snapshot.py), prefix `/api/v1/snapshot`.
- SDK namespace: [client.py](https://github.com/volcengine/OpenViking/blob/main/sdk/python/openviking_sdk/client.py), exposed as `client.snapshot.*`.
-- Underlying semantics: `commit` / `restore` / `show` / `log` / `diff` in [viking_fs.py](https://github.com/volcengine/OpenViking/blob/main/openviking/storage/viking_fs.py).
+- Underlying semantics: `commit` / `restore` / `show` / `log` / `diff` in [_snapshot.py](https://github.com/volcengine/OpenViking/blob/main/openviking/storage/viking_fs/_snapshot.py).
- CLI: the `SnapshotCmd` in [main.rs](https://github.com/volcengine/OpenViking/blob/main/crates/ov_cli/src/main.rs), subcommands in [snapshot.rs](https://github.com/volcengine/OpenViking/blob/main/crates/ov_cli/src/commands/snapshot.rs).
## API Reference
@@ -53,6 +53,8 @@ Existing nodes keep their current ACL after restore. Newly restored nodes inheri
Save the current workspace state as a new snapshot.
+Partial commits preserve the previous snapshot outside the requested scope. After deleting a file or directory, include that URI or its parent in `paths` to record the deletion. A trailing `/` does not declare its type. Non-ROOT commits use Exact for existing files and Tree for existing directories; missing targets are unlocked with the filesystem lock provider and use Tree with the cache provider. Concurrent recreation of a missing target is not guaranteed to be fully captured; see [commit scope and concurrency](../guides/15-snapshot.md#commit-scope-and-concurrency).
+
**Parameters**
| Parameter | Type | Required | Default | Description |
diff --git a/docs/en/concepts/09-transaction.md b/docs/en/concepts/09-transaction.md
index ff8a8110f..3835a184e 100644
--- a/docs/en/concepts/09-transaction.md
+++ b/docs/en/concepts/09-transaction.md
@@ -1,6 +1,6 @@
# Path Locks and Crash Recovery
-OpenViking uses two simple primitives — **path locks** and **persistent queue recovery** — to protect the consistency of core write operations (`rm`, `mv`, `add_resource`, `session.commit`), ensuring that VikingFS, VectorDB, and QueueManager remain consistent even when failures occur.
+OpenViking uses two simple primitives — **path locks** and **persistent queue recovery** — to protect the consistency of core write operations (`rm`, `mv`, `add_resource`, `session.commit`), coordinating concurrent writes and resuming queued session work after a process restart. These primitives do not form an atomic transaction across VikingFS, VectorDB, and QueueManager.
## Design Philosophy
@@ -10,8 +10,8 @@ OpenViking is a context database where FS is the source of truth and VectorDB is
## Design Principles
-1. **Write-exclusive**: Path locks ensure only one write operation can operate on a path at a time
-2. **On by default**: All data operations automatically acquire locks; no extra configuration needed
+1. **Write-exclusive**: Different owners participating in the protocol cannot hold conflicting path locks simultaneously
+2. **On by default**: Protected writes acquire locks by default; ordinary reads and low-level mkdir do not automatically acquire locks
3. **Lock as protection**: LockContext acquires locks on entry, releases on exit — no undo/journal/commit semantics
4. **Only session_memory needs crash recovery**: a persistent `session_commit` queue resumes Phase 2 after a process crash
5. **Queue operations run outside locks**: SemanticQueue/EmbeddingQueue enqueue operations are idempotent and retriable
@@ -276,24 +276,47 @@ The lock mechanism uses two lock types to handle different conflict patterns:
- **EXACT (E)**: Locks one concrete path. It can protect files, directory names, and not-yet-created target paths. Blocks if any ancestor holds a TreeLock.
- **TREE (T)**: Used for directory delete, directory move, resource lifecycle protection, and similar subtree-level operations. Logically covers the entire subtree but stores one provider token for the root path. Conflict checks cover descendants and Tree-locked ancestors within the provider's scope. The filesystem provider may create a missing target directory to place its lock file.
+### Path scope and target type
+
+Exact and Tree describe an operation's scope; file, directory, and missing describe the target's current state. These are independent. Locks protect path names, including names that do not currently exist.
+
+The following conflicts apply to different owners within the same provider scope:
+
+| Held lock | New request | Conflict |
+| --- | --- | --- |
+| Exact(`/docs/a.md`) | Exact or Tree(`/docs/a.md`) | Yes |
+| Exact(`/docs`) | Exact(`/docs/a.md`) | No |
+| Tree(`/docs`) | Exact or Tree(`/docs/a.md`) | Yes |
+| Exact(`/docs/a.md`) | Tree(`/docs`) | Yes |
+| Tree(`/docs/a.md`) | Exact(`/docs/b.md`) | No |
+
+`Tree(/docs/a.md)` does not expand to `Tree(/docs)`. Likewise, Exact on a directory name does not protect its descendants; recursive deletion needs Tree.
+
+Locks coordinate participating operations. The low-level `PathLockWrappedFS` uses Exact for create, write, truncate, and non-recursive remove, and Tree for remove_all. File rename uses Exact on both paths; directory rename uses Tree on the source and Exact on the destination. Read, stat, directory listing, and mkdir pass through, though higher layers may hold their own locks. The operating system does not block I/O that bypasses this protocol.
+
## Lock Mechanism
### Filesystem Provider Lock Protocol
-Lock file paths:
+The caller chooses the lock type; the resolver chooses the token location from the target's state:
+
+| Target state | Exact token | Tree token |
+| --- | --- | --- |
+| Existing file `/docs/a` | `/docs/.exact.ovlock.a.<hash>`, containing E | Same sidecar, containing T |
+| Existing directory `/docs/a` | `/docs/a/.path.ovlock`, containing E | Same file inside the directory, containing T |
+| Missing `/docs/a` | Parent sidecar, containing E | Create the target directory, then write `.path.ovlock` inside it, containing T |
+
+A sidecar sits beside its target but represents only that target, not its parent directory. `<hash>` is a SHA-1 prefix of the full backend path, not the business file's contents.
+
+`.exact.ovlock.*` can contain a Tree token, and `.path.ovlock` can contain an Exact token. Filenames are part of the storage protocol, but do not alone identify the logical lock type. Token contents are:
```text
-TreeLock(path) -> {path}/.path.ovlock
-ExactPathLock(existing dir path) -> {path}/.path.ovlock
-ExactPathLock(file or missing path) -> {parent}/.exact.ovlock.<name>.<hash>
+{owner_id}:{time_ns}:{lock_type}
```
-Lock file content (Fencing Token):
-```
-{handle_id}:{time_ns}:{lock_type}
-```
+`lock_type` is `E` or `T`. This ownership token supports conflict checks, refresh, and conditional release; it does not imply storage-side fencing of every business write.
-Where `lock_type` is `E` (EXACT) or `T` (TREE).
+A lease records logical scope in `covered_paths` separately from token locations in `lock_paths`. An owned lease controls refresh, release, and handoff. A borrowed lease proves coverage by an existing lock without permission to release the outer lock.
### Cache Provider Lock Protocol
@@ -344,7 +367,7 @@ Timeout (default 0 = no-wait) raises LockAcquisitionError
```
loop until timeout (poll interval: 200ms):
- 1. Check if target directory is locked by another operation
+ 1. Check if target path is locked by another operation
- Stale lock? -> remove and retry
- Active lock? -> wait
2. Check all ancestor directories for TREE locks
@@ -354,8 +377,8 @@ loop until timeout (poll interval: 200ms):
- Missing target directory? -> treat as no descendant locks
- Stale lock? -> remove and retry
- Active lock? -> wait
- 4. Ensure the target directory exists; create it if missing
- 5. Write TREE (T) lock file (only one file, at the root path)
+ 4. Ensure the resolved token parent exists; this creates a missing target as a directory
+ 5. Write the TREE (T) token (sidecar for an existing file, internal .path.ovlock otherwise)
6. TOCTOU double-check: re-scan descendants and ancestors
- Conflict found: compare (timestamp, handle_id)
- Later one (larger timestamp/handle_id) backs off (removes own lock) to prevent livelock
@@ -378,11 +401,15 @@ for conflicts first:
4. Step 3 does not roll back the empty directory
```
+Acquisition rollback and normal release clean up tokens without guaranteeing removal of directories created to store them. An Exact sidecar can also create a missing parent chain. Snapshots use a separate policy for missing targets; see [snapshot scope and concurrency](../guides/15-snapshot.md#commit-scope-and-concurrency).
+
### Lock Expiry Cleanup
+**Automatic refresh**: Rust PathLockManager refreshes active leases every `lock_expire / 3`. The default 30-second expiry is not a maximum operation duration. Refresh stops when the process exits.
+
**Stale lock detection**: PathLockEngine checks the ownership token timestamp. Locks older than `lock_expire` (default 30s) are considered stale and are removed automatically during acquisition.
-**In-process cleanup**: LockManager checks active LockHandles every 60 seconds. Handles that still own provider tokens but have been inactive for longer than `lock_expire` are force-released.
+**In-process cleanup**: During refresh, Rust PathLockManager checks leases that have not refreshed successfully for `2 × lock_expire` and attempts cleanup with ownership validation.
**Orphan locks**: Provider tokens left behind after a process crash are automatically removed via stale lock detection when a later acquisition checks the same path or scope.
diff --git a/docs/en/guides/15-snapshot.md b/docs/en/guides/15-snapshot.md
index 6ccef82d8..38733dc08 100644
--- a/docs/en/guides/15-snapshot.md
+++ b/docs/en/guides/15-snapshot.md
@@ -6,6 +6,40 @@ Multi-version management is powered by [gitoxide](https://github.com/Byron/gitox
> For the full API reference of each command's parameters and responses, see [Snapshots API](../api/11-snapshot.md).
+## When to commit
+
+Ordinary `write` and `rm` operations change the live workspace directly. Enabling snapshots does not automatically create a version after every write. A prior file version is recoverable through snapshots only if a successful snapshot commit covered that path. Commit after an import, before and after a batch change, or at a business checkpoint, and retain the returned `commit_oid`.
+
+Snapshots version the file tree. They do not roll back current ACLs or preserve historical vector indexes. After restoring files, `restore` rebuilds indexes asynchronously when needed. It changes the workspace; inspect its plan with `dry_run` first.
+
+`client.snapshot.commit()` saves file versions, while a session's `commit()` archives conversation and processes memory. Some memory workflows call snapshot after updating experiences; this does not make snapshots automatic for every write.
+
+## Commit scope and concurrency
+
+USER / ADMIN callers must supply `paths` for `commit` and `log`, `project_dir` for `restore`, and a file `path` for `show`. Only local ROOT mode may omit those scope parameters. Committing a project directory with write access keeps unrelated resources out of the change.
+
+| `commit(paths=...)` input | Meaning |
+| --- | --- |
+| Existing file URI | Process that file |
+| Existing directory URI | Recursively process current files and record deletions of previously snapshotted files in that subtree |
+| Missing URI | Record deletion of that path and its subtree from the previous snapshot; no change if neither existed there |
+| `[]` | Explicit empty scope; no changes |
+| `None` / omitted | Entire account tree; ROOT only |
+
+A partial commit starts from the branch's previous snapshot and preserves its versions outside the requested scope. After deletion, include the deleted URI or a covering parent directory in the next commit. Dropping the URI from `paths` would omit the deletion. A trailing `/` does not declare a file/directory type, and the current API has no explicit per-target type parameter.
+
+For non-ROOT explicit-path commits, locks are selected from current target state before scope authorization and snapshot creation:
+
+| Target state | Filesystem PathLock | Cache (Redis) PathLock |
+| --- | --- | --- |
+| Existing file | Exact | Exact |
+| Existing directory | Tree | Tree |
+| Missing path | Skip locking this target | Tree |
+
+Filesystem skips missing targets to avoid creating a target directory or a missing parent chain just to store a lock token. The target still participates in snapshot deletion processing. A concurrent recreation may be missed or read before its write finishes; a later commit records the final state. ROOT bypasses this explicit-path locking flow.
+
+Snapshots are not a globally atomic view of arbitrary concurrent I/O. For a deterministic checkpoint, finish writes in the intended scope before committing. Locks coordinate only operations participating in the [PathLock protocol](../concepts/09-transaction.md).
+
## Prerequisites
- You already have a working `ov.conf`.
@@ -142,7 +176,7 @@ client.write(
content="# Guide\n\nv1 content\n",
mode="create",
)
-v1 = client.snapshot.commit(message="v1 initial import")
+v1 = client.snapshot.commit(message="v1 initial import", paths=[root])
print("v1:", v1["commit_oid"])
# 2. Modify and commit v2
@@ -151,14 +185,14 @@ client.write(
content="# Guide\n\nv2 content\n",
mode="replace",
)
-v2 = client.snapshot.commit(message="v2 update")
+v2 = client.snapshot.commit(message="v2 update", paths=[root])
# 3. Walk history
-for c in client.snapshot.log(limit=10):
+for c in client.snapshot.log(limit=10, paths=[root]):
print(c["oid"][:8], c["message"])
-# 4. Inspect a commit's metadata
-print(client.snapshot.show(v1["commit_oid"])["message"])
+# 4. Read historical file content
+print(client.snapshot.show(v1["commit_oid"], path=f"{root}/guide.md"))
# 5. Restore the workspace to v1 (creates a new "forward" commit on top of v2)
client.snapshot.restore(project_dir=root, source_commit=v1["commit_oid"], message="restore to v1")
@@ -172,13 +206,13 @@ The CLI subcommands live under `ov snapshot`:
```bash
# Commit the current workspace state
-ov snapshot commit -m "v1 initial import" -o json
+ov snapshot commit -m "v1 initial import" --paths viking://resources/my_project -o json
# Walk history (newest first)
-ov snapshot log --limit 10 -o json
+ov snapshot log --paths viking://resources/my_project --limit 10 -o json
-# View commit metadata
-ov snapshot show <commit_oid> -o json
+# Read historical file content
+ov snapshot show <commit_oid> --path viking://resources/my_project/guide.md
# Read a file's content from a commit (defaults to stdout; use --out-file to write a local file)
ov snapshot show <commit_oid> --path viking://resources/my_project/guide.md --out-file ./guide.md
@@ -197,14 +231,14 @@ ov snapshot restore <commit_oid> viking://resources/my_project --dry-run -o json
curl -X POST "http://localhost:1933/api/v1/snapshot/commit" \
-H "Content-Type: application/json" \
-H "X-API-Key: your-key" \
- -d '{"message": "v1 initial import"}'
+ -d '{"message": "v1 initial import", "paths": ["viking://resources/my_project"]}'
# Walk history
-curl -X GET "http://localhost:1933/api/v1/snapshot/log?branch=main&limit=10" \
+curl -X GET "http://localhost:1933/api/v1/snapshot/log?branch=main&limit=10&paths=viking://resources/my_project" \
-H "X-API-Key: your-key"
-# View commit metadata
-curl -X GET "http://localhost:1933/api/v1/snapshot/show?target_ref=<commit_oid>" \
+# Read historical file content
+curl -X GET "http://localhost:1933/api/v1/snapshot/show?target_ref=<commit_oid>&path=viking://resources/my_project/guide.md" \
-H "X-API-Key: your-key"
# Restore
@@ -262,7 +296,7 @@ client.snapshot.delete_gitignore()
On subsequent commits, files matching the rules are excluded, and the response's `ignored` field reports how many candidate paths were skipped:
```python
-v = client.snapshot.commit(message="with ignore")
+v = client.snapshot.commit(message="with ignore", paths=["viking://resources/my_project"])
print(v["result"], v.get("ignored")) # created, 1
```
diff --git a/docs/zh/agent-integrations/03-openclaw.md b/docs/zh/agent-integrations/03-openclaw.md
index a2c1141a7..2ea59ed90 100644
--- a/docs/zh/agent-integrations/03-openclaw.md
+++ b/docs/zh/agent-integrations/03-openclaw.md
@@ -82,6 +82,42 @@ openclaw openviking setup --base-url http://your-server:1933 --api-key sk-xxx --
新配置请使用 `sender`;已有的 `peer_role=person` 配置仍兼容,并按 `sender` 处理。OpenViking 会为每个用户初始化受管的 `peers/` 容器,因此 `none` 的含义是不使用具体的 `peers/<peer_id>/memories` 子树。Actor-peer 召回同时包含用户共享记忆和当前 peer 记忆;切换 scope 不会搬迁已有记忆。
+## assemble 如何组装上下文
+
+插件占用 OpenClaw 的 `contextEngine` 槽位。会话历史、长期记忆召回和本轮新输入分别处理;`assemble()` 返回供本次模型请求使用的上下文,不把组装出的摘要或召回内容持久化到 OpenClaw 的 session transcript,也不通过该调用向 OV session 追加消息。宿主可以用返回的 messages 更新本轮内存状态,这与写入持久化对话记录不同。
+
+主 assemble 在每轮新输入开始执行时准备历史上下文。插件通过参数中是否包含 `prompt`、`availableTools`、`citationsMode` 中任一字段识别该调用。
+
+### 主 assemble:历史和当前输入分开
+
+主分支调用 `getSessionContext(tokenBudget)`,用返回的内容构造:
+
+```text
+summaryMessage = { role: "user", content: "[Session History Summary]\n" + latest_archive_overview }
+messages = [summaryMessage] + OV active messages
+systemPromptAddition = Session Context Guide(有归档时)+ 本轮召回结果(有命中时)
+```
+
+`latest_archive_overview` 是服务端返回的摘要正文,`[Session History Summary]` 是插件加在正文前的固定文本标题。仅在 overview 非空时插入这条合成 user 消息;active messages 保留近期未压缩对话。当前 `prompt` 由宿主加入本轮;插件只用它查询记忆,不把它重复追加到返回的历史中。召回结果属于本次请求的上下文,不直接作为新对话写回 OV。
+
+overview 由 OV 服务端的工作记忆流程生成,插件读取结果。服务端先为 active messages 分配预算,剩余空间不足时不返回 overview;`pre_archive_abstracts` 当前为空数组。因此返回结果不是完整归档索引,需要原始细节时通过 `ov_archive_search` 查询归档。
+
+插件为模型输出预留 token 空间,扣除使用指南和摘要的实际估算量,再从 active messages 头部裁掉超预算内容,并整理工具调用/结果等 provider 消息格式。摘要不会按插件计算出的 archive 预算硬截断,因此这些预算不能当作各层的严格配额;新增召回块若使总估算量超过 `tokenBudget`,该块会被省略。
+
+OV 无数据、无归档且消息数少于宿主输入、转换后为空或读取失败时,历史分支回退到宿主 messages。即使历史透传,只要有合法 `prompt` 且启用了 `autoRecall`,主分支仍可尝试召回。召回无命中或失败不会阻止对话。
+
+### transformContext
+
+`transformContext` 在每次 LLM 调用前执行,无论最后一条消息是 user message 还是 tool response。该 hook 在 OV 集成中的最佳使用方式暂未明确。
+
+### 捕获和压缩
+
+- `ingest()` / `ingestBatch()` 不写入消息。常规捕获通过 `afterTurn`;插件对 OpenClaw 2026.9.3 及之后的稳定版本支持由 `commitTurn` 接收宿主交付的已结束轮次,旧版和独立 runner 使用 `afterTurn`。无法识别版本时,`commitTurn` 拒绝确认,避免确认未捕获的数据。
+- 捕获逻辑清洗注入内容、转换文本和工具消息,再写入 OV session。达到 `pending_tokens >= tokenBudget × commitTokenThresholdRatio` 时,发起异步 session commit;默认比例为 `0.5`,默认保留最近 `10` 条消息,也可选 `turn_budget` 保留策略。`pending_tokens` 是服务端按保留策略计算的待归档消息 token 数,不是整个模型请求的 token 数。
+- `ownsCompaction: true` 表示插件负责压缩。正常 `compact()` 提交 OV session(`wait=true`、保留数为 `0`),读取 overview 作为压缩摘要;下一次主 assemble 用摘要和 active messages 重建历史。被 bypass 的会话尝试委托宿主压缩器,宿主 bridge 不可用时返回跳过。
+
+这里的 **session commit** 负责会话归档和记忆处理,与保存资源文件版本的 [snapshot commit](../guides/15-snapshot.md) 是不同操作。
+
## 验证
```bash
diff --git a/docs/zh/api/11-snapshot.md b/docs/zh/api/11-snapshot.md
index 30201e150..12813248f 100644
--- a/docs/zh/api/11-snapshot.md
+++ b/docs/zh/api/11-snapshot.md
@@ -44,7 +44,7 @@ USER 和 ADMIN 调用 `commit`、`log`、`restore` 时必须显式传入 `paths`
- HTTP 路由:[snapshot.py](https://github.com/volcengine/OpenViking/blob/main/openviking/server/routers/snapshot.py),前缀 `/api/v1/snapshot`。
- 命名空间(SDK):[client.py](https://github.com/volcengine/OpenViking/blob/main/sdk/python/openviking_sdk/client.py),暴露为 `client.snapshot.*`。
-- 底层语义实现:[viking_fs.py](https://github.com/volcengine/OpenViking/blob/main/openviking/storage/viking_fs.py) 的 `commit` / `restore` / `show` / `log` / `diff`。
+- 底层语义实现:[_snapshot.py](https://github.com/volcengine/OpenViking/blob/main/openviking/storage/viking_fs/_snapshot.py) 的 `commit` / `restore` / `show` / `log` / `diff`。
- CLI 命令:[main.rs](https://github.com/volcengine/OpenViking/blob/main/crates/ov_cli/src/main.rs) 的 `SnapshotCmd`,子命令 [snapshot.rs](https://github.com/volcengine/OpenViking/blob/main/crates/ov_cli/src/commands/snapshot.rs)。
## API 参考
@@ -53,12 +53,14 @@ USER 和 ADMIN 调用 `commit`、`log`、`restore` 时必须显式传入 `paths`
把当前工作区状态保存成一个新的快照。
+局部提交保留范围外的上次快照内容。删除文件或目录后,仍需把该 URI 或其父目录传入 `paths` 才会记录删除。末尾 `/` 不声明类型。非 ROOT 提交对现存文件加 Exact、现存目录加 Tree;缺失路径在 filesystem 锁后端不加锁,在 cache 后端加 Tree。缺失目标的并发重建不保证被本次快照完整记录,见 [提交范围与并发](../guides/15-snapshot.md#提交范围与并发)。
+
**参数**
| 参数 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| message | str | 是 | - | 提交说明 |
-| paths | List[str] | 否 | null | 限定本次快照的 `viking://` URI 列表,条目可以是文件或目录;目录会按照快照的剪枝规则递归展开。USER/ADMIN 必须显式传入;`null` 只保留给本地 ROOT 模式的整棵账号树快照。传入空列表 `[]` 表示显式的空路径集(不会产生改动)。如果某个路径在 VFS 和前一次快照中都不存在,会输出一条 warn,并按"对该名称下任何子树执行删除"处理 |
+| paths | List[str] | 否 | null | 限定本次快照的 `viking://` URI 列表,条目可以是文件或目录;目录会按照快照的剪枝规则递归展开。USER/ADMIN 必须显式传入;`null` 只保留给本地 ROOT 模式的整棵账号树快照。传入空列表 `[]` 表示显式的空路径集(不会产生改动)。缺失路径会从新快照移除此前的同名文件及其子树;若此前也不存在则告警并无改动 |
| branch | str | 否 | `main` | 要推进的分支 |
| author_name | str | 否 | null | 覆盖默认的提交者名字(默认 `viking-bot`) |
| author_email | str | 否 | null | 覆盖默认的提交者邮箱 |
diff --git a/docs/zh/concepts/09-transaction.md b/docs/zh/concepts/09-transaction.md
index 947669f22..96fecfca5 100644
--- a/docs/zh/concepts/09-transaction.md
+++ b/docs/zh/concepts/09-transaction.md
@@ -1,6 +1,6 @@
# 路径锁与崩溃恢复
-OpenViking 通过**路径锁**和**持久化队列恢复**两个简单原语保护核心写操作(`rm`、`mv`、`add_resource`、`session.commit`)的一致性,确保 VikingFS、VectorDB、QueueManager 三个子系统在故障时不会出现数据不一致。
+OpenViking 通过**路径锁**和**持久化队列恢复**两个简单原语保护核心写操作(`rm`、`mv`、`add_resource`、`session.commit`)的一致性,协调并发写入,并在进程重启后继续处理已入队的会话任务。路径锁和队列恢复不构成跨 VikingFS、VectorDB、QueueManager 的原子事务。
## 设计哲学
@@ -10,8 +10,8 @@ OpenViking 是上下文数据库,FS 是源数据,VectorDB 是派生索引。
## 设计原则
-1. **写互斥**:通过路径锁保证同一路径同一时间只有一个写操作
-2. **默认生效**:所有数据操作命令自动加锁,用户无需额外配置
+1. **写互斥**:参与锁协议的不同 owner 不能同时取得冲突路径的锁
+2. **默认生效**:受保护的写操作默认加锁;普通读取和底层 mkdir 不自动加锁
3. **锁即保护**:进入 LockContext 时加锁,退出时释放,没有 undo/journal/commit 语义
4. **仅 session_memory 需要崩溃恢复**:通过持久化 `session_commit` 队列在进程崩溃后恢复 Phase 2
5. **Queue 操作在锁外执行**:SemanticQueue/EmbeddingQueue 的 enqueue 是幂等的,失败可重试
@@ -270,24 +270,47 @@ async with LockContext(lock_manager, [src], lock_mode="mv", mv_dst_path=dst):
- **EXACT (E)**:锁定一个具体路径本身。文件、目录名、尚未创建的目标路径都可以使用;若祖先目录持有 TreeLock 则阻塞。
- **TREE (T)**:用于删除目录、移动目录、资源生命周期保护等。逻辑上覆盖整棵子树,但只为根路径保存一个 Provider token。冲突检查覆盖 Provider scope 内的后代和持有 Tree 锁的祖先。Filesystem Provider 可能为了写锁文件而创建尚不存在的目标目录。
+### 路径范围与目标类型
+
+Exact 和 Tree 表达操作范围,文件、目录或缺失路径表达目标的当前状态,两者独立。锁保护路径名字,目标不存在也可以申请锁。
+
+以下冲突关系限定为不同 owner、同一 Provider scope 内的请求:
+
+| 已持有 | 新请求 | 冲突 |
+| --- | --- | --- |
+| Exact(`/docs/a.md`) | Exact 或 Tree(`/docs/a.md`) | 是 |
+| Exact(`/docs`) | Exact(`/docs/a.md`) | 否 |
+| Tree(`/docs`) | Exact 或 Tree(`/docs/a.md`) | 是 |
+| Exact(`/docs/a.md`) | Tree(`/docs`) | 是 |
+| Tree(`/docs/a.md`) | Exact(`/docs/b.md`) | 否 |
+
+`Tree(/docs/a.md)` 不会扩大为 `Tree(/docs)`。反过来,目录自身的 Exact 也不能保护子树,递归删除需要 Tree。
+
+锁只协调参与协议的操作。底层 `PathLockWrappedFS` 对 create、write、truncate、非递归 remove 使用 Exact,对 remove_all 使用 Tree;文件 rename 锁源和目标的 Exact,目录 rename 锁源 Tree 和目标 Exact。read、stat、列目录和 mkdir 直接转发,上层可另行持锁。绕过协议的 I/O 不会被操作系统自动阻断。
+
## 锁机制
### Filesystem Provider 锁协议
-锁文件路径:
+锁类型由调用者选择,Resolver 根据目标状态决定 token 位置:
+
+| 目标状态 | Exact token | Tree token |
+| --- | --- | --- |
+| 现存文件 `/docs/a` | `/docs/.exact.ovlock.a.<hash>`,内容为 E | 同一 sidecar,内容为 T |
+| 现存目录 `/docs/a` | `/docs/a/.path.ovlock`,内容为 E | 同一目录内文件,内容为 T |
+| 缺失路径 `/docs/a` | 父目录 sidecar,内容为 E | 创建目标目录后写内部 `.path.ovlock`,内容为 T |
+
+sidecar 位于目标旁边,但只代表该目标,不会锁住整个父目录。`<hash>` 来自完整后端路径的 SHA-1 前缀,与业务文件内容无关。
+
+`.exact.ovlock.*` 可以存 Tree token,`.path.ovlock` 也可以存 Exact token。文件名是存储协议的一部分,不能单凭名字判断逻辑锁类型。token 内容为:
```text
-TreeLock(path) -> {path}/.path.ovlock
-ExactPathLock(已存在目录 path) -> {path}/.path.ovlock
-ExactPathLock(文件或未创建路径) -> {parent}/.exact.ovlock.<name>.<hash>
+{owner_id}:{time_ns}:{lock_type}
```
-锁文件内容(Fencing Token):
-```
-{handle_id}:{time_ns}:{lock_type}
-```
+`lock_type` 为 `E` 或 `T`。该归属 token 用于竞争检查、续期和条件释放,不代表所有业务写入都有存储端 fencing 校验。
-其中 `lock_type` 为 `E`(EXACT)或 `T`(TREE)。
+lease 将逻辑范围 `covered_paths` 与 token 位置 `lock_paths` 分开记录。Owned lease 控制续期、释放和交接;Borrowed lease 仅提供已有锁的覆盖证明,不能释放外层锁。
### Cache Provider 锁协议
@@ -348,8 +371,8 @@ HASH。跨 scope batch 会被拒绝。
- 目标目录不存在? -> 视为无后代锁
- 陈旧锁? -> 移除后重试
- 活跃锁? -> 等待
- 4. 确保目标目录存在;如果不存在则创建目录
- 5. 写入 TREE (T) 锁文件(只写一个文件,在根路径)
+ 4. 确保 Resolver 选定的 token 父目录存在;缺失目标会因此被创建成目录
+ 5. 写入 TREE (T) token(现存文件用 sidecar,其余用内部 .path.ovlock)
6. TOCTOU 双重检查:重新扫描后代目录和祖先目录
- 发现冲突:比较 (timestamp, handle_id)
- 后到者(更大的 timestamp/handle_id)主动让步(删除自己的锁),防止活锁
@@ -386,11 +409,15 @@ HASH。跨 scope batch 会被拒绝。
fencing token 校验通过的一方成功持有 `TreeLock(java-guide)`;失败方会删除自己的锁,
已创建出来的空目录可以保留。
+获取失败的回滚和正常释放只清理 token,不保证删除为存放 token 创建的目录。Exact sidecar 也可能创建缺失的父目录链。Snapshot 对缺失目标采用单独的策略,见 [快照的范围与并发](../guides/15-snapshot.md#提交范围与并发)。
+
### 锁过期清理
+**自动续期**:Rust PathLockManager 每隔 `lock_expire / 3` 刷新活跃 lease;默认过期时间为 30 秒,不是业务操作的最长运行时间。进程退出后续期停止。
+
**陈旧锁检测**:PathLockEngine 检查归属 token 中的时间戳。超过 `lock_expire`(默认 30s)的锁被视为陈旧锁,在加锁过程中自动移除。
-**进程内清理**:LockManager 每 60 秒检查活跃的 LockHandle。仍持有 Provider token 且失活时间超过 `lock_expire` 的 handle 会被强制释放。
+**进程内清理**:Rust PathLockManager 在续期循环中检查长期未成功续期的 lease,以 `2 × lock_expire` 为阈值尝试清理,并校验归属后释放 token。
**孤儿锁**:进程崩溃后遗留的 Provider token,在后续 acquire 检查同一路径或 scope 时通过 stale lock 检测自动移除。
diff --git a/docs/zh/guides/15-snapshot.md b/docs/zh/guides/15-snapshot.md
index 7f6a7da42..d6df37a2f 100644
--- a/docs/zh/guides/15-snapshot.md
+++ b/docs/zh/guides/15-snapshot.md
@@ -6,6 +6,40 @@
> 关于各命令参数和响应结构的完整 API 参考,见 [多版本管理 API](../api/11-snapshot.md)。
+## 何时需要 commit
+
+普通 `write`、`rm` 等操作直接改变当前工作区,启用快照不代表每次写入都会自动生成版本。没有成功覆盖该路径的 snapshot commit,就不能通过快照找回它此前的内容。建议在导入完成、批量修改前后或一个业务阶段结束时提交,并保存返回的 `commit_oid`。
+
+Snapshot 保存文件树的版本,不会回滚当前 ACL,也不保存向量索引的历史。`restore` 写回文件后按需异步重建索引;它会修改工作区,执行前先用 `dry_run` 检查计划。
+
+`client.snapshot.commit()` 与会话的 `session.commit()` 职责不同:前者保存文件版本,后者归档对话并处理记忆。部分记忆流程会在更新 experience 后调用 snapshot,这不等于所有写入都自动快照。
+
+## 提交范围与并发
+
+USER / ADMIN 必须给 `commit` 和 `log` 传入 `paths`,给 `restore` 传入 `project_dir`,给 `show` 传入文件 `path`。只有本地 ROOT 可以省略这些范围参数。对自己有写权限的项目目录提交,可以避免把无关资源纳入同一次快照。
+
+| `commit(paths=...)` 输入 | 含义 |
+| --- | --- |
+| 现存文件 URI | 处理该文件 |
+| 现存目录 URI | 递归处理当前文件,并记录此前快照中该子树内文件的删除 |
+| 缺失 URI | 记录此前快照中该路径及其子树的删除;此前也不存在则无改动 |
+| `[]` | 显式空范围,不产生改动 |
+| `None` / 省略 | 整棵账号树,仅限 ROOT |
+
+局部提交沿用分支上一次快照作为基础,范围外文件保留原快照版本。删除后仍要提交被删除 URI 或覆盖它的父目录;从 `paths` 中去掉该 URI 会漏记删除。路径末尾 `/` 不是文件/目录类型声明,当前接口没有逐目标的显式类型参数。
+
+非 ROOT 的显式路径提交先按当前状态选锁,再检查范围权限并生成快照:
+
+| 目标状态 | Filesystem PathLock | Cache(Redis)PathLock |
+| --- | --- | --- |
+| 现存文件 | Exact | Exact |
+| 现存目录 | Tree | Tree |
+| 缺失路径 | 跳过该目标的锁 | Tree |
+
+Filesystem 对缺失目标跳过锁,避免锁文件创建目标目录或缺失的父目录链;该目标仍参与快照删除处理。此时并发重建同一路径可能被漏记或读到尚未写完的内容,后续提交才能记录最终状态。ROOT 不经过这段显式路径加锁流程。
+
+快照不能视为任意并发 I/O 的全局原子视图。需要确定的业务检查点时,应先结束该范围内的写入,再提交;锁只协调参与 [PathLock 协议](../concepts/09-transaction.md) 的操作。
+
## 前置条件
- 已有可用的 `ov.conf`。
@@ -30,7 +64,7 @@
"author_name": "viking-bot",
"author_email": "[email protected]",
"local": {
- "base_dir": "",
+ "base_dir": ""
}
}
}
@@ -142,7 +176,7 @@ client.write(
content="# Guide\n\nv1 content\n",
mode="create",
)
-v1 = client.snapshot.commit(message="v1 initial import")
+v1 = client.snapshot.commit(message="v1 initial import", paths=[root])
print("v1:", v1["commit_oid"])
# 2. 修改后再提交 v2
@@ -151,14 +185,14 @@ client.write(
content="# Guide\n\nv2 content\n",
mode="replace",
)
-v2 = client.snapshot.commit(message="v2 update")
+v2 = client.snapshot.commit(message="v2 update", paths=[root])
# 3. 查看历史
-for c in client.snapshot.log(limit=10):
+for c in client.snapshot.log(limit=10, paths=[root]):
print(c["oid"][:8], c["message"])
-# 4. 查看某个提交的元数据
-print(client.snapshot.show(v1["commit_oid"])["message"])
+# 4. 读取历史文件内容
+print(client.snapshot.show(v1["commit_oid"], path=f"{root}/guide.md"))
# 5. 把工作区恢复到 v1(会在 v2 之上生成一个新的“正向”提交)
client.snapshot.restore(project_dir=root, source_commit=v1["commit_oid"], message="restore to v1")
@@ -172,13 +206,13 @@ CLI 子命令位于 `ov snapshot` 下:
```bash
# 提交当前工作区状态
-ov snapshot commit -m "v1 initial import" -o json
+ov snapshot commit -m "v1 initial import" --paths viking://resources/my_project -o json
# 回溯历史(最新在前)
-ov snapshot log --limit 10 -o json
+ov snapshot log --paths viking://resources/my_project --limit 10 -o json
-# 查看提交元数据
-ov snapshot show <commit_oid> -o json
+# 读取历史文件内容
+ov snapshot show <commit_oid> --path viking://resources/my_project/guide.md
# 读取某个提交中的文件内容(默认输出到 stdout,可用 --out-file 写入本地文件)
ov snapshot show <commit_oid> --path viking://resources/my_project/guide.md --out-file ./guide.md
@@ -197,14 +231,14 @@ ov snapshot restore <commit_oid> viking://resources/my_project --dry-run -o json
curl -X POST "http://localhost:1933/api/v1/snapshot/commit" \
-H "Content-Type: application/json" \
-H "X-API-Key: your-key" \
- -d '{"message": "v1 initial import"}'
+ -d '{"message": "v1 initial import", "paths": ["viking://resources/my_project"]}'
# 回溯历史
-curl -X GET "http://localhost:1933/api/v1/snapshot/log?branch=main&limit=10" \
+curl -X GET "http://localhost:1933/api/v1/snapshot/log?branch=main&limit=10&paths=viking://resources/my_project" \
-H "X-API-Key: your-key"
-# 查看提交元数据
-curl -X GET "http://localhost:1933/api/v1/snapshot/show?target_ref=<commit_oid>" \
+# 读取历史文件内容
+curl -X GET "http://localhost:1933/api/v1/snapshot/show?target_ref=<commit_oid>&path=viking://resources/my_project/guide.md" \
-H "X-API-Key: your-key"
# 恢复
@@ -262,7 +296,7 @@ client.snapshot.delete_gitignore()
随后提交时,匹配规则的文件会被排除,响应里的 `ignored` 字段给出本次被排除的候选路径数:
```python
-v = client.snapshot.commit(message="with ignore")
+v = client.snapshot.commit(message="with ignore", paths=["viking://resources/my_project"])
print(v["result"], v.get("ignored")) # created, 1
```