Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

Fix/add resource (#4344)

zihenglib8c89099be13
diff --git a/docs/en/api/02-resources.md b/docs/en/api/02-resources.md
index 8fb6a10ac..ef840a2a7 100644
--- a/docs/en/api/02-resources.md
+++ b/docs/en/api/02-resources.md
@@ -47,7 +47,7 @@ OpenViking supports various resource types, categorized by functionality:
| Type | Description |
|------|-------------|
-| Feishu/Lark | URL-based, supports doc/docx, wiki, sheets, bitable. By default uses app credentials from FEISHU_APP_ID and FEISHU_APP_SECRET; user-token imports can pass `args.feishu_access_token`, and user-token watches also pass `args.feishu_refresh_token` |
+| Feishu/Lark | URL-based, supports doc/docx, wiki, sheets, bitable. By default uses app credentials from FEISHU_APP_ID and FEISHU_APP_SECRET; user-token imports can pass `args.feishu_access_token`, and user-token watches also pass `args.feishu_refresh_token` plus an optional `args.feishu_app_id` / `args.feishu_app_secret` pair |
**Web Pages (recursive web crawler)**
@@ -200,9 +200,9 @@ This endpoint is the core entry point for resource management, supporting adding
- When `watch_interval > 0`, the watch task binds to `to` if provided; otherwise it binds to the `root_uri` returned by this import. If no stable `root_uri` is available, the request fails and asks for an explicit `to`.
- Feishu/Lark app-token imports do not pass `args.feishu_access_token`. OpenViking keeps the existing app credential flow and the SDK obtains an app/tenant token from `app_id` and `app_secret`. This mode supports both one-time imports and `watch_interval > 0`.
- Feishu/Lark one-time user-token imports pass `args={"feishu_access_token": "u-..."}` with `watch_interval <= 0`. OpenViking uses that user token only for the current import and does not store it.
-- Feishu/Lark user-token watches pass `args={"feishu_access_token": "u-...", "feishu_refresh_token": "r-..."}` with `watch_interval > 0`. OpenViking stores the token state in the private watch task state, refreshes it with the configured Feishu app credentials, and uses the refreshed user token for later watch runs.
-- Feishu/Lark user-token watches require `FEISHU_APP_ID` and `FEISHU_APP_SECRET` (or `feishu.app_id` and `feishu.app_secret` in `ov.conf`) because Feishu refresh tokens are bound to the app that issued them. The supplied user token must come from the same Feishu app configured in OpenViking.
-- Watch task token state is stored in the internal `viking://resources/.watch_tasks.json` control file and is hidden from watch API/MCP/CLI responses. If VikingFS file encryption is enabled, this control file is encrypted at rest; otherwise the server-side control file contains plaintext token state.
+- Feishu/Lark user-token watches pass `args={"feishu_access_token": "u-...", "feishu_refresh_token": "r-..."}` with `watch_interval > 0`. They may also pass `feishu_app_id` and `feishu_app_secret` together; OpenViking stores the pair in the private watch task state and uses it to refresh that watch's user token.
+- If the request omits the app pair, user-token watches use `FEISHU_APP_ID` and `FEISHU_APP_SECRET`, or `feishu.app_id` and `feishu.app_secret` in `ov.conf`. Feishu refresh tokens are bound to their issuing app, so whichever app credentials are used must match the supplied user token.
+- Watch task token state and request-supplied app credentials are stored in the internal `viking://resources/.watch_tasks.json` control file and hidden from watch API/MCP/CLI responses. If VikingFS file encryption is enabled, this control file is encrypted at rest; otherwise the server-side control file contains this private state in plaintext.
- For local directory inputs, scanning respects `.gitignore` files (root and nested) with standard Git semantics; `ignore_dirs`, `include`, and `exclude` further refine what is ingested.
- `args.parse_mode=no_split` still invokes the normal format Parser. PDF, Word, PowerPoint, HTML, and other supported documents are converted to Markdown, but heading-, paragraph-, and size-based splitting is skipped. A directory import applies this independently to each supported document and continues to honor `.gitignore`, filters, and `preserve_structure`.
- For a single-file input in `no_split` mode, when parsing produces exactly one visible file and `to` is omitted, that file is stored directly under the resolved parent (for example, `guide.md` becomes `viking://resources/guide.md`). No wrapper directory or directory-level `.abstract.md` / `.overview.md` is created. If parsing also produces images or other visible files, the wrapper directory is retained. An explicit `to` is always preserved as the exact final URI.
@@ -317,7 +317,9 @@ curl -X POST http://localhost:1933/api/v1/resources \
"watch_interval": 1440,
"args": {
"feishu_access_token": "u-...",
- "feishu_refresh_token": "r-..."
+ "feishu_refresh_token": "r-...",
+ "feishu_app_id": "cli_...",
+ "feishu_app_secret": "..."
}
}'
```
@@ -410,6 +412,8 @@ client.add_resource(
"args": {
"feishu_access_token": "u-...",
"feishu_refresh_token": "r-...",
+ "feishu_app_id": "cli_...",
+ "feishu_app_secret": "...",
},
},
)
@@ -484,7 +488,9 @@ ov add-resource https://example.feishu.cn/docx/doc_token \
--to viking://resources/feishu/doc \
--watch-interval 1440 \
--args feishu_access_token:u-... \
- --args feishu_refresh_token:r-...
+ --args feishu_refresh_token:r-... \
+ --args feishu_app_id:cli_... \
+ --args feishu_app_secret:...
# Add with parent directory (parent must exist)
ov add-resource ./documents/guide.md --parent viking://resources/docs
diff --git a/docs/en/api/15-watches.md b/docs/en/api/15-watches.md
index d77e01e15..b15f16a29 100644
--- a/docs/en/api/15-watches.md
+++ b/docs/en/api/15-watches.md
@@ -140,6 +140,7 @@ Listing tasks returns:
{
"task_id": "7f02e980-8df9-4f27-a570-4d8428cbed8a",
"path": "https://example.com/guide.md",
+ "source_type": "url",
"to_uri": "viking://resources/guide.md",
"parent_uri": "viking://resources",
"reason": "keep documentation current",
@@ -162,6 +163,10 @@ Listing tasks returns:
}
```
+`source_type` is optional provenance metadata. Explicit Connector `add_type` values take
+precedence (for example, `tos` or `feishu_project`); native imports report `feishu`, `git`,
+`url`, or `local`. Legacy or unclassified watches return `null`.
+
Getting one task and a successful update return the same task object directly in `result`. Delete and trigger return:
```json
diff --git a/docs/en/api/17-tasks.md b/docs/en/api/17-tasks.md
index 1af002252..c577b3159 100644
--- a/docs/en/api/17-tasks.md
+++ b/docs/en/api/17-tasks.md
@@ -255,8 +255,11 @@ List background tasks visible to the current caller, supporting filtering by typ
| task_type | str | No | None | Filter by task type, for example `session_commit` |
| status | str | No | None | Filter by task status: `pending`, `running`, `cancelling`, `completed`, `failed`, `cancelled` |
| resource_id | str | No | None | Filter by task resource ID, for example a session ID |
+| include_internal | bool | No | false | Include internal child tasks created by Connector imports |
| limit | int | No | 50 | Maximum number of task records to return |
+By default, only user-visible tasks are returned. Pass `include_internal=true` when diagnosing a Connector import to include its internal `add_resource` child tasks.
+
#### 3. Usage Examples
**HTTP API**
diff --git a/docs/en/guides/06-mcp-integration.md b/docs/en/guides/06-mcp-integration.md
index 2bbc702b8..577f8d9ad 100644
--- a/docs/en/guides/06-mcp-integration.md
+++ b/docs/en/guides/06-mcp-integration.md
@@ -141,7 +141,7 @@ Once connected, OpenViking exposes 15 tools:
| `remember` | Store messages into long-term memory (triggers extraction) | `messages` (list of `{role, content}`) |
| `write` | Write text to a `viking://` file (create/overwrite/append). Parent directories are created automatically; use `read` first to see current content before overwriting, and prefer `edit` for changing part of an existing file | `uri`, `content`, `mode` (optional: `replace` default — overwrites or creates if missing; `append` — appends or creates if missing; `create` — fails if it exists), `wait` (optional, block until re-indexed), `timeout` (optional) |
| `edit` | Replace an exact string with new text in an existing `viking://` file — for targeted changes instead of a full rewrite. The file is left unchanged if `old_string` is not found, or matches multiple times while `replace_all` is false | `uri`, `old_string`, `new_string`, `replace_all` (optional), `wait` (optional, block until re-indexed), `timeout` (optional) |
-| `add_resource` | Add a local file or URL as a resource (local files trigger a progressive upload flow) | `path`, `temp_file_id` (optional), `description` (optional), `watch_interval` (optional, minutes — auto-refresh cadence for remote URLs), `processing_mode` (optional: `semantic_and_vectors` default, or `vectors_only` to skip VLM semantic understanding and only vectorize current files), `to` (optional, target `viking://resources/...` URI; if omitted when `watch_interval > 0`, the watch auto-binds to the resource's created URI), `args` (optional parser-specific options, including `{"parse_mode":"no_split"}` to parse each source document into one Markdown body, `{"feishu_access_token":"u-..."}` for one-time Feishu user-token imports, or `{"feishu_access_token":"u-...","feishu_refresh_token":"r-..."}` for Feishu user-token watches) |
+| `add_resource` | Add a local file or URL as a resource (local files trigger a progressive upload flow) | `path`, `temp_file_id` (optional), `description` (optional), `watch_interval` (optional, minutes — auto-refresh cadence for remote URLs), `processing_mode` (optional: `semantic_and_vectors` default, or `vectors_only` to skip VLM semantic understanding and only vectorize current files), `to` (optional, target `viking://resources/...` URI; if omitted when `watch_interval > 0`, the watch auto-binds to the resource's created URI), `args` (optional parser-specific options, including `{"parse_mode":"no_split"}` to parse each source document into one Markdown body, `{"feishu_access_token":"u-..."}` for one-time Feishu user-token imports, or access/refresh tokens plus an optional `feishu_app_id` / `feishu_app_secret` pair for Feishu user-token watches) |
| `list_watches` | List watch tasks (auto-refresh subscriptions) visible to the current agent. Each entry shows target URI, refresh interval (minutes), active/paused status, and next scheduled execution time | none |
| `cancel_watch` | Cancel (delete) a watch task by its target URI. To change the cadence or pause temporarily, cancel and re-add with a new `watch_interval` | `to_uri` (must match the watch task's `to` value, e.g. `viking://resources/...`) |
| `grep` | Regex content search across `viking://` files | `uri`, `pattern` (string or array), `case_insensitive`, `node_limit` |
@@ -163,7 +163,7 @@ user spaces, not a shortcut to yours. See
> **Note**: MCP exposes the minimum closure for watch management (`list_watches` + `cancel_watch`). Pause / resume / trigger and the unified `update` verb are intentionally not exposed here — use the REST `/api/v1/watches/*` endpoints or the `ov task watch` CLI for those operations.
-> Feishu/Lark imports without `args.feishu_access_token` keep the existing app/tenant-token behavior and can be watched. Feishu/Lark one-time user-token imports pass only `args.feishu_access_token`; Feishu/Lark user-token watches must also pass `args.feishu_refresh_token` and require the same Feishu app credentials configured on the OpenViking server.
+> Feishu/Lark imports without `args.feishu_access_token` keep the existing app/tenant-token behavior and can be watched. One-time user-token imports pass only `args.feishu_access_token`; user-token watches must also pass `args.feishu_refresh_token`. They may pass `args.feishu_app_id` and `args.feishu_app_secret` together for that watch, or fall back to the server app credentials. The app must match the issuer of the user token.
> `processing_mode=vectors_only` skips the VLM semantic-understanding stage. It does not generate or refresh `.abstract.md` / `.overview.md`; it only vectorizes current non-hidden resource files, preserving any older semantic artifacts that already exist.
diff --git a/docs/zh/api/02-resources.md b/docs/zh/api/02-resources.md
index 1465c03d9..2acf9e5b1 100644
--- a/docs/zh/api/02-resources.md
+++ b/docs/zh/api/02-resources.md
@@ -42,7 +42,7 @@ OpenViking 支持多种资源类型,按照功能分类如下:
云文档类
| 类型 | 说明 |
|------|------|
-| 飞书/Lark | URL 方式,支持 doc/docx, wiki, sheets, bitable。默认使用 FEISHU_APP_ID 和 FEISHU_APP_SECRET 应用凭证;用户 token 导入可传 `args.feishu_access_token`,用户 token watch 还需传 `args.feishu_refresh_token` |
+| 飞书/Lark | URL 方式,支持 doc/docx, wiki, sheets, bitable。默认使用 FEISHU_APP_ID 和 FEISHU_APP_SECRET 应用凭证;用户 token 导入可传 `args.feishu_access_token`,用户 token watch 还需传 `args.feishu_refresh_token`,并可选传入 `args.feishu_app_id` / `args.feishu_app_secret` |
网页类(递归网页爬虫)
| 类型 | 资源名 | 说明 |
@@ -195,9 +195,9 @@ URL/文件 Parser TreeBuilder AGFS Summarizer/Vector
- `watch_interval > 0` 时,如果指定了 `to`,监控任务绑定该目标;如果未指定 `to`,监控任务绑定本次导入返回的 `root_uri`。如果无法得到稳定 `root_uri`,请求会报错并要求显式传 `to`。
- 飞书/Lark 应用 token 导入不传 `args.feishu_access_token`。OpenViking 保持原有应用凭证流程,由 SDK 使用 `app_id` 和 `app_secret` 自动获取 app/tenant token。该模式支持一次性导入和 `watch_interval > 0`。
- 飞书/Lark 一次性用户 token 导入通过 `args={"feishu_access_token": "u-..."}` 传入,且 `watch_interval <= 0`。OpenViking 只在本次导入使用该用户 token,不保存。
-- 飞书/Lark 用户 token watch 通过 `args={"feishu_access_token": "u-...", "feishu_refresh_token": "r-..."}` 传入,且 `watch_interval > 0`。OpenViking 会把 token 状态保存在 watch task 私有状态里,用配置的飞书应用凭证刷新,并在后续 watch 重跑中使用刷新后的用户 token。
-- 飞书/Lark 用户 token watch 需要 `FEISHU_APP_ID` 和 `FEISHU_APP_SECRET`,或 `ov.conf` 中的 `feishu.app_id` 和 `feishu.app_secret`。飞书 refresh token 绑定签发它的应用,因此传入的用户 token 必须来自 OpenViking 当前配置的同一个飞书应用。
-- Watch task 的 token 状态保存在内部控制文件 `viking://resources/.watch_tasks.json` 中,不会出现在 watch API/MCP/CLI 返回里。若启用了 VikingFS 文件加密,该控制文件会静态加密;否则服务端控制文件中会包含明文 token 状态。
+- 飞书/Lark 用户 token watch 通过 `args={"feishu_access_token": "u-...", "feishu_refresh_token": "r-..."}` 传入,且 `watch_interval > 0`。还可同时传入 `feishu_app_id` 和 `feishu_app_secret`;OpenViking 会将其保存在 watch task 私有状态中,并用于刷新该 watch 的用户 token。
+- 请求未传应用凭证时,用户 token watch 回退使用 `FEISHU_APP_ID` 和 `FEISHU_APP_SECRET`,或 `ov.conf` 中的 `feishu.app_id` 和 `feishu.app_secret`。飞书 refresh token 绑定签发它的应用,因此实际使用的应用凭证必须与传入的用户 token 匹配。
+- Watch task 的 token 状态和请求传入的应用凭证保存在内部控制文件 `viking://resources/.watch_tasks.json` 中,不会出现在 watch API/MCP/CLI 返回里。若启用了 VikingFS 文件加密,该控制文件会静态加密;否则服务端控制文件中会包含这些明文私有状态。
- 本地目录输入会遵循 `.gitignore`(根目录和子目录,标准 Git 语义);`ignore_dirs`、`include`、`exclude` 会在此基础上进一步过滤。
- `args.parse_mode=no_split` 仍调用正常的格式 Parser。PDF、Word、PowerPoint、HTML 等受支持文档会转换为 Markdown,但跳过按标题、段落和长度拆分。目录导入会对每个受支持文档分别应用该规则,并继续遵循 `.gitignore`、筛选参数和 `preserve_structure`。
- 对单文件输入使用 `no_split` 时,如果解析结果恰好只有一个可见文件且未指定 `to`,该文件会直接放到解析出的父目录下(例如 `guide.md` 写入 `viking://resources/guide.md`),不会创建同名上层目录,也不会生成目录级 `.abstract.md` / `.overview.md`。如果解析结果还包含图片等其他可见文件,则保留上层目录。显式指定的 `to` 始终作为最终 URI 原样保留。
@@ -323,7 +323,9 @@ curl -X POST http://localhost:1933/api/v1/resources \
"watch_interval": 1440,
"args": {
"feishu_access_token": "u-...",
- "feishu_refresh_token": "r-..."
+ "feishu_refresh_token": "r-...",
+ "feishu_app_id": "cli_...",
+ "feishu_app_secret": "..."
}
}'
```
@@ -416,6 +418,8 @@ client.add_resource(
"args": {
"feishu_access_token": "u-...",
"feishu_refresh_token": "r-...",
+ "feishu_app_id": "cli_...",
+ "feishu_app_secret": "...",
},
},
)
@@ -490,7 +494,9 @@ ov add-resource https://example.feishu.cn/docx/doc_token \
--to viking://resources/feishu/doc \
--watch-interval 1440 \
--args feishu_access_token:u-... \
- --args feishu_refresh_token:r-...
+ --args feishu_refresh_token:r-... \
+ --args feishu_app_id:cli_... \
+ --args feishu_app_secret:...
# 添加到指定父目录(父目录必须存在)
ov add-resource ./documents/guide.md --parent viking://resources/docs
diff --git a/docs/zh/api/15-watches.md b/docs/zh/api/15-watches.md
index 15dd713e0..5b484ee6f 100644
--- a/docs/zh/api/15-watches.md
+++ b/docs/zh/api/15-watches.md
@@ -140,6 +140,7 @@ ov task watch rm viking://resources/guide.md
{
"task_id": "7f02e980-8df9-4f27-a570-4d8428cbed8a",
"path": "https://example.com/guide.md",
+ "source_type": "url",
"to_uri": "viking://resources/guide.md",
"parent_uri": "viking://resources",
"reason": "keep documentation current",
@@ -162,6 +163,10 @@ ov task watch rm viking://resources/guide.md
}
```
+`source_type` 是可选的来源元数据。显式 Connector `add_type` 优先(例如 `tos` 或
+`feishu_project`);原生导入返回 `feishu`、`git`、`url` 或 `local`。历史任务或
+无法分类的任务返回 `null`。
+
查看单个任务以及成功更新时,`result` 直接是同一结构的任务对象。删除和触发分别返回:
```json
diff --git a/docs/zh/api/17-tasks.md b/docs/zh/api/17-tasks.md
index 602ae16c8..5476f1f60 100644
--- a/docs/zh/api/17-tasks.md
+++ b/docs/zh/api/17-tasks.md
@@ -253,8 +253,11 @@ ov task cancel uuid-xxx
| task_type | str | 否 | None | 按任务类型过滤,例如 `session_commit` |
| status | str | 否 | None | 按任务状态过滤:`pending`、`running`、`cancelling`、`completed`、`failed`、`cancelled` |
| resource_id | str | 否 | None | 按资源 ID 过滤,例如会话 ID |
+| include_internal | bool | 否 | false | 是否包含 Connector 导入产生的内部子任务 |
| limit | int | 否 | 50 | 最多返回的任务条数 |
+默认仅返回用户可见任务;排查 Connector 导入时可传 `include_internal=true` 查看其内部 `add_resource` 子任务。
+
#### 3. 使用示例
**HTTP API**
diff --git a/docs/zh/guides/06-mcp-integration.md b/docs/zh/guides/06-mcp-integration.md
index 1e4cba8ce..8f03a1c3f 100644
--- a/docs/zh/guides/06-mcp-integration.md
+++ b/docs/zh/guides/06-mcp-integration.md
@@ -133,7 +133,7 @@ claude mcp add --transport http openviking \
| `remember` | 存储消息到长期记忆(触发记忆提取) | `messages`(`{role, content}` 列表) |
| `write` | 向 `viking://` 文件写入文本(创建/覆盖/追加)。自动创建缺失的父目录;覆盖前请先用 `read` 查看当前内容;只改文件局部时优先用 `edit` | `uri`, `content`, `mode`(可选:默认 `replace` — 覆盖或在缺失时创建,`append` — 追加或在缺失时创建,`create` — 已存在则失败), `wait`(可选,阻塞直到重建索引完成), `timeout`(可选) |
| `edit` | 在已有 `viking://` 文件中把精确字符串替换为新文本——用于局部修改,避免整文件重写。若 `old_string` 找不到、或匹配多处且 `replace_all` 为 false,则编辑失败且文件保持不变 | `uri`, `old_string`, `new_string`, `replace_all`(可选), `wait`(可选,阻塞直到重建索引完成), `timeout`(可选) |
-| `add_resource` | 添加本地文件或 URL 作为资源(本地文件触发渐进式上传流) | `path`, `temp_file_id`(可选), `description`(可选), `watch_interval`(可选,分钟数 — 远程 URL 的自动刷新周期), `processing_mode`(可选:默认 `semantic_and_vectors`;传 `vectors_only` 时跳过 VLM 语义理解,只向量化当前文件), `to`(可选,目标 `viking://resources/...` URI;`watch_interval > 0` 时若省略 `to`,watch 将自动绑定到本次 add 创建的资源 URI), `args`(可选,特定 parser 参数,包括 `{"parse_mode":"no_split"}` 用于正常解析但每个源文档只生成一个 Markdown 正文、飞书一次性用户 token 导入使用 `{"feishu_access_token":"u-..."}`,或飞书用户 token watch 使用 `{"feishu_access_token":"u-...","feishu_refresh_token":"r-..."}`) |
+| `add_resource` | 添加本地文件或 URL 作为资源(本地文件触发渐进式上传流) | `path`, `temp_file_id`(可选), `description`(可选), `watch_interval`(可选,分钟数 — 远程 URL 的自动刷新周期), `processing_mode`(可选:默认 `semantic_and_vectors`;传 `vectors_only` 时跳过 VLM 语义理解,只向量化当前文件), `to`(可选,目标 `viking://resources/...` URI;`watch_interval > 0` 时若省略 `to`,watch 将自动绑定到本次 add 创建的资源 URI), `args`(可选,特定 parser 参数,包括 `{"parse_mode":"no_split"}` 用于正常解析但每个源文档只生成一个 Markdown 正文、飞书一次性用户 token 导入使用 `{"feishu_access_token":"u-..."}`,或飞书用户 token watch 使用 access/refresh token,并可选传入 `feishu_app_id` / `feishu_app_secret`) |
| `list_watches` | 列出当前 Agent 可见的 watch 任务(自动刷新订阅),每行显示目标 URI、刷新间隔(分钟)、active/paused 状态以及下一次调度时间 | 无 |
| `cancel_watch` | 按目标 URI 取消(删除)watch 任务。若需调整刷新周期或临时暂停,请取消后使用新的 `watch_interval` 重新添加 | `to_uri`(必须匹配 watch 任务的 `to` 值,例如 `viking://resources/...`) |
| `grep` | 在 `viking://` 文件中进行正则内容搜索 | `uri`, `pattern`(字符串或数组), `case_insensitive`, `node_limit` |
@@ -153,7 +153,7 @@ claude mcp add --transport http openviking \
> **注**:MCP 仅暴露 watch 管理的最小闭包(`list_watches` + `cancel_watch`)。pause / resume / trigger 和统一的 `update` 动作刻意不在此处暴露,请通过 REST `/api/v1/watches/*` 接口或 `ov task watch` CLI 使用上述操作。
-> 未传 `args.feishu_access_token` 的飞书/Lark 导入保持现有应用/tenant token 行为,也支持 watch。飞书/Lark 一次性用户 token 导入只传 `args.feishu_access_token`;飞书/Lark 用户 token watch 还必须传 `args.feishu_refresh_token`,并要求 OpenViking 服务端配置同一个飞书应用凭证。
+> 未传 `args.feishu_access_token` 的飞书/Lark 导入保持现有应用/tenant token 行为,也支持 watch。一次性用户 token 导入只传 `args.feishu_access_token`;用户 token watch 还必须传 `args.feishu_refresh_token`。可为该 watch 同时传入 `args.feishu_app_id` 和 `args.feishu_app_secret`,也可回退使用服务端应用凭证;实际使用的应用必须与用户 token 的签发应用一致。
> `processing_mode=vectors_only` 会跳过 VLM 语义理解阶段,不生成或刷新 `.abstract.md` / `.overview.md`;它只向量化当前非隐藏资源文件,并保留已存在的旧语义产物。
diff --git a/openviking/parse/accessors/web_importer.py b/openviking/parse/accessors/web_importer.py
index 06a099587..5b781cc30 100644
--- a/openviking/parse/accessors/web_importer.py
+++ b/openviking/parse/accessors/web_importer.py
@@ -20,7 +20,7 @@ from openviking.parse.accessors.web_feed_accessor import (
_sanitize_filename,
url_to_relpath,
)
-from openviking_cli.exceptions import InvalidArgumentError
+from openviking_cli.exceptions import FailedPreconditionError, InvalidArgumentError
DEPTH_UNLIMITED = -1
MAX_PAGES_UNLIMITED = -1
@@ -93,7 +93,10 @@ class WebImporter:
success_pages = self._dedupe_success_pages(crawl_result.pages)
if not any(page.depth == 0 for page in success_pages):
detail = self._entry_failure_detail(crawl_result.pages)
- raise RuntimeError(_entry_failure_message(root_url, detail))
+ message = _entry_failure_message(root_url, detail)
+ if detail and "robots.txt" in detail.lower():
+ raise FailedPreconditionError(message)
+ raise RuntimeError(message)
temp_root = Path(tempfile.mkdtemp(prefix="ov_web_"))
temp_dir = temp_root / _host_name(root_url)
diff --git a/openviking/resource/feishu_watch_auth.py b/openviking/resource/feishu_watch_auth.py
index 1c72b7849..2415ad1d6 100644
--- a/openviking/resource/feishu_watch_auth.py
+++ b/openviking/resource/feishu_watch_auth.py
@@ -13,6 +13,8 @@ from typing import Any, Dict, Optional
FEISHU_AUTH_PROVIDER = "feishu"
FEISHU_ACCESS_TOKEN_ARG = "feishu_access_token"
FEISHU_REFRESH_TOKEN_ARG = "feishu_refresh_token"
+FEISHU_APP_ID_ARG = "feishu_app_id"
+FEISHU_APP_SECRET_ARG = "feishu_app_secret"
FEISHU_REFRESH_GRANT_TYPE = "refresh_token"
FEISHU_REFRESH_SKEW = timedelta(minutes=5)
@@ -44,34 +46,50 @@ class FeishuTenantTokenError(Exception):
"""Raised when an app credential cannot produce a tenant token."""
-def load_feishu_app_credentials() -> FeishuAppCredentials:
+def load_feishu_app_credentials(
+ *,
+ app_id: Optional[str] = None,
+ app_secret: Optional[str] = None,
+) -> FeishuAppCredentials:
"""Load Feishu app credentials from OpenViking config or environment."""
from openviking_cli.utils.config import get_openviking_config
config = get_openviking_config().feishu
- app_id = (config.app_id or os.getenv("FEISHU_APP_ID", "")).strip()
- app_secret = (config.app_secret or os.getenv("FEISHU_APP_SECRET", "")).strip()
- if not app_id or not app_secret:
+ if app_id is None and app_secret is None:
+ resolved_app_id = (config.app_id or os.getenv("FEISHU_APP_ID", "")).strip()
+ resolved_app_secret = (config.app_secret or os.getenv("FEISHU_APP_SECRET", "")).strip()
+ else:
+ resolved_app_id = app_id.strip() if isinstance(app_id, str) else ""
+ resolved_app_secret = app_secret.strip() if isinstance(app_secret, str) else ""
+ if not resolved_app_id or not resolved_app_secret:
raise ValueError(
"Feishu credentials not configured. Set FEISHU_APP_ID and "
"FEISHU_APP_SECRET environment variables, or configure in ov.conf."
)
return FeishuAppCredentials(
- app_id=app_id,
- app_secret=app_secret,
+ app_id=resolved_app_id,
+ app_secret=resolved_app_secret,
domain=(config.domain or "https://open.feishu.cn").strip(),
request_timeout=float(config.request_timeout or 30.0),
)
-def create_feishu_auth_state(access_token: str, refresh_token: str) -> Dict[str, Any]:
+def create_feishu_auth_state(
+ access_token: str,
+ refresh_token: str,
+ app_credentials: Optional[FeishuAppCredentials] = None,
+) -> Dict[str, Any]:
"""Create the private watch auth state for Feishu user-token watch tasks."""
- return {
+ state = {
"provider": FEISHU_AUTH_PROVIDER,
"access_token": access_token,
"refresh_token": refresh_token,
"expires_at": None,
}
+ if app_credentials is not None:
+ state["app_id"] = app_credentials.app_id
+ state["app_secret"] = app_credentials.app_secret
+ return state
def is_feishu_auth_state(auth_state: Optional[Dict[str, Any]]) -> bool:
@@ -127,6 +145,21 @@ class FeishuOAuthClient:
def from_config(cls) -> "FeishuOAuthClient":
return cls(load_feishu_app_credentials())
+ @classmethod
+ def from_auth_state(cls, auth_state: Dict[str, Any]) -> "FeishuOAuthClient":
+ app_id = auth_state.get("app_id")
+ app_secret = auth_state.get("app_secret")
+ if app_id is None and app_secret is None:
+ return cls.from_config()
+ try:
+ credentials = load_feishu_app_credentials(app_id=app_id, app_secret=app_secret)
+ except ValueError as exc:
+ raise FeishuTokenRefreshError(
+ "Feishu app credentials in watch task are invalid.",
+ permanent=True,
+ ) from exc
+ return cls(credentials)
+
async def get_tenant_access_token(self) -> str:
return await asyncio.to_thread(self._get_tenant_access_token_sync)
diff --git a/openviking/resource/watch_manager.py b/openviking/resource/watch_manager.py
index 5add06103..8f8682ab9 100644
--- a/openviking/resource/watch_manager.py
+++ b/openviking/resource/watch_manager.py
@@ -59,6 +59,7 @@ class WatchTask(BaseModel):
default_factory=lambda: str(uuid.uuid4()), description="Unique task identifier"
)
path: str = Field(..., description="Resource path to monitor")
+ source_type: Optional[str] = Field(None, description="Original resource source type")
to_uri: Optional[str] = Field(None, description="Target URI")
to_is_directory: Optional[bool] = Field(
None,
@@ -97,6 +98,7 @@ class WatchTask(BaseModel):
return {
"task_id": self.task_id,
"path": self.path,
+ "source_type": self.source_type,
"to_uri": self.to_uri,
"parent_uri": self.parent_uri,
"reason": self.reason,
@@ -391,6 +393,7 @@ class WatchManager:
account_id: str = "default",
user_id: str = "default",
original_role: str = "user",
+ source_type: Optional[str] = None,
to_uri: Optional[str] = None,
to_is_directory: Optional[bool] = None,
parent_uri: Optional[str] = None,
@@ -419,6 +422,7 @@ class WatchManager:
task = WatchTask(
path=path,
+ source_type=source_type,
to_uri=to_uri,
to_is_directory=to_is_directory,
parent_uri=parent_uri,
@@ -456,6 +460,7 @@ class WatchManager:
user_id: str,
role: str,
path: Optional[str] = None,
+ source_type: Optional[str] = None,
to_uri: Optional[str] = None,
to_is_directory: Optional[bool] = None,
parent_uri: Optional[str] = None,
@@ -497,6 +502,7 @@ class WatchManager:
account_id=account_id,
user_id=user_id,
path=path,
+ source_type=source_type,
to_uri=to_uri,
to_is_directory=to_is_directory,
parent_uri=parent_uri,
@@ -518,6 +524,7 @@ class WatchManager:
account_id: str,
user_id: str,
path: Optional[str],
+ source_type: Optional[str],
to_uri: Optional[str],
to_is_directory: Optional[bool],
parent_uri: Optional[str],
@@ -541,6 +548,8 @@ class WatchManager:
old_to_uri = task.to_uri
if path is not None:
task.path = path
+ if source_type is not None:
+ task.source_type = source_type
if to_uri is not None:
task.to_uri = to_uri
if to_is_directory is not None:
diff --git a/openviking/resource/watch_scheduler.py b/openviking/resource/watch_scheduler.py
index 53666f322..aea690a37 100644
--- a/openviking/resource/watch_scheduler.py
+++ b/openviking/resource/watch_scheduler.py
@@ -75,7 +75,6 @@ class WatchScheduler:
self._scheduler_task: Optional[asyncio.Task] = None
self._executing_tasks: Set[str] = set()
self._lock = asyncio.Lock()
- self._feishu_oauth_client: Optional[Any] = None
@property
def watch_manager(self) -> Optional[WatchManager]:
@@ -435,17 +434,14 @@ class WatchScheduler:
return auth_state
refresh_token = auth_state.get("refresh_token")
- refreshed = await self._get_feishu_oauth_client().refresh_user_access_token(refresh_token)
+ refreshed = await FeishuOAuthClient.from_auth_state(auth_state).refresh_user_access_token(
+ refresh_token
+ )
updated = apply_feishu_refreshed_token(auth_state, refreshed)
if self._watch_manager is not None:
await self._watch_manager.update_auth_state(task.task_id, updated)
return updated
- def _get_feishu_oauth_client(self):
- if self._feishu_oauth_client is None:
- self._feishu_oauth_client = FeishuOAuthClient.from_config()
- return self._feishu_oauth_client
-
def _check_resource_exists(self, path: str) -> bool:
"""Check if a resource path exists.
diff --git a/openviking/server/openviking_assets.py b/openviking/server/openviking_assets.py
index 43c959708..ab1ed47e7 100644
--- a/openviking/server/openviking_assets.py
+++ b/openviking/server/openviking_assets.py
@@ -10,6 +10,7 @@ import hashlib
import math
import os
import re
+import signal
from typing import Any
import yaml
@@ -22,6 +23,7 @@ from openviking.server.local_input_guard import require_remote_resource_source
from openviking.utils.git_auth import (
GitHttpAuthConfig,
build_git_http_auth_env,
+ reject_git_http_userinfo,
)
from openviking_cli.exceptions import (
DeadlineExceededError,
@@ -281,6 +283,7 @@ async def preflight_git_repository(
"""Verify that the server can read a Git source without cloning or creating a task."""
_validate_clone_url(repo_url, asset_name)
+ reject_git_http_userinfo(repo_url)
repo_url = require_remote_resource_source(repo_url)
if timeout <= 0 or not math.isfinite(timeout):
raise InvalidArgumentError("Git preflight timeout must be a positive finite number")
@@ -344,21 +347,23 @@ async def preflight_git_repository(
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=env,
+ start_new_session=os.name == "posix",
)
except FileNotFoundError as exc:
raise UnavailableError("git", "executable not found on the OpenViking Server") from exc
try:
_stdout, stderr = await asyncio.wait_for(process.communicate(), timeout=timeout)
- except TimeoutError as exc:
- process.kill()
- await process.communicate()
- raise DeadlineExceededError("Git repository permission preflight", timeout) from exc
- except BaseException:
+ except BaseException as exc:
with contextlib.suppress(ProcessLookupError):
- process.kill()
+ if os.name == "posix":
+ os.killpg(process.pid, signal.SIGKILL)
+ else:
+ process.kill()
with contextlib.suppress(Exception):
- await asyncio.shield(process.wait())
+ await asyncio.wait_for(asyncio.shield(process.communicate()), timeout=1.0)
+ if isinstance(exc, asyncio.TimeoutError):
+ raise DeadlineExceededError("Git repository permission preflight", timeout) from exc
raise
locator = normalize_repo_url(repo_url)
diff --git a/openviking/server/routers/resources.py b/openviking/server/routers/resources.py
index 4b8fae761..8828c2938 100644
--- a/openviking/server/routers/resources.py
+++ b/openviking/server/routers/resources.py
@@ -24,6 +24,9 @@ from openviking_cli.exceptions import InvalidArgumentError
router = APIRouter(prefix="/api/v1", tags=["resources"])
+_CONNECTOR_TASK_ORIGIN_HEADER = "X-OpenViking-Task-Origin"
+_CONNECTOR_TASK_ORIGIN = "connector_import"
+
class AddResourceRequest(BaseModel):
"""Request model for add_resource.
@@ -63,7 +66,8 @@ class AddResourceRequest(BaseModel):
watch_interval > 0 the credentials are stored in private watch state.
For Feishu one-time user-token imports,
pass {"feishu_access_token": "..."}. For Feishu user-token watches,
- pass {"feishu_access_token": "...", "feishu_refresh_token": "..."}.
+ also pass "feishu_refresh_token". The optional "feishu_app_id" and
+ "feishu_app_secret" pair overrides the server app for that watch.
watch_interval: Watch interval in minutes for automatic resource monitoring.
- watch_interval > 0: Creates or updates a watch task. The resource will be
automatically re-processed at the specified interval.
@@ -296,6 +300,10 @@ async def add_resource(
tag_mode=request.tag_mode,
allow_local_path_resolution=allow_local_path_resolution,
enforce_public_remote_targets=True,
+ internal_task=(
+ http_request.headers.get(_CONNECTOR_TASK_ORIGIN_HEADER, "").strip().lower()
+ == _CONNECTOR_TASK_ORIGIN
+ ),
args=request.args,
**kwargs,
)
diff --git a/openviking/server/routers/tasks.py b/openviking/server/routers/tasks.py
index ccc52e141..6c0f13ea8 100644
--- a/openviking/server/routers/tasks.py
+++ b/openviking/server/routers/tasks.py
@@ -89,6 +89,7 @@ async def list_tasks(
description="Filter by status (pending/running/cancelling/completed/failed/cancelled)",
),
resource_id: Optional[str] = Query(None, description="Filter by resource ID (e.g. session_id)"),
+ include_internal: bool = Query(False, description="Include internal Connector child tasks"),
limit: int = Query(50, le=200, description="Max results"),
_ctx: RequestContext = Depends(get_request_context),
):
@@ -102,12 +103,14 @@ async def list_tasks(
limit=limit,
account_id=SYSTEM_TASK_ACCOUNT_ID,
user_id=SYSTEM_TASK_USER_ID,
+ include_internal=include_internal,
)
cached_tasks = await tracker.list_tasks(
task_type=task_type,
status=status,
resource_id=resource_id,
limit=limit,
+ include_internal=include_internal,
)
tasks_by_id = {task.task_id: task for task in cached_tasks}
tasks_by_id.update({task.task_id: task for task in system_tasks})
@@ -120,5 +123,6 @@ async def list_tasks(
limit=limit,
account_id=_ctx.account_id,
user_id=_ctx.user.user_id,
+ include_internal=include_internal,
)
return Response(status="ok", result=[t.to_dict() for t in tasks])
diff --git a/openviking/service/resource_service.py b/openviking/service/resource_service.py
index ca2fc1901..d00e31109 100644
--- a/openviking/service/resource_service.py
+++ b/openviking/service/resource_service.py
@@ -10,6 +10,8 @@ import asyncio
import contextlib
import inspect
import json
+import os
+import signal
import time
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
@@ -24,8 +26,11 @@ from openviking.parse.mode import ParseMode, normalize_parse_mode
from openviking.parse.parsers.constants import MPEG_TS_EXTENSION_ALIAS
from openviking.resource.feishu_watch_auth import (
FEISHU_ACCESS_TOKEN_ARG,
+ FEISHU_APP_ID_ARG,
+ FEISHU_APP_SECRET_ARG,
FEISHU_AUTH_PROVIDER,
FEISHU_REFRESH_TOKEN_ARG,
+ FeishuAppCredentials,
create_feishu_auth_state,
is_feishu_auth_state,
load_feishu_app_credentials,
@@ -128,6 +133,7 @@ _ADD_RESOURCE_ARGS_RESERVED_FIELDS = frozenset(
"prepared_resource",
"tags",
"tag_mode",
+ "internal_task",
}
)
_ADD_RESOURCE_TRANSIENT_ARGS = frozenset({"tos_signature", "tos_access"})
@@ -251,6 +257,20 @@ class ResourceService:
watch_kwargs["tag_mode"] = tag_mode
return watch_kwargs
+ @staticmethod
+ def _infer_watch_source_type(path: str) -> Optional[str]:
+ if not path:
+ return None
+ from openviking.parse.accessors.feishu_accessor import FeishuAccessor
+
+ if FeishuAccessor._is_feishu_url(path):
+ return "feishu"
+ if is_git_repo_url(path):
+ return "git"
+ if is_remote_resource_source(path):
+ return "url"
+ return "local"
+
def _validate_add_resource_tag_policy(
self,
*,
@@ -298,6 +318,7 @@ class ResourceService:
processor_kwargs: Dict[str, Any],
watch_auth_state: Optional[Dict[str, Any]],
ctx: RequestContext,
+ source_type: Optional[str] = None,
) -> None:
if not watch_manager or not manage_watch:
return
@@ -339,6 +360,7 @@ class ResourceService:
processing_mode=processing_mode,
processor_kwargs=sanitized,
auth_state=watch_auth_state,
+ source_type=source_type or self._infer_watch_source_type(path),
ctx=ctx,
)
except ConflictError:
@@ -386,6 +408,9 @@ class ResourceService:
raise InvalidArgumentError(str(exc).replace("parse_mode", "args.parse_mode")) from exc
token = normalized.get(FEISHU_ACCESS_TOKEN_ARG)
refresh_token = normalized.pop(FEISHU_REFRESH_TOKEN_ARG, None)
+ app_id = normalized.pop(FEISHU_APP_ID_ARG, None)
+ app_secret = normalized.pop(FEISHU_APP_SECRET_ARG, None)
+ has_app_credentials = app_id is not None or app_secret is not None
watch_auth_state = None
if token is not None:
if not isinstance(token, str) or not token.strip():
@@ -398,28 +423,61 @@ class ResourceService:
"args.feishu_refresh_token must be a non-empty string when "
"args.feishu_access_token is used with watch_interval > 0."
)
- self._ensure_feishu_credentials_for_watch()
- watch_auth_state = create_feishu_auth_state(token, refresh_token.strip())
+ app_credentials = self._load_feishu_credentials_for_watch(app_id, app_secret)
+ watch_auth_state = create_feishu_auth_state(
+ token,
+ refresh_token.strip(),
+ app_credentials,
+ )
elif refresh_token is not None:
raise InvalidArgumentError(
"args.feishu_refresh_token is only supported with "
"args.feishu_access_token and watch_interval > 0."
)
+ elif has_app_credentials:
+ raise InvalidArgumentError(
+ "args.feishu_app_id and args.feishu_app_secret are only supported with "
+ "args.feishu_access_token and watch_interval > 0."
+ )
elif refresh_token is not None:
raise InvalidArgumentError(
"args.feishu_refresh_token requires args.feishu_access_token."
)
+ elif has_app_credentials:
+ raise InvalidArgumentError(
+ "args.feishu_app_id and args.feishu_app_secret require "
+ "args.feishu_access_token and watch_interval > 0."
+ )
return _NormalizedAddResourceArgs(normalized, watch_auth_state, parse_mode)
- def _ensure_feishu_credentials_for_watch(self) -> None:
+ def _load_feishu_credentials_for_watch(
+ self,
+ app_id: Any,
+ app_secret: Any,
+ ) -> Optional[FeishuAppCredentials]:
+ supplied = app_id is not None or app_secret is not None
+ if supplied and (
+ not isinstance(app_id, str)
+ or not app_id.strip()
+ or not isinstance(app_secret, str)
+ or not app_secret.strip()
+ ):
+ raise InvalidArgumentError(
+ "args.feishu_app_id and args.feishu_app_secret must be non-empty strings "
+ "and provided together."
+ )
try:
- load_feishu_app_credentials()
+ credentials = load_feishu_app_credentials(
+ app_id=app_id.strip() if supplied else None,
+ app_secret=app_secret.strip() if supplied else None,
+ )
except Exception as exc:
raise InvalidArgumentError(
"Feishu user-token watch requires FEISHU_APP_ID and "
"FEISHU_APP_SECRET, or feishu.app_id and feishu.app_secret in ov.conf."
) from exc
+ return credentials if supplied else None
def _ensure_initialized(self) -> None:
"""Ensure all dependencies are initialized."""
@@ -504,7 +562,10 @@ class ResourceService:
account_id=msg.account_id,
user_id=msg.user_id,
task_id=msg.task_id,
- meta={"source_path": msg.source_path},
+ meta={
+ "source_path": msg.source_path,
+ **({"internal": True} if msg.internal_task else {}),
+ },
auth=task_auth,
)
await get_queue_manager().enqueue(queue_name, msg.to_dict())
@@ -625,6 +686,7 @@ class ResourceService:
stage_callback=stage_callback,
watch_auth_state=watch_auth_state,
prepared_resource=prepared_resource,
+ internal_task=msg.internal_task,
**internal_kwargs,
)
if msg.staged_source is not None:
@@ -897,6 +959,7 @@ class ResourceService:
allow_local_path_resolution: bool,
enforce_public_remote_targets: bool,
processor_kwargs: Dict[str, Any],
+ internal_task: bool,
) -> Dict[str, Any]:
from openviking.storage.queuefs.add_resource_msg import AddResourceMsg
@@ -973,6 +1036,7 @@ class ResourceService:
args=plan.processor_args,
defer_target_resolution=defer_target_resolution,
understanding_response_id=plan.understanding_response_id,
+ internal_task=internal_task,
)
def transfer_staged_source() -> None:
@@ -1077,18 +1141,27 @@ class ResourceService:
env=env,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
+ start_new_session=os.name == "posix",
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=10.0)
- except TimeoutError as exc:
+ except BaseException as exc:
if proc is not None:
+ with contextlib.suppress(ProcessLookupError):
+ if os.name == "posix":
+ os.killpg(proc.pid, signal.SIGKILL)
+ else:
+ proc.kill()
with contextlib.suppress(Exception):
- proc.kill()
- await proc.communicate()
- raise InvalidArgumentError(
- "Cannot access Git repository; the preflight timed out after 10s."
- ) from exc
- except Exception as exc:
- raise InvalidArgumentError("Cannot access Git repository during preflight.") from exc
+ await asyncio.wait_for(
+ asyncio.shield(proc.communicate()), timeout=1.0
+ )
+ if isinstance(exc, asyncio.TimeoutError):
+ raise InvalidArgumentError(
+ "Cannot access Git repository; the preflight timed out after 10s."
+ ) from exc
+ if isinstance(exc, Exception):
+ raise InvalidArgumentError("Cannot access Git repository during preflight.") from exc
+ raise
if proc.returncode != 0:
raise InvalidArgumentError("Cannot access Git repository; git ls-remote failed.")
@@ -1118,6 +1191,7 @@ class ResourceService:
allow_local_path_resolution: bool = True,
enforce_public_remote_targets: bool = False,
add_type: Optional[str] = None,
+ internal_task: bool = False,
args: Optional[Dict[str, Any]] = None,
**kwargs,
) -> Dict[str, Any]:
@@ -1149,6 +1223,7 @@ class ResourceService:
tag_mode=tag_mode,
allow_local_path_resolution=allow_local_path_resolution,
enforce_public_remote_targets=enforce_public_remote_targets,
+ internal_task=internal_task,
args=args,
**kwargs,
)
@@ -1218,6 +1293,7 @@ class ResourceService:
parse_mode: ParseMode | str | None = None,
allow_local_path_resolution: bool = True,
enforce_public_remote_targets: bool = False,
+ internal_task: bool = False,
args: Optional[Dict[str, Any]] = None,
**kwargs,
) -> Dict[str, Any]:
@@ -1410,6 +1486,7 @@ class ResourceService:
processor_kwargs=connector_watch_processor_kwargs,
watch_auth_state=watch_auth_state,
ctx=ctx,
+ source_type=resolved[0],
)
on_success = create_watch_after_success
@@ -1444,6 +1521,7 @@ class ResourceService:
processor_kwargs=connector_watch_processor_kwargs,
watch_auth_state=watch_auth_state,
ctx=ctx,
+ source_type=resolved[0],
)
return result
@@ -1481,6 +1559,7 @@ class ResourceService:
allow_local_path_resolution=allow_local_path_resolution,
enforce_public_remote_targets=enforce_public_remote_targets,
processor_kwargs=kwargs,
+ internal_task=internal_task,
)
else:
result = await self._execute_resource_ingestion(
@@ -1504,6 +1583,7 @@ class ResourceService:
allow_local_path_resolution=allow_local_path_resolution,
enforce_public_remote_targets=enforce_public_remote_targets,
watch_auth_state=normalized_args.watch_auth_state,
+ internal_task=internal_task,
**kwargs,
)
get_current_telemetry().set("resource.flags.wait", wait)
@@ -1560,6 +1640,7 @@ class ResourceService:
resource_lock: Optional[Dict[str, Any]] = None,
stage_callback: Optional[Callable[[str], Any]] = None,
prepared_resource: Optional["LocalResource"] = None,
+ internal_task: bool = False,
**kwargs,
) -> Dict[str, Any]:
"""Execute an already-routed resource ingestion."""
@@ -1688,6 +1769,7 @@ class ResourceService:
skip_watch_management=True,
tags=tags,
tag_mode=tag_mode,
+ internal_task=internal_task,
)
enqueue_lock = deferred_lock
deferred_lock = None
@@ -1849,6 +1931,7 @@ class ResourceService:
processing_mode: ProcessingMode,
processor_kwargs: Dict[str, Any],
auth_state: Optional[Dict[str, Any]],
+ source_type: Optional[str],
ctx: RequestContext,
) -> None:
"""Handle creation or update of watch task.
@@ -1884,6 +1967,7 @@ class ResourceService:
user_id=ctx.user.user_id,
role=str(ctx.role),
path=path,
+ source_type=source_type,
to_uri=to_uri,
to_is_directory=to_is_directory,
parent_uri=parent_uri,
@@ -1907,6 +1991,7 @@ class ResourceService:
account_id=ctx.account_id,
user_id=ctx.user.user_id,
original_role=str(ctx.role),
+ source_type=source_type,
to_uri=to_uri,
to_is_directory=to_is_directory,
parent_uri=parent_uri,
diff --git a/openviking/service/task_tracker.py b/openviking/service/task_tracker.py
index 04874001b..cd1df387d 100644
--- a/openviking/service/task_tracker.py
+++ b/openviking/service/task_tracker.py
@@ -877,6 +877,7 @@ class TaskTracker:
limit: int = 50,
account_id: Optional[str] = None,
user_id: Optional[str] = None,
+ include_internal: bool = True,
) -> List[TaskRecord]:
"""List tasks with optional filters. Most-recent first. Returns snapshot copies."""
return await self._dispatcher.run(
@@ -887,6 +888,7 @@ class TaskTracker:
limit,
account_id,
user_id,
+ include_internal,
)
)
@@ -898,11 +900,14 @@ class TaskTracker:
limit: int,
account_id: Optional[str],
user_id: Optional[str],
+ include_internal: bool,
) -> List[TaskRecord]:
if account_id is not None:
self._merge_loaded_tasks(await self._load_all_from_store(account_id, user_id))
source = self._cache_snapshot()
tasks = [self._copy(t) for t in source if self._matches_owner(t, account_id, user_id)]
+ if not include_internal:
+ tasks = [t for t in tasks if t.meta.get("internal") is not True]
if task_type:
tasks = [t for t in tasks if t.task_type == task_type]
if status:
diff --git a/openviking/storage/queuefs/add_resource_msg.py b/openviking/storage/queuefs/add_resource_msg.py
index ae0cb7461..ff6f1b564 100644
--- a/openviking/storage/queuefs/add_resource_msg.py
+++ b/openviking/storage/queuefs/add_resource_msg.py
@@ -55,6 +55,7 @@ class AddResourceMsg:
parse_mode: str = "default"
tags: Optional[list[str]] = None
tag_mode: str = "replace"
+ internal_task: bool = False
def __post_init__(self) -> None:
inferred = (
@@ -174,4 +175,5 @@ class AddResourceMsg:
parse_mode=str(data.get("parse_mode") or "default"),
tags=(list(data["tags"]) if isinstance(data.get("tags"), list) else None),
tag_mode=str(data.get("tag_mode") or "replace"),
+ internal_task=bool(data.get("internal_task", False)),
)
diff --git a/openviking/storage/queuefs/add_resource_processor.py b/openviking/storage/queuefs/add_resource_processor.py
index 3414ffa09..d7fda1189 100644
--- a/openviking/storage/queuefs/add_resource_processor.py
+++ b/openviking/storage/queuefs/add_resource_processor.py
@@ -114,7 +114,10 @@ class AddResourceProcessor(DequeueHandlerBase):
account_id=ctx.account_id,
user_id=ctx.user.user_id,
task_id=msg.task_id,
- meta={"source_path": msg.source_path},
+ meta={
+ "source_path": msg.source_path,
+ **({"internal": True} if msg.internal_task else {}),
+ },
)
if task.status in (
TaskStatus.CANCELLING,
diff --git a/tests/parse/test_feishu_parser_api.py b/tests/parse/test_feishu_parser_api.py
index 3da2820b0..118c29bc3 100644
--- a/tests/parse/test_feishu_parser_api.py
+++ b/tests/parse/test_feishu_parser_api.py
@@ -383,6 +383,7 @@ def test_add_resource_message_round_trips_internal_fields():
role="user",
defer_target_resolution=True,
understanding_response_id="response-1",
+ internal_task=True,
)
restored = AddResourceMsg.from_dict(msg.to_dict())
@@ -392,6 +393,7 @@ def test_add_resource_message_round_trips_internal_fields():
assert restored.defer_target_resolution is True
assert restored.understanding_response_id == "response-1"
assert restored.job_phase is AddResourcePhase.SOURCE
+ assert restored.internal_task is True
def test_add_resource_message_round_trips_processing_mode():
@@ -502,6 +504,7 @@ async def test_uat_producer_payload_reaches_worker_without_persisting_token(
assert task_tracker.create.await_args.kwargs["resource_id"] == (
None if preflight_name is None else root_uri
)
+ assert task_tracker.create.await_args.kwargs["meta"] == {"source_path": source}
submit_understanding.assert_awaited_once_with(
source,
feishu_access_token="u-secret",
@@ -912,6 +915,7 @@ async def test_add_resource_processor_persists_final_uri_and_cleans_staged_sourc
role="user",
telemetry_id=telemetry_id,
defer_target_resolution=True,
+ internal_task=True,
staged_source={
"temp_uri": "viking://temp/account-1/task-1",
"source_uri": "viking://temp/account-1/task-1/source/document.md",
@@ -937,7 +941,7 @@ async def test_add_resource_processor_persists_final_uri_and_cleans_staged_sourc
account_id="account-1",
user_id="user-1",
task_id="task-1",
- meta={"source_path": ""},
+ meta={"source_path": "", "internal": True},
)
assert task_tracker.complete.await_count == 2
first_complete = task_tracker.complete.await_args_list[0]
diff --git a/tests/resource/test_feishu_watch_auth.py b/tests/resource/test_feishu_watch_auth.py
index 2f993b0af..8a4102b4d 100644
--- a/tests/resource/test_feishu_watch_auth.py
+++ b/tests/resource/test_feishu_watch_auth.py
@@ -4,6 +4,7 @@
from datetime import datetime, timedelta, timezone
+from openviking.resource import feishu_watch_auth
from openviking.resource.feishu_watch_auth import (
FeishuAppCredentials,
FeishuOAuthClient,
@@ -14,6 +15,26 @@ from openviking.resource.feishu_watch_auth import (
)
+def test_oauth_client_uses_watch_app_credentials(monkeypatch):
+ seen = {}
+
+ def fake_load_credentials(*, app_id=None, app_secret=None):
+ seen.update(app_id=app_id, app_secret=app_secret)
+ return FeishuAppCredentials(
+ app_id=app_id,
+ app_secret=app_secret,
+ domain="https://open.feishu.cn",
+ request_timeout=30,
+ )
+
+ monkeypatch.setattr(feishu_watch_auth, "load_feishu_app_credentials", fake_load_credentials)
+
+ client = FeishuOAuthClient.from_auth_state({"app_id": "cli-test", "app_secret": "secret-test"})
+
+ assert seen == {"app_id": "cli-test", "app_secret": "secret-test"}
+ assert client._credentials.app_id == "cli-test"
+
+
def test_feishu_auth_state_refresh_window():
now = datetime(2026, 6, 10, 12, 0, tzinfo=timezone.utc)
state = create_feishu_auth_state("u-old", "r-old")
diff --git a/tests/resource/test_watch_manager.py b/tests/resource/test_watch_manager.py
index 95251d9a0..937fb8bed 100644
--- a/tests/resource/test_watch_manager.py
+++ b/tests/resource/test_watch_manager.py
@@ -62,6 +62,22 @@ async def mock_viking_fs(temp_storage: Path) -> MockVikingFS:
return MockVikingFS(root_path=str(temp_storage))
+@pytest_asyncio.fixture
+async def watch_manager(mock_viking_fs: MockVikingFS) -> WatchManager:
+ """Create an initialized WatchManager with isolated storage."""
+ manager = WatchManager(viking_fs=mock_viking_fs)
+ await manager.initialize()
+ return manager
+
+
+@pytest_asyncio.fixture
+async def watch_manager_no_fs() -> WatchManager:
+ """Create an initialized in-memory WatchManager."""
+ manager = WatchManager(viking_fs=None)
+ await manager.initialize()
+ return manager
+
+
class TestWatchTask:
"""Tests for WatchTask data model."""
@@ -70,6 +86,7 @@ class TestWatchTask:
task = WatchTask(path="/test/path")
assert task.path == "/test/path"
+ assert task.source_type is None
assert task.task_id is not None
assert task.to_uri is None
assert task.parent_uri is None
@@ -88,6 +105,7 @@ class TestWatchTask:
task = WatchTask(
task_id="test-task-id",
path="/test/path",
+ source_type="feishu_project",
to_uri="viking://resources/test",
parent_uri="viking://resources",
reason="Test reason",
@@ -102,6 +120,7 @@ class TestWatchTask:
assert task.task_id == "test-task-id"
assert task.path == "/test/path"
+ assert task.source_type == "feishu_project"
assert task.to_uri == "viking://resources/test"
assert task.parent_uri == "viking://resources"
assert task.reason == "Test reason"
@@ -118,6 +137,7 @@ class TestWatchTask:
task = WatchTask(
task_id="test-id",
path="/test/path",
+ source_type="url",
to_uri="viking://test",
auth_state={
"provider": "feishu",
@@ -132,6 +152,7 @@ class TestWatchTask:
assert data["task_id"] == "test-id"
assert data["path"] == "/test/path"
+ assert data["source_type"] == "url"
assert data["to_uri"] == "viking://test"
assert data["created_at"] == now.isoformat()
assert data["is_active"] is True
@@ -144,6 +165,7 @@ class TestWatchTask:
data = {
"task_id": "test-id",
"path": "/test/path",
+ "source_type": "git",
"to_uri": "viking://test",
"parent_uri": "viking://parent",
"reason": "Test",
@@ -160,6 +182,7 @@ class TestWatchTask:
assert task.task_id == "test-id"
assert task.path == "/test/path"
+ assert task.source_type == "git"
assert task.to_uri == "viking://test"
assert task.watch_interval == 45.0
assert task.processing_mode == "vectors_only"
@@ -172,6 +195,7 @@ class TestWatchTask:
assert task.processing_mode == "semantic_and_vectors"
assert task.to_is_directory is None
+ assert task.source_type is None
def test_calculate_next_execution_time(self):
"""Test calculating next execution time."""
@@ -596,6 +620,7 @@ class TestWatchManagerPersistence:
task = await manager1.create_task(
path="/test/path",
+ source_type="local",
to_uri="viking://resources/test",
reason="Test task",
watch_interval=45.0,
@@ -609,6 +634,7 @@ class TestWatchManagerPersistence:
assert loaded_task is not None
assert loaded_task.path == "/test/path"
+ assert loaded_task.source_type == "local"
assert loaded_task.to_uri == "viking://resources/test"
assert loaded_task.reason == "Test task"
assert loaded_task.watch_interval == 45.0
diff --git a/tests/server/test_api_resources.py b/tests/server/test_api_resources.py
index 3a7d0278b..6832c1d1b 100644
--- a/tests/server/test_api_resources.py
+++ b/tests/server/test_api_resources.py
@@ -171,6 +171,30 @@ async def test_add_resource_forwards_args_to_service(
assert resp.status_code == 200
assert seen["args"] == {"feishu_access_token": "u-test"}
+ assert seen["internal_task"] is False
+
+
+async def test_add_resource_marks_connector_child_task_internal(
+ client: httpx.AsyncClient,
+ service,
+ monkeypatch,
+):
+ seen = {}
+
+ async def fake_add_resource(**kwargs):
+ seen.update(kwargs)
+ return {"status": "success", "root_uri": "viking://resources/demo"}
+
+ monkeypatch.setattr(service.resources, "add_resource", fake_add_resource)
+
+ resp = await client.post(
+ "/api/v1/resources",
+ headers={"X-OpenViking-Task-Origin": "connector_import"},
+ json={"path": "https://example.com/demo.md"},
+ )
+
+ assert resp.status_code == 200
+ assert seen["internal_task"] is True
async def test_add_resource_forwards_processing_mode_to_service(monkeypatch):
@@ -187,7 +211,10 @@ async def test_add_resource_forwards_processing_mode_to_service(monkeypatch):
monkeypatch.setattr(resources_router, "get_service", lambda: service)
response = await resources_router.add_resource(
- SimpleNamespace(app=SimpleNamespace(state=SimpleNamespace(config=None))),
+ SimpleNamespace(
+ app=SimpleNamespace(state=SimpleNamespace(config=None)),
+ headers={},
+ ),
AddResourceRequest(
path="https://example.com/demo.md",
processing_mode="vectors_only",
diff --git a/tests/server/test_api_watches.py b/tests/server/test_api_watches.py
index 3e732b991..8d770831e 100644
--- a/tests/server/test_api_watches.py
+++ b/tests/server/test_api_watches.py
@@ -25,12 +25,14 @@ async def _seed(
role="user",
interval=60.0,
path="https://example.com/foo",
+ source_type=None,
):
return await wm.create_task(
path=path,
account_id=account,
user_id=user,
original_role=role,
+ source_type=source_type,
to_uri=to_uri,
watch_interval=interval,
)
@@ -45,7 +47,7 @@ async def test_list_empty(client: httpx.AsyncClient):
async def test_full_lifecycle(client: httpx.AsyncClient, watch_manager, monkeypatch):
- task = await _seed(watch_manager)
+ task = await _seed(watch_manager, source_type="url")
# List
resp = await client.get("/api/v1/watches")
@@ -53,6 +55,7 @@ async def test_full_lifecycle(client: httpx.AsyncClient, watch_manager, monkeypa
assert body["status"] == "ok"
assert body["result"]["total"] == 1
assert body["result"]["tasks"][0]["task_id"] == task.task_id
+ assert body["result"]["tasks"][0]["source_type"] == "url"
assert body["result"]["tasks"][0]["to_uri"] == task.to_uri
# Get by ID
diff --git a/tests/server/test_openviking_assets.py b/tests/server/test_openviking_assets.py
index fe85c8d60..9e489f7ec 100644
--- a/tests/server/test_openviking_assets.py
+++ b/tests/server/test_openviking_assets.py
@@ -4,7 +4,9 @@
import asyncio
import hashlib
-from unittest.mock import AsyncMock, Mock
+import os
+import signal
+from unittest.mock import AsyncMock, Mock, call
import httpx
import pytest
@@ -17,6 +19,7 @@ from openviking.server.openviking_assets import (
resolve_openviking_assets,
)
from openviking_cli.exceptions import (
+ DeadlineExceededError,
InvalidArgumentError,
NotFoundError,
PermissionDeniedError,
@@ -345,16 +348,80 @@ async def test_git_preflight_rejects_unsafe_token_urls_before_spawn(monkeypatch,
exec_mock.assert_not_awaited()
-async def test_git_preflight_cancellation_kills_and_reaps_authenticated_process(monkeypatch):
- process = Mock()
- process.communicate = AsyncMock(side_effect=asyncio.CancelledError)
+async def test_git_preflight_rejects_http_userinfo_before_spawn(monkeypatch):
+ exec_mock = AsyncMock(return_value=_GitProcess(0))
+ monkeypatch.setattr("asyncio.create_subprocess_exec", exec_mock)
+
+ with pytest.raises(InvalidArgumentError, match="cannot contain userinfo"):
+ await preflight_git_repository(
+ asset_name="private",
+ repo_url="https://user:[email protected]/org/private.git",
+ )
+
+ exec_mock.assert_not_awaited()
+
+
[email protected](os.name != "posix", reason="process groups are POSIX-only")
+async def test_git_preflight_timeout_kills_process_group_and_bounds_reap(monkeypatch):
+ process = Mock(pid=43210)
+ process.communicate = AsyncMock(return_value=(b"", b""))
+ process.kill = Mock()
+ captured = {}
+ wait_timeouts = []
+ killpg = Mock()
+
+ async def fake_exec(*_args, **kwargs):
+ captured.update(kwargs)
+ return process
+
+ async def fake_wait_for(awaitable, timeout):
+ wait_timeouts.append(timeout)
+ if len(wait_timeouts) == 1:
+ awaitable.close()
+ raise asyncio.TimeoutError
+ return await awaitable
+
+ monkeypatch.setattr("asyncio.create_subprocess_exec", fake_exec)
+ monkeypatch.setattr("asyncio.wait_for", fake_wait_for)
+ monkeypatch.setattr(os, "killpg", killpg)
+
+ with pytest.raises(DeadlineExceededError):
+ await preflight_git_repository(
+ asset_name="private",
+ repo_url="https://github.com/org/private.git",
+ token="secret-token",
+ timeout=7.0,
+ )
+
+ assert (
+ captured.get("start_new_session"),
+ killpg.call_args_list,
+ wait_timeouts,
+ ) == (True, [call(process.pid, signal.SIGKILL)], [7.0, 1.0])
+ process.kill.assert_not_called()
+
+
[email protected](os.name != "posix", reason="process groups are POSIX-only")
+async def test_git_preflight_cancellation_kills_process_group_and_bounds_reap(monkeypatch):
+ process = Mock(pid=43210)
+ process.communicate = AsyncMock(side_effect=[asyncio.CancelledError, (b"", b"")])
process.kill = Mock()
process.wait = AsyncMock()
+ captured = {}
+ wait_timeouts = []
+ killpg = Mock()
- async def fake_exec(*_args, **_kwargs):
+ async def fake_exec(*_args, **kwargs):
+ captured.update(kwargs)
return process
+ async def fake_wait_for(awaitable, timeout):
+ wait_timeouts.append(timeout)
+ return await awaitable
+
monkeypatch.setattr("asyncio.create_subprocess_exec", fake_exec)
+ monkeypatch.setattr("asyncio.wait_for", fake_wait_for)
+ monkeypatch.setattr(os, "killpg", killpg)
with pytest.raises(asyncio.CancelledError):
await preflight_git_repository(
@@ -363,8 +430,12 @@ async def test_git_preflight_cancellation_kills_and_reaps_authenticated_process(
token="secret-token",
)
- process.kill.assert_called_once_with()
- process.wait.assert_awaited_once_with()
+ assert (
+ captured.get("start_new_session"),
+ killpg.call_args_list,
+ wait_timeouts,
+ ) == (True, [call(process.pid, signal.SIGKILL)], [15.0, 1.0])
+ process.kill.assert_not_called()
async def test_git_preflight_preserves_ssh_username(monkeypatch):
diff --git a/tests/service/test_resource_service_connector.py b/tests/service/test_resource_service_connector.py
index 36ad1fd99..021ef6062 100644
--- a/tests/service/test_resource_service_connector.py
+++ b/tests/service/test_resource_service_connector.py
@@ -248,6 +248,7 @@ async def test_connector_watch_stores_only_encrypted_replay_state(
role=str(Role.USER),
)
assert task is not None
+ assert task.source_type == "tos"
assert task.auth_state["provider"] == "connector_encrypted"
assert "secret" not in json.dumps(task.auth_state)
assert "auth_state" not in task.to_dict()
@@ -506,6 +507,41 @@ async def test_connector_watch_allows_plaintext_private_state_without_encryption
) == ("secret", "tos", {})
+async def test_declared_connector_watch_returns_original_add_type(
+ connector_config,
+ ctx,
+ service,
+):
+ connector_config.allowed_add_types = ["feishu_project"]
+ watch_manager = WatchManager()
+ service._watch_scheduler = SimpleNamespace(watch_manager=watch_manager)
+ submitted = {}
+
+ async def submit(**kwargs):
+ submitted.update(kwargs)
+ return {"status": "accepted"}
+
+ service._connector.submit = AsyncMock(side_effect=submit)
+ await service.add_resource(
+ path="project-ptat4n",
+ ctx=ctx,
+ add_type="feishu_project",
+ to="viking://resources/Project/ptat4n",
+ watch_interval=5,
+ )
+ await submitted["on_success"]()
+
+ task = await watch_manager.get_task_by_uri(
+ "viking://resources/Project/ptat4n",
+ account_id="acct",
+ user_id="alice",
+ role=str(Role.USER),
+ )
+ assert task is not None
+ assert task.source_type == "feishu_project"
+
+
@pytest.mark.asyncio
async def test_connector_watch_scheduler_restores_request_credentials(
connector_config,
diff --git a/tests/service/test_resource_service_git_preflight.py b/tests/service/test_resource_service_git_preflight.py
new file mode 100644
index 000000000..5fd8b8b4a
--- /dev/null
+++ b/tests/service/test_resource_service_git_preflight.py
@@ -0,0 +1,81 @@
+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
+# SPDX-License-Identifier: AGPL-3.0
+"""Regression tests for Git preflight process cleanup."""
+
+import asyncio
+import os
+import signal
+from unittest.mock import AsyncMock, Mock, call
+
+import pytest
+
+from openviking.service.resource_service import ResourceService
+from openviking_cli.exceptions import InvalidArgumentError
+
+pytestmark = pytest.mark.skipif(os.name != "posix", reason="process groups are POSIX-only")
+
+
+async def test_git_preflight_timeout_kills_process_group_and_bounds_reap(monkeypatch):
+ process = Mock(pid=43210)
+ process.communicate = AsyncMock(return_value=(b"", b""))
+ process.kill = Mock()
+ captured = {}
+ wait_timeouts = []
+ killpg = Mock()
+
+ async def fake_exec(*_args, **kwargs):
+ captured.update(kwargs)
+ return process
+
+ async def fake_wait_for(awaitable, timeout):
+ wait_timeouts.append(timeout)
+ if len(wait_timeouts) == 1:
+ awaitable.close()
+ raise asyncio.TimeoutError
+ return await awaitable
+
+ monkeypatch.setattr(asyncio, "create_subprocess_exec", fake_exec)
+ monkeypatch.setattr(asyncio, "wait_for", fake_wait_for)
+ monkeypatch.setattr(os, "killpg", killpg)
+
+ with pytest.raises(InvalidArgumentError, match="preflight timed out"):
+ await ResourceService()._preflight_git_source("https://github.com/org/private")
+
+ assert (
+ captured.get("start_new_session"),
+ killpg.call_args_list,
+ wait_timeouts,
+ ) == (True, [call(process.pid, signal.SIGKILL)], [10.0, 1.0])
+ process.kill.assert_not_called()
+
+
+async def test_git_preflight_cancellation_kills_process_group_and_bounds_reap(monkeypatch):
+ process = Mock(pid=43210)
+ process.communicate = AsyncMock(side_effect=[asyncio.CancelledError, (b"", b"")])
+ process.wait = AsyncMock()
+ process.kill = Mock()
+ captured = {}
+ wait_timeouts = []
+ killpg = Mock()
+
+ async def fake_exec(*_args, **kwargs):
+ captured.update(kwargs)
+ return process
+
+ async def fake_wait_for(awaitable, timeout):
+ wait_timeouts.append(timeout)
+ return await awaitable
+
+ monkeypatch.setattr(asyncio, "create_subprocess_exec", fake_exec)
+ monkeypatch.setattr(asyncio, "wait_for", fake_wait_for)
+ monkeypatch.setattr(os, "killpg", killpg)
+
+ with pytest.raises(asyncio.CancelledError):
+ await ResourceService()._preflight_git_source("https://github.com/org/private")
+
+ assert (
+ captured.get("start_new_session"),
+ killpg.call_args_list,
+ wait_timeouts,
+ ) == (True, [call(process.pid, signal.SIGKILL)], [10.0, 1.0])
+ process.kill.assert_not_called()
diff --git a/tests/service/test_resource_service_watch.py b/tests/service/test_resource_service_watch.py
index 248eb5670..d8f51ac02 100644
--- a/tests/service/test_resource_service_watch.py
+++ b/tests/service/test_resource_service_watch.py
@@ -9,6 +9,7 @@ from unittest.mock import AsyncMock, MagicMock
import pytest
import pytest_asyncio
+from openviking.resource.feishu_watch_auth import FeishuAppCredentials
from openviking.resource.watch_manager import WatchManager
from openviking.server.identity import RequestContext, Role
from openviking.service import resource_service as resource_service_module
@@ -171,6 +172,7 @@ class TestWatchTaskCreation:
task = await get_task_by_uri(resource_service, to_uri, request_context)
assert task is not None
assert task.path == "/test/path"
+ assert task.source_type == "local"
assert task.to_uri == to_uri
assert task.reason == "Test monitoring"
assert task.instruction == "Monitor for changes"
@@ -457,7 +459,12 @@ class TestAddResourceArgs:
monkeypatch.setattr(
resource_service_module,
"load_feishu_app_credentials",
- lambda: object(),
+ lambda app_id=None, app_secret=None: FeishuAppCredentials(
+ app_id=app_id or "configured-app",
+ app_secret=app_secret or "configured-secret",
+ domain="https://open.feishu.cn",
+ request_timeout=30,
+ ),
)
disable_task_tracker(monkeypatch)
to_uri = "viking://resources/feishu_user_watch"
@@ -480,6 +487,8 @@ class TestAddResourceArgs:
args={
"feishu_access_token": " u-test ",
"feishu_refresh_token": " r-test ",
+ "feishu_app_id": " cli-test ",
+ "feishu_app_secret": " secret-test ",
},
)
@@ -492,7 +501,10 @@ class TestAddResourceArgs:
"access_token": "u-test",
"refresh_token": "r-test",
"expires_at": None,
+ "app_id": "cli-test",
+ "app_secret": "secret-test",
}
+ assert "secret-test" not in str(message.to_dict())
await resource_service.execute_add_resource_job(
message,
ctx=request_context,
@@ -504,18 +516,41 @@ class TestAddResourceArgs:
processor = resource_service._resource_processor
assert processor.calls[-1]["feishu_access_token"] == "u-test"
assert "feishu_refresh_token" not in processor.calls[-1]
+ assert "feishu_app_id" not in processor.calls[-1]
+ assert "feishu_app_secret" not in processor.calls[-1]
task = await get_task_by_uri(resource_service, to_uri, request_context)
assert task is not None
+ assert task.source_type == "feishu"
assert task.processor_kwargs == {}
assert task.auth_state == {
"provider": "feishu",
"access_token": "u-test",
"refresh_token": "r-test",
"expires_at": None,
+ "app_id": "cli-test",
+ "app_secret": "secret-test",
}
assert "auth_state" not in task.to_dict()
+ @pytest.mark.asyncio
+ async def test_feishu_user_token_watch_rejects_partial_app_credentials(
+ self,
+ resource_service: ResourceService,
+ request_context: RequestContext,
+ ):
+ with pytest.raises(InvalidArgumentError, match="must be non-empty strings"):
+ await resource_service.add_resource(
+ path="https://example.feishu.cn/docx/doc_token",
+ ctx=request_context,
+ watch_interval=30,
+ args={
+ "feishu_access_token": "u-test",
+ "feishu_refresh_token": "r-test",
+ "feishu_app_id": "cli-test",
+ },
+ )
+
@pytest.mark.asyncio
async def test_git_token_watch_stores_private_auth_state(
self,
@@ -576,6 +611,7 @@ class TestAddResourceArgs:
task = await get_task_by_uri(resource_service, to_uri, request_context)
assert task is not None
+ assert task.source_type == "git"
assert task.processor_kwargs == {
"branch": "main",
"source_name": "private",
diff --git a/tests/service/test_watch_recovery.py b/tests/service/test_watch_recovery.py
index c183e72b0..53e81b69a 100644
--- a/tests/service/test_watch_recovery.py
+++ b/tests/service/test_watch_recovery.py
@@ -12,7 +12,7 @@ from unittest.mock import AsyncMock
import pytest
import pytest_asyncio
-from openviking.resource.feishu_watch_auth import FeishuRefreshedToken
+from openviking.resource.feishu_watch_auth import FeishuOAuthClient, FeishuRefreshedToken
from openviking.resource.git_watch_auth import create_git_http_auth_state
from openviking.resource.watch_manager import WatchManager
from openviking.resource.watch_scheduler import WatchScheduler
@@ -425,7 +425,10 @@ class TestResourceExistenceCheck:
@pytest.mark.asyncio
async def test_feishu_user_token_watch_refreshes_before_execution(
- self, temp_storage: Path, request_context: RequestContext
+ self,
+ monkeypatch: pytest.MonkeyPatch,
+ temp_storage: Path,
+ request_context: RequestContext,
):
resource_processor = MockResourceProcessor()
resource_service = ResourceService(
@@ -440,7 +443,19 @@ class TestResourceExistenceCheck:
)
scheduler = WatchScheduler(resource_service=resource_service, viking_fs=None)
await scheduler.start()
- scheduler._feishu_oauth_client = FakeFeishuOAuthClient()
+ feishu_client = FakeFeishuOAuthClient()
+
+ def from_auth_state(_cls, auth_state):
+ assert auth_state["app_id"] == "cli-test"
+ assert auth_state["app_secret"] == "secret-test"
+ return feishu_client
+
+ monkeypatch.setattr(
+ FeishuOAuthClient,
+ "from_auth_state",
+ classmethod(from_auth_state),
+ raising=False,
+ )
watch_manager = scheduler.watch_manager
task = await watch_manager.create_task(
@@ -452,12 +467,14 @@ class TestResourceExistenceCheck:
"access_token": "u-old",
"refresh_token": "r-old",
"expires_at": None,
+ "app_id": "cli-test",
+ "app_secret": "secret-test",
},
)
await scheduler._execute_task(task)
- assert scheduler._feishu_oauth_client.calls == ["r-old"]
+ assert feishu_client.calls == ["r-old"]
assert resource_service.refresh_resource.await_args.kwargs["feishu_access_token"] == "u-new"
updated_task = await watch_manager.get_task(task.task_id)
diff --git a/tests/test_session_task_tracking.py b/tests/test_session_task_tracking.py
index 25f5e8093..c43a5fdf2 100644
--- a/tests/test_session_task_tracking.py
+++ b/tests/test_session_task_tracking.py
@@ -265,6 +265,34 @@ async def test_list_tasks(api_client):
assert tasks[0]["task_type"] == "session_commit"
+async def test_list_tasks_hides_internal_tasks_by_default(api_client):
+ client, _ = api_client
+ tracker = get_task_tracker()
+ visible = await tracker.create(
+ "add_resource",
+ account_id="default",
+ user_id="default",
+ )
+ internal = await tracker.create(
+ "add_resource",
+ account_id="default",
+ user_id="default",
+ meta={"internal": True},
+ )
+
+ resp = await client.get("/api/v1/tasks", params={"task_type": "add_resource"})
+ task_ids = {task["task_id"] for task in resp.json()["result"]}
+ assert visible.task_id in task_ids
+ assert internal.task_id not in task_ids
+
+ resp = await client.get(
+ "/api/v1/tasks",
+ params={"task_type": "add_resource", "include_internal": True},
+ )
+ assert internal.task_id in {task["task_id"] for task in resp.json()["result"]}
+ assert (await client.get(f"/api/v1/tasks/{internal.task_id}")).status_code == 200
+
+
async def test_list_tasks_filter_status(api_client):
client, service = api_client
diff --git a/tests/test_task_tracker.py b/tests/test_task_tracker.py
index aa880568e..6deed5469 100644
--- a/tests/test_task_tracker.py
+++ b/tests/test_task_tracker.py
@@ -291,6 +291,17 @@ async def test_list_limit(tracker: TaskTracker):
assert len(tasks) == 3
+async def test_list_can_hide_internal_tasks_before_limit(tracker: TaskTracker):
+ visible = await tracker.create("add_resource", meta={}, **_owner_kwargs())
+ internal = await tracker.create("add_resource", meta={"internal": True}, **_owner_kwargs())
+
+ assert [task.task_id for task in await tracker.list_tasks(limit=1)] == [internal.task_id]
+ assert [
+ task.task_id
+ for task in await tracker.list_tasks(limit=1, include_internal=False)
+ ] == [visible.task_id]
+
+
async def test_list_order_most_recent_first(tracker: TaskTracker):
await tracker.create("session_commit", resource_id="first", **_owner_kwargs())
await tracker.create("session_commit", resource_id="second", **_owner_kwargs())
diff --git a/tests/unit/test_web_importer.py b/tests/unit/test_web_importer.py
index 4fb20579b..4152adffb 100644
--- a/tests/unit/test_web_importer.py
+++ b/tests/unit/test_web_importer.py
@@ -4,11 +4,11 @@ from types import SimpleNamespace
import pytest
from openviking.parse.accessors.web_importer import (
- WebImportOptions,
WebImporter,
+ WebImportOptions,
parse_web_import_options,
)
-from openviking_cli.exceptions import InvalidArgumentError
+from openviking_cli.exceptions import FailedPreconditionError, InvalidArgumentError
class TestParseWebImportOptions:
@@ -244,12 +244,13 @@ class TestWebImporter:
monkeypatch.setattr("openviking.parse.accessors.web_importer.ScrapyWebCrawler", FakeCrawler)
- with pytest.raises(RuntimeError) as exc_info:
+ with pytest.raises(FailedPreconditionError) as exc_info:
await WebImporter().import_to_directory(
root_url="https://mp.weixin.qq.com/s/abc",
options=WebImportOptions(),
)
message = str(exc_info.value)
+ assert exc_info.value.code == "FAILED_PRECONDITION"
assert "Forbidden by robots.txt" not in message
assert "compliance" in message
assert "local file" in message