Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
fix(resources): support paused watches for native Git imports (#5066)
zihenglifd719cbb7c97
diff --git a/docs/en/api/02-resources.md b/docs/en/api/02-resources.mdindex 33c6bed96..88a175e66 100644--- a/docs/en/api/02-resources.md+++ b/docs/en/api/02-resources.md@@ -185,7 +185,7 @@ This endpoint is the core entry point for resource management. It supports vario| preserve_structure | bool | No | None | Whether to preserve directory structure || args | object | No | `{}` | Parser-specific import options forwarded to the source parser/accessor. Native HTTPS Git imports and watches accept HTTP Basic credentials over TLS as `args.auth_config={"username":"oauth2","token":"..."}`; `username` defaults to `oauth2`. Git `branch` or `commit` remains at the top level of `args`. To import a private TOS object through its HTTP(S) URL, pass exactly one non-empty string: `args.tos_signature` (sent as `X-Tos-Signature`) or `args.tos_access` (sent as `X-Tos-Access`). TOS credentials are used only for the current HEAD/GET fetch, which is staged as a snapshot; they are not persisted to resource metadata or queue jobs. `args.parse_mode` accepts `default` (existing splitting behavior) or `no_split` (parse and convert each source document to one Markdown body). E.g. `args.site=true/false` forces/opts out of whole-site (sitemap/RSS) ingestion, `args.max_pages` etc. override the `webfeed` config; the recursive web crawler accepts `args.depth`, `args.max_pages`, `args.include_paths`, `args.exclude_paths`, `args.allow_external_links`, `args.skip_download_links`; Feishu user-token imports pass `args.feishu_access_token`. Core `add_resource` fields such as `path`, `to`, `watch_interval`, `include`, and `exclude` are not allowed inside `args` || watch_interval | float | No | 0 | Scheduled update interval (minutes). >0 creates a new Watch for a re-readable source, subject to target ownership rules; uploaded `temp_file_id` snapshots cannot be watched. <=0 creates no Watch: native imports with explicit `to` pause a single accessible task (409 if ambiguous), while Connector imports leave Watches untouched. Explicit `to` wins, otherwise the Watch binds to the imported `root_uri`. |-| is_active | bool | No | True | Initial Watch scheduling state. `false` requires `watch_interval > 0` and either `to` or `parent`. `parent` is supported for native Feishu URL imports; Connector imports still require an exact `to`. The initial import still runs once and the Watch remains paused afterward |+| is_active | bool | No | True | Initial Watch scheduling state. `false` requires `watch_interval > 0` and either `to` or `parent`. `parent` is supported for native Feishu URL and Git imports; Connector imports still require an exact `to`. The initial import still runs once and the Watch remains paused afterward || processing_mode | string | No | `semantic_and_vectors` | Post-ingest processing mode. `semantic_and_vectors` is the normal flow: generate semantic artifacts (`.abstract.md`, `.overview.md`) and vectors. `vectors_only` skips semantic understanding/VLM summarization and only vectorizes current resource files || telemetry | TelemetryRequest | No | False | Whether to return telemetry data |@@ -207,7 +207,7 @@ This endpoint is the core entry point for resource management. It supports vario- `processing_mode=vectors_only` does not call the VLM semantic-understanding stage and does not generate or refresh `.abstract.md` / `.overview.md`. For existing targets, it preserves existing semantic artifacts and existing semantic vectors. It still updates the resource tree, vectorizes current non-hidden files when `build_index=true`, and removes detail vectors for files deleted during refresh.- `processing_mode` belongs to `add_resource`. The admin `reindex` API/CLI continues to use `mode` (`vectors_only`, `semantic_and_vectors`, `prune_orphans`) for maintenance operations on already-ingested data.- When `watch_interval > 0`, the watch task binds to `to` if provided; otherwise it binds to the `root_uri` returned by this import. If no stable `root_uri` is available, the request fails and asks for an explicit `to`.-- For Connector imports, `is_active=false` creates the paused Watch before submission. Native Feishu imports carry `is_active` through the resource queue and create the Watch after resolving the imported resource URI. In both cases, the initial import still runs once and periodic scheduling remains disabled.+- For Connector imports, `is_active=false` creates the paused Watch before submission. Native Feishu and Git imports carry `is_active` through the resource queue and create the Watch after resolving the imported resource URI. In both cases, the initial import still runs once and periodic scheduling remains disabled.- Feishu/Lark app-token imports do not pass `args.feishu_access_token`. OpenViking keeps the existing app credential flow and the SDK obtains an app/tenant token from `app_id` and `app_secret`. This mode supports both one-time imports and `watch_interval > 0`.- Feishu/Lark one-time user-token imports pass `args={"feishu_access_token": "u-..."}` with `watch_interval <= 0`. OpenViking uses that user token only for the current import and does not store it.- Feishu/Lark user-token watches pass `args={"feishu_access_token": "u-...", "feishu_refresh_token": "r-..."}` with `watch_interval > 0`. They may also pass `feishu_app_id` and `feishu_app_secret` together; OpenViking stores the pair in the private watch task state and uses it to refresh that watch's user token.diff --git a/docs/zh/api/02-resources.md b/docs/zh/api/02-resources.mdindex eb409fccc..433d32f9d 100644--- a/docs/zh/api/02-resources.md+++ b/docs/zh/api/02-resources.md@@ -178,7 +178,7 @@ URL/文件 Parser TreeBuilder AGFS Summarizer/Vector| preserve_structure | bool | 否 | None | 是否保留目录结构 || args | object | 否 | `{}` | 传给特定 parser/accessor 的导入参数。原生 HTTPS Git 导入和 Watch 可通过 `args.auth_config={"username":"oauth2","token":"..."}` 在 TLS 上传递 HTTP Basic 凭据;`username` 默认为 `oauth2`。Git 的 `branch` 或 `commit` 仍放在 `args` 顶层。通过 HTTP(S) URL 导入私有 TOS 对象时,二选一传入非空字符串:`args.tos_signature`(映射为 `X-Tos-Signature`)或 `args.tos_access`(映射为 `X-Tos-Access`)。TOS 凭证只用于当前 HEAD/GET 抓取;资源会先保存为快照,凭证不会写入资源元数据或队列任务。`args.parse_mode` 支持 `default`(保持现有拆分行为)和 `no_split`(正常解析并将每个源文档正文保存为一个 Markdown 文件)。例如 `args.site=true/false` 强制/禁用整站(sitemap/RSS)导入,`args.max_pages` 等可覆盖 `webfeed` 配置;递归网页爬虫支持 `args.depth`、`args.max_pages`、`args.include_paths`、`args.exclude_paths`、`args.allow_external_links`、`args.skip_download_links`;飞书用户 token 导入传 `args.feishu_access_token`。`path`、`to`、`watch_interval`、`include`、`exclude` 等 `add_resource` 核心字段不能放入 `args` || watch_interval | float | 否 | 0 | 定时更新间隔(分钟)。>0 按目标占用规则为可重新读取的来源创建新 Watch;通过 `temp_file_id` 上传的一次性快照不能创建 Watch。≤0 不创建 Watch:原生导入显式指定 `to` 时暂停唯一可访问的任务(存在歧义时返回 409),Connector 导入不影响已有 Watch。显式 `to` 优先,否则绑定本次导入的 `root_uri`。 |-| is_active | bool | 否 | True | Watch 初始调度状态。设为 `false` 时要求 `watch_interval > 0`,并在 `to`、`parent` 中二选一。`parent` 仅支持原生飞书 URL 导入;Connector 仍要求精确的 `to`。首次导入仍执行一次,随后保持暂停 |+| is_active | bool | 否 | True | Watch 初始调度状态。设为 `false` 时要求 `watch_interval > 0`,并在 `to`、`parent` 中二选一。`parent` 支持原生飞书 URL 和 Git 导入;Connector 仍要求精确的 `to`。首次导入仍执行一次,随后保持暂停 || processing_mode | string | 否 | `semantic_and_vectors` | 入库后的处理模式。`semantic_and_vectors` 是默认流程:生成语义产物(`.abstract.md`、`.overview.md`)并生成向量。`vectors_only` 跳过语义理解/VLM 总结,只对当前资源文件生成向量 || tags | string[] | 否 | None | 导入时写入向量检索记录的显式检索标签,格式必须是 `k=v`,例如 `["team=search", "env=test"]`。搜索接口可用同名 `tags` 参数过滤召回 || tag_mode | string | 否 | `"replace"` | `tags` 的写入模式,可选 `replace` 或 `append`。导入新资源时会随本次生成的每条向量记录写入;不会在完成后额外调用 `set_tags`,响应也不返回 `tags_result` |@@ -202,7 +202,7 @@ URL/文件 Parser TreeBuilder AGFS Summarizer/Vector- `processing_mode=vectors_only` 不调用 VLM 语义理解阶段,也不会生成或刷新 `.abstract.md` / `.overview.md`。对已存在目标,它会保留旧的语义产物和旧的语义向量;仍会更新资源树,在 `build_index=true` 时向量化当前非隐藏文件,并清理由本次刷新删除的文件 detail 向量。- `processing_mode` 只属于 `add_resource`。管理员维护已有数据时,`reindex` API/CLI 仍使用 `mode`(`vectors_only`、`semantic_and_vectors`、`prune_orphans`)。- `watch_interval > 0` 时,如果指定了 `to`,监控任务绑定该目标;如果未指定 `to`,监控任务绑定本次导入返回的 `root_uri`。如果无法得到稳定 `root_uri`,请求会报错并要求显式传 `to`。-- Connector 导入设置 `is_active=false` 时会在提交前创建暂停状态的 Watch;原生飞书导入会通过资源队列透传 `is_active`,解析出最终资源 URI 后再创建 Watch。两种情况下首次导入均执行一次,周期调度保持关闭。+- Connector 导入设置 `is_active=false` 时会在提交前创建暂停状态的 Watch;原生飞书和 Git 导入会通过资源队列透传 `is_active`,解析出最终资源 URI 后再创建 Watch。两种情况下首次导入均执行一次,周期调度保持关闭。- 飞书/Lark 应用 token 导入不传 `args.feishu_access_token`。OpenViking 保持原有应用凭证流程,由 SDK 使用 `app_id` 和 `app_secret` 自动获取 app/tenant token。该模式支持一次性导入和 `watch_interval > 0`。- 飞书/Lark 一次性用户 token 导入通过 `args={"feishu_access_token": "u-..."}` 传入,且 `watch_interval <= 0`。OpenViking 只在本次导入使用该用户 token,不保存。- 飞书/Lark 用户 token watch 通过 `args={"feishu_access_token": "u-...", "feishu_refresh_token": "r-..."}` 传入,且 `watch_interval > 0`。还可同时传入 `feishu_app_id` 和 `feishu_app_secret`;OpenViking 会将其保存在 watch task 私有状态中,并用于刷新该 watch 的用户 token。diff --git a/openviking/server/routers/resources.py b/openviking/server/routers/resources.pyindex f79b11aa7..880484a4f 100644--- a/openviking/server/routers/resources.py+++ b/openviking/server/routers/resources.py@@ -74,7 +74,7 @@ class AddResourceRequest(BaseModel):create no Watch: native imports with explicit ``to`` pause a single accessibleWatch (409 if ambiguous); Connector imports leave Watches untouched.See the endpoint's Watch ownership rules.- is_active: Initial Watch state for Connector and native Feishu imports. When false,+ is_active: Initial Watch state for Connector, native Feishu, and native Git imports. When false,requires watch_interval > 0 and an explicit to or parent target and creates the Watchpaused; it stays paused until updated, regardless of the import result."""diff --git a/openviking/service/resource_service.py b/openviking/service/resource_service.pyindex 9055cd9bb..899ced6dc 100644--- a/openviking/service/resource_service.py+++ b/openviking/service/resource_service.py@@ -1551,7 +1551,7 @@ class ResourceService:or the imported ``root_uri``. Nonpositive values create no Watch:native imports with explicit ``to`` pause a single accessible Watch(ConflictError if ambiguous); Connector imports leave Watches untouched.- is_active: When false, the Connector or native Feishu Watch is created paused.+ is_active: When false, the Connector, native Feishu, or native Git Watch is created paused.Requires watch_interval > 0 and an explicit to or parent target.enforce_public_remote_targets: When True, reject non-public remote hosts andvalidate each outbound HTTP request URL during fetch.@@ -1787,9 +1787,11 @@ class ResourceService:from openviking.parse.accessors.feishu_accessor import FeishuAccessor- if is_active is False and not FeishuAccessor._is_feishu_url(path):+ if is_active is False and not (+ FeishuAccessor._is_feishu_url(path) or is_git_repo_url(path)+ ):raise InvalidArgumentError(- "is_active=false is only supported for Connector or native Feishu imports."+ "is_active=false is only supported for Connector, native Feishu, or native Git imports.")if enforce_public_remote_targets and is_remote_resource_source(path):path = require_remote_resource_source(path)diff --git a/tests/service/test_resource_service_watch.py b/tests/service/test_resource_service_watch.pyindex d4e3f58fb..c9b496666 100644--- a/tests/service/test_resource_service_watch.py+++ b/tests/service/test_resource_service_watch.py@@ -664,11 +664,21 @@ class TestAddResourceArgs:)@pytest.mark.asyncio+ @pytest.mark.parametrize(+ ("target", "is_active"),+ [+ ({"to": "viking://resources/git_private_watch"}, True),+ ({"to": "viking://resources/git_private_watch"}, False),+ ({"parent": "viking://resources"}, False),+ ],+ )async def test_git_token_watch_stores_private_auth_state(self,monkeypatch: pytest.MonkeyPatch,resource_service: ResourceService,request_context: RequestContext,+ target,+ is_active,):monkeypatch.setattr(resource_service_module, "is_git_repo_url", lambda _path: True)disable_task_tracker(monkeypatch)@@ -688,7 +698,8 @@ class TestAddResourceArgs:await resource_service.add_resource(path=repo_url,ctx=request_context,- to=to_uri,+ **target,+ is_active=is_active,watch_interval=30,args={"branch": "main",@@ -701,6 +712,7 @@ class TestAddResourceArgs:enqueue_call = resource_service._enqueue_add_resource_job.await_argsmessage = enqueue_call.args[0]+ assert AddResourceMsg.from_dict(message.to_dict()).is_active is is_activetask_auth = enqueue_call.kwargs["task_auth"]assert "git-secret" not in str(message.to_dict())assert task_auth == {@@ -725,6 +737,10 @@ class TestAddResourceArgs:task = await get_task_by_uri(resource_service, to_uri, request_context)assert task is not None+ assert len(processor.calls) == 1+ assert task.is_active is is_active+ assert (task.next_execution_time is not None) is is_active+ assert message.watch_task_id == task.task_idassert task.source_type == "git"assert task.processor_kwargs == {"branch": "main",