Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
feat: live-reload Agent Evolution and add file usage sink (#3573)
agent0ec2bb0ec528
diff --git a/docs/design/openviking-usage-count-record-implementation-plan.md b/docs/design/openviking-usage-count-record-implementation-plan.mdindex d0f96894d..00f24ad5d 100644--- a/docs/design/openviking-usage-count-record-implementation-plan.md+++ b/docs/design/openviking-usage-count-record-implementation-plan.md@@ -1,228 +1,100 @@-# HTTP Usage CountRecord Implementation Plan+# File Usage CountRecord Implementation Plan-> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.+**Goal:** 将 Usage Reporter 生成的 `UsageEvent` 转换为稳定的+`CountRecord` 日志协议,供部署侧日志采集系统读取并投递到下游。-**Goal:** 将内置 HTTP Usage Sink 上报的事件元素改为 `CountRecord`,同时保持内部 `UsageEvent`、自定义 Sink 和 outbox 可靠性语义不变。+**Architecture:** `MemoryUsageExtractor` 继续生成内部 `UsageEvent`。+`FileLogUsageSink` 在写入专用日志文件前执行单向转换,每行保存一个包含 Kafka+message key 和对应 `CountRecord` 的 JSON envelope。`unique_id` 作为稳定事件标识,+供下游在 best-effort 投递发生重复时去重。-**Architecture:** `MemoryUsageExtractor` 继续生成 `UsageEvent`。`HttpUsageSink` 在持久化批次前执行单向转换,HTTP 批次中的 `events` 改为 `CountRecord`;`uniqueId` 继续承担重放去重键,并用于计算稳定 `batch_id`。--**Tech Stack:** Python 3.10+、dataclasses、标准库 datetime/json、pytest、Ruff。+**Tech Stack:** Python 3.10+、dataclasses、标准库+`datetime` / `json` / `logging`、pytest、Ruff。---## 文件结构-- Modify: `openviking/usage_reporter/http_sink.py`+- `openviking/usage_reporter/file_log_sink.py`- 定义 `UsageEvent -> CountRecord` 私有转换。- - 调整批次 ID 从 `uniqueId` 计算。-- Modify: `tests/unit/usage_reporter/test_http_sink.py`+ - 构造资源归属 message key。+ - 将记录追加到按 UTC 小时滚动的专用日志文件。+- `tests/unit/usage_reporter/test_file_log_sink.py`- 固定 recall/inject 字段映射、毫秒时间戳、扩展字段和未知事件行为。-- Modify: `docs/design/openviking-usage-reporter-sink-design.md`- - 补充 `attributes -> extra.attributes` 的无损映射说明。--### Task 1: 固定 CountRecord 映射契约--**Files:**-- Test: `tests/unit/usage_reporter/test_http_sink.py`--- [ ] **Step 1: 扩展测试事件构造器**--让 `FakeUsageEvent` 支持 `event_type`、`occurred_at`、`resource_uri`、-`resource_type`、`task_id`、`evidence` 和 `attributes`,并继续通过-`to_dict()` 模拟真实 `UsageEvent`。--- [ ] **Step 2: 写 recall 映射失败测试**--构造 `memory.recalled` 事件并调用 `sink.write()`,读取 pending JSON,断言:--```python-assert payload["events"] == [- {- "CountName": "experience.recall.count",- "OpType": "add",- "amount": 1.0,- "timestamp": 1785124800000,- "uniqueId": "ue_recall",- "tags": {- "account_id": "2101858484",- "user_id": "user-1",- "resource_uri": "viking://user/user-1/memories/experiences/exchange.md",- "resource_type": "experience",- },- "extra": {- "session_id": "session-1",- "task_id": "task-1",- "archive_uri": "viking://user/user-1/sessions/session-1/history/archive_001",- "message_id": "msg-1",- "tool_call_id": "call-1",- "tool_name": "search_experience",- "attributes": {"rank": 1},- },+ - 验证多 worker 共享文件时的写入和滚动行为。+- `docs/design/openviking-usage-reporter-sink-design.md`+ - 定义 Usage Reporter、Sink 扩展点和文件日志协议。++## CountRecord 映射契约++`memory.recalled` 事件映射为:++```json+{+ "count_name": "experience.recall.count",+ "op_type": "add",+ "amount": 1.0,+ "timestamp": 1785124800000,+ "unique_id": "ue_recall",+ "tags": {+ "account_id": "2101858484",+ "user_id": "user-1",+ "resource_uri": "viking://user/user-1/memories/experiences/exchange.md",+ "resource_type": "experience"+ },+ "extra": {+ "session_id": "session-1",+ "task_id": "task-1",+ "archive_uri": "viking://user/user-1/sessions/session-1/history/archive_001",+ "message_id": "msg-1",+ "tool_call_id": "call-1",+ "tool_name": "search_experience",+ "attributes": {+ "rank": 1}-]-```--- [ ] **Step 3: 运行测试并确认 RED**--Run:--```bash-uv run pytest -q --no-cov tests/unit/usage_reporter/test_http_sink.py::TestHttpUsageSink::test_write_persists_recall_as_count_record-```--Expected: FAIL,当前 payload 仍包含 `event_type` 等 `UsageEvent` 字段。--- [ ] **Step 4: 写 inject、空字段和未知事件失败测试**--分别断言:--- `memory.injected -> experience.inject.count`-- `task_id=None`、空 `evidence`、空 `attributes` 不产生对应 `extra` 字段-- 未知 `event_type` 在 `sink.write()` 时抛出 `ValueError`--- [ ] **Step 5: 运行新增测试并确认 RED**--Run:--```bash-uv run pytest -q --no-cov tests/unit/usage_reporter/test_http_sink.py -k "count_record or unknown_event_type"-```--Expected: FAIL,CountRecord 转换尚不存在。--### Task 2: 实现 HTTP 边界转换--**Files:**-- Modify: `openviking/usage_reporter/http_sink.py`--- [ ] **Step 1: 增加事件类型映射和毫秒时间转换**--```python-_COUNT_NAMES = {- "memory.recalled": "experience.recall.count",- "memory.injected": "experience.inject.count",+ },+ "prefix": "ov-resource-id"}---def _timestamp_millis(value: str) -> int:- parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))- if parsed.tzinfo is None:- parsed = parsed.replace(tzinfo=timezone.utc)- return int(parsed.timestamp() * 1000)-```--- [ ] **Step 2: 增加 CountRecord 转换**--```python-def _to_count_record(event: UsageEvent) -> dict[str, Any]:- record = event.to_dict()- event_type = str(record.get("event_type") or "")- try:- count_name = _COUNT_NAMES[event_type]- except KeyError as exc:- raise ValueError(f"unsupported usage event type: {event_type}") from exc-- evidence = record.get("evidence")- evidence = evidence if isinstance(evidence, dict) else {}- attributes = record.get("attributes")- attributes = attributes if isinstance(attributes, dict) else {}- extra = {- "session_id": str(record.get("session_id") or ""),- **{key: value for key, value in evidence.items() if value not in (None, "")},- }- if record.get("task_id") not in (None, ""):- extra["task_id"] = record["task_id"]- if attributes:- extra["attributes"] = attributes-- return {- "CountName": count_name,- "OpType": "add",- "amount": 1.0,- "timestamp": _timestamp_millis(str(record["occurred_at"])),- "uniqueId": str(record["event_id"]),- "tags": {- "account_id": str(record.get("account_id") or ""),- "user_id": str(record.get("user_id") or ""),- "resource_uri": str(record.get("resource_uri") or ""),- "resource_type": str(record.get("resource_type") or ""),- },- "extra": extra,- }```-- [ ] **Step 3: 在持久化入口使用转换**--`_persist_events()` 先校验 `event_id`,再调用 `_to_count_record()`。移除旧的-逐事件 `prefix` 字段;资源归属继续由批次 `resource_id` 和-`V-Resource-Id` header 表达。--- [ ] **Step 4: 改用 uniqueId 生成 batch_id**--```python-unique_ids = "\n".join(str(event["uniqueId"]) for event in events)-batch_id = f"ub_{hashlib.sha256(unique_ids.encode('utf-8')).hexdigest()}"-```--- [ ] **Step 5: 运行新增测试并确认 GREEN**--Run:--```bash-uv run pytest -q --no-cov tests/unit/usage_reporter/test_http_sink.py -k "count_record or unknown_event_type"-```--Expected: PASS。--### Task 3: 回归可靠性与文档--**Files:**-- Modify: `docs/design/openviking-usage-reporter-sink-design.md`-- Test: `tests/unit/usage_reporter/test_http_sink.py`+映射规则:-- [ ] **Step 1: 补充 attributes 映射说明**+- `memory.recalled` 映射为+ `count_name=experience.recall.count`。+- `memory.injected` 映射为+ `count_name=experience.inject.count`。+- `op_type` 固定为 `add`,`amount` 固定为 `1.0`。+- `occurred_at` 转换为毫秒时间戳。+- `event_id` 写入 `unique_id`,为空时拒绝写入。+- 非空 `UsageEvent.attributes` 写入 `extra.attributes`。+- `resource_id_env` 指定的环境变量写入 `prefix`。+- 未知 `event_type` 拒绝写入,避免产生无法解释的计量记录。-在 CountRecord 字段映射中明确:非空 `UsageEvent.attributes` 写入-`extra.attributes`,避免事件类型扩展字段丢失。+## 文件日志协议-- [ ] **Step 2: 运行 HTTP Sink 全量测试**+日志行格式:-Run:--```bash-uv run pytest -q --no-cov --tb=short tests/unit/usage_reporter/test_http_sink.py+```text+{"key":"<resource_id>|<account_id>|<user_id>|<resource_uri>","value":<CountRecord JSON>}```-Expected: 全部 PASS,覆盖 outbox、重试、拆包、dead-letter 和 destination-隔离。+当 `resource_uri` 为空时,message key 的最后一段使用 `session_id`。+整行使用 JSON envelope,key 中的 `=`、空格或其他字符不会影响 key/value 拆分。+日志文件不复用 OpenViking stdout,按 UTC 小时滚动,并保留配置数量的历史+文件。多个 server worker 写入同一路径时,文件追加和滚动通过进程间锁串行化。-- [ ] **Step 3: 运行 Usage Reporter 回归**+文件落盘及后续采集均采用 best-effort 语义。下游必须按+`CountRecord.unique_id` 去重。-Run:+## 验证```bash-uv run pytest -q --no-cov --tb=short tests/unit/usage_reporter-```--Expected: 全部 PASS。--- [ ] **Step 4: 运行静态检查**--Run:--```bash-uv run ruff check openviking/usage_reporter/http_sink.py tests/unit/usage_reporter/test_http_sink.py-uv run ruff format --check openviking/usage_reporter/http_sink.py tests/unit/usage_reporter/test_http_sink.py+uv run pytest -q --no-cov tests/unit/usage_reporter+uv run ruff check \+ openviking/usage_reporter/file_log_sink.py \+ tests/unit/usage_reporter/test_file_log_sink.py+uv run ruff format --check \+ openviking/usage_reporter/file_log_sink.py \+ tests/unit/usage_reporter/test_file_log_sink.pygit diff --check```--Expected: 全部通过。--- [ ] **Step 5: 提交实现**--```bash-git add openviking/usage_reporter/http_sink.py \- tests/unit/usage_reporter/test_http_sink.py \- docs/design/openviking-usage-reporter-sink-design.md-git commit -m "feat(usage): emit CountRecord over HTTP"-```diff --git a/docs/design/openviking-usage-reporter-sink-design.md b/docs/design/openviking-usage-reporter-sink-design.mdindex 9b7e3fb16..c9ba42c35 100644--- a/docs/design/openviking-usage-reporter-sink-design.md+++ b/docs/design/openviking-usage-reporter-sink-design.md@@ -100,7 +100,7 @@ class UsageSink:...```-具体 Sink 可以作为外部扩展通过 `class_path` 动态加载。开源包同时提供不依赖第三方消息队列 SDK 的内置 HTTP Sink,用于需要本地持久化重试的部署。+具体 Sink 可以作为外部扩展通过 `class_path` 动态加载。开源包同时提供不依赖第三方消息队列 SDK 的内置文件日志 Sink,供日志采集系统读取。配置示例:@@ -130,53 +130,40 @@ def load_class(class_path: str):每个 Sink 的 `write()` 调用最多等待 5 秒。超时或异常只记录日志,不影响其他 Sink。Reporter 在应用生命周期内只创建一次,应用退出时调用 Sink 可选的 `close()` 方法。同步和异步 `close()` 均受同一超时限制;同步 hook 在独立 daemon 线程中执行,超时后不会阻塞事件循环、后续 Sink 清理或进程退出。-内置 HTTP Sink 在 `write()` 返回前把事件批次写入本地 outbox,再由后台线程发送。它只依赖 Python 标准库,不引入 Kafka 等厂商依赖。HTTP Sink 支持指数退避、`413` 拆包、不可恢复批次转入 dead-letter,以及有界磁盘容量。+内置文件日志 Sink 将事件立即追加到专用日志文件,不写默认 stdout,也不发起+HTTP 请求。日志文件使用 UTC 小时滚动,默认保留 168 个小时文件。部署侧应将+专用目录挂载到日志采集系统可见的宿主机路径。多个 server worker 写入同一路径+时,通过进程间文件锁串行化写入和滚动;Windows worker 写入后主动关闭文件句柄,+避免其他进程滚动重命名失败。-### 6.1 HTTP CountRecord 协议+### 6.1 文件日志 CountRecord 协议`UsageEvent` 继续作为 OpenViking 内部抽取结果和自定义 Sink 的稳定协议。内置-HTTP Sink 在写入 outbox 前,将 `UsageEvent` 转换为计量接收端使用的-`CountRecord`。这样计量平台协议不会侵入 Extractor,也不会改变自定义 Sink-的输入。+文件日志 Sink 将 `UsageEvent` 转换为计量接收端使用的 `CountRecord`。每个事件+写成一行包含 Kafka key/value 的 JSON envelope:-HTTP 批次外层继续保留 `schema_version`、`resource_id`、`batch_id`、-`created_at`、`attempt`、`next_retry_at` 和 `events`。`events` 数组中的元素-采用以下格式:--```json-{- "CountName": "experience.recall.count",- "OpType": "add",- "amount": 1.0,- "timestamp": 1785124800000,- "uniqueId": "ue_<sha256>",- "tags": {- "account_id": "new",- "user_id": "test",- "resource_uri": "viking://user/test/memories/experiences/example.md",- "resource_type": "experience"- },- "extra": {- "session_id": "session-id",- "task_id": "task-id",- "archive_uri": "viking://user/test/sessions/session-id/history/archive_001",- "message_id": "message-id",- "tool_call_id": "tool-call-id",- "tool_name": "search_experience"- }-}+```text+{"key":"ov-xxx|new|test|viking://user/test/memories/experiences/example.md","value":{"count_name":"experience.recall.count","op_type":"add","amount":1.0,"timestamp":1785124800000,"unique_id":"ue_<sha256>","tags":{"account_id":"new","user_id":"test","resource_uri":"viking://user/test/memories/experiences/example.md","resource_type":"experience"},"extra":{"session_id":"session-id","task_id":"task-id","archive_uri":"viking://user/test/sessions/session-id/history/archive_001","message_id":"message-id","tool_call_id":"tool-call-id","tool_name":"search_experience"},"prefix":"ov-xxx"}}```字段映射:-- `memory.recalled` 映射为 `CountName=experience.recall.count`。-- `memory.injected` 映射为 `CountName=experience.inject.count`。-- `OpType` 固定为 `add`。+- envelope 的 `key` 与原 Kafka message key 一致,格式为+ `resource_id|account_id|user_id|resource_uri`;`resource_uri` 为空时回退+ 到 `session_id`。+- envelope 的 `value` 是原 Kafka message value 的完整 JSON,不拆分内部字段。+- 整行使用 JSON envelope,key 中的分隔符不会影响 key/value 解析。+- JSON 中的 `prefix` 为 OpenViking 实例的 resource ID。+- `memory.recalled` 映射为 `count_name=experience.recall.count`。+- `memory.injected` 映射为 `count_name=experience.inject.count`。+- `op_type` 固定为 `add`。- `amount` 固定为 `1.0`。- `timestamp` 由 `occurred_at` 转换为 Unix 毫秒时间戳。-- `uniqueId` 使用稳定的 `event_id`。-- `tags` 保存 `account_id`、`user_id`、`resource_uri` 和 `resource_type`。-- `extra` 保存 `session_id`、可选的 `task_id` 以及 `evidence` 中的审计字段。+- JSON 中的 `unique_id` 使用稳定的 `event_id`。+- JSON 中的 `tags` 保存 `account_id`、`user_id`、`resource_uri` 和+ `resource_type`。+- JSON 中的 `extra` 保存 `session_id`、可选的 `task_id` 以及 `evidence`+ 中的审计字段。- 非空的 `UsageEvent.attributes` 原样写入 `extra.attributes`。- 空的可选字段不写入 `extra`。@@ -206,19 +193,19 @@ server:endpoint: https://usage.example.com/events```-内置 HTTP Sink:+内置文件日志 Sink:```yamlserver:usage_reporter:enabled: truesinks:- - type: http+ - type: file_logconfig:- endpoint: https://usage.example.com/events+ path: /var/log/openviking_usage/usage.logresource_id_env: OV_RESOURCE_ID- outbox_dir: /var/lib/openviking/.usage_outbox- max_outbox_bytes: 268435456+ rotation_interval_hours: 1+ backup_count: 168```## 8. 对 OpenViking 的侵入@@ -289,6 +276,6 @@ schema_version任务恢复或附加属性变化破坏幂等性。同一 session message 中同一 tool call对同一资源产生的事件,在 phase2 重放后仍得到相同 `event_id`。-内置 HTTP Sink 提供持久化 outbox、失败重试和发送确认。所有 `2xx` 响应视为确认,瞬时错误进入指数退避重试,`400` / `422` 进入 dead-letter。HTTP Sink 可能重复发送同一事件,消费端需按由内部 `event_id` 映射得到的 `CountRecord.uniqueId` 去重。--HTTP outbox 使用 `max_outbox_bytes` 限制磁盘占用。达到上限时先淘汰最旧的 dead-letter,再淘汰最旧的 pending;inflight 不被淘汰。因此即使启用 HTTP Sink,长期下游故障和容量压力下仍可能丢失事件,整体机制保持 best-effort,不承诺端到端 at-least-once。+内置文件日志 Sink 在 `write()` 返回前完成本地追加,但不负责 TLS 采集、Kafka+投递或下游确认。文件写入、TLS 采集和下游消费任一阶段都可能在故障时产生丢失+或重复,消费端需按 `unique_id` 去重,整体保持 best-effort 语义。diff --git a/docs/en/api/01-overview.md b/docs/en/api/01-overview.mdindex 56f0d850d..074bb50bd 100644--- a/docs/en/api/01-overview.md+++ b/docs/en/api/01-overview.md@@ -532,6 +532,7 @@ This catalog follows the routes actually mounted by the server. Each group headi| Method | Path | Description ||--------|------|-------------|+| GET | `/api/v1/admin/agent-evolution` | Get the live instance-wide Agent Evolution status || POST | `/api/v1/admin/accounts` | Create an account and its first administrator || GET | `/api/v1/admin/accounts` | List accounts || POST | `/api/v1/admin/migrate` | Migrate legacy identity data |diff --git a/docs/en/api/08-admin.md b/docs/en/api/08-admin.mdindex c81133415..02c46f085 100644--- a/docs/en/api/08-admin.md+++ b/docs/en/api/08-admin.md@@ -57,6 +57,40 @@ Configure `root_api_key` in `~/.openviking/ovcli.conf`:## API Reference+### get_agent_evolution_status++Return the live instance-wide Agent Evolution switch and the configured default+account ID. The endpoint is ROOT-only.++**HTTP API**++```+GET /api/v1/admin/agent-evolution+```++```bash+curl http://localhost:1933/api/v1/admin/agent-evolution \+ -H "X-API-Key: <root-key>"+```++**Response Example**++```json+{+ "status": "ok",+ "result": {+ "enabled": false,+ "account_id": "default"+ },+ "time": 0.1+}+```++`enabled` is read from the live `server.agent_evolution.enabled` value used by+session commits. `account_id` is the deployment's configured default account.++---+### create_account#### 1. API Implementation Overviewdiff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.mdindex 297273ed1..088415dc2 100644--- a/docs/en/guides/01-configuration.md+++ b/docs/en/guides/01-configuration.md@@ -1506,11 +1506,11 @@ When running OpenViking as an HTTP service, add a `server` section to `ov.conf`:When `root_api_key` is configured in `api_key` mode, the server enables multi-tenant authentication. Use the Admin API to create accounts and user keys. In `trusted` mode, ordinary requests do not require user registration first; each request is resolved as `USER` from the injected identity headers. However, skipping `root_api_key` in `trusted` mode is allowed only on localhost. Development mode only applies when `auth_mode = "api_key"` and `root_api_key` is not set.-`user_config_defaults` only provides per-user defaults for add targets. For add operations, explicit request targets still win: `add_resource.to` / `add_resource.parent` take precedence over user defaults, and `add_skill.target_uri` takes precedence over user defaults. `agent_evolution.enabled` is shared by the entire OpenViking instance and has no per-user override; restart the server after changing it.+`user_config_defaults` only provides per-user defaults for add targets. For add operations, explicit request targets still win: `add_resource.to` / `add_resource.parent` take precedence over user defaults, and `add_skill.target_uri` takes precedence over user defaults. `agent_evolution.enabled` is shared by the entire OpenViking instance and has no per-user override. Running HTTP server workers read the current value from the resolved `ov.conf` when a session commits, so a valid file update applies without restarting the server.### Usage Reporter-The optional Usage Reporter extracts memory usage events from committed session tool parts. The built-in HTTP sink persists batches to a local outbox before delivering them to a collector:+The optional Usage Reporter extracts memory usage events from committed session tool parts. The built-in file log sink writes each event as a `{"key": ..., "value": ...}` JSON envelope to a dedicated hourly rotating file:```json{@@ -1520,17 +1520,12 @@ The optional Usage Reporter extracts memory usage events from committed session"extractors": ["memory_usage"],"sinks": [{- "type": "http",+ "type": "file_log","config": {- "endpoint": "https://collector.example.com/openviking/usage",+ "path": "/var/log/openviking_usage/usage.log","resource_id_env": "OV_RESOURCE_ID",- "outbox_dir": "/var/lib/openviking/.usage_outbox",- "request_timeout_seconds": 10,- "inflight_lease_seconds": 60,- "retry_base_seconds": 1,- "retry_max_seconds": 300,- "max_batch_bytes": 1048576,- "max_outbox_bytes": 268435456+ "rotation_interval_hours": 1,+ "backup_count": 168}}]@@ -1539,11 +1534,14 @@ The optional Usage Reporter extracts memory usage events from committed session}```-Set the environment variable named by `resource_id_env` before starting the server. If `outbox_dir` is omitted, it defaults to `~/.openviking/data/.usage_outbox` for the operating-system user running OpenViking.+The built-in `file_log` sink replaces the earlier `http` sink. Deployments+using `"type": "http"` must migrate to `file_log` and collect the dedicated+log files, or configure a `custom` sink that implements their delivery+contract.-HTTP delivery uses a durable local retry queue and may deliver the same event more than once. Collectors should deduplicate `CountRecord` entries by `uniqueId`. Every `2xx` response acknowledges a batch; transient failures are retried with exponential backoff. `400` and `422` responses move a batch to `dead_letter`, while `413` splits multi-event batches. The outbox is bounded by `max_outbox_bytes`; when the limit is reached, the oldest dead-letter batches are removed first, followed by the oldest pending batches. In-flight batches are not evicted. Because capacity pressure can discard pending data, reporting remains best-effort and does not provide an end-to-end at-least-once guarantee.+Set the environment variable named by `resource_id_env` before starting the server. The sink creates the parent directory, appends events immediately, rotates the active file every UTC hour, and retains `backup_count` rotated files. It does not write to the default OpenViking stdout log.-`inflight_lease_seconds` must be greater than `request_timeout_seconds`. The lease is refreshed whenever a worker claims a batch, preventing another worker from recovering an active delivery.+Each line is a JSON envelope with `key` and `value` fields. `key` matches the original Kafka message key and has the form `resource_id|account_id|user_id|resource_uri`; it falls back to `session_id` when `resource_uri` is empty. `value` is the complete object used as the original Kafka message value, containing `count_name`, `op_type`, `amount`, `timestamp`, `unique_id`, `tags`, `extra`, and `prefix`. The JSON envelope preserves delimiters that appear inside the key. File collection and downstream delivery remain best-effort, so consumers should deduplicate by `value.unique_id`.Supported add target URIs:diff --git a/docs/zh/api/01-overview.md b/docs/zh/api/01-overview.mdindex 301f5636f..862bc3d9b 100644--- a/docs/zh/api/01-overview.md+++ b/docs/zh/api/01-overview.md@@ -527,6 +527,7 @@ JSON 输出 - 错误:| 方法 | 路径 | 说明 ||------|------|------|+| GET | `/api/v1/admin/agent-evolution` | 获取实例级 Agent 进化开关的实时状态 || POST | `/api/v1/admin/accounts` | 创建账号及首个管理员 || GET | `/api/v1/admin/accounts` | 列出账号 || POST | `/api/v1/admin/migrate` | 迁移旧版身份数据 |diff --git a/docs/zh/api/08-admin.md b/docs/zh/api/08-admin.mdindex 947fee47f..e7c3f189f 100644--- a/docs/zh/api/08-admin.md+++ b/docs/zh/api/08-admin.md@@ -57,6 +57,41 @@ Admin API 用于多租户环境下的账户和用户管理。包括工作区(a## API 参考+### get_agent_evolution_status++返回实例级 Agent 进化开关的实时状态和已配置的默认 account ID。该接口仅限+ROOT 调用。++**HTTP API**++```+GET /api/v1/admin/agent-evolution+```++```bash+curl http://localhost:1933/api/v1/admin/agent-evolution \+ -H "X-API-Key: <root-key>"+```++**响应示例**++```json+{+ "status": "ok",+ "result": {+ "enabled": false,+ "account_id": "default"+ },+ "time": 0.1+}+```++`enabled` 来自 session commit 实际使用的+`server.agent_evolution.enabled` 实时值;`account_id` 是部署配置的默认+account。++---+### create_account#### 1. API 实现介绍diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.mdindex ec1d016d1..3015d3e7a 100644--- a/docs/zh/guides/01-configuration.md+++ b/docs/zh/guides/01-configuration.md@@ -1471,11 +1471,11 @@ openviking add-resource ./docs --exclude "*.tmp"在 `api_key` 模式下配置 `root_api_key` 后,服务端启用正式多租户认证,并通过 Admin API 创建工作区和用户 key。在 `trusted` 模式下,普通请求不需要先注册 user key;每个请求都会根据注入的身份头解析成 `USER`。只有在 `auth_mode = "api_key"` 且未配置 `root_api_key` 时,服务端才会进入开发模式。-`user_config_defaults` 仅用于添加目标的用户级默认配置。添加操作中,显式请求目标仍然优先:`add_resource.to` / `add_resource.parent` 优先于用户默认值,`add_skill.target_uri` 优先于用户默认值。`agent_evolution.enabled` 是当前 OpenViking 实例的统一开关,不支持用户级覆盖;修改后需重启服务加载配置。+`user_config_defaults` 仅用于添加目标的用户级默认配置。添加操作中,显式请求目标仍然优先:`add_resource.to` / `add_resource.parent` 优先于用户默认值,`add_skill.target_uri` 优先于用户默认值。`agent_evolution.enabled` 是当前 OpenViking 实例的统一开关,不支持用户级覆盖。HTTP Server 的 worker 会在 session commit 时从启动阶段解析出的 `ov.conf` 路径读取当前值,因此合法的文件更新无需重启服务即可生效。### Usage Reporter-可选的 Usage Reporter 从已 commit session 的 tool parts 中抽取记忆使用事件。内置 HTTP Sink 会先把事件批次持久化到本地 outbox,再发送给采集服务:+可选的 Usage Reporter 从已 commit session 的 tool parts 中抽取记忆使用事件。内置文件日志 Sink 将每个事件写成一行 `{"key": ..., "value": ...}` JSON envelope,并按小时滚动专用日志文件:```json{@@ -1485,17 +1485,12 @@ openviking add-resource ./docs --exclude "*.tmp""extractors": ["memory_usage"],"sinks": [{- "type": "http",+ "type": "file_log","config": {- "endpoint": "https://collector.example.com/openviking/usage",+ "path": "/var/log/openviking_usage/usage.log","resource_id_env": "OV_RESOURCE_ID",- "outbox_dir": "/var/lib/openviking/.usage_outbox",- "request_timeout_seconds": 10,- "inflight_lease_seconds": 60,- "retry_base_seconds": 1,- "retry_max_seconds": 300,- "max_batch_bytes": 1048576,- "max_outbox_bytes": 268435456+ "rotation_interval_hours": 1,+ "backup_count": 168}}]@@ -1504,11 +1499,13 @@ openviking add-resource ./docs --exclude "*.tmp"}```-启动服务前,需要设置 `resource_id_env` 指定的环境变量。未配置 `outbox_dir` 时,默认使用 OpenViking 运行用户的 `~/.openviking/data/.usage_outbox`。+内置 `file_log` Sink 替代了此前的 `http` Sink。原来使用+`"type": "http"` 的部署需要迁移为 `file_log` 并采集专用日志文件,或配置实现+原投递协议的 `custom` Sink。-HTTP Sink 使用本地持久化队列重试,同一事件可能被重复发送,采集端需要按 `CountRecord.uniqueId` 去重。所有 `2xx` 响应都表示批次已确认;瞬时失败使用指数退避重试;`400` 和 `422` 会将批次移入 `dead_letter`;`413` 会拆分包含多条事件的批次。outbox 总量受 `max_outbox_bytes` 限制,达到上限后先删除最旧的 dead-letter 批次,再删除最旧的 pending 批次,正在发送的批次不会被淘汰。由于容量压力可能丢弃 pending 数据,整体上报仍是 best-effort,不提供端到端 at-least-once 保证。+启动服务前,需要设置 `resource_id_env` 指定的环境变量。Sink 会自动创建父目录、立即追加事件、按 UTC 每小时滚动文件,并保留 `backup_count` 个历史文件;它不会写入 OpenViking 默认 stdout 日志。-`inflight_lease_seconds` 必须大于 `request_timeout_seconds`。worker 领取批次时会刷新 lease,避免其他 worker 把正在发送的批次误判为过期任务。+每行是包含 `key` 和 `value` 字段的 JSON envelope。`key` 与原 Kafka 消息键一致,格式为 `resource_id|account_id|user_id|resource_uri`;`resource_uri` 为空时使用 `session_id`。`value` 是原 Kafka 消息的完整对象,包含 `count_name`、`op_type`、`amount`、`timestamp`、`unique_id`、`tags`、`extra` 和 `prefix`。JSON envelope 能完整保留 key 内部的分隔符。文件采集和下游投递仍为 best-effort,消费端应按 `value.unique_id` 去重。支持的 add target URI:diff --git a/openviking/server/agent_evolution_config.py b/openviking/server/agent_evolution_config.pynew file mode 100644index 000000000..091396165--- /dev/null+++ b/openviking/server/agent_evolution_config.py@@ -0,0 +1,68 @@+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.+# SPDX-License-Identifier: AGPL-3.0+"""Live access to the instance-wide Agent Evolution switch."""++from __future__ import annotations++from pathlib import Path+from typing import Optional++from openviking.server.config import ServerConfig+from openviking_cli.utils.config import load_json_config+from openviking_cli.utils.logger import get_logger++logger = get_logger(__name__)+++class AgentEvolutionConfigProvider:+ """Read the live Agent Evolution value from the configured ov.conf.++ Kubernetes projects Secret updates into the mounted config directory+ atomically. Reading the file at commit time lets already-created sessions+ observe the new value without restarting the server.+ """++ def __init__(+ self,+ *,+ default_enabled: bool,+ config_path: Optional[str | Path] = None,+ ) -> None:+ self._last_valid_enabled = bool(default_enabled)+ self._config_path = Path(config_path).expanduser() if config_path else None++ def set_default_enabled(self, enabled: bool) -> None:+ self._last_valid_enabled = bool(enabled)++ def is_enabled(self) -> bool:+ config_path = self._config_path+ if config_path is None:+ return self._last_valid_enabled++ try:+ payload = load_json_config(config_path)+ if not isinstance(payload, dict):+ raise ValueError("config root must be an object")+ server_config = payload.get("server", {})+ if server_config is None:+ server_config = {}+ if not isinstance(server_config, dict):+ raise ValueError("server must be an object")+ enabled = ServerConfig.model_validate(server_config).agent_evolution.enabled+ except OSError as exc:+ logger.warning(+ "Failed to access Agent Evolution config file %s, using last valid value: %s",+ config_path,+ exc,+ )+ return self._last_valid_enabled+ except (KeyError, TypeError, ValueError) as exc:+ logger.warning(+ "Failed to read Agent Evolution config from %s, using last valid value: %s",+ config_path,+ exc,+ )+ return self._last_valid_enabled++ self._last_valid_enabled = enabled+ return enableddiff --git a/openviking/server/app.py b/openviking/server/app.pyindex 1c1e4830c..42169fd5e 100644--- a/openviking/server/app.py+++ b/openviking/server/app.py@@ -58,7 +58,12 @@ from openviking.service.core import OpenVikingServicefrom openviking.service.task_tracker import get_task_trackerfrom openviking_cli.exceptions import OpenVikingErrorfrom openviking_cli.utils import get_logger-from openviking_cli.utils.config import get_openviking_config+from openviking_cli.utils.config import (+ DEFAULT_OV_CONF,+ OPENVIKING_CONFIG_ENV,+ get_openviking_config,+ resolve_config_path,+)from openviking_cli.utils.logger import init_otel_log_handler_from_server_configlogger = get_logger(__name__)@@ -69,14 +74,20 @@ WORKER_BOT_API_URL_ENV = "OPENVIKING_WORKER_BOT_API_URL"def create_worker_app() -> FastAPI:"""Load file config and replay parent-process Bot CLI overrides."""- config = load_server_config()+ resolved_config_path = resolve_config_path(+ None,+ OPENVIKING_CONFIG_ENV,+ DEFAULT_OV_CONF,+ )+ config_path = str(resolved_config_path) if resolved_config_path is not None else None+ config = load_server_config(config_path)with_bot = os.environ.get(WORKER_WITH_BOT_ENV)if with_bot is not None:config.with_bot = with_bot == "1"bot_api_url = os.environ.get(WORKER_BOT_API_URL_ENV)if bot_api_url is not None:config.bot_api_url = bot_api_url- return create_app(config)+ return create_app(config, config_path=config_path)async def _initialize_auth_plugin(@@ -203,18 +214,27 @@ def _message_from_http_detail(detail: object) -> str:def create_app(config: Optional[ServerConfig] = None,service: Optional[OpenVikingService] = None,+ config_path: Optional[str] = None,) -> FastAPI:"""Create FastAPI application.Args:config: Server configuration. If None, loads from default location.service: Pre-initialized OpenVikingService (optional).+ config_path: Resolved ov.conf path used for live configuration reload.Returns:FastAPI application instance"""+ resolved_config_path = (+ resolve_config_path(config_path, OPENVIKING_CONFIG_ENV, DEFAULT_OV_CONF)+ if config_path is not None or config is None+ else None+ )if config is None:- config = load_server_config()+ config = load_server_config(+ str(resolved_config_path) if resolved_config_path is not None else config_path+ )validate_server_config(config)@@ -242,6 +262,15 @@ def create_app(agent_evolution_setter = getattr(sessions, "set_agent_evolution_config", None)if callable(agent_evolution_setter):agent_evolution_setter(config.agent_evolution)+ agent_evolution_path_setter = getattr(+ sessions,+ "set_agent_evolution_config_path",+ None,+ )+ if callable(agent_evolution_path_setter):+ agent_evolution_path_setter(+ str(resolved_config_path) if resolved_config_path is not None else None+ )if service is not None:_configure_session_runtime(service)diff --git a/openviking/server/bootstrap.py b/openviking/server/bootstrap.pyindex 8c1575328..4a35548e7 100644--- a/openviking/server/bootstrap.py+++ b/openviking/server/bootstrap.py@@ -207,6 +207,11 @@ def main():# Load server config from ov.conftry:+ resolved_config_path = resolve_config_path(+ args.config,+ OPENVIKING_CONFIG_ENV,+ DEFAULT_OV_CONF,+ )config = load_server_config(args.config)OpenVikingConfigSingleton.initialize(config_path=args.config)except (FileNotFoundError, ValueError) as e:@@ -279,7 +284,12 @@ def main():sys.exit(1)# Create and run server app- app = create_app(config)+ app = create_app(+ config,+ config_path=(+ str(resolved_config_path) if resolved_config_path is not None else args.config+ ),+ )workers_info = f" (workers: {config.workers})" if config.workers > 1 else ""print(f"OpenViking HTTP Server is running on {config.host}:{config.port}{workers_info}")diff --git a/openviking/server/config.py b/openviking/server/config.pyindex 71d232bb3..27afda988 100644--- a/openviking/server/config.py+++ b/openviking/server/config.py@@ -196,7 +196,7 @@ class UsageAuditConfig(BaseModel):class UsageReporterSinkConfig(BaseModel):"""Usage reporter sink configuration."""- type: Literal["custom", "http"] = "custom"+ type: Literal["custom", "file_log"] = "custom"class_path: Optional[str] = Noneconfig: Dict[str, object] = Field(default_factory=dict)diff --git a/openviking/server/routers/admin.py b/openviking/server/routers/admin.pyindex 56c686ad9..4726b3bc5 100644--- a/openviking/server/routers/admin.py+++ b/openviking/server/routers/admin.py@@ -33,6 +33,7 @@ from openviking_cli.exceptions import (PermissionDeniedError,)from openviking_cli.session.user_id import UserIdentifier+from openviking_cli.utils.config import get_openviking_configfrom openviking_cli.utils.logger import get_loggerlogger = get_logger(__name__)@@ -66,6 +67,24 @@ class MigrateLegacyDataRequest(BaseModel):action: str = "migrate"[email protected]("/agent-evolution")+@require_auth_root+async def get_agent_evolution_status(+ request: Request,+ ctx: RequestContext = Depends(get_request_context),+):+ """Return the live instance-wide Agent Evolution switch."""+ del request+ del ctx+ return Response(+ status="ok",+ result={+ "enabled": get_service().sessions.get_agent_evolution_enabled(),+ "account_id": get_openviking_config().default_account,+ },+ )++def _get_api_key_manager(request: Request):"""Get APIKeyManager from app state."""return get_api_key_manager_or_raise(request)diff --git a/openviking/service/session_service.py b/openviking/service/session_service.pyindex 25b8704cf..14f225d04 100644--- a/openviking/service/session_service.py+++ b/openviking/service/session_service.py@@ -9,6 +9,7 @@ Provides session management operations: session, sessions, add_message, commit,from typing import TYPE_CHECKING, Any, Dict, List, Optionalfrom openviking.core.namespace import canonical_session_uri+from openviking.server.agent_evolution_config import AgentEvolutionConfigProviderfrom openviking.server.config import AgentEvolutionConfig, ToolOutputExternalizationConfigfrom openviking.server.identity import RequestContextfrom openviking.service.task_tracker import get_task_tracker@@ -48,6 +49,7 @@ class SessionService:# Agent memory behavior; HTTP servers always override this from# server.agent_evolution during app setup.self._agent_evolution_enabled = True+ self._agent_evolution_config_provider: Optional[AgentEvolutionConfigProvider] = Noneself._usage_reporter: Optional["UsageReporter"] = Nonedef set_dependencies(@@ -70,6 +72,25 @@ class SessionService:def set_agent_evolution_config(self, config: AgentEvolutionConfig) -> None:"""Set the instance-wide Agent Evolution switch."""self._agent_evolution_enabled = config.enabled+ if self._agent_evolution_config_provider is not None:+ self._agent_evolution_config_provider.set_default_enabled(config.enabled)++ def set_agent_evolution_config_path(self, config_path: Optional[str]) -> None:+ """Enable live reload from the HTTP server's resolved ov.conf path."""+ self._agent_evolution_config_provider = (+ AgentEvolutionConfigProvider(+ default_enabled=self._agent_evolution_enabled,+ config_path=config_path,+ )+ if config_path+ else None+ )++ def get_agent_evolution_enabled(self) -> bool:+ """Return the live instance-wide Agent Evolution switch."""+ if self._agent_evolution_config_provider is None:+ return self._agent_evolution_enabled+ return self._agent_evolution_config_provider.is_enabled()def set_usage_reporter(self, usage_reporter: Optional["UsageReporter"]) -> None:"""Set the usage reporter for newly created sessions."""@@ -134,7 +155,8 @@ class SessionService:session_id=session_id,session_uri=session_uri,tool_output_externalization_config=self._tool_output_externalization_config,- agent_evolution_enabled=self._agent_evolution_enabled,+ agent_evolution_enabled=self.get_agent_evolution_enabled(),+ agent_evolution_enabled_provider=self.get_agent_evolution_enabled,usage_reporter=self._usage_reporter,)@@ -377,7 +399,7 @@ class SessionService:session_id=session_id,ctx=ctx,archive_uri=archive_uri,- agent_evolution_enabled=self._agent_evolution_enabled,+ agent_evolution_enabled=self.get_agent_evolution_enabled(),)self._record_lifecycle_metric("extract", "ok")return memoriesdiff --git a/openviking/session/compressor_v3.py b/openviking/session/compressor_v3.pyindex 95e611a5a..a198817e3 100644--- a/openviking/session/compressor_v3.py+++ b/openviking/session/compressor_v3.py@@ -111,10 +111,10 @@ async def _commit_experience_snapshot(]if not paths:return- archive_name = archive_uri.rstrip("/").rsplit("/", 1)[-1] if archive_uri else "unknown"+ archive_ref = archive_uri.rstrip("/") if archive_uri else "unknown"try:await commit(- message=f"Update experience memories from session commit {archive_name}",+ message=f"Update experience memories from session commit {archive_ref}",paths=paths,ctx=ctx,)diff --git a/openviking/session/session.py b/openviking/session/session.pyindex b28a8745e..80ec7f356 100644--- a/openviking/session/session.py+++ b/openviking/session/session.py@@ -544,6 +544,7 @@ class Session:tool_output_externalization_config: Optional[ToolOutputExternalizationConfig] = None,agent_evolution_enabled: bool = True,usage_reporter: Optional["UsageReporter"] = None,+ agent_evolution_enabled_provider: Optional[Callable[[], bool]] = None,):self._viking_fs = viking_fsself._vikingdb_manager = vikingdb_manager@@ -575,6 +576,7 @@ class Session:else ToolOutputExternalizationConfig())self._agent_evolution_enabled = agent_evolution_enabled+ self._agent_evolution_enabled_provider = agent_evolution_enabled_providerself._usage_reporter = usage_reporterlogger.info(f"Session created: {self.session_id} for user {self.user}")@@ -1710,7 +1712,11 @@ class Session:memory_policy if memory_policy is not None else self._meta.memory_policy)_validate_memory_policy_types(effective_policy)- agent_evolution_enabled = self._agent_evolution_enabled+ agent_evolution_enabled = (+ self._agent_evolution_enabled_provider()+ if self._agent_evolution_enabled_provider is not None+ else self._agent_evolution_enabled+ )effective_policy = _apply_agent_evolution_setting(effective_policy,agent_evolution_enabled=agent_evolution_enabled,diff --git a/openviking/usage_reporter/__init__.py b/openviking/usage_reporter/__init__.pyindex d38f5d585..c52fe5536 100644--- a/openviking/usage_reporter/__init__.py+++ b/openviking/usage_reporter/__init__.py@@ -3,14 +3,14 @@"""Usage reporting extension points for OpenViking."""from .extractors import MemoryUsageExtractor, UsageExtractor-from .http_sink import HttpUsageSink+from .file_log_sink import FileLogUsageSinkfrom .models import UsageContext, UsageEventfrom .reporter import UsageReporterfrom .sinks import UsageSink__all__ = [+ "FileLogUsageSink","MemoryUsageExtractor",- "HttpUsageSink","UsageContext","UsageEvent","UsageExtractor",diff --git a/openviking/usage_reporter/config.py b/openviking/usage_reporter/config.pyindex f6d0ecf05..ad8645cca 100644--- a/openviking/usage_reporter/config.py+++ b/openviking/usage_reporter/config.py@@ -10,7 +10,7 @@ from typing import Anyfrom openviking.server.config import UsageReporterConfig, UsageReporterSinkConfigfrom .extractors import MemoryUsageExtractor-from .http_sink import HttpUsageSink+from .file_log_sink import FileLogUsageSinkfrom .reporter import UsageReporterfrom .sinks import UsageSink@@ -25,8 +25,8 @@ def _load_class(class_path: str) -> type:def _build_sink(config: UsageReporterSinkConfig) -> UsageSink:- if config.type == "http":- return HttpUsageSink(**dict(config.config))+ if config.type == "file_log":+ return FileLogUsageSink(**dict(config.config))if config.type == "custom":if not config.class_path:diff --git a/openviking/usage_reporter/file_log_sink.py b/openviking/usage_reporter/file_log_sink.pynew file mode 100644index 000000000..69e72f027--- /dev/null+++ b/openviking/usage_reporter/file_log_sink.py@@ -0,0 +1,263 @@+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.+# SPDX-License-Identifier: AGPL-3.0+"""Hourly rotating key-value usage log sink."""++from __future__ import annotations++import asyncio+import json+import logging+import os+from contextlib import contextmanager+from datetime import datetime, timezone+from logging.handlers import TimedRotatingFileHandler+from pathlib import Path+from typing import TYPE_CHECKING, Any, Iterator++if TYPE_CHECKING:+ from .models import UsageEvent++_COUNT_NAMES = {+ "memory.recalled": "experience.recall.count",+ "memory.injected": "experience.inject.count",+}+++@contextmanager+def _interprocess_lock(path: Path) -> Iterator[None]:+ """Serialize log rotation across server worker processes."""+ descriptor = os.open(path, os.O_CREAT | os.O_RDWR, 0o600)+ try:+ if os.name == "nt":+ import msvcrt++ if os.fstat(descriptor).st_size == 0:+ os.write(descriptor, b"\0")+ os.lseek(descriptor, 0, os.SEEK_SET)+ msvcrt.locking(descriptor, msvcrt.LK_LOCK, 1)+ else:+ import fcntl++ fcntl.flock(descriptor, fcntl.LOCK_EX)+ yield+ finally:+ try:+ if os.name == "nt":+ import msvcrt++ os.lseek(descriptor, 0, os.SEEK_SET)+ msvcrt.locking(descriptor, msvcrt.LK_UNLCK, 1)+ else:+ import fcntl++ fcntl.flock(descriptor, fcntl.LOCK_UN)+ finally:+ os.close(descriptor)+++class _ProcessSafeTimedRotatingFileHandler(TimedRotatingFileHandler):+ """Timed handler that coordinates rollover with sibling worker processes."""++ def __init__(self, filename: Path, **kwargs: Any) -> None:+ self._process_lock_path = filename.with_name(f".{filename.name}.lock")+ super().__init__(filename=filename, **kwargs)+ self._base_file_identity = self._get_base_file_identity()++ def emit(self, record: logging.LogRecord) -> None:+ with _interprocess_lock(self._process_lock_path):+ self._refresh_after_external_rollover()+ try:+ super().emit(record)+ finally:+ self._base_file_identity = self._get_base_file_identity()+ # Windows does not allow another process to rename an open+ # file. Release the handle so the next worker can roll it.+ if os.name == "nt" and self.stream is not None:+ self.stream.close()+ self.stream = None++ @staticmethod+ def _file_identity(stat_result: os.stat_result) -> tuple[int, int]:+ return stat_result.st_dev, stat_result.st_ino++ def _get_base_file_identity(self) -> tuple[int, int] | None:+ try:+ return self._file_identity(os.stat(self.baseFilename))+ except FileNotFoundError:+ return None++ def _refresh_after_external_rollover(self) -> None:+ """Reopen a base file rotated by another process and sync its deadline."""+ try:+ path_stat = os.stat(self.baseFilename)+ except FileNotFoundError:+ path_stat = None+ path_identity = self._file_identity(path_stat) if path_stat is not None else None++ stream_stat = None+ if self.stream is not None:+ try:+ stream_stat = os.fstat(self.stream.fileno())+ except (OSError, ValueError):+ stream_stat = None++ if stream_stat is not None and path_stat is not None:+ if self._file_identity(stream_stat) == path_identity:+ self._base_file_identity = path_identity+ return+ elif self.stream is None and path_identity == self._base_file_identity:+ # Windows closes the stream after every emit. The missing handle+ # alone does not mean another worker rotated the file, so preserve+ # this handler's existing rollover deadline.+ return++ if self.stream is not None:+ self.stream.close()+ self.stream = None+ self._base_file_identity = path_identity+ if path_stat is not None:+ self.rolloverAt = self.computeRollover(int(path_stat.st_mtime))+++def _timestamp_millis(value: str) -> int:+ parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))+ if parsed.tzinfo is None:+ parsed = parsed.replace(tzinfo=timezone.utc)+ return int(parsed.timestamp() * 1000)+++def _to_count_record(event: UsageEvent, *, resource_id: str) -> dict[str, Any]:+ record = event.to_dict()+ event_type = str(record.get("event_type") or "")+ try:+ count_name = _COUNT_NAMES[event_type]+ except KeyError as exc:+ raise ValueError(f"unsupported usage event type: {event_type}") from exc++ evidence = record.get("evidence")+ evidence = evidence if isinstance(evidence, dict) else {}+ attributes = record.get("attributes")+ attributes = attributes if isinstance(attributes, dict) else {}++ extra = {key: value for key, value in evidence.items() if value not in (None, "")}+ session_id = str(record.get("session_id") or "")+ if session_id:+ extra["session_id"] = session_id+ task_id = record.get("task_id")+ if task_id not in (None, ""):+ extra["task_id"] = task_id+ if attributes:+ extra["attributes"] = attributes++ unique_id = str(record.get("event_id") or "").strip()+ if not unique_id:+ raise ValueError("event_id is required")++ count_record = {+ "count_name": count_name,+ "op_type": "add",+ "amount": 1.0,+ "timestamp": _timestamp_millis(str(record.get("occurred_at") or "")),+ "unique_id": unique_id,+ "tags": {+ "account_id": str(record.get("account_id") or ""),+ "user_id": str(record.get("user_id") or ""),+ "resource_uri": str(record.get("resource_uri") or ""),+ "resource_type": str(record.get("resource_type") or ""),+ },+ "extra": extra,+ }+ count_record["prefix"] = resource_id+ return count_record+++def _build_message_key(record: dict[str, Any]) -> str:+ tags = record.get("tags")+ tags = tags if isinstance(tags, dict) else {}+ extra = record.get("extra")+ extra = extra if isinstance(extra, dict) else {}+ target = str(tags.get("resource_uri") or extra.get("session_id") or "")+ return "|".join(+ (+ str(record.get("prefix") or ""),+ str(tags.get("account_id") or ""),+ str(tags.get("user_id") or ""),+ target,+ )+ )+++def _to_log_record(event: UsageEvent, *, resource_id: str) -> dict[str, Any]:+ value = _to_count_record(event, resource_id=resource_id)+ return {+ "key": _build_message_key(value),+ "value": value,+ }+++def _format_record(record: dict[str, Any]) -> str:+ return json.dumps(record, ensure_ascii=False, separators=(",", ":"))+++class FileLogUsageSink:+ """Write usage records to a dedicated hourly rotating log file."""++ def __init__(+ self,+ *,+ path: str | os.PathLike[str],+ resource_id_env: str = "OV_RESOURCE_ID",+ rotation_interval_hours: int = 1,+ backup_count: int = 168,+ ) -> None:+ raw_path = str(path).strip()+ if not raw_path:+ raise ValueError("path is required")+ self._resource_id = os.environ.get(resource_id_env, "").strip()+ if not self._resource_id:+ raise ValueError(f"{resource_id_env} is required")+ if rotation_interval_hours <= 0:+ raise ValueError("rotation_interval_hours must be positive")+ if backup_count < 0:+ raise ValueError("backup_count must be non-negative")++ self._path = Path(os.path.expandvars(os.path.expanduser(raw_path)))+ self._path.parent.mkdir(parents=True, exist_ok=True)+ self._handler = _ProcessSafeTimedRotatingFileHandler(+ filename=self._path,+ when="H",+ interval=rotation_interval_hours,+ backupCount=backup_count,+ encoding="utf-8",+ delay=True,+ utc=True,+ )+ self._handler.setFormatter(logging.Formatter("%(message)s"))+ self._logger = logging.getLogger(f"openviking.usage_reporter.file_log.{id(self)}")+ self._logger.setLevel(logging.INFO)+ self._logger.propagate = False+ self._logger.addHandler(self._handler)+ self._closed = False++ @property+ def path(self) -> Path:+ return self._path++ async def write(self, *, events: list[UsageEvent]) -> None:+ if not events:+ return+ records = [_to_log_record(event, resource_id=self._resource_id) for event in events]+ await asyncio.to_thread(self._write_records, records)++ def _write_records(self, records: list[dict[str, Any]]) -> None:+ if self._closed:+ raise RuntimeError("usage log sink is closed")+ for record in records:+ self._logger.info(_format_record(record))++ def close(self) -> None:+ if self._closed:+ return+ self._closed = True+ self._logger.removeHandler(self._handler)+ self._handler.close()diff --git a/openviking/usage_reporter/http_sink.py b/openviking/usage_reporter/http_sink.pydeleted file mode 100644index ead3e0c85..000000000--- a/openviking/usage_reporter/http_sink.py+++ /dev/null@@ -1,524 +0,0 @@-# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.-# SPDX-License-Identifier: AGPL-3.0-"""Durable HTTP usage event sink."""--from __future__ import annotations--import asyncio-import hashlib-import json-import logging-import os-import random-import threading-import time-import urllib.error-import urllib.request-from collections.abc import Iterator-from datetime import datetime, timezone-from pathlib import Path-from typing import TYPE_CHECKING, Any--if TYPE_CHECKING:- from .models import UsageEvent--logger = logging.getLogger(__name__)--_COUNT_NAMES = {- "memory.recalled": "experience.recall.count",- "memory.injected": "experience.inject.count",-}---def _timestamp_millis(value: str) -> int:- parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))- if parsed.tzinfo is None:- parsed = parsed.replace(tzinfo=timezone.utc)- return int(parsed.timestamp() * 1000)---def _to_count_record(event: UsageEvent) -> dict[str, Any]:- record = event.to_dict()- event_type = str(record.get("event_type") or "")- try:- count_name = _COUNT_NAMES[event_type]- except KeyError as exc:- raise ValueError(f"unsupported usage event type: {event_type}") from exc-- evidence = record.get("evidence")- evidence = evidence if isinstance(evidence, dict) else {}- attributes = record.get("attributes")- attributes = attributes if isinstance(attributes, dict) else {}-- extra = {key: value for key, value in evidence.items() if value not in (None, "")}- session_id = str(record.get("session_id") or "")- if session_id:- extra["session_id"] = session_id- task_id = record.get("task_id")- if task_id not in (None, ""):- extra["task_id"] = task_id- if attributes:- extra["attributes"] = attributes-- return {- "CountName": count_name,- "OpType": "add",- "amount": 1.0,- "timestamp": _timestamp_millis(str(record.get("occurred_at") or "")),- "uniqueId": str(record.get("event_id") or ""),- "tags": {- "account_id": str(record.get("account_id") or ""),- "user_id": str(record.get("user_id") or ""),- "resource_uri": str(record.get("resource_uri") or ""),- "resource_type": str(record.get("resource_type") or ""),- },- "extra": extra,- }---class HttpUsageSink:- """Persist usage events before delivering them to an HTTP collector."""-- def __init__(- self,- *,- endpoint: str,- resource_id_env: str = "OV_RESOURCE_ID",- outbox_dir: str | os.PathLike[str] | None = None,- request_timeout_seconds: float = 10.0,- inflight_lease_seconds: float = 60.0,- retry_base_seconds: float = 1.0,- retry_max_seconds: float = 300.0,- max_batch_bytes: int = 1024 * 1024,- max_outbox_bytes: int = 256 * 1024 * 1024,- ) -> None:- self._endpoint = endpoint.strip()- self._resource_id = os.environ.get(resource_id_env, "").strip()- if not self._endpoint:- raise ValueError("endpoint is required")- if not self._resource_id:- raise ValueError(f"{resource_id_env} is required")- self._outbox_scope = hashlib.sha256(- f"{self._endpoint}\0{self._resource_id}".encode("utf-8")- ).hexdigest()[:24]-- self._outbox_dir = (- Path(outbox_dir)- if outbox_dir is not None- else Path.home() / ".openviking" / "data" / ".usage_outbox"- )- self._pending_dir = self._outbox_dir / "pending"- self._inflight_dir = self._outbox_dir / "inflight"- self._dead_letter_dir = self._outbox_dir / "dead_letter"- self._metadata_dir = self._outbox_dir / "metadata"- for directory in (- self._pending_dir,- self._inflight_dir,- self._dead_letter_dir,- self._metadata_dir,- ):- directory.mkdir(parents=True, exist_ok=True)-- self._request_timeout_seconds = float(request_timeout_seconds)- self._inflight_lease_seconds = float(inflight_lease_seconds)- self._retry_base_seconds = float(retry_base_seconds)- self._retry_max_seconds = float(retry_max_seconds)- self._max_batch_bytes = int(max_batch_bytes)- self._max_outbox_bytes = int(max_outbox_bytes)- if self._request_timeout_seconds <= 0:- raise ValueError("request_timeout_seconds must be positive")- if self._inflight_lease_seconds <= 0:- raise ValueError("inflight_lease_seconds must be positive")- if self._inflight_lease_seconds <= self._request_timeout_seconds:- raise ValueError("inflight_lease_seconds must be greater than request_timeout_seconds")- if self._retry_base_seconds < 0:- raise ValueError("retry_base_seconds must be non-negative")- if self._retry_max_seconds < 0:- raise ValueError("retry_max_seconds must be non-negative")- if self._max_batch_bytes <= 0:- raise ValueError("max_batch_bytes must be positive")- if self._max_outbox_bytes <= 0:- raise ValueError("max_outbox_bytes must be positive")- self._outbox_lock = threading.Lock()- self._wake_event = threading.Event()- self._stop_event = threading.Event()- self._worker: threading.Thread | None = None- self._start_worker()-- async def write(self, *, events: list[UsageEvent]) -> None:- if not events:- return- await asyncio.to_thread(self._persist_events, events)-- def close(self) -> None:- self._stop_event.set()- self._wake_event.set()- if self._worker is not None:- self._worker.join(timeout=self._request_timeout_seconds + 1)-- def _persist_events(self, events: list[UsageEvent]) -> None:- records: list[dict[str, Any]] = []- for event in events:- event_id = str(event.to_dict().get("event_id") or "").strip()- if not event_id:- raise ValueError("event_id is required")- records.append(_to_count_record(event))-- batch: list[dict[str, Any]] = []- for record in records:- candidate = [*batch, record]- if batch and (- len(candidate) > 100- or self._encoded_payload_size(candidate) > self._max_batch_bytes- ):- self._persist_batch(batch)- batch = [record]- else:- batch = candidate- if batch:- self._persist_batch(batch)- self._wake_event.set()-- def _persist_batch(self, events: list[dict[str, Any]]) -> bool:- return self._persist_batches([events])-- def _persist_batches(- self,- event_batches: list[list[dict[str, Any]]],- *,- replace_path: Path | None = None,- ) -> bool:- encoded_batches: list[tuple[dict[str, Any], bytes, Path, Path]] = []- for index, events in enumerate(event_batches):- payload = self._build_payload(events)- encoded = json.dumps(- payload,- ensure_ascii=False,- separators=(",", ":"),- ).encode("utf-8")- file_stem = f"{self._outbox_scope}_{time.time_ns()}_{index}_{payload['batch_id']}"- encoded_batches.append(- (- payload,- encoded,- self._pending_dir / f".{file_stem}.tmp",- self._pending_dir / f"{file_stem}.json",- )- )-- incoming_bytes = sum(len(encoded) for _, encoded, _, _ in encoded_batches)- with self._outbox_lock:- reclaimable_bytes = 0- if replace_path is not None:- try:- reclaimable_bytes = replace_path.stat().st_size- except FileNotFoundError:- return False- if not self._ensure_outbox_capacity(- incoming_bytes,- reclaimable_bytes=reclaimable_bytes,- ):- batch_ids = [payload["batch_id"] for payload, _, _, _ in encoded_batches]- logger.warning(- "Dropping OpenViking usage batch because the outbox is full, "- "batch_ids=%s size=%s max_outbox_bytes=%s",- batch_ids,- incoming_bytes,- self._max_outbox_bytes,- )- return False- temporary_paths: list[Path] = []- try:- for _payload, encoded, temporary_path, pending_path in encoded_batches:- temporary_paths.append(temporary_path)- with temporary_path.open("wb") as output:- output.write(encoded)- output.flush()- os.fsync(output.fileno())- os.replace(temporary_path, pending_path)- self._fsync_directory(self._pending_dir)- if replace_path is not None:- replace_path.unlink()- self._fsync_directory(replace_path.parent)- finally:- for temporary_path in temporary_paths:- try:- temporary_path.unlink()- except FileNotFoundError:- pass- return True-- def _ensure_outbox_capacity(- self,- incoming_bytes: int,- *,- reclaimable_bytes: int = 0,- ) -> bool:- if incoming_bytes > self._max_outbox_bytes:- return False-- eviction_candidates: list[Path] = []- total_bytes = 0- for directory in (- self._dead_letter_dir,- self._pending_dir,- self._inflight_dir,- ):- for path in self._owned_batches(directory):- try:- total_bytes += path.stat().st_size- except FileNotFoundError:- continue- if directory != self._inflight_dir:- eviction_candidates.append(path)-- effective_total_bytes = max(0, total_bytes - reclaimable_bytes)- if effective_total_bytes + incoming_bytes <= self._max_outbox_bytes:- return True-- def _eviction_order(path: Path) -> tuple[int, int, str]:- try:- modified_at = path.stat().st_mtime_ns- except FileNotFoundError:- modified_at = 0- directory_priority = 0 if path.parent == self._dead_letter_dir else 1- return directory_priority, modified_at, path.name-- for path in sorted(eviction_candidates, key=_eviction_order):- try:- size = path.stat().st_size- path.unlink()- except FileNotFoundError:- continue- effective_total_bytes -= size- logger.warning("Evicted OpenViking usage outbox batch: %s", path)- if effective_total_bytes + incoming_bytes <= self._max_outbox_bytes:- return True- return effective_total_bytes + incoming_bytes <= self._max_outbox_bytes-- def _build_payload(self, events: list[dict[str, Any]]) -> dict[str, Any]:- unique_ids = "\n".join(str(event["uniqueId"]) for event in events)- batch_id = f"ub_{hashlib.sha256(unique_ids.encode('utf-8')).hexdigest()}"- created_at = (- datetime.now(timezone.utc)- .isoformat(timespec="milliseconds")- .replace(- "+00:00",- "Z",- )- )- return {- "schema_version": "v1",- "resource_id": self._resource_id,- "batch_id": batch_id,- "created_at": created_at,- "attempt": 0,- "next_retry_at": created_at,- "events": events,- }-- def _encoded_payload_size(self, events: list[dict[str, Any]]) -> int:- return len(- json.dumps(- self._build_payload(events),- ensure_ascii=False,- separators=(",", ":"),- ).encode("utf-8")- )-- @staticmethod- def _fsync_directory(directory: Path) -> None:- if os.name == "nt":- return- descriptor = os.open(directory, os.O_RDONLY)- try:- os.fsync(descriptor)- finally:- os.close(descriptor)-- def _start_worker(self) -> None:- self._recover_stale_inflight()- with self._outbox_lock:- self._ensure_outbox_capacity(0)- self._worker = threading.Thread(- target=self._worker_loop,- name="openviking-usage-http-sink",- daemon=True,- )- self._worker.start()-- def _worker_loop(self) -> None:- while not self._stop_event.is_set():- try:- self._recover_stale_inflight()- handled_batch = self._handle_next_batch()- except Exception:- logger.exception("OpenViking usage outbox worker failed")- handled_batch = False- if handled_batch:- continue- self._wake_event.wait(timeout=0.05)- self._wake_event.clear()-- def _handle_next_batch(self) -> bool:- now = datetime.now(timezone.utc)- for pending_path in sorted(self._owned_batches(self._pending_dir)):- try:- payload = self._read_payload(pending_path)- except (OSError, ValueError, json.JSONDecodeError):- self._move_to_dead_letter(pending_path)- continue- next_retry_at = self._parse_datetime(str(payload.get("next_retry_at") or ""))- if next_retry_at is not None and next_retry_at > now:- continue-- inflight_path = self._inflight_dir / pending_path.name- try:- os.replace(pending_path, inflight_path)- os.utime(inflight_path, None)- except FileNotFoundError:- continue-- try:- status = self._post_payload(payload)- except Exception:- logger.exception(- "OpenViking usage delivery failed unexpectedly, batch_id=%s",- payload.get("batch_id"),- )- self._schedule_retry(inflight_path, payload)- return True- if 200 <= status < 300:- try:- inflight_path.unlink()- except FileNotFoundError:- pass- return True- if status == 413:- self._split_batch(inflight_path, payload)- return True- if status in (400, 422):- self._move_to_dead_letter(inflight_path)- return True-- self._schedule_retry(inflight_path, payload)- return True- return False-- def _split_batch(self, inflight_path: Path, payload: dict[str, Any]) -> None:- events = payload.get("events")- if not isinstance(events, list) or len(events) < 2:- self._move_to_dead_letter(inflight_path)- return- midpoint = len(events) // 2- if not self._persist_batches(- [events[:midpoint], events[midpoint:]],- replace_path=inflight_path,- ):- self._move_to_dead_letter(inflight_path)- self._wake_event.set()-- def _post_payload(self, payload: dict[str, Any]) -> int:- request = urllib.request.Request(- self._endpoint,- data=json.dumps(- payload,- ensure_ascii=False,- separators=(",", ":"),- ).encode("utf-8"),- headers={- "Content-Type": "application/json",- "V-Resource-Id": self._resource_id,- },- method="POST",- )- try:- with urllib.request.urlopen(- request,- timeout=self._request_timeout_seconds,- ) as response:- response.read()- return response.status- except urllib.error.HTTPError as error:- error.read()- return error.code- except (OSError, TimeoutError, urllib.error.URLError):- return 0-- def _schedule_retry(self, inflight_path: Path, payload: dict[str, Any]) -> None:- attempt = int(payload.get("attempt") or 0) + 1- delay = min(- self._retry_max_seconds,- self._retry_base_seconds * (2 ** max(0, attempt - 1)),- )- if delay > 0:- delay *= random.uniform(0.8, 1.2)- payload["attempt"] = attempt- payload["next_retry_at"] = (- datetime.fromtimestamp(time.time() + delay, tz=timezone.utc)- .isoformat(timespec="milliseconds")- .replace("+00:00", "Z")- )- try:- self._replace_payload(inflight_path, payload)- os.replace(inflight_path, self._pending_dir / inflight_path.name)- except FileNotFoundError:- return- self._wake_event.set()-- def _replace_payload(self, path: Path, payload: dict[str, Any]) -> None:- temporary_path = path.with_name(f".{path.name}.tmp")- encoded = json.dumps(- payload,- ensure_ascii=False,- separators=(",", ":"),- ).encode("utf-8")- try:- with temporary_path.open("wb") as output:- output.write(encoded)- output.flush()- os.fsync(output.fileno())- os.replace(temporary_path, path)- self._fsync_directory(path.parent)- finally:- try:- temporary_path.unlink()- except FileNotFoundError:- pass-- @staticmethod- def _read_payload(path: Path) -> dict[str, Any]:- with path.open("r", encoding="utf-8") as source:- payload = json.load(source)- if not isinstance(payload, dict):- raise ValueError("outbox payload must be an object")- return payload-- @staticmethod- def _parse_datetime(value: str) -> datetime | None:- if not value:- return None- try:- return datetime.fromisoformat(value.replace("Z", "+00:00"))- except ValueError:- return None-- def _move_to_dead_letter(self, path: Path) -> None:- try:- os.replace(path, self._dead_letter_dir / path.name)- except FileNotFoundError:- return-- def _recover_stale_inflight(self) -> None:- stale_before = time.time() - self._inflight_lease_seconds- with self._outbox_lock:- for inflight_path in self._owned_batches(self._inflight_dir):- try:- if inflight_path.stat().st_mtime > stale_before:- continue- os.replace(inflight_path, self._pending_dir / inflight_path.name)- except FileNotFoundError:- continue-- def _owned_batches(self, directory: Path) -> Iterator[Path]:- return directory.glob(f"{self._outbox_scope}_*.json")diff --git a/tests/server/test_agent_evolution_global_setting.py b/tests/server/test_agent_evolution_global_setting.pyindex ec0c9b7e5..3c191a9a7 100644--- a/tests/server/test_agent_evolution_global_setting.py+++ b/tests/server/test_agent_evolution_global_setting.py@@ -1,12 +1,20 @@# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.# SPDX-License-Identifier: AGPL-3.0+import json+from types import SimpleNamespacefrom unittest.mock import AsyncMockimport httpx+from openviking.server.agent_evolution_config import AgentEvolutionConfigProvider+from openviking.server.app import create_appfrom openviking.server.config import AgentEvolutionConfig, ServerConfig, UserConfig+from openviking.server.identity import RequestContext, Rolefrom openviking.service.session_service import SessionService+from openviking.session import Session+from openviking_cli.session.user_id import UserIdentifier+from openviking_cli.utils.config import OPENVIKING_CONFIG_ENVdef test_agent_evolution_is_disabled_by_default():@@ -19,10 +27,190 @@ def test_agent_evolution_can_be_enabled_for_the_server():assert config.agent_evolution.enabled is True-def test_embedded_session_service_preserves_agent_evolution_default():+def test_embedded_session_service_preserves_agent_evolution_default(+ tmp_path,+ monkeypatch,+):+ config_path = tmp_path / "ov.conf"+ config_path.write_text(+ json.dumps({"server": {"agent_evolution": {"enabled": False}}}),+ encoding="utf-8",+ )+ monkeypatch.setenv(OPENVIKING_CONFIG_ENV, str(config_path))+service = SessionService()- assert service._agent_evolution_enabled is True+ assert service.get_agent_evolution_enabled() is True+++def test_existing_session_observes_updated_runtime_value():+ service = SessionService(viking_fs=object())+ session = service.session(+ RequestContext(user=UserIdentifier.the_default_user(), role=Role.ROOT)+ )++ service.set_agent_evolution_config(AgentEvolutionConfig(enabled=False))++ assert session._agent_evolution_enabled_provider() is False+++def test_session_positional_usage_reporter_remains_compatible():+ reporter = object()++ session = Session(+ object(),+ None,+ None,+ None,+ None,+ "session-id",+ None,+ 8000,+ None,+ True,+ reporter,+ )++ assert session._usage_reporter is reporter+ assert session._agent_evolution_enabled_provider is None+++def test_agent_evolution_provider_reloads_config_file(tmp_path):+ config_path = tmp_path / "ov.conf"+ config_path.write_text(+ json.dumps({"server": {"agent_evolution": {"enabled": False}}}),+ encoding="utf-8",+ )+ provider = AgentEvolutionConfigProvider(+ default_enabled=True,+ config_path=config_path,+ )++ assert provider.is_enabled() is False++ config_path.write_text(+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),+ encoding="utf-8",+ )++ assert provider.is_enabled() is True+++def test_session_service_reloads_from_resolved_server_config_path(tmp_path):+ config_path = tmp_path / "ov.conf"+ config_path.write_text(+ json.dumps({"server": {"agent_evolution": {"enabled": False}}}),+ encoding="utf-8",+ )+ service = SessionService()+ service.set_agent_evolution_config(AgentEvolutionConfig(enabled=True))+ service.set_agent_evolution_config_path(str(config_path))++ assert service.get_agent_evolution_enabled() is False+++def test_http_app_wires_resolved_server_config_path(tmp_path):+ config_path = tmp_path / "ov.conf"+ config_path.write_text(+ json.dumps({"server": {"agent_evolution": {"enabled": False}}}),+ encoding="utf-8",+ )+ sessions = SessionService()++ create_app(+ config=ServerConfig(agent_evolution=AgentEvolutionConfig(enabled=True)),+ service=SimpleNamespace(sessions=sessions),+ config_path=str(config_path),+ )++ assert sessions.get_agent_evolution_enabled() is False++ config_path.write_text(+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),+ encoding="utf-8",+ )++ assert sessions.get_agent_evolution_enabled() is True+++def test_agent_evolution_provider_applies_missing_section_default(tmp_path):+ config_path = tmp_path / "ov.conf"+ config_path.write_text(+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),+ encoding="utf-8",+ )+ provider = AgentEvolutionConfigProvider(+ default_enabled=False,+ config_path=config_path,+ )+ assert provider.is_enabled() is True++ config_path.write_text(json.dumps({"server": {}}), encoding="utf-8")++ assert provider.is_enabled() is False+++def test_agent_evolution_provider_uses_standard_config_loading(tmp_path, monkeypatch):+ config_path = tmp_path / "ov.conf"+ config_path.write_text(+ '\ufeff{"server": {"agent_evolution": {"enabled": ${AGENT_EVOLUTION_ENABLED}}}}',+ encoding="utf-8",+ )+ monkeypatch.setenv("AGENT_EVOLUTION_ENABLED", "true")+ provider = AgentEvolutionConfigProvider(+ default_enabled=False,+ config_path=config_path,+ )++ assert provider.is_enabled() is True+++def test_agent_evolution_provider_keeps_last_valid_value(tmp_path):+ config_path = tmp_path / "ov.conf"+ config_path.write_text(+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),+ encoding="utf-8",+ )+ provider = AgentEvolutionConfigProvider(+ default_enabled=False,+ config_path=config_path,+ )+ assert provider.is_enabled() is True++ config_path.write_text("{", encoding="utf-8")++ assert provider.is_enabled() is True++ config_path.write_text("[]", encoding="utf-8")++ assert provider.is_enabled() is True+++def test_agent_evolution_provider_rejects_invalid_unrelated_server_config(tmp_path):+ config_path = tmp_path / "ov.conf"+ config_path.write_text(+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),+ encoding="utf-8",+ )+ provider = AgentEvolutionConfigProvider(+ default_enabled=False,+ config_path=config_path,+ )+ assert provider.is_enabled() is True++ config_path.write_text(+ json.dumps(+ {+ "server": {+ "port": "not-an-integer",+ "agent_evolution": {"enabled": False},+ }+ }+ ),+ encoding="utf-8",+ )++ assert provider.is_enabled() is Truedef test_deprecated_user_agent_evolution_config_is_not_persisted():@@ -40,6 +228,21 @@ async def test_agent_evolution_user_endpoint_is_not_registered(assert response.status_code == 404+async def test_agent_evolution_admin_endpoint_returns_runtime_value(+ service,+ client: httpx.AsyncClient,+):+ service.sessions.set_agent_evolution_config(AgentEvolutionConfig(enabled=True))++ response = await client.get("/api/v1/admin/agent-evolution")++ assert response.status_code == 200, response.text+ assert response.json()["result"] == {+ "enabled": True,+ "account_id": "default",+ }++async def test_manual_extract_respects_disabled_agent_evolution(service,client: httpx.AsyncClient,diff --git a/tests/server/test_bootstrap.py b/tests/server/test_bootstrap.pyindex 181e3c1e4..64a7f0198 100644--- a/tests/server/test_bootstrap.py+++ b/tests/server/test_bootstrap.py@@ -26,7 +26,7 @@ def test_main_keeps_config_host_when_cli_host_is_omitted(monkeypatch):monkeypatch.setattr(bootstrap,"create_app",- lambda config: "app",+ lambda config, **kwargs: "app",)monkeypatch.setattr(bootstrap,@@ -80,7 +80,7 @@ def test_main_coerces_cli_host_all_to_none(monkeypatch):monkeypatch.setattr(bootstrap,"create_app",- lambda config: "app",+ lambda config, **kwargs: "app",)monkeypatch.setattr(bootstrap,@@ -128,7 +128,7 @@ def test_main_enables_bot_logging_when_with_bot_comes_from_config(monkeypatch):bot_process = object()monkeypatch.setattr(bootstrap, "load_server_config", lambda config_path: config)- monkeypatch.setattr(bootstrap, "create_app", lambda config: "app")+ monkeypatch.setattr(bootstrap, "create_app", lambda config, **kwargs: "app")monkeypatch.setattr(bootstrap, "configure_uvicorn_logging", lambda: None)monkeypatch.setattr(OpenVikingConfigSingleton,@@ -187,7 +187,7 @@ def test_bot_alias_propagates_resolved_config_to_workers(monkeypatch):monkeypatch.setattr(bootstrap.argparse.ArgumentParser, "parse_args", parse_bot_alias)monkeypatch.setattr(bootstrap, "load_server_config", lambda config_path: config)- monkeypatch.setattr(bootstrap, "create_app", lambda config: "parent-app")+ monkeypatch.setattr(bootstrap, "create_app", lambda config, **kwargs: "parent-app")monkeypatch.setattr(bootstrap, "configure_uvicorn_logging", lambda: None)monkeypatch.setattr(bootstrap, "_abort_if_port_in_use", lambda port, label: None)monkeypatch.setattr(bootstrap, "_start_vikingbot_gateway", lambda *args, **kwargs: object())@@ -221,15 +221,40 @@ def test_bot_alias_propagates_resolved_config_to_workers(monkeypatch):def test_worker_factory_replays_bot_overrides(monkeypatch):config = ServerConfig(with_bot=False, bot_api_url="http://from-config")+ captured = {}monkeypatch.setenv(app_module.WORKER_WITH_BOT_ENV, "1")monkeypatch.setenv(app_module.WORKER_BOT_API_URL_ENV, "http://127.0.0.1:19000")- monkeypatch.setattr(app_module, "load_server_config", lambda: config)- monkeypatch.setattr(app_module, "create_app", lambda worker_config: worker_config)+ monkeypatch.setattr(+ app_module,+ "resolve_config_path",+ lambda config_path, env_var, default_name: "/tmp/worker-ov.conf",+ )+ monkeypatch.setattr(+ app_module,+ "load_server_config",+ lambda config_path: captured.update({"load_path": config_path}) or config,+ )+ monkeypatch.setattr(+ app_module,+ "create_app",+ lambda worker_config, config_path: captured.update(+ {+ "app_config": worker_config,+ "app_config_path": config_path,+ }+ )+ or worker_config,+ )worker_config = app_module.create_worker_app()assert worker_config.with_bot is Trueassert worker_config.bot_api_url == "http://127.0.0.1:19000"+ assert captured == {+ "load_path": "/tmp/worker-ov.conf",+ "app_config": config,+ "app_config_path": "/tmp/worker-ov.conf",+ }def test_resolve_queuefs_mount_point_defaults_to_shared():diff --git a/tests/session/test_compressor_v3.py b/tests/session/test_compressor_v3.pyindex dfce4eb70..fff4ffbe6 100644--- a/tests/session/test_compressor_v3.py+++ b/tests/session/test_compressor_v3.py@@ -1495,7 +1495,10 @@ async def test_v3_training_links_case_to_trajectory_and_experience_via_trajectorassert any(link["from_uri"] == case_uri for link in exp_file.backlinks)assert fs.commits == [{- "message": "Update experience memories from session commit archive_001",+ "message": (+ "Update experience memories from session commit "+ "viking://user/u/sessions/session-1/history/archive_001"+ ),"paths": [exp_uri, deleted_exp_uri],"ctx": _ctx(),}diff --git a/tests/session/test_session_commit.py b/tests/session/test_session_commit.pyindex 9ed194543..04ed3b851 100644--- a/tests/session/test_session_commit.py+++ b/tests/session/test_session_commit.py@@ -82,7 +82,7 @@ class TestCommit:async def test_commit_default_disables_agent_memory_but_keeps_archive(self, session_with_messages: Session):- session_with_messages._agent_evolution_enabled = False+ session_with_messages._agent_evolution_enabled_provider = lambda: Falsesession_with_messages._session_compressor.extract_long_term_memories = AsyncMock(return_value=[])@@ -130,7 +130,7 @@ class TestCommit:async def test_disabled_agent_evolution_keeps_working_memory(self, session_with_messages: Session, monkeypatch):- session_with_messages._agent_evolution_enabled = False+ session_with_messages._agent_evolution_enabled_provider = lambda: Falsesummary_called = Falseasync def fake_summary(_session, messages, latest_archive_overview=""):diff --git a/tests/unit/usage_reporter/test_file_log_sink.py b/tests/unit/usage_reporter/test_file_log_sink.pynew file mode 100644index 000000000..8faaf4e21--- /dev/null+++ b/tests/unit/usage_reporter/test_file_log_sink.py@@ -0,0 +1,263 @@+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.+# SPDX-License-Identifier: AGPL-3.0++from __future__ import annotations++import json+import time++import pytest++from openviking.usage_reporter.file_log_sink import FileLogUsageSink+++class FakeUsageEvent:+ def __init__(+ self,+ *,+ event_id: str = "ue_recall",+ event_type: str = "memory.recalled",+ session_id: str = "session-1",+ resource_uri: str = "viking://user/user-1/memories/experiences/exchange.md",+ ) -> None:+ self._record = {+ "schema_version": "v1",+ "event_id": event_id,+ "event_type": event_type,+ "account_id": "2101858484",+ "user_id": "user-1",+ "session_id": session_id,+ "task_id": "task-1",+ "occurred_at": "2026-07-27T04:00:00Z",+ "resource_uri": resource_uri,+ "resource_type": "experience",+ "evidence": {+ "archive_uri": ("viking://user/user-1/sessions/session-1/history/archive_001"),+ "message_id": "msg-1",+ "tool_call_id": "call-1",+ "tool_name": "search_experience",+ },+ "attributes": {"note": "contains spaces\tand tabs"},+ }++ def to_dict(self) -> dict[str, object]:+ return dict(self._record)+++def _parse_line(line: str) -> tuple[str, dict[str, object]]:+ record = json.loads(line)+ return record["key"], record["value"]+++async def test_file_log_sink_writes_original_kafka_key_and_value(tmp_path, monkeypatch):+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")+ log_path = tmp_path / "dedicated" / "usage.log"+ sink = FileLogUsageSink(path=log_path)++ try:+ await sink.write(events=[FakeUsageEvent()])+ finally:+ sink.close()++ lines = log_path.read_text(encoding="utf-8").splitlines()+ assert len(lines) == 1+ key, value = _parse_line(lines[0])+ assert key == "ov-test|2101858484|user-1|viking://user/user-1/memories/experiences/exchange.md"+ assert value == {+ "count_name": "experience.recall.count",+ "op_type": "add",+ "amount": 1.0,+ "timestamp": 1785124800000,+ "unique_id": "ue_recall",+ "tags": {+ "account_id": "2101858484",+ "user_id": "user-1",+ "resource_uri": "viking://user/user-1/memories/experiences/exchange.md",+ "resource_type": "experience",+ },+ "extra": {+ "archive_uri": "viking://user/user-1/sessions/session-1/history/archive_001",+ "message_id": "msg-1",+ "tool_call_id": "call-1",+ "tool_name": "search_experience",+ "session_id": "session-1",+ "task_id": "task-1",+ "attributes": {"note": "contains spaces\tand tabs"},+ },+ "prefix": "ov-test",+ }+++async def test_file_log_sink_message_key_falls_back_to_session_id(tmp_path, monkeypatch):+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")+ log_path = tmp_path / "usage.log"+ sink = FileLogUsageSink(path=log_path)++ try:+ await sink.write(events=[FakeUsageEvent(resource_uri="", session_id="session-fallback")])+ finally:+ sink.close()++ key, _value = _parse_line(log_path.read_text(encoding="utf-8").strip())+ assert key == "ov-test|2101858484|user-1|session-fallback"+++async def test_file_log_sink_preserves_equals_signs_in_key(tmp_path, monkeypatch):+ monkeypatch.setenv("OV_RESOURCE_ID", "ov=test")+ log_path = tmp_path / "usage.log"+ sink = FileLogUsageSink(path=log_path)++ try:+ await sink.write(+ events=[+ FakeUsageEvent(+ resource_uri=("viking://user/user-1/memories/experiences/exchange=delivered.md")+ )+ ]+ )+ finally:+ sink.close()++ key, value = _parse_line(log_path.read_text(encoding="utf-8").strip())+ assert key == (+ "ov=test|2101858484|user-1|viking://user/user-1/memories/experiences/exchange=delivered.md"+ )+ assert value["prefix"] == "ov=test"+ assert (+ value["tags"]["resource_uri"]+ == "viking://user/user-1/memories/experiences/exchange=delivered.md"+ )+++async def test_file_log_sink_preserves_records_when_workers_roll_over(tmp_path, monkeypatch):+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")+ log_path = tmp_path / "usage.log"+ first = FileLogUsageSink(path=log_path)+ second = FileLogUsageSink(path=log_path)++ try:+ await first.write(events=[FakeUsageEvent(event_id="ue_first_before")])+ await second.write(events=[FakeUsageEvent(event_id="ue_second_before")])++ first._handler.rolloverAt = 0+ second._handler.rolloverAt = 0+ await first.write(events=[FakeUsageEvent(event_id="ue_first_after")])+ await second.write(events=[FakeUsageEvent(event_id="ue_second_after")])+ finally:+ first.close()+ second.close()++ unique_ids = []+ for path in tmp_path.glob("usage.log*"):+ for line in path.read_text(encoding="utf-8").splitlines():+ _key, value = _parse_line(line)+ unique_ids.append(value["unique_id"])+ assert sorted(unique_ids) == [+ "ue_first_after",+ "ue_first_before",+ "ue_second_after",+ "ue_second_before",+ ]+++async def test_closed_handle_preserves_overdue_rollover_deadline(tmp_path, monkeypatch):+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")+ log_path = tmp_path / "usage.log"+ sink = FileLogUsageSink(path=log_path)++ try:+ await sink.write(events=[FakeUsageEvent(event_id="ue_before")])+ sink._handler.stream.close()+ sink._handler.stream = None+ sink._handler.rolloverAt = int(time.time()) - 1++ await sink.write(events=[FakeUsageEvent(event_id="ue_after")])+ finally:+ sink.close()++ log_files = list(tmp_path.glob("usage.log*"))+ assert len(log_files) == 2+ unique_ids = []+ for path in log_files:+ for line in path.read_text(encoding="utf-8").splitlines():+ _key, value = _parse_line(line)+ unique_ids.append(value["unique_id"])+ assert sorted(unique_ids) == ["ue_after", "ue_before"]+++def test_file_log_sink_uses_hourly_utc_rotation(tmp_path, monkeypatch):+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")+ sink = FileLogUsageSink(+ path=tmp_path / "usage.log",+ rotation_interval_hours=1,+ backup_count=168,+ )++ try:+ assert sink._handler.when == "H"+ assert sink._handler.interval == 60 * 60+ assert sink._handler.backupCount == 168+ assert sink._handler.utc is True+ assert sink._handler.suffix == "%Y-%m-%d_%H"+ finally:+ sink.close()+++ ("environment", "kwargs", "message"),+ [+ ({}, {"path": "usage.log"}, "OV_RESOURCE_ID is required"),+ ({"OV_RESOURCE_ID": "ov-test"}, {"path": ""}, "path is required"),+ (+ {"OV_RESOURCE_ID": "ov-test"},+ {"path": "usage.log", "rotation_interval_hours": 0},+ "rotation_interval_hours must be positive",+ ),+ (+ {"OV_RESOURCE_ID": "ov-test"},+ {"path": "usage.log", "backup_count": -1},+ "backup_count must be non-negative",+ ),+ ],+)+def test_file_log_sink_validates_config(+ monkeypatch,+ environment,+ kwargs,+ message,+):+ monkeypatch.delenv("OV_RESOURCE_ID", raising=False)+ for key, value in environment.items():+ monkeypatch.setenv(key, value)++ with pytest.raises(ValueError, match=message):+ FileLogUsageSink(**kwargs)+++async def test_file_log_sink_rejects_unsupported_event(tmp_path, monkeypatch):+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")+ sink = FileLogUsageSink(path=tmp_path / "usage.log")++ try:+ with pytest.raises(ValueError, match="unsupported usage event type"):+ await sink.write(events=[FakeUsageEvent(event_type="memory.unknown")])+ finally:+ sink.close()+++async def test_file_log_sink_requires_event_id(tmp_path, monkeypatch):+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")+ sink = FileLogUsageSink(path=tmp_path / "usage.log")++ try:+ with pytest.raises(ValueError, match="event_id is required"):+ await sink.write(events=[FakeUsageEvent(event_id="")])+ finally:+ sink.close()diff --git a/tests/unit/usage_reporter/test_http_sink.py b/tests/unit/usage_reporter/test_http_sink.pydeleted file mode 100644index 90449ec01..000000000--- a/tests/unit/usage_reporter/test_http_sink.py+++ /dev/null@@ -1,692 +0,0 @@-# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.-# SPDX-License-Identifier: AGPL-3.0--from __future__ import annotations--import asyncio-import hashlib-import json-import os-import tempfile-import threading-import time-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer-from pathlib import Path-from unittest import mock--import pytest--from openviking.usage_reporter.http_sink import HttpUsageSink---class FakeUsageEvent:- def __init__(- self,- event_id: str,- session_id: str,- attributes: dict[str, object] | None = None,- *,- event_type: str = "memory.recalled",- occurred_at: str = "2026-07-27T04:00:00Z",- resource_uri: str = "viking://user/user-1/memories/experiences/exchange.md",- resource_type: str = "experience",- task_id: str | None = None,- evidence: dict[str, object] | None = None,- ) -> None:- self._record = {- "schema_version": "v1",- "event_id": event_id,- "event_type": event_type,- "account_id": "2101858484",- "user_id": "user-1",- "session_id": session_id,- "task_id": task_id,- "occurred_at": occurred_at,- "resource_uri": resource_uri,- "resource_type": resource_type,- "evidence": evidence or {},- "attributes": attributes or {},- }-- def to_dict(self) -> dict[str, object]:- return dict(self._record)---class UsageRequestHandler(BaseHTTPRequestHandler):- def do_POST(self) -> None:- content_length = int(self.headers["Content-Length"])- body = json.loads(self.rfile.read(content_length))- self.server.requests.append(body)- self.server.resource_ids.append(self.headers["V-Resource-Id"])- response_status = self.server.response_statuses.pop(0)- self.send_response(response_status)- self.end_headers()- self.wfile.write(b"{}")-- def log_message(self, format: str, *args: object) -> None:- return---class UsageHTTPServer(ThreadingHTTPServer):- requests: list[dict[str, object]]- resource_ids: list[str]- response_statuses: list[int]---def wait_until(predicate: object, timeout: float = 3.0) -> None:- deadline = time.monotonic() + timeout- while time.monotonic() < deadline:- if predicate():- return- time.sleep(0.01)- raise AssertionError("condition was not met before timeout")---class TestHttpUsageSink:- def test_write_persists_recall_as_count_record(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(outbox_dir),- )- asyncio.run(- sink.write(- events=[- FakeUsageEvent(- "ue_recall",- "session-1",- {"rank": 1},- task_id="task-1",- evidence={- "archive_uri": (- "viking://user/user-1/sessions/session-1/"- "history/archive_001"- ),- "message_id": "msg-1",- "tool_call_id": "call-1",- "tool_name": "search_experience",- },- )- ]- )- )-- pending_path = next((outbox_dir / "pending").glob("*.json"))- payload = json.loads(pending_path.read_text(encoding="utf-8"))- assert payload["events"] == [- {- "CountName": "experience.recall.count",- "OpType": "add",- "amount": 1.0,- "timestamp": 1785124800000,- "uniqueId": "ue_recall",- "tags": {- "account_id": "2101858484",- "user_id": "user-1",- "resource_uri": ("viking://user/user-1/memories/experiences/exchange.md"),- "resource_type": "experience",- },- "extra": {- "session_id": "session-1",- "task_id": "task-1",- "archive_uri": (- "viking://user/user-1/sessions/session-1/history/archive_001"- ),- "message_id": "msg-1",- "tool_call_id": "call-1",- "tool_name": "search_experience",- "attributes": {"rank": 1},- },- }- ]-- def test_write_maps_injected_event_to_count_record(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(outbox_dir),- )- asyncio.run(- sink.write(- events=[- FakeUsageEvent(- "ue_inject",- "session-1",- event_type="memory.injected",- )- ]- )- )-- pending_path = next((outbox_dir / "pending").glob("*.json"))- payload = json.loads(pending_path.read_text(encoding="utf-8"))- assert payload["events"][0]["CountName"] == "experience.inject.count"-- def test_write_omits_empty_optional_count_record_extra_fields(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(outbox_dir),- )- asyncio.run(sink.write(events=[FakeUsageEvent("ue_minimal", "session-1")]))-- pending_path = next((outbox_dir / "pending").glob("*.json"))- payload = json.loads(pending_path.read_text(encoding="utf-8"))- assert payload["events"][0]["extra"] == {"session_id": "session-1"}-- def test_write_rejects_unknown_usage_event_type(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(Path(data_dir) / ".usage_outbox"),- )- with pytest.raises(- ValueError,- match="unsupported usage event type: memory.unknown",- ):- asyncio.run(- sink.write(- events=[- FakeUsageEvent(- "ue_unknown",- "session-1",- event_type="memory.unknown",- )- ]- )- )-- def test_write_atomically_persists_stable_batch(self) -> None:- events = [- FakeUsageEvent("ue_1", "session-1"),- FakeUsageEvent("ue_2", "session-1"),- ]- expected_digest = hashlib.sha256(b"ue_1\nue_2").hexdigest()-- with tempfile.TemporaryDirectory() as data_dir:- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/api/openviking/ReportOpenVikingUsage",- outbox_dir=str(Path(data_dir) / ".usage_outbox"),- )- asyncio.run(sink.write(events=events))-- pending_files = list((Path(data_dir) / ".usage_outbox" / "pending").glob("*.json"))- temporary_files = list((Path(data_dir) / ".usage_outbox" / "pending").glob("*.tmp"))-- assert len(pending_files) == 1- assert temporary_files == []- payload = json.loads(pending_files[0].read_text(encoding="utf-8"))- assert payload["schema_version"] == "v1"- assert payload["resource_id"] == "ov-test"- assert payload["batch_id"] == f"ub_{expected_digest}"- assert [event["uniqueId"] for event in payload["events"]] == [- "ue_1",- "ue_2",- ]- assert all("prefix" not in event for event in payload["events"])-- def test_write_splits_batches_before_http_limit(self) -> None:- events = [- FakeUsageEvent("ue_large_1", "session-large", {"text": "a" * 700}),- FakeUsageEvent("ue_large_2", "session-large", {"text": "b" * 700}),- ]-- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(outbox_dir),- max_batch_bytes=1024,- )- asyncio.run(sink.write(events=events))-- batches = [- json.loads(path.read_text(encoding="utf-8"))- for path in (outbox_dir / "pending").glob("*.json")- ]- assert len(batches) == 2- assert sorted(len(batch["events"]) for batch in batches) == [1, 1]-- def test_worker_retries_then_deletes_acknowledged_batch(self) -> None:- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)- server.requests = []- server.resource_ids = []- server.response_statuses = [503, 200]- server_thread = threading.Thread(target=server.serve_forever, daemon=True)- server_thread.start()-- try:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- sink = HttpUsageSink(- endpoint=f"http://127.0.0.1:{server.server_port}/usage",- outbox_dir=str(outbox_dir),- request_timeout_seconds=0.5,- retry_base_seconds=0.01,- retry_max_seconds=0.02,- )- asyncio.run(sink.write(events=[FakeUsageEvent("ue_retry", "session-retry")]))- wait_until(lambda: len(server.requests) == 2)- wait_until(- lambda: (- not list((outbox_dir / "pending").glob("*.json"))- and not list((outbox_dir / "inflight").glob("*.json"))- )- )- sink.close()-- assert [request["batch_id"] for request in server.requests] == [- server.requests[0]["batch_id"],- server.requests[0]["batch_id"],- ]- assert server.resource_ids == ["ov-test", "ov-test"]- finally:- server.shutdown()- server.server_close()- server_thread.join(timeout=1)-- def test_worker_accepts_any_successful_http_status(self) -> None:- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)- server.requests = []- server.resource_ids = []- server.response_statuses = [204]- server_thread = threading.Thread(target=server.serve_forever, daemon=True)- server_thread.start()-- try:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- sink = HttpUsageSink(- endpoint=f"http://127.0.0.1:{server.server_port}/usage",- outbox_dir=str(outbox_dir),- request_timeout_seconds=0.5,- )- asyncio.run(sink.write(events=[FakeUsageEvent("ue_204", "session-success")]))- wait_until(lambda: len(server.requests) == 1)- wait_until(- lambda: (- not list((outbox_dir / "pending").glob("*.json"))- and not list((outbox_dir / "inflight").glob("*.json"))- )- )- sink.close()- finally:- server.shutdown()- server.server_close()- server_thread.join(timeout=1)-- def test_outbox_capacity_evicts_oldest_pending_batches(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(outbox_dir),- max_outbox_bytes=1800,- )- for index in range(4):- asyncio.run(- sink.write(- events=[- FakeUsageEvent(- f"ue_capacity_{index}",- "session-capacity",- {"text": str(index) * 600},- )- ]- )- )-- pending_files = list((outbox_dir / "pending").glob("*.json"))- total_bytes = sum(path.stat().st_size for path in pending_files)- persisted_ids = {- event["uniqueId"]- for path in pending_files- for event in json.loads(path.read_text(encoding="utf-8"))["events"]- }- assert total_bytes <= 1800- assert "ue_capacity_0" not in persisted_ids- assert "ue_capacity_3" in persisted_ids-- def test_outbox_capacity_counts_inflight_without_evicting_it(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(outbox_dir),- max_outbox_bytes=1800,- )- inflight_path = outbox_dir / "inflight" / f"{sink._outbox_scope}_active.json"- inflight_path.write_bytes(b"x" * 1000)- asyncio.run(- sink.write(- events=[- FakeUsageEvent(- "ue_while_inflight",- "session-capacity",- {"text": "x" * 900},- )- ]- )- )-- assert inflight_path.exists()- assert list((outbox_dir / "pending").glob("*.json")) == []-- def test_default_outbox_directory_uses_current_user_home(self) -> None:- with tempfile.TemporaryDirectory() as home_dir:- expected = Path(home_dir) / ".openviking" / "data" / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(Path, "home", return_value=Path(home_dir)):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(endpoint="http://127.0.0.1:1/usage")-- assert sink._outbox_dir == expected- assert expected.is_dir()-- def test_shared_outbox_root_isolated_by_endpoint_and_resource(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- outbox_root = Path(data_dir) / ".usage_outbox"- with mock.patch.object(HttpUsageSink, "_start_worker"):- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "resource-a"}, clear=False):- sink_a = HttpUsageSink(- endpoint="http://collector-a.example/usage",- outbox_dir=str(outbox_root),- )- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "resource-b"}, clear=False):- other_resource_sink = HttpUsageSink(- endpoint="http://collector-a.example/usage",- outbox_dir=str(outbox_root),- )- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "resource-a"}, clear=False):- other_endpoint_sink = HttpUsageSink(- endpoint="http://collector-b.example/usage",- outbox_dir=str(outbox_root),- )-- asyncio.run(sink_a.write(events=[FakeUsageEvent("ue_a", "session-a")]))- sink_a._post_payload = mock.Mock(return_value=200)- other_resource_sink._post_payload = mock.Mock(return_value=200)- other_endpoint_sink._post_payload = mock.Mock(return_value=200)-- assert other_resource_sink._handle_next_batch() is False- other_resource_sink._post_payload.assert_not_called()- assert other_endpoint_sink._handle_next_batch() is False- other_endpoint_sink._post_payload.assert_not_called()- assert sink_a._handle_next_batch() is True- sink_a._post_payload.assert_called_once()-- def test_worker_splits_payload_rejected_as_too_large(self) -> None:- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)- server.requests = []- server.resource_ids = []- server.response_statuses = [413, 200, 200]- server_thread = threading.Thread(target=server.serve_forever, daemon=True)- server_thread.start()-- try:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- sink = HttpUsageSink(- endpoint=f"http://127.0.0.1:{server.server_port}/usage",- outbox_dir=str(outbox_dir),- request_timeout_seconds=0.5,- )- asyncio.run(- sink.write(- events=[- FakeUsageEvent("ue_left", "session-split"),- FakeUsageEvent("ue_right", "session-split"),- ]- )- )- wait_until(lambda: len(server.requests) == 3)- wait_until(- lambda: (- not list((outbox_dir / "pending").glob("*.json"))- and not list((outbox_dir / "inflight").glob("*.json"))- )- )- sink.close()-- original_batch_id = server.requests[0]["batch_id"]- child_batch_ids = {request["batch_id"] for request in server.requests[1:]}- assert [len(request["events"]) for request in server.requests] == [2, 1, 1]- assert original_batch_id not in child_batch_ids- assert len(child_batch_ids) == 2- finally:- server.shutdown()- server.server_close()- server_thread.join(timeout=1)-- def test_split_replaces_parent_without_evicting_child_batches(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- events = [- FakeUsageEvent("ue_split_left", "session-split", {"text": "a" * 500}),- FakeUsageEvent("ue_split_right", "session-split", {"text": "b" * 500}),- ]- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(outbox_dir),- max_outbox_bytes=10_000,- )- asyncio.run(sink.write(events=events))-- parent_path = next((outbox_dir / "pending").glob("*.json"))- payload = json.loads(parent_path.read_text(encoding="utf-8"))- inflight_path = outbox_dir / "inflight" / parent_path.name- os.replace(parent_path, inflight_path)- child_size = sum(- sink._encoded_payload_size([event]) for event in payload["events"]- )- sink._max_outbox_bytes = child_size + 50-- sink._split_batch(inflight_path, payload)-- children = [- json.loads(path.read_text(encoding="utf-8"))- for path in (outbox_dir / "pending").glob("*.json")- ]- assert not inflight_path.exists()- assert len(children) == 2- assert {event["uniqueId"] for child in children for event in child["events"]} == {- "ue_split_left",- "ue_split_right",- }-- def test_worker_moves_unrecoverable_payload_to_dead_letter(self) -> None:- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)- server.requests = []- server.resource_ids = []- server.response_statuses = [422]- server_thread = threading.Thread(target=server.serve_forever, daemon=True)- server_thread.start()-- try:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- sink = HttpUsageSink(- endpoint=f"http://127.0.0.1:{server.server_port}/usage",- outbox_dir=str(outbox_dir),- )- asyncio.run(sink.write(events=[FakeUsageEvent("ue_bad", "session-bad")]))- wait_until(lambda: len(list((outbox_dir / "dead_letter").glob("*.json"))) == 1)- sink.close()-- assert len(server.requests) == 1- assert not list((outbox_dir / "pending").glob("*.json"))- finally:- server.shutdown()- server.server_close()- server_thread.join(timeout=1)-- def test_startup_recovers_only_stale_inflight_batch(self) -> None:- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)- server.requests = []- server.resource_ids = []- server.response_statuses = [200]- server_thread = threading.Thread(target=server.serve_forever, daemon=True)- server_thread.start()-- try:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- first_sink = HttpUsageSink(- endpoint=f"http://127.0.0.1:{server.server_port}/usage",- outbox_dir=str(outbox_dir),- )- asyncio.run(- first_sink.write(events=[FakeUsageEvent("ue_stale", "session-stale")])- )- pending_path = next((outbox_dir / "pending").glob("*.json"))- inflight_path = outbox_dir / "inflight" / pending_path.name- os.replace(pending_path, inflight_path)- stale_time = time.time() - 10- os.utime(inflight_path, (stale_time, stale_time))-- second_sink = HttpUsageSink(- endpoint=f"http://127.0.0.1:{server.server_port}/usage",- outbox_dir=str(outbox_dir),- request_timeout_seconds=0.5,- inflight_lease_seconds=1,- )- wait_until(lambda: len(server.requests) == 1)- wait_until(lambda: not inflight_path.exists())- second_sink.close()- finally:- server.shutdown()- server.server_close()- server_thread.join(timeout=1)-- def test_worker_recovers_inflight_batch_when_lease_expires_after_startup(self) -> None:- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)- server.requests = []- server.resource_ids = []- server.response_statuses = [200]- server_thread = threading.Thread(target=server.serve_forever, daemon=True)- server_thread.start()-- try:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- first_sink = HttpUsageSink(- endpoint=f"http://127.0.0.1:{server.server_port}/usage",- outbox_dir=str(outbox_dir),- )- asyncio.run(- first_sink.write(events=[FakeUsageEvent("ue_fresh", "session-fresh")])- )- pending_path = next((outbox_dir / "pending").glob("*.json"))- inflight_path = outbox_dir / "inflight" / pending_path.name- os.replace(pending_path, inflight_path)-- second_sink = HttpUsageSink(- endpoint=f"http://127.0.0.1:{server.server_port}/usage",- outbox_dir=str(outbox_dir),- request_timeout_seconds=0.01,- inflight_lease_seconds=0.05,- )- wait_until(lambda: len(server.requests) == 1)- wait_until(lambda: not inflight_path.exists())- second_sink.close()- finally:- server.shutdown()- server.server_close()- server_thread.join(timeout=1)-- def test_worker_keeps_batch_after_unexpected_delivery_exception(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(outbox_dir),- retry_base_seconds=0.01,- retry_max_seconds=0.02,- )- attempts = 0-- def flaky_post(payload: dict[str, object]) -> int:- nonlocal attempts- attempts += 1- if attempts == 1:- raise RuntimeError("unexpected delivery failure")- return 200-- sink._post_payload = flaky_post- with mock.patch(- "openviking.usage_reporter.http_sink.logger.exception"- ) as log_exception:- asyncio.run(- sink.write(events=[FakeUsageEvent("ue_exception", "session-exception")])- )- wait_until(lambda: attempts == 2)- wait_until(- lambda: (- not list((outbox_dir / "pending").glob("*.json"))- and not list((outbox_dir / "inflight").glob("*.json"))- )- )- sink.close()- assert log_exception.call_count == 1- assert (- "usage delivery failed unexpectedly"- in log_exception.call_args.args[0].lower()- )-- def test_claiming_pending_batch_refreshes_inflight_lease(self) -> None:- with tempfile.TemporaryDirectory() as data_dir:- outbox_dir = Path(data_dir) / ".usage_outbox"- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):- with mock.patch.object(HttpUsageSink, "_start_worker"):- sink = HttpUsageSink(- endpoint="http://127.0.0.1:1/usage",- outbox_dir=str(outbox_dir),- request_timeout_seconds=0.01,- inflight_lease_seconds=0.05,- )- asyncio.run(sink.write(events=[FakeUsageEvent("ue_claim", "session-claim")]))- pending_path = next((outbox_dir / "pending").glob("*.json"))- stale_time = time.time() - 60- os.utime(pending_path, (stale_time, stale_time))- claimed_mtime = 0.0-- def record_claim(_payload: dict[str, object]) -> int:- nonlocal claimed_mtime- inflight_path = outbox_dir / "inflight" / pending_path.name- claimed_mtime = inflight_path.stat().st_mtime- return 200-- sink._post_payload = record_claim- assert sink._handle_next_batch() is True-- assert claimed_mtime >= time.time() - 1diff --git a/tests/unit/usage_reporter/test_usage_reporter_config.py b/tests/unit/usage_reporter/test_usage_reporter_config.pyindex 8b631407e..49f13fcff 100644--- a/tests/unit/usage_reporter/test_usage_reporter_config.py+++ b/tests/unit/usage_reporter/test_usage_reporter_config.py@@ -9,9 +9,8 @@ import pytestfrom openviking.server.app import create_appfrom openviking.server.config import ServerConfig, UsageReporterConfig-from openviking.usage_reporter import UsageContext, UsageEvent+from openviking.usage_reporter import FileLogUsageSink, UsageContext, UsageEventfrom openviking.usage_reporter.config import build_usage_reporter-from openviking.usage_reporter.http_sink import HttpUsageSink@pytest.mark.asyncio@@ -80,8 +79,9 @@ class CustomUsageSink:assert "source" not in payload-def test_builtin_http_sink_is_built_from_config(tmp_path, monkeypatch):+def test_builtin_file_log_sink_is_built_from_config(tmp_path, monkeypatch):monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")+ log_path = tmp_path / "usage" / "usage.log"reporter = build_usage_reporter(UsageReporterConfig(@@ -89,10 +89,10 @@ def test_builtin_http_sink_is_built_from_config(tmp_path, monkeypatch):extractors=["memory_usage"],sinks=[{- "type": "http",+ "type": "file_log","config": {- "endpoint": "http://127.0.0.1:1/usage",- "outbox_dir": str(tmp_path / ".usage_outbox"),+ "path": str(log_path),+ "rotation_interval_hours": 1,},}],@@ -101,7 +101,8 @@ def test_builtin_http_sink_is_built_from_config(tmp_path, monkeypatch):assert reporter is not Noneassert len(reporter.sinks) == 1- assert isinstance(reporter.sinks[0], HttpUsageSink)+ assert isinstance(reporter.sinks[0], FileLogUsageSink)+ assert reporter.sinks[0].path == log_pathreporter.sinks[0].close()