Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

feat: live-reload Agent Evolution and add file usage sink (#3573)

agent0ec2bb0ec528
diff --git a/docs/design/openviking-usage-count-record-implementation-plan.md b/docs/design/openviking-usage-count-record-implementation-plan.md
index d0f96894d..00f24ad5d 100644
--- a/docs/design/openviking-usage-count-record-implementation-plan.md
+++ b/docs/design/openviking-usage-count-record-implementation-plan.md
@@ -1,228 +1,100 @@
-# HTTP Usage CountRecord Implementation Plan
+# File Usage CountRecord Implementation Plan
-> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+**Goal:** 将 Usage Reporter 生成的 `UsageEvent` 转换为稳定的
+`CountRecord` 日志协议,供部署侧日志采集系统读取并投递到下游。
-**Goal:** 将内置 HTTP Usage Sink 上报的事件元素改为 `CountRecord`,同时保持内部 `UsageEvent`、自定义 Sink 和 outbox 可靠性语义不变。
+**Architecture:** `MemoryUsageExtractor` 继续生成内部 `UsageEvent`。
+`FileLogUsageSink` 在写入专用日志文件前执行单向转换,每行保存一个包含 Kafka
+message key 和对应 `CountRecord` 的 JSON envelope。`unique_id` 作为稳定事件标识,
+供下游在 best-effort 投递发生重复时去重。
-**Architecture:** `MemoryUsageExtractor` 继续生成 `UsageEvent`。`HttpUsageSink` 在持久化批次前执行单向转换,HTTP 批次中的 `events` 改为 `CountRecord`;`uniqueId` 继续承担重放去重键,并用于计算稳定 `batch_id`。
-
-**Tech Stack:** Python 3.10+、dataclasses、标准库 datetime/json、pytest、Ruff。
+**Tech Stack:** Python 3.10+、dataclasses、标准库
+`datetime` / `json` / `logging`、pytest、Ruff。
---
## 文件结构
-- Modify: `openviking/usage_reporter/http_sink.py`
+- `openviking/usage_reporter/file_log_sink.py`
- 定义 `UsageEvent -> CountRecord` 私有转换。
- - 调整批次 ID 从 `uniqueId` 计算。
-- Modify: `tests/unit/usage_reporter/test_http_sink.py`
+ - 构造资源归属 message key。
+ - 将记录追加到按 UTC 小时滚动的专用日志文件。
+- `tests/unit/usage_reporter/test_file_log_sink.py`
- 固定 recall/inject 字段映射、毫秒时间戳、扩展字段和未知事件行为。
-- Modify: `docs/design/openviking-usage-reporter-sink-design.md`
- - 补充 `attributes -> extra.attributes` 的无损映射说明。
-
-### Task 1: 固定 CountRecord 映射契约
-
-**Files:**
-- Test: `tests/unit/usage_reporter/test_http_sink.py`
-
-- [ ] **Step 1: 扩展测试事件构造器**
-
-让 `FakeUsageEvent` 支持 `event_type`、`occurred_at`、`resource_uri`、
-`resource_type`、`task_id`、`evidence` 和 `attributes`,并继续通过
-`to_dict()` 模拟真实 `UsageEvent`。
-
-- [ ] **Step 2: 写 recall 映射失败测试**
-
-构造 `memory.recalled` 事件并调用 `sink.write()`,读取 pending JSON,断言:
-
-```python
-assert payload["events"] == [
- {
- "CountName": "experience.recall.count",
- "OpType": "add",
- "amount": 1.0,
- "timestamp": 1785124800000,
- "uniqueId": "ue_recall",
- "tags": {
- "account_id": "2101858484",
- "user_id": "user-1",
- "resource_uri": "viking://user/user-1/memories/experiences/exchange.md",
- "resource_type": "experience",
- },
- "extra": {
- "session_id": "session-1",
- "task_id": "task-1",
- "archive_uri": "viking://user/user-1/sessions/session-1/history/archive_001",
- "message_id": "msg-1",
- "tool_call_id": "call-1",
- "tool_name": "search_experience",
- "attributes": {"rank": 1},
- },
+ - 验证多 worker 共享文件时的写入和滚动行为。
+- `docs/design/openviking-usage-reporter-sink-design.md`
+ - 定义 Usage Reporter、Sink 扩展点和文件日志协议。
+
+## CountRecord 映射契约
+
+`memory.recalled` 事件映射为:
+
+```json
+{
+ "count_name": "experience.recall.count",
+ "op_type": "add",
+ "amount": 1.0,
+ "timestamp": 1785124800000,
+ "unique_id": "ue_recall",
+ "tags": {
+ "account_id": "2101858484",
+ "user_id": "user-1",
+ "resource_uri": "viking://user/user-1/memories/experiences/exchange.md",
+ "resource_type": "experience"
+ },
+ "extra": {
+ "session_id": "session-1",
+ "task_id": "task-1",
+ "archive_uri": "viking://user/user-1/sessions/session-1/history/archive_001",
+ "message_id": "msg-1",
+ "tool_call_id": "call-1",
+ "tool_name": "search_experience",
+ "attributes": {
+ "rank": 1
}
-]
-```
-
-- [ ] **Step 3: 运行测试并确认 RED**
-
-Run:
-
-```bash
-uv run pytest -q --no-cov tests/unit/usage_reporter/test_http_sink.py::TestHttpUsageSink::test_write_persists_recall_as_count_record
-```
-
-Expected: FAIL,当前 payload 仍包含 `event_type` 等 `UsageEvent` 字段。
-
-- [ ] **Step 4: 写 inject、空字段和未知事件失败测试**
-
-分别断言:
-
-- `memory.injected -> experience.inject.count`
-- `task_id=None`、空 `evidence`、空 `attributes` 不产生对应 `extra` 字段
-- 未知 `event_type` 在 `sink.write()` 时抛出 `ValueError`
-
-- [ ] **Step 5: 运行新增测试并确认 RED**
-
-Run:
-
-```bash
-uv run pytest -q --no-cov tests/unit/usage_reporter/test_http_sink.py -k "count_record or unknown_event_type"
-```
-
-Expected: FAIL,CountRecord 转换尚不存在。
-
-### Task 2: 实现 HTTP 边界转换
-
-**Files:**
-- Modify: `openviking/usage_reporter/http_sink.py`
-
-- [ ] **Step 1: 增加事件类型映射和毫秒时间转换**
-
-```python
-_COUNT_NAMES = {
- "memory.recalled": "experience.recall.count",
- "memory.injected": "experience.inject.count",
+ },
+ "prefix": "ov-resource-id"
}
-
-
-def _timestamp_millis(value: str) -> int:
- parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
- if parsed.tzinfo is None:
- parsed = parsed.replace(tzinfo=timezone.utc)
- return int(parsed.timestamp() * 1000)
-```
-
-- [ ] **Step 2: 增加 CountRecord 转换**
-
-```python
-def _to_count_record(event: UsageEvent) -> dict[str, Any]:
- record = event.to_dict()
- event_type = str(record.get("event_type") or "")
- try:
- count_name = _COUNT_NAMES[event_type]
- except KeyError as exc:
- raise ValueError(f"unsupported usage event type: {event_type}") from exc
-
- evidence = record.get("evidence")
- evidence = evidence if isinstance(evidence, dict) else {}
- attributes = record.get("attributes")
- attributes = attributes if isinstance(attributes, dict) else {}
- extra = {
- "session_id": str(record.get("session_id") or ""),
- **{key: value for key, value in evidence.items() if value not in (None, "")},
- }
- if record.get("task_id") not in (None, ""):
- extra["task_id"] = record["task_id"]
- if attributes:
- extra["attributes"] = attributes
-
- return {
- "CountName": count_name,
- "OpType": "add",
- "amount": 1.0,
- "timestamp": _timestamp_millis(str(record["occurred_at"])),
- "uniqueId": str(record["event_id"]),
- "tags": {
- "account_id": str(record.get("account_id") or ""),
- "user_id": str(record.get("user_id") or ""),
- "resource_uri": str(record.get("resource_uri") or ""),
- "resource_type": str(record.get("resource_type") or ""),
- },
- "extra": extra,
- }
```
-- [ ] **Step 3: 在持久化入口使用转换**
-
-`_persist_events()` 先校验 `event_id`,再调用 `_to_count_record()`。移除旧的
-逐事件 `prefix` 字段;资源归属继续由批次 `resource_id` 和
-`V-Resource-Id` header 表达。
-
-- [ ] **Step 4: 改用 uniqueId 生成 batch_id**
-
-```python
-unique_ids = "\n".join(str(event["uniqueId"]) for event in events)
-batch_id = f"ub_{hashlib.sha256(unique_ids.encode('utf-8')).hexdigest()}"
-```
-
-- [ ] **Step 5: 运行新增测试并确认 GREEN**
-
-Run:
-
-```bash
-uv run pytest -q --no-cov tests/unit/usage_reporter/test_http_sink.py -k "count_record or unknown_event_type"
-```
-
-Expected: PASS。
-
-### Task 3: 回归可靠性与文档
-
-**Files:**
-- Modify: `docs/design/openviking-usage-reporter-sink-design.md`
-- Test: `tests/unit/usage_reporter/test_http_sink.py`
+映射规则:
-- [ ] **Step 1: 补充 attributes 映射说明**
+- `memory.recalled` 映射为
+ `count_name=experience.recall.count`。
+- `memory.injected` 映射为
+ `count_name=experience.inject.count`。
+- `op_type` 固定为 `add`,`amount` 固定为 `1.0`。
+- `occurred_at` 转换为毫秒时间戳。
+- `event_id` 写入 `unique_id`,为空时拒绝写入。
+- 非空 `UsageEvent.attributes` 写入 `extra.attributes`。
+- `resource_id_env` 指定的环境变量写入 `prefix`。
+- 未知 `event_type` 拒绝写入,避免产生无法解释的计量记录。
-在 CountRecord 字段映射中明确:非空 `UsageEvent.attributes` 写入
-`extra.attributes`,避免事件类型扩展字段丢失。
+## 文件日志协议
-- [ ] **Step 2: 运行 HTTP Sink 全量测试**
+日志行格式:
-Run:
-
-```bash
-uv run pytest -q --no-cov --tb=short tests/unit/usage_reporter/test_http_sink.py
+```text
+{"key":"<resource_id>|<account_id>|<user_id>|<resource_uri>","value":<CountRecord JSON>}
```
-Expected: 全部 PASS,覆盖 outbox、重试、拆包、dead-letter 和 destination
-隔离。
+当 `resource_uri` 为空时,message key 的最后一段使用 `session_id`。
+整行使用 JSON envelope,key 中的 `=`、空格或其他字符不会影响 key/value 拆分。
+日志文件不复用 OpenViking stdout,按 UTC 小时滚动,并保留配置数量的历史
+文件。多个 server worker 写入同一路径时,文件追加和滚动通过进程间锁串行化。
-- [ ] **Step 3: 运行 Usage Reporter 回归**
+文件落盘及后续采集均采用 best-effort 语义。下游必须按
+`CountRecord.unique_id` 去重。
-Run:
+## 验证
```bash
-uv run pytest -q --no-cov --tb=short tests/unit/usage_reporter
-```
-
-Expected: 全部 PASS。
-
-- [ ] **Step 4: 运行静态检查**
-
-Run:
-
-```bash
-uv run ruff check openviking/usage_reporter/http_sink.py tests/unit/usage_reporter/test_http_sink.py
-uv run ruff format --check openviking/usage_reporter/http_sink.py tests/unit/usage_reporter/test_http_sink.py
+uv run pytest -q --no-cov tests/unit/usage_reporter
+uv run ruff check \
+ openviking/usage_reporter/file_log_sink.py \
+ tests/unit/usage_reporter/test_file_log_sink.py
+uv run ruff format --check \
+ openviking/usage_reporter/file_log_sink.py \
+ tests/unit/usage_reporter/test_file_log_sink.py
git diff --check
```
-
-Expected: 全部通过。
-
-- [ ] **Step 5: 提交实现**
-
-```bash
-git add openviking/usage_reporter/http_sink.py \
- tests/unit/usage_reporter/test_http_sink.py \
- docs/design/openviking-usage-reporter-sink-design.md
-git commit -m "feat(usage): emit CountRecord over HTTP"
-```
diff --git a/docs/design/openviking-usage-reporter-sink-design.md b/docs/design/openviking-usage-reporter-sink-design.md
index 9b7e3fb16..c9ba42c35 100644
--- a/docs/design/openviking-usage-reporter-sink-design.md
+++ b/docs/design/openviking-usage-reporter-sink-design.md
@@ -100,7 +100,7 @@ class UsageSink:
...
```
-具体 Sink 可以作为外部扩展通过 `class_path` 动态加载。开源包同时提供不依赖第三方消息队列 SDK 的内置 HTTP Sink,用于需要本地持久化重试的部署。
+具体 Sink 可以作为外部扩展通过 `class_path` 动态加载。开源包同时提供不依赖第三方消息队列 SDK 的内置文件日志 Sink,供日志采集系统读取。
配置示例:
@@ -130,53 +130,40 @@ def load_class(class_path: str):
每个 Sink 的 `write()` 调用最多等待 5 秒。超时或异常只记录日志,不影响其他 Sink。Reporter 在应用生命周期内只创建一次,应用退出时调用 Sink 可选的 `close()` 方法。同步和异步 `close()` 均受同一超时限制;同步 hook 在独立 daemon 线程中执行,超时后不会阻塞事件循环、后续 Sink 清理或进程退出。
-内置 HTTP Sink 在 `write()` 返回前把事件批次写入本地 outbox,再由后台线程发送。它只依赖 Python 标准库,不引入 Kafka 等厂商依赖。HTTP Sink 支持指数退避、`413` 拆包、不可恢复批次转入 dead-letter,以及有界磁盘容量。
+内置文件日志 Sink 将事件立即追加到专用日志文件,不写默认 stdout,也不发起
+HTTP 请求。日志文件使用 UTC 小时滚动,默认保留 168 个小时文件。部署侧应将
+专用目录挂载到日志采集系统可见的宿主机路径。多个 server worker 写入同一路径
+时,通过进程间文件锁串行化写入和滚动;Windows worker 写入后主动关闭文件句柄,
+避免其他进程滚动重命名失败。
-### 6.1 HTTP CountRecord 协议
+### 6.1 文件日志 CountRecord 协议
`UsageEvent` 继续作为 OpenViking 内部抽取结果和自定义 Sink 的稳定协议。内置
-HTTP Sink 在写入 outbox 前,将 `UsageEvent` 转换为计量接收端使用的
-`CountRecord`。这样计量平台协议不会侵入 Extractor,也不会改变自定义 Sink
-的输入。
+文件日志 Sink 将 `UsageEvent` 转换为计量接收端使用的 `CountRecord`。每个事件
+写成一行包含 Kafka key/value 的 JSON envelope:
-HTTP 批次外层继续保留 `schema_version`、`resource_id`、`batch_id`、
-`created_at`、`attempt`、`next_retry_at` 和 `events`。`events` 数组中的元素
-采用以下格式:
-
-```json
-{
- "CountName": "experience.recall.count",
- "OpType": "add",
- "amount": 1.0,
- "timestamp": 1785124800000,
- "uniqueId": "ue_<sha256>",
- "tags": {
- "account_id": "new",
- "user_id": "test",
- "resource_uri": "viking://user/test/memories/experiences/example.md",
- "resource_type": "experience"
- },
- "extra": {
- "session_id": "session-id",
- "task_id": "task-id",
- "archive_uri": "viking://user/test/sessions/session-id/history/archive_001",
- "message_id": "message-id",
- "tool_call_id": "tool-call-id",
- "tool_name": "search_experience"
- }
-}
+```text
+{"key":"ov-xxx|new|test|viking://user/test/memories/experiences/example.md","value":{"count_name":"experience.recall.count","op_type":"add","amount":1.0,"timestamp":1785124800000,"unique_id":"ue_<sha256>","tags":{"account_id":"new","user_id":"test","resource_uri":"viking://user/test/memories/experiences/example.md","resource_type":"experience"},"extra":{"session_id":"session-id","task_id":"task-id","archive_uri":"viking://user/test/sessions/session-id/history/archive_001","message_id":"message-id","tool_call_id":"tool-call-id","tool_name":"search_experience"},"prefix":"ov-xxx"}}
```
字段映射:
-- `memory.recalled` 映射为 `CountName=experience.recall.count`。
-- `memory.injected` 映射为 `CountName=experience.inject.count`。
-- `OpType` 固定为 `add`。
+- envelope 的 `key` 与原 Kafka message key 一致,格式为
+ `resource_id|account_id|user_id|resource_uri`;`resource_uri` 为空时回退
+ 到 `session_id`。
+- envelope 的 `value` 是原 Kafka message value 的完整 JSON,不拆分内部字段。
+- 整行使用 JSON envelope,key 中的分隔符不会影响 key/value 解析。
+- JSON 中的 `prefix` 为 OpenViking 实例的 resource ID。
+- `memory.recalled` 映射为 `count_name=experience.recall.count`。
+- `memory.injected` 映射为 `count_name=experience.inject.count`。
+- `op_type` 固定为 `add`。
- `amount` 固定为 `1.0`。
- `timestamp` 由 `occurred_at` 转换为 Unix 毫秒时间戳。
-- `uniqueId` 使用稳定的 `event_id`。
-- `tags` 保存 `account_id`、`user_id`、`resource_uri` 和 `resource_type`。
-- `extra` 保存 `session_id`、可选的 `task_id` 以及 `evidence` 中的审计字段。
+- JSON 中的 `unique_id` 使用稳定的 `event_id`。
+- JSON 中的 `tags` 保存 `account_id`、`user_id`、`resource_uri` 和
+ `resource_type`。
+- JSON 中的 `extra` 保存 `session_id`、可选的 `task_id` 以及 `evidence`
+ 中的审计字段。
- 非空的 `UsageEvent.attributes` 原样写入 `extra.attributes`。
- 空的可选字段不写入 `extra`。
@@ -206,19 +193,19 @@ server:
endpoint: https://usage.example.com/events
```
-内置 HTTP Sink:
+内置文件日志 Sink:
```yaml
server:
usage_reporter:
enabled: true
sinks:
- - type: http
+ - type: file_log
config:
- endpoint: https://usage.example.com/events
+ path: /var/log/openviking_usage/usage.log
resource_id_env: OV_RESOURCE_ID
- outbox_dir: /var/lib/openviking/.usage_outbox
- max_outbox_bytes: 268435456
+ rotation_interval_hours: 1
+ backup_count: 168
```
## 8. 对 OpenViking 的侵入
@@ -289,6 +276,6 @@ schema_version
任务恢复或附加属性变化破坏幂等性。同一 session message 中同一 tool call
对同一资源产生的事件,在 phase2 重放后仍得到相同 `event_id`。
-内置 HTTP Sink 提供持久化 outbox、失败重试和发送确认。所有 `2xx` 响应视为确认,瞬时错误进入指数退避重试,`400` / `422` 进入 dead-letter。HTTP Sink 可能重复发送同一事件,消费端需按由内部 `event_id` 映射得到的 `CountRecord.uniqueId` 去重。
-
-HTTP outbox 使用 `max_outbox_bytes` 限制磁盘占用。达到上限时先淘汰最旧的 dead-letter,再淘汰最旧的 pending;inflight 不被淘汰。因此即使启用 HTTP Sink,长期下游故障和容量压力下仍可能丢失事件,整体机制保持 best-effort,不承诺端到端 at-least-once。
+内置文件日志 Sink 在 `write()` 返回前完成本地追加,但不负责 TLS 采集、Kafka
+投递或下游确认。文件写入、TLS 采集和下游消费任一阶段都可能在故障时产生丢失
+或重复,消费端需按 `unique_id` 去重,整体保持 best-effort 语义。
diff --git a/docs/en/api/01-overview.md b/docs/en/api/01-overview.md
index 56f0d850d..074bb50bd 100644
--- a/docs/en/api/01-overview.md
+++ b/docs/en/api/01-overview.md
@@ -532,6 +532,7 @@ This catalog follows the routes actually mounted by the server. Each group headi
| Method | Path | Description |
|--------|------|-------------|
+| GET | `/api/v1/admin/agent-evolution` | Get the live instance-wide Agent Evolution status |
| POST | `/api/v1/admin/accounts` | Create an account and its first administrator |
| GET | `/api/v1/admin/accounts` | List accounts |
| POST | `/api/v1/admin/migrate` | Migrate legacy identity data |
diff --git a/docs/en/api/08-admin.md b/docs/en/api/08-admin.md
index c81133415..02c46f085 100644
--- a/docs/en/api/08-admin.md
+++ b/docs/en/api/08-admin.md
@@ -57,6 +57,40 @@ Configure `root_api_key` in `~/.openviking/ovcli.conf`:
## API Reference
+### get_agent_evolution_status
+
+Return the live instance-wide Agent Evolution switch and the configured default
+account ID. The endpoint is ROOT-only.
+
+**HTTP API**
+
+```
+GET /api/v1/admin/agent-evolution
+```
+
+```bash
+curl http://localhost:1933/api/v1/admin/agent-evolution \
+ -H "X-API-Key: <root-key>"
+```
+
+**Response Example**
+
+```json
+{
+ "status": "ok",
+ "result": {
+ "enabled": false,
+ "account_id": "default"
+ },
+ "time": 0.1
+}
+```
+
+`enabled` is read from the live `server.agent_evolution.enabled` value used by
+session commits. `account_id` is the deployment's configured default account.
+
+---
+
### create_account
#### 1. API Implementation Overview
diff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.md
index 297273ed1..088415dc2 100644
--- a/docs/en/guides/01-configuration.md
+++ b/docs/en/guides/01-configuration.md
@@ -1506,11 +1506,11 @@ When running OpenViking as an HTTP service, add a `server` section to `ov.conf`:
When `root_api_key` is configured in `api_key` mode, the server enables multi-tenant authentication. Use the Admin API to create accounts and user keys. In `trusted` mode, ordinary requests do not require user registration first; each request is resolved as `USER` from the injected identity headers. However, skipping `root_api_key` in `trusted` mode is allowed only on localhost. Development mode only applies when `auth_mode = "api_key"` and `root_api_key` is not set.
-`user_config_defaults` only provides per-user defaults for add targets. For add operations, explicit request targets still win: `add_resource.to` / `add_resource.parent` take precedence over user defaults, and `add_skill.target_uri` takes precedence over user defaults. `agent_evolution.enabled` is shared by the entire OpenViking instance and has no per-user override; restart the server after changing it.
+`user_config_defaults` only provides per-user defaults for add targets. For add operations, explicit request targets still win: `add_resource.to` / `add_resource.parent` take precedence over user defaults, and `add_skill.target_uri` takes precedence over user defaults. `agent_evolution.enabled` is shared by the entire OpenViking instance and has no per-user override. Running HTTP server workers read the current value from the resolved `ov.conf` when a session commits, so a valid file update applies without restarting the server.
### Usage Reporter
-The optional Usage Reporter extracts memory usage events from committed session tool parts. The built-in HTTP sink persists batches to a local outbox before delivering them to a collector:
+The optional Usage Reporter extracts memory usage events from committed session tool parts. The built-in file log sink writes each event as a `{"key": ..., "value": ...}` JSON envelope to a dedicated hourly rotating file:
```json
{
@@ -1520,17 +1520,12 @@ The optional Usage Reporter extracts memory usage events from committed session
"extractors": ["memory_usage"],
"sinks": [
{
- "type": "http",
+ "type": "file_log",
"config": {
- "endpoint": "https://collector.example.com/openviking/usage",
+ "path": "/var/log/openviking_usage/usage.log",
"resource_id_env": "OV_RESOURCE_ID",
- "outbox_dir": "/var/lib/openviking/.usage_outbox",
- "request_timeout_seconds": 10,
- "inflight_lease_seconds": 60,
- "retry_base_seconds": 1,
- "retry_max_seconds": 300,
- "max_batch_bytes": 1048576,
- "max_outbox_bytes": 268435456
+ "rotation_interval_hours": 1,
+ "backup_count": 168
}
}
]
@@ -1539,11 +1534,14 @@ The optional Usage Reporter extracts memory usage events from committed session
}
```
-Set the environment variable named by `resource_id_env` before starting the server. If `outbox_dir` is omitted, it defaults to `~/.openviking/data/.usage_outbox` for the operating-system user running OpenViking.
+The built-in `file_log` sink replaces the earlier `http` sink. Deployments
+using `"type": "http"` must migrate to `file_log` and collect the dedicated
+log files, or configure a `custom` sink that implements their delivery
+contract.
-HTTP delivery uses a durable local retry queue and may deliver the same event more than once. Collectors should deduplicate `CountRecord` entries by `uniqueId`. Every `2xx` response acknowledges a batch; transient failures are retried with exponential backoff. `400` and `422` responses move a batch to `dead_letter`, while `413` splits multi-event batches. The outbox is bounded by `max_outbox_bytes`; when the limit is reached, the oldest dead-letter batches are removed first, followed by the oldest pending batches. In-flight batches are not evicted. Because capacity pressure can discard pending data, reporting remains best-effort and does not provide an end-to-end at-least-once guarantee.
+Set the environment variable named by `resource_id_env` before starting the server. The sink creates the parent directory, appends events immediately, rotates the active file every UTC hour, and retains `backup_count` rotated files. It does not write to the default OpenViking stdout log.
-`inflight_lease_seconds` must be greater than `request_timeout_seconds`. The lease is refreshed whenever a worker claims a batch, preventing another worker from recovering an active delivery.
+Each line is a JSON envelope with `key` and `value` fields. `key` matches the original Kafka message key and has the form `resource_id|account_id|user_id|resource_uri`; it falls back to `session_id` when `resource_uri` is empty. `value` is the complete object used as the original Kafka message value, containing `count_name`, `op_type`, `amount`, `timestamp`, `unique_id`, `tags`, `extra`, and `prefix`. The JSON envelope preserves delimiters that appear inside the key. File collection and downstream delivery remain best-effort, so consumers should deduplicate by `value.unique_id`.
Supported add target URIs:
diff --git a/docs/zh/api/01-overview.md b/docs/zh/api/01-overview.md
index 301f5636f..862bc3d9b 100644
--- a/docs/zh/api/01-overview.md
+++ b/docs/zh/api/01-overview.md
@@ -527,6 +527,7 @@ JSON 输出 - 错误:
| 方法 | 路径 | 说明 |
|------|------|------|
+| GET | `/api/v1/admin/agent-evolution` | 获取实例级 Agent 进化开关的实时状态 |
| POST | `/api/v1/admin/accounts` | 创建账号及首个管理员 |
| GET | `/api/v1/admin/accounts` | 列出账号 |
| POST | `/api/v1/admin/migrate` | 迁移旧版身份数据 |
diff --git a/docs/zh/api/08-admin.md b/docs/zh/api/08-admin.md
index 947fee47f..e7c3f189f 100644
--- a/docs/zh/api/08-admin.md
+++ b/docs/zh/api/08-admin.md
@@ -57,6 +57,41 @@ Admin API 用于多租户环境下的账户和用户管理。包括工作区(a
## API 参考
+### get_agent_evolution_status
+
+返回实例级 Agent 进化开关的实时状态和已配置的默认 account ID。该接口仅限
+ROOT 调用。
+
+**HTTP API**
+
+```
+GET /api/v1/admin/agent-evolution
+```
+
+```bash
+curl http://localhost:1933/api/v1/admin/agent-evolution \
+ -H "X-API-Key: <root-key>"
+```
+
+**响应示例**
+
+```json
+{
+ "status": "ok",
+ "result": {
+ "enabled": false,
+ "account_id": "default"
+ },
+ "time": 0.1
+}
+```
+
+`enabled` 来自 session commit 实际使用的
+`server.agent_evolution.enabled` 实时值;`account_id` 是部署配置的默认
+account。
+
+---
+
### create_account
#### 1. API 实现介绍
diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.md
index ec1d016d1..3015d3e7a 100644
--- a/docs/zh/guides/01-configuration.md
+++ b/docs/zh/guides/01-configuration.md
@@ -1471,11 +1471,11 @@ openviking add-resource ./docs --exclude "*.tmp"
在 `api_key` 模式下配置 `root_api_key` 后,服务端启用正式多租户认证,并通过 Admin API 创建工作区和用户 key。在 `trusted` 模式下,普通请求不需要先注册 user key;每个请求都会根据注入的身份头解析成 `USER`。只有在 `auth_mode = "api_key"` 且未配置 `root_api_key` 时,服务端才会进入开发模式。
-`user_config_defaults` 仅用于添加目标的用户级默认配置。添加操作中,显式请求目标仍然优先:`add_resource.to` / `add_resource.parent` 优先于用户默认值,`add_skill.target_uri` 优先于用户默认值。`agent_evolution.enabled` 是当前 OpenViking 实例的统一开关,不支持用户级覆盖;修改后需重启服务加载配置。
+`user_config_defaults` 仅用于添加目标的用户级默认配置。添加操作中,显式请求目标仍然优先:`add_resource.to` / `add_resource.parent` 优先于用户默认值,`add_skill.target_uri` 优先于用户默认值。`agent_evolution.enabled` 是当前 OpenViking 实例的统一开关,不支持用户级覆盖。HTTP Server 的 worker 会在 session commit 时从启动阶段解析出的 `ov.conf` 路径读取当前值,因此合法的文件更新无需重启服务即可生效。
### Usage Reporter
-可选的 Usage Reporter 从已 commit session 的 tool parts 中抽取记忆使用事件。内置 HTTP Sink 会先把事件批次持久化到本地 outbox,再发送给采集服务:
+可选的 Usage Reporter 从已 commit session 的 tool parts 中抽取记忆使用事件。内置文件日志 Sink 将每个事件写成一行 `{"key": ..., "value": ...}` JSON envelope,并按小时滚动专用日志文件:
```json
{
@@ -1485,17 +1485,12 @@ openviking add-resource ./docs --exclude "*.tmp"
"extractors": ["memory_usage"],
"sinks": [
{
- "type": "http",
+ "type": "file_log",
"config": {
- "endpoint": "https://collector.example.com/openviking/usage",
+ "path": "/var/log/openviking_usage/usage.log",
"resource_id_env": "OV_RESOURCE_ID",
- "outbox_dir": "/var/lib/openviking/.usage_outbox",
- "request_timeout_seconds": 10,
- "inflight_lease_seconds": 60,
- "retry_base_seconds": 1,
- "retry_max_seconds": 300,
- "max_batch_bytes": 1048576,
- "max_outbox_bytes": 268435456
+ "rotation_interval_hours": 1,
+ "backup_count": 168
}
}
]
@@ -1504,11 +1499,13 @@ openviking add-resource ./docs --exclude "*.tmp"
}
```
-启动服务前,需要设置 `resource_id_env` 指定的环境变量。未配置 `outbox_dir` 时,默认使用 OpenViking 运行用户的 `~/.openviking/data/.usage_outbox`。
+内置 `file_log` Sink 替代了此前的 `http` Sink。原来使用
+`"type": "http"` 的部署需要迁移为 `file_log` 并采集专用日志文件,或配置实现
+原投递协议的 `custom` Sink。
-HTTP Sink 使用本地持久化队列重试,同一事件可能被重复发送,采集端需要按 `CountRecord.uniqueId` 去重。所有 `2xx` 响应都表示批次已确认;瞬时失败使用指数退避重试;`400` 和 `422` 会将批次移入 `dead_letter`;`413` 会拆分包含多条事件的批次。outbox 总量受 `max_outbox_bytes` 限制,达到上限后先删除最旧的 dead-letter 批次,再删除最旧的 pending 批次,正在发送的批次不会被淘汰。由于容量压力可能丢弃 pending 数据,整体上报仍是 best-effort,不提供端到端 at-least-once 保证。
+启动服务前,需要设置 `resource_id_env` 指定的环境变量。Sink 会自动创建父目录、立即追加事件、按 UTC 每小时滚动文件,并保留 `backup_count` 个历史文件;它不会写入 OpenViking 默认 stdout 日志。
-`inflight_lease_seconds` 必须大于 `request_timeout_seconds`。worker 领取批次时会刷新 lease,避免其他 worker 把正在发送的批次误判为过期任务。
+每行是包含 `key` 和 `value` 字段的 JSON envelope。`key` 与原 Kafka 消息键一致,格式为 `resource_id|account_id|user_id|resource_uri`;`resource_uri` 为空时使用 `session_id`。`value` 是原 Kafka 消息的完整对象,包含 `count_name`、`op_type`、`amount`、`timestamp`、`unique_id`、`tags`、`extra` 和 `prefix`。JSON envelope 能完整保留 key 内部的分隔符。文件采集和下游投递仍为 best-effort,消费端应按 `value.unique_id` 去重。
支持的 add target URI:
diff --git a/openviking/server/agent_evolution_config.py b/openviking/server/agent_evolution_config.py
new file mode 100644
index 000000000..091396165
--- /dev/null
+++ b/openviking/server/agent_evolution_config.py
@@ -0,0 +1,68 @@
+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
+# SPDX-License-Identifier: AGPL-3.0
+"""Live access to the instance-wide Agent Evolution switch."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Optional
+
+from openviking.server.config import ServerConfig
+from openviking_cli.utils.config import load_json_config
+from openviking_cli.utils.logger import get_logger
+
+logger = get_logger(__name__)
+
+
+class AgentEvolutionConfigProvider:
+ """Read the live Agent Evolution value from the configured ov.conf.
+
+ Kubernetes projects Secret updates into the mounted config directory
+ atomically. Reading the file at commit time lets already-created sessions
+ observe the new value without restarting the server.
+ """
+
+ def __init__(
+ self,
+ *,
+ default_enabled: bool,
+ config_path: Optional[str | Path] = None,
+ ) -> None:
+ self._last_valid_enabled = bool(default_enabled)
+ self._config_path = Path(config_path).expanduser() if config_path else None
+
+ def set_default_enabled(self, enabled: bool) -> None:
+ self._last_valid_enabled = bool(enabled)
+
+ def is_enabled(self) -> bool:
+ config_path = self._config_path
+ if config_path is None:
+ return self._last_valid_enabled
+
+ try:
+ payload = load_json_config(config_path)
+ if not isinstance(payload, dict):
+ raise ValueError("config root must be an object")
+ server_config = payload.get("server", {})
+ if server_config is None:
+ server_config = {}
+ if not isinstance(server_config, dict):
+ raise ValueError("server must be an object")
+ enabled = ServerConfig.model_validate(server_config).agent_evolution.enabled
+ except OSError as exc:
+ logger.warning(
+ "Failed to access Agent Evolution config file %s, using last valid value: %s",
+ config_path,
+ exc,
+ )
+ return self._last_valid_enabled
+ except (KeyError, TypeError, ValueError) as exc:
+ logger.warning(
+ "Failed to read Agent Evolution config from %s, using last valid value: %s",
+ config_path,
+ exc,
+ )
+ return self._last_valid_enabled
+
+ self._last_valid_enabled = enabled
+ return enabled
diff --git a/openviking/server/app.py b/openviking/server/app.py
index 1c1e4830c..42169fd5e 100644
--- a/openviking/server/app.py
+++ b/openviking/server/app.py
@@ -58,7 +58,12 @@ from openviking.service.core import OpenVikingService
from openviking.service.task_tracker import get_task_tracker
from openviking_cli.exceptions import OpenVikingError
from openviking_cli.utils import get_logger
-from openviking_cli.utils.config import get_openviking_config
+from openviking_cli.utils.config import (
+ DEFAULT_OV_CONF,
+ OPENVIKING_CONFIG_ENV,
+ get_openviking_config,
+ resolve_config_path,
+)
from openviking_cli.utils.logger import init_otel_log_handler_from_server_config
logger = get_logger(__name__)
@@ -69,14 +74,20 @@ WORKER_BOT_API_URL_ENV = "OPENVIKING_WORKER_BOT_API_URL"
def create_worker_app() -> FastAPI:
"""Load file config and replay parent-process Bot CLI overrides."""
- config = load_server_config()
+ resolved_config_path = resolve_config_path(
+ None,
+ OPENVIKING_CONFIG_ENV,
+ DEFAULT_OV_CONF,
+ )
+ config_path = str(resolved_config_path) if resolved_config_path is not None else None
+ config = load_server_config(config_path)
with_bot = os.environ.get(WORKER_WITH_BOT_ENV)
if with_bot is not None:
config.with_bot = with_bot == "1"
bot_api_url = os.environ.get(WORKER_BOT_API_URL_ENV)
if bot_api_url is not None:
config.bot_api_url = bot_api_url
- return create_app(config)
+ return create_app(config, config_path=config_path)
async def _initialize_auth_plugin(
@@ -203,18 +214,27 @@ def _message_from_http_detail(detail: object) -> str:
def create_app(
config: Optional[ServerConfig] = None,
service: Optional[OpenVikingService] = None,
+ config_path: Optional[str] = None,
) -> FastAPI:
"""Create FastAPI application.
Args:
config: Server configuration. If None, loads from default location.
service: Pre-initialized OpenVikingService (optional).
+ config_path: Resolved ov.conf path used for live configuration reload.
Returns:
FastAPI application instance
"""
+ resolved_config_path = (
+ resolve_config_path(config_path, OPENVIKING_CONFIG_ENV, DEFAULT_OV_CONF)
+ if config_path is not None or config is None
+ else None
+ )
if config is None:
- config = load_server_config()
+ config = load_server_config(
+ str(resolved_config_path) if resolved_config_path is not None else config_path
+ )
validate_server_config(config)
@@ -242,6 +262,15 @@ def create_app(
agent_evolution_setter = getattr(sessions, "set_agent_evolution_config", None)
if callable(agent_evolution_setter):
agent_evolution_setter(config.agent_evolution)
+ agent_evolution_path_setter = getattr(
+ sessions,
+ "set_agent_evolution_config_path",
+ None,
+ )
+ if callable(agent_evolution_path_setter):
+ agent_evolution_path_setter(
+ str(resolved_config_path) if resolved_config_path is not None else None
+ )
if service is not None:
_configure_session_runtime(service)
diff --git a/openviking/server/bootstrap.py b/openviking/server/bootstrap.py
index 8c1575328..4a35548e7 100644
--- a/openviking/server/bootstrap.py
+++ b/openviking/server/bootstrap.py
@@ -207,6 +207,11 @@ def main():
# Load server config from ov.conf
try:
+ resolved_config_path = resolve_config_path(
+ args.config,
+ OPENVIKING_CONFIG_ENV,
+ DEFAULT_OV_CONF,
+ )
config = load_server_config(args.config)
OpenVikingConfigSingleton.initialize(config_path=args.config)
except (FileNotFoundError, ValueError) as e:
@@ -279,7 +284,12 @@ def main():
sys.exit(1)
# Create and run server app
- app = create_app(config)
+ app = create_app(
+ config,
+ config_path=(
+ str(resolved_config_path) if resolved_config_path is not None else args.config
+ ),
+ )
workers_info = f" (workers: {config.workers})" if config.workers > 1 else ""
print(f"OpenViking HTTP Server is running on {config.host}:{config.port}{workers_info}")
diff --git a/openviking/server/config.py b/openviking/server/config.py
index 71d232bb3..27afda988 100644
--- a/openviking/server/config.py
+++ b/openviking/server/config.py
@@ -196,7 +196,7 @@ class UsageAuditConfig(BaseModel):
class UsageReporterSinkConfig(BaseModel):
"""Usage reporter sink configuration."""
- type: Literal["custom", "http"] = "custom"
+ type: Literal["custom", "file_log"] = "custom"
class_path: Optional[str] = None
config: Dict[str, object] = Field(default_factory=dict)
diff --git a/openviking/server/routers/admin.py b/openviking/server/routers/admin.py
index 56c686ad9..4726b3bc5 100644
--- a/openviking/server/routers/admin.py
+++ b/openviking/server/routers/admin.py
@@ -33,6 +33,7 @@ from openviking_cli.exceptions import (
PermissionDeniedError,
)
from openviking_cli.session.user_id import UserIdentifier
+from openviking_cli.utils.config import get_openviking_config
from openviking_cli.utils.logger import get_logger
logger = get_logger(__name__)
@@ -66,6 +67,24 @@ class MigrateLegacyDataRequest(BaseModel):
action: str = "migrate"
[email protected]("/agent-evolution")
+@require_auth_root
+async def get_agent_evolution_status(
+ request: Request,
+ ctx: RequestContext = Depends(get_request_context),
+):
+ """Return the live instance-wide Agent Evolution switch."""
+ del request
+ del ctx
+ return Response(
+ status="ok",
+ result={
+ "enabled": get_service().sessions.get_agent_evolution_enabled(),
+ "account_id": get_openviking_config().default_account,
+ },
+ )
+
+
def _get_api_key_manager(request: Request):
"""Get APIKeyManager from app state."""
return get_api_key_manager_or_raise(request)
diff --git a/openviking/service/session_service.py b/openviking/service/session_service.py
index 25b8704cf..14f225d04 100644
--- a/openviking/service/session_service.py
+++ b/openviking/service/session_service.py
@@ -9,6 +9,7 @@ Provides session management operations: session, sessions, add_message, commit,
from typing import TYPE_CHECKING, Any, Dict, List, Optional
from openviking.core.namespace import canonical_session_uri
+from openviking.server.agent_evolution_config import AgentEvolutionConfigProvider
from openviking.server.config import AgentEvolutionConfig, ToolOutputExternalizationConfig
from openviking.server.identity import RequestContext
from openviking.service.task_tracker import get_task_tracker
@@ -48,6 +49,7 @@ class SessionService:
# Agent memory behavior; HTTP servers always override this from
# server.agent_evolution during app setup.
self._agent_evolution_enabled = True
+ self._agent_evolution_config_provider: Optional[AgentEvolutionConfigProvider] = None
self._usage_reporter: Optional["UsageReporter"] = None
def set_dependencies(
@@ -70,6 +72,25 @@ class SessionService:
def set_agent_evolution_config(self, config: AgentEvolutionConfig) -> None:
"""Set the instance-wide Agent Evolution switch."""
self._agent_evolution_enabled = config.enabled
+ if self._agent_evolution_config_provider is not None:
+ self._agent_evolution_config_provider.set_default_enabled(config.enabled)
+
+ def set_agent_evolution_config_path(self, config_path: Optional[str]) -> None:
+ """Enable live reload from the HTTP server's resolved ov.conf path."""
+ self._agent_evolution_config_provider = (
+ AgentEvolutionConfigProvider(
+ default_enabled=self._agent_evolution_enabled,
+ config_path=config_path,
+ )
+ if config_path
+ else None
+ )
+
+ def get_agent_evolution_enabled(self) -> bool:
+ """Return the live instance-wide Agent Evolution switch."""
+ if self._agent_evolution_config_provider is None:
+ return self._agent_evolution_enabled
+ return self._agent_evolution_config_provider.is_enabled()
def set_usage_reporter(self, usage_reporter: Optional["UsageReporter"]) -> None:
"""Set the usage reporter for newly created sessions."""
@@ -134,7 +155,8 @@ class SessionService:
session_id=session_id,
session_uri=session_uri,
tool_output_externalization_config=self._tool_output_externalization_config,
- agent_evolution_enabled=self._agent_evolution_enabled,
+ agent_evolution_enabled=self.get_agent_evolution_enabled(),
+ agent_evolution_enabled_provider=self.get_agent_evolution_enabled,
usage_reporter=self._usage_reporter,
)
@@ -377,7 +399,7 @@ class SessionService:
session_id=session_id,
ctx=ctx,
archive_uri=archive_uri,
- agent_evolution_enabled=self._agent_evolution_enabled,
+ agent_evolution_enabled=self.get_agent_evolution_enabled(),
)
self._record_lifecycle_metric("extract", "ok")
return memories
diff --git a/openviking/session/compressor_v3.py b/openviking/session/compressor_v3.py
index 95e611a5a..a198817e3 100644
--- a/openviking/session/compressor_v3.py
+++ b/openviking/session/compressor_v3.py
@@ -111,10 +111,10 @@ async def _commit_experience_snapshot(
]
if not paths:
return
- archive_name = archive_uri.rstrip("/").rsplit("/", 1)[-1] if archive_uri else "unknown"
+ archive_ref = archive_uri.rstrip("/") if archive_uri else "unknown"
try:
await commit(
- message=f"Update experience memories from session commit {archive_name}",
+ message=f"Update experience memories from session commit {archive_ref}",
paths=paths,
ctx=ctx,
)
diff --git a/openviking/session/session.py b/openviking/session/session.py
index b28a8745e..80ec7f356 100644
--- a/openviking/session/session.py
+++ b/openviking/session/session.py
@@ -544,6 +544,7 @@ class Session:
tool_output_externalization_config: Optional[ToolOutputExternalizationConfig] = None,
agent_evolution_enabled: bool = True,
usage_reporter: Optional["UsageReporter"] = None,
+ agent_evolution_enabled_provider: Optional[Callable[[], bool]] = None,
):
self._viking_fs = viking_fs
self._vikingdb_manager = vikingdb_manager
@@ -575,6 +576,7 @@ class Session:
else ToolOutputExternalizationConfig()
)
self._agent_evolution_enabled = agent_evolution_enabled
+ self._agent_evolution_enabled_provider = agent_evolution_enabled_provider
self._usage_reporter = usage_reporter
logger.info(f"Session created: {self.session_id} for user {self.user}")
@@ -1710,7 +1712,11 @@ class Session:
memory_policy if memory_policy is not None else self._meta.memory_policy
)
_validate_memory_policy_types(effective_policy)
- agent_evolution_enabled = self._agent_evolution_enabled
+ agent_evolution_enabled = (
+ self._agent_evolution_enabled_provider()
+ if self._agent_evolution_enabled_provider is not None
+ else self._agent_evolution_enabled
+ )
effective_policy = _apply_agent_evolution_setting(
effective_policy,
agent_evolution_enabled=agent_evolution_enabled,
diff --git a/openviking/usage_reporter/__init__.py b/openviking/usage_reporter/__init__.py
index d38f5d585..c52fe5536 100644
--- a/openviking/usage_reporter/__init__.py
+++ b/openviking/usage_reporter/__init__.py
@@ -3,14 +3,14 @@
"""Usage reporting extension points for OpenViking."""
from .extractors import MemoryUsageExtractor, UsageExtractor
-from .http_sink import HttpUsageSink
+from .file_log_sink import FileLogUsageSink
from .models import UsageContext, UsageEvent
from .reporter import UsageReporter
from .sinks import UsageSink
__all__ = [
+ "FileLogUsageSink",
"MemoryUsageExtractor",
- "HttpUsageSink",
"UsageContext",
"UsageEvent",
"UsageExtractor",
diff --git a/openviking/usage_reporter/config.py b/openviking/usage_reporter/config.py
index f6d0ecf05..ad8645cca 100644
--- a/openviking/usage_reporter/config.py
+++ b/openviking/usage_reporter/config.py
@@ -10,7 +10,7 @@ from typing import Any
from openviking.server.config import UsageReporterConfig, UsageReporterSinkConfig
from .extractors import MemoryUsageExtractor
-from .http_sink import HttpUsageSink
+from .file_log_sink import FileLogUsageSink
from .reporter import UsageReporter
from .sinks import UsageSink
@@ -25,8 +25,8 @@ def _load_class(class_path: str) -> type:
def _build_sink(config: UsageReporterSinkConfig) -> UsageSink:
- if config.type == "http":
- return HttpUsageSink(**dict(config.config))
+ if config.type == "file_log":
+ return FileLogUsageSink(**dict(config.config))
if config.type == "custom":
if not config.class_path:
diff --git a/openviking/usage_reporter/file_log_sink.py b/openviking/usage_reporter/file_log_sink.py
new file mode 100644
index 000000000..69e72f027
--- /dev/null
+++ b/openviking/usage_reporter/file_log_sink.py
@@ -0,0 +1,263 @@
+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
+# SPDX-License-Identifier: AGPL-3.0
+"""Hourly rotating key-value usage log sink."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import os
+from contextlib import contextmanager
+from datetime import datetime, timezone
+from logging.handlers import TimedRotatingFileHandler
+from pathlib import Path
+from typing import TYPE_CHECKING, Any, Iterator
+
+if TYPE_CHECKING:
+ from .models import UsageEvent
+
+_COUNT_NAMES = {
+ "memory.recalled": "experience.recall.count",
+ "memory.injected": "experience.inject.count",
+}
+
+
+@contextmanager
+def _interprocess_lock(path: Path) -> Iterator[None]:
+ """Serialize log rotation across server worker processes."""
+ descriptor = os.open(path, os.O_CREAT | os.O_RDWR, 0o600)
+ try:
+ if os.name == "nt":
+ import msvcrt
+
+ if os.fstat(descriptor).st_size == 0:
+ os.write(descriptor, b"\0")
+ os.lseek(descriptor, 0, os.SEEK_SET)
+ msvcrt.locking(descriptor, msvcrt.LK_LOCK, 1)
+ else:
+ import fcntl
+
+ fcntl.flock(descriptor, fcntl.LOCK_EX)
+ yield
+ finally:
+ try:
+ if os.name == "nt":
+ import msvcrt
+
+ os.lseek(descriptor, 0, os.SEEK_SET)
+ msvcrt.locking(descriptor, msvcrt.LK_UNLCK, 1)
+ else:
+ import fcntl
+
+ fcntl.flock(descriptor, fcntl.LOCK_UN)
+ finally:
+ os.close(descriptor)
+
+
+class _ProcessSafeTimedRotatingFileHandler(TimedRotatingFileHandler):
+ """Timed handler that coordinates rollover with sibling worker processes."""
+
+ def __init__(self, filename: Path, **kwargs: Any) -> None:
+ self._process_lock_path = filename.with_name(f".{filename.name}.lock")
+ super().__init__(filename=filename, **kwargs)
+ self._base_file_identity = self._get_base_file_identity()
+
+ def emit(self, record: logging.LogRecord) -> None:
+ with _interprocess_lock(self._process_lock_path):
+ self._refresh_after_external_rollover()
+ try:
+ super().emit(record)
+ finally:
+ self._base_file_identity = self._get_base_file_identity()
+ # Windows does not allow another process to rename an open
+ # file. Release the handle so the next worker can roll it.
+ if os.name == "nt" and self.stream is not None:
+ self.stream.close()
+ self.stream = None
+
+ @staticmethod
+ def _file_identity(stat_result: os.stat_result) -> tuple[int, int]:
+ return stat_result.st_dev, stat_result.st_ino
+
+ def _get_base_file_identity(self) -> tuple[int, int] | None:
+ try:
+ return self._file_identity(os.stat(self.baseFilename))
+ except FileNotFoundError:
+ return None
+
+ def _refresh_after_external_rollover(self) -> None:
+ """Reopen a base file rotated by another process and sync its deadline."""
+ try:
+ path_stat = os.stat(self.baseFilename)
+ except FileNotFoundError:
+ path_stat = None
+ path_identity = self._file_identity(path_stat) if path_stat is not None else None
+
+ stream_stat = None
+ if self.stream is not None:
+ try:
+ stream_stat = os.fstat(self.stream.fileno())
+ except (OSError, ValueError):
+ stream_stat = None
+
+ if stream_stat is not None and path_stat is not None:
+ if self._file_identity(stream_stat) == path_identity:
+ self._base_file_identity = path_identity
+ return
+ elif self.stream is None and path_identity == self._base_file_identity:
+ # Windows closes the stream after every emit. The missing handle
+ # alone does not mean another worker rotated the file, so preserve
+ # this handler's existing rollover deadline.
+ return
+
+ if self.stream is not None:
+ self.stream.close()
+ self.stream = None
+ self._base_file_identity = path_identity
+ if path_stat is not None:
+ self.rolloverAt = self.computeRollover(int(path_stat.st_mtime))
+
+
+def _timestamp_millis(value: str) -> int:
+ parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
+ if parsed.tzinfo is None:
+ parsed = parsed.replace(tzinfo=timezone.utc)
+ return int(parsed.timestamp() * 1000)
+
+
+def _to_count_record(event: UsageEvent, *, resource_id: str) -> dict[str, Any]:
+ record = event.to_dict()
+ event_type = str(record.get("event_type") or "")
+ try:
+ count_name = _COUNT_NAMES[event_type]
+ except KeyError as exc:
+ raise ValueError(f"unsupported usage event type: {event_type}") from exc
+
+ evidence = record.get("evidence")
+ evidence = evidence if isinstance(evidence, dict) else {}
+ attributes = record.get("attributes")
+ attributes = attributes if isinstance(attributes, dict) else {}
+
+ extra = {key: value for key, value in evidence.items() if value not in (None, "")}
+ session_id = str(record.get("session_id") or "")
+ if session_id:
+ extra["session_id"] = session_id
+ task_id = record.get("task_id")
+ if task_id not in (None, ""):
+ extra["task_id"] = task_id
+ if attributes:
+ extra["attributes"] = attributes
+
+ unique_id = str(record.get("event_id") or "").strip()
+ if not unique_id:
+ raise ValueError("event_id is required")
+
+ count_record = {
+ "count_name": count_name,
+ "op_type": "add",
+ "amount": 1.0,
+ "timestamp": _timestamp_millis(str(record.get("occurred_at") or "")),
+ "unique_id": unique_id,
+ "tags": {
+ "account_id": str(record.get("account_id") or ""),
+ "user_id": str(record.get("user_id") or ""),
+ "resource_uri": str(record.get("resource_uri") or ""),
+ "resource_type": str(record.get("resource_type") or ""),
+ },
+ "extra": extra,
+ }
+ count_record["prefix"] = resource_id
+ return count_record
+
+
+def _build_message_key(record: dict[str, Any]) -> str:
+ tags = record.get("tags")
+ tags = tags if isinstance(tags, dict) else {}
+ extra = record.get("extra")
+ extra = extra if isinstance(extra, dict) else {}
+ target = str(tags.get("resource_uri") or extra.get("session_id") or "")
+ return "|".join(
+ (
+ str(record.get("prefix") or ""),
+ str(tags.get("account_id") or ""),
+ str(tags.get("user_id") or ""),
+ target,
+ )
+ )
+
+
+def _to_log_record(event: UsageEvent, *, resource_id: str) -> dict[str, Any]:
+ value = _to_count_record(event, resource_id=resource_id)
+ return {
+ "key": _build_message_key(value),
+ "value": value,
+ }
+
+
+def _format_record(record: dict[str, Any]) -> str:
+ return json.dumps(record, ensure_ascii=False, separators=(",", ":"))
+
+
+class FileLogUsageSink:
+ """Write usage records to a dedicated hourly rotating log file."""
+
+ def __init__(
+ self,
+ *,
+ path: str | os.PathLike[str],
+ resource_id_env: str = "OV_RESOURCE_ID",
+ rotation_interval_hours: int = 1,
+ backup_count: int = 168,
+ ) -> None:
+ raw_path = str(path).strip()
+ if not raw_path:
+ raise ValueError("path is required")
+ self._resource_id = os.environ.get(resource_id_env, "").strip()
+ if not self._resource_id:
+ raise ValueError(f"{resource_id_env} is required")
+ if rotation_interval_hours <= 0:
+ raise ValueError("rotation_interval_hours must be positive")
+ if backup_count < 0:
+ raise ValueError("backup_count must be non-negative")
+
+ self._path = Path(os.path.expandvars(os.path.expanduser(raw_path)))
+ self._path.parent.mkdir(parents=True, exist_ok=True)
+ self._handler = _ProcessSafeTimedRotatingFileHandler(
+ filename=self._path,
+ when="H",
+ interval=rotation_interval_hours,
+ backupCount=backup_count,
+ encoding="utf-8",
+ delay=True,
+ utc=True,
+ )
+ self._handler.setFormatter(logging.Formatter("%(message)s"))
+ self._logger = logging.getLogger(f"openviking.usage_reporter.file_log.{id(self)}")
+ self._logger.setLevel(logging.INFO)
+ self._logger.propagate = False
+ self._logger.addHandler(self._handler)
+ self._closed = False
+
+ @property
+ def path(self) -> Path:
+ return self._path
+
+ async def write(self, *, events: list[UsageEvent]) -> None:
+ if not events:
+ return
+ records = [_to_log_record(event, resource_id=self._resource_id) for event in events]
+ await asyncio.to_thread(self._write_records, records)
+
+ def _write_records(self, records: list[dict[str, Any]]) -> None:
+ if self._closed:
+ raise RuntimeError("usage log sink is closed")
+ for record in records:
+ self._logger.info(_format_record(record))
+
+ def close(self) -> None:
+ if self._closed:
+ return
+ self._closed = True
+ self._logger.removeHandler(self._handler)
+ self._handler.close()
diff --git a/openviking/usage_reporter/http_sink.py b/openviking/usage_reporter/http_sink.py
deleted file mode 100644
index ead3e0c85..000000000
--- a/openviking/usage_reporter/http_sink.py
+++ /dev/null
@@ -1,524 +0,0 @@
-# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
-# SPDX-License-Identifier: AGPL-3.0
-"""Durable HTTP usage event sink."""
-
-from __future__ import annotations
-
-import asyncio
-import hashlib
-import json
-import logging
-import os
-import random
-import threading
-import time
-import urllib.error
-import urllib.request
-from collections.abc import Iterator
-from datetime import datetime, timezone
-from pathlib import Path
-from typing import TYPE_CHECKING, Any
-
-if TYPE_CHECKING:
- from .models import UsageEvent
-
-logger = logging.getLogger(__name__)
-
-_COUNT_NAMES = {
- "memory.recalled": "experience.recall.count",
- "memory.injected": "experience.inject.count",
-}
-
-
-def _timestamp_millis(value: str) -> int:
- parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
- if parsed.tzinfo is None:
- parsed = parsed.replace(tzinfo=timezone.utc)
- return int(parsed.timestamp() * 1000)
-
-
-def _to_count_record(event: UsageEvent) -> dict[str, Any]:
- record = event.to_dict()
- event_type = str(record.get("event_type") or "")
- try:
- count_name = _COUNT_NAMES[event_type]
- except KeyError as exc:
- raise ValueError(f"unsupported usage event type: {event_type}") from exc
-
- evidence = record.get("evidence")
- evidence = evidence if isinstance(evidence, dict) else {}
- attributes = record.get("attributes")
- attributes = attributes if isinstance(attributes, dict) else {}
-
- extra = {key: value for key, value in evidence.items() if value not in (None, "")}
- session_id = str(record.get("session_id") or "")
- if session_id:
- extra["session_id"] = session_id
- task_id = record.get("task_id")
- if task_id not in (None, ""):
- extra["task_id"] = task_id
- if attributes:
- extra["attributes"] = attributes
-
- return {
- "CountName": count_name,
- "OpType": "add",
- "amount": 1.0,
- "timestamp": _timestamp_millis(str(record.get("occurred_at") or "")),
- "uniqueId": str(record.get("event_id") or ""),
- "tags": {
- "account_id": str(record.get("account_id") or ""),
- "user_id": str(record.get("user_id") or ""),
- "resource_uri": str(record.get("resource_uri") or ""),
- "resource_type": str(record.get("resource_type") or ""),
- },
- "extra": extra,
- }
-
-
-class HttpUsageSink:
- """Persist usage events before delivering them to an HTTP collector."""
-
- def __init__(
- self,
- *,
- endpoint: str,
- resource_id_env: str = "OV_RESOURCE_ID",
- outbox_dir: str | os.PathLike[str] | None = None,
- request_timeout_seconds: float = 10.0,
- inflight_lease_seconds: float = 60.0,
- retry_base_seconds: float = 1.0,
- retry_max_seconds: float = 300.0,
- max_batch_bytes: int = 1024 * 1024,
- max_outbox_bytes: int = 256 * 1024 * 1024,
- ) -> None:
- self._endpoint = endpoint.strip()
- self._resource_id = os.environ.get(resource_id_env, "").strip()
- if not self._endpoint:
- raise ValueError("endpoint is required")
- if not self._resource_id:
- raise ValueError(f"{resource_id_env} is required")
- self._outbox_scope = hashlib.sha256(
- f"{self._endpoint}\0{self._resource_id}".encode("utf-8")
- ).hexdigest()[:24]
-
- self._outbox_dir = (
- Path(outbox_dir)
- if outbox_dir is not None
- else Path.home() / ".openviking" / "data" / ".usage_outbox"
- )
- self._pending_dir = self._outbox_dir / "pending"
- self._inflight_dir = self._outbox_dir / "inflight"
- self._dead_letter_dir = self._outbox_dir / "dead_letter"
- self._metadata_dir = self._outbox_dir / "metadata"
- for directory in (
- self._pending_dir,
- self._inflight_dir,
- self._dead_letter_dir,
- self._metadata_dir,
- ):
- directory.mkdir(parents=True, exist_ok=True)
-
- self._request_timeout_seconds = float(request_timeout_seconds)
- self._inflight_lease_seconds = float(inflight_lease_seconds)
- self._retry_base_seconds = float(retry_base_seconds)
- self._retry_max_seconds = float(retry_max_seconds)
- self._max_batch_bytes = int(max_batch_bytes)
- self._max_outbox_bytes = int(max_outbox_bytes)
- if self._request_timeout_seconds <= 0:
- raise ValueError("request_timeout_seconds must be positive")
- if self._inflight_lease_seconds <= 0:
- raise ValueError("inflight_lease_seconds must be positive")
- if self._inflight_lease_seconds <= self._request_timeout_seconds:
- raise ValueError("inflight_lease_seconds must be greater than request_timeout_seconds")
- if self._retry_base_seconds < 0:
- raise ValueError("retry_base_seconds must be non-negative")
- if self._retry_max_seconds < 0:
- raise ValueError("retry_max_seconds must be non-negative")
- if self._max_batch_bytes <= 0:
- raise ValueError("max_batch_bytes must be positive")
- if self._max_outbox_bytes <= 0:
- raise ValueError("max_outbox_bytes must be positive")
- self._outbox_lock = threading.Lock()
- self._wake_event = threading.Event()
- self._stop_event = threading.Event()
- self._worker: threading.Thread | None = None
- self._start_worker()
-
- async def write(self, *, events: list[UsageEvent]) -> None:
- if not events:
- return
- await asyncio.to_thread(self._persist_events, events)
-
- def close(self) -> None:
- self._stop_event.set()
- self._wake_event.set()
- if self._worker is not None:
- self._worker.join(timeout=self._request_timeout_seconds + 1)
-
- def _persist_events(self, events: list[UsageEvent]) -> None:
- records: list[dict[str, Any]] = []
- for event in events:
- event_id = str(event.to_dict().get("event_id") or "").strip()
- if not event_id:
- raise ValueError("event_id is required")
- records.append(_to_count_record(event))
-
- batch: list[dict[str, Any]] = []
- for record in records:
- candidate = [*batch, record]
- if batch and (
- len(candidate) > 100
- or self._encoded_payload_size(candidate) > self._max_batch_bytes
- ):
- self._persist_batch(batch)
- batch = [record]
- else:
- batch = candidate
- if batch:
- self._persist_batch(batch)
- self._wake_event.set()
-
- def _persist_batch(self, events: list[dict[str, Any]]) -> bool:
- return self._persist_batches([events])
-
- def _persist_batches(
- self,
- event_batches: list[list[dict[str, Any]]],
- *,
- replace_path: Path | None = None,
- ) -> bool:
- encoded_batches: list[tuple[dict[str, Any], bytes, Path, Path]] = []
- for index, events in enumerate(event_batches):
- payload = self._build_payload(events)
- encoded = json.dumps(
- payload,
- ensure_ascii=False,
- separators=(",", ":"),
- ).encode("utf-8")
- file_stem = f"{self._outbox_scope}_{time.time_ns()}_{index}_{payload['batch_id']}"
- encoded_batches.append(
- (
- payload,
- encoded,
- self._pending_dir / f".{file_stem}.tmp",
- self._pending_dir / f"{file_stem}.json",
- )
- )
-
- incoming_bytes = sum(len(encoded) for _, encoded, _, _ in encoded_batches)
- with self._outbox_lock:
- reclaimable_bytes = 0
- if replace_path is not None:
- try:
- reclaimable_bytes = replace_path.stat().st_size
- except FileNotFoundError:
- return False
- if not self._ensure_outbox_capacity(
- incoming_bytes,
- reclaimable_bytes=reclaimable_bytes,
- ):
- batch_ids = [payload["batch_id"] for payload, _, _, _ in encoded_batches]
- logger.warning(
- "Dropping OpenViking usage batch because the outbox is full, "
- "batch_ids=%s size=%s max_outbox_bytes=%s",
- batch_ids,
- incoming_bytes,
- self._max_outbox_bytes,
- )
- return False
- temporary_paths: list[Path] = []
- try:
- for _payload, encoded, temporary_path, pending_path in encoded_batches:
- temporary_paths.append(temporary_path)
- with temporary_path.open("wb") as output:
- output.write(encoded)
- output.flush()
- os.fsync(output.fileno())
- os.replace(temporary_path, pending_path)
- self._fsync_directory(self._pending_dir)
- if replace_path is not None:
- replace_path.unlink()
- self._fsync_directory(replace_path.parent)
- finally:
- for temporary_path in temporary_paths:
- try:
- temporary_path.unlink()
- except FileNotFoundError:
- pass
- return True
-
- def _ensure_outbox_capacity(
- self,
- incoming_bytes: int,
- *,
- reclaimable_bytes: int = 0,
- ) -> bool:
- if incoming_bytes > self._max_outbox_bytes:
- return False
-
- eviction_candidates: list[Path] = []
- total_bytes = 0
- for directory in (
- self._dead_letter_dir,
- self._pending_dir,
- self._inflight_dir,
- ):
- for path in self._owned_batches(directory):
- try:
- total_bytes += path.stat().st_size
- except FileNotFoundError:
- continue
- if directory != self._inflight_dir:
- eviction_candidates.append(path)
-
- effective_total_bytes = max(0, total_bytes - reclaimable_bytes)
- if effective_total_bytes + incoming_bytes <= self._max_outbox_bytes:
- return True
-
- def _eviction_order(path: Path) -> tuple[int, int, str]:
- try:
- modified_at = path.stat().st_mtime_ns
- except FileNotFoundError:
- modified_at = 0
- directory_priority = 0 if path.parent == self._dead_letter_dir else 1
- return directory_priority, modified_at, path.name
-
- for path in sorted(eviction_candidates, key=_eviction_order):
- try:
- size = path.stat().st_size
- path.unlink()
- except FileNotFoundError:
- continue
- effective_total_bytes -= size
- logger.warning("Evicted OpenViking usage outbox batch: %s", path)
- if effective_total_bytes + incoming_bytes <= self._max_outbox_bytes:
- return True
- return effective_total_bytes + incoming_bytes <= self._max_outbox_bytes
-
- def _build_payload(self, events: list[dict[str, Any]]) -> dict[str, Any]:
- unique_ids = "\n".join(str(event["uniqueId"]) for event in events)
- batch_id = f"ub_{hashlib.sha256(unique_ids.encode('utf-8')).hexdigest()}"
- created_at = (
- datetime.now(timezone.utc)
- .isoformat(timespec="milliseconds")
- .replace(
- "+00:00",
- "Z",
- )
- )
- return {
- "schema_version": "v1",
- "resource_id": self._resource_id,
- "batch_id": batch_id,
- "created_at": created_at,
- "attempt": 0,
- "next_retry_at": created_at,
- "events": events,
- }
-
- def _encoded_payload_size(self, events: list[dict[str, Any]]) -> int:
- return len(
- json.dumps(
- self._build_payload(events),
- ensure_ascii=False,
- separators=(",", ":"),
- ).encode("utf-8")
- )
-
- @staticmethod
- def _fsync_directory(directory: Path) -> None:
- if os.name == "nt":
- return
- descriptor = os.open(directory, os.O_RDONLY)
- try:
- os.fsync(descriptor)
- finally:
- os.close(descriptor)
-
- def _start_worker(self) -> None:
- self._recover_stale_inflight()
- with self._outbox_lock:
- self._ensure_outbox_capacity(0)
- self._worker = threading.Thread(
- target=self._worker_loop,
- name="openviking-usage-http-sink",
- daemon=True,
- )
- self._worker.start()
-
- def _worker_loop(self) -> None:
- while not self._stop_event.is_set():
- try:
- self._recover_stale_inflight()
- handled_batch = self._handle_next_batch()
- except Exception:
- logger.exception("OpenViking usage outbox worker failed")
- handled_batch = False
- if handled_batch:
- continue
- self._wake_event.wait(timeout=0.05)
- self._wake_event.clear()
-
- def _handle_next_batch(self) -> bool:
- now = datetime.now(timezone.utc)
- for pending_path in sorted(self._owned_batches(self._pending_dir)):
- try:
- payload = self._read_payload(pending_path)
- except (OSError, ValueError, json.JSONDecodeError):
- self._move_to_dead_letter(pending_path)
- continue
- next_retry_at = self._parse_datetime(str(payload.get("next_retry_at") or ""))
- if next_retry_at is not None and next_retry_at > now:
- continue
-
- inflight_path = self._inflight_dir / pending_path.name
- try:
- os.replace(pending_path, inflight_path)
- os.utime(inflight_path, None)
- except FileNotFoundError:
- continue
-
- try:
- status = self._post_payload(payload)
- except Exception:
- logger.exception(
- "OpenViking usage delivery failed unexpectedly, batch_id=%s",
- payload.get("batch_id"),
- )
- self._schedule_retry(inflight_path, payload)
- return True
- if 200 <= status < 300:
- try:
- inflight_path.unlink()
- except FileNotFoundError:
- pass
- return True
- if status == 413:
- self._split_batch(inflight_path, payload)
- return True
- if status in (400, 422):
- self._move_to_dead_letter(inflight_path)
- return True
-
- self._schedule_retry(inflight_path, payload)
- return True
- return False
-
- def _split_batch(self, inflight_path: Path, payload: dict[str, Any]) -> None:
- events = payload.get("events")
- if not isinstance(events, list) or len(events) < 2:
- self._move_to_dead_letter(inflight_path)
- return
- midpoint = len(events) // 2
- if not self._persist_batches(
- [events[:midpoint], events[midpoint:]],
- replace_path=inflight_path,
- ):
- self._move_to_dead_letter(inflight_path)
- self._wake_event.set()
-
- def _post_payload(self, payload: dict[str, Any]) -> int:
- request = urllib.request.Request(
- self._endpoint,
- data=json.dumps(
- payload,
- ensure_ascii=False,
- separators=(",", ":"),
- ).encode("utf-8"),
- headers={
- "Content-Type": "application/json",
- "V-Resource-Id": self._resource_id,
- },
- method="POST",
- )
- try:
- with urllib.request.urlopen(
- request,
- timeout=self._request_timeout_seconds,
- ) as response:
- response.read()
- return response.status
- except urllib.error.HTTPError as error:
- error.read()
- return error.code
- except (OSError, TimeoutError, urllib.error.URLError):
- return 0
-
- def _schedule_retry(self, inflight_path: Path, payload: dict[str, Any]) -> None:
- attempt = int(payload.get("attempt") or 0) + 1
- delay = min(
- self._retry_max_seconds,
- self._retry_base_seconds * (2 ** max(0, attempt - 1)),
- )
- if delay > 0:
- delay *= random.uniform(0.8, 1.2)
- payload["attempt"] = attempt
- payload["next_retry_at"] = (
- datetime.fromtimestamp(time.time() + delay, tz=timezone.utc)
- .isoformat(timespec="milliseconds")
- .replace("+00:00", "Z")
- )
- try:
- self._replace_payload(inflight_path, payload)
- os.replace(inflight_path, self._pending_dir / inflight_path.name)
- except FileNotFoundError:
- return
- self._wake_event.set()
-
- def _replace_payload(self, path: Path, payload: dict[str, Any]) -> None:
- temporary_path = path.with_name(f".{path.name}.tmp")
- encoded = json.dumps(
- payload,
- ensure_ascii=False,
- separators=(",", ":"),
- ).encode("utf-8")
- try:
- with temporary_path.open("wb") as output:
- output.write(encoded)
- output.flush()
- os.fsync(output.fileno())
- os.replace(temporary_path, path)
- self._fsync_directory(path.parent)
- finally:
- try:
- temporary_path.unlink()
- except FileNotFoundError:
- pass
-
- @staticmethod
- def _read_payload(path: Path) -> dict[str, Any]:
- with path.open("r", encoding="utf-8") as source:
- payload = json.load(source)
- if not isinstance(payload, dict):
- raise ValueError("outbox payload must be an object")
- return payload
-
- @staticmethod
- def _parse_datetime(value: str) -> datetime | None:
- if not value:
- return None
- try:
- return datetime.fromisoformat(value.replace("Z", "+00:00"))
- except ValueError:
- return None
-
- def _move_to_dead_letter(self, path: Path) -> None:
- try:
- os.replace(path, self._dead_letter_dir / path.name)
- except FileNotFoundError:
- return
-
- def _recover_stale_inflight(self) -> None:
- stale_before = time.time() - self._inflight_lease_seconds
- with self._outbox_lock:
- for inflight_path in self._owned_batches(self._inflight_dir):
- try:
- if inflight_path.stat().st_mtime > stale_before:
- continue
- os.replace(inflight_path, self._pending_dir / inflight_path.name)
- except FileNotFoundError:
- continue
-
- def _owned_batches(self, directory: Path) -> Iterator[Path]:
- return directory.glob(f"{self._outbox_scope}_*.json")
diff --git a/tests/server/test_agent_evolution_global_setting.py b/tests/server/test_agent_evolution_global_setting.py
index ec0c9b7e5..3c191a9a7 100644
--- a/tests/server/test_agent_evolution_global_setting.py
+++ b/tests/server/test_agent_evolution_global_setting.py
@@ -1,12 +1,20 @@
# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
# SPDX-License-Identifier: AGPL-3.0
+import json
+from types import SimpleNamespace
from unittest.mock import AsyncMock
import httpx
+from openviking.server.agent_evolution_config import AgentEvolutionConfigProvider
+from openviking.server.app import create_app
from openviking.server.config import AgentEvolutionConfig, ServerConfig, UserConfig
+from openviking.server.identity import RequestContext, Role
from openviking.service.session_service import SessionService
+from openviking.session import Session
+from openviking_cli.session.user_id import UserIdentifier
+from openviking_cli.utils.config import OPENVIKING_CONFIG_ENV
def test_agent_evolution_is_disabled_by_default():
@@ -19,10 +27,190 @@ def test_agent_evolution_can_be_enabled_for_the_server():
assert config.agent_evolution.enabled is True
-def test_embedded_session_service_preserves_agent_evolution_default():
+def test_embedded_session_service_preserves_agent_evolution_default(
+ tmp_path,
+ monkeypatch,
+):
+ config_path = tmp_path / "ov.conf"
+ config_path.write_text(
+ json.dumps({"server": {"agent_evolution": {"enabled": False}}}),
+ encoding="utf-8",
+ )
+ monkeypatch.setenv(OPENVIKING_CONFIG_ENV, str(config_path))
+
service = SessionService()
- assert service._agent_evolution_enabled is True
+ assert service.get_agent_evolution_enabled() is True
+
+
+def test_existing_session_observes_updated_runtime_value():
+ service = SessionService(viking_fs=object())
+ session = service.session(
+ RequestContext(user=UserIdentifier.the_default_user(), role=Role.ROOT)
+ )
+
+ service.set_agent_evolution_config(AgentEvolutionConfig(enabled=False))
+
+ assert session._agent_evolution_enabled_provider() is False
+
+
+def test_session_positional_usage_reporter_remains_compatible():
+ reporter = object()
+
+ session = Session(
+ object(),
+ None,
+ None,
+ None,
+ None,
+ "session-id",
+ None,
+ 8000,
+ None,
+ True,
+ reporter,
+ )
+
+ assert session._usage_reporter is reporter
+ assert session._agent_evolution_enabled_provider is None
+
+
+def test_agent_evolution_provider_reloads_config_file(tmp_path):
+ config_path = tmp_path / "ov.conf"
+ config_path.write_text(
+ json.dumps({"server": {"agent_evolution": {"enabled": False}}}),
+ encoding="utf-8",
+ )
+ provider = AgentEvolutionConfigProvider(
+ default_enabled=True,
+ config_path=config_path,
+ )
+
+ assert provider.is_enabled() is False
+
+ config_path.write_text(
+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),
+ encoding="utf-8",
+ )
+
+ assert provider.is_enabled() is True
+
+
+def test_session_service_reloads_from_resolved_server_config_path(tmp_path):
+ config_path = tmp_path / "ov.conf"
+ config_path.write_text(
+ json.dumps({"server": {"agent_evolution": {"enabled": False}}}),
+ encoding="utf-8",
+ )
+ service = SessionService()
+ service.set_agent_evolution_config(AgentEvolutionConfig(enabled=True))
+ service.set_agent_evolution_config_path(str(config_path))
+
+ assert service.get_agent_evolution_enabled() is False
+
+
+def test_http_app_wires_resolved_server_config_path(tmp_path):
+ config_path = tmp_path / "ov.conf"
+ config_path.write_text(
+ json.dumps({"server": {"agent_evolution": {"enabled": False}}}),
+ encoding="utf-8",
+ )
+ sessions = SessionService()
+
+ create_app(
+ config=ServerConfig(agent_evolution=AgentEvolutionConfig(enabled=True)),
+ service=SimpleNamespace(sessions=sessions),
+ config_path=str(config_path),
+ )
+
+ assert sessions.get_agent_evolution_enabled() is False
+
+ config_path.write_text(
+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),
+ encoding="utf-8",
+ )
+
+ assert sessions.get_agent_evolution_enabled() is True
+
+
+def test_agent_evolution_provider_applies_missing_section_default(tmp_path):
+ config_path = tmp_path / "ov.conf"
+ config_path.write_text(
+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),
+ encoding="utf-8",
+ )
+ provider = AgentEvolutionConfigProvider(
+ default_enabled=False,
+ config_path=config_path,
+ )
+ assert provider.is_enabled() is True
+
+ config_path.write_text(json.dumps({"server": {}}), encoding="utf-8")
+
+ assert provider.is_enabled() is False
+
+
+def test_agent_evolution_provider_uses_standard_config_loading(tmp_path, monkeypatch):
+ config_path = tmp_path / "ov.conf"
+ config_path.write_text(
+ '\ufeff{"server": {"agent_evolution": {"enabled": ${AGENT_EVOLUTION_ENABLED}}}}',
+ encoding="utf-8",
+ )
+ monkeypatch.setenv("AGENT_EVOLUTION_ENABLED", "true")
+ provider = AgentEvolutionConfigProvider(
+ default_enabled=False,
+ config_path=config_path,
+ )
+
+ assert provider.is_enabled() is True
+
+
+def test_agent_evolution_provider_keeps_last_valid_value(tmp_path):
+ config_path = tmp_path / "ov.conf"
+ config_path.write_text(
+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),
+ encoding="utf-8",
+ )
+ provider = AgentEvolutionConfigProvider(
+ default_enabled=False,
+ config_path=config_path,
+ )
+ assert provider.is_enabled() is True
+
+ config_path.write_text("{", encoding="utf-8")
+
+ assert provider.is_enabled() is True
+
+ config_path.write_text("[]", encoding="utf-8")
+
+ assert provider.is_enabled() is True
+
+
+def test_agent_evolution_provider_rejects_invalid_unrelated_server_config(tmp_path):
+ config_path = tmp_path / "ov.conf"
+ config_path.write_text(
+ json.dumps({"server": {"agent_evolution": {"enabled": True}}}),
+ encoding="utf-8",
+ )
+ provider = AgentEvolutionConfigProvider(
+ default_enabled=False,
+ config_path=config_path,
+ )
+ assert provider.is_enabled() is True
+
+ config_path.write_text(
+ json.dumps(
+ {
+ "server": {
+ "port": "not-an-integer",
+ "agent_evolution": {"enabled": False},
+ }
+ }
+ ),
+ encoding="utf-8",
+ )
+
+ assert provider.is_enabled() is True
def test_deprecated_user_agent_evolution_config_is_not_persisted():
@@ -40,6 +228,21 @@ async def test_agent_evolution_user_endpoint_is_not_registered(
assert response.status_code == 404
+async def test_agent_evolution_admin_endpoint_returns_runtime_value(
+ service,
+ client: httpx.AsyncClient,
+):
+ service.sessions.set_agent_evolution_config(AgentEvolutionConfig(enabled=True))
+
+ response = await client.get("/api/v1/admin/agent-evolution")
+
+ assert response.status_code == 200, response.text
+ assert response.json()["result"] == {
+ "enabled": True,
+ "account_id": "default",
+ }
+
+
async def test_manual_extract_respects_disabled_agent_evolution(
service,
client: httpx.AsyncClient,
diff --git a/tests/server/test_bootstrap.py b/tests/server/test_bootstrap.py
index 181e3c1e4..64a7f0198 100644
--- a/tests/server/test_bootstrap.py
+++ b/tests/server/test_bootstrap.py
@@ -26,7 +26,7 @@ def test_main_keeps_config_host_when_cli_host_is_omitted(monkeypatch):
monkeypatch.setattr(
bootstrap,
"create_app",
- lambda config: "app",
+ lambda config, **kwargs: "app",
)
monkeypatch.setattr(
bootstrap,
@@ -80,7 +80,7 @@ def test_main_coerces_cli_host_all_to_none(monkeypatch):
monkeypatch.setattr(
bootstrap,
"create_app",
- lambda config: "app",
+ lambda config, **kwargs: "app",
)
monkeypatch.setattr(
bootstrap,
@@ -128,7 +128,7 @@ def test_main_enables_bot_logging_when_with_bot_comes_from_config(monkeypatch):
bot_process = object()
monkeypatch.setattr(bootstrap, "load_server_config", lambda config_path: config)
- monkeypatch.setattr(bootstrap, "create_app", lambda config: "app")
+ monkeypatch.setattr(bootstrap, "create_app", lambda config, **kwargs: "app")
monkeypatch.setattr(bootstrap, "configure_uvicorn_logging", lambda: None)
monkeypatch.setattr(
OpenVikingConfigSingleton,
@@ -187,7 +187,7 @@ def test_bot_alias_propagates_resolved_config_to_workers(monkeypatch):
monkeypatch.setattr(bootstrap.argparse.ArgumentParser, "parse_args", parse_bot_alias)
monkeypatch.setattr(bootstrap, "load_server_config", lambda config_path: config)
- monkeypatch.setattr(bootstrap, "create_app", lambda config: "parent-app")
+ monkeypatch.setattr(bootstrap, "create_app", lambda config, **kwargs: "parent-app")
monkeypatch.setattr(bootstrap, "configure_uvicorn_logging", lambda: None)
monkeypatch.setattr(bootstrap, "_abort_if_port_in_use", lambda port, label: None)
monkeypatch.setattr(bootstrap, "_start_vikingbot_gateway", lambda *args, **kwargs: object())
@@ -221,15 +221,40 @@ def test_bot_alias_propagates_resolved_config_to_workers(monkeypatch):
def test_worker_factory_replays_bot_overrides(monkeypatch):
config = ServerConfig(with_bot=False, bot_api_url="http://from-config")
+ captured = {}
monkeypatch.setenv(app_module.WORKER_WITH_BOT_ENV, "1")
monkeypatch.setenv(app_module.WORKER_BOT_API_URL_ENV, "http://127.0.0.1:19000")
- monkeypatch.setattr(app_module, "load_server_config", lambda: config)
- monkeypatch.setattr(app_module, "create_app", lambda worker_config: worker_config)
+ monkeypatch.setattr(
+ app_module,
+ "resolve_config_path",
+ lambda config_path, env_var, default_name: "/tmp/worker-ov.conf",
+ )
+ monkeypatch.setattr(
+ app_module,
+ "load_server_config",
+ lambda config_path: captured.update({"load_path": config_path}) or config,
+ )
+ monkeypatch.setattr(
+ app_module,
+ "create_app",
+ lambda worker_config, config_path: captured.update(
+ {
+ "app_config": worker_config,
+ "app_config_path": config_path,
+ }
+ )
+ or worker_config,
+ )
worker_config = app_module.create_worker_app()
assert worker_config.with_bot is True
assert worker_config.bot_api_url == "http://127.0.0.1:19000"
+ assert captured == {
+ "load_path": "/tmp/worker-ov.conf",
+ "app_config": config,
+ "app_config_path": "/tmp/worker-ov.conf",
+ }
def test_resolve_queuefs_mount_point_defaults_to_shared():
diff --git a/tests/session/test_compressor_v3.py b/tests/session/test_compressor_v3.py
index dfce4eb70..fff4ffbe6 100644
--- a/tests/session/test_compressor_v3.py
+++ b/tests/session/test_compressor_v3.py
@@ -1495,7 +1495,10 @@ async def test_v3_training_links_case_to_trajectory_and_experience_via_trajector
assert any(link["from_uri"] == case_uri for link in exp_file.backlinks)
assert fs.commits == [
{
- "message": "Update experience memories from session commit archive_001",
+ "message": (
+ "Update experience memories from session commit "
+ "viking://user/u/sessions/session-1/history/archive_001"
+ ),
"paths": [exp_uri, deleted_exp_uri],
"ctx": _ctx(),
}
diff --git a/tests/session/test_session_commit.py b/tests/session/test_session_commit.py
index 9ed194543..04ed3b851 100644
--- a/tests/session/test_session_commit.py
+++ b/tests/session/test_session_commit.py
@@ -82,7 +82,7 @@ class TestCommit:
async def test_commit_default_disables_agent_memory_but_keeps_archive(
self, session_with_messages: Session
):
- session_with_messages._agent_evolution_enabled = False
+ session_with_messages._agent_evolution_enabled_provider = lambda: False
session_with_messages._session_compressor.extract_long_term_memories = AsyncMock(
return_value=[]
)
@@ -130,7 +130,7 @@ class TestCommit:
async def test_disabled_agent_evolution_keeps_working_memory(
self, session_with_messages: Session, monkeypatch
):
- session_with_messages._agent_evolution_enabled = False
+ session_with_messages._agent_evolution_enabled_provider = lambda: False
summary_called = False
async def fake_summary(_session, messages, latest_archive_overview=""):
diff --git a/tests/unit/usage_reporter/test_file_log_sink.py b/tests/unit/usage_reporter/test_file_log_sink.py
new file mode 100644
index 000000000..8faaf4e21
--- /dev/null
+++ b/tests/unit/usage_reporter/test_file_log_sink.py
@@ -0,0 +1,263 @@
+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
+# SPDX-License-Identifier: AGPL-3.0
+
+from __future__ import annotations
+
+import json
+import time
+
+import pytest
+
+from openviking.usage_reporter.file_log_sink import FileLogUsageSink
+
+
+class FakeUsageEvent:
+ def __init__(
+ self,
+ *,
+ event_id: str = "ue_recall",
+ event_type: str = "memory.recalled",
+ session_id: str = "session-1",
+ resource_uri: str = "viking://user/user-1/memories/experiences/exchange.md",
+ ) -> None:
+ self._record = {
+ "schema_version": "v1",
+ "event_id": event_id,
+ "event_type": event_type,
+ "account_id": "2101858484",
+ "user_id": "user-1",
+ "session_id": session_id,
+ "task_id": "task-1",
+ "occurred_at": "2026-07-27T04:00:00Z",
+ "resource_uri": resource_uri,
+ "resource_type": "experience",
+ "evidence": {
+ "archive_uri": ("viking://user/user-1/sessions/session-1/history/archive_001"),
+ "message_id": "msg-1",
+ "tool_call_id": "call-1",
+ "tool_name": "search_experience",
+ },
+ "attributes": {"note": "contains spaces\tand tabs"},
+ }
+
+ def to_dict(self) -> dict[str, object]:
+ return dict(self._record)
+
+
+def _parse_line(line: str) -> tuple[str, dict[str, object]]:
+ record = json.loads(line)
+ return record["key"], record["value"]
+
+
+async def test_file_log_sink_writes_original_kafka_key_and_value(tmp_path, monkeypatch):
+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")
+ log_path = tmp_path / "dedicated" / "usage.log"
+ sink = FileLogUsageSink(path=log_path)
+
+ try:
+ await sink.write(events=[FakeUsageEvent()])
+ finally:
+ sink.close()
+
+ lines = log_path.read_text(encoding="utf-8").splitlines()
+ assert len(lines) == 1
+ key, value = _parse_line(lines[0])
+ assert key == "ov-test|2101858484|user-1|viking://user/user-1/memories/experiences/exchange.md"
+ assert value == {
+ "count_name": "experience.recall.count",
+ "op_type": "add",
+ "amount": 1.0,
+ "timestamp": 1785124800000,
+ "unique_id": "ue_recall",
+ "tags": {
+ "account_id": "2101858484",
+ "user_id": "user-1",
+ "resource_uri": "viking://user/user-1/memories/experiences/exchange.md",
+ "resource_type": "experience",
+ },
+ "extra": {
+ "archive_uri": "viking://user/user-1/sessions/session-1/history/archive_001",
+ "message_id": "msg-1",
+ "tool_call_id": "call-1",
+ "tool_name": "search_experience",
+ "session_id": "session-1",
+ "task_id": "task-1",
+ "attributes": {"note": "contains spaces\tand tabs"},
+ },
+ "prefix": "ov-test",
+ }
+
+
+async def test_file_log_sink_message_key_falls_back_to_session_id(tmp_path, monkeypatch):
+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")
+ log_path = tmp_path / "usage.log"
+ sink = FileLogUsageSink(path=log_path)
+
+ try:
+ await sink.write(events=[FakeUsageEvent(resource_uri="", session_id="session-fallback")])
+ finally:
+ sink.close()
+
+ key, _value = _parse_line(log_path.read_text(encoding="utf-8").strip())
+ assert key == "ov-test|2101858484|user-1|session-fallback"
+
+
+async def test_file_log_sink_preserves_equals_signs_in_key(tmp_path, monkeypatch):
+ monkeypatch.setenv("OV_RESOURCE_ID", "ov=test")
+ log_path = tmp_path / "usage.log"
+ sink = FileLogUsageSink(path=log_path)
+
+ try:
+ await sink.write(
+ events=[
+ FakeUsageEvent(
+ resource_uri=("viking://user/user-1/memories/experiences/exchange=delivered.md")
+ )
+ ]
+ )
+ finally:
+ sink.close()
+
+ key, value = _parse_line(log_path.read_text(encoding="utf-8").strip())
+ assert key == (
+ "ov=test|2101858484|user-1|viking://user/user-1/memories/experiences/exchange=delivered.md"
+ )
+ assert value["prefix"] == "ov=test"
+ assert (
+ value["tags"]["resource_uri"]
+ == "viking://user/user-1/memories/experiences/exchange=delivered.md"
+ )
+
+
+async def test_file_log_sink_preserves_records_when_workers_roll_over(tmp_path, monkeypatch):
+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")
+ log_path = tmp_path / "usage.log"
+ first = FileLogUsageSink(path=log_path)
+ second = FileLogUsageSink(path=log_path)
+
+ try:
+ await first.write(events=[FakeUsageEvent(event_id="ue_first_before")])
+ await second.write(events=[FakeUsageEvent(event_id="ue_second_before")])
+
+ first._handler.rolloverAt = 0
+ second._handler.rolloverAt = 0
+ await first.write(events=[FakeUsageEvent(event_id="ue_first_after")])
+ await second.write(events=[FakeUsageEvent(event_id="ue_second_after")])
+ finally:
+ first.close()
+ second.close()
+
+ unique_ids = []
+ for path in tmp_path.glob("usage.log*"):
+ for line in path.read_text(encoding="utf-8").splitlines():
+ _key, value = _parse_line(line)
+ unique_ids.append(value["unique_id"])
+ assert sorted(unique_ids) == [
+ "ue_first_after",
+ "ue_first_before",
+ "ue_second_after",
+ "ue_second_before",
+ ]
+
+
+async def test_closed_handle_preserves_overdue_rollover_deadline(tmp_path, monkeypatch):
+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")
+ log_path = tmp_path / "usage.log"
+ sink = FileLogUsageSink(path=log_path)
+
+ try:
+ await sink.write(events=[FakeUsageEvent(event_id="ue_before")])
+ sink._handler.stream.close()
+ sink._handler.stream = None
+ sink._handler.rolloverAt = int(time.time()) - 1
+
+ await sink.write(events=[FakeUsageEvent(event_id="ue_after")])
+ finally:
+ sink.close()
+
+ log_files = list(tmp_path.glob("usage.log*"))
+ assert len(log_files) == 2
+ unique_ids = []
+ for path in log_files:
+ for line in path.read_text(encoding="utf-8").splitlines():
+ _key, value = _parse_line(line)
+ unique_ids.append(value["unique_id"])
+ assert sorted(unique_ids) == ["ue_after", "ue_before"]
+
+
+def test_file_log_sink_uses_hourly_utc_rotation(tmp_path, monkeypatch):
+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")
+ sink = FileLogUsageSink(
+ path=tmp_path / "usage.log",
+ rotation_interval_hours=1,
+ backup_count=168,
+ )
+
+ try:
+ assert sink._handler.when == "H"
+ assert sink._handler.interval == 60 * 60
+ assert sink._handler.backupCount == 168
+ assert sink._handler.utc is True
+ assert sink._handler.suffix == "%Y-%m-%d_%H"
+ finally:
+ sink.close()
+
+
+ ("environment", "kwargs", "message"),
+ [
+ ({}, {"path": "usage.log"}, "OV_RESOURCE_ID is required"),
+ ({"OV_RESOURCE_ID": "ov-test"}, {"path": ""}, "path is required"),
+ (
+ {"OV_RESOURCE_ID": "ov-test"},
+ {"path": "usage.log", "rotation_interval_hours": 0},
+ "rotation_interval_hours must be positive",
+ ),
+ (
+ {"OV_RESOURCE_ID": "ov-test"},
+ {"path": "usage.log", "backup_count": -1},
+ "backup_count must be non-negative",
+ ),
+ ],
+)
+def test_file_log_sink_validates_config(
+ monkeypatch,
+ environment,
+ kwargs,
+ message,
+):
+ monkeypatch.delenv("OV_RESOURCE_ID", raising=False)
+ for key, value in environment.items():
+ monkeypatch.setenv(key, value)
+
+ with pytest.raises(ValueError, match=message):
+ FileLogUsageSink(**kwargs)
+
+
+async def test_file_log_sink_rejects_unsupported_event(tmp_path, monkeypatch):
+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")
+ sink = FileLogUsageSink(path=tmp_path / "usage.log")
+
+ try:
+ with pytest.raises(ValueError, match="unsupported usage event type"):
+ await sink.write(events=[FakeUsageEvent(event_type="memory.unknown")])
+ finally:
+ sink.close()
+
+
+async def test_file_log_sink_requires_event_id(tmp_path, monkeypatch):
+ monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")
+ sink = FileLogUsageSink(path=tmp_path / "usage.log")
+
+ try:
+ with pytest.raises(ValueError, match="event_id is required"):
+ await sink.write(events=[FakeUsageEvent(event_id="")])
+ finally:
+ sink.close()
diff --git a/tests/unit/usage_reporter/test_http_sink.py b/tests/unit/usage_reporter/test_http_sink.py
deleted file mode 100644
index 90449ec01..000000000
--- a/tests/unit/usage_reporter/test_http_sink.py
+++ /dev/null
@@ -1,692 +0,0 @@
-# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
-# SPDX-License-Identifier: AGPL-3.0
-
-from __future__ import annotations
-
-import asyncio
-import hashlib
-import json
-import os
-import tempfile
-import threading
-import time
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from pathlib import Path
-from unittest import mock
-
-import pytest
-
-from openviking.usage_reporter.http_sink import HttpUsageSink
-
-
-class FakeUsageEvent:
- def __init__(
- self,
- event_id: str,
- session_id: str,
- attributes: dict[str, object] | None = None,
- *,
- event_type: str = "memory.recalled",
- occurred_at: str = "2026-07-27T04:00:00Z",
- resource_uri: str = "viking://user/user-1/memories/experiences/exchange.md",
- resource_type: str = "experience",
- task_id: str | None = None,
- evidence: dict[str, object] | None = None,
- ) -> None:
- self._record = {
- "schema_version": "v1",
- "event_id": event_id,
- "event_type": event_type,
- "account_id": "2101858484",
- "user_id": "user-1",
- "session_id": session_id,
- "task_id": task_id,
- "occurred_at": occurred_at,
- "resource_uri": resource_uri,
- "resource_type": resource_type,
- "evidence": evidence or {},
- "attributes": attributes or {},
- }
-
- def to_dict(self) -> dict[str, object]:
- return dict(self._record)
-
-
-class UsageRequestHandler(BaseHTTPRequestHandler):
- def do_POST(self) -> None:
- content_length = int(self.headers["Content-Length"])
- body = json.loads(self.rfile.read(content_length))
- self.server.requests.append(body)
- self.server.resource_ids.append(self.headers["V-Resource-Id"])
- response_status = self.server.response_statuses.pop(0)
- self.send_response(response_status)
- self.end_headers()
- self.wfile.write(b"{}")
-
- def log_message(self, format: str, *args: object) -> None:
- return
-
-
-class UsageHTTPServer(ThreadingHTTPServer):
- requests: list[dict[str, object]]
- resource_ids: list[str]
- response_statuses: list[int]
-
-
-def wait_until(predicate: object, timeout: float = 3.0) -> None:
- deadline = time.monotonic() + timeout
- while time.monotonic() < deadline:
- if predicate():
- return
- time.sleep(0.01)
- raise AssertionError("condition was not met before timeout")
-
-
-class TestHttpUsageSink:
- def test_write_persists_recall_as_count_record(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(outbox_dir),
- )
- asyncio.run(
- sink.write(
- events=[
- FakeUsageEvent(
- "ue_recall",
- "session-1",
- {"rank": 1},
- task_id="task-1",
- evidence={
- "archive_uri": (
- "viking://user/user-1/sessions/session-1/"
- "history/archive_001"
- ),
- "message_id": "msg-1",
- "tool_call_id": "call-1",
- "tool_name": "search_experience",
- },
- )
- ]
- )
- )
-
- pending_path = next((outbox_dir / "pending").glob("*.json"))
- payload = json.loads(pending_path.read_text(encoding="utf-8"))
- assert payload["events"] == [
- {
- "CountName": "experience.recall.count",
- "OpType": "add",
- "amount": 1.0,
- "timestamp": 1785124800000,
- "uniqueId": "ue_recall",
- "tags": {
- "account_id": "2101858484",
- "user_id": "user-1",
- "resource_uri": ("viking://user/user-1/memories/experiences/exchange.md"),
- "resource_type": "experience",
- },
- "extra": {
- "session_id": "session-1",
- "task_id": "task-1",
- "archive_uri": (
- "viking://user/user-1/sessions/session-1/history/archive_001"
- ),
- "message_id": "msg-1",
- "tool_call_id": "call-1",
- "tool_name": "search_experience",
- "attributes": {"rank": 1},
- },
- }
- ]
-
- def test_write_maps_injected_event_to_count_record(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(outbox_dir),
- )
- asyncio.run(
- sink.write(
- events=[
- FakeUsageEvent(
- "ue_inject",
- "session-1",
- event_type="memory.injected",
- )
- ]
- )
- )
-
- pending_path = next((outbox_dir / "pending").glob("*.json"))
- payload = json.loads(pending_path.read_text(encoding="utf-8"))
- assert payload["events"][0]["CountName"] == "experience.inject.count"
-
- def test_write_omits_empty_optional_count_record_extra_fields(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(outbox_dir),
- )
- asyncio.run(sink.write(events=[FakeUsageEvent("ue_minimal", "session-1")]))
-
- pending_path = next((outbox_dir / "pending").glob("*.json"))
- payload = json.loads(pending_path.read_text(encoding="utf-8"))
- assert payload["events"][0]["extra"] == {"session_id": "session-1"}
-
- def test_write_rejects_unknown_usage_event_type(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(Path(data_dir) / ".usage_outbox"),
- )
- with pytest.raises(
- ValueError,
- match="unsupported usage event type: memory.unknown",
- ):
- asyncio.run(
- sink.write(
- events=[
- FakeUsageEvent(
- "ue_unknown",
- "session-1",
- event_type="memory.unknown",
- )
- ]
- )
- )
-
- def test_write_atomically_persists_stable_batch(self) -> None:
- events = [
- FakeUsageEvent("ue_1", "session-1"),
- FakeUsageEvent("ue_2", "session-1"),
- ]
- expected_digest = hashlib.sha256(b"ue_1\nue_2").hexdigest()
-
- with tempfile.TemporaryDirectory() as data_dir:
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/api/openviking/ReportOpenVikingUsage",
- outbox_dir=str(Path(data_dir) / ".usage_outbox"),
- )
- asyncio.run(sink.write(events=events))
-
- pending_files = list((Path(data_dir) / ".usage_outbox" / "pending").glob("*.json"))
- temporary_files = list((Path(data_dir) / ".usage_outbox" / "pending").glob("*.tmp"))
-
- assert len(pending_files) == 1
- assert temporary_files == []
- payload = json.loads(pending_files[0].read_text(encoding="utf-8"))
- assert payload["schema_version"] == "v1"
- assert payload["resource_id"] == "ov-test"
- assert payload["batch_id"] == f"ub_{expected_digest}"
- assert [event["uniqueId"] for event in payload["events"]] == [
- "ue_1",
- "ue_2",
- ]
- assert all("prefix" not in event for event in payload["events"])
-
- def test_write_splits_batches_before_http_limit(self) -> None:
- events = [
- FakeUsageEvent("ue_large_1", "session-large", {"text": "a" * 700}),
- FakeUsageEvent("ue_large_2", "session-large", {"text": "b" * 700}),
- ]
-
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(outbox_dir),
- max_batch_bytes=1024,
- )
- asyncio.run(sink.write(events=events))
-
- batches = [
- json.loads(path.read_text(encoding="utf-8"))
- for path in (outbox_dir / "pending").glob("*.json")
- ]
- assert len(batches) == 2
- assert sorted(len(batch["events"]) for batch in batches) == [1, 1]
-
- def test_worker_retries_then_deletes_acknowledged_batch(self) -> None:
- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)
- server.requests = []
- server.resource_ids = []
- server.response_statuses = [503, 200]
- server_thread = threading.Thread(target=server.serve_forever, daemon=True)
- server_thread.start()
-
- try:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- sink = HttpUsageSink(
- endpoint=f"http://127.0.0.1:{server.server_port}/usage",
- outbox_dir=str(outbox_dir),
- request_timeout_seconds=0.5,
- retry_base_seconds=0.01,
- retry_max_seconds=0.02,
- )
- asyncio.run(sink.write(events=[FakeUsageEvent("ue_retry", "session-retry")]))
- wait_until(lambda: len(server.requests) == 2)
- wait_until(
- lambda: (
- not list((outbox_dir / "pending").glob("*.json"))
- and not list((outbox_dir / "inflight").glob("*.json"))
- )
- )
- sink.close()
-
- assert [request["batch_id"] for request in server.requests] == [
- server.requests[0]["batch_id"],
- server.requests[0]["batch_id"],
- ]
- assert server.resource_ids == ["ov-test", "ov-test"]
- finally:
- server.shutdown()
- server.server_close()
- server_thread.join(timeout=1)
-
- def test_worker_accepts_any_successful_http_status(self) -> None:
- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)
- server.requests = []
- server.resource_ids = []
- server.response_statuses = [204]
- server_thread = threading.Thread(target=server.serve_forever, daemon=True)
- server_thread.start()
-
- try:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- sink = HttpUsageSink(
- endpoint=f"http://127.0.0.1:{server.server_port}/usage",
- outbox_dir=str(outbox_dir),
- request_timeout_seconds=0.5,
- )
- asyncio.run(sink.write(events=[FakeUsageEvent("ue_204", "session-success")]))
- wait_until(lambda: len(server.requests) == 1)
- wait_until(
- lambda: (
- not list((outbox_dir / "pending").glob("*.json"))
- and not list((outbox_dir / "inflight").glob("*.json"))
- )
- )
- sink.close()
- finally:
- server.shutdown()
- server.server_close()
- server_thread.join(timeout=1)
-
- def test_outbox_capacity_evicts_oldest_pending_batches(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(outbox_dir),
- max_outbox_bytes=1800,
- )
- for index in range(4):
- asyncio.run(
- sink.write(
- events=[
- FakeUsageEvent(
- f"ue_capacity_{index}",
- "session-capacity",
- {"text": str(index) * 600},
- )
- ]
- )
- )
-
- pending_files = list((outbox_dir / "pending").glob("*.json"))
- total_bytes = sum(path.stat().st_size for path in pending_files)
- persisted_ids = {
- event["uniqueId"]
- for path in pending_files
- for event in json.loads(path.read_text(encoding="utf-8"))["events"]
- }
- assert total_bytes <= 1800
- assert "ue_capacity_0" not in persisted_ids
- assert "ue_capacity_3" in persisted_ids
-
- def test_outbox_capacity_counts_inflight_without_evicting_it(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(outbox_dir),
- max_outbox_bytes=1800,
- )
- inflight_path = outbox_dir / "inflight" / f"{sink._outbox_scope}_active.json"
- inflight_path.write_bytes(b"x" * 1000)
- asyncio.run(
- sink.write(
- events=[
- FakeUsageEvent(
- "ue_while_inflight",
- "session-capacity",
- {"text": "x" * 900},
- )
- ]
- )
- )
-
- assert inflight_path.exists()
- assert list((outbox_dir / "pending").glob("*.json")) == []
-
- def test_default_outbox_directory_uses_current_user_home(self) -> None:
- with tempfile.TemporaryDirectory() as home_dir:
- expected = Path(home_dir) / ".openviking" / "data" / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(Path, "home", return_value=Path(home_dir)):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(endpoint="http://127.0.0.1:1/usage")
-
- assert sink._outbox_dir == expected
- assert expected.is_dir()
-
- def test_shared_outbox_root_isolated_by_endpoint_and_resource(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_root = Path(data_dir) / ".usage_outbox"
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "resource-a"}, clear=False):
- sink_a = HttpUsageSink(
- endpoint="http://collector-a.example/usage",
- outbox_dir=str(outbox_root),
- )
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "resource-b"}, clear=False):
- other_resource_sink = HttpUsageSink(
- endpoint="http://collector-a.example/usage",
- outbox_dir=str(outbox_root),
- )
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "resource-a"}, clear=False):
- other_endpoint_sink = HttpUsageSink(
- endpoint="http://collector-b.example/usage",
- outbox_dir=str(outbox_root),
- )
-
- asyncio.run(sink_a.write(events=[FakeUsageEvent("ue_a", "session-a")]))
- sink_a._post_payload = mock.Mock(return_value=200)
- other_resource_sink._post_payload = mock.Mock(return_value=200)
- other_endpoint_sink._post_payload = mock.Mock(return_value=200)
-
- assert other_resource_sink._handle_next_batch() is False
- other_resource_sink._post_payload.assert_not_called()
- assert other_endpoint_sink._handle_next_batch() is False
- other_endpoint_sink._post_payload.assert_not_called()
- assert sink_a._handle_next_batch() is True
- sink_a._post_payload.assert_called_once()
-
- def test_worker_splits_payload_rejected_as_too_large(self) -> None:
- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)
- server.requests = []
- server.resource_ids = []
- server.response_statuses = [413, 200, 200]
- server_thread = threading.Thread(target=server.serve_forever, daemon=True)
- server_thread.start()
-
- try:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- sink = HttpUsageSink(
- endpoint=f"http://127.0.0.1:{server.server_port}/usage",
- outbox_dir=str(outbox_dir),
- request_timeout_seconds=0.5,
- )
- asyncio.run(
- sink.write(
- events=[
- FakeUsageEvent("ue_left", "session-split"),
- FakeUsageEvent("ue_right", "session-split"),
- ]
- )
- )
- wait_until(lambda: len(server.requests) == 3)
- wait_until(
- lambda: (
- not list((outbox_dir / "pending").glob("*.json"))
- and not list((outbox_dir / "inflight").glob("*.json"))
- )
- )
- sink.close()
-
- original_batch_id = server.requests[0]["batch_id"]
- child_batch_ids = {request["batch_id"] for request in server.requests[1:]}
- assert [len(request["events"]) for request in server.requests] == [2, 1, 1]
- assert original_batch_id not in child_batch_ids
- assert len(child_batch_ids) == 2
- finally:
- server.shutdown()
- server.server_close()
- server_thread.join(timeout=1)
-
- def test_split_replaces_parent_without_evicting_child_batches(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- events = [
- FakeUsageEvent("ue_split_left", "session-split", {"text": "a" * 500}),
- FakeUsageEvent("ue_split_right", "session-split", {"text": "b" * 500}),
- ]
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(outbox_dir),
- max_outbox_bytes=10_000,
- )
- asyncio.run(sink.write(events=events))
-
- parent_path = next((outbox_dir / "pending").glob("*.json"))
- payload = json.loads(parent_path.read_text(encoding="utf-8"))
- inflight_path = outbox_dir / "inflight" / parent_path.name
- os.replace(parent_path, inflight_path)
- child_size = sum(
- sink._encoded_payload_size([event]) for event in payload["events"]
- )
- sink._max_outbox_bytes = child_size + 50
-
- sink._split_batch(inflight_path, payload)
-
- children = [
- json.loads(path.read_text(encoding="utf-8"))
- for path in (outbox_dir / "pending").glob("*.json")
- ]
- assert not inflight_path.exists()
- assert len(children) == 2
- assert {event["uniqueId"] for child in children for event in child["events"]} == {
- "ue_split_left",
- "ue_split_right",
- }
-
- def test_worker_moves_unrecoverable_payload_to_dead_letter(self) -> None:
- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)
- server.requests = []
- server.resource_ids = []
- server.response_statuses = [422]
- server_thread = threading.Thread(target=server.serve_forever, daemon=True)
- server_thread.start()
-
- try:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- sink = HttpUsageSink(
- endpoint=f"http://127.0.0.1:{server.server_port}/usage",
- outbox_dir=str(outbox_dir),
- )
- asyncio.run(sink.write(events=[FakeUsageEvent("ue_bad", "session-bad")]))
- wait_until(lambda: len(list((outbox_dir / "dead_letter").glob("*.json"))) == 1)
- sink.close()
-
- assert len(server.requests) == 1
- assert not list((outbox_dir / "pending").glob("*.json"))
- finally:
- server.shutdown()
- server.server_close()
- server_thread.join(timeout=1)
-
- def test_startup_recovers_only_stale_inflight_batch(self) -> None:
- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)
- server.requests = []
- server.resource_ids = []
- server.response_statuses = [200]
- server_thread = threading.Thread(target=server.serve_forever, daemon=True)
- server_thread.start()
-
- try:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- first_sink = HttpUsageSink(
- endpoint=f"http://127.0.0.1:{server.server_port}/usage",
- outbox_dir=str(outbox_dir),
- )
- asyncio.run(
- first_sink.write(events=[FakeUsageEvent("ue_stale", "session-stale")])
- )
- pending_path = next((outbox_dir / "pending").glob("*.json"))
- inflight_path = outbox_dir / "inflight" / pending_path.name
- os.replace(pending_path, inflight_path)
- stale_time = time.time() - 10
- os.utime(inflight_path, (stale_time, stale_time))
-
- second_sink = HttpUsageSink(
- endpoint=f"http://127.0.0.1:{server.server_port}/usage",
- outbox_dir=str(outbox_dir),
- request_timeout_seconds=0.5,
- inflight_lease_seconds=1,
- )
- wait_until(lambda: len(server.requests) == 1)
- wait_until(lambda: not inflight_path.exists())
- second_sink.close()
- finally:
- server.shutdown()
- server.server_close()
- server_thread.join(timeout=1)
-
- def test_worker_recovers_inflight_batch_when_lease_expires_after_startup(self) -> None:
- server = UsageHTTPServer(("127.0.0.1", 0), UsageRequestHandler)
- server.requests = []
- server.resource_ids = []
- server.response_statuses = [200]
- server_thread = threading.Thread(target=server.serve_forever, daemon=True)
- server_thread.start()
-
- try:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- first_sink = HttpUsageSink(
- endpoint=f"http://127.0.0.1:{server.server_port}/usage",
- outbox_dir=str(outbox_dir),
- )
- asyncio.run(
- first_sink.write(events=[FakeUsageEvent("ue_fresh", "session-fresh")])
- )
- pending_path = next((outbox_dir / "pending").glob("*.json"))
- inflight_path = outbox_dir / "inflight" / pending_path.name
- os.replace(pending_path, inflight_path)
-
- second_sink = HttpUsageSink(
- endpoint=f"http://127.0.0.1:{server.server_port}/usage",
- outbox_dir=str(outbox_dir),
- request_timeout_seconds=0.01,
- inflight_lease_seconds=0.05,
- )
- wait_until(lambda: len(server.requests) == 1)
- wait_until(lambda: not inflight_path.exists())
- second_sink.close()
- finally:
- server.shutdown()
- server.server_close()
- server_thread.join(timeout=1)
-
- def test_worker_keeps_batch_after_unexpected_delivery_exception(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(outbox_dir),
- retry_base_seconds=0.01,
- retry_max_seconds=0.02,
- )
- attempts = 0
-
- def flaky_post(payload: dict[str, object]) -> int:
- nonlocal attempts
- attempts += 1
- if attempts == 1:
- raise RuntimeError("unexpected delivery failure")
- return 200
-
- sink._post_payload = flaky_post
- with mock.patch(
- "openviking.usage_reporter.http_sink.logger.exception"
- ) as log_exception:
- asyncio.run(
- sink.write(events=[FakeUsageEvent("ue_exception", "session-exception")])
- )
- wait_until(lambda: attempts == 2)
- wait_until(
- lambda: (
- not list((outbox_dir / "pending").glob("*.json"))
- and not list((outbox_dir / "inflight").glob("*.json"))
- )
- )
- sink.close()
- assert log_exception.call_count == 1
- assert (
- "usage delivery failed unexpectedly"
- in log_exception.call_args.args[0].lower()
- )
-
- def test_claiming_pending_batch_refreshes_inflight_lease(self) -> None:
- with tempfile.TemporaryDirectory() as data_dir:
- outbox_dir = Path(data_dir) / ".usage_outbox"
- with mock.patch.dict(os.environ, {"OV_RESOURCE_ID": "ov-test"}, clear=False):
- with mock.patch.object(HttpUsageSink, "_start_worker"):
- sink = HttpUsageSink(
- endpoint="http://127.0.0.1:1/usage",
- outbox_dir=str(outbox_dir),
- request_timeout_seconds=0.01,
- inflight_lease_seconds=0.05,
- )
- asyncio.run(sink.write(events=[FakeUsageEvent("ue_claim", "session-claim")]))
- pending_path = next((outbox_dir / "pending").glob("*.json"))
- stale_time = time.time() - 60
- os.utime(pending_path, (stale_time, stale_time))
- claimed_mtime = 0.0
-
- def record_claim(_payload: dict[str, object]) -> int:
- nonlocal claimed_mtime
- inflight_path = outbox_dir / "inflight" / pending_path.name
- claimed_mtime = inflight_path.stat().st_mtime
- return 200
-
- sink._post_payload = record_claim
- assert sink._handle_next_batch() is True
-
- assert claimed_mtime >= time.time() - 1
diff --git a/tests/unit/usage_reporter/test_usage_reporter_config.py b/tests/unit/usage_reporter/test_usage_reporter_config.py
index 8b631407e..49f13fcff 100644
--- a/tests/unit/usage_reporter/test_usage_reporter_config.py
+++ b/tests/unit/usage_reporter/test_usage_reporter_config.py
@@ -9,9 +9,8 @@ import pytest
from openviking.server.app import create_app
from openviking.server.config import ServerConfig, UsageReporterConfig
-from openviking.usage_reporter import UsageContext, UsageEvent
+from openviking.usage_reporter import FileLogUsageSink, UsageContext, UsageEvent
from openviking.usage_reporter.config import build_usage_reporter
-from openviking.usage_reporter.http_sink import HttpUsageSink
@pytest.mark.asyncio
@@ -80,8 +79,9 @@ class CustomUsageSink:
assert "source" not in payload
-def test_builtin_http_sink_is_built_from_config(tmp_path, monkeypatch):
+def test_builtin_file_log_sink_is_built_from_config(tmp_path, monkeypatch):
monkeypatch.setenv("OV_RESOURCE_ID", "ov-test")
+ log_path = tmp_path / "usage" / "usage.log"
reporter = build_usage_reporter(
UsageReporterConfig(
@@ -89,10 +89,10 @@ def test_builtin_http_sink_is_built_from_config(tmp_path, monkeypatch):
extractors=["memory_usage"],
sinks=[
{
- "type": "http",
+ "type": "file_log",
"config": {
- "endpoint": "http://127.0.0.1:1/usage",
- "outbox_dir": str(tmp_path / ".usage_outbox"),
+ "path": str(log_path),
+ "rotation_interval_hours": 1,
},
}
],
@@ -101,7 +101,8 @@ def test_builtin_http_sink_is_built_from_config(tmp_path, monkeypatch):
assert reporter is not None
assert len(reporter.sinks) == 1
- assert isinstance(reporter.sinks[0], HttpUsageSink)
+ assert isinstance(reporter.sinks[0], FileLogUsageSink)
+ assert reporter.sinks[0].path == log_path
reporter.sinks[0].close()