Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
Feature/memory opt (#1099)
chenjw22e42ebc4674
diff --git a/openviking/prompts/templates/memory/cases.yaml b/openviking/prompts/templates/memory/cases.yamlindex 50d49af97..1475b6525 100644--- a/openviking/prompts/templates/memory/cases.yaml+++ b/openviking/prompts/templates/memory/cases.yaml@@ -5,8 +5,8 @@ description: |Cases should be about specific problems that have clear solutions.Each case should include: what the problem was (symptoms, error messages, context), what the solution was (steps taken, principles used), and why it worked.Case names should be in "Problem → Solution" format to make them easily searchable.-directory: "viking://agent/{agent_space}/memories/cases"-filename_template: "{case_name}.md"+directory: "viking://agent/{{ agent_space }}/memories/cases"+filename_template: "{{ case_name }}.md"enabled: falsefields:- name: case_namediff --git a/openviking/prompts/templates/memory/entities.yaml b/openviking/prompts/templates/memory/entities.yamlindex 3b2f693a9..579da2442 100644--- a/openviking/prompts/templates/memory/entities.yaml+++ b/openviking/prompts/templates/memory/entities.yaml@@ -5,8 +5,8 @@ description: |Relative path format: ../entities/entity_name.md- Example: [skin_itching](../entities/skin_itching.md) → see dermatologist- Cards should be rich and distributed - avoid putting all info in one card.-directory: "viking://user/{user_space}/memories/entities"-filename_template: "{name}.md"+directory: "viking://user/{{ user_space }}/memories/entities"+filename_template: "{{ name }}.md"enabled: truefields:diff --git a/openviking/prompts/templates/memory/events.yaml b/openviking/prompts/templates/memory/events.yamlindex 11bc6289f..0c00a8f56 100644--- a/openviking/prompts/templates/memory/events.yaml+++ b/openviking/prompts/templates/memory/events.yaml@@ -8,20 +8,15 @@ description: |- Use a third-person perspective.- If possible, combine the user's current behavior and reactions to speculate on the user's possible thoughts or actions.- Describe the complete content of an event within a single event as much as possible; do not split one event into multiple parts.-directory: "viking://user/{user_space}/memories/events"-filename_template: "{event_time}_{event_name}.md"+directory: "viking://user/{{ user_space }}/memories/events"+filename_template: "{{ extract_context.get_first_message_time_from_ranges(ranges) }}_{{ event_name }}.md"enabled: true# 操作模式:add_only 表示只新增记忆,不需要查看之前的记忆列表# upsert 表示新增或更新(默认行为)operation_mode: "add_only"content_template: |- {% set msg_range = extract_context.read_message_ranges(ranges|default('')) if extract_context else None %}- {% set first_time = msg_range.first_message_time() if msg_range else None %}- time: {{event_time|default(first_time if first_time else 'N/A')}}- {% if extract_context %}- {{ msg_range.pretty_print() if msg_range else '' }}- {% endif %}-+ time: {{extract_context.get_first_message_time_with_weekday_from_ranges(ranges|default(''))|default('N/A')}}+ {{ extract_context.read_message_ranges(ranges|default('')).pretty_print() if ranges else '' }}fields:- name: event_name@@ -38,13 +33,8 @@ fields:- name: summarytype: stringdescription: |- Based on the content of the above fields, compile a description to outline the complete Fact content, in English-- - name: event_time- type: string- description: |- Time when the event occurred, format “2026-03-17” / “2026-03” / “2026”. If unknown, use current time.- merge_op: immutable+ Based on the content of the above fields, compile a description to outline the complete Fact content.+ Use specific year-month-day dates instead of vague terms such as yesterday, last week, and tomorrow.- name: rangestype: stringdiff --git a/openviking/prompts/templates/memory/patterns.yaml b/openviking/prompts/templates/memory/patterns.yamlindex 6b6ba73b4..de9f3dcf1 100644--- a/openviking/prompts/templates/memory/patterns.yaml+++ b/openviking/prompts/templates/memory/patterns.yaml@@ -5,8 +5,8 @@ description: |Patterns should be about: how to approach certain types of tasks, what steps to follow, what considerations to keep in mind.Each pattern should include: trigger conditions (when to use this pattern), process steps (what to do), and considerations (what to watch out for).Pattern names should be in "Process name: Step description" format.-directory: "viking://agent/{agent_space}/memories/patterns"-filename_template: "{pattern_name}.md"+directory: "viking://agent/{{ agent_space }}/memories/patterns"+filename_template: "{{ pattern_name }}.md"enabled: falsefields:- name: pattern_namediff --git a/openviking/prompts/templates/memory/preferences.yaml b/openviking/prompts/templates/memory/preferences.yamlindex 98ec6c7e6..ae3f841fa 100644--- a/openviking/prompts/templates/memory/preferences.yaml+++ b/openviking/prompts/templates/memory/preferences.yaml@@ -5,8 +5,8 @@ description: |Each preference should be about a specific topic (not generic).Topics can be: code style, communication style, tools, workflow, food, commute, etc.Store different topics as separate memory files, do NOT mix unrelated preferences.-directory: "viking://user/{user_space}/memories/preferences"-filename_template: "{user}_{topic}.md"+directory: "viking://user/{{ user_space }}/memories/preferences"+filename_template: "{{ user }}_{{ topic }}.md"enabled: truefields:- name: userdiff --git a/openviking/prompts/templates/memory/profile.yaml b/openviking/prompts/templates/memory/profile.yamlindex 9bc8dcc09..3bc5c2896 100644--- a/openviking/prompts/templates/memory/profile.yaml+++ b/openviking/prompts/templates/memory/profile.yaml@@ -4,7 +4,7 @@ description: |Extract relatively stable personal attributes that define the user's identity, work style, and preferences.Include: profession, experience level, technical background, communication style, work habits, etc.Do NOT include transient conversation content or temporary mood states.-directory: "viking://user/{user_space}/memories"+directory: "viking://user/{{ user_space }}/memories"filename_template: "profile.md"enabled: truefields:diff --git a/openviking/prompts/templates/memory/skills.yaml b/openviking/prompts/templates/memory/skills.yamlindex 8d7a9d862..0358ce517 100644--- a/openviking/prompts/templates/memory/skills.yaml+++ b/openviking/prompts/templates/memory/skills.yaml@@ -1,8 +1,8 @@memory_type: skillsdescription: |Record all skills uses,-directory: "viking://agent/{agent_space}/memories/skills"-filename_template: "{skill_name}.md"+directory: "viking://agent/{{ agent_space }}/memories/skills"+filename_template: "{{ skill_name }}.md"enabled: truecontent_template: |Skill: {{ skill_name }}diff --git a/openviking/prompts/templates/memory/tools.yaml b/openviking/prompts/templates/memory/tools.yamlindex ec6e985cf..801a008df 100644--- a/openviking/prompts/templates/memory/tools.yaml+++ b/openviking/prompts/templates/memory/tools.yaml@@ -1,8 +1,8 @@memory_type: toolsdescription: |Record all tool calls-directory: "viking://agent/{agent_space}/memories/tools"-filename_template: "{tool_name}.md"+directory: "viking://agent/{{ agent_space }}/memories/tools"+filename_template: "{{ tool_name }}.md"enabled: truecontent_template: |Tool: {{ tool_name }}diff --git a/openviking/server/routers/bot.py b/openviking/server/routers/bot.pyindex 538146a52..cb89661a2 100644--- a/openviking/server/routers/bot.py+++ b/openviking/server/routers/bot.py@@ -8,9 +8,11 @@ import jsonfrom typing import AsyncGenerator, Optionalimport httpx-from fastapi import APIRouter, HTTPException, Request, status+from fastapi import APIRouter, Depends, HTTPException, Request, statusfrom fastapi.responses import StreamingResponse+from openviking.server.auth import get_request_context+from openviking.server.identity import RequestContextfrom openviking_cli.utils.logger import get_loggerrouter = APIRouter(prefix="", tags=["bot"])@@ -37,32 +39,6 @@ def get_bot_url() -> str:return BOT_API_URL-def extract_auth_token(request: Request) -> Optional[str]:- """Extract and return authorization token from request."""- # Try X-API-Key header first- api_key = request.headers.get("X-API-Key")- if api_key:- return api_key-- # Try Authorization header (Bearer token)- auth_header = request.headers.get("Authorization")- if auth_header and auth_header.startswith("Bearer "):- return auth_header[7:] # Remove "Bearer " prefix-- return None---def require_auth_token(request: Request) -> str:- """Return an auth token or raise 401 for bot proxy endpoints."""- auth_token = extract_auth_token(request)- if not auth_token:- raise HTTPException(- status_code=status.HTTP_401_UNAUTHORIZED,- detail="Missing authentication token",- )- return auth_token--@router.get("/health")async def health_check(request: Request):"""Health check endpoint for Bot API.@@ -96,14 +72,32 @@ async def health_check(request: Request):)+def extract_auth_token(request: Request) -> Optional[str]:+ """Extract and return authorization token from request."""+ # Try X-API-Key header first+ api_key = request.headers.get("X-API-Key")+ if api_key:+ return api_key++ # Try Authorization header (Bearer token)+ auth_header = request.headers.get("Authorization")+ if auth_header and auth_header.startswith("Bearer "):+ return auth_header[7:] # Remove "Bearer " prefix++ return None++@router.post("/chat")-async def chat(request: Request):+async def chat(+ request: Request,+ _ctx: RequestContext = Depends(get_request_context),+):"""Send a message to the bot and get a response.Proxies the request to Vikingbot OpenAPIChannel."""bot_url = get_bot_url()- auth_token = require_auth_token(request)+ auth_token = extract_auth_token(request)# Read request bodytry:@@ -116,8 +110,10 @@ async def chat(request: Request):try:async with httpx.AsyncClient() as client:- # Build headers- headers = {"Content-Type": "application/json", "X-API-Key": auth_token}+ # Build headers - only include X-API-Key if provided+ headers = {"Content-Type": "application/json"}+ if auth_token:+ headers["X-API-Key"] = auth_token# Forward to Vikingbot OpenAPIChannel chat endpointresponse = await client.post(@@ -149,13 +145,16 @@ async def chat(request: Request):@router.post("/chat/stream")-async def chat_stream(request: Request):+async def chat_stream(+ request: Request,+ _ctx: RequestContext = Depends(get_request_context),+):"""Send a message to the bot and get a streaming response.Proxies the request to Vikingbot OpenAPIChannel with SSE streaming."""bot_url = get_bot_url()- auth_token = require_auth_token(request)+ auth_token = extract_auth_token(request)# Read request bodytry:@@ -170,8 +169,10 @@ async def chat_stream(request: Request):"""Generate SSE events from bot response stream."""try:async with httpx.AsyncClient() as client:- # Build headers- headers = {"Content-Type": "application/json", "X-API-Key": auth_token}+ # Build headers - only include X-API-Key if provided+ headers = {"Content-Type": "application/json"}+ if auth_token:+ headers["X-API-Key"] = auth_token# Forward to Vikingbot OpenAPIChannel stream endpointasync with client.stream(diff --git a/openviking/session/compressor_v2.py b/openviking/session/compressor_v2.pyindex 816c65e68..2717f84b6 100644--- a/openviking/session/compressor_v2.py+++ b/openviking/session/compressor_v2.py@@ -140,9 +140,11 @@ class SessionCompressorV2:continueuser_space = ctx.user.user_space_name() if ctx and ctx.user else "default"agent_space = ctx.user.agent_space_name() if ctx and ctx.user else "default"- dir_path = schema.directory.replace("{user_space}", user_space).replace(- "{agent_space}", agent_space- )+ # 使用 Jinja2 渲染 directory+ import jinja2+ env = jinja2.Environment(autoescape=False)+ template = env.from_string(schema.directory)+ dir_path = template.render(user_space=user_space, agent_space=agent_space)dir_path = viking_fs._uri_to_path(dir_path, ctx)if dir_path not in memory_schema_dirs:memory_schema_dirs.append(dir_path)diff --git a/openviking/session/memory/extract_loop.py b/openviking/session/memory/extract_loop.pyindex 5cc668bbd..c28a9bf82 100644--- a/openviking/session/memory/extract_loop.py+++ b/openviking/session/memory/extract_loop.py@@ -118,6 +118,11 @@ class ExtractLoop:# 预计算 expected_fieldsself._expected_fields = ["reasoning", "edit_overview_uris", "delete_uris"]++ # 获取 ExtractContext(整个流程复用)+ self._extract_context = self.context_provider.get_extract_context()+ if self._extract_context is None:+ raise ValueError("Failed to get ExtractContext from provider")for schema in schemas:self._expected_fields.append(schema.memory_type)@@ -185,7 +190,6 @@ See the complete JSON Schema below:if tool_calls:await self._execute_tool_calls(messages, tool_calls, tools_used)- await self._mark_cache_breakpoint(messages)continue# If model returned final operations, check if refetch is needed@@ -201,7 +205,6 @@ See the complete JSON Schema below:max_iterations += 1logger.info(f"Extended max_iterations to {max_iterations} for refetch")- await self._mark_cache_breakpoint(messages)continuefinal_operations = operations@@ -280,12 +283,17 @@ See the complete JSON Schema below:registry = self.context_provider._get_registry()schemas = self.context_provider.get_memory_schemas(self.ctx)+ # Use pre-initialized extract_context+ if not hasattr(self, '_extract_context') or self._extract_context is None:+ raise ValueError("ExtractContext not initialized")+is_valid, errors = validate_operations_uris(operations,schemas,registry,user_space="default",agent_space="default",+ extract_context=self._extract_context,)if not is_valid:error_msg = "Invalid memory operations:\n" + "\n".join(f" - {err}" for err in errors)@@ -307,6 +315,9 @@ See the complete JSON Schema below:Returns:Tuple of (tool_calls, operations) - one will be None, the other set"""+ # 标记 cache breakpoint+ await self._mark_cache_breakpoint(messages)+# Call LLM with tools - use tools from strategytool_choice = "none" if force_final else None@@ -367,8 +378,7 @@ See the complete JSON Schema below:self._validate_operations(operations)return (None, operations)except Exception as e:- print(f"Error parsing operations: {e}")- logger.warning(f"Unexpected error parsing memory operations: {e}")+ logger.exception(f"Error parsing operations: {e}")# Case 3: No tool calls and no parsable operationsprint("No tool calls or operations parsed")diff --git a/openviking/session/memory/memory_type_registry.py b/openviking/session/memory/memory_type_registry.pyindex 3cd62ad25..203688e72 100644--- a/openviking/session/memory/memory_type_registry.py+++ b/openviking/session/memory/memory_type_registry.py@@ -73,12 +73,14 @@ class MemoryTypeRegistry:Returns:List of directory URIs from enabled schemas"""+ import jinja2+uris = []for schema in self.list_all(include_disabled=False):if schema.directory:- dir_path = schema.directory.replace("{user_space}", user_space).replace(- "{agent_space}", agent_space- )+ env = jinja2.Environment(autoescape=False)+ template = env.from_string(schema.directory)+ dir_path = template.render(user_space=user_space, agent_space=agent_space)uris.append(dir_path)return urisdiff --git a/openviking/session/memory/memory_updater.py b/openviking/session/memory/memory_updater.pyindex 417d11547..6ecc17ce6 100644--- a/openviking/session/memory/memory_updater.py+++ b/openviking/session/memory/memory_updater.py@@ -35,6 +35,20 @@ class ExtractContext:def __init__(self, messages: List[Message]):self.messages = messages+ def get_first_message_time_from_ranges(self, ranges_str: str) -> str | None:+ """根据 ranges 字符串获取第一条消息的时间(YAML 日期格式)"""+ if not ranges_str:+ return None+ msg_range = self.read_message_ranges(ranges_str)+ return msg_range._first_message_time()++ def get_first_message_time_with_weekday_from_ranges(self, ranges_str: str) -> str | None:+ """根据 ranges 字符串获取第一条消息的时间,带周几"""+ if not ranges_str:+ return None+ msg_range = self.read_message_ranges(ranges_str)+ return msg_range._first_message_time_with_weekday()+def read_message_ranges(self, ranges_str: str) -> "MessageRange":"""Parse ranges string like "0-10,50-60" or "7,9,11,13" and return combined MessageRange.@@ -100,8 +114,8 @@ class MessageRange:result.append(f"[{elem.role}]: {elem.content}")return "\n".join(result)- def first_message_time(self) -> str | None:- """获取第一条消息的时间(YAML 日期格式),如果没有消息则返回 None"""+ def _first_message_time(self) -> str | None:+ """获取第一条消息的时间(内部方法)"""for elem in self.elements:if isinstance(elem, str):continue@@ -109,6 +123,18 @@ class MessageRange:return elem.created_at.strftime("%Y-%m-%d")return None+ def _first_message_time_with_weekday(self) -> str | None:+ """获取第一条消息的时间,带周几(内部方法)"""+ for elem in self.elements:+ if isinstance(elem, str):+ continue+ if hasattr(elem, "created_at") and elem.created_at:+ # 获取周几的英文全称+ weekday_en = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"]+ weekday = weekday_en[elem.created_at.weekday()]+ return f"{elem.created_at.strftime('%Y-%m-%d')} ({weekday})"+ return None+class MemoryUpdateResult:"""Result of memory update operation."""@@ -205,12 +231,13 @@ class MemoryUpdater:user_space = ctx.user.user_space_name() if ctx and ctx.user else "default"agent_space = ctx.user.agent_space_name() if ctx and ctx.user else "default"- # Resolve all URIs first+ # Resolve all URIs first (pass extract_context for template rendering)resolved_ops = resolve_all_operations(operations,resolved_registry,user_space=user_space,agent_space=agent_space,+ extract_context=extract_context,)if resolved_ops.has_errors():diff --git a/openviking/session/memory/schema_model_generator.py b/openviking/session/memory/schema_model_generator.pyindex 275ef0a8f..b6c14596b 100644--- a/openviking/session/memory/schema_model_generator.py+++ b/openviking/session/memory/schema_model_generator.py@@ -399,11 +399,11 @@ class SchemaPromptGenerator:# Add variable substitution infolines.append("\n**Variable Substitution:**")- lines.append("- `{user_space}` → 'default'")- lines.append("- `{agent_space}` → 'default'")+ lines.append("- `{{ user_space }}` → 'default'")+ lines.append("- `{{ agent_space }}` → 'default'")if mt.fields:for field in mt.fields:- lines.append(f"- `{field.name}` → use value from fields")+ lines.append(f"- `{{ {field.name} }}` → use value from fields")if mt.fields:lines.append("\n**Fields:**")diff --git a/openviking/session/memory/session_extract_context_provider.py b/openviking/session/memory/session_extract_context_provider.pyindex 4fa7a1fd2..732a46ac5 100644--- a/openviking/session/memory/session_extract_context_provider.py+++ b/openviking/session/memory/session_extract_context_provider.py@@ -33,6 +33,15 @@ class SessionExtractContextProvider(ExtractContextProvider):self._output_language = self._detect_language()self._registry = None # 延迟加载self._schema_directories = None+ self._extract_context = None # 缓存 ExtractContext 实例++ def get_extract_context(self) -> "ExtractContext":+ """获取或创建 ExtractContext 实例(缓存)"""+ from openviking.session.memory.memory_updater import ExtractContext++ if self._extract_context is None and self.messages:+ self._extract_context = ExtractContext(self.messages)+ return self._extract_contextdef _detect_language(self) -> str:"""检测输出语言"""@@ -215,9 +224,10 @@ After exploring, analyze the conversation and output ALL memory write/edit/delet# Replace variables in directory path with actual user/agent spaceuser_space = ctx.user.user_space_name() if ctx and ctx.user else "default"agent_space = ctx.user.agent_space_name() if ctx and ctx.user else "default"- dir_path = schema.directory.replace("{user_space}", user_space).replace(- "{agent_space}", agent_space- )+ import jinja2+ env = jinja2.Environment(autoescape=False)+ template = env.from_string(schema.directory)+ dir_path = template.render(user_space=user_space, agent_space=agent_space)# Always add .overview.md to read listoverview_files.add(f"{dir_path}/.overview.md")@@ -227,10 +237,10 @@ After exploring, analyze the conversation and output ALL memory write/edit/delet# 只新增,不需要查看之前的记忆列表,只需要读取 .overview.mdcontinue- # Check if filename_template has variables (contains {xxx})+ # Check if filename_template has variables (contains {{ xxx }})has_variables = Falseif schema.filename_template:- has_variables = "{" in schema.filename_template and "}" in schema.filename_template+ has_variables = "{{" in schema.filename_template and "}}" in schema.filename_templateif has_variables or not schema.filename_template:# Multi-file schema or no filename template: ls the directorydiff --git a/openviking/session/memory/utils/uri.py b/openviking/session/memory/utils/uri.pyindex 081ddb007..667fb32c6 100644--- a/openviking/session/memory/utils/uri.py+++ b/openviking/session/memory/utils/uri.py@@ -8,6 +8,8 @@ import refrom dataclasses import dataclassfrom typing import Any, Dict, List, Optional, Set, Tuple+import jinja2+from openviking.session.memory.dataclass import MemoryTypeSchemafrom openviking.session.memory.memory_type_registry import MemoryTypeRegistryfrom openviking_cli.utils import get_logger@@ -15,6 +17,48 @@ from openviking_cli.utils import get_loggerlogger = get_logger(__name__)+def _render_jinja_template(template: str, context: Dict[str, Any]) -> str:+ """Render a Jinja2 template with the given context."""+ env = jinja2.Environment(+ autoescape=False,+ keep_trailing_newline=True,+ )+ jinja_template = env.from_string(template)+ return jinja_template.render(**context)+++def render_template(+ template: str,+ fields: Dict[str, Any],+ extract_context: Any = None,+) -> str:+ """+ Generic Jinja2 template rendering method.++ This is the same method used for rendering content_template in memory_updater.py.+ Used for rendering filename_template, directory, etc.++ Args:+ template: The template string with Jinja2 placeholders+ fields: Dictionary of field values for substitution+ extract_context: ExtractContext instance for template access to message ranges++ Returns:+ Rendered template string+ """+ # 创建 Jinja2 环境,允许未定义的变量(打印警告但不报错)+ env = jinja2.Environment(autoescape=False, undefined=jinja2.DebugUndefined)++ # 创建模板变量+ template_vars = fields.copy()+ # 始终传入 extract_context,即使是 None,避免模板中访问时 undefined+ template_vars["extract_context"] = extract_context++ # 渲染模板+ jinja_template = env.from_string(template)+ return jinja_template.render(**template_vars).strip()++@dataclassclass ResolvedOperation:"""A resolved memory operation with URI and memory_type."""@@ -29,6 +73,7 @@ def generate_uri(fields: Dict[str, Any],user_space: str = "default",agent_space: str = "default",+ extract_context: Any = None,) -> str:"""Generate a full URI from memory type schema and field values.@@ -36,8 +81,9 @@ def generate_uri(Args:memory_type: The memory type schema with directory and filename_templatefields: The field values to use for template replacement- user_space: The user space to substitute for {user_space}- agent_space: The agent space to substitute for {agent_space}+ user_space: The user space to substitute for {{ user_space }}+ agent_space: The agent space to substitute for {{ agent_space }}+ extract_context: ExtractContext instance for template rendering (same as content_template)Returns:The fully generated URI@@ -58,27 +104,16 @@ def generate_uri(if not uri_template:raise ValueError("Memory type has neither directory nor filename_template")- # Build the replacement dictionary- replacements = {+ # Build the context for Jinja2 rendering+ context = {"user_space": user_space,"agent_space": agent_space,}+ # Add all fields to context+ context.update(fields)- # Add all fields to replacements- replacements.update(fields)-- # Replace all template variables- def replace_var(match: re.Match) -> str:- var_name = match.group(1)- if var_name not in replacements:- raise ValueError(f"Missing template variable '{var_name}' in fields")- value = replacements[var_name]- if value is None:- raise ValueError(f"Template variable '{var_name}' has None value")- return str(value)-- # Replace {variable} patterns- uri = re.sub(r"\{([^}]+)\}", replace_var, uri_template)+ # Render using unified render_template method (same as content_template)+ uri = render_template(uri_template, context, extract_context)return uri@@ -99,9 +134,10 @@ def validate_uri_template(memory_type: MemoryTypeSchema) -> bool:# Check that all variables in filename_template exist in fieldsif memory_type.filename_template:field_names = {f.name for f in memory_type.fields}- template_vars = set(re.findall(r"\{([^}]+)\}", memory_type.filename_template))+ # Match Jinja2 {{ variable }} patterns+ template_vars = set(re.findall(r"\{\{\s*(\w+)\s*\}\}", memory_type.filename_template))- # {user_space} and {agent_space} are built-in, not from fields+ # {{ user_space }} and {{ agent_space }} are built-in, not from fieldsbuilt_in_vars = {"user_space", "agent_space"}required_field_vars = template_vars - built_in_vars@@ -116,14 +152,16 @@ def collect_allowed_directories(schemas: List[MemoryTypeSchema],user_space: str = "default",agent_space: str = "default",+ extract_context: Any = None,) -> Set[str]:"""Collect all allowed directories from activated schemas.Args:schemas: List of activated memory type schemas- user_space: User space to substitute for {user_space}- agent_space: Agent space to substitute for {agent_space}+ user_space: User space to substitute for {{ user_space }}+ agent_space: Agent space to substitute for {{ agent_space }}+ extract_context: ExtractContext instance for template renderingReturns:Set of allowed directory paths with variables replaced@@ -131,9 +169,9 @@ def collect_allowed_directories(allowed_dirs = set()for schema in schemas:if schema.directory:- dir_path = schema.directory.replace("{user_space}", user_space).replace(- "{agent_space}", agent_space- )+ context = {"user_space": user_space, "agent_space": agent_space}+ # Use unified render_template for consistent rendering+ dir_path = render_template(schema.directory, context, extract_context)allowed_dirs.add(dir_path)return allowed_dirs@@ -142,18 +180,20 @@ def collect_allowed_path_patterns(schemas: List[MemoryTypeSchema],user_space: str = "default",agent_space: str = "default",+ extract_context: Any = None,) -> Set[str]:"""Collect all allowed full path patterns from activated schemas.Args:schemas: List of activated memory type schemas- user_space: User space to substitute for {user_space}- agent_space: Agent space to substitute for {agent_space}+ user_space: User space to substitute for {{ user_space }}+ agent_space: Agent space to substitute for {{ agent_space }}+ extract_context: ExtractContext instance for template renderingReturns:- Set of allowed path patterns with {user_space} and {agent_space} replaced- (other variables like {topic}, {tool_name}, etc. remain as patterns)+ Set of allowed path patterns with {{ user_space }} and {{ agent_space }} replaced+ (other variables like {{ topic }}, {{ tool_name }}, etc. remain as patterns)"""allowed_patterns = set()for schema in schemas:@@ -166,24 +206,24 @@ def collect_allowed_path_patterns(pattern_parts.append(schema.filename_template)if pattern_parts:pattern = "/".join(pattern_parts)- pattern = pattern.replace("{user_space}", user_space).replace(- "{agent_space}", agent_space- )+ context = {"user_space": user_space, "agent_space": agent_space}+ # Use unified render_template for consistent rendering+ pattern = render_template(pattern, context, extract_context)allowed_patterns.add(pattern)return allowed_patternsdef _pattern_matches_uri(pattern: str, uri: str) -> bool:"""- Check if a URI matches a pattern with variables like {topic}, {tool_name}, etc.+ Check if a URI matches a pattern with variables like {{ topic }}, {{ tool_name }}, etc.The pattern matching is flexible:- - {variable} matches any sequence of characters except '/'+ - {{ variable }} matches any sequence of characters except '/'- * matches any sequence of characters except '/' (shell-style)- ** matches any sequence of characters including '/' (shell-style)Args:- pattern: The pattern to match against (may contain {variables} or * wildcards)+ pattern: The pattern to match against (may contain {{ variables }} or * wildcards)uri: The URI to checkReturns:@@ -196,7 +236,9 @@ def _pattern_matches_uri(pattern: str, uri: str) -> bool:pattern = re.escape(pattern)# Unescape {, }, * that we need to handle speciallypattern = pattern.replace(r"\{", "{").replace(r"\}", "}").replace(r"\*", "*")- # Convert {variable} to [^/]++ # Convert {{ variable }} to [^/]++ pattern = re.sub(r"\{\{\s*[^}]+\s*\}\}", r"[^/]+", pattern)+ # Also support legacy {variable} formatpattern = re.sub(r"\{[^}]+\}", r"[^/]+", pattern)# Convert ** to .* and * to [^/]*pattern = pattern.replace("**", ".*")@@ -246,14 +288,14 @@ def is_uri_allowed_for_schema(Args:uri: The URI to checkschemas: List of activated memory type schemas- user_space: User space to substitute for {user_space}- agent_space: Agent space to substitute for {agent_space}+ user_space: User space to substitute for {{ user_space }}+ agent_space: Agent space to substitute for {{ agent_space }}Returns:True if the URI is allowed"""- allowed_dirs = collect_allowed_directories(schemas, user_space, agent_space)- allowed_patterns = collect_allowed_path_patterns(schemas, user_space, agent_space)+ allowed_dirs = collect_allowed_directories(schemas, user_space, agent_space, extract_context)+ allowed_patterns = collect_allowed_path_patterns(schemas, user_space, agent_space, extract_context)return is_uri_allowed(uri, allowed_dirs, allowed_patterns)@@ -289,6 +331,7 @@ def resolve_flat_model_uri(user_space: str = "default",agent_space: str = "default",memory_type: Optional[str] = None,+ extract_context: Any = None,) -> str:"""Resolve URI for a flat model (used for both write and edit operations).@@ -299,6 +342,7 @@ def resolve_flat_model_uri(user_space: User space for substitutionagent_space: Agent space for substitutionmemory_type: Optional memory_type - if provided, use it instead of reading from model+ extract_context: ExtractContext instance for template rendering (same as content_template)Returns:Resolved URI@@ -328,7 +372,7 @@ def resolve_flat_model_uri(# Extract URI fields and generate URIuri_fields = extract_uri_fields_from_flat_model(flat_model, schema)- return generate_uri(schema, uri_fields, user_space, agent_space)+ return generate_uri(schema, uri_fields, user_space, agent_space, extract_context)def resolve_overview_edit_uri(@@ -368,10 +412,9 @@ def resolve_overview_edit_uri(if not schema.directory:raise ValueError(f"Memory type {memory_type_str} has no directory configured")- # Substitute user_space and agent_space in directory- directory = schema.directory.replace("{user_space}", user_space).replace(- "{agent_space}", agent_space- )+ # Render directory using Jinja2+ context = {"user_space": user_space, "agent_space": agent_space}+ directory = _render_jinja_template(schema.directory, context)# Return the .overview.md URIreturn f"{directory}/.overview.md"@@ -398,6 +441,7 @@ def resolve_all_operations(registry: MemoryTypeRegistry,user_space: str = "default",agent_space: str = "default",+ extract_context: Any = None,) -> ResolvedOperations:"""Resolve URIs for all operations.@@ -409,6 +453,7 @@ def resolve_all_operations(registry: MemoryTypeRegistry to get schemasuser_space: User space for substitutionagent_space: Agent space for substitution+ extract_context: ExtractContext instance for template rendering (same as content_template)Returns:ResolvedOperations with all URIs resolved@@ -435,7 +480,8 @@ def resolve_all_operations(item_dict = dict(item) if hasattr(item, "model_dump") else dict(item)try:uri = resolve_flat_model_uri(- item_dict, registry, user_space, agent_space, memory_type=field_name+ item_dict, registry, user_space, agent_space,+ memory_type=field_name, extract_context=extract_context)if is_edit:resolved.edit_operations.append(@@ -454,7 +500,9 @@ def resolve_all_operations(for op in write_uris:try:- uri = resolve_flat_model_uri(op, registry, user_space, agent_space)+ uri = resolve_flat_model_uri(+ op, registry, user_space, agent_space, extract_context=extract_context+ )# Legacy format: try to get memory_type from model, otherwise emptymemory_type = op.get("memory_type", "") if isinstance(op, dict) else ""resolved.write_operations.append(@@ -465,7 +513,9 @@ def resolve_all_operations(for op in edit_uris:try:- uri = resolve_flat_model_uri(op, registry, user_space, agent_space)+ uri = resolve_flat_model_uri(+ op, registry, user_space, agent_space, extract_context=extract_context+ )memory_type = op.get("memory_type", "") if isinstance(op, dict) else ""resolved.edit_operations.append(ResolvedOperation(model=op, uri=uri, memory_type=memory_type)@@ -500,6 +550,7 @@ def validate_operations_uris(registry: MemoryTypeRegistry,user_space: str = "default",agent_space: str = "default",+ extract_context: Any = None,) -> Tuple[bool, List[str]]:"""Validate that all URIs in StructuredMemoryOperations are allowed.@@ -508,19 +559,20 @@ def validate_operations_uris(operations: The StructuredMemoryOperations to validateschemas: List of activated memory type schemasregistry: MemoryTypeRegistry for URI resolution- user_space: User space to substitute for {user_space}- agent_space: Agent space to substitute for {agent_space}+ user_space: User space to substitute for {{ user_space }}+ agent_space: Agent space to substitute for {{ agent_space }}+ extract_context: ExtractContext instance for template renderingReturns:Tuple of (is_valid, list of error messages)"""- allowed_dirs = collect_allowed_directories(schemas, user_space, agent_space)- allowed_patterns = collect_allowed_path_patterns(schemas, user_space, agent_space)+ allowed_dirs = collect_allowed_directories(schemas, user_space, agent_space, extract_context)+ allowed_patterns = collect_allowed_path_patterns(schemas, user_space, agent_space, extract_context)errors = []# First resolve all URIs- resolved = resolve_all_operations(operations, registry, user_space, agent_space)+ resolved = resolve_all_operations(operations, registry, user_space, agent_space, extract_context)if resolved.has_errors():errors.extend(resolved.errors)diff --git a/tests/models/vlm/test_volcengine_cache.py b/tests/models/vlm/test_volcengine_cache.pynew file mode 100644index 000000000..73defabcf--- /dev/null+++ b/tests/models/vlm/test_volcengine_cache.py@@ -0,0 +1,253 @@+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.+# SPDX-License-Identifier: AGPL-3.0+"""Tests for VolcEngineVLM cache logic."""++import pytest+from unittest.mock import AsyncMock, MagicMock, patch++from openviking.models.vlm.backends.volcengine_vlm import VolcEngineVLM+from openviking.models.vlm.backends.volcengine_vlm import VolcEngineVLM as VLMClass+++def make_message(role: str, content: str, cache_control: bool = False) -> dict:+ """Helper to create a message dict."""+ msg = {"role": role, "content": content}+ if cache_control:+ msg["cache_control"] = {"type": "ephemeral"}+ return msg+++class TestGetOrCreateFromSegments:+ """Tests for _get_or_create_from_segments method."""++ def _create_vlm_with_mock_cache(self):+ """Create a VLM instance with mocked dependencies."""+ vlm = VLMClass(+ model="test-model",+ api_key="test-key",+ api_base="https://ark.cn-beijing.volces.com/api/v3",+ )+ # Mock the cache+ vlm._response_cache = MagicMock()+ vlm.get_async_client = MagicMock()+ return vlm++ def test_single_segment_with_cache_hit(self):+ """Test: Single segment, cache exists."""+ vlm = self._create_vlm_with_mock_cache()++ # 只有一个 segment [msg0, msg1(cache_control)]+ segments = [+ [make_message("system", "You are a helpful assistant"), make_message("user", "Hello", cache_control=True)]+ ]++ # Mock cache hit+ vlm._response_cache.get.return_value = "resp_123"++ result = VLMClass._get_or_create_from_segments(vlm, segments, 1)++ assert result == "resp_123"+ vlm._response_cache.get.assert_called_once()+ vlm.get_async_client.assert_not_called()++ @pytest.mark.asyncio+ async def test_single_segment_cache_miss_create_new(self):+ """Test: Single segment, cache miss, create new cache."""+ vlm = self._create_vlm_with_mock_cache()++ segments = [+ [make_message("system", "You are a helpful assistant"), make_message("user", "Hello", cache_control=True)]+ ]++ # Mock cache miss+ vlm._response_cache.get.return_value = None++ # Mock API response+ mock_response = MagicMock()+ mock_response.id = "resp_new_123"+ mock_client = AsyncMock()+ mock_client.responses.create = AsyncMock(return_value=mock_response)+ vlm.get_async_client.return_value = mock_client++ result = await vlm._get_or_create_from_segments(segments, 1)++ assert result == "resp_new_123"+ vlm._response_cache.get.assert_called_once()+ vlm._response_cache.set.assert_called_once()++ @pytest.mark.asyncio+ async def test_two_segments_both_cached(self):+ """Test: Two segments, both have cache."""+ vlm = self._create_vlm_with_mock_cache()++ # segments = [[msg0, msg1(cc)], [msg2, msg3(cc)]]+ segments = [+ [make_message("system", "You are a helpful assistant"), make_message("user", "Hello", cache_control=True)],+ [make_message("user", "How are you?", cache_control=True)],+ ]++ # Mock cache hits for both segments+ def cache_get(key):+ if "seg0" in key:+ return "resp_seg0"+ if "seg1" in key:+ return "resp_seg1"+ return None++ vlm._response_cache.get.side_effect = cache_get++ # Should use seg0 cache with previous_response_id to create seg1+ mock_response = MagicMock()+ mock_response.id = "resp_combined"+ mock_client = AsyncMock()+ mock_client.responses.create = AsyncMock(return_value=mock_response)+ vlm.get_async_client.return_value = mock_client++ result = await vlm._get_or_create_from_segments(segments, 2)++ # Should return combined response id+ assert result == "resp_combined"++ @pytest.mark.asyncio+ async def test_two_segments_first_not_cached(self):+ """Test: Two segments, first not cached, second cached."""+ vlm = self._create_vlm_with_mock_cache()++ segments = [+ [make_message("system", "System"), make_message("user", "Hello", cache_control=True)],+ [make_message("user", "How are you?", cache_control=True)],+ ]++ # First segment not cached, second is cached+ cache_returns = {+ "prefix:seg0_system_hello": None, # First segment - not cached+ "prefix:seg1_how_are_you": "resp_seg1", # Second segment - cached+ }++ def cache_get(key):+ return cache_returns.get(key)++ vlm._response_cache.get.side_effect = cache_get++ # Mock API: first call creates first segment cache, second call extends with previous_response_id+ call_count = 0+ mock_responses = ["resp_seg0", "resp_combined"]++ async def mock_create(**kwargs):+ nonlocal call_count+ resp = MagicMock()+ resp.id = mock_responses[call_count]+ call_count += 1+ return resp++ mock_client = AsyncMock()+ mock_client.responses.create = mock_create+ vlm.get_async_client.return_value = mock_client++ result = await vlm._get_or_create_from_segments(segments, 2)++ # Should create first segment, then extend with second+ assert result == "resp_combined"++ @pytest.mark.asyncio+ async def test_two_segments_neither_cached(self):+ """Test: Two segments, neither cached."""+ vlm = self._create_vlm_with_mock_cache()++ segments = [+ [make_message("system", "System"), make_message("user", "Hello", cache_control=True)],+ [make_message("user", "How are you?", cache_control=True)],+ ]++ # Neither segment cached+ vlm._response_cache.get.return_value = None++ # Mock API responses+ call_count = 0+ mock_responses = ["resp_seg0", "resp_combined"]++ async def mock_create(**kwargs):+ nonlocal call_count+ resp = MagicMock()+ resp.id = mock_responses[call_count]+ call_count += 1+ return resp++ mock_client = AsyncMock()+ mock_client.responses.create = mock_create+ vlm.get_async_client.return_value = mock_client++ result = await vlm._get_or_create_from_segments(segments, 2)++ # Should create both segments+ assert result == "resp_combined"++ @pytest.mark.asyncio+ async def test_three_segments_with_middle_cached(self):+ """Test: Three segments, middle one cached, others not."""+ vlm = self._create_vlm_with_mock_cache()++ segments = [+ [make_message("system", "System"), make_message("user", "Hello", cache_control=True)],+ [make_message("user", "How are you?", cache_control=True)],+ [make_message("user", "Tell me a story", cache_control=True)],+ ]++ # Only middle segment cached+ cache_returns = {+ "prefix:seg0_system_hello": None,+ "prefix:seg1_how_are_you": "resp_seg1", # Cached+ "prefix:seg2_tell_story": None,+ }++ def cache_get(key):+ return cache_returns.get(key)++ vlm._response_cache.get.side_effect = cache_get++ # Mock API: create seg0, extend to seg1, extend to seg2+ call_count = 0+ mock_responses = ["resp_seg0", "resp_01", "resp_012"]++ async def mock_create(**kwargs):+ nonlocal call_count+ resp = MagicMock()+ resp.id = mock_responses[call_count]+ call_count += 1+ return resp++ mock_client = AsyncMock()+ mock_client.responses.create = mock_create+ vlm.get_async_client.return_value = mock_client++ result = await vlm._get_or_create_from_segments(segments, 3)++ # Should chain: seg0 -> seg1 (cached) -> seg2+ assert result == "resp_012"++ def test_zero_segments(self):+ """Test: end_idx = 0 returns None."""+ vlm = self._create_vlm_with_mock_cache()++ segments = [[make_message("system", "System")]]++ result = VLMClass._get_or_create_from_segments(vlm, segments, 0)++ assert result is None+++class TestCacheKeyGeneration:+ """Tests for cache key generation logic."""++ def test_cache_key_includes_prefix(self):+ """Test that cache keys include 'prefix:' prefix."""+ vlm = VLMClass(+ model="test-model",+ api_key="test-key",+ )++ messages = [make_message("system", "Hello")]+ key = vlm._get_response_id_cache_key(messages)++ # Should include prefix in the key+ assert "prefix:" in key or key.startswith("prefix:")\ No newline at end of filediff --git a/tests/server/test_bot_proxy_auth.py b/tests/server/test_bot_proxy_auth.pyindex 977702500..bf5b580f0 100644--- a/tests/server/test_bot_proxy_auth.py+++ b/tests/server/test_bot_proxy_auth.py@@ -6,7 +6,7 @@import httpximport pytestimport pytest_asyncio-from fastapi import FastAPI, HTTPException, Request+from fastapi import FastAPI, Requestimport openviking.server.routers.bot as bot_router_module@@ -27,21 +27,6 @@ def make_request(headers: dict[str, str]) -> Request:)-@pytest_asyncio.fixture-async def bot_auth_client() -> httpx.AsyncClient:- """Client mounted with bot router and bot backend configured."""- app = FastAPI()- old_bot_api_url = bot_router_module.BOT_API_URL- bot_router_module.set_bot_api_url("http://bot-backend.local")- app.include_router(bot_router_module.router, prefix="/bot/v1")- transport = httpx.ASGITransport(app=app)- try:- async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as client:- yield client- finally:- bot_router_module.BOT_API_URL = old_bot_api_url--@pytest.mark.parametrize(("headers", "expected"),[@@ -52,25 +37,3 @@ async def bot_auth_client() -> httpx.AsyncClient:def test_extract_auth_token(headers: dict[str, str], expected: str):"""Accepted auth header formats should both produce a token."""assert bot_router_module.extract_auth_token(make_request(headers)) == expected---def test_require_auth_token_rejects_missing_token():- """Missing credentials should raise a 401 before proxying."""- with pytest.raises(HTTPException) as exc_info:- bot_router_module.require_auth_token(make_request({}))-- assert exc_info.value.status_code == 401- assert exc_info.value.detail == "Missing authentication token"--[email protected]("path", ["/bot/v1/chat", "/bot/v1/chat/stream"])-async def test_bot_proxy_requires_auth_token(bot_auth_client: httpx.AsyncClient, path: str):- """Bot proxy endpoints should reject missing auth with 401."""- response = await bot_auth_client.post(- path,- json={"message": "hello"},- )-- assert response.status_code == 401- assert response.json()["detail"] == "Missing authentication token"