Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

feat(embedder): support extra_body passthrough in OpenAI embedder config (#3495)

Colter Dahlberge97b74d2e53a
diff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.md
index 292f9b2f6..ef08e90f9 100644
--- a/docs/en/guides/01-configuration.md
+++ b/docs/en/guides/01-configuration.md
@@ -221,6 +221,7 @@ Embedding model configuration for vector search, supporting dense, sparse, and h
| `input` | str | Input type: `"text"` or `"multimodal"` |
| `batch_size` | int | Batch size for embedding requests |
| `encoding_format` | str | (OpenAI / Azure only) Wire format for embedding values: `"float"` or `"base64"`. Leave unset to use the OpenAI Python SDK default. Set to `"float"` when the upstream gateway cannot deserialize base64 embedding payloads correctly. |
+| `extra_body` | object | (OpenAI / Azure only) Extra JSON body fields merged into every embeddings request. Useful for OpenAI-compatible gateways that accept vendor-specific fields, e.g. OpenRouter provider routing `{"provider": {"sort": "latency"}}`. Explicit `query_param`/`document_param` keys take precedence on conflict. |
`embedding.max_retries` only applies to transient errors such as `429`, `5xx`, timeouts, and connection failures. Permanent errors such as `400`, `401`, `403`, and `AccountOverdue` are not retried automatically. The backoff strategy is exponential backoff with jitter, starting at `0.5s` and capped at `8s`.
@@ -289,6 +290,29 @@ With `input: "multimodal"`, OpenViking can embed text, images (PNG, JPG, etc.),
`encoding_format` is optional and is only forwarded for `provider: "openai"` and `provider: "azure"`. Leave it unset for the OpenAI Python SDK default. Set it to `"float"` when an OpenAI-compatible upstream gateway cannot deserialize base64 embedding payloads correctly.
+**OpenRouter example with provider routing:**
+
+```json
+{
+ "embedding": {
+ "dense": {
+ "provider": "openai",
+ "api_key": "your-openrouter-api-key",
+ "api_base": "https://openrouter.ai/api/v1",
+ "model": "qwen/qwen3-embedding-8b",
+ "dimension": 4096,
+ "extra_body": {
+ "provider": {
+ "sort": "latency"
+ }
+ }
+ }
+ }
+}
+```
+
+`extra_body` is merged into every embeddings request, so OpenAI-compatible gateways that accept vendor-specific fields (such as OpenRouter's provider routing preferences) can be tuned without code changes. It is only forwarded for `provider: "openai"` and `provider: "azure"`.
+
**Azure OpenAI provider example with JSON float embeddings:**
```json
diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.md
index b0bf49156..93fa75613 100644
--- a/docs/zh/guides/01-configuration.md
+++ b/docs/zh/guides/01-configuration.md
@@ -223,6 +223,7 @@ openviking-server doctor
| `input` | str | 输入类型:`"text"` 或 `"multimodal"` |
| `batch_size` | int | 批量请求大小 |
| `encoding_format` | str | (仅 OpenAI / Azure)Embedding 值的传输格式:`"float"` 或 `"base64"`。留空时使用 OpenAI Python SDK 默认值;当上游网关无法正确处理 base64 embedding payload 时,可设置为 `"float"`。 |
+| `extra_body` | object | (仅 OpenAI / Azure)合并进每次 embedding 请求体的额外 JSON 字段。适用于接受厂商专有字段的 OpenAI 兼容网关,例如 OpenRouter 的 provider 路由 `{"provider": {"sort": "latency"}}`。发生冲突时,显式设置的 `query_param`/`document_param` 键优先。 |
`embedding.max_retries` 仅对瞬时错误生效,例如 `429`、`5xx`、超时和连接错误;`400`、`401`、`403`、`AccountOverdue` 这类永久错误不会自动重试。退避策略为指数退避,初始延迟 `0.5s`,上限 `8s`,并带随机抖动。
@@ -291,6 +292,29 @@ openviking-server doctor
`encoding_format` 是可选字段,只会传给 `provider: "openai"` 和 `provider: "azure"`。留空时使用 OpenAI Python SDK 默认行为;如果 OpenAI 兼容上游网关无法正确反序列化 base64 embedding payload,可设置为 `"float"`。
+**OpenRouter provider 路由示例:**
+
+```json
+{
+ "embedding": {
+ "dense": {
+ "provider": "openai",
+ "api_key": "your-openrouter-api-key",
+ "api_base": "https://openrouter.ai/api/v1",
+ "model": "qwen/qwen3-embedding-8b",
+ "dimension": 4096,
+ "extra_body": {
+ "provider": {
+ "sort": "latency"
+ }
+ }
+ }
+ }
+}
+```
+
+`extra_body` 会合并进每次 embedding 请求,因此无需改动代码即可调优接受厂商专有字段的 OpenAI 兼容网关(例如 OpenRouter 的 provider 路由偏好)。该字段只会传给 `provider: "openai"` 和 `provider: "azure"`。
+
**Azure OpenAI provider 的 JSON float embedding 示例:**
```json
diff --git a/openviking/models/embedder/openai_embedders.py b/openviking/models/embedder/openai_embedders.py
index 5b9f1c382..5a738c40c 100644
--- a/openviking/models/embedder/openai_embedders.py
+++ b/openviking/models/embedder/openai_embedders.py
@@ -78,6 +78,7 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):
provider: str = "openai",
configured_provider: Optional[str] = None,
encoding_format: Optional[Literal["float", "base64"]] = None,
+ extra_body: Optional[Dict[str, Any]] = None,
):
"""Initialize OpenAI-Compatible Dense Embedder
@@ -107,6 +108,10 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):
``"float"`` to send/receive plain JSON arrays — the recommended
workaround when the upstream gateway cannot deserialize base64
embedding payloads correctly.
+ extra_body: Extra JSON body fields merged into every embeddings request
+ (e.g., OpenRouter provider routing:
+ {'provider': {'sort': 'latency'}}). Keys set explicitly via
+ query_param/document_param take precedence on conflict.
Raises:
ValueError: If api_key is not provided and env vars are not set
@@ -126,6 +131,7 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):
self.query_param = query_param
self.document_param = document_param
self.encoding_format = encoding_format
+ self.extra_body = extra_body
self._provider = provider.lower()
self.provider = (configured_provider or provider).lower()
self._client_kwargs: Dict[str, Any] = {"api_key": self.api_key or "no-key"}
@@ -236,15 +242,21 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):
def _build_extra_body(self, is_query: bool = False) -> Optional[Dict[str, Any]]:
"""Build extra_body dict for OpenAI-compatible parameters
+ Starts from the configured ``extra_body`` (a general passthrough for
+ OpenAI-compatible gateways, e.g. OpenRouter provider routing), then
+ applies the query/document parameters on top so explicit
+ query_param/document_param keys always win on conflict.
+
Args:
is_query: Flag to indicate if this is for query embeddings
Returns:
- Dict containing input_type and other parameters if non-symmetric mode is active.
- Supports key=value format for multiple parameters (e.g., "input_type=query,task=search").
+ Dict containing the configured extra body plus input_type and other
+ parameters if non-symmetric mode is active. Supports key=value
+ format for multiple parameters (e.g., "input_type=query,task=search").
Only supported by OpenAI-compatible third-party models.
"""
- extra_body = {}
+ extra_body = dict(self.extra_body) if self.extra_body else {}
# Determine which parameter to use based on is_query flag
active_param = None
diff --git a/openviking_cli/utils/config/embedding_config.py b/openviking_cli/utils/config/embedding_config.py
index e6687c487..210db140e 100644
--- a/openviking_cli/utils/config/embedding_config.py
+++ b/openviking_cli/utils/config/embedding_config.py
@@ -94,6 +94,16 @@ class EmbeddingModelConfig(BaseModel):
"when the upstream gateway cannot serialize base64 responses correctly."
),
)
+ extra_body: Optional[dict[str, Any]] = Field(
+ default=None,
+ description=(
+ "Extra JSON body fields merged into every embeddings request. "
+ "Applies to OpenAI / Azure providers. Useful for OpenAI-compatible "
+ "gateways that accept vendor-specific fields, e.g. OpenRouter provider "
+ "routing {'provider': {'sort': 'latency'}}. Explicit "
+ "query_param/document_param keys take precedence on conflict."
+ ),
+ )
api_version: Optional[str] = Field(
default=None,
description="API version for Azure OpenAI (e.g., '2025-01-01-preview').",
@@ -751,6 +761,7 @@ class EmbeddingConfig(BaseModel):
if cfg.encoding_format is not None
else {}
),
+ **({"extra_body": cfg.extra_body} if cfg.extra_body else {}),
},
),
("azure", "dense"): (
@@ -772,6 +783,7 @@ class EmbeddingConfig(BaseModel):
if cfg.encoding_format is not None
else {}
),
+ **({"extra_body": cfg.extra_body} if cfg.extra_body else {}),
},
),
("volcengine", "dense"): (
@@ -1049,6 +1061,7 @@ class EmbeddingConfig(BaseModel):
# Model-behavior fields are shared by all credentials of the
# same model and live only on the parent config.
encoding_format=config.encoding_format,
+ extra_body=config.extra_body,
model_path=config.model_path,
cache_dir=config.cache_dir,
enable_fusion=config.enable_fusion,
diff --git a/tests/unit/test_extra_body_embedding.py b/tests/unit/test_extra_body_embedding.py
new file mode 100644
index 000000000..6db9ba320
--- /dev/null
+++ b/tests/unit/test_extra_body_embedding.py
@@ -0,0 +1,177 @@
+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
+# SPDX-License-Identifier: AGPL-3.0
+"""Tests for extra_body support in OpenAIDenseEmbedder and EmbeddingConfig factory.
+
+Covers:
+ 1. extra_body is merged into every embeddings.create call
+ 2. explicit query_param/document_param keys override extra_body on conflict
+ 3. omitting extra_body does not inject an extra_body kwarg
+ 4. factory (_create_embedder) transparently forwards extra_body
+"""
+
+from unittest.mock import MagicMock, patch
+
+from openviking.models.embedder import OpenAIDenseEmbedder
+from openviking_cli.utils.config.embedding_config import EmbeddingConfig, EmbeddingModelConfig
+
+
+def _make_mock_client():
+ """Build a MagicMock openai client that returns a minimal valid embedding response."""
+ mock_client = MagicMock()
+ mock_client.embeddings.create.return_value = MagicMock(
+ data=[MagicMock(embedding=[0.1] * 8)],
+ usage=None,
+ )
+ return mock_client
+
+
+class TestExtraBodyDirectConstruction:
+ """Test extra_body behaviour when constructing OpenAIDenseEmbedder directly."""
+
+ @patch("openviking.models.embedder.openai_embedders.openai.OpenAI")
+ def test_extra_body_merged_into_request_kwargs(self, mock_openai_class):
+ """extra_body dict must arrive as the extra_body kwarg in embeddings.create()."""
+ mock_client = _make_mock_client()
+ mock_openai_class.return_value = mock_client
+
+ embedder = OpenAIDenseEmbedder(
+ model_name="qwen/qwen3-embedding-8b",
+ api_key="sk-test",
+ api_base="https://openrouter.ai/api/v1",
+ dimension=8,
+ extra_body={"provider": {"sort": "latency"}},
+ )
+ embedder.embed("hello")
+
+ mock_client.embeddings.create.assert_called_once()
+ call_kwargs = mock_client.embeddings.create.call_args[1]
+ assert call_kwargs["extra_body"] == {"provider": {"sort": "latency"}}
+
+ @patch("openviking.models.embedder.openai_embedders.openai.OpenAI")
+ def test_no_extra_body_omits_kwarg(self, mock_openai_class):
+ """When extra_body is not provided, extra_body must NOT appear in embeddings.create()."""
+ mock_client = _make_mock_client()
+ mock_openai_class.return_value = mock_client
+
+ embedder = OpenAIDenseEmbedder(
+ model_name="text-embedding-3-small",
+ api_key="sk-test",
+ dimension=8,
+ )
+ embedder.embed("hello")
+
+ mock_client.embeddings.create.assert_called_once()
+ call_kwargs = mock_client.embeddings.create.call_args[1]
+ assert "extra_body" not in call_kwargs
+
+ @patch("openviking.models.embedder.openai_embedders.openai.OpenAI")
+ def test_query_document_params_override_extra_body_on_conflict(self, mock_openai_class):
+ """Explicit query_param/document_param keys win over extra_body keys."""
+ mock_client = _make_mock_client()
+ mock_openai_class.return_value = mock_client
+
+ embedder = OpenAIDenseEmbedder(
+ model_name="bge-m3",
+ api_key="sk-test",
+ api_base="https://your-api-endpoint.com/v1",
+ dimension=8,
+ query_param="query",
+ document_param="passage",
+ extra_body={"input_type": "default", "provider": {"sort": "latency"}},
+ )
+
+ embedder.embed("search text", is_query=True)
+ call_kwargs = mock_client.embeddings.create.call_args[1]
+ assert call_kwargs["extra_body"]["input_type"] == "query"
+ assert call_kwargs["extra_body"]["provider"] == {"sort": "latency"}
+
+ embedder.embed("document text", is_query=False)
+ call_kwargs = mock_client.embeddings.create.call_args[1]
+ assert call_kwargs["extra_body"]["input_type"] == "passage"
+ assert call_kwargs["extra_body"]["provider"] == {"sort": "latency"}
+
+ @patch("openviking.models.embedder.openai_embedders.openai.OpenAI")
+ def test_extra_body_is_copied_not_mutated(self, mock_openai_class):
+ """Merging query/document params must not mutate the configured dict."""
+ mock_client = _make_mock_client()
+ mock_openai_class.return_value = mock_client
+
+ extra_body = {"input_type": "default"}
+ embedder = OpenAIDenseEmbedder(
+ model_name="bge-m3",
+ api_key="sk-test",
+ api_base="https://your-api-endpoint.com/v1",
+ dimension=8,
+ query_param="query",
+ extra_body=extra_body,
+ )
+ embedder.embed("search text", is_query=True)
+
+ assert extra_body == {"input_type": "default"}
+
+
+class TestExtraBodyViaFactory:
+ """Test extra_body forwarding through EmbeddingConfig._create_embedder."""
+
+ @patch("openai.OpenAI")
+ def test_factory_passes_extra_body(self, mock_openai_class):
+ """Factory must forward config extra_body into every embeddings.create() call."""
+ mock_client = _make_mock_client()
+ mock_openai_class.return_value = mock_client
+
+ cfg = EmbeddingModelConfig(
+ provider="openai",
+ model="qwen/qwen3-embedding-8b",
+ api_key="sk-test",
+ api_base="https://openrouter.ai/api/v1",
+ dimension=8,
+ extra_body={"provider": {"sort": "latency"}},
+ )
+ embedder = EmbeddingConfig(dense=cfg)._create_embedder("openai", "dense", cfg)
+ embedder.embed("hello")
+
+ mock_client.embeddings.create.assert_called_once()
+ call_kwargs = mock_client.embeddings.create.call_args[1]
+ assert call_kwargs["extra_body"] == {"provider": {"sort": "latency"}}
+
+ @patch("openai.OpenAI")
+ def test_factory_omits_extra_body_when_none(self, mock_openai_class):
+ """Factory must NOT inject extra_body when it is None."""
+ mock_client = _make_mock_client()
+ mock_openai_class.return_value = mock_client
+
+ cfg = EmbeddingModelConfig(
+ provider="openai",
+ model="text-embedding-3-small",
+ api_key="sk-test",
+ dimension=8,
+ )
+ embedder = EmbeddingConfig(dense=cfg)._create_embedder("openai", "dense", cfg)
+ embedder.embed("hello")
+
+ mock_client.embeddings.create.assert_called_once()
+ call_kwargs = mock_client.embeddings.create.call_args[1]
+ assert "extra_body" not in call_kwargs
+
+
+class TestEmbeddingModelConfigExtraBody:
+ """Test that EmbeddingModelConfig accepts and stores the extra_body field."""
+
+ def test_openai_config_accepts_extra_body_field(self):
+ """EmbeddingModelConfig should store extra_body without validation error."""
+ cfg = EmbeddingModelConfig(
+ provider="openai",
+ model="qwen/qwen3-embedding-8b",
+ api_key="sk-test",
+ extra_body={"provider": {"sort": "latency"}},
+ )
+ assert cfg.extra_body == {"provider": {"sort": "latency"}}
+
+ def test_extra_body_defaults_to_none(self):
+ """extra_body field should default to None when not supplied."""
+ cfg = EmbeddingModelConfig(
+ provider="openai",
+ model="text-embedding-3-small",
+ api_key="sk-test",
+ )
+ assert cfg.extra_body is None