Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

feat(embedder): expose encoding_format for OpenAI/Azure providers (#2092)

Simon Shi72b407ffa8ef
diff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.md
index 1a8722d2d..9f5570998 100644
--- a/docs/en/guides/01-configuration.md
+++ b/docs/en/guides/01-configuration.md
@@ -214,12 +214,13 @@ Embedding model configuration for vector search, supporting dense, sparse, and h
| `max_retries` | int | Maximum retry attempts for transient embedding provider errors (`embedding.max_retries`, default: `3`; `0` disables retry) |
| `text_source` | str | Text used for vectorizing text files. `content_only` reads raw content, `summary_first` uses summary when available and falls back to content, `summary_only` uses only summary. Default: `content_only` |
| `max_input_tokens` | int | Maximum estimated raw text tokens sent to the embedding model when content is used. Default: `4096` |
-| `provider` | str | `"volcengine"`, `"openai"`, `"vikingdb"`, `"jina"`, `"voyage"`, `"dashscope"`, or `"gemini"` |
+| `provider` | str | `"openai"`, `"azure"`, `"volcengine"`, `"vikingdb"`, `"jina"`, `"ollama"`, `"gemini"`, `"voyage"`, `"dashscope"`, `"minimax"`, `"cohere"`, `"litellm"`, or `"local"` |
| `api_key` | str | API key |
| `model` | str | Model name |
| `dimension` | int | Vector dimension. For Voyage, this maps to `output_dimension` |
| `input` | str | Input type: `"text"` or `"multimodal"` |
| `batch_size` | int | Batch size for embedding requests |
+| `encoding_format` | str | (OpenAI / Azure only) Wire format for embedding values: `"float"` or `"base64"`. Leave unset to use the OpenAI Python SDK default. Set to `"float"` when the upstream gateway cannot deserialize base64 embedding payloads correctly. |
`embedding.max_retries` only applies to transient errors such as `429`, `5xx`, timeouts, and connection failures. Permanent errors such as `400`, `401`, `403`, and `AccountOverdue` are not retried automatically. The backoff strategy is exponential backoff with jitter, starting at `0.5s` and capped at `8s`.
@@ -256,12 +257,57 @@ With `input: "multimodal"`, OpenViking can embed text, images (PNG, JPG, etc.),
**Supported providers:**
- `openai`: OpenAI Embedding API
+- `azure`: Azure OpenAI Embedding API
- `volcengine`: Volcengine Embedding API
- `vikingdb`: VikingDB Embedding API
- `jina`: Jina AI Embedding API
+- `ollama`: Ollama local OpenAI-compatible Embedding API
- `voyage`: Voyage AI Embedding API
- `minimax`: MiniMax Embedding API
+- `cohere`: Cohere Embedding API
- `gemini`: Google Gemini Embedding API (text-only; requires `google-genai>=1.0.0`)
+- `dashscope`: DashScope (Alibaba Tongyi) Embedding API
+- `litellm`: LiteLLM Embedding API
+- `local`: Local GGUF embedding models
+
+**OpenAI-compatible provider example with JSON float embeddings:**
+
+```json
+{
+ "embedding": {
+ "dense": {
+ "provider": "openai",
+ "api_key": "your-api-key",
+ "api_base": "https://your-openai-compatible-endpoint/v1",
+ "model": "text-embedding-3-large",
+ "dimension": 3072,
+ "encoding_format": "float"
+ }
+ }
+}
+```
+
+`encoding_format` is optional and is only forwarded for `provider: "openai"` and `provider: "azure"`. Leave it unset for the OpenAI Python SDK default. Set it to `"float"` when an OpenAI-compatible upstream gateway cannot deserialize base64 embedding payloads correctly.
+
+**Azure OpenAI provider example with JSON float embeddings:**
+
+```json
+{
+ "embedding": {
+ "dense": {
+ "provider": "azure",
+ "api_key": "your-azure-api-key",
+ "api_base": "https://your-resource-name.openai.azure.com",
+ "api_version": "2025-01-01-preview",
+ "model": "your-embedding-deployment-name",
+ "dimension": 3072,
+ "encoding_format": "float"
+ }
+ }
+}
+```
+
+For Azure OpenAI, `model` must be the embedding deployment name configured in Azure.
**minimax provider example:**
@@ -1289,7 +1335,8 @@ For detailed encryption explanations, see [Data Encryption](../concepts/10-encry
"api_key": "string",
"model": "string",
"dimension": 1024,
- "input": "multimodal"
+ "input": "multimodal",
+ "encoding_format": "float|base64"
}
},
"vlm": {
diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.md
index 2137ef221..dadd5c15f 100644
--- a/docs/zh/guides/01-configuration.md
+++ b/docs/zh/guides/01-configuration.md
@@ -216,12 +216,13 @@ openviking-server doctor
| `max_retries` | int | Embedding provider 瞬时错误的最大重试次数(`embedding.max_retries`,默认:`3`;`0` 表示禁用重试) |
| `text_source` | str | 文本文件向量化时使用的文本来源。`content_only` 读取原文内容;`summary_first` 优先使用摘要,没有摘要时回退到原文;`summary_only` 只使用摘要。默认:`content_only` |
| `max_input_tokens` | int | 使用原文内容向量化时,发送给 embedding 模型的最大估算 token 数。默认:`4096` |
-| `provider` | str | `"volcengine"`、`"openai"`、`"vikingdb"`、`"jina"`、`"voyage"`、`"minimax"`、`"dashscope"` 或 `"gemini"` |
+| `provider` | str | `"openai"`、`"azure"`、`"volcengine"`、`"vikingdb"`、`"jina"`、`"ollama"`、`"gemini"`、`"voyage"`、`"dashscope"`、`"minimax"`、`"cohere"`、`"litellm"` 或 `"local"` |
| `api_key` | str | API Key |
| `model` | str | 模型名称 |
| `dimension` | int | 向量维度 |
| `input` | str | 输入类型:`"text"` 或 `"multimodal"` |
| `batch_size` | int | 批量请求大小 |
+| `encoding_format` | str | (仅 OpenAI / Azure)Embedding 值的传输格式:`"float"` 或 `"base64"`。留空时使用 OpenAI Python SDK 默认值;当上游网关无法正确处理 base64 embedding payload 时,可设置为 `"float"`。 |
`embedding.max_retries` 仅对瞬时错误生效,例如 `429`、`5xx`、超时和连接错误;`400`、`401`、`403`、`AccountOverdue` 这类永久错误不会自动重试。退避策略为指数退避,初始延迟 `0.5s`,上限 `8s`,并带随机抖动。
@@ -258,13 +259,57 @@ openviking-server doctor
**支持的 provider:**
- `openai`: OpenAI Embedding API
+- `azure`: Azure OpenAI Embedding API
- `volcengine`: 火山引擎 Embedding API
- `vikingdb`: VikingDB Embedding API
- `jina`: Jina AI Embedding API
+- `ollama`: Ollama 本地 OpenAI 兼容 Embedding API
- `voyage`: Voyage AI Embedding API
- `minimax`: MiniMax Embedding API
+- `cohere`: Cohere Embedding API
- `gemini`: Google Gemini Embedding API(仅文本;需安装 `google-genai>=1.0.0`)
- `dashscope`: DashScope(阿里通义)Embedding API
+- `litellm`: LiteLLM Embedding API
+- `local`: 本地 GGUF embedding 模型
+
+**OpenAI 兼容 provider 的 JSON float embedding 示例:**
+
+```json
+{
+ "embedding": {
+ "dense": {
+ "provider": "openai",
+ "api_key": "your-api-key",
+ "api_base": "https://your-openai-compatible-endpoint/v1",
+ "model": "text-embedding-3-large",
+ "dimension": 3072,
+ "encoding_format": "float"
+ }
+ }
+}
+```
+
+`encoding_format` 是可选字段,只会传给 `provider: "openai"` 和 `provider: "azure"`。留空时使用 OpenAI Python SDK 默认行为;如果 OpenAI 兼容上游网关无法正确反序列化 base64 embedding payload,可设置为 `"float"`。
+
+**Azure OpenAI provider 的 JSON float embedding 示例:**
+
+```json
+{
+ "embedding": {
+ "dense": {
+ "provider": "azure",
+ "api_key": "your-azure-api-key",
+ "api_base": "https://your-resource-name.openai.azure.com",
+ "api_version": "2025-01-01-preview",
+ "model": "your-embedding-deployment-name",
+ "dimension": 3072,
+ "encoding_format": "float"
+ }
+ }
+}
+```
+
+对于 Azure OpenAI,`model` 必须填写 Azure 中配置的 embedding deployment name。
**minimax provider 配置示例:**
@@ -1263,7 +1308,8 @@ openviking add-resource ./docs --exclude "*.tmp"
"api_key": "string",
"model": "string",
"dimension": 1024,
- "input": "multimodal"
+ "input": "multimodal",
+ "encoding_format": "float|base64"
}
},
"vlm": {
diff --git a/openviking/models/embedder/openai_embedders.py b/openviking/models/embedder/openai_embedders.py
index 554a159af..311044e7a 100644
--- a/openviking/models/embedder/openai_embedders.py
+++ b/openviking/models/embedder/openai_embedders.py
@@ -2,7 +2,7 @@
# SPDX-License-Identifier: AGPL-3.0
"""OpenAI Embedder Implementation"""
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List, Literal, Optional
import openai
@@ -76,6 +76,7 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):
input_type: Optional[str] = None,
provider: str = "openai",
configured_provider: Optional[str] = None,
+ encoding_format: Optional[Literal["float", "base64"]] = None,
):
"""Initialize OpenAI-Compatible Dense Embedder
@@ -100,6 +101,11 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):
config: Additional configuration dict
extra_headers: Extra HTTP headers to include in API requests (e.g., for OpenRouter:
{'HTTP-Referer': 'https://your-site.com', 'X-Title': 'Your App'})
+ encoding_format: Wire format for embedding values. ``None`` (default) lets the
+ OpenAI Python SDK pick its default (currently ``base64``). Set to
+ ``"float"`` to send/receive plain JSON arrays — the recommended
+ workaround when the upstream gateway cannot deserialize base64
+ embedding payloads correctly.
Raises:
ValueError: If api_key is not provided and env vars are not set
@@ -118,6 +124,7 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):
self.dimension = dimension
self.query_param = query_param
self.document_param = document_param
+ self.encoding_format = encoding_format
self._provider = provider.lower()
self.provider = (configured_provider or provider).lower()
self._client_kwargs: Dict[str, Any] = {"api_key": self.api_key or "no-key"}
@@ -260,6 +267,8 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):
kwargs: Dict[str, Any] = {"input": text_input, "model": self.model_name}
if self.dimension and self._should_send_dimensions():
kwargs["dimensions"] = self.dimension
+ if self.encoding_format is not None:
+ kwargs["encoding_format"] = self.encoding_format
extra_body = self._build_extra_body(is_query=is_query)
if extra_body:
diff --git a/openviking_cli/utils/config/embedding_config.py b/openviking_cli/utils/config/embedding_config.py
index cd9c0bb9e..3774aeca1 100644
--- a/openviking_cli/utils/config/embedding_config.py
+++ b/openviking_cli/utils/config/embedding_config.py
@@ -1,6 +1,6 @@
# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
# SPDX-License-Identifier: AGPL-3.0
-from typing import Any, Optional, cast
+from typing import Any, Literal, Optional, cast
from pydantic import BaseModel, Field, model_validator
@@ -59,6 +59,16 @@ class EmbeddingModelConfig(BaseModel):
"or other OpenAI-compatible providers that require custom headers."
),
)
+ encoding_format: Optional[Literal["float", "base64"]] = Field(
+ default=None,
+ description=(
+ "Wire format for embedding values. Applies to OpenAI / Azure providers. "
+ "Leave unset to use the OpenAI Python SDK default (currently 'base64', "
+ "which is bandwidth-efficient and decoded client-side). Set to 'float' "
+ "to send/receive plain JSON arrays. This is the recommended workaround "
+ "when the upstream gateway cannot serialize base64 responses correctly."
+ ),
+ )
api_version: Optional[str] = Field(
default=None,
description="API version for Azure OpenAI (e.g., '2025-01-01-preview').",
@@ -470,6 +480,11 @@ class EmbeddingConfig(BaseModel):
**({"query_param": cfg.query_param} if cfg.query_param else {}),
**({"document_param": cfg.document_param} if cfg.document_param else {}),
**({"extra_headers": cfg.extra_headers} if cfg.extra_headers else {}),
+ **(
+ {"encoding_format": cfg.encoding_format}
+ if cfg.encoding_format is not None
+ else {}
+ ),
},
),
("azure", "dense"): (
@@ -486,6 +501,11 @@ class EmbeddingConfig(BaseModel):
**({"query_param": cfg.query_param} if cfg.query_param else {}),
**({"document_param": cfg.document_param} if cfg.document_param else {}),
**({"extra_headers": cfg.extra_headers} if cfg.extra_headers else {}),
+ **(
+ {"encoding_format": cfg.encoding_format}
+ if cfg.encoding_format is not None
+ else {}
+ ),
},
),
("volcengine", "dense"): (
diff --git a/tests/unit/test_encoding_format_embedding.py b/tests/unit/test_encoding_format_embedding.py
new file mode 100644
index 000000000..278a3b7df
--- /dev/null
+++ b/tests/unit/test_encoding_format_embedding.py
@@ -0,0 +1,142 @@
+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
+# SPDX-License-Identifier: AGPL-3.0
+"""Tests for the configurable encoding_format option on OpenAI-compatible embedders.
+
+Covers:
+ 1. EmbeddingModelConfig accepts encoding_format ('float' / 'base64' / unset).
+ 2. The OpenAI / Azure factory branches forward encoding_format to
+ OpenAIDenseEmbedder.
+ 3. OpenAIDenseEmbedder forwards the value to ``client.embeddings.create`` and
+ omits the kwarg entirely when unset, preserving the SDK's own default.
+"""
+
+from unittest.mock import MagicMock, patch
+
+import pytest
+from pydantic import ValidationError
+
+from openviking.models.embedder import OpenAIDenseEmbedder
+from openviking_cli.utils.config.embedding_config import EmbeddingConfig, EmbeddingModelConfig
+
+
+def _mock_openai_client():
+ """Return a mock openai.OpenAI client with a minimal valid embedding response."""
+ client = MagicMock()
+ client.embeddings.create.return_value = MagicMock(
+ data=[MagicMock(embedding=[0.1] * 8)],
+ usage=None,
+ )
+ return client
+
+
+class TestEncodingFormatConfig:
+ def test_default_is_unset(self):
+ cfg = EmbeddingModelConfig(
+ provider="openai",
+ model="text-embedding-3-small",
+ api_key="sk-test",
+ )
+ assert cfg.encoding_format is None
+
+ def test_accepts_float(self):
+ cfg = EmbeddingModelConfig(
+ provider="openai",
+ model="text-embedding-3-small",
+ api_key="sk-test",
+ encoding_format="float",
+ )
+ assert cfg.encoding_format == "float"
+
+ def test_accepts_base64(self):
+ cfg = EmbeddingModelConfig(
+ provider="openai",
+ model="text-embedding-3-small",
+ api_key="sk-test",
+ encoding_format="base64",
+ )
+ assert cfg.encoding_format == "base64"
+
+ def test_rejects_unknown_value(self):
+ with pytest.raises(ValidationError):
+ EmbeddingModelConfig(
+ provider="openai",
+ model="text-embedding-3-small",
+ api_key="sk-test",
+ encoding_format="hex", # type: ignore[arg-type]
+ )
+
+
+class TestEncodingFormatRuntime:
+ @patch("openviking.models.embedder.openai_embedders.openai.OpenAI")
+ def test_kwarg_forwarded_when_set(self, mock_openai_class):
+ mock_openai_class.return_value = _mock_openai_client()
+
+ embedder = OpenAIDenseEmbedder(
+ model_name="bge-m3",
+ api_key="sk-test",
+ api_base="https://gateway.example.com/v1",
+ dimension=1024,
+ encoding_format="float",
+ )
+ embedder.embed("hello")
+
+ call_kwargs = mock_openai_class.return_value.embeddings.create.call_args[1]
+ assert call_kwargs["encoding_format"] == "float"
+
+ @patch("openviking.models.embedder.openai_embedders.openai.OpenAI")
+ def test_kwarg_omitted_when_unset(self, mock_openai_class):
+ mock_openai_class.return_value = _mock_openai_client()
+
+ embedder = OpenAIDenseEmbedder(
+ model_name="text-embedding-3-small",
+ api_key="sk-test",
+ )
+ embedder.embed("hello")
+
+ call_kwargs = mock_openai_class.return_value.embeddings.create.call_args[1]
+ assert "encoding_format" not in call_kwargs
+
+
+class TestEncodingFormatFactory:
+ @patch("openai.OpenAI")
+ def test_openai_factory_forwards(self, mock_openai_class):
+ mock_openai_class.return_value = _mock_openai_client()
+
+ cfg = EmbeddingModelConfig(
+ provider="openai",
+ model="text-embedding-3-small",
+ api_key="sk-test",
+ encoding_format="float",
+ )
+ embedder = EmbeddingConfig(dense=cfg)._create_embedder("openai", "dense", cfg)
+ assert isinstance(embedder, OpenAIDenseEmbedder)
+ assert embedder.encoding_format == "float"
+
+ @patch("openai.AzureOpenAI")
+ def test_azure_factory_forwards(self, mock_azure_class):
+ mock_azure_class.return_value = _mock_openai_client()
+
+ cfg = EmbeddingModelConfig(
+ provider="azure",
+ model="text-embedding-3-small",
+ api_key="sk-test",
+ api_base="https://example.openai.azure.com",
+ api_version="2024-12-01-preview",
+ encoding_format="float",
+ )
+ embedder = EmbeddingConfig(dense=cfg)._create_embedder("azure", "dense", cfg)
+ assert isinstance(embedder, OpenAIDenseEmbedder)
+ assert embedder.encoding_format == "float"
+
+ @patch("openai.OpenAI")
+ def test_factory_omits_when_unset(self, mock_openai_class):
+ mock_openai_class.return_value = _mock_openai_client()
+
+ cfg = EmbeddingModelConfig(
+ provider="openai",
+ model="text-embedding-3-small",
+ api_key="sk-test",
+ )
+ embedder = EmbeddingConfig(dense=cfg)._create_embedder("openai", "dense", cfg)
+ assert isinstance(embedder, OpenAIDenseEmbedder)
+ assert embedder.encoding_format is None