Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
feat(embedder): expose encoding_format for OpenAI/Azure providers (#2092)
Simon Shi72b407ffa8ef
diff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.mdindex 1a8722d2d..9f5570998 100644--- a/docs/en/guides/01-configuration.md+++ b/docs/en/guides/01-configuration.md@@ -214,12 +214,13 @@ Embedding model configuration for vector search, supporting dense, sparse, and h| `max_retries` | int | Maximum retry attempts for transient embedding provider errors (`embedding.max_retries`, default: `3`; `0` disables retry) || `text_source` | str | Text used for vectorizing text files. `content_only` reads raw content, `summary_first` uses summary when available and falls back to content, `summary_only` uses only summary. Default: `content_only` || `max_input_tokens` | int | Maximum estimated raw text tokens sent to the embedding model when content is used. Default: `4096` |-| `provider` | str | `"volcengine"`, `"openai"`, `"vikingdb"`, `"jina"`, `"voyage"`, `"dashscope"`, or `"gemini"` |+| `provider` | str | `"openai"`, `"azure"`, `"volcengine"`, `"vikingdb"`, `"jina"`, `"ollama"`, `"gemini"`, `"voyage"`, `"dashscope"`, `"minimax"`, `"cohere"`, `"litellm"`, or `"local"` || `api_key` | str | API key || `model` | str | Model name || `dimension` | int | Vector dimension. For Voyage, this maps to `output_dimension` || `input` | str | Input type: `"text"` or `"multimodal"` || `batch_size` | int | Batch size for embedding requests |+| `encoding_format` | str | (OpenAI / Azure only) Wire format for embedding values: `"float"` or `"base64"`. Leave unset to use the OpenAI Python SDK default. Set to `"float"` when the upstream gateway cannot deserialize base64 embedding payloads correctly. |`embedding.max_retries` only applies to transient errors such as `429`, `5xx`, timeouts, and connection failures. Permanent errors such as `400`, `401`, `403`, and `AccountOverdue` are not retried automatically. The backoff strategy is exponential backoff with jitter, starting at `0.5s` and capped at `8s`.@@ -256,12 +257,57 @@ With `input: "multimodal"`, OpenViking can embed text, images (PNG, JPG, etc.),**Supported providers:**- `openai`: OpenAI Embedding API+- `azure`: Azure OpenAI Embedding API- `volcengine`: Volcengine Embedding API- `vikingdb`: VikingDB Embedding API- `jina`: Jina AI Embedding API+- `ollama`: Ollama local OpenAI-compatible Embedding API- `voyage`: Voyage AI Embedding API- `minimax`: MiniMax Embedding API+- `cohere`: Cohere Embedding API- `gemini`: Google Gemini Embedding API (text-only; requires `google-genai>=1.0.0`)+- `dashscope`: DashScope (Alibaba Tongyi) Embedding API+- `litellm`: LiteLLM Embedding API+- `local`: Local GGUF embedding models++**OpenAI-compatible provider example with JSON float embeddings:**++```json+{+ "embedding": {+ "dense": {+ "provider": "openai",+ "api_key": "your-api-key",+ "api_base": "https://your-openai-compatible-endpoint/v1",+ "model": "text-embedding-3-large",+ "dimension": 3072,+ "encoding_format": "float"+ }+ }+}+```++`encoding_format` is optional and is only forwarded for `provider: "openai"` and `provider: "azure"`. Leave it unset for the OpenAI Python SDK default. Set it to `"float"` when an OpenAI-compatible upstream gateway cannot deserialize base64 embedding payloads correctly.++**Azure OpenAI provider example with JSON float embeddings:**++```json+{+ "embedding": {+ "dense": {+ "provider": "azure",+ "api_key": "your-azure-api-key",+ "api_base": "https://your-resource-name.openai.azure.com",+ "api_version": "2025-01-01-preview",+ "model": "your-embedding-deployment-name",+ "dimension": 3072,+ "encoding_format": "float"+ }+ }+}+```++For Azure OpenAI, `model` must be the embedding deployment name configured in Azure.**minimax provider example:**@@ -1289,7 +1335,8 @@ For detailed encryption explanations, see [Data Encryption](../concepts/10-encry"api_key": "string","model": "string","dimension": 1024,- "input": "multimodal"+ "input": "multimodal",+ "encoding_format": "float|base64"}},"vlm": {diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.mdindex 2137ef221..dadd5c15f 100644--- a/docs/zh/guides/01-configuration.md+++ b/docs/zh/guides/01-configuration.md@@ -216,12 +216,13 @@ openviking-server doctor| `max_retries` | int | Embedding provider 瞬时错误的最大重试次数(`embedding.max_retries`,默认:`3`;`0` 表示禁用重试) || `text_source` | str | 文本文件向量化时使用的文本来源。`content_only` 读取原文内容;`summary_first` 优先使用摘要,没有摘要时回退到原文;`summary_only` 只使用摘要。默认:`content_only` || `max_input_tokens` | int | 使用原文内容向量化时,发送给 embedding 模型的最大估算 token 数。默认:`4096` |-| `provider` | str | `"volcengine"`、`"openai"`、`"vikingdb"`、`"jina"`、`"voyage"`、`"minimax"`、`"dashscope"` 或 `"gemini"` |+| `provider` | str | `"openai"`、`"azure"`、`"volcengine"`、`"vikingdb"`、`"jina"`、`"ollama"`、`"gemini"`、`"voyage"`、`"dashscope"`、`"minimax"`、`"cohere"`、`"litellm"` 或 `"local"` || `api_key` | str | API Key || `model` | str | 模型名称 || `dimension` | int | 向量维度 || `input` | str | 输入类型:`"text"` 或 `"multimodal"` || `batch_size` | int | 批量请求大小 |+| `encoding_format` | str | (仅 OpenAI / Azure)Embedding 值的传输格式:`"float"` 或 `"base64"`。留空时使用 OpenAI Python SDK 默认值;当上游网关无法正确处理 base64 embedding payload 时,可设置为 `"float"`。 |`embedding.max_retries` 仅对瞬时错误生效,例如 `429`、`5xx`、超时和连接错误;`400`、`401`、`403`、`AccountOverdue` 这类永久错误不会自动重试。退避策略为指数退避,初始延迟 `0.5s`,上限 `8s`,并带随机抖动。@@ -258,13 +259,57 @@ openviking-server doctor**支持的 provider:**- `openai`: OpenAI Embedding API+- `azure`: Azure OpenAI Embedding API- `volcengine`: 火山引擎 Embedding API- `vikingdb`: VikingDB Embedding API- `jina`: Jina AI Embedding API+- `ollama`: Ollama 本地 OpenAI 兼容 Embedding API- `voyage`: Voyage AI Embedding API- `minimax`: MiniMax Embedding API+- `cohere`: Cohere Embedding API- `gemini`: Google Gemini Embedding API(仅文本;需安装 `google-genai>=1.0.0`)- `dashscope`: DashScope(阿里通义)Embedding API+- `litellm`: LiteLLM Embedding API+- `local`: 本地 GGUF embedding 模型++**OpenAI 兼容 provider 的 JSON float embedding 示例:**++```json+{+ "embedding": {+ "dense": {+ "provider": "openai",+ "api_key": "your-api-key",+ "api_base": "https://your-openai-compatible-endpoint/v1",+ "model": "text-embedding-3-large",+ "dimension": 3072,+ "encoding_format": "float"+ }+ }+}+```++`encoding_format` 是可选字段,只会传给 `provider: "openai"` 和 `provider: "azure"`。留空时使用 OpenAI Python SDK 默认行为;如果 OpenAI 兼容上游网关无法正确反序列化 base64 embedding payload,可设置为 `"float"`。++**Azure OpenAI provider 的 JSON float embedding 示例:**++```json+{+ "embedding": {+ "dense": {+ "provider": "azure",+ "api_key": "your-azure-api-key",+ "api_base": "https://your-resource-name.openai.azure.com",+ "api_version": "2025-01-01-preview",+ "model": "your-embedding-deployment-name",+ "dimension": 3072,+ "encoding_format": "float"+ }+ }+}+```++对于 Azure OpenAI,`model` 必须填写 Azure 中配置的 embedding deployment name。**minimax provider 配置示例:**@@ -1263,7 +1308,8 @@ openviking add-resource ./docs --exclude "*.tmp""api_key": "string","model": "string","dimension": 1024,- "input": "multimodal"+ "input": "multimodal",+ "encoding_format": "float|base64"}},"vlm": {diff --git a/openviking/models/embedder/openai_embedders.py b/openviking/models/embedder/openai_embedders.pyindex 554a159af..311044e7a 100644--- a/openviking/models/embedder/openai_embedders.py+++ b/openviking/models/embedder/openai_embedders.py@@ -2,7 +2,7 @@# SPDX-License-Identifier: AGPL-3.0"""OpenAI Embedder Implementation"""-from typing import Any, Dict, List, Optional+from typing import Any, Dict, List, Literal, Optionalimport openai@@ -76,6 +76,7 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):input_type: Optional[str] = None,provider: str = "openai",configured_provider: Optional[str] = None,+ encoding_format: Optional[Literal["float", "base64"]] = None,):"""Initialize OpenAI-Compatible Dense Embedder@@ -100,6 +101,11 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):config: Additional configuration dictextra_headers: Extra HTTP headers to include in API requests (e.g., for OpenRouter:{'HTTP-Referer': 'https://your-site.com', 'X-Title': 'Your App'})+ encoding_format: Wire format for embedding values. ``None`` (default) lets the+ OpenAI Python SDK pick its default (currently ``base64``). Set to+ ``"float"`` to send/receive plain JSON arrays — the recommended+ workaround when the upstream gateway cannot deserialize base64+ embedding payloads correctly.Raises:ValueError: If api_key is not provided and env vars are not set@@ -118,6 +124,7 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):self.dimension = dimensionself.query_param = query_paramself.document_param = document_param+ self.encoding_format = encoding_formatself._provider = provider.lower()self.provider = (configured_provider or provider).lower()self._client_kwargs: Dict[str, Any] = {"api_key": self.api_key or "no-key"}@@ -260,6 +267,8 @@ class OpenAIDenseEmbedder(DenseEmbedderBase):kwargs: Dict[str, Any] = {"input": text_input, "model": self.model_name}if self.dimension and self._should_send_dimensions():kwargs["dimensions"] = self.dimension+ if self.encoding_format is not None:+ kwargs["encoding_format"] = self.encoding_formatextra_body = self._build_extra_body(is_query=is_query)if extra_body:diff --git a/openviking_cli/utils/config/embedding_config.py b/openviking_cli/utils/config/embedding_config.pyindex cd9c0bb9e..3774aeca1 100644--- a/openviking_cli/utils/config/embedding_config.py+++ b/openviking_cli/utils/config/embedding_config.py@@ -1,6 +1,6 @@# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.# SPDX-License-Identifier: AGPL-3.0-from typing import Any, Optional, cast+from typing import Any, Literal, Optional, castfrom pydantic import BaseModel, Field, model_validator@@ -59,6 +59,16 @@ class EmbeddingModelConfig(BaseModel):"or other OpenAI-compatible providers that require custom headers."),)+ encoding_format: Optional[Literal["float", "base64"]] = Field(+ default=None,+ description=(+ "Wire format for embedding values. Applies to OpenAI / Azure providers. "+ "Leave unset to use the OpenAI Python SDK default (currently 'base64', "+ "which is bandwidth-efficient and decoded client-side). Set to 'float' "+ "to send/receive plain JSON arrays. This is the recommended workaround "+ "when the upstream gateway cannot serialize base64 responses correctly."+ ),+ )api_version: Optional[str] = Field(default=None,description="API version for Azure OpenAI (e.g., '2025-01-01-preview').",@@ -470,6 +480,11 @@ class EmbeddingConfig(BaseModel):**({"query_param": cfg.query_param} if cfg.query_param else {}),**({"document_param": cfg.document_param} if cfg.document_param else {}),**({"extra_headers": cfg.extra_headers} if cfg.extra_headers else {}),+ **(+ {"encoding_format": cfg.encoding_format}+ if cfg.encoding_format is not None+ else {}+ ),},),("azure", "dense"): (@@ -486,6 +501,11 @@ class EmbeddingConfig(BaseModel):**({"query_param": cfg.query_param} if cfg.query_param else {}),**({"document_param": cfg.document_param} if cfg.document_param else {}),**({"extra_headers": cfg.extra_headers} if cfg.extra_headers else {}),+ **(+ {"encoding_format": cfg.encoding_format}+ if cfg.encoding_format is not None+ else {}+ ),},),("volcengine", "dense"): (diff --git a/tests/unit/test_encoding_format_embedding.py b/tests/unit/test_encoding_format_embedding.pynew file mode 100644index 000000000..278a3b7df--- /dev/null+++ b/tests/unit/test_encoding_format_embedding.py@@ -0,0 +1,142 @@+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.+# SPDX-License-Identifier: AGPL-3.0+"""Tests for the configurable encoding_format option on OpenAI-compatible embedders.++Covers:+ 1. EmbeddingModelConfig accepts encoding_format ('float' / 'base64' / unset).+ 2. The OpenAI / Azure factory branches forward encoding_format to+ OpenAIDenseEmbedder.+ 3. OpenAIDenseEmbedder forwards the value to ``client.embeddings.create`` and+ omits the kwarg entirely when unset, preserving the SDK's own default.+"""++from unittest.mock import MagicMock, patch++import pytest+from pydantic import ValidationError++from openviking.models.embedder import OpenAIDenseEmbedder+from openviking_cli.utils.config.embedding_config import EmbeddingConfig, EmbeddingModelConfig+++def _mock_openai_client():+ """Return a mock openai.OpenAI client with a minimal valid embedding response."""+ client = MagicMock()+ client.embeddings.create.return_value = MagicMock(+ data=[MagicMock(embedding=[0.1] * 8)],+ usage=None,+ )+ return client+++class TestEncodingFormatConfig:+ def test_default_is_unset(self):+ cfg = EmbeddingModelConfig(+ provider="openai",+ model="text-embedding-3-small",+ api_key="sk-test",+ )+ assert cfg.encoding_format is None++ def test_accepts_float(self):+ cfg = EmbeddingModelConfig(+ provider="openai",+ model="text-embedding-3-small",+ api_key="sk-test",+ encoding_format="float",+ )+ assert cfg.encoding_format == "float"++ def test_accepts_base64(self):+ cfg = EmbeddingModelConfig(+ provider="openai",+ model="text-embedding-3-small",+ api_key="sk-test",+ encoding_format="base64",+ )+ assert cfg.encoding_format == "base64"++ def test_rejects_unknown_value(self):+ with pytest.raises(ValidationError):+ EmbeddingModelConfig(+ provider="openai",+ model="text-embedding-3-small",+ api_key="sk-test",+ encoding_format="hex", # type: ignore[arg-type]+ )+++class TestEncodingFormatRuntime:+ @patch("openviking.models.embedder.openai_embedders.openai.OpenAI")+ def test_kwarg_forwarded_when_set(self, mock_openai_class):+ mock_openai_class.return_value = _mock_openai_client()++ embedder = OpenAIDenseEmbedder(+ model_name="bge-m3",+ api_key="sk-test",+ api_base="https://gateway.example.com/v1",+ dimension=1024,+ encoding_format="float",+ )+ embedder.embed("hello")++ call_kwargs = mock_openai_class.return_value.embeddings.create.call_args[1]+ assert call_kwargs["encoding_format"] == "float"++ @patch("openviking.models.embedder.openai_embedders.openai.OpenAI")+ def test_kwarg_omitted_when_unset(self, mock_openai_class):+ mock_openai_class.return_value = _mock_openai_client()++ embedder = OpenAIDenseEmbedder(+ model_name="text-embedding-3-small",+ api_key="sk-test",+ )+ embedder.embed("hello")++ call_kwargs = mock_openai_class.return_value.embeddings.create.call_args[1]+ assert "encoding_format" not in call_kwargs+++class TestEncodingFormatFactory:+ @patch("openai.OpenAI")+ def test_openai_factory_forwards(self, mock_openai_class):+ mock_openai_class.return_value = _mock_openai_client()++ cfg = EmbeddingModelConfig(+ provider="openai",+ model="text-embedding-3-small",+ api_key="sk-test",+ encoding_format="float",+ )+ embedder = EmbeddingConfig(dense=cfg)._create_embedder("openai", "dense", cfg)+ assert isinstance(embedder, OpenAIDenseEmbedder)+ assert embedder.encoding_format == "float"++ @patch("openai.AzureOpenAI")+ def test_azure_factory_forwards(self, mock_azure_class):+ mock_azure_class.return_value = _mock_openai_client()++ cfg = EmbeddingModelConfig(+ provider="azure",+ model="text-embedding-3-small",+ api_key="sk-test",+ api_base="https://example.openai.azure.com",+ api_version="2024-12-01-preview",+ encoding_format="float",+ )+ embedder = EmbeddingConfig(dense=cfg)._create_embedder("azure", "dense", cfg)+ assert isinstance(embedder, OpenAIDenseEmbedder)+ assert embedder.encoding_format == "float"++ @patch("openai.OpenAI")+ def test_factory_omits_when_unset(self, mock_openai_class):+ mock_openai_class.return_value = _mock_openai_client()++ cfg = EmbeddingModelConfig(+ provider="openai",+ model="text-embedding-3-small",+ api_key="sk-test",+ )+ embedder = EmbeddingConfig(dense=cfg)._create_embedder("openai", "dense", cfg)+ assert isinstance(embedder, OpenAIDenseEmbedder)+ assert embedder.encoding_format is None