Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
feat: add Jev choice-based reranking (#5486)
chenxiaobin-monkey1394d4769a34
diff --git a/docs/en/configuration/01-server.md b/docs/en/configuration/01-server.mdindex c19b91a6f..09454356d 100644--- a/docs/en/configuration/01-server.md+++ b/docs/en/configuration/01-server.md@@ -144,13 +144,14 @@ Changing the model or `dimension` can make existing vector collections incompati|---|---|---|---|| `provider` | `vikingdb`, `cohere`, `openai`, `litellm`, `jev` / `null` | `null` | Rerank service; inferred from credentials when omitted || `model` | string / `null` | `null` | OpenAI-compatible, LiteLLM, or Jev rerank model |+| `mode` | `noul`, `choice`, or `null` | `noul` | Jev rerank mode; `null` also uses `noul` || `threshold` | number | `0.1` | Minimum score considered relevant || `max_input_tokens` | integer; `0` or `>= 128` | `0` | Maximum estimated tokens per query-document pair; `0` disables truncation || `log_payloads` | boolean | `false` | Log complete rerank request and response payloads; may expose query and document content |Rerank has no separate `enabled` field. It becomes available when the required provider credentials are configured.-`jev` supports direct TypeSafe access (`https://api.typesafe.ai`, model `jev-latest`) and Vercel AI Gateway's TypeSafe-compatible endpoint (`https://ai-gateway.vercel.sh/typesafe`, model `typesafe-ai/jev`) through the existing `api_base` and `model` fields; both speak the same protocol. It sends the query and candidate documents as structured `state`, asks one independent relevance question per candidate, and uses each yes probability as its rerank score. Setting `provider` explicitly requires the credentials that provider needs: `ak` and `sk` for `vikingdb`, `api_key` for `cohere` and `jev`, `api_key` and `api_base` for `openai`, `model` for `litellm`. An incomplete block is rejected when the configuration loads.+`jev` supports direct TypeSafe access (`https://api.typesafe.ai`, model `jev-latest`) and Vercel AI Gateway's TypeSafe-compatible endpoint (`https://ai-gateway.vercel.sh/typesafe`, model `typesafe-ai/jev`) through the existing `api_base` and `model` fields; both speak the same protocol. Its default `noul` mode scores each candidate independently. The optional `choice` mode compares all candidates and returns relative probabilities; set `threshold` to `0` when using it. Setting `provider` explicitly requires the credentials that provider needs: `ak` and `sk` for `vikingdb`, `api_key` for `cohere` and `jev`, `api_key` and `api_base` for `openai`, `model` for `litellm`. An incomplete block is rejected when the configuration loads.## Retrieval Settingsdiff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.mdindex 5586cac0b..597d21697 100644--- a/docs/en/guides/01-configuration.md+++ b/docs/en/guides/01-configuration.md@@ -1072,10 +1072,21 @@ Notes for Vercel:- `api_base` must include the `/typesafe` suffix. `https://ai-gateway.vercel.sh/v1`is Vercel's own evaluate protocol and is not supported by the adapter.-The Jev adapter sends the query and candidate documents as structured System One-`state`, then asks one independent Noul relevance question per candidate. Each returned-yes probability becomes that document's rerank score. All questions are evaluated in-parallel in one request, and scores do not compete or have to sum to 1.+The Jev adapter sends the query and candidate documents as shared System One `state`+and supports two modes:++- `noul` (default): one independent relevance question per document, using its returned yes+ probability as the score. All questions are sent in one HTTP request, and document+ scores do not have to sum to 1.+- `choice`: one question compares all candidate documents, with `candidate_{index}`+ mapped to input indices. Scores come from `probabilities` in document order.++Select the mode with `rerank.mode`; omitting it or setting it to `null` keeps the+default `noul` behavior. Choice scores are relative probabilities summing to 1 within+the candidate pool, not absolute relevance probabilities, so they are unsuitable for+absolute score filtering. When enabling Choice, set `"threshold": 0`. The MCP `find`+and `search` `min_score` argument overrides that setting, so pass `min_score=0` when+using Choice through MCP.**Parameters**@@ -1088,6 +1099,7 @@ parallel in one request, and scores do not compete or have to sum to 1.| `api_key` | str | API key (for `openai`, `cohere`, or `jev` providers) || `api_base` | str | Endpoint URL (for `openai` or `jev`; Jev defaults to `https://api.typesafe.ai`, Vercel uses `https://ai-gateway.vercel.sh/typesafe`) || `model` | str | Model name for OpenAI-compatible, LiteLLM, or Jev providers |+| `mode` | `"noul"`, `"choice"`, or `null` | Jev rerank mode. `null` and omission use `"noul"` || `timeout` | float | HTTP request timeout in seconds for HTTP rerank providers, including Jev. Default: `30.0` || `max_input_tokens` | int | Maximum estimated raw-text tokens in each query-document pair sent to the reranker. Oversized inputs retain their beginning and end. `0` disables. Default: `0` || `log_payloads` | bool | Log complete rerank request and response payloads. May expose query and document content. Default: `false` |@@ -1099,7 +1111,7 @@ parallel in one request, and scores do not compete or have to sum to 1.- `cohere`: Cohere Rerank API- `openai`: OpenAI-compatible Rerank API- `litellm`: LiteLLM Rerank API-- `jev`: Jev (TypeSafe System One) structured-decision API; each document receives an independent Noul relevance score+- `jev`: Jev (TypeSafe System One) structured-decision API with Choice comparison and independent Noul scoringIf rerank is not configured, search uses vector similarity only.diff --git a/docs/zh/configuration/01-server.md b/docs/zh/configuration/01-server.mdindex d9915f29d..5a9a225ae 100644--- a/docs/zh/configuration/01-server.md+++ b/docs/zh/configuration/01-server.md@@ -144,13 +144,14 @@ API 型 `embedding`、`vlm`、`query_planner` 和 `rerank` 配置会复用部分|---|---|---|---|| `provider` | `vikingdb`、`cohere`、`openai`、`litellm`、`jev` / `null` | `null` | Rerank 服务类型;省略时根据凭证字段推断 || `model` | string / `null` | `null` | OpenAI 兼容、LiteLLM 或 Jev Rerank 模型 |+| `mode` | `noul`、`choice` 或 `null` | `noul` | Jev Rerank 模式;`null` 同样使用 `noul` || `threshold` | number | `0.1` | 判定结果相关的最低分数 || `max_input_tokens` | integer;`0` 或 `>= 128` | `0` | 每个 query-document pair 的最大估算 token;`0` 表示不截断 || `log_payloads` | boolean | `false` | 记录完整 rerank 请求和响应;日志可能包含 query 和文档内容 |Rerank 没有单独的 `enabled` 字段;配置了对应 provider 所需的凭证后才会启用。-`jev` 通过现有 `api_base` 和 `model` 字段同时支持 TypeSafe 直连(`https://api.typesafe.ai`,模型 `jev-latest`)和 Vercel AI Gateway 的 TypeSafe 兼容端点(`https://ai-gateway.vercel.sh/typesafe`,模型 `typesafe-ai/jev`),两者协议相同。它将 query 和候选文档作为结构化 `state`,为每个候选提出一个独立的相关性问题,并将各自的 yes 概率作为 rerank 分数。显式指定 `provider` 时必须提供该 provider 所需的凭证:`vikingdb` 需要 `ak` 和 `sk`,`cohere` 和 `jev` 需要 `api_key`,`openai` 需要 `api_key` 和 `api_base`,`litellm` 需要 `model`。凭证不全的配置在加载时即被拒绝。+`jev` 通过现有 `api_base` 和 `model` 字段同时支持 TypeSafe 直连(`https://api.typesafe.ai`,模型 `jev-latest`)和 Vercel AI Gateway 的 TypeSafe 兼容端点(`https://ai-gateway.vercel.sh/typesafe`,模型 `typesafe-ai/jev`),两者协议相同。默认 `noul` 模式独立评估每个候选;可选的 `choice` 模式横向比较所有候选并返回相对概率,使用时应将 `threshold` 设为 `0`。显式指定 `provider` 时必须提供该 provider 所需的凭证:`vikingdb` 需要 `ak` 和 `sk`,`cohere` 和 `jev` 需要 `api_key`,`openai` 需要 `api_key` 和 `api_base`,`litellm` 需要 `model`。凭证不全的配置在加载时即被拒绝。## 检索配置diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.mdindex ab0c4461f..4264e3dba 100644--- a/docs/zh/guides/01-configuration.md+++ b/docs/zh/guides/01-configuration.md@@ -1036,9 +1036,18 @@ PDF 解析配置。支持三种策略:`local`(本地 pdfplumber)、`mineru- `api_base` 必须带 `/typesafe` 后缀;`https://ai-gateway.vercel.sh/v1` 是Vercel 自有的 evaluate 协议,适配器不支持。-Jev 适配器将 query 和候选文档作为结构化 System One `state`,并为每个候选-提出一个独立的 Noul 相关性问题。每个问题返回的 yes 概率就是该文档的 rerank-分数。所有问题在一次请求中并行计算,各文档分数互不竞争,也不要求总和为 1。+Jev 适配器将 query 和候选文档作为共享的 System One `state`,支持两种模式:++- `noul`(默认):为每篇文档提出独立的相关性问题,返回的 yes 概率作为分数。+ 所有问题放在一次 HTTP 请求中,各文档分数不要求总和为 1。+- `choice`:用一个问题横向比较所有候选文档,以 `candidate_{index}` 对应输入+ 下标,并按原文档顺序读取 `probabilities` 作为分数。++通过 `rerank.mode` 选择模式;未配置或配置为 `null` 时默认使用 `noul`。Choice+返回候选池内总和为 1 的相对概率,不是绝对相关概率,不适合使用绝对分数阈值。+启用 Choice 时建议同时设置 `"threshold": 0`。MCP `find` 和 `search` 的+`min_score` 会覆盖该配置,因此通过 MCP 使用 Choice 时还需要显式传入+`min_score=0`。**参数**@@ -1051,6 +1060,7 @@ Jev 适配器将 query 和候选文档作为结构化 System One `state`,并| `api_key` | str | API Key(用于 `openai`、`cohere` 或 `jev` 提供方) || `api_base` | str | 接口地址(用于 `openai` 或 `jev`;Jev 默认为 `https://api.typesafe.ai`,Vercel 使用 `https://ai-gateway.vercel.sh/typesafe`) || `model` | str | 模型名称(用于 OpenAI 兼容、LiteLLM 或 `jev` 提供方) |+| `mode` | `"noul"`、`"choice"` 或 `null` | Jev Rerank 模式;省略或 `null` 时使用 `"noul"` || `timeout` | float | HTTP Rerank provider(包括 Jev)的请求超时时间,单位为秒。默认:`30.0` || `max_input_tokens` | int | 每个 query-document 对发送给 reranker 的最大估算原始文本 token 数;超长输入会保留开头和结尾。`0` 表示不截断。默认:`0` || `log_payloads` | bool | 记录完整 rerank 请求和响应;日志可能包含 query 和文档内容。默认:`false` |@@ -1062,7 +1072,7 @@ Jev 适配器将 query 和候选文档作为结构化 System One `state`,并- `cohere`: Cohere Rerank API- `openai`: OpenAI 兼容的 Rerank 接口- `litellm`: LiteLLM Rerank 接口-- `jev`: Jev (TypeSafe System One) 结构化判定接口,为每篇文档独立计算 Noul 相关性分数+- `jev`: Jev (TypeSafe System One) 结构化判定接口,支持 Choice 横向比较和 Noul 独立评分如果未配置 Rerank,搜索仅使用向量相似度。diff --git a/openviking/models/rerank/jev_rerank.py b/openviking/models/rerank/jev_rerank.pyindex 9f73831a3..351248cf2 100644--- a/openviking/models/rerank/jev_rerank.py+++ b/openviking/models/rerank/jev_rerank.py@@ -3,9 +3,8 @@"""Jev (TypeSafe System One) Rerank API Client.-TypeSafe has no native rerank endpoint; each document is evaluated by an-independent Noul question against the query. The returned yes probability is-used as the relevance score.+Choice compares candidates in one question and returns relative probabilities.+Noul evaluates each document independently and returns its yes probability.Speaks only the TypeSafe System One protocol. Vercel AI Gateway exposes aTypeSafe-compatible endpoint (https://ai-gateway.vercel.sh/typesafe), so routing@@ -26,6 +25,81 @@ from openviking_cli.utils import get_loggerlogger = get_logger(__name__)+MODE_NOUL = "noul"+MODE_CHOICE = "choice"+++def _validate_score(score: object) -> float:+ if isinstance(score, bool) or not isinstance(score, (int, float)) or not 0 <= score <= 1:+ raise ValueError(f"Invalid relevance score: {score!r}")+ return float(score)+++class NoulRerankMode:+ """One independent relevance question per document."""++ def build_questions(self, documents: List[str]) -> Dict[str, dict]:+ """Build one independent relevance question per document."""+ return {+ f"relevance_{index}": {+ "type": MODE_NOUL,+ "instructions": {+ "candidate_index": index,+ "question": (+ "Does candidate_documents[candidate_index] answer or match "+ "the retrieval intent of query?"+ ),+ },+ "criteria": {+ "true": "The candidate directly answers or is relevant to the query",+ "false": "The candidate does not answer and is not relevant to the query",+ },+ }+ for index in range(len(documents))+ }++ def parse_scores(self, answers: dict, document_count: int) -> List[float]:+ scores = []+ for index in range(document_count):+ answer = answers.get(f"relevance_{index}")+ if not isinstance(answer, dict) or answer.get("type") != MODE_NOUL:+ raise ValueError(f"Missing or malformed Noul answer for item {index}")+ scores.append(_validate_score(answer.get("noul")))+ return scores+++class ChoiceRerankMode:+ """One choice question comparing all documents."""++ def build_questions(self, documents: List[str]) -> Dict[str, dict]:+ return {+ "relevance": {+ "type": MODE_CHOICE,+ "instructions": (+ "Which candidate document best answers or matches the retrieval intent of query?"+ ),+ "criteria": {+ f"candidate_{index}": (+ f"candidate_documents[{index}] directly answers or is relevant to the query"+ )+ for index in range(len(documents))+ },+ }+ }++ def parse_scores(self, answers: dict, document_count: int) -> List[float]:+ answer = answers.get("relevance")+ if not isinstance(answer, dict) or answer.get("type") != MODE_CHOICE:+ raise ValueError("Missing or malformed Choice answer")+ probabilities = answer.get("probabilities")+ if not isinstance(probabilities, dict):+ raise ValueError("Choice answer has no probabilities object")+ # Use candidate keys, never response iteration order or winner confidence.+ return [+ _validate_score(probabilities.get(f"candidate_{index}"))+ for index in range(document_count)+ ]+class JevRerankClient(RerankBase):"""Jev rerank client — same interface as VikingDB RerankClient."""@@ -37,8 +111,14 @@ class JevRerankClient(RerankBase):api_base: str = "https://api.typesafe.ai",timeout: float = 30.0,log_payloads: bool = False,+ mode: str = MODE_NOUL,):super().__init__()+ modes = {MODE_NOUL: NoulRerankMode, MODE_CHOICE: ChoiceRerankMode}+ if mode not in modes:+ raise ValueError(f"Unknown Jev rerank mode: {mode!r}")+ self.mode = mode+ self._strategy = modes[mode]()self.api_key = api_keyself.model_name = model_nameself.api_base = api_base.rstrip("/")@@ -58,26 +138,6 @@ class JevRerankClient(RerankBase):timeout=timeout,)- def _build_questions(self, documents: List[str]) -> Dict[str, dict]:- """Build one independent relevance question per document."""- return {- f"relevance_{index}": {- "type": "noul",- "instructions": {- "candidate_index": index,- "question": (- "Does candidate_documents[candidate_index] answer or match "- "the retrieval intent of query?"- ),- },- "criteria": {- "true": "The candidate directly answers or is relevant to the query",- "false": "The candidate does not answer and is not relevant to the query",- },- }- for index in range(len(documents))- }-def rerank_batch(self, query: str, documents: List[str]) -> Optional[List[float]]:"""Rerank documents against a query using the Jev System One API.@@ -87,14 +147,12 @@ class JevRerankClient(RerankBase):if not documents:return []- questions = self._build_questions(documents)- body = {- "model": self.model_name,- "state": {"query": query, "candidate_documents": documents},- "questions": questions,- }-try:+ body = {+ "model": self.model_name,+ "state": {"query": query, "candidate_documents": documents},+ "questions": self._strategy.build_questions(documents),+ }if self.log_payloads:logger.warning("[JevRerank] Request items=%s payload=%s",@@ -119,17 +177,7 @@ class JevRerankClient(RerankBase):logger.warning("[JevRerank] Response has no answers object")return None- scores = []- for index in range(len(documents)):- answer = answers.get(f"relevance_{index}")- if not isinstance(answer, dict) or answer.get("type") != "noul":- logger.warning("[JevRerank] Missing or malformed answer for item %s", index)- return None- score = answer.get("noul")- if not isinstance(score, (int, float)) or not 0 <= score <= 1:- logger.warning("[JevRerank] Invalid relevance score for item %s", index)- return None- scores.append(float(score))+ scores = self._strategy.parse_scores(answers, len(documents))usage = data.get("usage") if isinstance(data.get("usage"), dict) else {}input_tokens = usage.get("input_tokens", 0)@@ -170,4 +218,5 @@ class JevRerankClient(RerankBase):api_base=api_base,timeout=config.timeout,log_payloads=config.log_payloads,+ mode=config.mode or MODE_NOUL,)diff --git a/openviking_cli/utils/config/rerank_config.py b/openviking_cli/utils/config/rerank_config.pyindex 96454308c..62ece0963 100644--- a/openviking_cli/utils/config/rerank_config.py+++ b/openviking_cli/utils/config/rerank_config.py@@ -30,6 +30,10 @@ class RerankConfig(BaseModel):model: Optional[str] = Field(default=None, description="Model name for OpenAI-compatible, LiteLLM, or Jev providers")+ mode: Optional[str] = Field(+ default="noul",+ description="Jev rerank mode: 'noul' or 'choice'",+ )extra_headers: Optional[Dict[str, str]] = Field(default=None, description="Extra HTTP headers for OpenAI-compatible providers"@@ -84,6 +88,11 @@ class RerankConfig(BaseModel):raise ValueError("Rerank max_input_tokens must be 0 or at least 128")provider = self._effective_provider()+ if provider == "jev" and self.mode is not None:+ self.mode = self.mode.strip().lower()+ if self.mode not in ("noul", "choice"):+ raise ValueError("Jev rerank mode must be one of ['noul', 'choice']")+if provider and provider not in ["vikingdb","cohere",diff --git a/tests/unit/models/rerank/test_jev_rerank.py b/tests/unit/models/rerank/test_jev_rerank.pyindex 25df84a03..6858e9dbe 100644--- a/tests/unit/models/rerank/test_jev_rerank.py+++ b/tests/unit/models/rerank/test_jev_rerank.py@@ -8,6 +8,7 @@ import httpximport pytestfrom openviking.models.rerank import JevRerankClient, RerankClient+from openviking.models.rerank.jev_rerank import MODE_CHOICE, MODE_NOULfrom openviking_cli.utils.config.rerank_config import RerankConfig@@ -41,6 +42,7 @@ class TestJevRerankClient:client = JevRerankClient(api_key="test-key")scores = client.rerank_batch("What is UCW?", ["doc A", "doc B", "doc C"])+ assert client.mode == MODE_NOULassert scores == [0.97, 0.04, 0.07]body = mock_client.post.call_args[1]["json"]assert body["model"] == "jev-latest"@@ -64,6 +66,7 @@ class TestJevRerankClient:api_key="vercel-key",api_base="https://ai-gateway.vercel.sh/typesafe",model_name="typesafe-ai/jev",+ mode=MODE_NOUL,)assert client.rerank_batch("query", ["first", "second"]) == [0.91, 0.08]@@ -80,7 +83,7 @@ class TestJevRerankClient:mock_client_class.return_value = mock_clientmock_client.post.return_value = _mock_response(_systemone_payload([0.9]))- client = JevRerankClient(api_key="secret-key", log_payloads=True)+ client = JevRerankClient(api_key="secret-key", log_payloads=True, mode=MODE_NOUL)assert client.rerank_batch("query", ["document"]) == [0.9]messages = [str(call) for call in mock_log.call_args_list]@@ -95,7 +98,7 @@ class TestJevRerankClient:mock_client_class.return_value = mock_clientmock_client.post.return_value = _mock_response(_systemone_payload([0.9]))- client = JevRerankClient(api_key="secret-key")+ client = JevRerankClient(api_key="secret-key", mode=MODE_NOUL)assert client.rerank_batch("query", ["document"]) == [0.9]mock_log.assert_not_called()@@ -111,7 +114,7 @@ class TestJevRerankClient:error = httpx.HTTPStatusError("401", request=MagicMock(), response=MagicMock())mock_client.post.return_value = _mock_response({}, status_error=error)- client = JevRerankClient(api_key="bad-key")+ client = JevRerankClient(api_key="bad-key", mode=MODE_NOUL)assert client.rerank_batch("query", ["doc"]) is None@patch("openviking.models.rerank.jev_rerank.httpx.Client")@@ -122,7 +125,7 @@ class TestJevRerankClient:{"answers": {}, "usage": {"input_tokens": 1, "output_tokens": 1}})- client = JevRerankClient(api_key="key")+ client = JevRerankClient(api_key="key", mode=MODE_NOUL)assert client.rerank_batch("q", ["doc"]) is None@patch("openviking.models.rerank.jev_rerank.httpx.Client")@@ -131,7 +134,7 @@ class TestJevRerankClient:mock_client_class.return_value = mock_clientmock_client.post.return_value = _mock_response(_systemone_payload([0.2, 0.8, 0.4]))- client = JevRerankClient(api_key="key")+ client = JevRerankClient(api_key="key", mode=MODE_NOUL)scores = client.rerank_batch("q", ["a", "b", "c"])assert scores == [0.2, 0.8, 0.4]@@ -144,7 +147,7 @@ class TestJevRerankClient:mock_client_class.return_value = mock_clientmock_client.post.return_value = _mock_response(_systemone_payload([1.1]))- client = JevRerankClient(api_key="key")+ client = JevRerankClient(api_key="key", mode=MODE_NOUL)assert client.rerank_batch("q", ["doc"]) is None@patch("openviking.models.rerank.jev_rerank.httpx.Client")@@ -156,11 +159,141 @@ class TestJevRerankClient:mock_client.close.assert_called_once()+class TestChoiceRerank:+ @patch("openviking.models.rerank.jev_rerank.httpx.Client")+ def test_choice_preserves_document_order(self, mock_client_class):+ mock_client = mock_client_class.return_value+ mock_client.post.return_value = _mock_response(+ {+ "answers": {+ "relevance": {+ "type": "choice",+ "choice": "candidate_1",+ "confidence": 0.55,+ "probabilities": {+ "candidate_2": 0.2,+ "candidate_1": 0.7,+ "candidate_0": 0.1,+ },+ }+ },+ "usage": {"input_tokens": None, "output_tokens": 1},+ }+ )+ client = JevRerankClient(api_key="key", mode=MODE_CHOICE)+ assert client.mode == MODE_CHOICE+ assert client.rerank_batch("query", ["first", "second", "third"]) == [0.1, 0.7, 0.2]+ body = mock_client.post.call_args.kwargs["json"]+ assert body["state"] == {+ "query": "query",+ "candidate_documents": ["first", "second", "third"],+ }+ assert body["questions"] == {+ "relevance": {+ "type": "choice",+ "instructions": (+ "Which candidate document best answers or matches the retrieval intent of query?"+ ),+ "criteria": {+ "candidate_0": (+ "candidate_documents[0] directly answers or is relevant to the query"+ ),+ "candidate_1": (+ "candidate_documents[1] directly answers or is relevant to the query"+ ),+ "candidate_2": (+ "candidate_documents[2] directly answers or is relevant to the query"+ ),+ },+ }+ }+ mock_client.post.assert_called_once_with("https://api.typesafe.ai/v1/systemone", json=body)++ @pytest.mark.parametrize("count", [1, 30])+ @patch("openviking.models.rerank.jev_rerank.httpx.Client")+ def test_choice_does_not_impose_candidate_limit(self, mock_client_class, count):+ probabilities = {f"candidate_{index}": 1 / count for index in range(count)}+ mock_client_class.return_value.post.return_value = _mock_response(+ {"answers": {"relevance": {"type": "choice", "probabilities": probabilities}}}+ )+ client = JevRerankClient(api_key="key", mode=MODE_CHOICE)+ assert client.rerank_batch("query", ["doc"] * count) == [1 / count] * count+ criteria = mock_client_class.return_value.post.call_args.kwargs["json"]["questions"][+ "relevance"+ ]["criteria"]+ assert list(criteria) == list(probabilities)++ @patch("openviking.models.rerank.jev_rerank.httpx.Client")+ def test_choice_empty_input_does_not_send_request(self, mock_client_class):+ client = JevRerankClient(api_key="key", mode=MODE_CHOICE)+ assert client.rerank_batch("query", []) == []+ mock_client_class.return_value.post.assert_not_called()++ @pytest.mark.parametrize(+ "answer",+ [+ None,+ {"type": "noul", "noul": 0.8},+ {"type": "choice", "choice": "candidate_0", "confidence": 0.8},+ {"type": "choice", "probabilities": []},+ {"type": "choice", "probabilities": {"candidate_0": 0.8}},+ {+ "type": "choice",+ "probabilities": {"candidate_0": 0.8, "candidate_1": "0.2"},+ },+ {"type": "choice", "probabilities": {"candidate_0": 0.8, "candidate_1": True}},+ {"type": "choice", "probabilities": {"candidate_0": 0.8, "candidate_1": -0.1}},+ {"type": "choice", "probabilities": {"candidate_0": 0.8, "candidate_1": 1.1}},+ {+ "type": "choice",+ "probabilities": {"candidate_0": 0.8, "candidate_1": float("nan")},+ },+ ],+ )+ @patch("openviking.models.rerank.jev_rerank.httpx.Client")+ def test_invalid_choice_answer_returns_none(self, mock_client_class, answer):+ mock_client_class.return_value.post.return_value = _mock_response(+ {"answers": {"relevance": answer}}+ )+ client = JevRerankClient(api_key="key", mode=MODE_CHOICE)+ assert client.rerank_batch("query", ["first", "second"]) is None+ mock_client_class.return_value.post.assert_called_once()++ @patch("openviking.models.rerank.jev_rerank.httpx.Client")+ def test_unknown_mode_rejected_before_client_creation(self, mock_client_class):+ with pytest.raises(ValueError, match="Unknown Jev rerank mode"):+ JevRerankClient(api_key="key", mode="unknown")+ mock_client_class.assert_not_called()++class TestJevRerankConfig:def test_explicit_jev_provider(self):config = RerankConfig(provider="jev", api_key="key")assert config._effective_provider() == "jev"assert config.is_available() is True+ assert config.mode == MODE_NOUL++ @pytest.mark.parametrize("mode", [MODE_CHOICE, "CHOICE"])+ def test_explicit_choice_mode(self, mode):+ config = RerankConfig(provider="jev", api_key="key", mode=mode)+ assert config.mode == MODE_CHOICE++ def test_null_mode_uses_client_default(self):+ config = RerankConfig(provider="jev", api_key="key", mode=None)+ assert config.mode is None++ def test_unknown_mode_rejected(self):+ with pytest.raises(ValueError, match="Jev rerank mode"):+ RerankConfig(provider="jev", api_key="key", mode="unknown")++ def test_non_jev_provider_does_not_validate_jev_mode(self):+ config = RerankConfig(+ provider="openai",+ api_key="key",+ api_base="https://example.com/rerank",+ mode="listwise",+ )+ assert config.mode == "listwise"def test_jev_auto_detected_from_api_base(self):config = RerankConfig(api_key="key", api_base="https://api.typesafe.ai")@@ -191,6 +324,14 @@ class TestJevDispatch:assert isinstance(client, JevRerankClient)assert client.api_key == "jev-key"assert client.model_name == "jev-latest"+ assert client.mode == MODE_NOUL++ @pytest.mark.parametrize("mode", [MODE_CHOICE, None])+ @patch("openviking.models.rerank.jev_rerank.httpx.Client")+ def test_from_config_resolves_mode(self, mock_client_class, mode):+ config = RerankConfig(provider="jev", api_key="key", mode=mode)+ client = RerankClient.from_config(config)+ assert client.mode == (mode or MODE_NOUL)@patch("openviking.models.rerank.jev_rerank.httpx.Client")def test_from_config_uses_custom_model(self, mock_client_class):