Skip to content

🤖 Open-source, local-first alternative to Grok Bot, OpenAI dots and Claude Duet. Your own AI bots on any model: GPT, Claude, Gemini, Grok, DeepSeek, Kimi, GLM and 70+ more through one Modelflare key. Reach them in the app, WeChat or Feishu; they remember you, work in the background, run commands and use your computer. Pay per use.

feat(agent): 联网搜索不再依赖 DeepSeek key,默认免费后端 + 可选 Tavily/Exa/博查/智谱/SearXNG

Marla27ec38e89c68
diff --git a/docs/development.md b/docs/development.md
index 8d6033e..cf0a0fe 100644
--- a/docs/development.md
+++ b/docs/development.md
@@ -13,7 +13,7 @@
```
src/
core/ 统一消息类型、会话队列与去重、运行时(/model /reset、流式)、重启后的提醒、准入(主人 + 允许名单)、持久化出站队列
- agent/ 代理模式:DSH 宿主与桥接、审批、权限预设、记忆与上下文管家、生图 / 生视频、电脑操作、Windows PowerShell
+ agent/ 代理模式:DSH 宿主与桥接、审批、权限预设、记忆与上下文管家、生图 / 生视频、联网搜索、电脑操作、Windows PowerShell
prompts/ 交给代理模型的全部提示词:人设、聊天规则、记忆说明、每段第一条消息、技能说明
llm/ OpenAI 兼容客户端(模型接入接口)
magpie/ 内置 magpie 守护:隔离目录、空闲端口、自动生成密钥、健康检查、崩溃重启、干净退出
@@ -96,6 +96,7 @@ docker exec amb cat /data/ui-token # 打开 http://<主机>:3600/?token=<令
- 机器人预算(需要登录):机器人设置 → 预算,开启后这个机器人用自己的 Key(内置网关里单独的 provider),最多花设定的美元数;关闭后回到共用的 Key。
- 生图 / 生视频:代理模式下机器人有 `media_models`(可用模型、默认模型、参数范围、参考价)、`media_docs`(从 docs.modelflare.dev 实时读该模型的请求格式)、`generate_image`、`generate_video` 四个工具,并自带一份 media 技能说明。各模型的参数格式不同,机器人按文档写请求体,工作区里的参考图写成 `workspace:路径` 即可;Key 始终只在本程序里,由本程序带着该机器人用的 Key 调 Modelflare,结果存进机器人工作区的 `media/`,再交付到聊天。
- 生成前确认:直接生成,不打扰你。机器人选了不是默认的模型、要求没说清,都不问:它按最合理的理解生成,再说一句用了什么。只在预计花费超过 $0.5 时(按参考价、张数、视频时长和分辨率估算;视频估不出来也算)先在聊天里问你这次用哪个模型(附提示词、参数、参考价和为什么问)。可以确认、换成别的模型或取消;微信回复编号或模型名,飞书 / 网页点选。10 分钟内没回答就不生成。想每次都确认:机器人设置 → 生图 / 生视频 → 生成前确认 →「每次都问」。
+- 联网搜索:代理的 `web_search` 不用 DeepSeek 自带的搜索(它要 DeepSeek 的 Key),经 `amb-web-search` 插件交给本程序搜。默认免费、不用设置:依次试 Exa 的公开 MCP、必应、DuckDuckGo,上次成功的先试,出错的(限流、验证页、必应给出不相干的结果)跳过 10 分钟。设置 → 网络与访问 → 联网搜索 可以换成 Tavily、Exa、博查、智谱(填 Key)或自己的 SearXNG,填了就先用它,出错时退回免费的。Key 和渠道令牌一样存在本机数据库里,代理进程拿不到。桌面版搜索走系统代理;命令行版用 Node 的 fetch(设 `NODE_USE_ENV_PROXY=1` 后认 `HTTPS_PROXY`)。
- 默认图片 / 视频模型:首次设置时和默认文本模型一起选,之后在 设置 → Modelflare 修改;选「自动」时用推荐顺序里第一个不需要参考素材的模型。
- 模型选择器里显示上下文长度、视觉 / 工具 / 推理能力,以及每百万 tokens 输入 / 输出的起步参考价(来自公开价格目录,以账单为准)。
- 调用归因:经 Modelflare 的请求都带 `X-Modelflare-Skill: anymodel-bot/<版本>`,链接都带 `utm_source=anymodelbot&utm_campaign=amb_<位置>`(应用内 `amb_signup`、`amb_topup`、`amb_console` 等,README 与仓库文档 `amb_readme`,官网 `amb_homepage`;格式由 `test/utm.unit.test.ts` 固定)。
diff --git a/docs/development_en.md b/docs/development_en.md
index e3dcde6..577227f 100644
--- a/docs/development_en.md
+++ b/docs/development_en.md
@@ -13,7 +13,7 @@ This guide is for people who want to run AnyModelBot from source, package it, or
```
src/
core/ shared message types, session queue and dedup, runtime (/model /reset, streaming), the notice after a restart, admission (owner + allowlist), durable outbox
- agent/ agent mode: DSH host and bridge, approvals, permission presets, memory and context steward, image / video generation, computer use, Windows PowerShell
+ agent/ agent mode: DSH host and bridge, approvals, permission presets, memory and context steward, image / video generation, web search, computer use, Windows PowerShell
prompts/ every prompt the agent model reads: persona, chat rules, memory guide, a segment's first message, skills
llm/ OpenAI-compatible client (model access interface)
magpie/ magpie supervisor: isolated directories, free port, generated secret, health checks, crash restart, clean exit
@@ -96,6 +96,7 @@ Inside the container the console listens on 0.0.0.0, so token access is enforced
- Bot budgets (requires sign-in): Bot settings → Budget. When enabled, the bot uses its own key (a separate provider in the gateway) capped at the dollar amount you set; when disabled it goes back to the shared key.
- Image / video generation: in agent mode, bots get four tools — `media_models` (available models, defaults, parameter ranges, reference prices), `media_docs` (reads the model's request format live from docs.modelflare.dev), `generate_image` and `generate_video` — plus a bundled media skill. Parameter formats differ per model, so the bot writes the request body from the docs; reference images in the workspace are written as `workspace:path`. The key never leaves the app: the app calls Modelflare with the bot's key, saves results in the bot workspace's `media/`, and delivers them to the chat.
- Confirmation before generation: the image or video is generated right away. A model other than the default, or a loose request, is not a reason to ask: the bot takes the most reasonable reading, generates, and says in one line what it used. The chat asks which model to use (with prompt, parameters, reference price and the reason for asking) only when the estimated cost is above $0.5 (from the reference price, number of images, video length and resolution; a video whose cost cannot be estimated counts too). You can confirm, switch models or cancel — reply with a number or model name on WeChat, or click on Feishu / the web console. With no answer within 10 minutes, nothing is generated. To confirm every time: bot settings → Images / videos → Confirm before generating → "Every time".
+- Web search: the agent's `web_search` does not use DeepSeek's built-in search (it needs a DeepSeek key); the `amb-web-search` plugin hands each search to the app. By default it is free and needs no setup: it tries Exa's public MCP, Bing and DuckDuckGo in turn, the last one that worked first, and skips one that fails (rate limit, bot check, Bing returning unrelated results) for 10 minutes. Settings → Network & access → Web search can switch to Tavily, Exa, Bocha or Zhipu (with a key) or your own SearXNG; when set, it is tried first and the free search takes over if it fails. Keys are stored in the local database like channel tokens and never reach the agent process. The desktop app searches through the system proxy; the command-line build uses Node's fetch (it honours `HTTPS_PROXY` with `NODE_USE_ENV_PROXY=1`).
- Default image / video models: chosen together with the default text model during first-run setup, changeable later in Settings → Modelflare. "Auto" uses the first model in the recommended order that needs no reference material.
- The model picker shows context length, vision / tools / reasoning capabilities, and starting reference prices per million input / output tokens (from the public price catalog; your bill is authoritative).
- Attribution: requests through Modelflare carry `X-Modelflare-Skill: anymodel-bot/<version>`, and links carry `utm_source=anymodelbot&utm_campaign=amb_<placement>` (`amb_signup`, `amb_topup`, `amb_console` and so on inside the app, `amb_readme` for the README and repository docs, `amb_homepage` for the website; `test/utm.unit.test.ts` pins the format).
diff --git a/dsh-runtime/plugins/amb-web-search.mjs b/dsh-runtime/plugins/amb-web-search.mjs
new file mode 100644
index 0000000..865eec7
--- /dev/null
+++ b/dsh-runtime/plugins/amb-web-search.mjs
@@ -0,0 +1,58 @@
+// anymodel bot web search provider (MIT, part of anymodel bot).
+//
+// Registers the "amb" search provider in DSH's web seam (ctx.web). The patch written by the host selects it with
+// web.searchProvider, so DeepSeek's own provider (it needs a DeepSeek API key the app never has) is never picked.
+// web_search keeps its DSH tool, schema and result card; each query goes to the host over the loopback bridge
+// (AMB_BRIDGE_URL / AMB_BRIDGE_TOKEN), which searches with the app's network settings: free engines by default, or
+// the search API the user set up. The agent process holds no search key.
+import { WebError } from '@deepseek-ai/dsh-web'
+
+export const name = 'amb-web-search'
+export const inject = ['web']
+
+/** Shown to the model when the host could not search; nothing here points at settings the model cannot change. */
+const UNAVAILABLE = 'Web search is temporarily unavailable. If you know a relevant page, open it directly with web_fetch.'
+
+const text = (v) => (typeof v === 'string' && v.length > 0 ? v : undefined)
+
+export function apply(ctx) {
+ const url = process.env.AMB_BRIDGE_URL
+ const token = process.env.AMB_BRIDGE_TOKEN
+ const botId = process.env.AMB_BOT_ID ?? ''
+
+ const aborted = (signal, cause) => new WebError('web search aborted', 'WEB_ABORTED', { cause: signal?.reason ?? cause })
+
+ ctx.web.registerSearchProvider({
+ id: 'amb',
+ available: () => true,
+ async search(request, signal) {
+ if (signal?.aborted) throw aborted(signal)
+ if (!url || !token) throw new WebError(UNAVAILABLE, 'WEB_PROVIDER_UNAVAILABLE')
+ let reply
+ try {
+ const res = await fetch(url + '/dsh/search', {
+ method: 'POST',
+ headers: { authorization: `Bearer ${token}`, 'content-type': 'application/json' },
+ body: JSON.stringify({ botId, query: request.query, maxResults: request.maxResults }),
+ signal,
+ })
+ if (!res.ok) throw new Error(`bridge /dsh/search ${res.status}`)
+ reply = await res.json()
+ } catch (error) {
+ if (signal?.aborted) throw aborted(signal, error)
+ throw new WebError(UNAVAILABLE, 'WEB_PROVIDER_ERROR', { cause: error })
+ }
+ if (reply?.error === 'aborted') throw aborted(signal)
+ if (reply?.error) throw new WebError(UNAVAILABLE, 'WEB_PROVIDER_ERROR')
+ const sources = (Array.isArray(reply?.sources) ? reply.sources : [])
+ .filter((s) => s && typeof s.url === 'string' && s.url.length > 0)
+ .map((s) => ({
+ url: s.url,
+ ...(text(s.title) ? { title: s.title } : {}),
+ ...(text(s.snippet) ? { snippet: s.snippet } : {}),
+ ...(text(s.publishedAt) ? { publishedAt: s.publishedAt } : {}),
+ }))
+ return { sources, truncated: reply?.truncated === true }
+ },
+ })
+}
diff --git a/src/agent/bridge.ts b/src/agent/bridge.ts
index 1cb9d79..7e81888 100644
--- a/src/agent/bridge.ts
+++ b/src/agent/bridge.ts
@@ -22,6 +22,8 @@ export interface BridgeHandlers {
steward?(body: any): Promise<unknown>;
/** amb-media 插件:生图 / 生视频 */
media?(body: any): Promise<unknown>;
+ /** amb-web-search 插件:web_search 的一次搜索(signal = 连接断了:工具超时或回合被中止) */
+ search?(body: any, signal: AbortSignal): Promise<unknown>;
/** amb-control 插件:长轮询取宿主的指令(signal = 连接断了)、回报结果 */
control?: { poll(body: any, signal: AbortSignal): Promise<unknown>; result(body: any): void };
/** 服务端压缩:长会话的 /v1/responses 换上上游压缩过的历史(见 server-compaction.ts) */
@@ -52,6 +54,7 @@ export const BOT_HEADER = "x-amb-bot";
* - /v1/*:模型代理(Chat Completions、Responses、Anthropic Messages),把令牌换成真实网关密钥(子进程环境里没有任何真实密钥)
* - /dsh/ask|wait|cancel:amb-bridge 插件的审批/提问长轮询
* - /dsh/control/poll|result:amb-control 插件取宿主的指令(中止、插话、换模型)并回报
+ * - /dsh/search:amb-web-search 插件(web_search 由宿主联网搜索)
* - /dsh/task-report|task-settled|task-tool|task-config:amb-tasks 插件(后台任务的报告、收尾结束、聊天代理的任务工具、规则会话的配置)
*/
export class AgentBridge {
@@ -270,6 +273,11 @@ export class AgentBridge {
else if (url === "/dsh/memory" && this.h.memory) out = await this.h.memory(body);
else if (url === "/dsh/steward" && this.h.steward) out = await this.h.steward(body);
else if (url === "/dsh/media" && this.h.media) return this.longCall(res, this.h.media(body));
+ else if (url === "/dsh/search" && this.h.search) {
+ const ac = new AbortController();
+ res.on("close", () => ac.abort());
+ out = await this.h.search(body, ac.signal);
+ }
else if (url === "/dsh/control/poll" && this.h.control) {
const ac = new AbortController();
res.on("close", () => ac.abort());
diff --git a/src/agent/host.ts b/src/agent/host.ts
index b4d65d5..ef8245c 100644
--- a/src/agent/host.ts
+++ b/src/agent/host.ts
@@ -726,6 +726,12 @@ export class AgentHost {
...providerPatchLines(providerProfiles(list, this.o.bridge.url, this.o.botId)),
// 长命令转后台;Windows:命令用检测到的 PowerShell 7(DSH 自己只看 Program Files\PowerShell\7 和 PATH,找不到就退回 5.1)
...shellPatchLines(pwshPath),
+ // web_search 用 amb-web-search 插件注册的 amb(经 bridge 由宿主搜索),不用 DeepSeek 自带的搜索(要 DeepSeek 的 Key);
+ // 补丁整体替换这一行的 config,网页抓取照旧写上 http
+ "- id: web",
+ " config:",
+ " searchProvider: amb",
+ " fetchProvider: http",
// 人设:个人助手(替换 DSH 的 coding agent 人设),不要 "powered by DeepSeek Harness" 那句身份
"- id: system-prompt",
" config:",
@@ -746,6 +752,9 @@ export class AgentHost {
// 生图 / 生视频:宿主带着本机器人的 Key 调 Modelflare,文件存进工作区 media/;技能说明在 skills/media
" - id: amb-media",
` name: ${yamlStr(path.join(this.o.runtimeDir, "plugins", "amb-media.mjs"))}`,
+ // 联网搜索:默认免费(Exa MCP、必应、DuckDuckGo),设置里可换成带 Key 的接口(search.ts)
+ " - id: amb-web-search",
+ ` name: ${yamlStr(path.join(this.o.runtimeDir, "plugins", "amb-web-search.mjs"))}`,
// sdk 配置默认不带这两个工具:ask_user_question 走 user-questions → 插件 → 审批中心;
// present 只声明交付文件,由宿主在回合结束后经渠道发送。
" - id: amb-tool-ask-user",
diff --git a/src/agent/manager.ts b/src/agent/manager.ts
index 90d03a8..caa161b 100644
--- a/src/agent/manager.ts
+++ b/src/agent/manager.ts
@@ -9,6 +9,7 @@ import { ServerCompaction, type CompactEndpoint, type CompactionEvent } from "./
import { ActivityLog, promotedJob, type ActivityStep } from "./activity.js";
import { ApiFallbacks, type ApiFallback } from "./api-fallback.js";
import type { MediaModel, MediaPlan, MediaService } from "./media.js";
+import { SearchAborted, type SearchService, type WebSource } from "./search.js";
import { APPROVAL_TTL_MS, ApprovalCenter, TASK_APPROVAL_TTL_MS, type QuestionItem } from "./approvals.js";
import { MemoryError, MemoryStore, type Scope } from "./memory.js";
import type { TaskHost, TaskService } from "../tasks/service.js";
@@ -49,6 +50,8 @@ export interface AgentManagerOptions {
reviewDelayMs?: number;
/** 生图 / 生视频(amb-media 插件经 bridge 调用);缺省时工具返回不可用 */
media?: MediaService;
+ /** 联网搜索(amb-web-search 插件经 bridge 调用,DSH 的 web_search 用它);缺省时搜索返回不可用 */
+ search?: SearchService;
/** 需要让用户看到的提示(代理进程起不来等) */
onNotice?: (text: string) => void;
/** 模型的上下文长度(Modelflare 模型资料),写进 DSH 补丁 */
@@ -187,6 +190,7 @@ export class AgentManager {
memory: (b) => this.memoryOp(b),
steward: (b) => this.stewardCall(b),
media: (b) => this.mediaOp(b),
+ search: (b, signal) => this.searchOp(b, signal),
// amb-control 插件按机器人取指令;不认识的机器人(已删除、已停止)让它别再来
control: {
poll: (b, signal) => this.hosts.get(String(b?.botId ?? ""))?.control.poll(b, signal) ?? Promise.resolve({ stop: true }),
@@ -642,6 +646,23 @@ export class AgentManager {
return res;
}
+ /**
+ * 联网搜索:聊天和后台任务都能用。出错只回一个 error(插件给代理一句"搜索暂时不可用,可以直接用 web_fetch 打开网页"),
+ * 哪个后端、为什么出错记在日志里(search.ts)。
+ */
+ private async searchOp(b: any, signal: AbortSignal): Promise<{ sources?: WebSource[]; truncated?: boolean; error?: string }> {
+ const search = this.o.search;
+ if (!search || !this.hosts.has(String(b?.botId ?? ""))) return { error: "unavailable" };
+ const query = String(b?.query ?? "").trim();
+ if (!query) return { sources: [], truncated: false };
+ try {
+ const r = await search.search(query, Number(b?.maxResults) || 8, signal);
+ return { sources: r.sources, truncated: false };
+ } catch (e) {
+ return { error: e instanceof SearchAborted ? "aborted" : "unavailable" };
+ }
+ }
+
/**
* 生图 / 生视频:只在该机器人有回合进行中时受理(工具调用只会发生在回合里)。
* models / docs 直接回答;image / video 直接生成,预计花费超过上限或设成每次都问时(mediaAsk)先问用户用哪个模型。
diff --git a/src/agent/search.ts b/src/agent/search.ts
new file mode 100644
index 0000000..c341bb6
--- /dev/null
+++ b/src/agent/search.ts
@@ -0,0 +1,751 @@
+/**
+ * 联网搜索:代理的 web_search 工具(DSH 的 ctx.web 选中 amb-web-search 插件注册的 "amb" 提供方)经内部 bridge 调到这里,
+ * 由宿主搜索。代理子进程里没有任何搜索 Key,也不用 DeepSeek 自带的搜索(它要 DeepSeek 的 Key)。
+ *
+ * - 网络:桌面版用 Chromium 的网络栈(Electron net.fetch:系统代理、PAC、系统证书),和下载 PowerShell 一样;
+ * 命令行 / Docker 用 Node 的 fetch(Node 22.21+ 设 NODE_USE_ENV_PROXY=1 后认 HTTPS_PROXY)。
+ * - 默认(自动)不用 Key:Exa 的公开 MCP → 必应网页 → DuckDuckGo 网页,上一次成功的排最前。某个出错(网络、超时、
+ * 限流、人机验证页、必应给出与问题无关的结果)就冷却一段时间不用;全都在冷却时照样都试。某个说"没有结果"不算出错,
+ * 接着问下一个。
+ * - 设置里选了带 Key 的接口(Tavily / Exa / 博查 / 智谱)或填了 SearXNG 地址时先用它;它出错就退回免费的
+ * (Key 被拒记一条警告,设置页也会提示),同样冷却一段时间。
+ * - 结果按 URL 去重、截到 maxResults 条;日志只记用了哪个后端、几条、耗时,不记查询内容和 Key。
+ */
+import { logger, registerSecret, mask } from "../log.js";
+import { appLang, type Lang } from "../i18n/index.js";
+
+const log = logger("agent.search");
+
+// ---------- 设置 ----------
+
+export const SEARCH_PROVIDERS = ["auto", "tavily", "exa", "bocha", "zhipu", "searxng"] as const;
+export type SearchProvider = (typeof SEARCH_PROVIDERS)[number];
+/** 带 Key 的接口 */
+export type KeyedProvider = Exclude<SearchProvider, "auto" | "searxng">;
+const KEYED: readonly KeyedProvider[] = ["tavily", "exa", "bocha", "zhipu"];
+
+/** 应用级(不分机器人):存在 SQLite 的 kv 表,和渠道令牌一样只靠数据目录的权限保护;Key 按接口分开存,切换不丢 */
+export interface WebSearchSettings {
+ provider: SearchProvider;
+ keys: Partial<Record<KeyedProvider, string>>;
+ /** SearXNG 实例地址 */
+ url: string;
+}
+export const WEB_SEARCH_KV = "web-search";
+
+export function normalizeWebSearch(v: unknown): WebSearchSettings {
+ const o = (v && typeof v === "object" ? v : {}) as Record<string, any>;
+ const provider = SEARCH_PROVIDERS.includes(o.provider) ? (o.provider as SearchProvider) : "auto";
+ const keys: WebSearchSettings["keys"] = {};
+ for (const p of KEYED) {
+ const k = typeof o.keys?.[p] === "string" ? o.keys[p].trim() : "";
+ if (k) keys[p] = k;
+ }
+ const url = typeof o.url === "string" ? o.url.trim() : "";
+ return { provider, keys, url };
+}
+
+/** SearXNG 地址:http(s),去掉末尾的 /search 和斜杠;不合法返回 undefined */
+export function normalizeSearxUrl(v: string): string | undefined {
+ const s = v.trim();
+ if (!s) return "";
+ try {
+ const u = new URL(s);
+ if (u.protocol !== "http:" && u.protocol !== "https:") return undefined;
+ u.search = "";
+ u.hash = "";
+ return u.href.replace(/\/+$/, "").replace(/\/search$/, "");
+ } catch {
+ return undefined;
+ }
+}
+
+interface KvStore {
+ getKV(key: string): string | undefined;
+ setKV(key: string, value: string): void;
+}
+export function loadWebSearch(store: KvStore): WebSearchSettings {
+ let raw: unknown;
+ try {
+ raw = JSON.parse(store.getKV(WEB_SEARCH_KV) ?? "null");
+ } catch {
+ raw = undefined;
+ }
+ const s = normalizeWebSearch(raw);
+ for (const k of Object.values(s.keys)) registerSecret(k);
+ return s;
+}
+export function saveWebSearch(store: KvStore, s: WebSearchSettings): void {
+ for (const k of Object.values(s.keys)) registerSecret(k);
+ store.setKV(WEB_SEARCH_KV, JSON.stringify(s));
+}
+
+/** 设置页看到的:接口、当前接口 Key 的末 4 位、SearXNG 地址;Key 最近被拒时 keyRejected */
+export interface WebSearchView {
+ provider: SearchProvider;
+ keyTail: string;
+ url: string;
+ keyRejected: boolean;
+}
+
+// ---------- 结果与错误 ----------
+
+export interface WebSource {
+ url: string;
+ title?: string;
+ snippet?: string;
+ publishedAt?: string;
+}
+
+/** network 网络不通;timeout 超时;auth Key 被拒;rate 限流;blocked 人机验证 / 拦截;unrelated 结果与问题无关;format 看不懂的回复 */
+export type FailKind = "network" | "timeout" | "http" | "auth" | "rate" | "blocked" | "unrelated" | "format";
+
+export class SearchFailure extends Error {
+ constructor(
+ readonly backend: string,
+ readonly kind: FailKind,
+ message: string,
+ readonly status?: number,
+ ) {
+ super(message);
+ }
+}
+/** 调用方取消(工具超时、用户中止):不算后端出错 */
+export class SearchAborted extends Error {}
+
+export interface BackendContext {
+ fetch: typeof fetch;
+ signal: AbortSignal;
+ lang: Lang;
+}
+/** 一个搜索后端:返回结果(空数组 = 确实没有结果);出错抛 SearchFailure */
+export interface SearchBackend {
+ id: string;
+ search(query: string, count: number, ctx: BackendContext): Promise<WebSource[]>;
+}
+
+// ---------- 文本工具(不依赖 HTML 解析库:搜索页只取标题、链接、摘要几样) ----------
+
+const ENTITIES: Record<string, string> = {
+ amp: "&", lt: "<", gt: ">", quot: '"', apos: "'", nbsp: " ", ensp: " ", emsp: " ", thinsp: " ",
+ middot: "·", hellip: "…", ndash: "–", mdash: "—", lsquo: "‘", rsquo: "’", ldquo: "“", rdquo: "”",
+ laquo: "«", raquo: "»", bull: "•", copy: "©", reg: "®", trade: "™", times: "×", deg: "°",
+};
+
+export function decodeEntities(s: string): string {
+ return s.replace(/&(#x[0-9a-f]+|#\d+|[a-z]+);/gi, (m, e: string) => {
+ if (e[0] === "#") {
+ const n = e[1] === "x" || e[1] === "X" ? parseInt(e.slice(2), 16) : parseInt(e.slice(1), 10);
+ return Number.isFinite(n) && n > 0 && n <= 0x10ffff ? String.fromCodePoint(n) : m;
+ }
+ return ENTITIES[e.toLowerCase()] ?? m;
+ });
+}
+
+/** HTML 片段 → 一行纯文本(行内标签如命中词的 <strong> 直接去掉,其他标签换成空格) */
+export function htmlText(s: string): string {
+ return decodeEntities(
+ s
+ .replace(/<(script|style)\b[\s\S]*?<\/\1>/gi, " ")
+ .replace(/<\/?(b|strong|em|i|u|mark|span|a|abbr|code|sup|sub|small|font|wbr)\b[^>]*>/gi, "")
+ .replace(/<[^>]*>/g, " "),
+ )
+ .replace(/\s+/g, " ")
+ .trim();
+}
+
+/** 开始标签里的一个属性值(已解码实体) */
+export function attr(tag: string, name: string): string | undefined {
+ const m = tag.match(new RegExp(`\\s${name}\\s*=\\s*(?:"([^"]*)"|'([^']*)'|([^\\s>]+))`, "i"));
+ return m ? decodeEntities(m[1] ?? m[2] ?? m[3] ?? "") : undefined;
+}
+
+const SNIPPET_MAX = 300;
+/** 摘要:去掉 Markdown 标题符号和分隔线,压成一行,截到 SNIPPET_MAX */
+export function tidy(s: string, max = SNIPPET_MAX): string {
+ const t = s
+ .replace(/^\s*#{1,6}\s+/gm, "")
+ .replace(/^\s*(-{3,}|\*{3,}|\.{3})\s*$/gm, " ")
+ .replace(/\s+/g, " ")
+ .trim();
+ return t.length > max ? t.slice(0, max - 1).trimEnd() + "…" : t;
+}
+
+function source(url: string, title?: string, snippet?: string, publishedAt?: string): WebSource {
+ const t = title ? tidy(title, 200) : "";
+ const s = snippet ? tidy(snippet) : "";
+ const d = (publishedAt ?? "").trim().replace(/T00:00:00(\.0+)?Z$/, "");
+ return { url, ...(t ? { title: t } : {}), ...(s ? { snippet: s } : {}), ...(d && !/^n\/?a$/i.test(d) ? { publishedAt: d } : {}) };
+}
+
+const httpUrl = (u: string | undefined): u is string => !!u && /^https?:\/\/[^\s/]+/i.test(u);
+
+// ---------- HTTP ----------
+
+/** 普通桌面浏览器的请求头(必应、DuckDuckGo 的网页版对脚本样的请求给验证页或无关结果) */
+export const BROWSER_UA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36";
+function browserHeaders(lang: Lang): Record<string, string> {
+ return {
+ "user-agent": BROWSER_UA,
+ accept: "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
+ "accept-language": lang === "zh-CN" ? "zh-CN,zh;q=0.9,en;q=0.8" : "en-US,en;q=0.9",
+ };
+}
+
+async function send(backend: string, ctx: BackendContext, url: string, init: RequestInit = {}): Promise<Response> {
+ try {
+ return await ctx.fetch(url, { ...init, signal: ctx.signal });
+ } catch (e) {
+ if (ctx.signal.aborted) throw ctx.signal.reason ?? e;
+ throw new SearchFailure(backend, "network", errText(e));
+ }
+}
+
+async function read(backend: string, ctx: BackendContext, res: Response): Promise<string> {
+ try {
+ return await res.text();
+ } catch (e) {
+ if (ctx.signal.aborted) throw ctx.signal.reason ?? e;
+ throw new SearchFailure(backend, "network", errText(e));
+ }
+}
+
+/** 非 2xx:401/403 对带 Key 的接口是 Key 被拒,对网页是被拦截;429 限流 */
+async function httpFailure(backend: string, res: Response, keyed: boolean): Promise<SearchFailure> {
+ const body = (await res.text().catch(() => "")).replace(/\s+/g, " ").slice(0, 200);
+ const s = res.status;
+ const kind: FailKind = s === 401 || s === 403 ? (keyed ? "auth" : "blocked") : s === 429 ? "rate" : "http";
+ return new SearchFailure(backend, kind, `HTTP ${s}${body ? `: ${body}` : ""}`, s);
+}
+
+function errText(e: unknown): string {
+ const err = e as { message?: string; cause?: { code?: string; message?: string } };
+ const cause = err?.cause?.code ?? err?.cause?.message;
+ return `${err?.message ?? String(e)}${cause ? ` (${cause})` : ""}`.slice(0, 200);
+}
+
+// ---------- Exa 公开 MCP(免 Key,有频率限制) ----------
+
+export const EXA_MCP_URL = "https://mcp.exa.ai/mcp";
+
+/** MCP 的 HTTP 回复:JSON,或 SSE(data: 行)里的一条 JSON-RPC 消息 */
+export function parseRpcReply(body: string): any {
+ const t = body.trim();
+ if (t.startsWith("{")) return JSON.parse(t);
+ let found: any;
+ for (const event of t.split(/\r?\n\r?\n/)) {
+ const data = event
+ .split(/\r?\n/)
+ .filter((l) => l.startsWith("data:"))
+ .map((l) => l.slice(5).replace(/^ /, ""))
+ .join("\n");
+ if (!data) continue;
+ try {
+ const msg = JSON.parse(data);
+ if (msg && typeof msg === "object" && ("result" in msg || "error" in msg)) {
+ if (msg.id === 1) return msg;
+ found ??= msg;
+ }
+ } catch {
+ /* 不是 JSON 的事件跳过 */
+ }
+ }
+ if (found) return found;
+ throw new Error("no JSON-RPC message in the reply");
+}
+
+/**
+ * web_search_exa 的文字结果:每条是 "Title: …\nURL: …\nPublished: …\nAuthor: …\nHighlights:\n…",条与条之间 "---"。
+ * 正文里也可能有 "---",所以按"Title: 行后面紧跟 URL: 行"找每条的开头。也认旧格式(Published Date: / Text:)和 JSON。
+ */
+export function parseExaText(text: string): WebSource[] {
+ const t = text.trim();
+ if (t.startsWith("{") || t.startsWith("[")) {
+ try {
+ const j = JSON.parse(t);
+ const list: any[] = Array.isArray(j) ? j : Array.isArray(j?.results) ? j.results : [];
+ return list
+ .filter((r) => httpUrl(r?.url))
+ .map((r) => source(r.url, str(r.title), str(r.text) || (Array.isArray(r.highlights) ? r.highlights.join(" ") : "") || str(r.summary), str(r.publishedDate)));
+ } catch {
+ /* 不是 JSON:按文字解析 */
+ }
+ }
+ const lines = t.split(/\r?\n/);
+ const starts: number[] = [];
+ for (let i = 0; i < lines.length; i++) {
+ if (/^Title:/.test(lines[i]) && lines.slice(i + 1, i + 4).some((l) => /^URL:\s*\S/.test(l))) starts.push(i);
+ }
+ const out: WebSource[] = [];
+ starts.forEach((s, n) => {
+ const rec = lines.slice(s, starts[n + 1] ?? lines.length);
+ const title = rec[0].replace(/^Title:\s*/, "");
+ let url = "";
+ let published = "";
+ let body: string[] = [];
+ for (let i = 1; i < rec.length; i++) {
+ const l = rec[i];
+ const m = l.match(/^(URL|Published(?: Date)?|Author|Highlights|Text|Summary|Content):\s*(.*)$/);
+ if (!m) continue;
+ if (m[1] === "URL") url ||= m[2].trim();
+ else if (m[1].startsWith("Published")) published ||= m[2].trim();
+ else if (m[1] !== "Author") {
+ body = [m[2], ...rec.slice(i + 1)];
+ break;
+ }
+ }
+ while (body.length && /^\s*(-{3,})?\s*$/.test(body[body.length - 1])) body.pop();
+ if (httpUrl(url)) out.push(source(url, title, body.join("\n"), published));
+ });
+ return out;
+}
+
+const str = (v: unknown): string => (typeof v === "string" ? v : "");
+
+/** tools/call 的回复 → 结果;限流、出错抛 SearchFailure;"没有结果"返回空 */
+export function exaMcpResults(msg: any): WebSource[] {
+ if (msg?.error) throw new SearchFailure("exa-mcp", "format", `JSON-RPC error ${msg.error.code ?? ""}: ${String(msg.error.message ?? "").slice(0, 200)}`);
+ const result = msg?.result;
+ const text = (Array.isArray(result?.content) ? result.content : [])
+ .filter((c: any) => c?.type === "text" && typeof c.text === "string")
+ .map((c: any) => c.text)
+ .join("\n");
+ if (result?._meta?.["ai.exa/rateLimited"] === true || (/rate limit/i.test(text) && !/^URL:/m.test(text))) {
+ throw new SearchFailure("exa-mcp", "rate", "free MCP rate limit");
+ }
+ if (result?.isError) throw new SearchFailure("exa-mcp", "format", text.slice(0, 200) || "tool error");
+ const sources = parseExaText(text);
+ if (sources.length) return sources;
+ if (/no (search )?results found/i.test(text) || !text.trim()) return [];
+ throw new SearchFailure("exa-mcp", "format", `unrecognized reply: ${text.slice(0, 120)}`);
+}
+
+export const exaMcp: SearchBackend = {
+ id: "exa-mcp",
+ async search(query, count, ctx) {
+ const body = {
+ jsonrpc: "2.0",
+ id: 1,
+ method: "tools/call",
+ params: { name: "web_search_exa", arguments: { query, objective: query.slice(0, 4000), numResults: count } },
+ };
+ const res = await send("exa-mcp", ctx, EXA_MCP_URL, {
+ method: "POST",
+ headers: { "content-type": "application/json", accept: "application/json, text/event-stream" },
+ body: JSON.stringify(body),
+ });
+ if (!res.ok) throw await httpFailure("exa-mcp", res, false);
+ const text = await read("exa-mcp", ctx, res);
+ let msg: unknown;
+ try {
+ msg = parseRpcReply(text);
+ } catch (e) {
+ throw new SearchFailure("exa-mcp", "format", errText(e));
+ }
+ return exaMcpResults(msg);
+ },
+};
+
+// ---------- 必应网页 ----------
+
+/** 必应的跳转链接 bing.com/ck/a?…&u=a1<base64url> → 真实地址;不是跳转链接原样返回 */
+export function bingTarget(href: string): string | undefined {
+ let u: URL;
+ try {
+ u = new URL(href, "https://www.bing.com/");
+ } catch {
+ return undefined;
+ }
+ if (/(^|\.)bing\.com$/i.test(u.hostname) && u.pathname === "/ck/a") {
+ const p = u.searchParams.get("u") ?? "";
+ if (!p.startsWith("a1")) return undefined;
+ try {
+ const real = Buffer.from(p.slice(2).replace(/-/g, "+").replace(/_/g, "/"), "base64").toString("utf8");
+ return httpUrl(real) ? real : undefined;
+ } catch {
+ return undefined;
+ }
+ }
+ return httpUrl(u.href) ? u.href : undefined;
+}
+
+/** 必应结果页:自然结果(li.b_algo:h2 > a 是标题和链接,.b_caption p 是摘要,span.news_dt 是日期);noResults = 明确说没有结果 */
+export function parseBing(html: string): { results: WebSource[]; noResults: boolean; page: boolean } {
+ const results: WebSource[] = [];
+ const starts = [...html.matchAll(/<li\s+class="b_algo[\s"]/g)].map((m) => m.index!);
+ starts.forEach((s, i) => {
+ let block = html.slice(s, starts[i + 1] ?? html.length);
+ const end = block.indexOf("</ol>");
+ if (end > 0) block = block.slice(0, end);
+ const h2 = block.match(/<h2\b[^>]*>([\s\S]*?)<\/h2>/i);
+ const a = h2?.[1].match(/<a\b([^>]*)>([\s\S]*?)<\/a>/i);
+ if (!h2 || !a) return;
+ const url = bingTarget(attr(a[0], "href") ?? "");
+ if (!url) return;
+ const rest = block.slice(h2.index! + h2[0].length);
+ const p = rest.match(/<p\b[^>]*>([\s\S]*?)<\/p>/i);
+ let snippet = p?.[1] ?? "";
+ const date = snippet.match(/<span class="news_dt">([\s\S]*?)<\/span>/i);
+ if (date) snippet = snippet.replace(date[0], "");
+ snippet = htmlText(snippet).replace(/^[\s·•|-]+/, "");
+ results.push(source(url, htmlText(a[2]), snippet, date ? htmlText(date[1]) : undefined));
+ });
+ return { results, noResults: /class="b_no"/.test(html), page: /id="b_results"/.test(html) };
+}
+
+const STOPWORDS = new Set(
+ "the and for with what how why who when where which are was were is of to in on an by from about into vs not you your can does this that best new free top".split(" "),
+);
+/** 用来判断结果是否切题的词:拉丁等字母文字取 ≥3 个字符的词(去掉常见虚词),中日韩文字取相邻两字 */
+export function queryTerms(q: string): string[] {
+ const lower = q.toLowerCase();
+ const out = new Set<string>();
+ for (const w of lower.match(/[\p{L}\p{N}][\p{L}\p{N}.+#'_-]*/gu) ?? []) {
+ const word = w.replace(/[.'_-]+$/, "");
+ if (/[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}\p{Script=Hangul}]/u.test(word)) continue;
+ if (word.length >= 3 && /\p{L}/u.test(word) && !STOPWORDS.has(word)) out.add(word);
+ }
+ for (const run of lower.match(/[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}\p{Script=Hangul}]+/gu) ?? []) {
+ if (run.length === 1) out.add(run);
+ for (let i = 0; i + 1 < run.length; i++) out.add(run.slice(i, i + 2));
+ }
+ return [...out];
+}
+
+/**
+ * 结果是否切题:至少三分之一的结果(标题、摘要、网址)含有查询里的词。必应对它认为是脚本的请求(常见于机房 IP)
+ * 返回 200 和一页别的热门搜索的结果,只能这样识别出来,交给下一个后端。
+ */
+export function relevant(query: string, results: WebSource[]): boolean {
+ const terms = queryTerms(query);
+ if (!terms.length || !results.length) return true;
+ const hit = results.filter((r) => {
+ const hay = `${r.title ?? ""} ${r.snippet ?? ""} ${decodeURIComponentSafe(r.url)}`.toLowerCase();
+ return terms.some((t) => hay.includes(t));
+ }).length;
+ return hit >= 1 && hit * 3 >= results.length;
+}
+
+function decodeURIComponentSafe(s: string): string {
+ try {
+ return decodeURIComponent(s);
+ } catch {
+ return s;
+ }
+}
+
+export const bing: SearchBackend = {
+ id: "bing",
+ async search(query, count, ctx) {
+ // 中文界面先试 cn.bing.com(中国大陆直接可用);在境外它会跳回 www.bing.com 首页,那就再试 www.bing.com
+ const hosts = ctx.lang === "zh-CN" ? ["cn.bing.com", "www.bing.com"] : ["www.bing.com"];
+ let last: SearchFailure | undefined;
+ for (const host of hosts) {
+ const res = await send("bing", ctx, `https://${host}/search?q=${encodeURIComponent(query)}`, { headers: browserHeaders(ctx.lang) });
+ if (!res.ok) {
+ last = await httpFailure("bing", res, false);
+ continue;
+ }
+ const page = parseBing(await read("bing", ctx, res));
+ if (page.results.length) {
+ if (!relevant(query, page.results)) throw new SearchFailure("bing", "unrelated", "results do not match the query");
+ return page.results.slice(0, count);
+ }
+ if (page.noResults) return [];
+ last = new SearchFailure("bing", "blocked", page.page ? "results page without results" : "not a results page");
+ }
+ throw last ?? new SearchFailure("bing", "blocked", "not a results page");
+ },
+};
+
+// ---------- DuckDuckGo 网页(html.duckduckgo.com) ----------
+
+/** DuckDuckGo 的跳转链接 //duckduckgo.com/l/?uddg=<编码后的地址> → 真实地址;广告(y.js)不要 */
+export function ddgTarget(href: string): string | undefined {
+ let u: URL;
+ try {
+ u = new URL(href.startsWith("//") ? `https:${href}` : href, "https://html.duckduckgo.com/");
+ } catch {
+ return undefined;
+ }
+ if (/(^|\.)duckduckgo\.com$/i.test(u.hostname)) {
+ if (u.pathname === "/l/" || u.pathname === "/l") {
+ const real = u.searchParams.get("uddg") ?? "";
+ if (!httpUrl(real)) return undefined;
+ try {
+ if (/(^|\.)duckduckgo\.com$/i.test(new URL(real).hostname)) return undefined;
+ } catch {
+ return undefined;
+ }
+ return real;
+ }
+ return undefined;
+ }
+ return httpUrl(u.href) ? u.href : undefined;
+}
+
+/** DuckDuckGo 结果页:.result(去掉 .result--ad),a.result__a 是标题和链接,.result__snippet 是摘要;challenge = 人机验证页 */
+export function parseDdg(html: string): { results: WebSource[]; noResults: boolean; challenge: boolean } {
+ const results: WebSource[] = [];
+ const heads = [...html.matchAll(/<div\s+class="(result\b[^"]*)"/g)].filter((m) => /(^|\s)result(\s|$)/.test(m[1]));
+ heads.forEach((m, i) => {
+ if (/\bresult--ad\b/.test(m[1])) return;
+ const block = html.slice(m.index!, heads[i + 1]?.index ?? html.length);
+ const a = block.match(/<a\b([^>]*\bclass="[^"]*\bresult__a\b[^"]*"[^>]*)>([\s\S]*?)<\/a>/i);
+ if (!a) return;
+ const url = ddgTarget(attr(a[0], "href") ?? "");
+ if (!url) return;
+ const sn = block.match(/<(a|div|td|span)\b[^>]*\bclass="[^"]*\bresult__snippet\b[^"]*"[^>]*>([\s\S]*?)<\/\1>/i);
+ results.push(source(url, htmlText(a[2]), sn ? htmlText(sn[2]) : ""));
+ });
+ return {
+ results,
+ noResults: /class="no-results"/.test(html),
+ challenge: /anomaly-modal|anomaly\.js|challenge-form/.test(html),
+ };
+}
+
+export const duckduckgo: SearchBackend = {
+ id: "duckduckgo",
+ async search(query, count, ctx) {
+ const res = await send("duckduckgo", ctx, `https://html.duckduckgo.com/html/?q=${encodeURIComponent(query)}`, { headers: browserHeaders(ctx.lang) });
+ // 人机验证页回 202
+ if (res.status === 202) throw new SearchFailure("duckduckgo", "blocked", "bot challenge", 202);
+ if (!res.ok) throw await httpFailure("duckduckgo", res, false);
+ const page = parseDdg(await read("duckduckgo", ctx, res));
+ if (page.results.length) return page.results.slice(0, count);
+ if (page.challenge) throw new SearchFailure("duckduckgo", "blocked", "bot challenge");
+ if (page.noResults) return [];
+ // 200 却一条也解析不出来、也没说没有结果:多半是验证页或改版,当出错
+ throw new SearchFailure("duckduckgo", "blocked", "no results parsed");
+ },
+};
+
+export const FREE_BACKENDS: readonly SearchBackend[] = [exaMcp, bing, duckduckgo];
+
+// ---------- 带 Key 的接口 ----------
+
+/** 带 Key 的接口(或 SearXNG)的一次请求:地址、请求参数,和把回复变成结果的函数 */
+export interface KeyedRequest {
+ url: string;
+ init: RequestInit;
+ parse(j: any): WebSource[];
+}
+
+/** 按设置构造请求;没填 Key / 地址时返回 undefined(直接用免费的) */
+export function keyedRequest(s: WebSearchSettings, query: string, count: number): KeyedRequest | undefined {
+ const json = (headers: Record<string, string>, body: unknown): RequestInit => ({
+ method: "POST",
+ headers: { "content-type": "application/json", accept: "application/json", ...headers },
+ body: JSON.stringify(body),
+ });
+ const list = (v: unknown): any[] => (Array.isArray(v) ? v : []);
+ if (s.provider === "searxng") {
+ if (!s.url) return undefined;
+ return {
+ url: `${s.url}/search?q=${encodeURIComponent(query)}&format=json`,
+ init: { headers: { accept: "application/json" } },
+ parse: (j) => list(j?.results).filter((r) => httpUrl(r?.url)).map((r) => source(r.url, str(r.title), str(r.content), str(r.publishedDate))),
+ };
+ }
+ if (s.provider === "auto") return undefined;
+ const key = s.keys[s.provider];
+ if (!key) return undefined;
+ switch (s.provider) {
+ case "tavily":
+ return {
+ url: "https://api.tavily.com/search",
+ init: json({ authorization: `Bearer ${key}` }, { query, max_results: Math.min(count, 20) }),
+ parse: (j) => list(j?.results).filter((r) => httpUrl(r?.url)).map((r) => source(r.url, str(r.title), str(r.content), str(r.published_date))),
+ };
+ case "exa":
+ return {
+ url: "https://api.exa.ai/search",
+ init: json({ "x-api-key": key }, { query, numResults: Math.min(count, 25), type: "auto", contents: { highlights: { maxCharacters: 300 } } }),
+ parse: (j) =>
+ list(j?.results)
+ .filter((r) => httpUrl(r?.url))
+ .map((r) => source(r.url, str(r.title), list(r.highlights).filter((h) => typeof h === "string").join(" ") || str(r.summary) || str(r.text), str(r.publishedDate))),
+ };
+ case "bocha":
+ return {
+ url: "https://api.bochaai.com/v1/web-search",
+ init: json({ authorization: `Bearer ${key}` }, { query, count: Math.min(count, 50), summary: true }),
+ parse: (j) => {
+ if (j?.code !== undefined && Number(j.code) !== 200) throw new SearchFailure("bocha", "http", `code ${j.code}: ${str(j.msg).slice(0, 200)}`);
+ return list(j?.data?.webPages?.value)
+ .filter((r) => httpUrl(r?.url))
+ .map((r) => source(r.url, str(r.name), str(r.summary) || str(r.snippet), str(r.datePublished)));
+ },
+ };
+ case "zhipu":
+ return {
+ url: "https://open.bigmodel.cn/api/paas/v4/web_search",
+ init: json({ authorization: `Bearer ${key}` }, { search_query: query.slice(0, 70), search_engine: "search_std", search_intent: false, count: Math.min(count, 50) }),
+ parse: (j) => list(j?.search_result).filter((r) => httpUrl(r?.link)).map((r) => source(r.link, str(r.title), str(r.content), str(r.publish_date))),
+ };
+ }
+}
+
+/** 设置对应的后端(没填 Key / 地址时 undefined) */
+export function keyedBackend(s: WebSearchSettings): SearchBackend | undefined {
+ if (!keyedRequest(s, "q", 1)) return undefined;
+ const id = s.provider;
+ const keyed = s.provider !== "searxng";
+ return {
+ id,
+ async search(query, count, ctx) {
+ const r = keyedRequest(s, query, count)!;
+ const res = await send(id, ctx, r.url, r.init);
+ if (!res.ok) throw await httpFailure(id, res, keyed);
+ const text = await read(id, ctx, res);
+ let j: unknown;
+ try {
+ j = JSON.parse(text);
+ } catch {
+ throw new SearchFailure(id, "format", `not JSON: ${text.slice(0, 120)}`);
+ }
+ return r.parse(j);
+ },
+ };
+}
+
+// ---------- 服务 ----------
+
+export interface SearchServiceOptions {
+ settings: () => WebSearchSettings;
+ /** 桌面版是 Electron 的 net.fetch(系统代理);缺省 Node 的 fetch。每次搜索时取 */
+ fetch?: () => typeof fetch | undefined;
+ lang?: () => Lang;
+ /** 每个后端最多等多久(DSH 的 web_search 工具总共等 60 秒) */
+ timeoutMs?: number;
+ /** 出错后多久不再用它 */
+ cooldownMs?: number;
+ now?: () => number;
+ /** 测试注入:免费后端 */
+ free?: readonly SearchBackend[];
+}
+
+export interface SearchOutcome {
+ sources: WebSource[];
+ /** 给出结果的后端;都没有结果时是最后问的那个 */
+ backend: string;
+}
+
+export const SEARCH_TIMEOUT_MS = 12_000;
+export const SEARCH_COOLDOWN_MS = 10 * 60_000;
+
+export class SearchService {
+ private cooldown = new Map<string, number>();
+ private lastGood: string | undefined;
+ /** 带 Key 的接口最近一次出错(设置变了就清掉);Key 被拒时设置页提示 */
+ private keyed: { sig: string; failure?: SearchFailure } = { sig: "" };
+
+ constructor(private o: SearchServiceOptions) {}
+
+ /** 设置页:当前接口、Key 末 4 位、SearXNG 地址、Key 是否刚被拒 */
+ view(): WebSearchView {
+ const s = this.o.settings();
+ const key = s.provider === "auto" || s.provider === "searxng" ? "" : (s.keys[s.provider] ?? "");
+ const f = this.keyed.sig === sig(s) ? this.keyed.failure : undefined;
+ return { provider: s.provider, keyTail: key ? mask(key).slice(-4) : "", url: s.url, keyRejected: f?.kind === "auth" };
+ }
+
+ async search(query: string, maxResults = 8, signal?: AbortSignal): Promise<SearchOutcome> {
+ const q = query.trim();
+ const n = Math.max(1, Math.min(20, Math.floor(maxResults) || 8));
+ const t0 = Date.now();
+ const s = this.o.settings();
+ if (this.keyed.sig !== sig(s)) {
+ this.keyed = { sig: sig(s) };
+ for (const k of [...this.cooldown.keys()]) if (k.startsWith("keyed:")) this.cooldown.delete(k);
+ }
+ const tried: string[] = [];
+ let empty: string | undefined;
+ const keyed = keyedBackend(s);
+ const queue: Array<{ b: SearchBackend; slot: string }> = [];
+ if (keyed && this.ready(`keyed:${keyed.id}`)) queue.push({ b: keyed, slot: `keyed:${keyed.id}` });
+ for (const b of this.order()) queue.push({ b, slot: b.id });
+ for (const { b, slot } of queue) {
+ try {
+ const got = dedupe(await this.attempt(b, q, n, signal)).slice(0, n);
+ if (slot === `keyed:${b.id}`) this.keyed.failure = undefined;
+ if (!got.length) {
+ empty ??= b.id;
+ tried.push(`${b.id}: no results`);
+ continue;
+ }
+ this.cooldown.delete(slot);
+ if (slot === b.id) this.lastGood = b.id;
+ log.info("web search", { backend: b.id, results: got.length, ms: Date.now() - t0, ...(tried.length ? { before: tried } : {}) });
+ return { sources: got, backend: b.id };
+ } catch (e) {
+ if (e instanceof SearchAborted) throw e;
+ const f = e instanceof SearchFailure ? e : new SearchFailure(b.id, "format", errText(e));
+ this.cooldown.set(slot, this.now() + (this.o.cooldownMs ?? SEARCH_COOLDOWN_MS));
+ tried.push(`${b.id}: ${f.kind}`);
+ if (slot === `keyed:${b.id}`) {
+ this.keyed.failure = f;
+ if (f.kind === "auth") log.warn("web search key rejected; using the free search", { provider: b.id, status: f.status });
+ else log.warn("web search provider failed; using the free search", { provider: b.id, kind: f.kind, status: f.status, error: f.message });
+ } else log.warn("web search backend failed", { backend: b.id, kind: f.kind, status: f.status, error: f.message });
+ }
+ }
+ if (empty) {
+ log.info("web search", { backend: empty, results: 0, ms: Date.now() - t0, tried });
+ return { sources: [], backend: empty };
+ }
+ log.warn("web search failed on every backend", { ms: Date.now() - t0, tried });
+ throw new SearchFailure("all", "network", tried.join("; "));
+ }
+
+ /** 免费后端的顺序:上次成功的在前;冷却中的跳过,全在冷却时按冷却先结束的顺序都试 */
+ private order(): SearchBackend[] {
+ const all = [...(this.o.free ?? FREE_BACKENDS)];
+ const ready = all.filter((b) => this.ready(b.id));
+ if (!ready.length) return all.sort((a, b) => (this.cooldown.get(a.id) ?? 0) - (this.cooldown.get(b.id) ?? 0));
+ const i = ready.findIndex((b) => b.id === this.lastGood);
+ if (i > 0) ready.unshift(...ready.splice(i, 1));
+ return ready;
+ }
+
+ private ready(slot: string): boolean {
+ return (this.cooldown.get(slot) ?? 0) <= this.now();
+ }
+
+ private now(): number {
+ return this.o.now?.() ?? Date.now();
+ }
+
+ private async attempt(b: SearchBackend, q: string, n: number, signal?: AbortSignal): Promise<WebSource[]> {
+ if (signal?.aborted) throw new SearchAborted("aborted");
+ const timeout = AbortSignal.timeout(this.o.timeoutMs ?? SEARCH_TIMEOUT_MS);
+ const ctx: BackendContext = {
+ fetch: this.o.fetch?.() ?? fetch,
+ signal: signal ? AbortSignal.any([signal, timeout]) : timeout,
+ lang: this.o.lang?.() ?? appLang(),
+ };
+ try {
+ return await b.search(q, n, ctx);
+ } catch (e) {
+ if (signal?.aborted) throw new SearchAborted("aborted");
+ if (timeout.aborted && !(e instanceof SearchFailure)) throw new SearchFailure(b.id, "timeout", `no answer in ${this.o.timeoutMs ?? SEARCH_TIMEOUT_MS} ms`);
+ throw e;
+ }
+ }
+}
+
+function sig(s: WebSearchSettings): string {
+ return s.provider === "auto" ? "auto" : s.provider === "searxng" ? `searxng|${s.url}` : `${s.provider}|${s.keys[s.provider] ?? ""}`;
+}
+
+/** 按网址去重(忽略 #片段和末尾斜杠),保留先出现的 */
+export function dedupe(list: WebSource[]): WebSource[] {
+ const seen = new Set<string>();
+ return list.filter((r) => {
+ const k = r.url.replace(/#.*$/, "").replace(/\/+$/, "").toLowerCase();
+ if (seen.has(k)) return false;
+ seen.add(k);
+ return true;
+ });
+}
diff --git a/src/app.ts b/src/app.ts
index c2cac71..17132d1 100644
--- a/src/app.ts
+++ b/src/app.ts
@@ -44,6 +44,7 @@ import { AccountManager } from "./account/manager.js";
import { AccountVault, type SecretBox } from "./account/vault.js";
import { loadModelInfo, type ModelInfo, type Rates } from "./account/model-info.js";
import { MediaService, pickDefault } from "./agent/media.js";
+import { SearchService, loadWebSearch, saveWebSearch, type SearchProvider, type WebSearchSettings, type WebSearchView } from "./agent/search.js";
import { COMPACT_ENDPOINT, serverCompactionEnabled, type CompactEndpoint } from "./agent/server-compaction.js";
import type { BotSettings } from "./agent/settings.js";
import { TaskService } from "./tasks/service.js";
@@ -156,6 +157,9 @@ export class App {
readonly ruleEvents: RuleEvents;
/** "需要你":所有机器人里在等人的事(侧栏、托盘角标、系统通知) */
readonly attention: Attention;
+ /** 联网搜索(代理的 web_search):默认免费,设置 › 网络 里可换成带 Key 的接口 */
+ readonly search: SearchService;
+ private webSearch: WebSearchSettings;
web: WebServer | undefined;
desktop: DesktopHooks | undefined;
/** Modelflare 账户(登录、余额、预算);start() 时创建,桌面外壳的钥匙串那时已就绪 */
@@ -210,6 +214,9 @@ export class App {
this.magpie
? { baseUrl: this.magpie.baseUrl, apiKey: this.magpie.gatewayKey }
: { baseUrl: config.magpie.externalBaseUrl.replace(/\/+$/, ""), apiKey: config.magpie.externalApiKey };
+ this.webSearch = loadWebSearch(this.store);
+ // 桌面版搜索走 Chromium 的网络栈(系统代理、PAC、系统证书);命令行 / Docker 用 Node 的 fetch
+ this.search = new SearchService({ settings: () => this.webSearch, fetch: () => this.desktop?.fetch });
this.agent = new AgentManager({
store: this.store,
dataDir: config.dataDir,
@@ -253,6 +260,7 @@ export class App {
pricingUrl: () => `${modelflareIssuer()}/api/pricing`,
})
: undefined,
+ search: this.search,
onNotice: (text) => this.notice("warn", text),
});
this.attention = new Attention({ store: this.store });
@@ -736,6 +744,27 @@ export class App {
saveConfigFile(this.config.dataDir, (f) => (f.llm = { ...(f.llm ?? {}), defaultModel: model }));
}
+ /** 联网搜索设置(设置 › 网络):Key 只给末 4 位 */
+ webSearchView(): WebSearchView {
+ return this.search.view();
+ }
+
+ /** 改联网搜索(已校验):provider 换接口,各接口的 Key 分开留着;key 存给当前接口,空 = 删掉;url 是 SearXNG 地址 */
+ setWebSearch(patch: { provider?: SearchProvider; key?: string; url?: string }): WebSearchView {
+ const next: WebSearchSettings = { ...this.webSearch, keys: { ...this.webSearch.keys } };
+ if (patch.provider) next.provider = patch.provider;
+ const p = next.provider;
+ if (patch.key !== undefined && p !== "auto" && p !== "searxng") {
+ if (patch.key) next.keys[p] = patch.key;
+ else delete next.keys[p];
+ }
+ if (patch.url !== undefined) next.url = patch.url;
+ saveWebSearch(this.store, next);
+ this.webSearch = next;
+ log.info("web search settings changed", { provider: p, key: p !== "auto" && p !== "searxng" && !!next.keys[p], url: !!next.url });
+ return this.search.view();
+ }
+
/** 默认的图片 / 视频模型;空字符串 = 自动(推荐顺序的第一个) */
setMediaModels(patch: { imageModel?: string; videoModel?: string }): void {
Object.assign(this.config.media, patch);
diff --git a/src/i18n/messages/web.ts b/src/i18n/messages/web.ts
index 329f8ae..b8d8c59 100644
--- a/src/i18n/messages/web.ts
+++ b/src/i18n/messages/web.ts
@@ -12,6 +12,8 @@ export const zh = {
lowBalanceRange: "提醒线要在 0 到 1000 美元之间",
badZoom: "不支持的缩放比例",
badLang: "不支持的语言",
+ badSearchProvider: "不支持的搜索接口",
+ badSearchUrl: "SearXNG 地址要以 http:// 或 https:// 开头",
noAutostart: "当前环境不支持开机自启",
desktopOnly: "仅桌面版可用",
unknownGuideStep: "未知的引导步骤",
@@ -51,6 +53,8 @@ export const en: Catalog<typeof zh> = {
lowBalanceRange: "The alert threshold must be between $0 and $1,000",
badZoom: "Unsupported zoom level",
badLang: "Unsupported language",
+ badSearchProvider: "Unsupported search provider",
+ badSearchUrl: "The SearXNG address must start with http:// or https://",
noAutostart: "Launch at login isn't supported here",
desktopOnly: "Only available in the desktop app",
unknownGuideStep: "Unknown checklist step",
diff --git a/src/web/server.ts b/src/web/server.ts
index 0db2475..f9df0d2 100644
--- a/src/web/server.ts
+++ b/src/web/server.ts
@@ -21,6 +21,7 @@ import { connectionFromPids } from "../agent/peer-guard.js";
import type { ApprovalRow, Message, Store } from "../store/db.js";
import { defaultLine } from "../agent/approvals.js";
import { normalizeSettings } from "../agent/settings.js";
+import { SEARCH_PROVIDERS, normalizeSearxUrl, type SearchProvider } from "../agent/search.js";
import { scopeLabel } from "../agent/memory.js";
import { pwshProbe, type PwshMode, type PwshStatus } from "../agent/pwsh.js";
import { ACCESS_CHANNELS, peerName, shortPeer, whoText, type AccessChannel } from "../core/access.js";
@@ -428,6 +429,17 @@ export async function startWebServer(app: App, opts: { port?: number; host?: str
if (!app.desktop?.autostart().supported) throw new HttpError(400, t("web.noAutostart"));
app.desktop.setAutostart(!!b.autostart);
}
+ if (b.webSearch && typeof b.webSearch === "object") {
+ const w = b.webSearch as Record<string, unknown>;
+ if (w.provider !== undefined && !SEARCH_PROVIDERS.includes(w.provider as SearchProvider)) throw new HttpError(400, t("web.badSearchProvider"));
+ const url = w.url === undefined ? undefined : normalizeSearxUrl(String(w.url));
+ if (url === undefined && w.url !== undefined) throw new HttpError(400, t("web.badSearchUrl"));
+ app.setWebSearch({
+ ...(w.provider !== undefined ? { provider: w.provider as SearchProvider } : {}),
+ ...(w.key !== undefined ? { key: String(w.key).trim() } : {}),
+ ...(url !== undefined ? { url } : {}),
+ });
+ }
} else if (m !== "GET") throw new HttpError(405, "method not allowed");
return send(res, 200, {
defaultModel: cfg.llm.defaultModel,
@@ -440,6 +452,7 @@ export async function startWebServer(app: App, opts: { port?: number; host?: str
desktop: !!app.desktop,
listen: { host, port: boundPort, tokenRequired: !!token },
dataDir: cfg.dataDir,
+ webSearch: app.webSearchView(),
});
}
if (p === "/notices" && m === "GET") {
diff --git a/test/agent-bridge.unit.test.ts b/test/agent-bridge.unit.test.ts
index 0b5f1f5..7258f30 100644
--- a/test/agent-bridge.unit.test.ts
+++ b/test/agent-bridge.unit.test.ts
@@ -75,3 +75,33 @@ describe("bridge /dsh/media", () => {
expect(JSON.parse(r.chunks.join(""))).toEqual({ error: "boom" });
});
});
+
+describe("bridge /dsh/search", () => {
+ it("hands the query to the host and answers with its JSON; the plugin hanging up cancels the search", async () => {
+ const seen: unknown[] = [];
+ let cancelled = false;
+ bridge = new AgentBridge({
+ upstream: () => ({ baseUrl: "", apiKey: "" }),
+ ask: async () => ({ id: "" }),
+ wait: async () => ({}),
+ cancel: async () => undefined,
+ search: (b, signal) => {
+ seen.push(b);
+ if (b.query !== "hang") return Promise.resolve({ sources: [{ url: "https://a.example/" }], truncated: false });
+ return new Promise((resolve) => signal.addEventListener("abort", () => ((cancelled = true), resolve({ error: "aborted" }))));
+ },
+ });
+ await bridge.start();
+ const r = post(bridge, "/dsh/search", { botId: "b", query: "q", maxResults: 5 });
+ await r.end;
+ expect(JSON.parse(r.chunks.join(""))).toEqual({ sources: [{ url: "https://a.example/" }], truncated: false });
+ expect(seen).toEqual([{ botId: "b", query: "q", maxResults: 5 }]);
+ // web_search 超时或回合中止时插件的 fetch 断开连接:宿主那边的搜索收到取消
+ const ac = new AbortController();
+ const pending = fetch(bridge.url + "/dsh/search", { method: "POST", headers: { authorization: `Bearer ${bridge.token}`, "content-type": "application/json" }, body: JSON.stringify({ botId: "b", query: "hang" }), signal: ac.signal }).catch(() => "aborted");
+ await vi.waitFor(() => expect(seen).toHaveLength(2));
+ ac.abort();
+ expect(await pending).toBe("aborted");
+ await vi.waitFor(() => expect(cancelled).toBe(true));
+ });
+});
diff --git a/test/agent.integration.test.ts b/test/agent.integration.test.ts
index a521c3f..5063dc2 100644
--- a/test/agent.integration.test.ts
+++ b/test/agent.integration.test.ts
@@ -209,7 +209,7 @@ describe.skipIf(!runtime || !hasBwrap || process.platform !== "linux")("agent ho
function plan(messages: any[]): Step {
let lastUser = -1;
for (let i = messages.length - 1; i >= 0; i--) {
- if (messages[i].role === "user" && /QUICKSORT|ESCALATE|SECRET|READFILE|LOOPBACK|PRESENT|ASKQ|NATIVES|LOOKIMG|HANG|COUNTER|OUTWRITE|RMRF|RMWS|CHILDRM|SUBRM|SUBASK|SUBHELLO|PADDED|STEERME|SLOWOK|BGJOB|TOKENS|^background job |^Background subagent |\[用户消息\]/.test(textOf(messages[i].content))) { lastUser = i; break; }
+ if (messages[i].role === "user" && /QUICKSORT|ESCALATE|SECRET|READFILE|LOOPBACK|PRESENT|ASKQ|NATIVES|LOOKIMG|HANG|COUNTER|OUTWRITE|RMRF|RMWS|CHILDRM|SUBRM|SUBASK|SUBHELLO|PADDED|STEERME|SLOWOK|BGJOB|TOKENS|WEBSEARCH|^background job |^Background subagent |\[用户消息\]/.test(textOf(messages[i].content))) { lastUser = i; break; }
}
const user = textOf(messages[lastUser]?.content);
const tools = messages.slice(lastUser + 1).filter((m) => m.role === "tool");
@@ -259,6 +259,8 @@ describe.skipIf(!runtime || !hasBwrap || process.platform !== "linux")("agent ho
...(canStripTypes ? [{ name: "workflow", args: { script: "return { answer: 6 * 7 }", meta: { name: "calc", description: "multiply" } } }] : []),
]],
[/COUNTER/, [{ name: "bash", args: { description: "run", command: "echo ran >> counter.txt && wc -l < counter.txt" } }]],
+ // 联网搜索:DSH 的 web_search 经 amb-web-search 插件交给宿主(不是 DeepSeek 的搜索)
+ [/WEBSEARCH/, [{ name: "web_search", args: { queries: ["amb probe query"] } }]],
[/SLOWOK/, [{ name: "bash", args: { description: "run", command: `${gated(SLOW_GATE, 20)}; echo slow-done` } }]],
// 跑得比转后台的阈值久:命令转到后台,这一轮先回复"还在跑"
[/BGJOB/, [{ name: "bash", args: { description: "run", command: `${gated(BG_GATE, 30)}; echo bg-finished-42` } }]],
@@ -428,6 +430,20 @@ describe.skipIf(!runtime || !hasBwrap || process.platform !== "linux")("agent ho
expect(JSON.stringify(hist)).not.toMatch(/🔧|tool|bash:/);
}, 120_000);
+ it("web_search searches through the host (amb-web-search → bridge → app.search), not DeepSeek's search", async () => {
+ const seen: Array<[string, number]> = [];
+ const original = app.search.search;
+ app.search.search = async (q: string, n?: number) => (seen.push([q, n ?? 0]), { backend: "test", sources: [{ url: "https://probe.example/page", title: "Probe Title", snippet: "probe snippet" }] });
+ try {
+ const r = await chat("WEBSEARCH please");
+ expect(seen).toEqual([["amb probe query", 8]]);
+ expect(r.reply).toContain("[Probe Title](https://probe.example/page) — probe snippet");
+ expect(r.reply).not.toMatch(/DeepSeek|API key/i);
+ } finally {
+ app.search.search = original;
+ }
+ }, 120_000);
+
// —— 系统提示词:我们的人设与机器人段(每次请求现读),而不是 DSH 的 coding agent 人设 ——
/** 模型请求里的系统提示词(Chat:system 消息;Responses:instructions 与 system 条目;Messages:system) */
const systemOf = (c: { api: string; body: any }): string => {
diff --git a/test/agent.unit.test.ts b/test/agent.unit.test.ts
index f5affbc..2a32efd 100644
--- a/test/agent.unit.test.ts
+++ b/test/agent.unit.test.ts
@@ -418,6 +418,15 @@ describe("agent API per model", () => {
expect(text).toContain(' - id: amb-control\n name: "/rt/plugins/amb-control.mjs"\n');
});
+ it("web_search searches through the host: the amb-web-search plugin, and a top-level web row selecting it (fetch stays http)", () => {
+ const text = patchOf([route("m", "openai-completions")]);
+ expect(text).toContain(' - id: amb-web-search\n name: "/rt/plugins/amb-web-search.mjs"\n');
+ // 顶层的行补丁在插件 insert 列表之前;补丁整体替换 config,所以网页抓取的 http 也要写上。DeepSeek 自带的搜索不会被选中
+ expect(text).toContain("\n- id: web\n config:\n searchProvider: amb\n fetchProvider: http\n- id: ");
+ expect(text.indexOf("\n- id: web\n")).toBeLessThan(text.indexOf("\n- insert:\n"));
+ expect(text).not.toMatch(/deepseek-official|DEEPSEEK_API_KEY/);
+ });
+
it("long commands move to a background job: shell and job-wait limits sized for a chat assistant; PowerShell keeps its path", () => {
const text = patchOf([route("m", "openai-completions")], "C:\\PS7\\pwsh.exe");
expect(text).toContain(`- id: bash-sandbox\n config:\n timeoutMs: ${SHELL_LIMITS.bashMs}\n maxTimeoutMs: ${SHELL_LIMITS.bashMaxMs}\n`);
@@ -607,6 +616,9 @@ describe("full access patch", () => {
// 网页/渠道的审批、读围栏照旧
expect(text).toContain("- id: amb-bridge");
expect(text).toContain("- id: amb-permission");
+ // 搜索走宿主:完全权限下照样
+ expect(text).toContain(" - id: amb-web-search\n");
+ expect(text).toContain("\n- id: web\n config:\n searchProvider: amb\n fetchProvider: http\n");
expect(JSON.parse(env.AMB_DENY_READ)).toHaveLength(1);
});
it("other presets keep DSH's own permission rows", () => {
diff --git a/test/e2e.test.ts b/test/e2e.test.ts
index ea0da39..06bdb50 100644
--- a/test/e2e.test.ts
+++ b/test/e2e.test.ts
@@ -277,6 +277,25 @@ describe("Web 界面 / API", () => {
expect((await api("/settings", { method: "PUT", body: { autostart: true } })).status).toBe(400);
await api("/settings", { method: "PUT", body: { defaultModel: "", closeToTray: true, zoom: 1 } });
});
+ it("联网搜索设置:默认自动、不用 Key;Key 只回末 4 位、按接口分开存;SearXNG 地址规整;非法值拒绝", async () => {
+ expect((await api("/settings")).json.webSearch).toEqual({ provider: "auto", keyTail: "", url: "", keyRejected: false });
+ let r = await api("/settings", { method: "PUT", body: { webSearch: { provider: "tavily" } } });
+ expect(r.json.webSearch).toEqual({ provider: "tavily", keyTail: "", url: "", keyRejected: false });
+ r = await api("/settings", { method: "PUT", body: { webSearch: { key: " tvly-abcdef123456 " } } });
+ expect(r.json.webSearch).toMatchObject({ provider: "tavily", keyTail: "3456" });
+ expect(JSON.stringify(r.json)).not.toContain("tvly-abcdef");
+ expect(JSON.parse(app.store.getKV("web-search")!)).toEqual({ provider: "tavily", keys: { tavily: "tvly-abcdef123456" }, url: "" });
+ // 换接口不丢别的接口的 Key;空 Key = 删掉当前接口的
+ expect((await api("/settings", { method: "PUT", body: { webSearch: { provider: "exa" } } })).json.webSearch.keyTail).toBe("");
+ expect((await api("/settings", { method: "PUT", body: { webSearch: { provider: "tavily" } } })).json.webSearch.keyTail).toBe("3456");
+ expect((await api("/settings", { method: "PUT", body: { webSearch: { key: "" } } })).json.webSearch.keyTail).toBe("");
+ r = await api("/settings", { method: "PUT", body: { webSearch: { provider: "searxng", url: " https://searx.example/search/ " } } });
+ expect(r.json.webSearch).toMatchObject({ provider: "searxng", url: "https://searx.example" });
+ expect((await api("/settings", { method: "PUT", body: { webSearch: { provider: "google" } } })).status).toBe(400);
+ expect((await api("/settings", { method: "PUT", body: { webSearch: { url: "ftp://x" } } })).status).toBe(400);
+ await api("/settings", { method: "PUT", body: { webSearch: { provider: "auto", url: "" } } });
+ expect((await api("/settings")).json.webSearch).toEqual({ provider: "auto", keyTail: "", url: "", keyRejected: false });
+ });
it("窗口按钮配色:浏览器模式下没有桌面外壳,静默忽略", async () => {
const r = await api("/desktop/title-bar", { method: "POST", body: { dark: true } });
expect(r.status).toBe(200);
diff --git a/test/fixtures/web-search/bing-unrelated.html b/test/fixtures/web-search/bing-unrelated.html
new file mode 100644
index 0000000..5690574
--- /dev/null
+++ b/test/fixtures/web-search/bing-unrelated.html
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="zh"><head><meta charset="utf-8"><title>Bing</title></head><body><ol id="b_results" class=""><li class="b_algo" data-id iid=SERP.5308><div class="b_tpcn"><a class="tilk" aria-label="zhihu.com" RedirectUrl="" tabindex="-1" target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=1c4ab9cc864d4a4cd2f921ba2fdce9aa6d12459d7436281a73ae15002a461449JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=17f7c988-9a2c-6e28-070b-d18c9bb86f51&amp;u=a1aHR0cHM6Ly93d3cuemhpaHUuY29tL3F1ZXN0aW9uLzQ5Njg4NTQ5&amp;ntb=1" h="ID=SERP,5120.1"><div class="tpic"><div class="wr_fav" data-priority="2"><div class="cico siteicon" style="width:32px;height:32px;"><div class="rms_iac" style="height:32px;line-height:32px;width:32px;" data-height="32" data-width="32" data-alt="Global web icon" data-class="rms_img" data-src="https://th.bing.com/th/id/ODLS.A2450BEC-5595-40BA-9F13-D9EC6AB74B9F?w=32&amp;h=32&amp;qlt=90&amp;pcl=fffffa&amp;o=6&amp;pid=1.2"></div></div></div></div><div class="tptxt"><div class="tptt">zhihu.com</div><div class="tpmeta"><div class="b_attribution" tabindex="-1"><cite>https://www.zhihu.com › question</cite></div></div></div></a></div><h2><a target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=1c4ab9cc864d4a4cd2f921ba2fdce9aa6d12459d7436281a73ae15002a461449JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=17f7c988-9a2c-6e28-070b-d18c9bb86f51&amp;u=a1aHR0cHM6Ly93d3cuemhpaHUuY29tL3F1ZXN0aW9uLzQ5Njg4NTQ5&amp;ntb=1" h="ID=SERP,5120.2">applemusic和spotify哪个音质更好? - 知乎</a></h2><div class="b_caption"><p class="b_lineclamp2" data-rslinkclamp-iid="">Genius和Spotify之前也有合作的 边播歌下面还有storyline或者behind the lyrics(即歌手们传递创作信息或者想传递的故事等等)。 二 …</p></div><div elementtiming=frp.MiddleOfPage aria-hidden="true" style="pointer-events:none;margin-top:-1px;height:1px;width:1px;font-size:1px;position:absolute;top:0;right:0;" tabindex="-1"> &nbsp; </div></li><li class="b_algo" data-id iid=SERP.5309><div class="b_tpcn"><a class="tilk" aria-label="zhihu.com" RedirectUrl="" tabindex="-1" target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=cae62fa6ab85f32f7e816d91803430f960e8a2606a160121c789de0d134a6ce4JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=17f7c988-9a2c-6e28-070b-d18c9bb86f51&amp;u=a1aHR0cHM6Ly93d3cuemhpaHUuY29tL3F1ZXN0aW9uLzM2MjU1NzU2Mw&amp;ntb=1" h="ID=SERP,5133.1"><div class="tpic"><div class="wr_fav" data-priority="2"><div class="cico siteicon" style="width:32px;height:32px;"><div class="rms_iac" style="height:32px;line-height:32px;width:32px;" data-height="32" data-width="32" data-alt="Global web icon" data-class="rms_img" data-src="https://th.bing.com/th/id/ODLS.A2450BEC-5595-40BA-9F13-D9EC6AB74B9F?w=32&amp;h=32&amp;qlt=91&amp;pcl=fffffa&amp;o=6&amp;pid=1.2"></div></div></div></div><div class="tptxt"><div class="tptt">zhihu.com</div><div class="tpmeta"><div class="b_attribution" tabindex="-1"><cite>https://www.zhihu.com › question</cite></div></div></div></a></div><h2><a target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=cae62fa6ab85f32f7e816d91803430f960e8a2606a160121c789de0d134a6ce4JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=17f7c988-9a2c-6e28-070b-d18c9bb86f51&amp;u=a1aHR0cHM6Ly93d3cuemhpaHUuY29tL3F1ZXN0aW9uLzM2MjU1NzU2Mw&amp;ntb=1" h="ID=SERP,5133.2">有哪位大佬知道spotify怎么用吗? - 知乎</a></h2><div class="b_caption"><p class="b_lineclamp2" data-rslinkclamp-iid="">Spotify,即声田或声破天,是一家享誉全球的正版音乐服务平台,2008年10月在瑞典首都斯德哥尔摩正式上线。作为目前全球最大的 …</p></div></li><li class="b_algo" data-id iid=SERP.5310><div class="b_tpcn"><a class="tilk" aria-label="baidu.com" RedirectUrl="" tabindex="-1" target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=76e9be8e9d96f067c4d527c497baa61d2c9cb82c5c239e71e8f932aadd770b3eJmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=17f7c988-9a2c-6e28-070b-d18c9bb86f51&amp;u=a1aHR0cHM6Ly9qaW5neWFuLmJhaWR1LmNvbS9hcnRpY2xlL2YwMDYyMjI4NTdhNjJiYmFkMmYwYzgyYS5odG1s&amp;ntb=1" h="ID=SERP,5148.1"><div class="tpic"><div class="wr_fav" data-priority="2"><div class="cico siteicon" style="width:32px;height:32px;"><div class="rms_iac" style="height:32px;line-height:32px;width:32px;" data-height="32" data-width="32" data-alt="Global web icon" data-class="rms_img" data-src="https://th.bing.com/th/id/ODLS.A2450BEC-5595-40BA-9F13-D9EC6AB74B9F?w=32&amp;h=32&amp;qlt=92&amp;pcl=fffffa&amp;o=6&amp;pid=1.2"></div></div></div></div><div class="tptxt"><div class="tptt">baidu.com</div><div class="tpmeta"><div class="b_attribution" tabindex="-1"><cite>https://jingyan.baidu.com › article</cite></div></div></div></a></div><h2><a target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=76e9be8e9d96f067c4d527c497baa61d2c9cb82c5c239e71e8f932aadd770b3eJmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=17f7c988-9a2c-6e28-070b-d18c9bb86f51&amp;u=a1aHR0cHM6Ly9qaW5neWFuLmJhaWR1LmNvbS9hcnRpY2xlL2YwMDYyMjI4NTdhNjJiYmFkMmYwYzgyYS5odG1s&amp;ntb=1" h="ID=SERP,5148.2">spotify如何分享到朋友圈 - 百度经验</a></h2><div class="b_caption"><p class="b_lineclamp2" data-rslinkclamp-iid=""><span class="news_dt">Oct 20, 2023</span>&nbsp;&#0183;&#32;spotify如何分享到朋友圈,otify如何分享到朋友圈?当你在otify上听一首歌,觉得很喜欢的时候,可以一键分享给朋友 …</p></div></li><li class="b_ad"><ul><li><div class="sb_add"><h2><a href="https://www.bing.com/aclick?ld=x">Ad title</a></h2></div></li></ul></li></ol></body></html>
diff --git a/test/fixtures/web-search/bing.html b/test/fixtures/web-search/bing.html
new file mode 100644
index 0000000..06c2cb0
--- /dev/null
+++ b/test/fixtures/web-search/bing.html
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="zh"><head><meta charset="utf-8"><title>Bing</title></head><body><ol id="b_results" class=""><li class="b_algo" data-id iid=SERP.5329><div class="b_tpcn"><a class="tilk" aria-label="weather.com.cn" RedirectUrl="" tabindex="-1" target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=29e123c0ee80e27825a6689fd10d272b940894f27674d0ac958686b501d6e392JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=3864e560-5d14-631f-381a-fd645cfb6240&amp;u=a1aHR0cHM6Ly93d3cud2VhdGhlci5jb20uY24vd2VhdGhlci8xMDEwMTAxMDAuc2h0bWw&amp;ntb=1" h="ID=SERP,5124.1"><div class="tpic"><div class="wr_fav" data-priority="2"><div class="cico siteicon" style="width:32px;height:32px;"><div class="rms_iac" style="height:32px;line-height:32px;width:32px;" data-height="32" data-width="32" data-alt="全球 Web 图标" data-class="rms_img" data-src="https://th.bing.com/th/id/ODLS.A2450BEC-5595-40BA-9F13-D9EC6AB74B9F?w=32&amp;h=32&amp;qlt=90&amp;pcl=fffffa&amp;o=6&amp;pid=1.2"></div></div></div></div><div class="tptxt"><div class="tptt">weather.com.cn</div><div class="tpmeta"><div class="b_attribution" tabindex="-1"><cite>https://www.weather.com.cn <span class="b_ctdif">› weather</span></cite></div></div></div></a></div><h2><a target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=29e123c0ee80e27825a6689fd10d272b940894f27674d0ac958686b501d6e392JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=3864e560-5d14-631f-381a-fd645cfb6240&amp;u=a1aHR0cHM6Ly93d3cud2VhdGhlci5jb20uY24vd2VhdGhlci8xMDEwMTAxMDAuc2h0bWw&amp;ntb=1" h="ID=SERP,5124.2"><strong>北京天气</strong>预报,<strong>北京</strong>7天<strong>天气</strong>预报,<strong>北京</strong>15天<strong>天气</strong>预报,<strong>北京天气</strong>查询</a></h2><div class="b_caption"><p class="b_lineclamp2" data-rslinkclamp-iid="">高清图集 北京降雨过后路面湿滑 体感湿凉 北京秋色斑斓 元宝枫叶呈现渐变色 新疆昌吉:雪落丹霞 绘就冬日壮阔画卷 北京上空现“丁 …</p></div><div elementtiming=frp.MiddleOfPage aria-hidden="true" style="pointer-events:none;margin-top:-1px;height:1px;width:1px;font-size:1px;position:absolute;top:0;right:0;" tabindex="-1"> &nbsp; </div></li><li class="b_algo" data-id iid=SERP.5330><div class="b_tpcn"><a class="tilk" aria-label="nmc.cn" RedirectUrl="" tabindex="-1" target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=21c28ee1dd4c04ee6376ee784638a7cde51d981e0980d2fbeb36616379ef37a8JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=3864e560-5d14-631f-381a-fd645cfb6240&amp;u=a1aHR0cHM6Ly93d3cubm1jLmNuL3B1Ymxpc2gvZm9yZWNhc3QvQUJKL2JlaWppbmcuaHRtbA&amp;ntb=1" h="ID=SERP,5140.1"><div class="tpic"><div class="wr_fav" data-priority="2"><div class="cico siteicon" style="width:32px;height:32px;"><div class="rms_iac" style="height:32px;line-height:32px;width:32px;" data-height="32" data-width="32" data-alt="全球 Web 图标" data-class="rms_img" data-src="https://th.bing.com/th/id/ODLS.A2450BEC-5595-40BA-9F13-D9EC6AB74B9F?w=32&amp;h=32&amp;qlt=91&amp;pcl=fffffa&amp;o=6&amp;pid=1.2"></div></div></div></div><div class="tptxt"><div class="tptt">nmc.cn</div><div class="tpmeta"><div class="b_attribution" tabindex="-1"><cite>https://www.nmc.cn <span class="b_ctdif">› publish › forecast › ABJ › beijing.html</span></cite></div></div></div></a></div><h2><a target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=21c28ee1dd4c04ee6376ee784638a7cde51d981e0980d2fbeb36616379ef37a8JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=3864e560-5d14-631f-381a-fd645cfb6240&amp;u=a1aHR0cHM6Ly93d3cubm1jLmNuL3B1Ymxpc2gvZm9yZWNhc3QvQUJKL2JlaWppbmcuaHRtbA&amp;ntb=1" h="ID=SERP,5140.2"><strong>北京-天气</strong>预报</a></h2><div class="b_caption"><p class="b_lineclamp2" data-rslinkclamp-iid="">当前位置: 首页 北京市 北京天气预报 省份: 北京 北京 北京 城市: 北京 北京 北京 日出 北京 日落 降水量 相对湿度 体感温度 雷达图 …</p></div></li><li class="b_algo" data-id iid=SERP.5331><div class="b_tpcn"><a class="tilk" aria-label="msn.com" RedirectUrl="" tabindex="-1" target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=d75bc3ad726f80ffa238009ad01181b7f2ff416fcbc82427a684c8a44d8fce81JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=3864e560-5d14-631f-381a-fd645cfb6240&amp;u=a1aHR0cHM6Ly93d3cubXNuLmNvbS96aC1jbi93ZWF0aGVyL2ZvcmVjYXN0L2luLSVFNSU4QyU5NyVFNCVCQSVBQw&amp;ntb=1" h="ID=SERP,5156.1"><div class="tpic"><div class="wr_fav" data-priority="2"><div class="cico siteicon" style="width:32px;height:32px;"><div class="rms_iac" style="height:32px;line-height:32px;width:32px;" data-height="32" data-width="32" data-alt="全球 Web 图标" data-class="rms_img" data-src="https://th.bing.com/th/id/ODLS.A2450BEC-5595-40BA-9F13-D9EC6AB74B9F?w=32&amp;h=32&amp;qlt=92&amp;pcl=fffffa&amp;o=6&amp;pid=1.2"></div></div></div></div><div class="tptxt"><div class="tptt">msn.com</div><div class="tpmeta"><div class="b_attribution" tabindex="-1"><cite>https://www.msn.com <span class="b_ctdif">› zh-cn › weather › forecast › in-北京</span></cite></div></div></div></a></div><h2><a target="_blank" href="https://www.bing.com/ck/a?!&amp;&amp;p=d75bc3ad726f80ffa238009ad01181b7f2ff416fcbc82427a684c8a44d8fce81JmltdHM9MTc5MzQ5MTIwMA&amp;ptn=3&amp;ver=2&amp;hsh=4&amp;fclid=3864e560-5d14-631f-381a-fd645cfb6240&amp;u=a1aHR0cHM6Ly93d3cubXNuLmNvbS96aC1jbi93ZWF0aGVyL2ZvcmVjYXN0L2luLSVFNSU4QyU5NyVFNCVCQSVBQw&amp;ntb=1" h="ID=SERP,5156.2"><strong>北京 天气</strong>预报 | MSN <strong>天气</strong></a></h2><div class="b_caption"><p class="b_lineclamp2" data-rslinkclamp-iid="">12 &#176;C 以晴为主 体感温度 11&#176; 大部分地区天气晴朗。 最低气温3&#176;。 空气质量 空气优 (0-50) 主要污染物 PM10 21 μg/m&#179; 36 风速 轻风 (2 …</p></div></li><li class="b_ad"><ul><li><div class="sb_add"><h2><a href="https://www.bing.com/aclick?ld=x">Ad title</a></h2></div></li></ul></li></ol></body></html>
diff --git a/test/fixtures/web-search/ddg-challenge.html b/test/fixtures/web-search/ddg-challenge.html
new file mode 100644
index 0000000..deb986f
--- /dev/null
+++ b/test/fixtures/web-search/ddg-challenge.html
@@ -0,0 +1,8 @@
+<!DOCTYPE html><html lang="en"><head><title>DuckDuckGo</title></head><body><center id="lite_wrapper"><form id="challenge-form" action="//duckduckgo.com/anomaly.js?sv=html&cc=sre&st=1793545342&gk=d4cd0dabcf4caa22ad92fab40844c786&p=393ec9411ba7b347f6135c29ba523749-67d08ca414af695b4e35d2f424a0fb7f-c4265459ef57c6a30c7bca3f80465c43-d9911e6988e57da3a85d21f61f591fe1-10b707df866e8e33cfbabeee6b8e1401-d00e62f93da14d23e5aa76f46040ff07-8cdafec8fff79c64917d7341c9340be3-4bfac92d830434253251e17211d34e8b-c520a17980965ba05bc0623583384cb1&q=北京天气&o=mc08i4uc7n%2FgNtuAZHnFJd%2F72nKcACi3KR1zgWqrrflVCMRvmAX69ZzA4rjBjtiV6pYIpZUz7UxB%0AB%2Bs4voNWAQ%3D%3D%0A&r=ase" method="POST">
+ <div class="anomaly-modal__mask">
+ <div class="anomaly-modal__modal is-ie" data-testid="anomaly-modal">
+ <div class="anomaly-modal__title">Unfortunately, bots use DuckDuckGo too.</div>
+ <div class="anomaly-modal__description">Please complete the following challenge to confirm this search was made by a human.</div>
+
+
+ </div></div></form></center></body></html>
diff --git a/test/fixtures/web-search/ddg.html b/test/fixtures/web-search/ddg.html
new file mode 100644
index 0000000..637843c
--- /dev/null
+++ b/test/fixtures/web-search/ddg.html
@@ -0,0 +1,66 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8"><title>node.js 22 release notes at DuckDuckGo</title></head><body><div class="serp__results"><div id="links" class="results">
+<div class="result results_links results_links_deep result--ad "> <div class="links_main links_deep result__body"> <!-- This is the visible part --> <h2 class="result__title"> <a rel="nofollow" class="result__a" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fduckduckgo.com%2Fy.js%3Fad_domain%3Dcheapflights.com.sg%26ad_provider%3Dbingv7aa%26ad_type%3Dtxad%26click_metadata%3DD7EIQaFbQ9I8beQg0NuCt32_8v1Q9lAnoD9P8mym1S9kgSVTMPBVpL4vVN55c5Gi9RgVfVNuiEpFzk9LiNwVXSdrmWX9iv1yzWjAkKYaP4diUQkeUg_giCSZoIfjnpzugCxzqjsh1EY6uKA_1bvSDYOuvcTRIRFz2oNN4g7k204.gtoY4vXn93A34EkfadzwWw%26rut%3Dea3b2ff86d11272440e85a77cd5a402327da51a0dd3713eceefd66a83100b90f%26u3%3Dhttps%253A%252F%252Fwww.bing.com%252Faclick%253Fld%253De83doN2suEZJgJY2gFyi5p3zVUCUwAVN_TdImd7nViWzzrURCXd1FCJ8iwyGNa4UbYfWpQsKY8VGZ%252D3iOY%252DzHqm8JBpJR8IfPd62i8Ricj2lZvHIl2IlAohSIhGKAGElY4DLnthrqMP%252DUyix_3CWy55Ajmzj_v%252D0T0u2EBxHrUTR7PR_CX7glDRgzHeyLDzboueRyEpBXAeRvKhsDeoJBw7SVVQQA%2526u%253DaHR0cHMlM2ElMmYlMmZ3d3cuY2hlYXBmbGlnaHRzLmNvbS5zZyUyZnNlbWklMmZiaW5nYWRzc2VhcmNoJTJmZmxpZ2h0X2dlbmVyYWwlMmZhbnklMmZlbi5odG1sJTNmbXQlM2RwJTI2Ym10JTNkYmIlMjZvaWlkJTNkNzE5NTAxMjM0NDc1MzclMjZhaWQlM2QxMTUxMTkwNDY3MDEwNTI1JTI2YWRpZCUzZDcxOTQ5NTY4NTYzNjY2JTI2cGlkJTNkJTI2dGlkJTNka3dkLTcxOTUwMTIzNDQ3NTM3JTNhbG9jLTE2NCUyNmQlM2RjJTI2biUzZHMlMjZjbXBpZCUzZDU4MDgxOTg3NyUyNmsuYSUzZGJpbmdhZHNzZWFyY2glMjZtc2Nsa2lkJTNkZjkyYzljNzIwNTY1MWY4NDUzNWZmNjk1NTA3ZTFkYTQlMjZ1dG1fc291cmNlJTNkYmluZyUyNnV0bV9tZWRpdW0lM2RjcGMlMjZ1dG1fY2FtcGFpZ24lM2RDb21wZXRpdG9yX0dlbmVyaWMlMjUyMC0lMjUyMEdyb3VwJTI2dXRtX3Rlcm0lM2RiaWxsaWdmbGllZ2VyJTI2dXRtX2NvbnRlbnQlM2RUJTI1M0Rub25lJTI1MjAtJTI1MjBQJTI1M0Rub25lJTI1MjAtJTI1MjBPJTI1M0ROb25lJTI1MjAtJTI1MjBEJTI1M0ROb25lJTI1MjAtJTI1MjBTJTI1M0RiaWxsaWdmbGllZ2VyLmRl%2526rlid%253Df92c9c7205651f84535ff695507e1da4%26vqd%3D4%2D129894861294845585964898057611786943543%26iurl%3D%257B1%257DIG%253DAE2D3DE72FE84F4FB84457B82B0ADFA0%2526CID%253D25F5B2F01156648B3596AAF4104D6535%2526ID%253DDevEx%252C5036.1&amp;rut=16c59c41d81ab96040be5f3ee6d84183355b10a535d4297ddfe0df226041680c">Compare flights for free - cheapflights.com.sg</a> <div class="result__badge-wrap"> <button class="badge--ad">Ad</button> <div class="badge--ad__tooltip-wrap"> <div class="tooltip tooltip--right"> <div class="tooltip__triangle-outer"></div> <div class="tooltip__triangle-inner"></div> Viewing ads is privacy protected by DuckDuckGo. Ad clicks are managed by Microsoft's ad network (<a href="https://duckduckgo.com/duckduckgo-help-pages/company/ads-by-microsoft-on-duckduckgo-private-search" class="tooltip-link">more info</a>). </div> </div> </div> </h2> <div class="result__extras"> <div class="result__extras__url"> <!-- Disable Ad Favicon <span class="result__icon"> <a rel="nofollow" class="result__icon__img" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fduckduckgo.com%2Fy.js%3Fad_domain%3Dcheapflights.com.sg%26ad_provider%3Dbingv7aa%26ad_type%3Dtxad%26click_metadata%3DD7EIQaFbQ9I8beQg0NuCt32_8v1Q9lAnoD9P8mym1S9kgSVTMPBVpL4vVN55c5Gi9RgVfVNuiEpFzk9LiNwVXSdrmWX9iv1yzWjAkKYaP4diUQkeUg_giCSZoIfjnpzugCxzqjsh1EY6uKA_1bvSDYOuvcTRIRFz2oNN4g7k204.gtoY4vXn93A34EkfadzwWw%26rut%3Dea3b2ff86d11272440e85a77cd5a402327da51a0dd3713eceefd66a83100b90f%26u3%3Dhttps%253A%252F%252Fwww.bing.com%252Faclick%253Fld%253De83doN2suEZJgJY2gFyi5p3zVUCUwAVN_TdImd7nViWzzrURCXd1FCJ8iwyGNa4UbYfWpQsKY8VGZ%252D3iOY%252DzHqm8JBpJR8IfPd62i8Ricj2lZvHIl2IlAohSIhGKAGElY4DLnthrqMP%252DUyix_3CWy55Ajmzj_v%252D0T0u2EBxHrUTR7PR_CX7glDRgzHeyLDzboueRyEpBXAeRvKhsDeoJBw7SVVQQA%2526u%253DaHR0cHMlM2ElMmYlMmZ3d3cuY2hlYXBmbGlnaHRzLmNvbS5zZyUyZnNlbWklMmZiaW5nYWRzc2VhcmNoJTJmZmxpZ2h0X2dlbmVyYWwlMmZhbnklMmZlbi5odG1sJTNmbXQlM2RwJTI2Ym10JTNkYmIlMjZvaWlkJTNkNzE5NTAxMjM0NDc1MzclMjZhaWQlM2QxMTUxMTkwNDY3MDEwNTI1JTI2YWRpZCUzZDcxOTQ5NTY4NTYzNjY2JTI2cGlkJTNkJTI2dGlkJTNka3dkLTcxOTUwMTIzNDQ3NTM3JTNhbG9jLTE2NCUyNmQlM2RjJTI2biUzZHMlMjZjbXBpZCUzZDU4MDgxOTg3NyUyNmsuYSUzZGJpbmdhZHNzZWFyY2glMjZtc2Nsa2lkJTNkZjkyYzljNzIwNTY1MWY4NDUzNWZmNjk1NTA3ZTFkYTQlMjZ1dG1fc291cmNlJTNkYmluZyUyNnV0bV9tZWRpdW0lM2RjcGMlMjZ1dG1fY2FtcGFpZ24lM2RDb21wZXRpdG9yX0dlbmVyaWMlMjUyMC0lMjUyMEdyb3VwJTI2dXRtX3Rlcm0lM2RiaWxsaWdmbGllZ2VyJTI2dXRtX2NvbnRlbnQlM2RUJTI1M0Rub25lJTI1MjAtJTI1MjBQJTI1M0Rub25lJTI1MjAtJTI1MjBPJTI1M0ROb25lJTI1MjAtJTI1MjBEJTI1M0ROb25lJTI1MjAtJTI1MjBTJTI1M0RiaWxsaWdmbGllZ2VyLmRl%2526rlid%253Df92c9c7205651f84535ff695507e1da4%26vqd%3D4%2D129894861294845585964898057611786943543%26iurl%3D%257B1%257DIG%253DAE2D3DE72FE84F4FB84457B82B0ADFA0%2526CID%253D25F5B2F01156648B3596AAF4104D6535%2526ID%253DDevEx%252C5036.1&amp;rut=16c59c41d81ab96040be5f3ee6d84183355b10a535d4297ddfe0df226041680c"> <img width="16" height="16" alt="" src="//external-content.duckduckgo.com/ip3/.ico" name="i15" /> </a> </span> --> <a class="result__url" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fduckduckgo.com%2Fy.js%3Fad_domain%3Dcheapflights.com.sg%26ad_provider%3Dbingv7aa%26ad_type%3Dtxad%26click_metadata%3DD7EIQaFbQ9I8beQg0NuCt32_8v1Q9lAnoD9P8mym1S9kgSVTMPBVpL4vVN55c5Gi9RgVfVNuiEpFzk9LiNwVXSdrmWX9iv1yzWjAkKYaP4diUQkeUg_giCSZoIfjnpzugCxzqjsh1EY6uKA_1bvSDYOuvcTRIRFz2oNN4g7k204.gtoY4vXn93A34EkfadzwWw%26rut%3Dea3b2ff86d11272440e85a77cd5a402327da51a0dd3713eceefd66a83100b90f%26u3%3Dhttps%253A%252F%252Fwww.bing.com%252Faclick%253Fld%253De83doN2suEZJgJY2gFyi5p3zVUCUwAVN_TdImd7nViWzzrURCXd1FCJ8iwyGNa4UbYfWpQsKY8VGZ%252D3iOY%252DzHqm8JBpJR8IfPd62i8Ricj2lZvHIl2IlAohSIhGKAGElY4DLnthrqMP%252DUyix_3CWy55Ajmzj_v%252D0T0u2EBxHrUTR7PR_CX7glDRgzHeyLDzboueRyEpBXAeRvKhsDeoJBw7SVVQQA%2526u%253DaHR0cHMlM2ElMmYlMmZ3d3cuY2hlYXBmbGlnaHRzLmNvbS5zZyUyZnNlbWklMmZiaW5nYWRzc2VhcmNoJTJmZmxpZ2h0X2dlbmVyYWwlMmZhbnklMmZlbi5odG1sJTNmbXQlM2RwJTI2Ym10JTNkYmIlMjZvaWlkJTNkNzE5NTAxMjM0NDc1MzclMjZhaWQlM2QxMTUxMTkwNDY3MDEwNTI1JTI2YWRpZCUzZDcxOTQ5NTY4NTYzNjY2JTI2cGlkJTNkJTI2dGlkJTNka3dkLTcxOTUwMTIzNDQ3NTM3JTNhbG9jLTE2NCUyNmQlM2RjJTI2biUzZHMlMjZjbXBpZCUzZDU4MDgxOTg3NyUyNmsuYSUzZGJpbmdhZHNzZWFyY2glMjZtc2Nsa2lkJTNkZjkyYzljNzIwNTY1MWY4NDUzNWZmNjk1NTA3ZTFkYTQlMjZ1dG1fc291cmNlJTNkYmluZyUyNnV0bV9tZWRpdW0lM2RjcGMlMjZ1dG1fY2FtcGFpZ24lM2RDb21wZXRpdG9yX0dlbmVyaWMlMjUyMC0lMjUyMEdyb3VwJTI2dXRtX3Rlcm0lM2RiaWxsaWdmbGllZ2VyJTI2dXRtX2NvbnRlbnQlM2RUJTI1M0Rub25lJTI1MjAtJTI1MjBQJTI1M0Rub25lJTI1MjAtJTI1MjBPJTI1M0ROb25lJTI1MjAtJTI1MjBEJTI1M0ROb25lJTI1MjAtJTI1MjBTJTI1M0RiaWxsaWdmbGllZ2VyLmRl%2526rlid%253Df92c9c7205651f84535ff695507e1da4%26vqd%3D4%2D129894861294845585964898057611786943543%26iurl%3D%257B1%257DIG%253DAE2D3DE72FE84F4FB84457B82B0ADFA0%2526CID%253D25F5B2F01156648B3596AAF4104D6535%2526ID%253DDevEx%252C5036.1&amp;rut=16c59c41d81ab96040be5f3ee6d84183355b10a535d4297ddfe0df226041680c">cheapflights.com.sg</a> </div> </div> <a class="result__snippet" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fduckduckgo.com%2Fy.js%3Fad_domain%3Dcheapflights.com.sg%26ad_provider%3Dbingv7aa%26ad_type%3Dtxad%26click_metadata%3DD7EIQaFbQ9I8beQg0NuCt32_8v1Q9lAnoD9P8mym1S9kgSVTMPBVpL4vVN55c5Gi9RgVfVNuiEpFzk9LiNwVXSdrmWX9iv1yzWjAkKYaP4diUQkeUg_giCSZoIfjnpzugCxzqjsh1EY6uKA_1bvSDYOuvcTRIRFz2oNN4g7k204.gtoY4vXn93A34EkfadzwWw%26rut%3Dea3b2ff86d11272440e85a77cd5a402327da51a0dd3713eceefd66a83100b90f%26u3%3Dhttps%253A%252F%252Fwww.bing.com%252Faclick%253Fld%253De83doN2suEZJgJY2gFyi5p3zVUCUwAVN_TdImd7nViWzzrURCXd1FCJ8iwyGNa4UbYfWpQsKY8VGZ%252D3iOY%252DzHqm8JBpJR8IfPd62i8Ricj2lZvHIl2IlAohSIhGKAGElY4DLnthrqMP%252DUyix_3CWy55Ajmzj_v%252D0T0u2EBxHrUTR7PR_CX7glDRgzHeyLDzboueRyEpBXAeRvKhsDeoJBw7SVVQQA%2526u%253DaHR0cHMlM2ElMmYlMmZ3d3cuY2hlYXBmbGlnaHRzLmNvbS5zZyUyZnNlbWklMmZiaW5nYWRzc2VhcmNoJTJmZmxpZ2h0X2dlbmVyYWwlMmZhbnklMmZlbi5odG1sJTNmbXQlM2RwJTI2Ym10JTNkYmIlMjZvaWlkJTNkNzE5NTAxMjM0NDc1MzclMjZhaWQlM2QxMTUxMTkwNDY3MDEwNTI1JTI2YWRpZCUzZDcxOTQ5NTY4NTYzNjY2JTI2cGlkJTNkJTI2dGlkJTNka3dkLTcxOTUwMTIzNDQ3NTM3JTNhbG9jLTE2NCUyNmQlM2RjJTI2biUzZHMlMjZjbXBpZCUzZDU4MDgxOTg3NyUyNmsuYSUzZGJpbmdhZHNzZWFyY2glMjZtc2Nsa2lkJTNkZjkyYzljNzIwNTY1MWY4NDUzNWZmNjk1NTA3ZTFkYTQlMjZ1dG1fc291cmNlJTNkYmluZyUyNnV0bV9tZWRpdW0lM2RjcGMlMjZ1dG1fY2FtcGFpZ24lM2RDb21wZXRpdG9yX0dlbmVyaWMlMjUyMC0lMjUyMEdyb3VwJTI2dXRtX3Rlcm0lM2RiaWxsaWdmbGllZ2VyJTI2dXRtX2NvbnRlbnQlM2RUJTI1M0Rub25lJTI1MjAtJTI1MjBQJTI1M0Rub25lJTI1MjAtJTI1MjBPJTI1M0ROb25lJTI1MjAtJTI1MjBEJTI1M0ROb25lJTI1MjAtJTI1MjBTJTI1M0RiaWxsaWdmbGllZ2VyLmRl%2526rlid%253Df92c9c7205651f84535ff695507e1da4%26vqd%3D4%2D129894861294845585964898057611786943543%26iurl%3D%257B1%257DIG%253DAE2D3DE72FE84F4FB84457B82B0ADFA0%2526CID%253D25F5B2F01156648B3596AAF4104D6535%2526ID%253DDevEx%252C5036.1&amp;rut=16c59c41d81ab96040be5f3ee6d84183355b10a535d4297ddfe0df226041680c">Better deals with one easy search. Compare <b>cheap</b> <b>flights</b> to find your best fit. <b>cheapflights</b>.com.sg. Join millions of happy users and find your ideal <b>flight</b>.</a> <div class="clear"></div> </div> </div>
+<div class="result results_links results_links_deep web-result ">
+ <div class="links_main links_deep result__body"> <!-- This is the visible part -->
+ <h2 class="result__title">
+ <a rel="nofollow" class="result__a" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fnodejs.org%2Fen%2Fblog%2Fannouncements%2Fv22%2Drelease%2Dannounce&amp;rut=def750526e98d3b1942b91f7c2c8022519f132069fba606ea5206a3e6b4de1d1">Node.js 22 is now available!</a>
+ </h2>
+ <div class="result__extras">
+ <div class="result__extras__url">
+ <span class="result__icon">
+ <a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fnodejs.org%2Fen%2Fblog%2Fannouncements%2Fv22%2Drelease%2Dannounce&amp;rut=def750526e98d3b1942b91f7c2c8022519f132069fba606ea5206a3e6b4de1d1">
+ <img class="result__icon__img" width="16" height="16" alt="" src="//external-content.duckduckgo.com/ip3/nodejs.org.ico" name="i15" />
+ </a>
+ </span>
+ <a class="result__url" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fnodejs.org%2Fen%2Fblog%2Fannouncements%2Fv22%2Drelease%2Dannounce&amp;rut=def750526e98d3b1942b91f7c2c8022519f132069fba606ea5206a3e6b4de1d1">
+ nodejs.org/en/blog/announcements/v22-release-announce
+ </a>
+ </div>
+ </div>
+ <a class="result__snippet" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fnodejs.org%2Fen%2Fblog%2Fannouncements%2Fv22%2Drelease%2Dannounce&amp;rut=def750526e98d3b1942b91f7c2c8022519f132069fba606ea5206a3e6b4de1d1">Contributed by Raz Luvaton in #52505 Call to action Try out the new <b>Node.js</b> <b>22</b> <b>release</b>! We&#x27;re always happy to hear your feedback. Testing your applications and modules with <b>Node.js</b> <b>22</b> helps to ensure the future compatibility of your project with the latest <b>Node.js</b> changes and features.</a>
+ <div class="clear"></div>
+ </div>
+ </div>
+<div class="result results_links results_links_deep web-result ">
+ <div class="links_main links_deep result__body"> <!-- This is the visible part -->
+ <h2 class="result__title">
+ <a rel="nofollow" class="result__a" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fnodejs.org%2Fen%2Fblog%2Frelease%2Fv22.22.0&amp;rut=74d7c401b2694913937cd57c68d61df7b94aef50423e3219a52361fd16217b03">Node.js — Node.js 22.22.0 (LTS)</a>
+ </h2>
+ <div class="result__extras">
+ <div class="result__extras__url">
+ <span class="result__icon">
+ <a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fnodejs.org%2Fen%2Fblog%2Frelease%2Fv22.22.0&amp;rut=74d7c401b2694913937cd57c68d61df7b94aef50423e3219a52361fd16217b03">
+ <img class="result__icon__img" width="16" height="16" alt="" src="//external-content.duckduckgo.com/ip3/nodejs.org.ico" name="i15" />
+ </a>
+ </span>
+ <a class="result__url" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fnodejs.org%2Fen%2Fblog%2Frelease%2Fv22.22.0&amp;rut=74d7c401b2694913937cd57c68d61df7b94aef50423e3219a52361fd16217b03">
+ nodejs.org/en/blog/release/v22.22.0
+ </a>
+ </div>
+ </div>
+ <a class="result__snippet" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fnodejs.org%2Fen%2Fblog%2Frelease%2Fv22.22.0&amp;rut=74d7c401b2694913937cd57c68d61df7b94aef50423e3219a52361fd16217b03"><b>Node.js</b>® is a free, open-source, cross-platform JavaScript runtime environment that lets developers create servers, web apps, command line tools and scripts.</a>
+ <div class="clear"></div>
+ </div>
+ </div>
+<div class="result results_links results_links_deep web-result ">
+ <div class="links_main links_deep result__body"> <!-- This is the visible part -->
+ <h2 class="result__title">
+ <a rel="nofollow" class="result__a" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fversionlog.com%2Fnodejs%2F22%2F&amp;rut=6dad3efeee4ca236ab575553b04e3f06f0fe5a65b0dc6a902bbdf94ce6701668">Node.js 22 (LTS) - What&#x27;s New, Support Lifecycle &amp; EOL</a>
+ </h2>
+ <div class="result__extras">
+ <div class="result__extras__url">
+ <span class="result__icon">
+ <a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fversionlog.com%2Fnodejs%2F22%2F&amp;rut=6dad3efeee4ca236ab575553b04e3f06f0fe5a65b0dc6a902bbdf94ce6701668">
+ <img class="result__icon__img" width="16" height="16" alt="" src="//external-content.duckduckgo.com/ip3/versionlog.com.ico" name="i15" />
+ </a>
+ </span>
+ <a class="result__url" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fversionlog.com%2Fnodejs%2F22%2F&amp;rut=6dad3efeee4ca236ab575553b04e3f06f0fe5a65b0dc6a902bbdf94ce6701668">
+ versionlog.com/nodejs/22/
+ </a>
+ </div>
+ </div>
+ <a class="result__snippet" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fversionlog.com%2Fnodejs%2F22%2F&amp;rut=6dad3efeee4ca236ab575553b04e3f06f0fe5a65b0dc6a902bbdf94ce6701668">Discover what&#x27;s new in <b>Node.js</b> <b>22</b>. Complete <b>release</b> history, support lifecycle, <b>release</b> dates, LTS schedule and End of Life (EOL) status. <b>Node.js</b> is an...</a>
+ <div class="clear"></div>
+ </div>
+ </div>
+</div></div></body></html>
diff --git a/test/fixtures/web-search/exa-mcp-rate-limited.sse b/test/fixtures/web-search/exa-mcp-rate-limited.sse
new file mode 100644
index 0000000..e8cca48
--- /dev/null
+++ b/test/fixtures/web-search/exa-mcp-rate-limited.sse
@@ -0,0 +1,3 @@
+event: message
+data: {"result":{"_meta":{"ai.exa/rateLimited":true},"content":[{"type":"text","text":"You've hit Exa's free MCP rate limit. To continue using without limits, create your own Exa API key.\n\nFix: Create API key at https://dashboard.exa.ai/api-keys , and then update Exa MCP URL to this https://mcp.exa.ai/mcp?exaApiKey=YOUR_EXA_API_KEY"}]},"jsonrpc":"2.0","id":1}
+
diff --git a/test/fixtures/web-search/exa-mcp.sse b/test/fixtures/web-search/exa-mcp.sse
new file mode 100644
index 0000000..d266650
--- /dev/null
+++ b/test/fixtures/web-search/exa-mcp.sse
@@ -0,0 +1,3 @@
+event: message
+data: {"result": {"content": [{"type": "text", "text": "Title: doc/changelogs/CHANGELOG_V22.md\nURL: https://github.com/nodejs/node/blob/main/doc/changelogs/CHANGELOG_V22.md\nPublished: N/A\nAuthor: N/A\nHighlights:\n# Node.js 22 ChangeLog\n\n \n\n \n \n LTS 'Jod' \n Current \n \n \n \n 22.23.4 \n 22.23.3 \n 22.23.2 \n 22.23.1 \n 22.23.0 \n 22.22.3 \n 22.22.2 \n 22.22.1 \n 22.22.0 \n 22.21.1 \n 22.21.0 \n 22.20.0 \n 22.19.0 \n 22.18.0 \n 22.17.1 \n 22.17.0 \n 22.16.0 \n 22.15.1 \n 22.15.0 \n 22.14.0 \n 22.13.1 \n 22.13.0 \n 22.12.0 \n 22.11.0 \n \n \n 22.10.0 \n 22.9.0 \n 22.8.0 \n 22.7.0 \n 22.6.0 \n 22.5.1 \n 22.5.0 \n 22.4.1 \n 22.4.0 \n 22.3.0 \n 22.2.0 \n 22.1.0 \n 22.0.0\n\n---\n\nTitle: Node.js 22 is now available!\nURL: https://nodejs.org/en/blog/announcements/v22-release-announce\nPublished: N/A\nAuthor: N/A\nHighlights:\nNode.js — Node.js 22 is now available!\n\n# Node.js 22 is now available!\n\nThe Node.js Project\n\nWe're excited to announce the release of Node.js 22! Highlights include require()ing ES modules, a WebSocket client, updates of the V8 JavaScript engine, and more!\n\nNode.js 22 will enter long-term support (LTS) in October, but until then, it will be the \"Current\" release for the next six months. We encourage you to explore th\n\n---\n\nURL fragments in prose: see https://nodejs.org/en/blog for more.\n\n---\n\nTitle: Node.js — Node.js 22.0.0 (Current)\nURL: https://nodejs.org/en/blog/release/v22.0.0/\nPublished: 2024-04-24T00:00:00.000Z\nAuthor: N/A\nHighlights:\n# Node.js 22.0.0 (Current)\n\nRafael Gonzaga\n\n## 2024-04-24, Version 22.0.0 (Current), @RafaelGSS and @marco-ippolito\n\nWe're excited to announce the release of Node.js 22! Highlights include require()ing ESM graphs, WebSocket client, updates of the V8 JavaScript engine, and more! As a reminder, Node.js 22 will enter long-term support (LTS) in October, but until then, it will be the \"Current\" release for the next six mo", "_meta": {"searchTime": 1412.2}}]}, "jsonrpc": "2.0", "id": 1}
+
diff --git a/test/web-search.unit.test.ts b/test/web-search.unit.test.ts
new file mode 100644
index 0000000..b09e196
--- /dev/null
+++ b/test/web-search.unit.test.ts
@@ -0,0 +1,505 @@
+/** 联网搜索(src/agent/search.ts):免费后端的解析(真实页面裁剪的样本,不联网)、回退与冷却、带 Key 接口的请求。 */
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { describe, expect, it } from "vitest";
+import { AgentManager } from "../src/agent/manager.js";
+import { locateDshRuntime } from "../src/agent/host.js";
+import { findAppRoot } from "../src/magpie/supervisor.js";
+import { Store } from "../src/store/db.js";
+import {
+ SearchAborted,
+ SearchFailure,
+ SearchService,
+ bing,
+ bingTarget,
+ ddgTarget,
+ dedupe,
+ duckduckgo,
+ exaMcp,
+ exaMcpResults,
+ htmlText,
+ keyedBackend,
+ keyedRequest,
+ normalizeSearxUrl,
+ normalizeWebSearch,
+ parseBing,
+ parseDdg,
+ parseExaText,
+ parseRpcReply,
+ queryTerms,
+ relevant,
+ type BackendContext,
+ type SearchBackend,
+ type WebSearchSettings,
+ type WebSource,
+} from "../src/agent/search.js";
+
+const fixture = (f: string) => fs.readFileSync(path.join(__dirname, "fixtures", "web-search", f), "utf8");
+
+/** 假 fetch:按地址回页面,记下每次请求 */
+function fakeFetch(route: (url: string, init: RequestInit) => Response | Promise<Response>) {
+ const calls: Array<{ url: string; init: RequestInit }> = [];
+ const fn = (async (input: string | URL | Request, init: RequestInit = {}) => {
+ const url = String(input);
+ calls.push({ url, init });
+ return route(url, init);
+ }) as typeof fetch;
+ return { fn, calls };
+}
+const ctxOf = (f: typeof fetch, lang: "zh-CN" | "en" = "en"): BackendContext => ({ fetch: f, signal: AbortSignal.timeout(5000), lang });
+const html = (body: string, status = 200) => new Response(body, { status, headers: { "content-type": "text/html; charset=utf-8" } });
+
+describe("Exa 公开 MCP", () => {
+ it("SSE 回复:每条的标题、网址、发布日期、摘要;正文里的 --- 不会把一条切成两条", () => {
+ const got = exaMcpResults(parseRpcReply(fixture("exa-mcp.sse")));
+ expect(got.map((r) => r.url)).toEqual([
+ "https://github.com/nodejs/node/blob/main/doc/changelogs/CHANGELOG_V22.md",
+ "https://nodejs.org/en/blog/announcements/v22-release-announce",
+ "https://nodejs.org/en/blog/release/v22.0.0/",
+ ]);
+ expect(got[1].title).toBe("Node.js 22 is now available!");
+ expect(got[1].snippet).toMatch(/^Node\.js — Node\.js 22 is now available! /);
+ // 正文里的分隔线和后面的段落仍属于这一条
+ expect(got[1].snippet).not.toContain("---");
+ // "Published: N/A" 不算日期;午夜的 ISO 时间只留日期
+ expect(got[0].publishedAt).toBeUndefined();
+ expect(got[2].publishedAt).toBe("2024-04-24");
+ // 摘要压成一行、去掉 Markdown 标题符号、有长度上限
+ expect(got[0].snippet!.startsWith("Node.js 22 ChangeLog")).toBe(true);
+ for (const r of got) expect(r.snippet!.length).toBeLessThanOrEqual(300);
+ });
+
+ it("JSON 回复、旧格式(Published Date / Text)和 JSON 文本都认", () => {
+ const text = "Title: A\nURL: https://a.example/x\nPublished Date: 2025-01-02\nAuthor: me\nText: first line\nsecond line\n\nTitle: B\nURL: https://b.example/\nText: b";
+ const reply = JSON.stringify({ jsonrpc: "2.0", id: 1, result: { content: [{ type: "text", text }] } });
+ expect(exaMcpResults(parseRpcReply(reply))).toEqual([
+ { url: "https://a.example/x", title: "A", snippet: "first line second line", publishedAt: "2025-01-02" },
+ { url: "https://b.example/", title: "B", snippet: "b" },
+ ]);
+ expect(parseExaText(JSON.stringify({ results: [{ title: "J", url: "https://j.example/", publishedDate: "2025-02-03T00:00:00.000Z", highlights: ["h1", "h2"] }] }))).toEqual([
+ { url: "https://j.example/", title: "J", snippet: "h1 h2", publishedAt: "2025-02-03" },
+ ]);
+ });
+
+ it("SSE 里多个事件、CRLF 换行:取 id 对得上的那条", () => {
+ const body = 'event: message\r\ndata: {"jsonrpc":"2.0","method":"notifications/progress","params":{}}\r\n\r\nevent: message\r\ndata: {"jsonrpc":"2.0","id":1,"result":{"content":[{"type":"text","text":"Title: T\\nURL: https://t.example/"}]}}\r\n\r\n';
+ expect(exaMcpResults(parseRpcReply(body))).toEqual([{ url: "https://t.example/", title: "T" }]);
+ });
+
+ it("没有结果是空结果;限流、出错、看不懂都抛错(换下一个后端)", () => {
+ const reply = (result: unknown) => ({ jsonrpc: "2.0", id: 1, result });
+ expect(exaMcpResults(reply({ content: [{ type: "text", text: "No search results found. Please try a different query." }] }))).toEqual([]);
+ const fail = (msg: unknown) => {
+ try {
+ exaMcpResults(msg);
+ } catch (e) {
+ return e as SearchFailure;
+ }
+ throw new Error("expected a failure");
+ };
+ // 真实的限流回复(200 + _meta 标记 + 一段说明,没有结果)
+ expect(fail(parseRpcReply(fixture("exa-mcp-rate-limited.sse")))).toMatchObject({ backend: "exa-mcp", kind: "rate" });
+ expect(fail(reply({ content: [{ type: "text", text: "You've hit Exa's free MCP rate limit." }] })).kind).toBe("rate");
+ expect(fail(reply({ isError: true, content: [{ type: "text", text: "Invalid arguments" }] })).kind).toBe("format");
+ expect(fail({ jsonrpc: "2.0", id: 1, error: { code: -32602, message: "bad params" } }).kind).toBe("format");
+ expect(fail(reply({ content: [{ type: "text", text: "Something else entirely" }] })).kind).toBe("format");
+ });
+
+ it("请求:tools/call web_search_exa,同时接受 JSON 和 SSE;HTTP 出错按状态码分类", async () => {
+ const f = fakeFetch(() => new Response(fixture("exa-mcp.sse"), { headers: { "content-type": "text/event-stream" } }));
+ const got = await exaMcp.search("node 22", 3, ctxOf(f.fn));
+ expect(got).toHaveLength(3);
+ const { url, init } = f.calls[0];
+ expect(url).toBe("https://mcp.exa.ai/mcp");
+ expect((init.headers as Record<string, string>).accept).toBe("application/json, text/event-stream");
+ expect(JSON.parse(String(init.body))).toMatchObject({ jsonrpc: "2.0", method: "tools/call", params: { name: "web_search_exa", arguments: { query: "node 22", numResults: 3 } } });
+ const limited = fakeFetch(() => new Response("slow down", { status: 429 }));
+ await expect(exaMcp.search("q", 3, ctxOf(limited.fn))).rejects.toMatchObject({ kind: "rate", status: 429 });
+ const down = fakeFetch(() => {
+ throw new TypeError("fetch failed", { cause: { code: "ECONNRESET" } });
+ });
+ await expect(exaMcp.search("q", 3, ctxOf(down.fn))).rejects.toMatchObject({ kind: "network", message: "fetch failed (ECONNRESET)" });
+ });
+});
+
+describe("必应网页", () => {
+ it("自然结果:h2 > a 的标题(命中词的 <strong> 不留空格)、跳转链接还原、摘要;广告不要", () => {
+ const page = parseBing(fixture("bing.html"));
+ expect(page.page).toBe(true);
+ expect(page.results.map((r) => r.url)).toEqual(["https://www.weather.com.cn/weather/101010100.shtml", "https://www.nmc.cn/publish/forecast/ABJ/beijing.html", "https://www.msn.com/zh-cn/weather/forecast/in-%E5%8C%97%E4%BA%AC"]);
+ expect(page.results[0].title).toBe("北京天气预报,北京7天天气预报,北京15天天气预报,北京天气查询");
+ expect(page.results[1].snippet).toMatch(/^当前位置: 首页 北京市/);
+ expect(page.results.some((r) => /aclick|Ad title/.test(`${r.url} ${r.title}`))).toBe(false);
+ });
+
+ it("日期(span.news_dt)拿出来作发布日期,摘要里不留日期和分隔点", () => {
+ const block = `<ol id="b_results"><li class="b_algo"><h2><a href="https://example.org/a">Hello <strong>World</strong></a></h2><div class="b_caption"><p class="b_lineclamp2"><span class="news_dt">Oct 20, 2023</span>&nbsp;&#0183;&#32;Body &amp; more …</p></div></li></ol>`;
+ expect(parseBing(block).results).toEqual([{ url: "https://example.org/a", title: "Hello World", snippet: "Body & more …", publishedAt: "Oct 20, 2023" }]);
+ expect(parseBing('<ol id="b_results"><li class="b_no"><h1>There are no results</h1></li></ol>')).toMatchObject({ results: [], noResults: true });
+ });
+
+ it("跳转链接 bing.com/ck/a?…&u=a1<base64url>", () => {
+ const enc = Buffer.from("https://例子.example/路径?q=1&x=_-").toString("base64url");
+ expect(bingTarget(`https://www.bing.com/ck/a?!&&p=abc&u=a1${enc}&ntb=1`)).toBe("https://例子.example/路径?q=1&x=_-");
+ expect(bingTarget("https://www.bing.com/ck/a?!&&p=abc&ntb=1")).toBeUndefined();
+ expect(bingTarget("https://plain.example/x")).toBe("https://plain.example/x");
+ expect(bingTarget("javascript:void(0)")).toBeUndefined();
+ });
+
+ it("给出与问题无关的一页(必应对脚本样的请求这么干):当出错,交给下一个后端", async () => {
+ const unrelated = parseBing(fixture("bing-unrelated.html")).results;
+ expect(unrelated.length).toBe(3);
+ expect(relevant("node.js 22 release notes", unrelated)).toBe(false);
+ expect(relevant("北京天气", parseBing(fixture("bing.html")).results)).toBe(true);
+ const f = fakeFetch(() => html(fixture("bing-unrelated.html")));
+ await expect(bing.search("node.js 22 release notes", 5, ctxOf(f.fn))).rejects.toMatchObject({ backend: "bing", kind: "unrelated" });
+ });
+
+ it("切题词:拉丁文字取 ≥3 个字符的实词,中日韩取相邻两字", () => {
+ expect(queryTerms("What is the Node.js 22 release?")).toEqual(["node.js", "release"]);
+ expect(queryTerms("北京今天天气")).toEqual(["北京", "京今", "今天", "天天", "天气"]);
+ expect(queryTerms("Rust 教程")).toEqual(["rust", "教程"]);
+ // 查询里没有可用的词时不判断
+ expect(relevant("22 ?", [{ url: "https://x.example/" }])).toBe(true);
+ });
+
+ it("中文界面先试 cn.bing.com,在境外被跳回首页时再试 www.bing.com", async () => {
+ const f = fakeFetch((url) => (url.startsWith("https://cn.bing.com/") ? html("<html><body>homepage</body></html>") : html(fixture("bing.html"))));
+ const got = await bing.search("北京天气", 2, ctxOf(f.fn, "zh-CN"));
+ expect(got).toHaveLength(2);
+ expect(f.calls.map((c) => new URL(c.url).host)).toEqual(["cn.bing.com", "www.bing.com"]);
+ expect(f.calls[1].url).toBe(`https://www.bing.com/search?q=${encodeURIComponent("北京天气")}`);
+ const h = f.calls[1].init.headers as Record<string, string>;
+ expect(h["user-agent"]).toMatch(/^Mozilla\/5\.0 .*Chrome\//);
+ expect(h["accept-language"]).toMatch(/^zh-CN/);
+ // 英文界面只用 www.bing.com;都不是结果页就出错
+ const g = fakeFetch(() => html("<html><body>captcha</body></html>"));
+ await expect(bing.search("x", 2, ctxOf(g.fn))).rejects.toMatchObject({ kind: "blocked" });
+ expect(g.calls).toHaveLength(1);
+ });
+});
+
+describe("DuckDuckGo 网页", () => {
+ it("结果:a.result__a 的标题、uddg 跳转还原、摘要;广告不要", () => {
+ const page = parseDdg(fixture("ddg.html"));
+ expect(page.challenge).toBe(false);
+ expect(page.results).toEqual([
+ expect.objectContaining({ url: "https://nodejs.org/en/blog/announcements/v22-release-announce", title: "Node.js 22 is now available!" }),
+ expect.objectContaining({ url: "https://nodejs.org/en/blog/release/v22.22.0", title: "Node.js — Node.js 22.22.0 (LTS)" }),
+ expect.objectContaining({ url: "https://versionlog.com/nodejs/22/", title: "Node.js 22 (LTS) - What's New, Support Lifecycle & EOL" }),
+ ]);
+ expect(page.results[0].snippet).toMatch(/Try out the new Node\.js 22 release! We're always happy/);
+ });
+
+ it("跳转链接 //duckduckgo.com/l/?uddg=…;广告跳转(y.js)和站内链接不要", () => {
+ expect(ddgTarget("//duckduckgo.com/l/?uddg=https%3A%2F%2Fa.example%2Fp%3Fx%3D1&amp;rut=abc")).toBe("https://a.example/p?x=1");
+ expect(ddgTarget("//duckduckgo.com/l/?uddg=https%3A%2F%2Fduckduckgo.com%2Fy.js%3Fad_domain%3Dx")).toBeUndefined();
+ expect(ddgTarget("https://direct.example/")).toBe("https://direct.example/");
+ expect(ddgTarget("/html/?q=next")).toBeUndefined();
+ });
+
+ it("人机验证页(202,或 200 却解析不出结果)是出错,不是没有结果;明确没有结果才是空", async () => {
+ expect(parseDdg(fixture("ddg-challenge.html"))).toMatchObject({ results: [], challenge: true });
+ const challenge202 = fakeFetch(() => html(fixture("ddg-challenge.html"), 202));
+ await expect(duckduckgo.search("q", 5, ctxOf(challenge202.fn))).rejects.toMatchObject({ backend: "duckduckgo", kind: "blocked", status: 202 });
+ const challenge200 = fakeFetch(() => html(fixture("ddg-challenge.html")));
+ await expect(duckduckgo.search("q", 5, ctxOf(challenge200.fn))).rejects.toMatchObject({ kind: "blocked" });
+ const unknown = fakeFetch(() => html("<html><body>something new</body></html>"));
+ await expect(duckduckgo.search("q", 5, ctxOf(unknown.fn))).rejects.toMatchObject({ kind: "blocked" });
+ const none = fakeFetch(() => html('<div class="serp__results"><div class="no-results">No results.</div></div>'));
+ expect(await duckduckgo.search("q", 5, ctxOf(none.fn))).toEqual([]);
+ const ok = fakeFetch(() => html(fixture("ddg.html")));
+ expect(await duckduckgo.search("node 22", 2, ctxOf(ok.fn))).toHaveLength(2);
+ expect(ok.calls[0].url).toBe("https://html.duckduckgo.com/html/?q=node%2022");
+ });
+});
+
+it("文字工具:实体、标签、去重", () => {
+ expect(htmlText("<b>A</b>&nbsp;&amp;&#x27;b&#39;<br>c&unknown;")).toBe("A &'b' c&unknown;");
+ const r = (url: string): WebSource => ({ url });
+ expect(dedupe([r("https://a.example/x"), r("https://A.example/x/"), r("https://a.example/x#top"), r("https://b.example/")]).map((x) => x.url)).toEqual(["https://a.example/x", "https://b.example/"]);
+});
+
+/** 测试用后端:每次调用按脚本给结果或抛错,记下调用顺序 */
+function scripted(id: string, script: Array<WebSource[] | FailKindOrError | "hang">) {
+ const calls: string[] = [];
+ const b: SearchBackend & { calls: string[] } = {
+ id,
+ calls,
+ async search(query, _count, ctx) {
+ calls.push(query);
+ const step = script.length > 1 ? script.shift()! : script[0];
+ if (step === "hang") return new Promise<WebSource[]>((_, reject) => ctx.signal.addEventListener("abort", () => reject(ctx.signal.reason)));
+ if (Array.isArray(step)) return step;
+ throw typeof step === "string" ? new SearchFailure(id, step, `${id} ${step}`) : step;
+ },
+ };
+ return b;
+}
+type FailKindOrError = "network" | "rate" | "blocked" | "unrelated" | Error;
+const hits = (...urls: string[]): WebSource[] => urls.map((u) => ({ url: `https://${u}/`, title: u }));
+
+describe("回退与冷却", () => {
+ const settings = (s: Partial<WebSearchSettings> = {}) => () => normalizeWebSearch(s);
+ const clock = () => {
+ let t = 1_000_000;
+ return { now: () => t, advance: (ms: number) => (t += ms) };
+ };
+
+ it("按顺序试:出错的跳过并冷却,成功的下一次排最前;冷却过了再回来", async () => {
+ const c = clock();
+ const exa = scripted("exa-mcp", ["rate", hits("e1")]);
+ const bg = scripted("bing", [hits("b1", "b2")]);
+ const ddg = scripted("duckduckgo", [hits("d1")]);
+ const svc = new SearchService({ settings: settings(), free: [exa, bg, ddg], now: c.now, cooldownMs: 600_000 });
+ expect(await svc.search("q1")).toEqual({ backend: "bing", sources: hits("b1", "b2") });
+ // exa 在冷却,bing 上次成功:直接问 bing
+ expect(await svc.search("q2")).toMatchObject({ backend: "bing" });
+ expect(exa.calls).toEqual(["q1"]);
+ expect(bg.calls).toEqual(["q1", "q2"]);
+ // 冷却结束:上次成功的 bing 仍排第一
+ c.advance(600_001);
+ await svc.search("q3");
+ expect(bg.calls).toEqual(["q1", "q2", "q3"]);
+ expect(exa.calls).toEqual(["q1"]);
+ expect(ddg.calls).toEqual([]);
+ });
+
+ it("没有结果不算出错:接着问下一个;都没有结果时返回空", async () => {
+ const exa = scripted("exa-mcp", [[]]);
+ const bg = scripted("bing", [hits("b1")]);
+ const svc = new SearchService({ settings: settings(), free: [exa, bg] });
+ expect(await svc.search("q")).toMatchObject({ backend: "bing", sources: hits("b1") });
+ // exa 没有被冷却
+ expect(await svc.search("q2")).toMatchObject({ backend: "bing" });
+ expect(exa.calls).toEqual(["q"]);
+ const empty = new SearchService({ settings: settings(), free: [scripted("a", [[]]), scripted("b", ["network"])] });
+ expect(await empty.search("q")).toEqual({ backend: "a", sources: [] });
+ });
+
+ it("全部出错时报错;全都在冷却时照样都试(冷却先结束的先试)", async () => {
+ const c = clock();
+ const a = scripted("a", ["network", hits("a1")]);
+ const b = scripted("b", ["blocked", "blocked"]);
+ const svc = new SearchService({ settings: settings(), free: [a, b], now: c.now, cooldownMs: 1000 });
+ await expect(svc.search("q")).rejects.toMatchObject({ backend: "all", message: "a: network; b: blocked" });
+ c.advance(10);
+ expect(await svc.search("q2")).toMatchObject({ backend: "a", sources: hits("a1") });
+ expect(b.calls).toEqual(["q"]);
+ });
+
+ it("每个后端有超时;调用方取消时立即停下,不算后端出错", async () => {
+ const slow = scripted("slow", ["hang"]);
+ const fast = scripted("fast", [hits("f1")]);
+ const svc = new SearchService({ settings: settings(), free: [slow, fast], timeoutMs: 50 });
+ expect(await svc.search("q")).toMatchObject({ backend: "fast" });
+ const ac = new AbortController();
+ const hang = scripted("hang", ["hang"]);
+ const other = scripted("other", [hits("o1")]);
+ const svc2 = new SearchService({ settings: settings(), free: [hang, other], timeoutMs: 10_000 });
+ const p = svc2.search("q", 8, ac.signal);
+ setTimeout(() => ac.abort(), 20);
+ await expect(p).rejects.toBeInstanceOf(SearchAborted);
+ expect(other.calls).toEqual([]);
+ // 没有冷却:下一次还是先问它
+ const p2 = svc2.search("q2", 8, AbortSignal.timeout(20));
+ await expect(p2).rejects.toBeInstanceOf(SearchAborted);
+ expect(hang.calls).toEqual(["q", "q2"]);
+ });
+
+ it("去重并截到 maxResults", async () => {
+ const svc = new SearchService({ settings: settings(), free: [scripted("a", [[...hits("x", "y"), { url: "https://x/" }, ...hits("z", "w")]])] });
+ expect((await svc.search("q", 3)).sources.map((s) => s.url)).toEqual(["https://x/", "https://y/", "https://z/"]);
+ });
+
+ it("填了 Key 先用它;出错退回免费的并冷却;Key 被拒时设置页提示,换了 Key 就清掉", async () => {
+ let cfg = normalizeWebSearch({ provider: "tavily", keys: { tavily: "tvly-secret-1234" } });
+ const seen: string[] = [];
+ let tavily: () => Response = () => Response.json({ results: [{ title: "T", url: "https://t.example/", content: "c" }] });
+ const f = fakeFetch((url) => (seen.push(new URL(url).host), tavily()));
+ const free = scripted("duckduckgo", [hits("d1")]);
+ const svc = new SearchService({ settings: () => cfg, fetch: () => f.fn, free: [free] });
+ expect(await svc.search("q")).toEqual({ backend: "tavily", sources: [{ url: "https://t.example/", title: "T", snippet: "c" }] });
+ expect(svc.view()).toEqual({ provider: "tavily", keyTail: "1234", url: "", keyRejected: false });
+ tavily = () => new Response('{"detail":{"error":"Unauthorized"}}', { status: 401 });
+ expect(await svc.search("q2")).toMatchObject({ backend: "duckduckgo" });
+ expect(svc.view().keyRejected).toBe(true);
+ // 冷却中不再先问它
+ await svc.search("q3");
+ expect(seen).toEqual(["api.tavily.com", "api.tavily.com"]);
+ // 换了 Key:清掉提示和冷却,马上再试
+ cfg = normalizeWebSearch({ provider: "tavily", keys: { tavily: "tvly-other-5678" } });
+ tavily = () => Response.json({ results: [] });
+ expect(svc.view()).toMatchObject({ keyTail: "5678", keyRejected: false });
+ expect(await svc.search("q4")).toMatchObject({ backend: "duckduckgo" });
+ expect(seen).toHaveLength(3);
+ });
+
+ it("选了接口但没填 Key:直接用免费的", async () => {
+ const free = scripted("duckduckgo", [hits("d1")]);
+ const f = fakeFetch(() => {
+ throw new Error("should not be called");
+ });
+ const svc = new SearchService({ settings: settings({ provider: "exa" }), fetch: () => f.fn, free: [free] });
+ expect(await svc.search("q")).toMatchObject({ backend: "duckduckgo" });
+ expect(f.calls).toEqual([]);
+ });
+});
+
+describe("带 Key 的接口", () => {
+ const cfg = (provider: string, key = "k-123456", url = "") => normalizeWebSearch({ provider, keys: { [provider]: key }, url });
+ const body = (r: { init: RequestInit }) => JSON.parse(String(r.init.body));
+ const headers = (r: { init: RequestInit }) => r.init.headers as Record<string, string>;
+
+ it("Tavily:Bearer,query + max_results(≤20);results[].title / url / content / published_date", () => {
+ const r = keyedRequest(cfg("tavily"), "q", 30)!;
+ expect(r.url).toBe("https://api.tavily.com/search");
+ expect(headers(r).authorization).toBe("Bearer k-123456");
+ expect(body(r)).toEqual({ query: "q", max_results: 20 });
+ expect(r.parse({ results: [{ title: "A", url: "https://a/", content: "x", published_date: "2025-01-01" }, { title: "bad", url: "ftp://x" }] })).toEqual([{ url: "https://a/", title: "A", snippet: "x", publishedAt: "2025-01-01" }]);
+ });
+
+ it("Exa:x-api-key,numResults,contents.highlights;results[].highlights / publishedDate", () => {
+ const r = keyedRequest(cfg("exa"), "q", 8)!;
+ expect(r.url).toBe("https://api.exa.ai/search");
+ expect(headers(r)["x-api-key"]).toBe("k-123456");
+ expect(body(r)).toEqual({ query: "q", numResults: 8, type: "auto", contents: { highlights: { maxCharacters: 300 } } });
+ expect(r.parse({ results: [{ title: "E", url: "https://e/", publishedDate: "2025-03-04T00:00:00.000Z", highlights: ["one", "two"] }] })).toEqual([{ url: "https://e/", title: "E", snippet: "one two", publishedAt: "2025-03-04" }]);
+ });
+
+ it("博查:Bearer,query + count + summary;data.webPages.value[].name / url / summary|snippet / datePublished", () => {
+ const r = keyedRequest(cfg("bocha"), "天气", 8)!;
+ expect(r.url).toBe("https://api.bochaai.com/v1/web-search");
+ expect(headers(r).authorization).toBe("Bearer k-123456");
+ expect(body(r)).toEqual({ query: "天气", count: 8, summary: true });
+ const value = [{ name: "B", url: "https://b/", snippet: "s", summary: "long", datePublished: "2025-05-06T08:00:00+08:00" }, { name: "C", url: "https://c/", snippet: "s2" }];
+ expect(r.parse({ code: 200, data: { webPages: { value } } })).toEqual([
+ { url: "https://b/", title: "B", snippet: "long", publishedAt: "2025-05-06T08:00:00+08:00" },
+ { url: "https://c/", title: "C", snippet: "s2" },
+ ]);
+ expect(() => r.parse({ code: 403, msg: "余额不足" })).toThrow(/403/);
+ });
+
+ it("智谱:Bearer,search_query(≤70 字)+ search_engine + search_intent + count;search_result[].title / link / content / publish_date", () => {
+ const r = keyedRequest(cfg("zhipu"), "x".repeat(100), 8)!;
+ expect(r.url).toBe("https://open.bigmodel.cn/api/paas/v4/web_search");
+ expect(headers(r).authorization).toBe("Bearer k-123456");
+ expect(body(r)).toEqual({ search_query: "x".repeat(70), search_engine: "search_std", search_intent: false, count: 8 });
+ expect(r.parse({ search_result: [{ title: "Z", link: "https://z/", content: "zc", publish_date: "2025-06-07" }] })).toEqual([{ url: "https://z/", title: "Z", snippet: "zc", publishedAt: "2025-06-07" }]);
+ });
+
+ it("SearXNG:GET <地址>/search?q=…&format=json,不用 Key;results[].url / title / content / publishedDate", () => {
+ const r = keyedRequest(normalizeWebSearch({ provider: "searxng", url: "https://searx.example/sub" }), "a b", 8)!;
+ expect(r.url).toBe("https://searx.example/sub/search?q=a%20b&format=json");
+ expect(r.init.method).toBeUndefined();
+ expect(r.parse({ results: [{ url: "https://s/", title: "S", content: "sc", publishedDate: null }] })).toEqual([{ url: "https://s/", title: "S", snippet: "sc" }]);
+ expect(normalizeSearxUrl(" https://searx.example/search/ ")).toBe("https://searx.example");
+ expect(normalizeSearxUrl("https://searx.example/sub?x=1#y")).toBe("https://searx.example/sub");
+ expect(normalizeSearxUrl("ftp://x")).toBeUndefined();
+ expect(normalizeSearxUrl("not a url")).toBeUndefined();
+ expect(normalizeSearxUrl("")).toBe("");
+ });
+
+ it("没填 Key / 地址、或是自动:没有带 Key 的后端;401/403 是 Key 被拒,SearXNG 的 403 是没开 JSON", async () => {
+ expect(keyedRequest(normalizeWebSearch({ provider: "tavily" }), "q", 8)).toBeUndefined();
+ expect(keyedRequest(normalizeWebSearch({ provider: "searxng" }), "q", 8)).toBeUndefined();
+ expect(keyedRequest(normalizeWebSearch({ provider: "auto", keys: { tavily: "k" } }), "q", 8)).toBeUndefined();
+ const forbidden = fakeFetch(() => new Response("Forbidden", { status: 403 }));
+ await expect(keyedBackend(cfg("zhipu"))!.search("q", 5, ctxOf(forbidden.fn))).rejects.toMatchObject({ kind: "auth", status: 403 });
+ await expect(keyedBackend(normalizeWebSearch({ provider: "searxng", url: "https://s.example" }))!.search("q", 5, ctxOf(forbidden.fn))).rejects.toMatchObject({ kind: "blocked" });
+ const notJson = fakeFetch(() => new Response("<html>", { status: 200 }));
+ await expect(keyedBackend(cfg("tavily"))!.search("q", 5, ctxOf(notJson.fn))).rejects.toMatchObject({ kind: "format" });
+ });
+
+ it("设置:不认识的接口当自动;Key 按接口分开存;空 Key 不存", () => {
+ expect(normalizeWebSearch({ provider: "google", keys: { tavily: " t ", exa: "", bogus: "x" }, url: 3 })).toEqual({ provider: "auto", keys: { tavily: "t" }, url: "" });
+ expect(normalizeWebSearch(null)).toEqual({ provider: "auto", keys: {}, url: "" });
+ });
+});
+
+/** 宿主这一侧:bridge 的 /dsh/search 经 AgentManager 交给 SearchService */
+async function hostWith(backends: SearchBackend[]) {
+ const search = new SearchService({ settings: () => normalizeWebSearch({}), free: backends, timeoutMs: 10_000 });
+ const mgr = new AgentManager({ store: new Store(":memory:"), dataDir: fs.mkdtempSync(path.join(os.tmpdir(), "amb-search-")), runtimeDir: undefined, upstream: () => ({ baseUrl: "", apiKey: "" }), enabled: false, search });
+ (mgr as any).hosts.set("b1", {});
+ await mgr.bridge.start();
+ return mgr;
+}
+
+describe("bridge /dsh/search → 宿主搜索", () => {
+ it("已知机器人的查询交给搜索服务,回结果;不认识的机器人、全部出错都只回 unavailable", async () => {
+ const backend = scripted("duckduckgo", [hits("a.example", "b.example"), "blocked"]);
+ const mgr = await hostWith([backend]);
+ const post = (body: unknown) =>
+ fetch(`${mgr.bridge.url}/dsh/search`, { method: "POST", headers: { authorization: `Bearer ${mgr.bridge.token}`, "content-type": "application/json" }, body: JSON.stringify(body) }).then((r) => r.json());
+ try {
+ expect(await post({ botId: "b1", query: " node 22 ", maxResults: 1 })).toEqual({ sources: hits("a.example"), truncated: false });
+ expect(backend.calls).toEqual(["node 22"]);
+ expect(await post({ botId: "nobody", query: "q" })).toEqual({ error: "unavailable" });
+ expect(await post({ botId: "b1", query: "q2" })).toEqual({ error: "unavailable" });
+ expect(await post({ botId: "b1", query: " " })).toEqual({ sources: [], truncated: false });
+ } finally {
+ await mgr.bridge.stop();
+ }
+ });
+});
+
+describe("amb-web-search 插件", () => {
+ const runtime = locateDshRuntime(findAppRoot());
+
+ it.skipIf(!runtime)("注册 amb 搜索提供方:查询经 bridge 由宿主搜索;出错只说暂时不可用、建议 web_fetch;取消一路传到宿主", async () => {
+ let sawAbort = false;
+ const hang: SearchBackend = {
+ id: "hang",
+ search: (_q, _n, ctx) =>
+ new Promise((_, reject) =>
+ ctx.signal.addEventListener("abort", () => {
+ sawAbort = true;
+ reject(ctx.signal.reason);
+ }),
+ ),
+ };
+ let mode: "ok" | "fail" | "hang" = "ok";
+ const ok = scripted("ok", [[{ url: "https://a.example/", title: "A", snippet: "s", publishedAt: "2025-01-02" }, { url: "https://b.example/" }]]);
+ const mgr = await hostWith([
+ { id: "switch", search: (q, n, ctx) => (mode === "ok" ? ok.search(q, n, ctx) : mode === "hang" ? hang.search(q, n, ctx) : Promise.reject(new SearchFailure("switch", "network", "down"))) },
+ ]);
+ const env = { ...process.env };
+ try {
+ Object.assign(process.env, { AMB_BRIDGE_URL: mgr.bridge.url, AMB_BRIDGE_TOKEN: mgr.bridge.token, AMB_BOT_ID: "b1" });
+ const plugin = await import(path.join(runtime!, "plugins", "amb-web-search.mjs"));
+ expect(plugin.inject).toEqual(["web"]);
+ const providers: any[] = [];
+ plugin.apply({ web: { registerSearchProvider: (p: any) => providers.push(p) } });
+ expect(providers.map((p) => p.id)).toEqual(["amb"]);
+ const amb = providers[0];
+ expect(amb.available()).toBe(true);
+ expect(await amb.search({ query: "q", maxResults: 5 }, AbortSignal.timeout(5000))).toEqual({
+ sources: [{ url: "https://a.example/", title: "A", snippet: "s", publishedAt: "2025-01-02" }, { url: "https://b.example/" }],
+ truncated: false,
+ });
+ // 搜不了:DSH 的 WebError,给模型的话里没有 DeepSeek、Key、设置页
+ mode = "fail";
+ const err = await amb.search({ query: "q2", maxResults: 5 }).catch((e: unknown) => e);
+ expect(err).toMatchObject({ name: "WebError", code: "WEB_PROVIDER_ERROR" });
+ expect(err.message).toBe("Web search is temporarily unavailable. If you know a relevant page, open it directly with web_fetch.");
+ expect(err.message).not.toMatch(/deepseek|api key|setting/i);
+ // 工具超时 / 回合中止:插件的 fetch 断开,宿主那边的搜索跟着取消
+ mode = "hang";
+ const ac = new AbortController();
+ const p = amb.search({ query: "q3", maxResults: 5 }, ac.signal).catch((e: unknown) => e);
+ await new Promise((r) => setTimeout(r, 100));
+ ac.abort();
+ expect(await p).toMatchObject({ code: "WEB_ABORTED" });
+ await expect.poll(() => sawAbort).toBe(true);
+ // bridge 不在了:同样只是暂时不可用
+ await mgr.bridge.stop();
+ mode = "ok";
+ expect(await amb.search({ query: "q4" }).catch((e: unknown) => e)).toMatchObject({ code: "WEB_PROVIDER_ERROR" });
+ } finally {
+ process.env = env;
+ await mgr.bridge.stop();
+ }
+ });
+});
diff --git a/ui/css/settings.css b/ui/css/settings.css
index e2f55eb..ee0183c 100644
--- a/ui/css/settings.css
+++ b/ui/css/settings.css
@@ -45,6 +45,7 @@
.settings .row .ctl .pick ~ .switch { margin-left: var(--space-2); }
.settings input.field.num-in { width: 96px; --field-h: var(--ctl-md); text-align: right; font-variant-numeric: tabular-nums; }
.settings input.field.font-in { width: min(220px, 36vw); --field-h: var(--ctl-md); }
+.settings input.field.key-in { width: min(220px, 36vw); --field-h: var(--ctl-md); }
/* 外观预览(设置 › 外观顶部):一组 --well,里面一问一答 + 一行代码,用对话页同样的字号与字体 token,
跟着字体大小、界面字体、代码字体和代码换行实时变 */
diff --git a/ui/locales/en/settings.js b/ui/locales/en/settings.js
index a1dc364..deac451 100644
--- a/ui/locales/en/settings.js
+++ b/ui/locales/en/settings.js
@@ -284,6 +284,25 @@ export default {
sub: "Computer use screenshots go only to the model, never into chats",
keys: "privacy screenshot",
},
+ webSearch: {
+ name: "Web search",
+ sub: "Free search by default. A key you add is used first",
+ keys: "search web internet tavily exa bocha zhipu searxng",
+ auto: "Auto",
+ bocha: "Bocha",
+ zhipu: "Zhipu",
+ },
+ webSearchKey: {
+ name: "{provider} key",
+ placeholder: "Paste key",
+ rejected: "Key rejected. Using free search for now",
+ remove: "Remove",
+ keys: "search key api",
+ },
+ webSearchUrl: {
+ name: "SearXNG address",
+ keys: "search searxng url address",
+ },
listen: {
name: "Listen address",
sub: "To change it, set AMB_UI_HOST / AMB_UI_PORT and restart",
diff --git a/ui/locales/zh-CN/settings.js b/ui/locales/zh-CN/settings.js
index 6a0c59f..7acf168 100644
--- a/ui/locales/zh-CN/settings.js
+++ b/ui/locales/zh-CN/settings.js
@@ -284,6 +284,25 @@ export default {
sub: "电脑操作的截图只发给模型,不会发到聊天里",
keys: "隐私",
},
+ webSearch: {
+ name: "联网搜索",
+ sub: "默认免费搜索,填写 Key 后优先使用",
+ keys: "搜索 网页 search web tavily exa 博查 bocha 智谱 zhipu searxng",
+ auto: "自动",
+ bocha: "博查",
+ zhipu: "智谱",
+ },
+ webSearchKey: {
+ name: "{provider} Key",
+ placeholder: "粘贴 Key",
+ rejected: "Key 被拒绝,正在用免费搜索",
+ remove: "移除",
+ keys: "搜索 key api 密钥",
+ },
+ webSearchUrl: {
+ name: "SearXNG 地址",
+ keys: "搜索 searxng url 地址",
+ },
listen: {
name: "监听地址",
sub: "修改需设置 AMB_UI_HOST / AMB_UI_PORT 并重启",
diff --git a/ui/screens/settings.js b/ui/screens/settings.js
index 9576d36..0bca864 100644
--- a/ui/screens/settings.js
+++ b/ui/screens/settings.js
@@ -3,7 +3,7 @@
// 外观(主题、字体、动画、毛玻璃…)和对话习惯(发送键、通知)是本机界面偏好,存在浏览器里(lib/theme.js、lib/prefs.js);
// 其余设置走服务端 /settings。
import { useEffect, useMemo, useRef, useState } from "preact/hooks";
-import { html, Icon, Seg, Switch, Sheet, CopyButton, Skeleton } from "../components/ui.js";
+import { html, Icon, Seg, Switch, Sheet, CopyButton, Skeleton, MenuButton } from "../components/ui.js";
import { AccountButton } from "./sidebar.js";
import { PickModel, shortModel } from "../components/model-picker.js";
import { KeyForm, lineOptions } from "./onboarding.js";
@@ -97,6 +97,33 @@ function FontInput({ id, value, onChange, placeholder, suggest, fallback, label
<datalist id=${`${id}-list`}>${(fonts && fonts.length ? fonts : suggest).map((f) => html`<option key=${f} value=${f} />`)}</datalist>`;
}
+// 联网搜索的接口:自动 = 免费搜索,不用 Key
+const SEARCH_OPTIONS = () => [
+ ["auto", t("settings.rows.webSearch.auto")],
+ ["tavily", "Tavily"],
+ ["exa", "Exa"],
+ ["bocha", t("settings.rows.webSearch.bocha")],
+ ["zhipu", t("settings.rows.webSearch.zhipu")],
+ ["searxng", "SearXNG"],
+];
+
+/** 填 Key 或地址:回车或离开输入框时保存。Key 输入框总是空的(服务端只回末 4 位,显示在占位里),地址照原样显示 */
+function SaveInput({ id, value = "", secret, label, placeholder, onSave }) {
+ const ref = useRef(null);
+ const [v, setV] = useState(value);
+ useEffect(() => {
+ if (document.activeElement !== ref.current) setV(value);
+ }, [value]);
+ const commit = () => {
+ const x = v.trim();
+ if (secret ? x : x !== value) onSave(x);
+ if (secret) setV("");
+ };
+ return html`<input ref=${ref} id=${id} class="field mono key-in" type=${secret ? "password" : "url"} autocomplete="off" spellcheck="false"
+ aria-label=${label} placeholder=${placeholder} value=${v} onInput=${(e) => setV(e.target.value)} onBlur=${commit}
+ onKeyDown=${(e) => e.key === "Enter" && (e.preventDefault(), e.currentTarget.blur())} />`;
+}
+
/** 外观分页顶部的预览:一问一答 + 一行代码,跟着字体大小、字体、代码字体和代码换行实时变 */
function AppearancePreview() {
return html`<div class="ap-preview" aria-hidden="true">
@@ -235,6 +262,7 @@ export function SettingsScreen({ tab, query: q, onQuery: setQ, onTab, focusRow,
saved();
})
.catch(failed);
+ const putSearch = (patch) => put({ webSearch: patch }, { webSearch: { ...s.webSearch, ...(patch.provider ? { provider: patch.provider } : {}) } });
const desktop = !!(s && s.desktop);
const pickFont = (v) => {
setPrefs({ font: v });
@@ -374,6 +402,17 @@ export function SettingsScreen({ tab, query: q, onQuery: setQ, onTab, focusRow,
{ tab: "privacy", id: "sandbox", name: t("settings.rows.sandbox.name"), sub: t("settings.rows.sandbox.sub"), ctl: null, keys: t("settings.rows.sandbox.keys") },
{ tab: "privacy", id: "screens", name: t("settings.rows.screens.name"), sub: t("settings.rows.screens.sub"), ctl: null, keys: t("settings.rows.screens.keys") },
+ { tab: "network", id: "web-search", name: t("settings.rows.webSearch.name"), sub: t("settings.rows.webSearch.sub"), keys: t("settings.rows.webSearch.keys"),
+ ctl: html`<${MenuButton} label=${t("settings.rows.webSearch.name")} value=${s.webSearch.provider} width=${200} options=${SEARCH_OPTIONS()} onChange=${(v) => putSearch({ provider: v })} />` },
+ s.webSearch.provider === "searxng" && { tab: "network", id: "web-search-url", name: t("settings.rows.webSearchUrl.name"), keys: t("settings.rows.webSearchUrl.keys"),
+ ctl: html`<${SaveInput} id="web-search-url-in" value=${s.webSearch.url} label=${t("settings.rows.webSearchUrl.name")} placeholder="https://searx.example.org" onSave=${(url) => putSearch({ url })} />` },
+ s.webSearch.provider !== "auto" && s.webSearch.provider !== "searxng" && (() => {
+ const ws = s.webSearch;
+ const name = t("settings.rows.webSearchKey.name", { provider: SEARCH_OPTIONS().find(([v]) => v === ws.provider)[1] });
+ return { tab: "network", id: "web-search-key", name, sub: ws.keyRejected ? t("settings.rows.webSearchKey.rejected") : "", keys: t("settings.rows.webSearchKey.keys"),
+ ctl: html`<${SaveInput} key=${ws.provider} id="web-search-key-in" secret label=${name} placeholder=${ws.keyTail ? `••••${ws.keyTail}` : t("settings.rows.webSearchKey.placeholder")} onSave=${(key) => putSearch({ key })} />
+ ${ws.keyTail && html`<button type="button" class="text danger" onClick=${() => putSearch({ key: "" })}>${t("settings.rows.webSearchKey.remove")}</button>`}` };
+ })(),
{ tab: "network", id: "listen", name: t("settings.rows.listen.name"), sub: t("settings.rows.listen.sub"), keys: t("settings.rows.listen.keys"),
ctl: html`<span class="val">${s.listen.host}:${s.listen.port}</span><${CopyButton} text=${`http://${s.listen.host === "0.0.0.0" ? "127.0.0.1" : s.listen.host}:${s.listen.port}/`} label=${t("settings.rows.listen.copy")} />` },